Hi all,
Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces.
I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5.
Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again.
Yours,
Allan
On Wed, Feb 1, 2023 at 9:32 AM Allan Hansen <hansen@rc.org> wrote:
Hi all,
Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces.
I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5.
Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again.
Yours,
Allan
Instead of enabling the subscribers, I would instead look at a different way to address this for the list:
- Temporarily bump up the "Bounce score threshold" from 5 to 10 (to buy time.)
- Reduce the value of "Bounce info stale after" to something like 4 days.
These params are described on the list settings.
Then within a day or two, I'll have fixed the DKIM issues on the DNS front and the server (signing) side.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 1/31/23 22:32, Allan Hansen wrote:
Hi all,
Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces.
I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5.
Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again.
Yes, Google has started enforcing a policy which is effectively DMARC p=reject even for senders whose domains don't publish DMARC p=reject or quarantine.
To avoid this issue, you may need to set DMARC Mitigate unconditionally to Yes as well as DMARC mitigation action to Replace From: with list address if it isn't already.
As far as enabling without resetting the bounce score is concerned, I think you are correct that they will just be disabled again on the next bounce. This is a bug and should be fixed.
However, all of the above can be avoided without any DMARC changes by enabling bounce probes. in the [mta] section of mailman.cfg, set
verp_probes: yes
Since the probe is sent From: the list's domain, it should pass googles SPF and DKIM checks and not bounce.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thank you, Mark. By
"However, all of the above can be avoided without any DMARC changes by enabling bounce probes."
Do you mean that I can get rid of the 'From' mangling, which has been a frequent source of embarrassment because the list address is captured and reused by type-ahead when sending non-list messages and those messages instead going to 1000 subscribers.
__
AllanOn 2/1/23, 09:59, "Mark Sapiro" <mark@msapiro.net> wrote:
On 1/31/23 22:32, Allan Hansen wrote:
> Hi all,
>
> Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces.
>
> I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5.
>
> Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again.
Yes, Google has started enforcing a policy which is effectively DMARC
p=reject even for senders whose domains don't publish DMARC p=reject or
quarantine.
To avoid this issue, you may need to set DMARC Mitigate unconditionally
to Yes as well as DMARC mitigation action to Replace From: with list
address if it isn't already.
As far as enabling without resetting the bounce score is concerned, I
think you are correct that they will just be disabled again on the next
bounce. This is a bug and should be fixed.
However, all of the above can be avoided without any DMARC changes by
enabling bounce probes. in the [mta] section of mailman.cfg, set
```
verp_probes: yes
```
Since the probe is sent From: the list's domain, it should pass googles
SPF and DKIM checks and not bounce.
--
Mark Sapiro <mark@msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-leave@mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: mailman-users@mailman3.org <https://lists.mailman3.org/archives/list/<a href=>/message/56WVO6NIOGOBAOOS2B4YCIDJN7VCIXTF/">https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/56WVO6NIOGOBAOOS2B4YCIDJN7VCIXTF/
This message sent to hansen@rc.orgOn 2/1/23 10:32, Allan Hansen wrote:
Thank you, Mark. By
"However, all of the above can be avoided without any DMARC changes by enabling bounce probes."
Do you mean that I can get rid of the 'From' mangling, which has been a frequent source of embarrassment because the list address is captured and reused by type-ahead when sending non-list messages and those messages instead going to 1000 subscribers.
No. If you enable bounce probes, it will prevent users having delivery disabled and being unsubscribed by bounce if the only bounces are for DMARC policy reasons, but they will still be missing mail if you don't apply DMARC mitigations.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi all - new to this forum but am trying to work out a similar issue raised by a list member's IT department. I'm a bit out of my depth here and hoping you all can help.
They have recommended:
- When the mailing list relay an email from a member of the list to other members, the email from address should be the mailing list and not the user sending the original email. For example: list-name@mydomain.ca (on behalf of Sally Brown)
- The mailing list should not modify the original email and thus breaking the DKIM signature
I have clicked the DMARC mitigation unconditionally option which seems to solve the second point but haven't managed to have it sent by the list name but not as suggested in the first point. I either get "my email via list@mylist.ca" or list@mylist.ca" Is anyone aware of a setting to do this?
Many thanks
d
On 2/17/23 13:39, denise.mcgeachy--- via Mailman-users wrote:
Hi all - new to this forum but am trying to work out a similar issue raised by a list member's IT department. I'm a bit out of my depth here and hoping you all can help.
They have recommended:
- When the mailing list relay an email from a member of the list to other members, the email from address should be the mailing list and not the user sending the original email. For example: list-name@mydomain.ca (on behalf of Sally Brown)
- The mailing list should not modify the original email and thus breaking the DKIM signature
I have clicked the DMARC mitigation unconditionally option which seems to solve the second point but haven't managed to have it sent by the list name but not as suggested in the first point. I either get "my email via list@mylist.ca" or list@mylist.ca" Is anyone aware of a setting to do this?
Those two points are contradictory. You can't do both. Not modifying the message in ways that break DKIM requires no content filtering, no addition of list headers or footers and no munging of From: or Reply-To: headers. Normally the list will make some modification that will break DKIM signatures.
If you set DMARC mitigation action to Replace From: with list address, and set DMARC Mitigate unconditionally, that should address the first point and individual messages from the list will be From the list and that's all you should need. For example, the message to which I'm replying is
From: denise.mcgeachy--- via Mailman-users <mailman-users@mailman3.org>
While this is not identical to the format you quote, the address is the list's address which is the important thing.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 2023-02-17 21:39, denise.mcgeachy--- via Mailman-users wrote:
Hi all - new to this forum but am trying to work out a similar issue raised by a list member's IT department. I'm a bit out of my depth here and hoping you all can help.
They have recommended:
- When the mailing list relay an email from a member of the list to other members, the email from address should be the mailing list and not the user sending the original email. For example: list-name@mydomain.ca (on behalf of Sally Brown)
- The mailing list should not modify the original email and thus breaking the DKIM signature
I have clicked the DMARC mitigation unconditionally option which seems to solve the second point but haven't managed to have it sent by the list name but not as suggested in the first point. I either get "my email via list@mylist.ca" or list@mylist.ca" Is anyone aware of a setting to do this?
To distribute messages with valid DKIM signatures, I set
remove_dkim_headers: yes
in /etc/mailman3/mailman.cf and add a new DKIM signature for my mailing list domain.
- Jan
Jan Eden via Mailman-users writes:
To distribute messages with valid DKIM signatures, I set
remove_dkim_headers: yes
in /etc/mailman3/mailman.cf
That should be "mailman.cfg" for a standard installation, and it may not be in "/etc/mailman3" in a shared hosting situation.
Do you have evidence that removing DKIM headers helps you get mail delivered? (Not adding your own, IMO you should do that regardless of whether you strip existing ones, but removing originals.) The DKIM spec says that you shouldn't, and that consumers should treat invalid signatures no differently from missing signatures. I haven't run into a "public" host that treats an invalid signature as a problem in quite a while.
On the other hand, I know of at least one postmaster who is experimenting with simple "repairs" (namely, stripping list tags and serial numbers from Subject and unwrapping simple MIME structure for Mailman lists at least -- the case where Mailman wraps the original message in multipart/mixed and adds a text/plain part to the message with the footer in it) and checking the original DKIM signature and from alignment on the "repaired" message. I don't know if they're still working on that.
Steve
On 2023-02-20 19:03, Stephen J. Turnbull wrote:
Jan Eden via Mailman-users writes:
To distribute messages with valid DKIM signatures, I set
remove_dkim_headers: yes
in /etc/mailman3/mailman.cf
That should be "mailman.cfg" for a standard installation, and it may not be in "/etc/mailman3" in a shared hosting situation.
Do you have evidence that removing DKIM headers helps you get mail delivered? (Not adding your own, IMO you should do that regardless of whether you strip existing ones, but removing originals.) The DKIM spec says that you shouldn't, and that consumers should treat invalid signatures no differently from missing signatures. I haven't run into a "public" host that treats an invalid signature as a problem in quite a while.
Right – removing the invalid DKIM signature is just a matter of taste. I tested message distribution with and without signature removal, and two major mail providers did not handle the messages differently.
- Jan
Mark,
Do you have any tips for scripting the setting changes via mailman shell -l somelist@caltech.edu or a python script to run. eg. mailman shell -r dmarcsettings.py?
We have a number of lists that email Gmail addresses either directly or via aliases and virtual email domains using Google Workspace.
Thanks as always.
-- Dan Caballero Caltech
On 9/11/24 15:21, Dan Caballero wrote:
Mark,
Do you have any tips for scripting the setting changes via mailman shell -l somelist@caltech.edu or a python script to run. eg. mailman shell -r dmarcsettings.py?
We have a number of lists that email Gmail addresses either directly or via aliases and virtual email domains using Google Workspace.
If your Mailman version is >=3.3.9 you can add ^.*@gmail\.com to each
list's dmarc_addresses to apply dmarc_mitigate_action to any mail
From: a @gmail.com address. A mailman shell interaction to do that
and set dmarc_mitigate_action to munge_from for all lists is like
$ /opt/mailman/mm/bin/mailman shell
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
>>> for mlist in getUtility(IListManager):
... mlist.dmarc_addresses.append(r'^.*@gmail\.com')
... mlist.dmarc_mitigate_action = DMARCMitigateAction.munge_from
...
>>> commit()
If your Mailman is older than 3.3.9 or you don't want to use dmarc_addresses, replace the line
... m mlist.dmarc_addresses.append(r'^.*@gmail\.com')
with
... mlist.dmarc_mitigate_unconditionally = True
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
One more, sorry:
Looking in the mailman-users archive, I'm not sure if I understand the bounce settings.
Is it the case that if I set the 'Bounce info stale after' to 3d then in 3 days the bounces will be reset to 0 if no further bounces have arrived? I saw a post that made me doubt this.
Yours, Allan
On 2/1/23, 09:59, "Mark Sapiro" <mark@msapiro.net> wrote:
On 1/31/23 22:32, Allan Hansen wrote:
> Hi all,
>
> Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces.
>
> I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5.
>
> Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again.
Yes, Google has started enforcing a policy which is effectively DMARC
p=reject even for senders whose domains don't publish DMARC p=reject or
quarantine.
To avoid this issue, you may need to set DMARC Mitigate unconditionally
to Yes as well as DMARC mitigation action to Replace From: with list
address if it isn't already.
As far as enabling without resetting the bounce score is concerned, I
think you are correct that they will just be disabled again on the next
bounce. This is a bug and should be fixed.
However, all of the above can be avoided without any DMARC changes by
enabling bounce probes. in the [mta] section of mailman.cfg, set
```
verp_probes: yes
```
Since the probe is sent From: the list's domain, it should pass googles
SPF and DKIM checks and not bounce.
--
Mark Sapiro <mark@msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-leave@mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: mailman-users@mailman3.org <https://lists.mailman3.org/archives/list/<a href=>/message/56WVO6NIOGOBAOOS2B4YCIDJN7VCIXTF/">https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/56WVO6NIOGOBAOOS2B4YCIDJN7VCIXTF/
This message sent to hansen@rc.orgOn 2/1/23 10:49, Allan Hansen wrote:
Is it the case that if I set the 'Bounce info stale after' to 3d then in 3 days the bounces will be reset to 0 if no further bounces have arrived? I saw a post that made me doubt this.
Yes and no. If 'Bounce info stale after' is set to 3d and no bounces for a user are received in a 3 day period, the user's bounce score is not immediately reset, but it will be reset (to 1) the next time a bounce is received for the user. I.e., it is effectively the same thing unless you are looking at the user's bounce score which doesn't change until the next bounce.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark
On Wed, Feb 1, 2023 at 9:28 PM Mark Sapiro <mark@msapiro.net> wrote:
Yes and no. If 'Bounce info stale after' is set to 3d and no bounces for a user are received in a 3 day period, the user's bounce score is not immediately reset, but it will be reset (to 1) the next time a bounce is received for the user. I.e., it is effectively the same thing unless you are looking at the user's bounce score which doesn't change until the next bounce.
Isn't this a bug? We are now showing users' bounce scores on the Members page and this would be misleading. Is it difficult from internals perspective to have the bounce score reset immediately?
BR, Danil Smirnov
On 2/2/23 04:54, Danil Smirnov wrote:
Hi Mark
On Wed, Feb 1, 2023 at 9:28 PM Mark Sapiro <mark@msapiro.net> wrote:
Yes and no. If 'Bounce info stale after' is set to 3d and no bounces for a user are received in a 3 day period, the user's bounce score is not immediately reset, but it will be reset (to 1) the next time a bounce is received for the user. I.e., it is effectively the same thing unless you are looking at the user's bounce score which doesn't change until the next bounce.
Isn't this a bug? We are now showing users' bounce scores on the Members page and this would be misleading. Is it difficult from internals perspective to have the bounce score reset immediately?
Yes, it is not difficult, but it's a lot of processing. It would require something like a task runner task to periodically look at each row in the member table and if the bounce score is > 0, find the associated list's bounce_info_stale_after and if last_bounce_received + bounce_info_stale_after < now, reset the score.
Consider a site like mail.python.org that at this moment has 79024 rows in the member table and 7531 of those have bounce_score > 0. That's a lot of processing, and is it really an issue to show a non-zero bounce score that's stale?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 02-Feb-23 16:43, Mark Sapiro wrote:
On 2/2/23 04:54, Danil Smirnov wrote:
Hi Mark
On Wed, Feb 1, 2023 at 9:28 PM Mark Sapiro <mark@msapiro.net> wrote:
Yes and no. If 'Bounce info stale after' is set to 3d and no bounces for a user are received in a 3 day period, the user's bounce score is not immediately reset, but it will be reset (to 1) the next time a bounce is received for the user. I.e., it is effectively the same thing unless you are looking at the user's bounce score which doesn't change until the next bounce.
Isn't this a bug? We are now showing users' bounce scores on the Members page and this would be misleading. Is it difficult from internals perspective to have the bounce score reset immediately?
Yes, it is not difficult, but it's a lot of processing. It would require something like a task runner task to periodically look at each row in the member table and if the bounce score is > 0, find the associated list's bounce_info_stale_after and if last_bounce_received
- bounce_info_stale_after < now, reset the score.
Consider a site like mail.python.org that at this moment has 79024 rows in the member table and 7531 of those have bounce_score > 0. That's a lot of processing, and is it really an issue to show a non-zero bounce score that's stale?
There's a middle ground. Seems like the sort of thing to do daily. Loose consistency would be good enough. Waiting for the next bounce - which on a low traffic list could be a long time coming - does seem difficult to explain that to users. A daily update seems easier to explain.
Mark,
I can see your argument here, but if a 'Disabled by bounce' subscription is enabled, it would require just looking at that particular subscription and set the bounce count to 0, so that the next bounce (before staleness) does not disable it again.
__
Yours,
AllanOn 2/2/23, 13:44, "Mark Sapiro" <mark@msapiro.net> wrote:
On 2/2/23 04:54, Danil Smirnov wrote:
> Hi Mark
>
> On Wed, Feb 1, 2023 at 9:28 PM Mark Sapiro <mark@msapiro.net> wrote:
>
>> Yes and no. If 'Bounce info stale after' is set to 3d and no bounces for
>> a user are received in a 3 day period, the user's bounce score is not
>> immediately reset, but it will be reset (to 1) the next time a bounce is
>> received for the user. I.e., it is effectively the same thing unless you
>> are looking at the user's bounce score which doesn't change until the
>> next bounce.
>>
>
> Isn't this a bug? We are now showing users' bounce scores on the Members
> page and this would be misleading. Is it difficult from internals
> perspective to have the bounce score reset immediately?
Yes, it is not difficult, but it's a lot of processing. It would require
something like a task runner task to periodically look at each row in
the member table and if the bounce score is > 0, find the associated
list's bounce_info_stale_after and if last_bounce_received +
bounce_info_stale_after < now, reset the score.
Consider a site like mail.python.org that at this moment has 79024 rows
in the member table and 7531 of those have bounce_score > 0. That's a
lot of processing, and is it really an issue to show a non-zero bounce
score that's stale?
--
Mark Sapiro <mark@msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-leave@mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: mailman-users@mailman3.org <https://lists.mailman3.org/archives/list/<a href=>/message/ERW6LLR3VIJHUS5ECDA7A2P53WD7ZZEB/">https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/ERW6LLR3VIJHUS5ECDA7A2P53WD7ZZEB/
This message sent to hansen@rc.orgOn 2/2/23 18:19, Allan Hansen wrote:
Mark,
I can see your argument here, but if a 'Disabled by bounce' subscription is enabled, it would require just looking at that particular subscription and set the bounce count to 0, so that the next bounce (before staleness) does not disable it again.
We are talking about two different issues here. One issue is not resetting the bounce score to zero when (or before) a disabled by bounce user's delivery is enabled. This is https://gitlab.com/mailman/mailman/-/issues/1061 and I'm working on it.
The other issue is not resetting the user's bounce score when it goes stale as opposed to when another bounce is received and the score is stale.
Other than the fact that one might be misled by seeing a non-zero bounce score which is stale, there are no other issues from waiting for a bounce to reset a stale score.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 2/2/23 04:54, Danil Smirnov wrote:
Isn't this a bug?
Eye of the beholder. I will argue it's a doc bug.
We are now showing users' bounce scores on the Members page and this would be misleading.
If you mean the member's own page, yes, confusion is a problem. If you mean it's the Members page that the admins see, documentation and education is the answer, IMO.
Is it difficult from internals perspective to have the bounce score reset immediately?
Mark Sapiro writes:
Yes, it is not difficult, but it's a lot of processing. It would require something like a task runner task to periodically look at each row in the member table and if the bounce score is > 0, find the associated list's bounce_info_stale_after and if last_bounce_received + bounce_info_stale_after < now, reset the score.
Aside: In programming, all problems are solved by another level of indirection. In databases, all problems are solved by another index. :-) You'd still need a runner, but it would be a simple "update these rows to zero", I guess.
About the education and documentation for subscribers: I would argue that we can document the bounce score as a "recent highwater mark", since delivery isn't disabled yet. That's useful information for those with ears to hear it. That is, if you see "0" 95% of the time, occasionally "1", and today it's "2", maybe you wanna have a talk with your postmaster? On the other hand, if we reset immediately, I bet most users will rarely see anything greater than 0 because most visits are triggered by list posting,
Admins would be similar, except that they might see correlations with remote domains, or if there was a wave across domains it might be a symptom of getting registered on an RBL or two.
participants (10)
-
Allan Hansen -
Dan Caballero -
Danil Smirnov -
denise.mcgeachy＠gov.bc.ca -
Jan Eden -
Mark Sapiro -
Odhiambo Washington -
Stephen J. Turnbull -
Stephen J. Turnbull -
tlhackque