auto-away notices as bounces
Hi,
I had a user complaint that his subscription was disabled.
Looking into details, his mail provider sent out lots of auto aways notices marked with Auto-Submitted: auto-replied (vacation)
Should we not exclude these as being counted as bounce?
(List settings are "Individual" for "Personalize" and yes for both "Include RFC2369 headers" and "Include the list post header".
Regards, Andi
Andreas Barth writes:
Looking into details, his mail provider sent out lots of auto aways notices marked with Auto-Submitted: auto-replied (vacation)
Should we not exclude these as being counted as bounce?
First of all, until recently Mailman 3 wasn't doing any bounce processing at all. Depending on your version, it's possible that Mailman had nothing to do with it.
Now, are these going to the bounce address, or to the list? If they're going to the bounce address, I doubt they're being recognized as standard delivery status notifications. I think they're probably still counted as bounces if sent to a LIST-bounces address, but they should also be sent to the list owner as "unrecognized bounces". This decision isn't something Mailman should be doing because it depends on context and guessing what the subscriber wants. Really, this is an owner problem, not a Mailman problem.
On the other hand, if they are going to the list itself, I don't think we treat those as bounces at all.
It's not obvious what *we* should do. If Mailman continues sending posts to the user, it can fill his or her mailbox with mail that is in many cases available in archives. Not a good idea. In fact, typically the reason vacationers get unsubscribed is because their mailboxes fill up, and they generate real bounces. Or a moderator got tired of vacation messages to the list, and nuked the subscription. Are you sure one of those is not what happened here?
I don't understand how the user managed to get automatically unsubscribed unless you have aggressive bounce settings on the mailing list(s) or they were gone for quite a while. What should happen is after the first few bounces they will be disabled, and then checks for reactivation will be sent once a day or so. It should take at least two weeks, and often longer, to unsubscribe a user. If you live where people habitually take month-long vacations, you should change your settings to account for that.
Finally, if the user doesn't want to get unsubscribed while they're on vacation, they can set themselves to no-mail. They should also be able to resubscribe themselves; deleting a subscription for excessive bouncing doesn't delete the user's login account.
Steve
- Stephen J. Turnbull (stephenjturnbull@gmail.com) [210824 19:31]:
First of all, until recently Mailman 3 wasn't doing any bounce processing at all. Depending on your version, it's possible that Mailman had nothing to do with it.
Well, the mail adress got disabled, and the user got warning mails ("Your subscription for list mailing list has been disabled"). As none of the list owners did that, I don't think there is any other explanation. Please note that individualsation was enabled, so all mails are sent out with per-subscriber mail envelope address, which then makes bounce detection quite straight-forward.
Now, are these going to the bounce address, or to the list? If they're going to the bounce address, I doubt they're being recognized as standard delivery status notifications.
They are sent to the bounce address (more specifically list-bounces+user=domain@listdomain where user@domain is subscribed to the list).
I think they're probably still counted as bounces if sent to a LIST-bounces address, but they should also be sent to the list owner as "unrecognized bounces".
I didn't get them as list owner, so no. (I however have duplicates of the unprocessed mails, so I can check what was actually sent to the -bounce-adress). I only became aware of the fact after I got a complaint that the account had been disabled.
This decision isn't something Mailman should be doing because it depends on context and guessing what the subscriber wants. Really, this is an owner problem, not a Mailman problem.
What should I do different as owner? The user was set to disabled without any manual intervention from me as list owner.
It's not obvious what *we* should do.
My proposal would be to ignore messages sent to the bounce address with the 'Auto-Submitted: auto-replied (vacation)' header. I'm thinking about doing that change on the MTA level, but I believe it would be more appropriate at the mailman level.
If Mailman continues sending posts to the user, it can fill his or her mailbox with mail that is in many cases available in archives. Not a good idea.
I would say, that's up to the user.
In fact, typically the reason vacationers get unsubscribed is because their mailboxes fill up, and they generate real bounces.
This would be a real bounce, and of course should be handled as such.
Or a moderator got tired of vacation messages to the list, and nuked the subscription.
There was no vacation message at the list, and also no was sent to the moderators or the list address.
Are you sure one of those is not what happened here?
"Reasonably sure" I would say. Also, as written above, the user received the "Your subscription for list mailing list has been disabled" mail which specifies :
Your subscription has been disabled on the $mailinglist mailing list because it has received a number of bounces indicating that there may be a problem delivering messages to $address. You may want to check with your mail administrator for more help.
I don't understand how the user managed to get automatically unsubscribed unless you have aggressive bounce settings on the mailing list(s) or they were gone for quite a while. What should happen is after the first few bounces they will be disabled, and then checks for reactivation will be sent once a day or so.
Sorry, I meant "disabled". A contributing fact was for sure that the users mail setup sent out the vacation mail on any incoming mail, not only the first, and also on mails with precendence list. I adressed that already to the user (because I believe it's a bug), but of course, IMHO this should be adressed at both sides. (Especially as the mail provider is gmx, this is *the* largest webmail provider in Germany, so both the probability of fixes is rather low, and the probability of happening with users users is not too low.)
Thanks for your help
Andi
Andreas Barth writes:
- Stephen J. Turnbull (stephenjturnbull@gmail.com) [210824 19:31]:
[I]t's possible that Mailman had nothing to do with it.
Well, the mail adress got disabled, and the user got warning mails ("Your subscription for list mailing list has been disabled"). [...] Please note that individualsation was enabled,
OK. But all you told us was the user got disabled or unsubscribed.
so all mails are sent out with per-subscriber mail envelope address, which then makes bounce detection quite straight-forward.
We can identify the subscription easily, but there are still complexities. Is it a hard bounce, or a soft bounce, in particular, but it could also be spam or a rogue autoresponder.
They are sent to the bounce address (more specifically list-bounces+user=domain@listdomain where user@domain is subscribed to the list).
OK.
I didn't get them as list owner, so no.
You have to request that unparseable bounces be sent to owner in the list configuration. It defaults off because most owners wouldn't know what to do with them, although maybe we should default on and provide an explanation of how to forward them to us for future versions, or turn off the forwarding.
This decision isn't something Mailman should be doing because it depends on context and guessing what the subscriber wants. Really, this is an owner problem, not a Mailman problem.
What should I do different as owner? The user was set to disabled without any manual intervention from me as list owner.
As I wrote before, you can set up your lists with a large number of bounces allowed so that their accounts don't get disabled for the period of anticipated absence. You can set the list to send bounce mail that isn't parseable as a standard delivery status notifications (DSN) to you. In this case it would probably have saved everybody a bit of annoyance. Finally, you can filter on that header either using the standard Mailman header filtering[1], or better yet, in the MTA as you suggest:
It's not obvious what *we* should do.
My proposal would be to ignore messages sent to the bounce address with the 'Auto-Submitted: auto-replied (vacation)' header. I'm thinking about doing that change on the MTA level,
Filtering in the MTA the right thing to do. You can already filter and discard such messages in Mailman using the "header filtering" capabilities, I think. But as always, this kind of filtering is far more efficient at the MTA because the MTA is doing a lot of it anyway at most sites.
It's also not easy to be sure we get it right. The 'Auto-Submitted' field is standard (RFC 3834), but unfortunately "Auto-Submitted: auto-replied" is the RFC-3834-recommended format for DSNs, ie, bounce messages. We do parse standard DSNs for soft vs. hard bounces, so we can identify standard DSNs but unparsable DSNs still show up occasionally, which are not distinguishable from vacation messages using Auto-Submitted. Relying on the string "(vacation)", which is a comment, would suck for us. We would inevitably get requests for translations into a few dozen languages and additions of "(out of office)" etc, etc, all for a few users / providers (I can't recall hearing of this problem until you brought it up, although it seems like a plausible bug at providers).
but I believe it would be more appropriate at the mailman level.
I don't see a good reason to have a standard feature for this in Mailman, because (1) based on the Auto-Submitted field as defined in the standard we can't distinguish "vacation" from a DSN, (2) the comment is arbitrary which would impose a maintenance burden on us as variants and translations appear in that comment, and (3) both Mailman and MTAs already provide sufficiently configurable header filtering.
Sorry, I meant "disabled". A contributing fact was for sure that the users mail setup sent out the vacation mail on any incoming mail, not only the first, and also on mails with precendence list. I adressed that already to the user (because I believe it's a bug),
RFC 3834 which specifically calls for efforts to not spam from auto- responder users.
(Especially as the mail provider is gmx, this is *the* largest webmail provider in Germany, so both the probability of fixes is rather low, and the probability of happening with users users is not too low.)
That's important to you, and other admins in Germany, and I sympathize. But for us there's no guarantee that other providers will use exactly that format, or won't allow users to specify the comment (even worse). So I respectfully ask you to use the filtering capabilities you already have available to you.
Steve
Footnotes: [1] I should check this, as filtering probably takes place differently for bounce messages and the rest of Mailman. If we don't already have filtering on bounce messages, I would support adding it to the chain for bounce messages, with the same UI but allowing it to have a different set of filtering rules from ordinary posts and email commands for configuring subscripts.
On 8/24/21 11:17 AM, Andreas Barth wrote:
Well, the mail adress got disabled, and the user got warning mails ("Your subscription for list mailing list has been disabled"). As none of the list owners did that, I don't think there is any other explanation. Please note that individualsation was enabled, so all mails are sent out with per-subscriber mail envelope address, which then makes bounce detection quite straight-forward.
If by 'individualsation was enabled' you mean messages were VERPed, there could be a bug here.
There are a couple of considerations. First the vacation responder should not respond to list messages. See <https://www.rfc-editor.org/rfc/rfc5230.html#section-4.6>.
That said, bounce processing should treat even a VERPed message as unrecognized if it doesn't look like some kind of DSN. This may not be the case in MM 3.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
- Mark Sapiro (mark@msapiro.net) [210830 04:30]:
On 8/24/21 11:17 AM, Andreas Barth wrote:
Well, the mail adress got disabled, and the user got warning mails ("Your subscription for list mailing list has been disabled"). As none of the list owners did that, I don't think there is any other explanation. Please note that individualsation was enabled, so all mails are sent out with per-subscriber mail envelope address, which then makes bounce detection quite straight-forward.
If by 'individualsation was enabled' you mean messages were VERPed, there could be a bug here.
Yes, that's what happens.
There are a couple of considerations. First the vacation responder should not respond to list messages. See <https://www.rfc-editor.org/rfc/rfc5230.html#section-4.6>.
I fully agree with you, and already responded with the same remark to the user before sending my initial mail here.
That said, bounce processing should treat even a VERPed message as unrecognized if it doesn't look like some kind of DSN. This may not be the case in MM 3.
Thanks, that's what I also think. Looking at the returned mail, they definitly don't look like DSNs / bounces to me.
Anything I should be doing re this topic?
Thanks, Andi
On 8/29/21 10:19 PM, Andreas Barth wrote:
* Mark Sapiro (mark@msapiro.net) [210830 04:30]:
That said, bounce processing should treat even a VERPed message as unrecognized if it doesn't look like some kind of DSN. This may not be the case in MM 3.
Thanks, that's what I also think. Looking at the returned mail, they definitly don't look like DSNs / bounces to me.
Anything I should be doing re this topic?
I've created https://gitlab.com/mailman/mailman/-/issues/939 for this. I think this patch will fix it. --- a/src/mailman/runners/bounce.py +++ b/src/mailman/runners/bounce.py @@ -51,13 +51,18 @@ class BounceRunner(Runner): addresses = StandardVERP().get_verp(mlist, msg) if len(addresses) > 0: # Scan the message to see if it contained permanent or temporary - # failures. We'll ignore temporary failures, but even if there - # are no permanent failures, we'll assume VERP bounces are - # permanent. + # failures. We'll ignore temporary failures, and if there + # are no permanent failures, we'll assume this is a vacation + # response or similar. temporary, permanent = all_failures(msg) if len(temporary) > 0: # This was a temporary failure, so just ignore it. return False + if len(permanent) == 0: + log.info('VERPed bounce message but not a recognized DSN: %s', + msg.get('message-id', 'n/a')) + maybe_forward(mlist, msg) + return False else: # See if this was a probe message. addresses = ProbeVERP().get_verp(mlist, msg) (Watch for wrapped lines) -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Andreas Barth -
Mark Sapiro -
Stephen J. Turnbull