Hi,
It seems, that mailman bounces the bounces, so that the list-owner never sees them:
--8<---------------cut here---------------start------------->8--- postfix/smtpd[12002]: connect from out04.smtpout.orange.fr[193.252.22.213] postfix/smtpd[12002]: 879D6F12E8: client=out04.smtpout.orange.fr[193.252.22.213] postfix/cleanup[11841]: 879D6F12E8: message-id=<> postfix/qmgr[10092]: 879D6F12E8: from=<>, size=4048, nrcpt=1 (queue active) postfix/smtpd[12002]: disconnect from out04.smtpout.orange.fr[193.252.22.213] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5 postfix/lmtp[11846]: 879D6F12E8: to=<test-bounces@my-domain.org>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.05, delays=0.04/0/0/0.01, dsn=5.0.0, status=bounced (host 127.0.0.1[127.0.0.1] said: 550 No Message-ID header provided (in reply to end of DATA command)) postfix/qmgr[10092]: 879D6F12E8: removed --8<---------------cut here---------------end--------------->8---
What could I do please, to get the bounces? I would like to use VERP to identify wrong addresses...
TIA for any hints,
Peter
On 01/30/2018 12:15 AM, Peter Münster wrote:
It seems, that mailman bounces the bounces, so that the list-owner never sees them:
--8<---------------cut here---------------start------------->8--- postfix/smtpd[12002]: connect from out04.smtpout.orange.fr[193.252.22.213] postfix/smtpd[12002]: 879D6F12E8: client=out04.smtpout.orange.fr[193.252.22.213] postfix/cleanup[11841]: 879D6F12E8: message-id=<> postfix/qmgr[10092]: 879D6F12E8: from=<>, size=4048, nrcpt=1 (queue active) postfix/smtpd[12002]: disconnect from out04.smtpout.orange.fr[193.252.22.213] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5 postfix/lmtp[11846]: 879D6F12E8: to=<test-bounces@my-domain.org>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.05, delays=0.04/0/0/0.01, dsn=5.0.0, status=bounced (host 127.0.0.1[127.0.0.1] said: 550 No Message-ID header provided (in reply to end of DATA command)) postfix/qmgr[10092]: 879D6F12E8: removed --8<---------------cut here---------------end--------------->8---
There are a few things going on here. It appears that out04.smtpout.orange.fr has bounced a list post from the <test@my-domain.org> list and and sent a DSN back to Mailman at <test-bounces@my-domain.org>. It is this DSN that has no Message-ID:. Mailman's LMTP runner rejects such messages.
The underlying issue is the orange.fr domain is sending DSNs without a Message-ID: header.
What could I do please, to get the bounces? I would like to use VERP to identify wrong addresses...
If you used VERP (i.e. set verp_delivery_interval = 1 in the [mta] section of your config), this log message
postfix/lmtp[11846]: 879D6F12E8: to=<test-bounces@my-domain.org>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.05, delays=0.04/0/0/0.01, dsn=5.0.0, status=bounced (host 127.0.0.1[127.0.0.1] said: 550 No Message-ID header provided (in reply to end of DATA command))
would contain to=<test-bounces+user=example.com@my-domain.org> and you could identify the bouncing address (user@example.com in this case) that way.
Beyond that, there are still other issues, the biggest of which is currently, Mailman is not processing bounces other than storing them in the database. See <https://gitlab.com/mailman/mailman/issues/343>.
Also, the LMTP runner requires a Message-ID: at least in part because that is part of the algorithm for creating a unique ID for archiving. Given that messages to a -bounces address are handled differently and not archived, it would be possible to not require a Message-ID: for those messages, but arguing against that, if a recipient MTA can't create a DSN that has a Message-ID:, would you trust what else it does?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Jan 30 2018, Mark Sapiro wrote:
The underlying issue is the orange.fr domain is sending DSNs without a Message-ID: header.
Yes. Is a Message-ID header required in a DSN by any RFC?
If you used VERP (i.e. set verp_delivery_interval = 1 in the [mta] section of your config), this log message
I have verp_delivery_interval = 10 ...
would contain to=<test-bounces+user=example.com@my-domain.org> and you could identify the bouncing address (user@example.com in this case) that way.
Yes, that's my intention, but first of all, mailman must receive the DSN.
Beyond that, there are still other issues, the biggest of which is currently, Mailman is not processing bounces other than storing them in the database. See <https://gitlab.com/mailman/mailman/issues/343>.
Shouldn't the DSNs be forwarded to the owner of the list?
if a recipient MTA can't create a DSN that has a Message-ID:, would you trust what else it does?
I don't know. The question for me is: does the remote MTA needs a fix (i.e. the Message-ID is required by some RFC), or mailman (i.e. a DSN without Message-ID should be acceptable)?
-- Peter
On 01/30/2018 11:38 PM, Peter Münster wrote:
On Tue, Jan 30 2018, Mark Sapiro wrote:
The underlying issue is the orange.fr domain is sending DSNs without a Message-ID: header.
Yes. Is a Message-ID header required in a DSN by any RFC?
It's not a MUST, just a SHOULD, but RFC 5322, sec 3.6.4 says
Though listed as optional in the table in section 3.6, every message SHOULD have a "Message-ID:" field.
If you used VERP (i.e. set verp_delivery_interval = 1 in the [mta] section of your config), this log message
I have verp_delivery_interval = 10 ...
Which says only every 10th message will be VERPed. You want verp_delivery_interval = 1 to VERP every message.
would contain to=<test-bounces+user=example.com@my-domain.org> and you could identify the bouncing address (user@example.com in this case) that way.
Yes, that's my intention, but first of all, mailman must receive the DSN.
The message does appear in the mail log, but until the issues below are fixed, even if Mailman receives it, you won't see it.
Beyond that, there are still other issues, the biggest of which is currently, Mailman is not processing bounces other than storing them in the database. See <https://gitlab.com/mailman/mailman/issues/343>.
Shouldn't the DSNs be forwarded to the owner of the list?
They should be processed by Mailman and scored as bounces for the recipient if recognized and forwarded if not (depending on settings), but they currently are only stored.
if a recipient MTA can't create a DSN that has a Message-ID:, would you trust what else it does?
I don't know. The question for me is: does the remote MTA needs a fix (i.e. the Message-ID is required by some RFC), or mailman (i.e. a DSN without Message-ID should be acceptable)?
I have filed <https://gitlab.com/mailman/mailman/issues/448> for this.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, Jan 31 2018, Mark Sapiro wrote:
I have verp_delivery_interval = 10 ...
Which says only every 10th message will be VERPed. You want verp_delivery_interval = 1 to VERP every message.
Hi,
Why? The smtp relay host would get much more outgoing messages. With verp_delivery_interval = 10, only 10% of the messages would be personalized. Shouldn't this be enough to detect sooner or later, that a recipient address is no more valid? Or is it important, to detect such failure immediately?
The message does appear in the mail log, but until the issues below are fixed, even if Mailman receives it, you won't see it.
Ok. For now, I could do a grep of the syslog messages of Postfix, for example once per month, to check for DSNs.
They should be processed by Mailman and scored as bounces for the recipient if recognized and forwarded if not (depending on settings), but they currently are only stored.
Ok.
The question for me is: does the remote MTA needs a fix (i.e. the Message-ID is required by some RFC), or mailman (i.e. a DSN without Message-ID should be acceptable)?
I have filed <https://gitlab.com/mailman/mailman/issues/448> for this.
Thanks! That means, that I won't need to teach Postfix to add a Message-ID to the DSNs. :)
Kind regards,
PeterOn 01/31/2018 11:57 AM, Peter Münster wrote:
On Wed, Jan 31 2018, Mark Sapiro wrote:
I have verp_delivery_interval = 10 ...
Which says only every 10th message will be VERPed. You want verp_delivery_interval = 1 to VERP every message.
Hi,
Why? The smtp relay host would get much more outgoing messages. With verp_delivery_interval = 10, only 10% of the messages would be personalized. Shouldn't this be enough to detect sooner or later, that a recipient address is no more valid? Or is it important, to detect such failure immediately?
Yes, VERP does increase the number of SMTP transactions to deliver the mail from mailman to the outgoing MTA, but in general it has mush less effect on deliveries from the outgoing MTA to the recipient MXs, but with modern hardware, most people seem to find the added overhead of VERP to be minimal. Also, it is possible to have the MTA do the VERPing rather than Mailman. See <https://bugs.launchpad.net/mailman/+bug/558067> for a way to do this in Mailman 2.1. There's nothing for Mailman 3, but all that's needed is to configure Postfix to do it and add an appropriate mail_options= to the smtplib.SMTP.sendmail() calls to enable it.
In any case, consider (assuming automated bounce processing is working, which it currently isn't) a list with bounce_info_stale_after = 7 days that gets even a post per day and has a member who is bouncing with a DSN that can't be successfully parsed for the member address (maybe because the bounce is from a forward to a different address) so only a VERPed bounce will be successfully scored.
At some point a VERPed post bounces for this member and the member gets a bounce score of 1. Now, the next 9 posts to this list will not be VERPed and by the time the 10th post bounces it's more than a week since the first bounce and the score is reset so this member's delivery is never disabled.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Jan 30 2018, Mark Sapiro wrote:
Mailman is not processing bounces other than storing them in the database. See <https://gitlab.com/mailman/mailman/issues/343>.
Hi,
It seems, that the content (the body) of the bounce-message is not stored. How could I please get and see the content?
TIA for any hints,
PeterOn 12/11/19 8:16 AM, Peter Münster wrote:
On Tue, Jan 30 2018, Mark Sapiro wrote:
Mailman is not processing bounces other than storing them in the database. See <https://gitlab.com/mailman/mailman/issues/343>.
Hi,
It seems, that the content (the body) of the bounce-message is not stored. How could I please get and see the content?
You need to tweek your MTA to copy -bounces mail to you or just send them to you since they aren't processed anyway. Alternatively, you could patch Mailman to forward all bounces as 'unrecognized.
--- a/src/mailman/runners/bounce.py +++ b/src/mailman/runners/bounce.py @@ -77,6 +77,7 @@ class BounceRunner(Runner): # bytes/8-bit strings, but we must store them as unicodes in the # database. Assume utf-8 encoding, but be cautious. if len(addresses) > 0: + maybe_forward(mlist, msg) for address in addresses: if isinstance(address, bytes): try:
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, Dec 11 2019, Mark Sapiro wrote:
You need to tweek your MTA to copy -bounces mail to you or just send them to you since they aren't processed anyway.
Thanks, it was much simpler, than I guessed: virtual_alias_domains takes precedence over relay_recipient_maps, so I only needed to add some lines to /etc/postfix/virtual.
-- Peter
participants (3)
-
Mark Sapiro -
Peter Münster -
Peter Münster