New subject: [Suspected Spam]Exim, Mailman3 and missing Message-ID:

26 Jul 2019 · *very*


      Just a note that may help some people.
After a bit of a learning curve, managed to get Mailman3 working in Debian
10 using the Debian package.
I have mailman3-full + exim4 + apache2 setup:
I did quite a lot of testing and getting it working, then when all seemed
well, declared success and made it live.
As (bad) luck would have it, the very first real message bounced: :-(
2019-07-25 20:53:09 1hqjnY-0005ms-RD <= xxxxxxx@ntlworld.com H=cloudi.spamtitan.com [52.26.55.167] P=esmtps X=TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no S=7854 DKIM=ntlworld.com
2019-07-25 20:53:09 1hqjnY-0005ms-RD ** mylist@lentil.org R=mailman3_router T=mailman3_t H=127.0.0.1 [127.0.0.1]: LMTP error after DATA: 550 No Message-ID header provided
2019-07-25 20:53:09 1hqjnZ-0005mx-3F <= <> R=1hqjnY-0005ms-RD U=Debian-exim P=local S=9222
The user forwarded me the bounce, which contained the original message and
headers, and indeed the original message did not contain a Message-ID:
header. (Oddly, his message to me reporting the error using the same mail
client DID contain a Message-ID: header.)
Mail client: X-Mailer: Microsoft Outlook 16.0
So it seems in certain circumstances, Outlook doesn't put a Message-ID:
header in its messages.
The change from Mailman2.x to Mailman3 means it's now using LMTP to deliver
to mailman instead of a local submission seems to have exposed this issue.
Exim tries to be helpful for "locally submitted" messages:
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-message_process...
"12. The Message-ID: header line
If a locally-generated or submission-mode incoming message does not contain
a Message-ID: or Resent-Message-ID: header line, and the
suppress_local_fixups control is not set, Exim adds a suitable header line
to the message. If there are any Resent-: headers in the message, it creates
Resent-Message-ID:. The id is constructed from Exim's internal message id,
preceded by the letter E to ensure it starts with a letter, and followed by
@ and the primary host name."
However: "Note: Messages received over TCP/IP on the loopback interface
(127.0.0.1 or ::1) are not considered to be locally-originated. Exim does
not treat the loopback interface specially in any way."
Therefore, Mailman3 using LMTP means exim will NOT "fix up" this problem
before mailman3 receives the message.
My solution for now (apart from telling people not to use old broken
Outlook!) is to modify the exim mailman3_transport so that any message with
a missing Message-ID: will have one generated by exim before it's passed to
LMTP. This reproduces the same behaviour exim uses for locally submitted
messages.
(headers_remove ensures that we end up only with one Message-ID: header and
not two, then we put back either the original Message-ID, or if that's not
there, use Exim's ID to generate a new Message-ID header.)
Here's an updated version of the example at:
https://mailman.readthedocs.io/en/latest/src/mailman/docs/mta.html#exim
# /etc/exim4/conf.d/main/25_mm3_macros
# The colon-separated list of domains served by Mailman.
domainlist mm_domains=list.example.net
MM3_LMTP_PORT=8024
# MM3_HOME must be set to mailman's var directory, wherever it is
# according to your installation.
# debian default is /var/lib/mailman3
MM3_HOME=/opt/mailman/var
MM3_UID=list
MM3_GID=list
################################################################
# The configuration below is boilerplate:
# you should not need to change it.
# The path to the list receipt (used as the required file when
# matching list addresses)
MM3_LISTCHK=MM3_HOME/lists/${local_part}.${domain}
# /etc/exim4/conf.d/router/455_mm3_router
mailman3_router:
driver = accept
domains = +mm_domains
require_files = MM3_LISTCHK
local_part_suffix_optional
local_part_suffix = 

-bounces   : -bounces+* : 

-confirm   : -confirm+* : 

-join      : -leave     : 

-owner     : -request   : 

-subscribe : -unsubscribe
transport = mailman3_transport
# /etc/exim4/conf.d/transport/55_mm3_transport
mailman3_transport:
driver = smtp
protocol = lmtp
allow_localhost
hosts = localhost
port = MM3_LMTP_PORT
rcpt_include_affixes = true
mailman3_transport:
driver = smtp
protocol = lmtp
allow_localhost
hosts = localhost
port = MM3_LMTP_PORT
rcpt_include_affixes = true
headers_remove = message-id
headers_add = "Message-ID: ${if def:header_message-id:{$h_message-id:}{}}"

Exim, Mailman3 and missing Message-ID:

Robert Lister

Mark Sapiro

Stephen J. Turnbull

Mark Sapiro

Rob Lister

Mark Sapiro

Stephen J. Turnbull

tags

participants (4)