posting bounced with 501 Message has defects
Dear All,
One of our users posted an email using a Bluemail Desktop client on one of the mailing lists. This email bounced with the reason "501 Message has defects". However, it gets accepted if the same user posts the same email on the same mailing list using Webmail (RoundCube) or Thunderbird. There is nothing in mailman.log and bounce.log, even in debug mode. Following is the SMTP log for the same email posting and the email header. Any guidance in debugging this will be highly appreciated.
Regards, Hitachi.
*SMTP.LOG:*
Apr 21 18:00:47 2025 (1979121) Available AUTH mechanisms: LOGIN(builtin) PLAIN(builtin) Apr 21 18:00:47 2025 (1979121) Peer: ('127.0.0.1', 44502) Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) handling connection Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'220 lists.example.com GNU Mailman LMTP runner 2.0' Apr 21 18:00:47 2025 (1979121) _handle_client readline: b'LHLO lists\r\n' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) >> b'LHLO lists' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b' 250-lists.example.com' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'250-SIZE 33554432' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'250-8BITMIME' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'250 HELP' Apr 21 18:00:47 2025 (1979121) _handle_client readline: b'MAIL FROM:< user1@example.com> SIZE=3086\r\n' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) >> b'MAIL FROM:< user1@example.com> SIZE=3086' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) sender: user1@example.com Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'250 OK' Apr 21 18:00:47 2025 (1979121) _handle_client readline: b'RCPT TO:< prodlist-mm3@example.com>\r\n' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) >> b'RCPT TO:< prodlist-mm3@example.com>' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) recip: prodlist-mm3@example.com Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'250 Ok' Apr 21 18:00:47 2025 (1979121) _handle_client readline: b'DATA\r\n' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) >> b'DATA' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'354 End data with <CR><LF>.<CR><LF>' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: from relay.example.com (relay.example.com [10.198.5.2])\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tby lists (Postfix) with QMQP id 9AFF48D\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tfor < prodlist-mm3@example.com>; Mon, 21 Apr 2025 18:00:47 +0530 (IST)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Delivered-To: CLUSTERHOST relay.example.com prodlist-mm3@example.com\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: (qmail 7951 invoked from network); 21 Apr 2025 18:00:47 +0530\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: from unknown (HELO smtp2.example.com) ([10.198.12.2])\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' (envelope-sender < user1@example.com>)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' by relay.example.com (qmail-ldap-1.03) with SMTP\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' for < prodlist-mm3@example.com>; 21 Apr 2025 18:00:47 +0530\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: (qmail 2533 invoked by uid 510); 21 Apr 2025 18:00:47 +0530\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Qmail-Scanner-Diagnostics: from 10.198.12.3 by smtp2 (envelope-from < user1@example.com>, uid 501) with qmail-scanner-2.11\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' spamassassin: 3.4.1. mhr: 1.0. {clamdscan: 0.100.0/26337} \r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' Clear:RC:1(10.198.12.3):SA:0(-6.0/7.0):. Processed in 3.727185 secs; 21 Apr 2025 18:00:47 +0530\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp2.example.com\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Spam-Level: \r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Spam-Status: No, score=-6.0 required=7.0 tests=BAYES_20,HTML_MESSAGE,\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tIITB_ORIG,IITB_TARGET_ADDRESS,INTRA_IITB_MAIL,PROPER_IITB_MSGID\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tautolearn=disabled version=3.4.1\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Spam-Pyzor: Reported 0 times.\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Envelope-From: user1@example.com\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Qmail-Scanner-Mime-Attachments: |\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'X-Qmail-Scanner-Zip-Files: |\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: from unknown (HELO smtp3.example.com) (10.198.12.3)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' by smtp2.example.com with SMTP; 21 Apr 2025 18:00:43 +0530\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: from smtp-auth.example.com (smtpauth1.example.com [10.198.8.1])\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tby smtp3.example.com (Postfix) with ESMTP id B70D4341971\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tfor < prodlist-mm3@example.com>; Mon, 21 Apr 2025 18:00:43 +0530 (IST)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Received: from localhost ( pruthvi.example.com [10.100.100.95])\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\t(No client certificate requested)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\t(Authenticated sender: user1@example.com)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tby smtp-auth.example.com (Postfix) with ESMTPSA id AB0CB30C340\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\tfor < prodlist-mm3@example.com>; Mon, 21 Apr 2025 18:00:43 +0530 (IST)\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Content-Type: multipart/alternative;\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' boundary="----sinikael-?=_1-17452386440190.1454723563624738"\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'From: user1 < user1@example.com>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Subject: test mail 14\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'To: prodlist-mm3@example.com \r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Message-Id: < 934811260a4bf.44bbdcec3e1d9@example.com>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Content-Transfer-Encoding: quoted-printable\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Date: Mon, 21 Apr 2025 12:30:38 +0000\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'MIME-Version: 1.0\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'------sinikael-?=_1-17452386440190.1454723563624738\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Content-Type: text/plain\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Content-Transfer-Encoding: 7bit\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'test mail 14 Get BlueMail for Desktop\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'------sinikael-?=_1-17452386440190.1454723563624738\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Content-Type: text/html; format=flowed\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'Content-Transfer-Encoding: 7bit\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'<html><head></head><body><div>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' <div dir="auto" id="compose-body-wrapper">test mail 14</div><div \r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'dir="auto" id="compose-body-wrapper"><br></div><div dir="auto" \r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'id="compose-body-wrapper"><br>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' <div dir="auto"><br></div>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' <div dir="auto" id="tmjah_g_1299">\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' Get BlueMail for Desktop\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' </div>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' <div dir="auto"><br></div>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' </div>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b' </div></body></html>\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'------sinikael-?=_1-17452386440190.1454723563624738--\r\n' Apr 21 18:00:47 2025 (1979121) DATA readline: b'.\r\n' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'501 Message has defects' Apr 21 18:00:47 2025 (1979121) _handle_client readline: b'QUIT\r\n' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) >> b'QUIT' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'221 Bye' Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) connection lost Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) Connection lost during _handle_client()
*EMAIL HEADER:*
Content-Type: multipart/alternative; boundary="----sinikael-?=_1-17452386441150.6962994414278503" From: user1 <user1@example.com> Subject: test mail 14 To: prodlist-mm3@example.com Message-Id: <934811260a4bf.44bbdcec3e1d9@example.com> Content-Transfer-Encoding: quoted-printable Date: Mon, 21 Apr 2025 12:30:38 +0000 MIME-Version: 1.0
------sinikael-?=_1-17452386441150.6962994414278503 Content-Type: text/plain Content-Transfer-Encoding: 7bit
test mail 14 Get BlueMail for Desktop ------sinikael-?=_1-17452386441150.6962994414278503 Content-Type: text/html; format=flowed Content-Transfer-Encoding: 7bit
<html><head></head><body><div> <div dir="auto" id="compose-body-wrapper">test mail 14</div><div dir="auto" id="compose-body-wrapper"><br></div><div dir="auto" id="compose-body-wrapper"><br> <div dir="auto"><br></div> <div dir="auto" id="tmjah_g_1299"> Get BlueMail for Desktop </div> <div dir="auto"><br></div> </div> </div></body></html> ------sinikael-?=_1-17452386441150.6962994414278503--
Hitashi mancho via Mailman-users writes:
One of our users posted an email using a Bluemail Desktop client on one of the mailing lists. This email bounced with the reason "501 Message has defects".
Bouncing the message is arguably an overreaction to this particular defect, but in fact Bluemail is violating a "MUST NOT" in RFC 2045. Mailman takes the position that in principle we should not forward non-conforming mail. At least one expert has told me he considers this defect harmless, but (a) it's not clear to me that it is harmless (especially in the hands of a malicious actor targeting carelessly- written applications) and (b) it's hard to get excited about adding complicated logic to the code to make life easier for an application that doesn't conform to a trivial 30-year-old restriction.
However, it gets accepted if the same user posts the same email on the same mailing list using Webmail (RoundCube) or Thunderbird.
I'm not familiar with the maintainers of RoundCube, but Thunderbird has a long history of respect for Internet standards.
There is nothing in mailman.log and bounce.log, even in debug mode.
Since this is happening during receipt of the message I would guess that all messages related to the problem would be in Mailman's smtp.log. I don't think you'd see anything about this in mailman.log unless the defect managed to crash the lmtp runner or all of Mailman. bounce.log reports bounces *received* by Mailman, not bounces caused by Mailman.
Apr 21 18:00:47 2025 (1979121) ('127.0.0.1', 44502) << b'501 Message has defects'
I seem to recall that Mark submitted a patch to add the defect list to this message. I will follow up on that (although it doesn't help get the message delivered to subscribers, it will make diagnosis a lot easier and faster).
Content-Type: multipart/alternative; boundary="----sinikael-?=_1-17452386441150.6962994414278503" Content-Transfer-Encoding: quoted-printable
This the problem. When Content-Type is 'multipart', the only values allowed for Content-Transfer-Encoding are '7-bit', '8-bit' and 'binary'. This is a "what were they thinking?!" level bug.
-- GNU Mailman consultant (installation, migration, customization) Sirus Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
participants (2)
-
Hitashi mancho -
Stephen J. Turnbull