Apple Mail encoding again
Starting a new thread because I don't know how to reply to this one from November 2021:
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
A subscriber's post to a list bounced with a 501 bad encoding error. The subscriber uses Apple Mail on MacOS and PGP-signed his message.
The earlier post above suggests the problem could be a bug in either Apple Mail encoding or python's smtp module.
To rule out the smtp module, I sent a PGP-signed message using a different MUA (Thunderbird, in my case). Mailman3 accepted this post without complaint, and I reported back to list users that this appears to be an Apple Mail encoding issue.
Normally, that'd be the end of it. But this is a list for a bunch of Internet graybeards, including one of the authors of the SMTP protocols and the subscriber is a former IETF chair. He's grumbling that if this really were an Apple Mail issue, wouldn't it be far more widespread?
I don't think it would be, because (a) Mailman2 is still far more pervasive than Mailman3 and (b) the combination of PGP signing and Apple Mail on MacOS and Mailman3 is still pretty unusual.
Has anyone else hit this encoding issue, other than the post above? And did Mailman2 silently accept this because it was less strict about checking for bad encoding?
Thanks.
dn
On 4/12/22 11:45, David Newman wrote:
To rule out the smtp module, I sent a PGP-signed message using a different MUA (Thunderbird, in my case). Mailman3 accepted this post without complaint, and I reported back to list users that this appears to be an Apple Mail encoding issue.
Normally, that'd be the end of it. But this is a list for a bunch of Internet graybeards, including one of the authors of the SMTP protocols and the subscriber is a former IETF chair. He's grumbling that if this really were an Apple Mail issue, wouldn't it be far more widespread?
Maybe, but who knows.
The issue is with at least macOS 12.0.1, Mail.app Version 15.0 (3693.20.0.1.32), and GPG Mail 6.0, Build 2023.
That combination at least produces multipart/signed messages with multipart/* parts containing a Content-Transfer-Encoding: header.
The Python 3 email library considers this to be a InvalidMultipartContentTransferEncodingDefect() message defect.
I had a long exchange with Barry Warsaw about this. an excerpt is
It's definitely an Apple Mail issue. Both the message you sent and the original message part have the same defect: InvalidMultipartContentTransferEncodingDefect()
Also, My Thunderbird MUA wouldn't display the message content at all.
Interesting though that your original message was also signed and sent by Apple Mail and didn't have the defect. It's structure was
multipart/signed multipart/alternative text/plain text/html application/pgp-signature
and none of the multipart parts have Content-Transfer-Encoding.
The follow up you sent with the original message was
multipart/signed multipart/mixed * text/plain message/rfc822 (the attached message) * multipart/signed * text/plain application/pgp-signature text/plain (an empty plain text part) text/plain (a quote of my message to you) application/pgp-signature
All the asterisked parts have a Content-Transfer-Encoding header which is the defect.
It seems to be related to the presence of the empty text/plain part following the signature.
I don't think it would be, because (a) Mailman2 is still far more pervasive than Mailman3 and (b) the combination of PGP signing and Apple Mail on MacOS and Mailman3 is still pretty unusual.
Has anyone else hit this encoding issue, other than the post above? And did Mailman2 silently accept this because it was less strict about checking for bad encoding?
To my knowledge it's only been hit by Barry and the poster of the original report
Mailman 2.1 is has the Python 2 email package which doesn't report InvalidMultipartContentTransferEncodingDefect().
I reported it in November at https://feedbackassistant.apple.com/feedback/9768496 which is open with no reply.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
That combination at least produces multipart/signed messages with multipart/* parts containing a Content-Transfer-Encoding: header.
The Python 3 email library considers this to be a InvalidMultipartContentTransferEncodingDefect() message defect.
Merely containing C-T-E is not an error per RFC 2045, although it seems to consider it a stupid thing to do. If present, it may be "binary", "7-bit", or "8-bit" (below summarized as "cleartext").
Did Apple Mail apply a non-cleartext C-T-E? Or has an RFC tightening that restriction been published since? (I don't know why it would, though.)
Even with a non-cleartext C-T-E, it seems reasonable for Mailman to treat it as application/octet-stream and pass it through, unless the list is configured to do something like strip application/octet-stream or that requires decoding like spam-filtering (Postel principle, although one could argue it shouldn't be sent on on the same principle -- ah, the joys of being an e-mail Mediator). That would be especially true it if were quoted-printable, since any relevant MIME headers would (presumably) be pure ASCII and therefore unencoded.
multipart/signed multipart/mixed * text/plain message/rfc822 (the attached message) * multipart/signed * text/plain application/pgp-signature text/plain (an empty plain text part) text/plain (a quote of my message to you) application/pgp-signature
It's true that Apple Mail is horked if it produces that structure with C-T-E in the indicated places (but we already knew that :-( ).
Steve
On 4/14/22 08:21, Stephen J. Turnbull wrote:
Mark Sapiro writes:
That combination at least produces multipart/signed messages with multipart/* parts containing a Content-Transfer-Encoding: header.
The Python 3 email library considers this to be a InvalidMultipartContentTransferEncodingDefect() message defect.
Merely containing C-T-E is not an error per RFC 2045, although it seems to consider it a stupid thing to do. If present, it may be "binary", "7-bit", or "8-bit" (below summarized as "cleartext").
Did Apple Mail apply a non-cleartext C-T-E? Or has an RFC tightening that restriction been published since? (I don't know why it would, though.)
Yes. The C-T-E on all the multipart parts was quoted-printable, not allowed under RFC 2045.
Even with a non-cleartext C-T-E, it seems reasonable for Mailman to treat it as application/octet-stream and pass it through, unless the list is configured to do something like strip application/octet-stream or that requires decoding like spam-filtering (Postel principle, although one could argue it shouldn't be sent on on the same principle -- ah, the joys of being an e-mail Mediator). That would be especially true it if were quoted-printable, since any relevant MIME headers would (presumably) be pure ASCII and therefore unencoded.
The Python email library says the message is defective. Mailman's lmtp runner in turn rejects the defective message with a 501 status https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/runners/lmtp.py...
It could accept a defective message or one with specific defects, but this is problematic for further processing. For example, at least one MUA, Thunderbird, renders such a message with an empty body.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 4/14/22 9:50 AM, Mark Sapiro wrote:
On 4/14/22 08:21, Stephen J. Turnbull wrote:
Mark Sapiro writes:
> That combination at least produces multipart/signed messages with > multipart/* parts containing a Content-Transfer-Encoding: header. > > The Python 3 email library considers this to be a > InvalidMultipartContentTransferEncodingDefect() message defect.
Merely containing C-T-E is not an error per RFC 2045, although it seems to consider it a stupid thing to do. If present, it may be "binary", "7-bit", or "8-bit" (below summarized as "cleartext").
Did Apple Mail apply a non-cleartext C-T-E? Or has an RFC tightening that restriction been published since? (I don't know why it would, though.)
Yes. The C-T-E on all the multipart parts was quoted-printable, not allowed under RFC 2045.
Even with a non-cleartext C-T-E, it seems reasonable for Mailman to treat it as application/octet-stream and pass it through, unless the list is configured to do something like strip application/octet-stream or that requires decoding like spam-filtering (Postel principle, although one could argue it shouldn't be sent on on the same principle -- ah, the joys of being an e-mail Mediator). That would be especially true it if were quoted-printable, since any relevant MIME headers would (presumably) be pure ASCII and therefore unencoded.
The Python email library says the message is defective. Mailman's lmtp runner in turn rejects the defective message with a 501 status https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/runners/lmtp.py...
Yes.
To be clear, this is an Apple Mail issue. Mailman 3 accepted PGP-signed messages from Apple Mail on versions prior to 15.0 running on MacOS 12.0.1. Something changed on Apple's part.
It could accept a defective message or one with specific defects, but this is problematic for further processing. For example, at least one MUA, Thunderbird, renders such a message with an empty body.
That's not my experience. Mailman 3 accepted without complaint a PGP-signed message I sent from Thunderbird 91.8.0 and forwarded it to list subscribers with the message body intact.
dn
On 4/14/22 10:18, David Newman wrote:
On 4/14/22 9:50 AM, Mark Sapiro wrote:
It could accept a defective message or one with specific defects, but this is problematic for further processing. For example, at least one MUA, Thunderbird, renders such a message with an empty body.
That's not my experience. Mailman 3 accepted without complaint a PGP-signed message I sent from Thunderbird 91.8.0 and forwarded it to list subscribers with the message body intact.
Yes, and that's what we expect, but MM 3 does not accept the defective message, and T-bird does not render the body of such a message. I.e., if such a message composed by Apple Mail 15 is sent via a non-Mailman channel to a T-bird user, the user can't read it.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 4/14/22 10:45 AM, Mark Sapiro wrote:
On 4/14/22 10:18, David Newman wrote:
On 4/14/22 9:50 AM, Mark Sapiro wrote:
It could accept a defective message or one with specific defects, but this is problematic for further processing. For example, at least one MUA, Thunderbird, renders such a message with an empty body.
That's not my experience. Mailman 3 accepted without complaint a PGP-signed message I sent from Thunderbird 91.8.0 and forwarded it to list subscribers with the message body intact.
Yes, and that's what we expect, but MM 3 does not accept the defective message, and T-bird does not render the body of such a message. I.e., if such a message composed by Apple Mail 15 is sent via a non-Mailman channel to a T-bird user, the user can't read it.
Sorry, I misspoke. The Apple Mail sender used S/MIME signing, not PGP signing.
In Thunderbird, I was able to read an S/MIME-signed message the user sent directly to me complaining about the 501 bounce. Thunderbird said the signature was invalid due to the use of an untrusted CA, but I could read the message body.
dn
Mark Sapiro writes:
On 4/14/22 08:21, Stephen J. Turnbull wrote:
Yes. The C-T-E on all the multipart parts was quoted-printable, not allowed under RFC 2045.
That's what I expected (we're talking about Apple Mail), but thanks for confirming.
The Python email library says the message is defective. Mailman's lmtp runner in turn rejects the defective message with a 501 status https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/runners/lmtp.py...
It could accept a defective message or one with specific defects, but this is problematic for further processing. For example, at least one MUA, Thunderbird, renders such a message with an empty body.
Sure, but that's not our problem. I mean, I have my MUA set to ignore HTML so I get empty bodies all the time! :-) We are perfectly within our rights to just let the Apple Mail and Thunderbird users fight about it. :-) If it causes problems for *our* processing, that's another matter. I don't see why it would, though.
Whether we *want* to do that depends on how much trouble that causes for our main constituency (list and site admins), and whether we can provide discoverable documentation of the problem. My take is that this seems to be an extremely rare problem, and at least some MUAs are able to read the message, so we should let it through.
If we wanted to be really obnoxious, we could add a header to the message
This message does not conform to mail standards.
Some clients may omit some or all of the content.Or we could make passing it an option, perhaps as a pseudo-content type in "Pass these content types" or as a separate option.
I feel strongly enough that Postel implies "let it through" that I'll write up a PR shortly (a few days). Sleep on it, we bring in Abhilash and I guess Barry?, and if you all still don't like it, veto it -- my time is no big deal, it'll only be an hour to put it together I think.
Steve
Bring (I believe) unaffected directly by this, I'm just chiming in from the peanut gallery...
While I agree Apple should fix their software, shouldn't Postel's Law apply here, assuming there is a reasonable interpretation of multiple C-T-E headers?
Skip
Skip Montanaro writes:
While I agree Apple should fix their software, shouldn't Postel's Law apply here, assuming there is a reasonable interpretation of multiple C-T-E headers?
The thing about lists and the Postel Principle is that the Principle applies to both ends of the mailing list: we're supposed to accept garbage coming in to the mailing list, but we're not supposed to send it back out!
There isn't a consistent interpretation of these headers on multiparts with value quoted-printable, which is why they're specifically excluded by RFC 2045. I suspect that the reason they don't screw things up globally for messages produced by Apple Mail is that the entire message is ASCII anyway, so they're actually no-ops as long as the MUA doesn't explicit flag them (which is what Python 3's email package does, and that's why this thread is happening).
On 4/14/22 21:33, Stephen J. Turnbull wrote:
Whether we *want* to do that depends on how much trouble that causes for our main constituency (list and site admins), and whether we can provide discoverable documentation of the problem. My take is that this seems to be an extremely rare problem, and at least some MUAs are able to read the message, so we should let it through.
First, we are not silently dropping this message. We are rejecting it at LMTP time so the sender should be notified.
Then the broader question is should we accept all messages with defects, almost all of which are spam, or only certain defects and which ones.
I note that Apple has now released Mail 16 so maybe they've fixed this. My macs are still on Big Sur, so I can't test this.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
First, we are not silently dropping this message. We are rejecting it at LMTP time so the sender should be notified.
Sure, but you know how users are about being told that their broken MUAs are spewing crap into the Internet. And most of them will get their mail back and will have no idea how to diagnose what the problem is, let alone repair it.
Then the broader question is should we accept all messages with defects, almost all of which are spam, or only certain defects and which ones.
No (for the reason given), yes, and "it depends". This one I think we can pass through by default. Some lists might want to pass some of the defects that are characteristic of spam. For example, lists for projects developing MUAs in Japan. I regularly got mail with unencoded Japanese in Subject, display names, and occasionally Date at one point in time.
I note that Apple has now released Mail 16 so maybe they've fixed this. My macs are still on Big Sur, so I can't test this.
I have a Mac on Monterey but I haven't switched to it yet, and I'm not a regular user of signed mail and I've never ever used Apple Mail. I'll probably need a bit of handholding to set up a test.
Steve
Dear all,
as the original poster, I have asked the user to send a test mail through our mailman3 system (docker image maxking/mailman-core:0.4.2) and the mail passed.
The user is now on Apple Mail 16, maybe that does the trick. Please find enclosed the anonymizied mail source.
Best regards and thanks for caring even if the problems seems to affect only a small number of users! Rainer
On Fr, 15 Apr 2022 09:03:41 -0700 Mark Sapiro mark@msapiro.net wrote:
On 4/14/22 21:33, Stephen J. Turnbull wrote:
Whether we *want* to do that depends on how much trouble that causes for our main constituency (list and site admins), and whether we can provide discoverable documentation of the problem. My take is that this seems to be an extremely rare problem, and at least some MUAs are able to read the message, so we should let it through.
First, we are not silently dropping this message. We are rejecting it at LMTP time so the sender should be notified.
Then the broader question is should we accept all messages with defects, almost all of which are spam, or only certain defects and which ones.
I note that Apple has now released Mail 16 so maybe they've fixed this. My macs are still on Big Sur, so I can't test this.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Viele Grüße
Rainer Herbst Leiter IT-Service Phone: +49 331 7499-257 e-mail: rainer.herbst@aip.de https://www.aip.de
Leibniz-Institut für Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung bürgerlichen Rechts Stiftungsverzeichnis Brandenburg: 26 742-00/7026
Mailman's content filtering has removed the following MIME parts from this message.
Content-Type: application/x-pkcs7-signature Name: smime.p7s
Dear all,
my first post was to optimistic - the user tested some other combinations and when he tries to send a signed HTML mail, the server again refuses to accept it with "Diagnostic-Code: smtp; 501 Message has defects".
The user suggested that the problem is caused by a line containing only "-- " somewhere in the text, but I can not confirm this.
Please find enclosed the refused mail (anonymized).
Regards Rainer
On Mi, 20 Apr 2022 10:51:38 +0200 Rainer Herbst rainer.herbst@aip.de wrote:
Dear all,
as the original poster, I have asked the user to send a test mail through our mailman3 system (docker image maxking/mailman-core:0.4.2) and the mail passed.
The user is now on Apple Mail 16, maybe that does the trick. Please find enclosed the anonymizied mail source.
Best regards and thanks for caring even if the problems seems to affect only a small number of users! Rainer
On Fr, 15 Apr 2022 09:03:41 -0700 Mark Sapiro mark@msapiro.net wrote:
On 4/14/22 21:33, Stephen J. Turnbull wrote:
Whether we *want* to do that depends on how much trouble that causes for our main constituency (list and site admins), and whether we can provide discoverable documentation of the problem. My take is that this seems to be an extremely rare problem, and at least some MUAs are able to read the message, so we should let it through.
First, we are not silently dropping this message. We are rejecting it at LMTP time so the sender should be notified.
Then the broader question is should we accept all messages with defects, almost all of which are spam, or only certain defects and which ones.
I note that Apple has now released Mail 16 so maybe they've fixed this. My macs are still on Big Sur, so I can't test this.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Viele Grüße
Rainer Herbst Leiter IT-Service Phone: +49 331 7499-257 e-mail: rainer.herbst@aip.de https://www.aip.de
Leibniz-Institut für Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung bürgerlichen Rechts Stiftungsverzeichnis Brandenburg: 26 742-00/7026
Viele Grüße
Rainer Herbst Leiter IT-Service Phone: +49 331 7499-257 e-mail: rainer.herbst@aip.de https://www.aip.de
Leibniz-Institut für Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung bürgerlichen Rechts Stiftungsverzeichnis Brandenburg: 26 742-00/7026
On 4/20/22 07:54, Rainer Herbst wrote:
Dear all,
my first post was to optimistic - the user tested some other combinations and when he tries to send a signed HTML mail, the server again refuses to accept it with "Diagnostic-Code: smtp; 501 Message has defects".
The user suggested that the problem is caused by a line containing only "-- " somewhere in the text, but I can not confirm this.
Please find enclosed the refused mail (anonymized).
That message has no defects.
>>> import email
>>> from mailman.email.message import Message
>>> txt = b""" (the testmail-abgelehnt.txt contents) """
>>> msg = email.message_from_bytes(txt, Message)
>>> msg.defects
[]
>>>
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
On 4/20/22 07:54, Rainer Herbst wrote:
The user suggested that the problem is caused by a line containing only "-- " somewhere in the text, but I can not confirm this.
This is very unlikely. A very large portion of the mail processed at Mailman sites has "-- " in it because that is the conventional "start of .signature" marker, and much of that is multipart and even signed nowadays, yet we've never seen a problem with it.
Please find enclosed the refused mail (anonymized).
That message has no defects.
In particular, it has no Content-Transfer-Encoding in the header of any multipart part, which is the defect reported before. I guess there's no list of defects either in the returned message or in the MTA logs?
It seems strange that the message has no Received headers. Is this normal for aip.de address to aip.de address mail? Or is it a sent file copy of the sender or something like that?
Steve
On Mi, 20 Apr 2022 10:01:55 -0700 Mark Sapiro mark@msapiro.net wrote:
On 4/20/22 07:54, Rainer Herbst wrote:
Dear all,
my first post was to optimistic - the user tested some other combinations and when he tries to send a signed HTML mail, the server again refuses to accept it with "Diagnostic-Code: smtp; 501 Message has defects".
The user suggested that the problem is caused by a line containing only "-- " somewhere in the text, but I can not confirm this.
Please find enclosed the refused mail (anonymized).
That message has no defects.
>>> import email >>> from mailman.email.message import Message >>> txt = b""" (the testmail-abgelehnt.txt contents) """ >>> msg = email.message_from_bytes(txt, Message) >>> msg.defects [] >>>
Maybe I have deleted to much when anonymizing the message...
I tested the whole message and it gave me:
msg.defects [InvalidMultipartContentTransferEncodingDefect()]
Than I reduced the message content. The minimal message that provoked the defect is attached.
When I delete line 12 ("Content-Transfer-Encoding: quoted-printable") from the message, the defect disapears.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Viele Grüße
Rainer Herbst Leiter IT-Service Phone: +49 331 7499-257 e-mail: rainer.herbst@aip.de https://www.aip.de
Leibniz-Institut für Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam
Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung bürgerlichen Rechts Stiftungsverzeichnis Brandenburg: 26 742-00/7026
Mailman's content filtering has removed the following MIME parts from this message.
Content-Type: application/x-pkcs7-signature Name: smime.p7s
Rainer Herbst writes:
Maybe I have deleted to much when anonymizing the message...
I tested the whole message and it gave me:
msg.defects [InvalidMultipartContentTransferEncodingDefect()]
That's the same error. So I guess this is just an error that Apple wants Mail to produce. I'll work on the "pass these defects" patch eventually, but I don't know that it will even get in given that Mark hasn't been sympathetic so far. I've got other stuff scheduled already, so I'm not sure if I'll have time before summer.
Steve
On 21-Apr-22 05:52, Stephen J. Turnbull wrote:
Rainer Herbst writes:
Maybe I have deleted to much when anonymizing the message...
I tested the whole message and it gave me:
msg.defects [InvalidMultipartContentTransferEncodingDefect()]
That's the same error. So I guess this is just an error that Apple wants Mail to produce. I'll work on the "pass these defects" patch eventually, but I don't know that it will even get in given that Mark hasn't been sympathetic so far. I've got other stuff scheduled already, so I'm not sure if I'll have time before summer.
Steve
Mailman-users mailing list --mailman-users@mailman3.org To unsubscribe send an email tomailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Not that I count, but I'm not particularly sympathetic to this either.
An up-to-date Thunderbird user - the message with the "mail-with-defect.txt" attachment, as forwarded by [MME-users] (selected headers below for ID) displays as blank. Interestingly, the Thunderbird error console has no error message related to this.
So there's really no point in passing it along, unless "pass these defects" turns into "repair these defects".
(Does AppleMail present as "Samoware HTML5 6.3.6391280", or is this another instance of the same issue?)
Assuming it's an origin issue, this really seems like an issue that can only be fixed by the sending MUA (Apple).
But it's more complicated than that.
Note that MailMan is encapsulating a multipart/signed message in a multipart/mixed in order to append it's footer, and reports that it removed the signature part of the multipart/signed. This makes the signed message format invalid. So at least, MM3 needs to handle signed messages better: either include the signature or remove the signature headers (an unwrapping exercise to convert the message to unsigned)... Note that this can be recursive where a signed message contains a message/rfc822 part that contains a signed message, that....
I'm not sure it makes sense to encrypt messages to a mailing list, but the format issue is the same. (Well, it might make sense if MM knew how to decrypt with its own key, then archived and/or re-encrypted for list recipients. But I doubt that's of much interest.)
In either case, corrupting messages is a MM issue.
multipart/signed should imply accepting the signature part (in this case, application/x-pkcs7-signature). If signatures aren't allowed by policy, then multipart/signed should be bounced. It's also the case that if a signed message contains an banned part type, simply removing it will break the signature. So in that case, the choice would either be to bounce the message, convert the message to unsigned, or accept the banned part. (The latter might be acceptable in some environments, but has obvious risks.)
Clearly, the current behavior is problematic.
(I don't think it's reasonable to expect MM admins to "know" to configure their lists to pass signature parts if accepting signed - after all, this list is run by experts :-)
X-Mailer: Samoware HTML5 6.3.6391280
MIME-Version: 1.0
From: "Rainer Herbst"rainer.herbst@aip.de
In-Reply-To:ea404088-6355-e5f5-f7b1-160d18110127@msapiro.net
References:775f4ebb-da6e-414f-fac7-a11f99330980@networktest.comcb5d940b-9e03-aeaa-dcb9-fb571a9855cd@msapiro.netm27d7rofmq.fsf@sk.tsukuba.ac.jpa1acc0f8-f1c2-6c70-2ab6-081681d21385@msapiro.netm21qxznexe.fsf@sk.tsukuba.ac.jpd7c2d04d-ea1a-b5dd-b9a4-8e0cc133de03@msapiro.netximss-3471046@cgpro.aip.deximss-3473792@cgpro.aip.deea404088-6355-e5f5-f7b1-160d18110127@msapiro.net
To:mailman-users@mailman3.org
Date: Thu, 21 Apr 2022 10:52:36 +0200
Message-ID:ximss-3477556@cgpro.aip.de
Message-ID-Hash: B3QSDB5CY4543LILTSQBWIDIG6SRBFLX
X-Message-ID-Hash: B3QSDB5CY4543LILTSQBWIDIG6SRBFLX
X-MailFrom:rainer.herbst@aip.de
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-mailman-users.mailman3.org-0; header-match-mailman-users.mailman3.org-1; header-match-mailman-users.mailman3.org-2; header-match-mailman-users.mailman3.org-3; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
X-Content-Filtered-By: Mailman/MimeDel 3.3.6b1
X-Mailman-Version: 3.3.6b1
Precedence: list
Subject: [MM3-users] Re: Apple Mail encoding again
List-Id: General list for Mailman 3 users and others interested in Mailman 3
Actually, here's the entire source as received:
Received: from 10.217.136.93
by atlas308.free.mail.ne1.yahoo.com with HTTPS; Thu, 21 Apr 2022 08:53:17 +0000
Return-Path:mailman-users-bounces+tlhackque=yahoo.com@mailman3.org
X-Originating-Ip: [104.239.228.201]
Received-SPF: pass (domain of mailman3.org designates 104.239.228.201 as permitted sender)
Authentication-Results: atlas308.free.mail.ne1.yahoo.com;
dkim=passheader.i=@mailman3.org header.s=default;
spf=pass smtp.mailfrom=mailman3.org;
dmarc=unknown header.from=aip.de;
X-Apparently-To:tlhackque@yahoo.com; Thu, 21 Apr 2022 08:53:17 +0000
X-YMailISG: tnlCFPYWLDu1S3MzZHoCM6_b06G9UHWhoHP0YZkCNT2_Zjuk
ZJmHZ01FLsGoNsRY9kebXicB55yqEwl7Dp8pgbCN1itcyuDuU325aG22zhQN
4lrxl4EXR35ih5LRI3GeV46FsyI_Iwof4Y8JLU01Kegk_CZhiZAQX2x44ujY
2spNJjWWTfokpWFUpVd3JyFHAWNTUQZtXncb8dqSb6XSbFIuguPLNdiJ8GNY
tuo1i5NzfNXn7C_WfobGO8rejVDcrARFOguimABGWS04mpGgNUCru3kv7tls
vAOZkhcGS4LjFDpMVcPNYZiKn.IBshk19d1zpsGiLfmRwiK4zdAQKD4Bvel6
xeqL2CXqGuDpayiqSuQoOVQFVBBtj_0Y8TYIQRpWO7v1a9vuwrRo7pazNC.N
3LMqR11yQ7_5yKrj1.qPLUw1fpsT9tUz3MRro9.PPhJyF83_cRPWutvtmMin
Wl.25U6QfhgP.lpqhhmpOuSF0F23jHgSIwR_JEl3VkPXFez_Kz6BsTzzZQcq
QyBGHkExpRVY6nTylMY5LpozsE0RpQSR7mHgtXeFGX4.mmYMAT3B.ZKFYAP1
RWhStU1uPgDUMOBVoghykausb8V0u_40Bym0WHrFZP5mJI9BX9Soy2u0us75
quiKjNtNu3cj6FceiqmWwdN9egba4AOPkWKLBlhtDCaCf31V3OzMAdWcPEiV
Uk8vKqO4nHmSUIfKG8PAIJckKlwuMQiJcI_8vRXyl25jg8QbCx9khakNoTRY
VK7LWJpF2qXYo6jjjy2xegvgpi.BJ7_bG0Lcb9t7oXkUMdqyDHy8zJVKXn0x
22dsLAuW43u6rUb01TYztX3Naqr9CAHSI7xYPqPRHDsjiaznbYsknHIwtOTn
KT8YhWXEBz0sChBYLuPxk8NuJNG9osqKSS.jQ3ATOVNOOAwDcaJg5cG9T83z
v.MXzkXqIbBmV7OQw5j.9FX2gnPZOZx7exYSzJeVLjGRvXWlBrT8jqn94tEi
Pirkvajc8IcX3XWfoLg5LlcQACfCp4MtDxN1Pr3Dx.ZAZn0yIRy3Q5eA3Igb
AOHDIwzvBbl1CigzjY8k46yikrCNeiMxNAvoDK6_MZgg4b0o10ImSSclFSOH
oCJKUQoYxiLyFZnHz5WmX0fYkh1DnuUpct63JTmLkrI7EVRnq0rMb0C3GuA8
bQX390oWPMS.JvH1HgxadLSKi4YtJVJLUhKMVONcZPgr_cRNZzH83tVHrI9O
ysHPj0gLCsTnvSSlAlNsNA--
Received: from 104.239.228.201 (EHLO mail.mailman3.org)
by 10.217.136.93 with SMTPs
(version=TLS1_2 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256);
Thu, 21 Apr 2022 08:53:17 +0000
Received: from mailman.iad1.psf.io (ip6-localhost [127.0.0.1])
by mail.mailman3.org (Postfix) with ESMTP id 18C0680203
fortlhackque@yahoo.com; Thu, 21 Apr 2022 08:53:17 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mailman3.org;
s=default; t=1650531197;
bh=UNF2/5reEQ6ViWTr1kb3eTzfBxLgZt+4nMCH7Q6N7eQ=;
h=From:In-Reply-To:References:To:Date:Subject:List-Id:List-Archive:
List-Help:List-Owner:List-Post:List-Subscribe:List-Unsubscribe;
b=UbXFgtbfaWX5nf08zFw7LqiiHHObZzbxfYAK+V0pHh+U5mJmebsiUi839jjq9wLR7
tVknJnx5NkWseGcQeyi6bSFJJX5YHSG6C9C7geRPgj+L/k0mG5Y1HsQGBO2YGd/AcS
cXEcxyoLlQ/G80WEd6TBuRMeBB7MgCYnPfA4Yl2A=
Received: from mailout.aip.de (mailout.aip.de [141.33.160.3])
by mail.mailman3.org (Postfix) with ESMTPS id E4C078018F
formailman-users@mailman3.org; Thu, 21 Apr 2022 08:52:37 +0000 (UTC)
Received: from cgpro.aip.de (communigate.aip.de [141.33.207.216])
by mailout.aip.de (Postfix) with ESMTP id C4365C05D0
formailman-users@mailman3.org; Thu, 21 Apr 2022 10:52:36 +0200 (CEST)
Received: by cgpro.aip.de (Postfix, from userid 0)
id C248C383503; Thu, 21 Apr 2022 10:52:36 +0200 (CEST)
Received: from [141.33.50.228] (accountrherbst@aip.de)
by aip.de (CommuniGate Pro XIMSS 6.3.9)
with HTTPU id 3477555 formailman-users@mailman3.org; Thu, 21 Apr 2022 10:52:36 +0200
X-Mailer: Samoware HTML5 6.3.6391280
MIME-Version: 1.0
From: "Rainer Herbst"rainer.herbst@aip.de
In-Reply-To:ea404088-6355-e5f5-f7b1-160d18110127@msapiro.net
References:775f4ebb-da6e-414f-fac7-a11f99330980@networktest.comcb5d940b-9e03-aeaa-dcb9-fb571a9855cd@msapiro.netm27d7rofmq.fsf@sk.tsukuba.ac.jpa1acc0f8-f1c2-6c70-2ab6-081681d21385@msapiro.netm21qxznexe.fsf@sk.tsukuba.ac.jpd7c2d04d-ea1a-b5dd-b9a4-8e0cc133de03@msapiro.netximss-3471046@cgpro.aip.deximss-3473792@cgpro.aip.deea404088-6355-e5f5-f7b1-160d18110127@msapiro.net
To:mailman-users@mailman3.org
Date: Thu, 21 Apr 2022 10:52:36 +0200
Message-ID:ximss-3477556@cgpro.aip.de
Message-ID-Hash: B3QSDB5CY4543LILTSQBWIDIG6SRBFLX
X-Message-ID-Hash: B3QSDB5CY4543LILTSQBWIDIG6SRBFLX
X-MailFrom:rainer.herbst@aip.de
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-mailman-users.mailman3.org-0; header-match-mailman-users.mailman3.org-1; header-match-mailman-users.mailman3.org-2; header-match-mailman-users.mailman3.org-3; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
X-Content-Filtered-By: Mailman/MimeDel 3.3.6b1
X-Mailman-Version: 3.3.6b1
Precedence: list
Subject: [MM3-users] Re: Apple Mail encoding again
List-Id: General list for Mailman 3 users and others interested in Mailman 3
This is a signed S/MIME message
--===============7924952636743351526== Content-Type: multipart/signed;boundary="_===signed==3477555====cgpro.aip.de===_"; protocol="application/x-pkcs7-signature"; micalg="SHA1"
This is a signed S/MIME message
--_===signed==3477555====cgpro.aip.de===_ Content-Type: multipart/mixed;boundary="_===3477555====cgpro.aip.de===_"
--_===3477555====cgpro.aip.de===_ Content-Type: text/plain; charset="utf-8"; format="flowed" Content-Transfer-Encoding: base64
T24gTWksIDIwIEFwciAyMDIyIDEwOjAxOjU1IC0wNzAwDQogIE1hcmsgU2FwaXJvIDxtYXJrQG1z YXBpcm8ubmV0PiB3cm90ZToNCj4gT24gNC8yMC8yMiAwNzo1NCwgUmFpbmVyIEhlcmJzdCB3cm90 ZToNCj4+IERlYXIgYWxsLA0KPj4gDQo+PiBteSBmaXJzdCBwb3N0IHdhcyB0byBvcHRpbWlzdGlj IC0gdGhlIHVzZXIgdGVzdGVkIHNvbWUgb3RoZXIgDQo+PmNvbWJpbmF0aW9ucyBhbmQgd2hlbiBo ZSB0cmllcyB0byBzZW5kIGEgc2lnbmVkIEhUTUwgbWFpbCwgdGhlIHNlcnZlciANCj4+YWdhaW4g cmVmdXNlcyB0byBhY2NlcHQgaXQgd2l0aCAiRGlhZ25vc3RpYy1Db2RlOiBzbXRwOyA1MDEgTWVz c2FnZSANCj4+aGFzIGRlZmVjdHMiLg0KPj4gDQo+PiBUaGUgdXNlciBzdWdnZXN0ZWQgdGhhdCB0 aGUgcHJvYmxlbSBpcyBjYXVzZWQgYnkgYSBsaW5lIGNvbnRhaW5pbmcgDQo+Pm9ubHkgIi0tICIg c29tZXdoZXJlIGluIHRoZSB0ZXh0LCBidXQgSSBjYW4gbm90IGNvbmZpcm0gdGhpcy4NCj4+IA0K Pj4gUGxlYXNlIGZpbmQgZW5jbG9zZWQgdGhlIHJlZnVzZWQgbWFpbCAoYW5vbnltaXplZCkuDQo+ IA0KPiBUaGF0IG1lc3NhZ2UgaGFzIG5vIGRlZmVjdHMuDQo+IGBgYA0KPiA+Pj4gaW1wb3J0IGVt YWlsDQo+ID4+PiBmcm9tIG1haWxtYW4uZW1haWwubWVzc2FnZSBpbXBvcnQgTWVzc2FnZQ0KPiA+ Pj4gdHh0ID0gYiIiIiAodGhlIHRlc3RtYWlsLWFiZ2VsZWhudC50eHQgY29udGVudHMpICIiIg0K PiA+Pj4gbXNnID0gZW1haWwubWVzc2FnZV9mcm9tX2J5dGVzKHR4dCwgTWVzc2FnZSkNCj4gPj4+ IG1zZy5kZWZlY3RzDQo+IFtdDQo+ID4+Pg0KPiBgYGANCj4gDQoNCk1heWJlIEkgaGF2ZSBkZWxl dGVkIHRvIG11Y2ggd2hlbiBhbm9ueW1pemluZyB0aGUgbWVzc2FnZS4uLg0KDQpJIHRlc3RlZCB0 aGUgd2hvbGUgbWVzc2FnZSBhbmQgaXQgZ2F2ZSBtZToNCj4+PiBtc2cuZGVmZWN0cw0KW0ludmFs aWRNdWx0aXBhcnRDb250ZW50VHJhbnNmZXJFbmNvZGluZ0RlZmVjdCgpXQ0KDQpUaGFuIEkgcmVk dWNlZCB0aGUgbWVzc2FnZSBjb250ZW50LiBUaGUgbWluaW1hbCBtZXNzYWdlIHRoYXQgcHJvdm9r ZWQgdGhlIA0KZGVmZWN0IGlzIGF0dGFjaGVkLg0KDQpXaGVuIEkgZGVsZXRlIGxpbmUgMTIgKCJD b250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBxdW90ZWQtcHJpbnRhYmxlIikgZnJvbSANCnRoZSBt ZXNzYWdlLCB0aGUgZGVmZWN0IGRpc2FwZWFycy4NCg0KDQo+IC0tIA0KPiBNYXJrIFNhcGlybyA8 bWFya0Btc2FwaXJvLm5ldD4gICAgICAgIFRoZSBoaWdod2F5IGlzIGZvciBnYW1ibGVycywNCj4g U2FuIEZyYW5jaXNjbyBCYXkgQXJlYSwgQ2FsaWZvcm5pYSAgICBiZXR0ZXIgdXNlIHlvdXIgc2Vu c2UgLSBCLiANCj5EeWxhbg0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fXw0KPiBNYWlsbWFuLXVzZXJzIG1haWxpbmcgbGlzdCAtLSBtYWlsbWFuLXVzZXJz QG1haWxtYW4zLm9yZw0KPiBUbyB1bnN1YnNjcmliZSBzZW5kIGFuIGVtYWlsIHRvIG1haWxtYW4t dXNlcnMtbGVhdmVAbWFpbG1hbjMub3JnDQo+IGh0dHBzOi8vbGlzdHMubWFpbG1hbjMub3JnL21h aWxtYW4zL2xpc3RzL21haWxtYW4tdXNlcnMubWFpbG1hbjMub3JnLw0KPiANCg0KVmllbGUgR3LD vMOfZQ0KDQpSYWluZXIgSGVyYnN0DQpMZWl0ZXIgSVQtU2VydmljZQ0KUGhvbmU6ICs0OSAzMzEg NzQ5OS0yNTcNCmUtbWFpbDogcmFpbmVyLmhlcmJzdEBhaXAuZGUNCmh0dHBzOi8vd3d3LmFpcC5k ZQ0KDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KTGVpYm5pei1JbnN0 aXR1dCBmw7xyIEFzdHJvcGh5c2lrIFBvdHNkYW0gKEFJUCkNCkFuIGRlciBTdGVybndhcnRlIDE2 LCAxNDQ4MiBQb3RzZGFtDQoNClZvcnN0YW5kOiBQcm9mLiBEci4gTWF0dGhpYXMgU3RlaW5tZXR6 LCBXb2xmcmFtIFJvc2VuYmFjaA0KU3RpZnR1bmcgYsO8cmdlcmxpY2hlbiBSZWNodHMNClN0aWZ0 dW5nc3ZlcnplaWNobmlzIEJyYW5kZW5idXJnOiAyNiA3NDItMDAvNzAyNg0KLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KDQo=
--_===3477555====cgpro.aip.de===_ Content-Disposition: attachment; filename="mail-with-defect.txt" Content-Type: text/plain; name="mail-with-defect.txt" Content-Transfer-Encoding: base64
UmV0dXJuLVBhdGg6IDxzb21ldXNlckBhaXAuZGU+CkNvbnRlbnQtVHlwZTogbXVsdGlwYXJ0 L3NpZ25lZDsKCWJvdW5kYXJ5PSJBcHBsZS1NYWlsPV80NkExNzA4RC0xNDU3LTQ3NUQtOThF QS1FMjBCQzk3QUY5NjIiOwoJcHJvdG9jb2w9ImFwcGxpY2F0aW9uL3BrY3M3LXNpZ25hdHVy ZSI7CgltaWNhbGc9c2hhLTI1NgpNaW1lLVZlcnNpb246IDEuMCAoTWFjIE9TIFggTWFpbCAx Ni4wIFwoMzY5Ni44MC44Mi4xLjFcKSkKU3ViamVjdDogUmU6IFVuZGVsaXZlcmVkIE1haWwg UmV0dXJuZWQgdG8gU2VuZGVyCkZyb206IDxzb21ldXNlckBhaXAuZGU+CkluLVJlcGx5LVRv OiA8MkZDNjFFMzctMDNGNy00OERGLTgzMzgtMEIyQUI1NDM0RDkwQGFpcC5kZT4KRGF0ZTog V2VkLCAyMCBBcHIgMjAyMiAxMjoyMDoxNyArMDIwMApDYzogdGVzdGwyQGFpcC5kZQpDb250 ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBxdW90ZWQtcHJpbnRhYmxlCk1lc3NhZ2UtSWQ6IDw3 Qjk1QkQyNC1BMTI4LTQxQUQtODU2MS1DNDA1MTZFREE4ODlAYWlwLmRlPgpSZWZlcmVuY2Vz OiA8MjAyMjA0MjAxMDE3MTUuQjkwMEE2RTM4OEBtYWlsbWFuLmFpcC5kZT4KIDwyRkM2MUUz Ny0wM0Y3LTQ4REYtODMzOC0wQjJBQjU0MzREOTBAYWlwLmRlPgpUbzogPHNvbWVvdGhlcnVz ZXJAYWlwLmRlPgpYLU1haWxlcjogQXBwbGUgTWFpbCAoMi4zNjk2LjgwLjgyLjEuMSkKWC1T cGFtaWNpdHk6IEhhbSwgdGVzdHM9Ym9nb2ZpbHRlciwgc3BhbWljaXR5PTAuMDAwMDAwLCB2 ZXJzaW9uPTEuMi40CkxpbmVzOiAzMDcKCgotLUFwcGxlLU1haWw9XzQ2QTE3MDhELTE0NTct NDc1RC05OEVBLUUyMEJDOTdBRjk2MgpDb250ZW50LVRyYW5zZmVyLUVuY29kaW5nOiBxdW90 ZWQtcHJpbnRhYmxlCkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsKCWNoYXJzZXQ9dXRmLTgK CkVzIHNlaSBkZW5uLCBpY2ggbWFjaGUga2VpbmUgdGV4dC1zaWduYXR1ciBkcmF1ZiAobWl0 ICItLSAiIGdldHJlbm50KSwgPQpkYW5uIGZ1bmt0aW9uaWVydCBlcyBhdWNoIG1pdCBkaWdp dGFsZSBzaWduYXR1ci4KPj49MjAKPj49MjAKPj49MjAKPj0yMAo+PTIwCgoKLS1BcHBsZS1N YWlsPV80NkExNzA4RC0xNDU3LTQ3NUQtOThFQS1FMjBCQzk3QUY5NjIKQ29udGVudC1EaXNw b3NpdGlvbjogYXR0YWNobWVudDsKCWZpbGVuYW1lPXNtaW1lLnA3cwpDb250ZW50LVR5cGU6 IGFwcGxpY2F0aW9uL3BrY3M3LXNpZ25hdHVyZTsKCW5hbWU9c21pbWUucDdzCkNvbnRlbnQt VHJhbnNmZXItRW5jb2Rpbmc6IGJhc2U2NAoKTUlBR0NTcUdTSWIzRFFFSEFxQ0FNSUFDQVFF eER6QU5CZ2xnaGtnQlpRTUVBZ0VGQURDQUJna3Foa2lHOXcwQkJ3RUFBS0NDRU1RdwovM1Jr WDVLcXlaQ21vYmtQeWYxTHV0aW9MdHNDRERQSDV5aG10RURUWktpMlBsb25XK0ppUE9jemJ5 dHVNZlQ1TUNvVDZ2S3EwYURGCkR2Z0dLdmM5K1d2L2FwZVpIdUp6Zk5Odm5oSVd5aEQ3YjVi MTh3ZXVNSlhQQUFBQUFBQUEKLS1BcHBsZS1NYWlsPV80NkExNzA4RC0xNDU3LTQ3NUQtOThF QS1FMjBCQzk3QUY5NjItLQo=
--_===3477555====cgpro.aip.de===_--
--_===signed==3477555====cgpro.aip.de===_ Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit
Mailman's content filtering has removed the following MIME parts from this message.
Content-Type: application/x-pkcs7-signature Name: smime.p7s
--_===signed==3477555====cgpro.aip.de===_--
--===============7924952636743351526== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline
Mailman-users mailing list --mailman-users@mailman3.org To unsubscribe send an email tomailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
--===============7924952636743351526==--
On 21-Apr-22 07:53, tlhackque via Mailman-users wrote:
[Snip]
FWIW, I just opened a Thunderbird bug on its behavior.
https://bugzilla.mozilla.org/show_bug.cgi?id=1765762
Anyone with AppleMail-specific information might want to add it to the comments....
On 4/21/22 04:53, tlhackque via Mailman-users wrote:
Note that MailMan is encapsulating a multipart/signed message in a multipart/mixed in order to append it's footer, and reports that it removed the signature part of the multipart/signed. This makes the signed message format invalid. So at least, MM3 needs to handle signed messages better: either include the signature or remove the signature headers (an unwrapping exercise to convert the message to unsigned)... Note that this can be recursive where a signed message contains a message/rfc822 part that contains a signed message, that....
This was an issue with Mailman's content filtering for the mailmanusers@mailman3.org list. Pass types was:
multipart message/rfc822 text application/pgp-signature
So the application/x-pkcs7-signature part was removed. I have just updated Pass types to:
multipart message/rfc822 text application/pgp-signature application/x-pkcs7-signature
to not remove application/x-pkcs7-signature parts going forward.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 21-Apr-22 13:09, Mark Sapiro wrote:
On 4/21/22 04:53, tlhackque via Mailman-users wrote:
Note that MailMan is encapsulating a multipart/signed message in a multipart/mixed in order to append it's footer, and reports that it removed the signature part of the multipart/signed. This makes the signed message format invalid. So at least, MM3 needs to handle signed messages better: either include the signature or remove the signature headers (an unwrapping exercise to convert the message to unsigned)... Note that this can be recursive where a signed message contains a message/rfc822 part that contains a signed message, that....
This was an issue with Mailman's content filtering for the mailmanusers@mailman3.org list. Pass types was:
multipart message/rfc822 text application/pgp-signature
So the application/x-pkcs7-signature part was removed. I have just updated Pass types to:
multipart message/rfc822 text application/pgp-signature application/x-pkcs7-signature
to not remove application/x-pkcs7-signature parts going forward.
That's what I expected to be behind the behavior. Thanks for fixing it on this list.
But as I noted, it would be better if the application/*signature parts were automagically passed when a multipart/signed is present. Ideally those that correspond to the multipart/signed, but it's probably less work to allow them anywhere.
I've seen variations of this misconfiguration on a number of lists, all of whose administrators had to learn by experience. Seems like a good case for Mailman becoming smart enough to do the right thing...
tlhackque via Mailman-users writes:
Not that I count, but I'm not particularly sympathetic to this either.
In fact, you do count. If we can get a working consensus that Mailman shouldn't try to make up for MUA SNAFUs, no matter how many trillion dollars the manufacturer's stock is worth, life would be easier for us. ;-)
An up-to-date Thunderbird user - the message with the "mail-with-defect.txt" attachment, as forwarded by [MME-users] (selected headers below for ID) displays as blank. Interestingly, the Thunderbird error console has no error message related to this.
So there's really no point in passing it along, unless "pass these defects" turns into "repair these defects".
That's a Thunderbird problem, though, and nonconforming to the Postel Principle.
Assuming it's an origin issue, this really seems like an issue that can only be fixed by the sending MUA (Apple).
Obviously.
Note that MailMan is encapsulating a multipart/signed message in a multipart/mixed in order to append it's footer, and reports that it removed the signature part of the multipart/signed. This makes the signed message format invalid. So at least, MM3 needs to handle signed messages better: either include the signature or remove the signature headers (an unwrapping exercise to convert the message to unsigned)...
For the former, we already try pretty hard to include the signature. At least on this list "Pass types" is
multipart
message/rfc822
text
application/pgp-signature
application/x-pkcs7-signature
application/pkcs7-signaturealthough I see Mark just added that last one or two. Of course, "x-" tokens are deprecated except for private use, maybe we should be unsympathetic to x-pkcs7-signature, too? :-)
Note that this can be recursive where a signed message contains a message/rfc822 part that contains a signed message, that....
I'm not sure it makes sense to encrypt messages to a mailing list,
Of course it does. Maybe not to a public discussion list, but for a closed group's distribution list, I'm sure there are applications.
but the format issue is the same.
OK.
In either case, corrupting messages is a MM issue.
I disagree, in part. Although you do address this, let me use the example of this case where the signed body is multipart/alternative. I think it's perfectly reasonable for a list to configure to remove the text/html part, which will break the signature. Then there are three possible ways to go: (1a) leave the signature in place, broken, thus informing recipients it was supposed to be signed and leaving it to them to follow up, (1b) to change the multipart/signed content-type to multipart/mixed and replace the signature with a "removed" notice, and (2) to remove the signature and multipart structure, leaving recipients unaware that the message was intended to be signed. Each of those is problematic, each in its own way. Supporting all of them and the list configuration infrastructure to allow admins to choose would be an unwarranted burden on us, IMO.
multipart/signed should imply accepting the signature part (in this case, application/x-pkcs7-signature). If signatures aren't allowed by policy, then multipart/signed should be bounced.
I find it hard to believe that anyone would want a "deny signed bodies" policy, but it's easily implemented. Put multipart/signed in "Filter types". A configuration explicitly banning signature algorithms but not multipart/signed is a user error in that case. I guess I can imagine wanting to ban certain types (eg, known exploitable algorithms), and if one has such a use case and wants the message rejected rather than simply stripping the signature as unusable, "patches welcome" -- I can't see that level of care as worth our time, though.
It's also the case that if a signed message contains an banned part type, simply removing it will break the signature. So in that case, the choice would either be to bounce the message, convert the message to unsigned, or accept the banned part. (The latter might be acceptable in some environments, but has obvious risks.)
Clearly, the current behavior is problematic.
That's not obvious, except as it's cosmetic.
(I don't think it's reasonable to expect MM admins to "know" to configure their lists to pass signature parts if accepting signed - after all, this list is run by experts :-)
I agree with that. It's not clear that we should do something about it, though. There are two possibilities: (1) the list admin doesn't care about signatures, and simply wanted to allow multimedia mail, and (2) the list admin wants to admit signatures on purpose, for some purpose.
In case (1), this is mostly a cosmetic problem, and possibly involves minor annoyance. The signature validation will fail and maybe there's a text notice about removed parts, that's cosmetic. A subscriber who cares about the signature might have to contact the author for validation, that's the annoyance. In case (2), we do not know what the requirements are. We should refuse to guess and we can assume that list admin knows [more about what] she's doing [than we possibly can] and leave it up to her.
If you think that we can know a lot about a very common use case in principle, I'd like to hear about it, but for now that's my stance,
Steve
On 22-Apr-22 02:12, Stephen J. Turnbull wrote:
tlhackque via Mailman-users writes:
An up-to-date Thunderbird user - the message with the "mail-with-defect.txt" attachment, as forwarded by [MME-users] (selected headers below for ID) displays as blank. Interestingly, the Thunderbird error console has no error message related to this.
So there's really no point in passing it along, unless "pass these defects" turns into "repair these defects".
That's a Thunderbird problem, though, and nonconforming to the Postel Principle.
As I subsequently wrote, I agree and logged a bug report for TB. Apparently the other people who reported experiencing the issue here never did. However, it's not clear whether it's the same "blank display" issue - I'm pretty sure mine was due to the broken signature; others seemed to indicate the CTE headers. So I hope those with the ability to sort that out will add to the bug report...
The Postel Principle is a useful, sweeping generalization. It's not an absolute. It doesn't say "be so liberal in what you accept that you do something sensible with all possible input faults, including those that you never thought of and any input contrary to a specification". One could argue that TB was liberal enough not to crash. I don't expect any MUA to display non-conforming messages as the sender intended. It's unquestionably a bug that TB failed to display anything, even "I don't understand this malformed message". If it did that, further conformance to the Postel Principle would be a matter of degree. It could display the raw message body, try to intuit and format the most involved content, fall back to any text/plain part, offer to send a bug report to vendor of the MUA identified in the headers, or...
However, the point I tried to make with this example is that passing-on defects isn't a good solution. At best, receivers whose MUA tolerates a defect (or interprets it in a useful way) benefit. But to others, MM looks like a flaky source - some people can read some messages some of the time. And it amplifies the impact of the defect by distributing it to a large(r) audience. (Similar to the DDOS DNS amplification attacks.) So if there's an issue identified that causes pain for the MM community, I think it best to either (a) bounce the message with an explanation or (b) if possible (and in a Postelish spirit) repair the defect.
Note that MailMan is encapsulating a multipart/signed message in a multipart/mixed in order to append it's footer, and reports that it removed the signature part of the multipart/signed. This makes the signed message format invalid. So at least, MM3 needs to handle signed messages better: either include the signature or remove the signature headers (an unwrapping exercise to convert the message to unsigned)...
For the former, we already try pretty hard to include the signature. At least on this list "Pass types" is
multipart message/rfc822 text application/pgp-signature application/x-pkcs7-signature application/pkcs7-signaturealthough I see Mark just added that last one or two. Of course, "x-" tokens are deprecated except for private use, maybe we should be unsympathetic to x-pkcs7-signature, too? :-)
The last two were added in response to my analysis. There are a lot of x-mime types that were subsequently standardized. There's no good solution for migrating from an experiment to a standard. Your friend the Postel Principle gets in the way - since it's clear that the semantics are identical, you should generate the standard, but accept both the experimental and standard... :-)
Converting to unsigned, while possible, is a bad idea in the vast majority of cases. More later.
In either case, corrupting messages is a MM issue.
I disagree, in part. Although you do address this, let me use the example of this case where the signed body is multipart/alternative. I think it's perfectly reasonable for a list to configure to remove the text/html part, which will break the signature. Then there are three possible ways to go: (1a) leave the signature in place, broken, thus informing recipients it was supposed to be signed and leaving it to them to follow up, (1b) to change the multipart/signed content-type to multipart/mixed and replace the signature with a "removed" notice, and (2) to remove the signature and multipart structure, leaving recipients unaware that the message was intended to be signed. Each of those is problematic, each in its own way. Supporting all of them and the list configuration infrastructure to allow admins to choose would be an unwarranted burden on us, IMO.
The fourth, and probably correct, possible action is to bounce the message. Someone signed it for a reason. If its content violates the list policy, it can't be fixed and stay signed. So tell the sender why it can't be accepted, and give him the option to remove the banned part(s) or remove the signature. If someone signs a message, substituting the judgement of some random remote list manager (human or automaton) doesn't seem like the right thing to do...
multipart/signed should imply accepting the signature part (in this case, application/x-pkcs7-signature). If signatures aren't allowed by policy, then multipart/signed should be bounced.
I find it hard to believe that anyone would want a "deny signed bodies" policy
So would I. But I sign messages by default, and run into this. And the argument goes "the signature doesn't add anything. It occupies space (itself and it can cause e.g. base64 encoding of plain text) and my users don't care and it's a techie's vanity exercise and it causes problems (like the ones discussed here), and..."
, but it's easily implemented. Put multipart/signed in "Filter types". A configuration explicitly banning signature algorithms but not multipart/signed is a user error in that case.
Yes. And (I won't mention the PP here, but...) it's clear that even experienced administrators (e.g. Mark) fall into this trap. In fact, I think accept "multipart" (meaning multipart/*) is a default or at least recommended configuration. So MM should either detect it, or behave reasonably. I'm suggesting that a reasonable behavior would be:
If an incoming message is signed and the associated signature part is not explicitly banned, accept the signature part. If signed and the signature part is banned, or if signed and signed mail is banned: bounce the message.
I guess I can imagine wanting to ban certain types (eg, known exploitable algorithms), and if one has such a use case and wants the message rejected rather than simply stripping the signature as unusable, "patches welcome" -- I can't see that level of care as worth our time, though.
I'm not going as far as selective algorithm enforcement. That would only make sense if MM got smart enough to validate signatures of signed mail - e.g. to bounce mail with bad signatures. (Which might be a worthy goal, but out of scope of this exchange...)
But I stand by "if the sender signed it, then accept it or bounce it." Messing with it only leads to where there be dragons.
(I don't think it's reasonable to expect MM admins to "know" to configure their lists to pass signature parts if accepting signed - after all, this list is run by experts :-)
I agree with that. It's not clear that we should do something about it, though. There are two possibilities: (1) the list admin doesn't care about signatures, and simply wanted to allow multimedia mail, and (2) the list admin wants to admit signatures on purpose, for some purpose.
In case (1), this is mostly a cosmetic problem, and possibly involves minor annoyance. The signature validation will fail and maybe there's a text notice about removed parts, that's cosmetic. A subscriber who cares about the signature might have to contact the author for validation, that's the annoyance. In case (2), we do not know what the requirements are. We should refuse to guess and we can assume that list admin knows [more about what] she's doing [than we possibly can] and leave it up to her.
I'm taking the position that the originator's intent should control.
If (s)he doesn't sign a message, then MM/the admin is free to change the content per (hopefully published) list policy.
If (s)he did sign a message, the MM must either deliver it with the signature intact, or bounce it for policy reasons.
If you think that we can know a lot about a very common use case in principle, I'd like to hear about it, but for now that's my stance,
Steve
participants (6)
-
David Newman -
Mark Sapiro -
Rainer Herbst -
Skip Montanaro -
Stephen J. Turnbull -
tlhackque