On 10/15/21 3:19 AM, Philip Colmer wrote:

I'm not sure that the message ID identifies the author, does it?

No, but it identifies the message, and while Mailman has made some cosmetic modifications to the headers, it is still the same message.

I've just done a quick test of sending myself an email and then replying to it.

The original email has a message ID of CAKTSSThxADLoO=Cy5cNzHoihGb5dYbmB0iw_ThTqe9T9At9efw@mail.gmail.com

In the reply, the following headers are present:

References: <CAKTSSThxADLoO=Cy5cNzHoihGb5dYbmB0iw_ThTqe9T9At9efw@mail.gmail.com> In-Reply-To: <CAKTSSThxADLoO=Cy5cNzHoihGb5dYbmB0iw_ThTqe9T9At9efw@mail.gmail.com> Message-ID: <CAKTSSTh-Fgy63MbDQtWS=oPCw0kGxTmj5RxRe6r1U0W=Die4mQ@mail.gmail.com>

Would it be reasonable for Mailman 3 to use "References:" to keep the original Message ID and then add a new Message ID?

No, because the message from the list is not a reply to the incoming message. It is the incoming message.

I think that what Mailman 3 is doing is similar to forwarding the email. If I forward an email, again the headers change:

If you want Mailman to 'forward' the message, set DMARC mitigation action to wrap the message.

...

The IETF has addressed the issue of non-identical but equivalent messages with the Resent-Message-ID field (and in general the whole suite of Resent-* fields.

Mailman doesn't appear to be adding that header. Would that be an acceptable solution instead of the one I suggested above?

Mailman doesn't add the header, but even if it did, I doubt it would solve the problem with Gmail.

I appreciate your view that Google is broken. I'm just trying to find an acceptable solution to the end-user perception of "I've sent an email to the list but I haven't received a copy back". That is going to be a big issue/headache for us.

It's a long standing PITA with Google. See <https://wiki.list.org/x/4030680>. Your Gmail users can partially mitigate this by setting their Acknowledge posts option to Yes.

Note also that Mailman core communicates with archivers to determine what goes in the Archived-At: header and this value is usually based on the Message-ID:. Changing the Message-ID: could possibly render that value incorrect.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Philip Colmer writes:

I'm not sure that the message ID identifies the author, does it?

It does not identify any person. According to RFC 822 and all successors, it is an "author field", so third parties such as mailing lists should keep their hands off. The agent that causes a message to be sent is the "author", and it's up to them. While I am not the authoritative spokesperson for Mailman, I'm pretty good at channeling the core group, and I'm confident that the Mailman developers do not consider Mailman's role to be "authorship", even though Mailman may do things like delete attachments or HTML alternative parts, convert HTML-only messages to text/plain, and decorate messages with tags in Subject and headers and footers on the body.

There are good reasons for Mailman adopting this posture. In particular, while Google does an execrable job of handling the poster's own duplicates, eliminating duplicates via mailing lists is an important use case for "Message-ID" for everybody except us list admins, who desperately want access to both copies.

I think that what Mailman 3 is doing is similar to forwarding the email. If I forward an email, again the headers change:

There are two kinds of forwards. In one, you simply *resend* the original. (This function is sometimes called "bounce", not to be confused with a refused and returned message.) In that case, the Message-ID should stay the same so that tools can identify the message. In the other, you *attach* the message to a new message. MUAs allow you to add your own text, most folks do (they're citing the message as support for something they want to say), and since they can't read your mind, the MUA or the MTA will treat it as a new message and assign a new Message-ID. On the other hand, Mailman has no intention of adding (or usually subtracting) content any more than the mailman does when they put a postmark on the envelope, so we do not consider Mailman to be an author.

The idea that mailing lists should "take authorship" of posts comes up often, in many different contexts. It's almost always the case the issue that is raised is caused by other human or software agents' misbehavior.

...

The IETF has addressed the issue of non-identical but equivalent messages with the Resent-Message-ID field (and in general the whole suite of Resent-* fields.

Mailman doesn't appear to be adding that header. Would that be an acceptable solution instead of the one I suggested above?

I doubt it solves anything. I'm sure Google uses Message-ID to deduplicate. That's one of the things Message-ID is *for*, Google just does it in a way that creates maximum breakage, including for their own users.

I appreciate your view that Google is broken. I'm just trying to find an acceptable solution to the end-user perception of "I've sent an email to the list but I haven't received a copy back". That is going to be a big issue/headache for us.

I feel for you, but do you think that's a bigger headache than if Mailman is hacked to change the Message-ID and every CC'd subscriber starts receiving duplicates? Do you really want to punish people who use competent MUAs because others use a horrible one? Google is not just broken "in my view," it's logically broken. *Somebody* among your subscribers is going to be annoyed by whatever you do to deal with this.

Note that changing Message-ID and setting all subscribers to "no dupes" is not a 100% solution to this; it means that filtering on List-* headers breaks because Mailman doesn't even send them the message, and the subscribers don't get serial numbers, tags, and footers in posts they're cc'd on.

"It's broken *all the way down*." If I ever get invited on "The New Abnormal", GMail will get my FTG for this.

Steve