disable body transcoding
Hi,
I have disabled all message modifications and filtering from within the webUI, and created empty header and footer template files. Still, some list messages get their body encoding changed to base64, breaking DKIM. What can I do to prevent this? (I'm a newbie to mailman3, so please give somewhat complete commands. I have read other threads with similar questions, but did not understand some shortened configuration commands.)
BR
On 9/22/21 9:02 AM, Markus Schaaf wrote:
Hi,
I have disabled all message modifications and filtering from within the webUI, and created empty header and footer template files. Still, some list messages get their body encoding changed to base64, breaking DKIM. What can I do to prevent this? (I'm a newbie to mailman3, so please give somewhat complete commands. I have read other threads with similar questions, but did not understand some shortened configuration commands.)
This is probably due to the message body charset being utf-8. The Python email library used by Mailman encodes utf-8 message bodies as base64. There are no configuration settings in Mailman to avoid this. Calling email.charset.add_charset('utf-8') at some point in Mailman's processing might avoid this. E.g. ``` --- a/src/mailman/handlers/validate_authenticity.py +++ b/src/mailman/handlers/validate_authenticity.py @@ -23,6 +23,7 @@ import logging from authheaders import authenticate_message from authres import AuthenticationResultsHeader, SyntaxError from dns.resolver import Timeout +from email.charset import add_charset from itertools import chain from mailman.config import config from mailman.core.i18n import _ @@ -121,5 +122,7 @@ class ValidateAuthenticity: def process(self, mlist, msg, msgdata): """See `IHandler`.""" + # Try to avoid base64 encoding of utf-8. + add_charset('utf-8') if config.arc_enabled: authenticate(msg, msgdata) ``` Note this is just a guess and could be a disaster so test it before trying it in production. -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Am 22.09.21 um 19:26 schrieb Mark Sapiro:
This is probably due to the message body charset being utf-8. The Python email library used by Mailman encodes utf-8 message bodies as base64.
Thank you very much for that helpful clue.
Note this is just a guess and could be a disaster so test it before trying it in production.
It doesn't look dangerous ... :-)
But ok, maybe it's a good idea to have a playground anyway. I'll report back after setting up a test system and trying your patch.
BR
Am 22.09.21 um 19:54 schrieb Markus Schaaf:
But ok, maybe it's a good idea to have a playground anyway. I'll report back after setting up a test system and trying your patch.
Nothing broke, but it didn't work either. I'm a bit surprised that mailman3 was designed without the idea to preserve the body, after signatures have been a thing for at least a decade.
BR
Markus Schaaf writes:
Nothing broke, but it didn't work either. I'm a bit surprised that mailman3 was designed without the idea to preserve the body, after signatures have been a thing for at least a decade.
Patching (including monkey-patching) the Python standard library email package has caused headaches in the past. As you discovered, it's not the case that a one-line patch will do the job.
Regards, Steve
Am 23.09.21 um 10:08 schrieb Stephen J. Turnbull:
Markus Schaaf writes:
Nothing broke, but it didn't work either. I'm a bit surprised that mailman3 was designed without the idea to preserve the body, after signatures have been a thing for at least a decade.
Patching (including monkey-patching) the Python standard library email package has caused headaches in the past. As you discovered, it's not the case that a one-line patch will do the job.
It's not about patching. It's understood since the early days of MIME, that you cannot recreate an encoded representation using the plain text alone. If one had planned for a feature to preserve the body, one had to keep the encoded form (or meta data to recreate it) around and inject it into the smtp sender, if that option were enabled.
BR
Markus Schaaf writes:
It's not about patching.
Call it what you like, but here's the story. We use somebody else's software (the Python standard library 'email' package) for operations at any level below "whole message", and it seems 'email' doesn't do what you want it to do. That implies we need code that takes over functions that are already in the package.
If you want to do it, a merge request will be welcome. But it still involves changing code maintained by somebody else, and it will still require us to maintain it for some time (years), it will still likely will cause problems for users when some component is upgraded, and somebody will have to shepherd it through the Python contribution process to get things back in sync (which is what will take years, because it will not be accepted for any current versions of Python, so we will have to support it until we stop supporting Python 3.10).
If someone else doesn't do it, it goes in the priority queue. I don't know how high the priority will be.
Steve
participants (3)
-
Mark Sapiro -
Markus Schaaf -
Stephen J. Turnbull