- Mailman-users - lists.mailman3.org

Review requested for some merge requests
by Stephen J. Turnbull Jan. 10, 2024

Jan. 10, 2024

Reply-To set to Mailman Developers. Please check that your replies go there. Or you can comment on MRs directly if you have a GitLab login. I have three merge requests pending: [1] https://gitlab.com/mailman/mailman/-/merge_requests/1159 [2] https://gitlab.com/mailman/mailman/-/merge_requests/1160 [3] https://gitlab.com/mailman/mailman/-/merge_requests/1161 [1] fixes the logging initialization bug that prevented the `smtp` log from rotating properly, and maybe some others. I don't think it needs user review because it's a pure bugfix, but if you're curious and take a look, and you think it needs better documentation let me know. [2] adds a configuration variable `lmtp_max_size`. This is user-visible, so review of the documentation in docs/mta.rst and config/mailman.cfg for clarity and readability would be appreciated. [3] adds postfix-pgsql support for use of the PostgreSQL database to route Mailman traffic. It's really most useful in an edge case (migration of 10,000+ list Mailman 2 instances), but it's theoretically attractive on the principle of "single source of truth", and possibly for implementing a "microservices" style cloud instance (it eliminates the need for Mailman and Postfix to share a file system). Besides opinions on the value of the general idea I have two specific questions: - Postfix docs recommend not looking up domains in the database to reduce load on the database server. But on a dedicated Mailman system, you're going to be looking up pretty much everything twice anyway (once to check for local recipient and once to fetch the transport), and my experience so far is that this isn't a big load on Postgres, and it's not a drag on list traffic distribution. Should I implement domain lookup in Postgres? Should it be optional if the lists are being looked up in Postgres? - The current patch completely disables generation of the postfix_lmtp alias file, even for the `mailman aliases` command. Should I implement an option to force generation of the files for documentation (and maybe a backup in a very rare and contorted situation where Mailman can see Postgres but Postfix can't)? Thanks, Steve

3 3

Mailman 3 Core user assistance
by Tony Deshields Jan. 10, 2024

Jan. 10, 2024

I currently have a docker mailman stack that utilizes Django LDAP backend for authentication. I have my users attributes mapped as you see below in my settings.py file. Whenever a user logs into postorius with their 'uid', mailman-core uses their 'uid' as their email address instead of the 'email' attribute. Is there anyway to change this functionality? I would like my users to be able to login with their enterprise uid but be mapped to their email but I feel like there is a disconnect between Django and MM3 core. Any assistance is much appreciated. AUTH_LDAP_USER_ATTR_MAP = { "uid" : "userPrincipalName", "first_name": "givenName", "last_name": "sn", "email" : "mail", }

2 1

Docker mailman image can't access https://publicsuffix.org/
by bob B Jan. 9, 2024

Jan. 9, 2024

Thanks so it looks like Mailman can not get to publicsuffix. The wrinkle is I am running maxking's docker version. The host can get there, but not from the docker image. Any idea to to configure that? and this only comes into play when I set the dmarc settings.

3 8

The 'flufl.lock>=5.1' distribution was not found, but flufl.lock 8.0.2 was installed
by Nils Jan. 8, 2024

Jan. 8, 2024

3 2

The doubled entries issue
by Lars Bjørndal Jan. 7, 2024

Jan. 7, 2024

I've got the problem Multiple rows were found when one or none was required. I found the issue and a script here: https://gitlab.com/mailman/mailman/-/issues/657 I did su - mailman3 and ran: python scriptname.py and got: Traceback (most recent call last): File "/opt/mailman3/fix-doubled-entries.py", line 21, in <module> listmgr = get_list_manager() ^^^^^^^^^^^^^^^^^^ File "/opt/mailman3/fix-doubled-entries.py", line 6, in get_list_manager return getUtility(IListManager) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman3/venv/lib64/python3.11/site-packages/zope/component/_api.py ", line 180, in getUtility raise ComponentLookupError(interface, name) zope.interface.interfaces.ComponentLookupError: (<InterfaceClass mailman.interfa ces.listmanager.IListManager>, '') (venv) 1[mailman3@dalen ~]$ Do I miss something? How can I fix the problem? Thanks, Lars

4 10

Archive import fails: "IndexError: list index out of range"
by Johannes Rohr Jan. 6, 2024

Jan. 6, 2024

I am now importing the archives of my old mailman2 installation. Unfortunately, with some of the larger and older lists, import fails halfway through, with an exception, the last message of which is IndexError: list index out of range Below is an example. This means that the last few years are missing from the archive in several lists. Are there any suggestions what I could do to still get them importet? Thanks! Johannes File "/usr/lib/python3.8/email/_header_value_parser.py", line 2069, in get_msg_id token, value = get_dot_atom_text(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1334, in get_dot_atom_text raise errors.HeaderParseError("expected atom at a start of " email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found '@localhost>' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/opt/mailman/venv/lib/python3.8/site-packages/mailman_web/manage.py", line 37, in main execute_from_command_line(sys.argv) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/opt/mailman/venv/lib/python3.8/site-packages/hyperkitty/management/commands/hyperkitty_import.py", line 387, in handle importer.from_mbox(mbfile, report_name) File "/opt/mailman/venv/lib/python3.8/site-packages/hyperkitty/management/commands/hyperkitty_import.py", line 212, in from_mbox progress_marker.tick(unquote(message.get("message-id", 'n/a'))) File "/usr/lib/python3.8/email/message.py", line 471, in get return self.policy.header_fetch_parse(k, v) File "/usr/lib/python3.8/email/policy.py", line 163, in header_fetch_parse return self.header_factory(name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 607, in __call__ return self[name](name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 202, in __new__ cls.parse(value, kwds) File "/usr/lib/python3.8/email/headerregistry.py", line 535, in parse kwds['parse_tree'] = parse_tree = cls.value_parser(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2126, in parse_message_id token, value = get_msg_id(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2073, in get_msg_id token, value = get_obs_local_part(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1516, in get_obs_local_part if (obs_local_part[0].token_type == 'dot' or IndexError: list index out of range

6 10

Not receiving email from outlook or gmail.
by Mark Labeste Jan. 5, 2024

Jan. 5, 2024

From mailman web it was able to deliver the email to outlook and gmail but when replying or confirming subcription it can't reach the mailman. This error is bounce back from outlook. Server at destination.email.com (192.10.10.10) returned '400 4.4.7 Message delayed' 1/4/2024 12:16:13 PM - Server at destination.email.com (192.10.10.10) returned '451 4.4.397 Error communicating with target host. -> 421 4.2.1 Unable to connect -> SocketTimedout: Socket error code 10060 Thanks, mARK

2 1

Programmatically modify Thread-Topic *before* message is sent out the door?
by Thomas Ward Jan. 3, 2024

Jan. 3, 2024

We use a custom plugin to do numeric thread stamping, which has been working well in the code system. However, it was discovered by Microsoft that Microsoft Outlook and OWA prioritize the Thread-Topic header over the Subject header, which results in the oddness we've experienced in MS365 clients in testing. During our tasks though a custom handler goes through during the pipeline and modifies the Subject field accordingly, adding a thread tag to it. While this is 'fragile' it does work; the only problem is that Thread-Topic doesn't get altered. Programmatically, how would we alter the Thread-Topic during the handler's processing to prepend the corresponding prefix/tag so that when the message actually goes out the door it is properly sending with the modified Thread-Topic? Thomas

2 6

REST API for list owners/moderators
by mathias＠koerber.org Jan. 3, 2024

Jan. 3, 2024

Does Mailman 3 have an REST API that allows list-owners/moderators to perform list-management functions without having access to the whole server/other lists etc? ie, an API that uses the same credentials as Postorious but allows mass-subscription/un-subscription/membership-export etc and changes to list settings, membership settings (mod etc) via REST or python? The documentation seems to only indicate a single one-admin-user REST API that should not even be accessible from outside?

2 1

Message getting shunted with enabling Dmarc replace with list address
by bob B Jan. 3, 2024

Jan. 3, 2024

So we host a mailing list for domain X, the MX record points to us, we are domain y. If we have Dmarc set to "no dmarc mitigations" I can send a message from my email in domain Y and it goes to the list in domain x and is processed and sent out. If I set the dmarc to "replace with list address". the same message gets shunted. I included the logs below Not sure why it is trying to contact publicsuffix.org? ************* /mailman/core/var/logs/mailman.log Jan 02 20:41:50 2024 (29) No cached copy of the public suffix list found Jan 02 20:41:55 2024 (29) Uncaught runner exception: HTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/public_suffix_list.dat (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)')) Jan 02 20:41:55 2024 (29) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( File "/usr/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection raise err File "/usr/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection sock.connect(sa) TimeoutError: timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn conn.connect() File "/usr/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect self.sock = conn = self._new_conn() File "/usr/lib/python3.10/site-packages/urllib3/connection.py", line 179, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)') During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 489, in send resp = conn.urlopen( File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen retries = retries.increment( File "/usr/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/public_suffix_list.dat (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/incoming.py", line 80, in _dispose process(mlist, msg, msgdata, start_chain) File "/usr/lib/python3.10/site-packages/mailman/core/chains.py", line 57, in process if link.rule.check(mlist, msg, msgdata): File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 316, in check if maybe_mitigate(mlist, address): File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 292, in maybe_mitigate org_dom = get_organizational_domain(from_domain) File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 142, in get_organizational_domain parse_suffix_list() File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 109, in parse_suffix_list filename = ensure_current_suffix_list() File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 77, in ensure_current_suffix_list content = get(config.dmarc.org_domain_data_url) File "/usr/lib/python3.10/site-packages/mailman/utilities/protocols.py", line 39, in get response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws) File "/usr/lib/python3.10/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/usr/lib/python3.10/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 553, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/public_suffix_list.dat (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)')) Jan 02 20:41:55 2024 (29) SHUNTING: 1704228115.0417743+619bfacfe7e6d030991468aa2707d065418c56e5

2 1