January 2024 - Mailman-users - lists.mailman3.org

Find a smtp server to send out emails
by Leon March 12, 2025

March 12, 2025

Hi, This problem is not caused by mailman, but I still want to give it a shot here. I'm hosting a mailing list on top of mailman3. Emails are supposed to be sent out by AWS ses. However, ses requires that sender address must be verified, which leads to a problem that emails sent by subscribers to mailing list cannot be sent to other subcirbers, since their addresses are not verified. And it is impossible to verified every subscriber. Are there smtp service providers allowing unverified email address to send out emails, or do I have other solutions? Thanks. -- Best wishes, Leon

4 3

Moderation of held emails vs. spam
by David Krantz March 11, 2025

March 11, 2025

Hi, I now run Mailman 3.3.1, installed from pip. Is there now any way to prevent Mailman from forwarding the contents of the held message when notifying moderators of held messages? In my case these messages often are spam messages and if the entire contents are forwarded it triggers spam handling in the recipient address(es) which over time risks classifying all moderation emails as spam or even classifying the list domain or server as a spam source. Even my own outgoing filter classifies it as spam, which I will fix. I currently run rspamd for spam filtering, graylisting and DKIM and ARC verification/signing. Someone on this recommended it, thank you for that as it removes the need for other milters. I guess that this would be a simple change to make as a kludge but a proper implementation as a setting would be a lot better. I'd like it as a global setting but that might be too coarse. Cheers // David

3 4

Issues with a DMARC record leading to message being shunted
by Ralf Hildebrandt March 4, 2025

March 4, 2025

Hi! Recently a mail was not being distributed via our MM3 instance; I searched the logs and found: Dec 22 04:54:27 2023 (24) ACCEPT: <1d431542-b386-467b-8bf2-305f19ee7eb4(a)somenet.org> Dec 22 04:54:28 2023 (28) Uncaught runner exception: list index out of range Dec 22 04:54:28 2023 (28) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/lib/python3.10/site-packages/mailman/core/pipelines.py", line 53, in process handler.process(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 125, in process authenticate(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/utilities/retry.py", line 44, in f_retry return f(*args, **kwargs) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 93, in authenticate auth_result = authenticate_message( File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 395, in authenticate_message dmarc_result = check_dmarc(msg, spf_result, dkim_result, dnsfunc=dnsfunc, psddmarc=psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 343, in check_dmarc result, result_comment, from_domain, policy = dmarc_per_from(from_domain, spf_result, dkim_result, dnsfunc, psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 90, in dmarc_per_from record, orgdomain = receiver_record(from_domain) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 117, in receiver_record retval = lookup_receiver_record(hostSansDmarc, dnsfunc) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 92, in lookup_receiver_record tags = answer_to_dict(str(result)) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in answer_to_dict retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in <dictcomp> retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} IndexError: list index out of range Dec 22 04:54:28 2023 (28) SHUNTING: 1703220868.1418982+b27f5db154375fcd2c418d467172e4aad0d3d57a So the message has been shunted, due to this error. Something with DMARC and tags. So I checked the DMARC record for the sending domain somenet.org. It is "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;mailto:postmaster@somenet.org;ri=3600;fo=1;" which is syntactically incorrect. The extra ";mailto:postmaster@somenet.org" is wrong, I guess the tag "ruf=" is missing here! It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;" So, should mm3 somehow "catch" this error somehow? According to the DMARC specs, an syntactically incorrect DMARC record should be ignored / handled as if none were present. -- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin Tel. +49 30 450 570 155 ralf.hildebrandt(a)charite.de https://www.charite.de

5 12

How To Install Mailman 3 on Debian 10 (Complete Guide)
by Brian Carpenter Feb. 25, 2025

Feb. 25, 2025

I have finished my how-to install Mailman 3 on Debian 10 completed to the point where you can use it to get a fully functional Mailman 3 environment up and running. This is a comprehensive guide that walks you through installing EVERYTHING (currently!) that Mailman 3 needs to run. It also makes updating Mailman 3/Postorius/Hyperkitty very easy. I have not added the update directions yet but will very soon. I also plan on adding a section on setting up DKIM and Xapian with Mailman 3. I hope this helps those who wish to get a Mailman 3 server up and running quickly and brings a sense of sanity to the confusion regarding the various documentation out there concerning the installation of Mailman 3/Postorius/Hyperkitty. I intend to add a How-to for Ubuntu 20 at some point. Without further ado: https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 I am sure the above guide can use improvements. Feel free to contact me off-list to point out mistakes/improvements, etc. -- Brian Carpenter Harmonylists.com Emwd.com

13 38

How to remove Held Unsubscriptions
by Mark Jan. 6, 2025

Jan. 6, 2025

Hi As a moderator I receive an email each day for "Held Unsubscriptions" for a subscriber that was unsubscribed by the Owner sometime ago (using "Mass operations"). +++++++++++++++++++++++++++++++ The listname(a)domain.com list has 1 moderation requests waiting. Held Unsubscriptions: User: bart(a)example.com Please attend to this at your earliest convenience. +++++++++++++++++++++++++++++++ There is nothing showing under "Subscription requests > Pending approval". Could anyone tell me how I can stop these email notifications? I have access to the mailman shell and the databases. Thanks, Mark

4 7

Re: mail stuck in "out" queue
by Dmitry Makovey Dec. 18, 2024

Dec. 18, 2024

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

6 17

How to Increase outgoing mail volume for Mailman 3
by Brian Carpenter Dec. 9, 2024

Dec. 9, 2024

Hello everyone, We had a new client come on board that brought a list that had the largest out-going mail volume that I have ever dealt with. So I did what any sane server administrator would do that is running Mailman 3: I ran to Mark Sapiro for help! Here is what the Sage of all things Mailman counseled me to do: 1. Disabled VERP delivery. VERP can help improve mail delivery by delivering 1 message per SMTP transaction (I think I got that right). Using VERP is counter-productive as it can seriously delay outgoing mail delivery for a high-volume list. So you can use something like the following in the MTA section of your mailman.cfg: [mta] verp_confirmations: no verp_personalized_deliveries: no verp_delivery_interval: 0 2. I configured Mailman 3 to run 4 outgoing runners. The default is 1. This action was the game changer. It allows Mailman to handle 4 incoming messages at a time instead of 1. My client's incoming list volume was high enough where it was overwhelming the single outgoing runner. Here is what I added to the mailman.cfg file: [runner.out] instances: 4 Doing this step immediately solved the backlog in var/queue/out. 3. If you start up with a new server for a list that is high volume then keep an eye out for Proofpoint and Vade Security as they will block your new IP address quicker than Mark Sapiro can say "This is a bug. See <some Gitlab.com url>" Mark, thank you! -- Brian Carpenter Harmonylists.com Emwd.com

5 10

Aborted connection message in log in every 5 minutes
by zoltan＠circle-interactive.co.uk Dec. 3, 2024

Dec. 3, 2024

Hi Everyone, we have mailman 3.3.8 venv test installation which in general works fine however our mysql error log is filled up with messages below because they are so frequent. 2023-02-24 0:31:30 199567 [Warning] Aborted connection 199567 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:36:30 201063 [Warning] Aborted connection 201063 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:41:31 202559 [Warning] Aborted connection 202559 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:46:31 204056 [Warning] Aborted connection 204056 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) As suggested here https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… I tried to correlate with error logs and /opt/mailman/web/logs/mailmanweb.log shows some hyperkitty.lib.utils Failed to update the fulltext index: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (111)" /opt/mailman/web/logs/uwsgi-error.log Message: OperationalError(2013, "Lost connection to MySQL server at 'handshake: reading initial communication packet', sy stem error: 104") but they have not such frequency so unsure if they are related. Thanks for the suggestions

6 16

Backup and restore a single list
by sherin＠raidm.com Nov. 6, 2024

Nov. 6, 2024

Hello, I am new to mailman3 . I am looking for the procedure to backup and restore a single mailing list from mailman3. If any one know the procedure or the documentation , please point me

3 7

Logging subscribes/unsubscribes
by Brian Carpenter Oct. 22, 2024

Oct. 22, 2024

Does Mailman 3 log subscribes and unsubscribes for each list the way Mailman 2 did? Brian

5 7

Questions regarding message acceptance
by Pawel Grzywaczewski Sept. 19, 2024

Sept. 19, 2024

Hello, I have few questions regarding usage of Mailman3: 1) We have list1 and list2 with members. Can we specify that all members of list1 can post messages to list2? We could potentially write a script that would synchronize members of another list to “Accept these non-members” of another list, but with thousands of lists it will be difficult to maintain lists in synchro. So perhaps there is a concept of adding another list to “Accept these non-members” and mailman real-time evaluate membership of another list upon message arrival? 2) In explanation of “Accept these non-members” it’s written: “While non-regexp addresses can be entered here, it is preferred to add the address as a nonmember and set the nonmember's Moderation to Default Processing”. I noticed that if someone posts to a list then he is added automatically to non-members. Which means that next time he will be able to post. My understanding is that it’s better to use the feature “Accept these non-members”. Am I correct? Thank you, Paweł

5 7

Disabled by bounce
by Allan Hansen Sept. 11, 2024

Sept. 11, 2024

Hi all, Apparently Google just started to get very aggressive about DKIM settings. Some hundreds of my subscribers have had their subscriptions disabled because of Google's bounces. I tried to 'Enable' some of them, expecting that their bounce scores would be reset, but when I list the users, the bounce scores are still 5. Please advise. Will enabling them without the bounce score being reset just cause the server to disable them again. Yours, Allan

10 19

Display the last post time for a list or all lists.
by Henrik Rasmussen Aug. 9, 2024

Aug. 9, 2024

Does Mailman 3 have something similar to https://www.msapiro.net/scripts/last_post.py displaying the last post time for a list or all lists?

3 4

MM3 generates long lines?
by peter＠chubb.wattle.id.au July 28, 2024

July 28, 2024

I'm seeing emails being dropped when mm3 talks to the local smtp server, with log errors like: 2021-06-23 15:42:24 1lvvea-007r8u-VN H=localhost (<FQDN>) [::1] F=<announce-bounces@<FQDN>> rejected after DATA: maximum allowed line length is 998 octets, got 1036 The email itself did not have such long lines AFAICT. I can disable line length checking on the mailserver, but that sounds like a recipe for the next one along to drop the email. Peter C

4 8

Digests not working correctly
by Joel Lord June 8, 2024

June 8, 2024

So I seem to have problems with both periodic AND size-triggered digests. Size first, this is simpler. Just got one sent out, clearly triggered by a new email getting added to the queued up messages, but the setting for the list is the default 30KB and the digest came out at 23.2 KB. Is it measuring the accumulated queue as size on disk including metadata and filesystem block sizes or something, rather than the size of what the emailed digest will be? Periodic is much more complicated, even to explain. I have the cron job running nightly at 11:00 PM and it certainly seems to be working, except for one really odd thing. The list also definitely has periodic digests enabled in the web UI. This is a very bursty list, so it will go mad with traffic for a few weeks and then will go mostly idle for a month or so. The first digest of a burst is definitely size-triggered, but pretty reliably it has the last few messages from the previous burst in there. So the one I got today had three messages from today and one message from May 4th. But that May 4th message should have gone out at 11:00 PM on May 4th with the daily digest and then gotten flushed from the pending digest queue, right? -- Joel Lord

3 14

Change config to create a new list with all archives off
by Tibor Molnar April 21, 2024

April 21, 2024

Can archiving be changed anywhere in mailman3 configs, so then all new lists created do not archive by default?

5 8

Mailman 3 hosting query
by Bill Robertson April 18, 2024

April 18, 2024

Hi I'm investigating the possibility of transferring a fairly small mailing list I own from Google Groups to MM3 via a hosting service. I'd appreciate any pointers regarding what I should be looking out for with companies that offer this service It also seems very hard to find out which hosting companies are worth considering (In particular, reliability and support are important) Any help much is much appreciated Thanks

4 5

mailman3 digest sending questions and also adding docker to the mix
by bob B April 17, 2024

April 17, 2024

So I have been down the digest rabbit hole. So to confirm my assumptions/findings: Digest size threshold---- means when the total size of held messages hits this setting, a digest is sent Digest Volume Frequency. ---- specifies when the digest volume number gets incremented for digest messages, IT DOES NOT send out digests based on the setting, daily, weekly ... So what does the GUI setting "Send Digest Periodically yes no" do? how periodically does it send digests, if this is set to yes? If I want to send a digest out for a list every day, or every week, how do I set that, especially in the docker environment? Is this something that can be configured in the GUI and configured by each of my list admins? I found that ”'mailman digests --periodic' via cron” should be set in googling, but does that mean I have to do a cron job for any list that wants a scheduled digest sent? My use case is, that I have a list admin who set a size threshold, but regardless of that, they also want their list digest sent out daily, they thought it was the "Digest Volume Frequency" setting but that is not correct. However in their testing they are not getting daily digest, they are getting digests only when the threshold is reached, BUT they have "Send Digest Periodically" set to "yes" and "Enable Digests" is set to "yes"

7 17

hyperkitty.lib.utils Failed to update the fulltext index
by Ben Harris April 13, 2024

April 13, 2024

hello there, i migrated to a new host last night and i've been getting this error from a cronjob mail every minute. anyone have any ideas what would cause this? ERROR 2019-04-17 14:26:33,056 27205 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/hyperkitty/search_indexes.py", line 87, in update_index update_cmd.update_backend("hyperkitty", "default") File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 276, in update_backend last_max_pk=max_pk) File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/lib/python2.7/dist-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/lib/python2.7/dist-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1)) thanks, ben

5 8

Error -- postorius Un-handled exception
by Mark April 12, 2024

April 12, 2024

Hi All, I've just installed MM3 on a fresh Debian 12 using the Virtualenv method. (venv)...$ mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] Postorius Version 1.3.10 After creating a couple of lists, the lists are shown in the table on the "Mailing lists" webpage as they should, but clicking on any of the listnames returns an error page: Something went wrong HTTP Error 500: {"title": "500 Internal Server Error"} And the Mailman log is a mystery to me, and no error is reported in the Nginx log ... # tail -f /opt/mailman/web/logs/mailmanweb.log ERROR 2024-01-24 05:41:46,327 51799 postorius Un-handled exception: HTTP Error 500: {"title": "500 Internal Server Error"} Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 103, in view return self.dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/postorius/views/generic.py", line 76, in dispatch return super(MailingListView, self).dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 142, in dispatch return handler(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/postorius/views/list.py", line 359, in get and 'hyperkitty' in archivers # noqa: W504 ^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen _collections_abc>", line 780, in __contains__ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/base.py", line 146, in __getitem__ return self._get(key) ^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/base.py", line 88, in _get return self.rest_data[key] ^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/base.py", line 74, in rest_data response, content = self._connection.call(self._url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/connection.py", line 160, in call raise HTTPError(params.get('url'), response.status_code, urllib.error.HTTPError: HTTP Error 500: {"title": "500 Internal Server Error"} ERROR 2024-01-24 05:41:46,333 51799 django.request Internal Server Error: /postorius/lists/testing.example.com/ ERROR 2024-01-24 05:41:46,333 51799 django.request Internal Server Error: /postorius/lists/testing.example.com/ Could anyone steer me in the right direction for fixing this? Thanks, Mark

6 20

export/import list?
by Dave McGuire March 1, 2024

March 1, 2024

I'm migrating to a new Mailman3 instance, finally escaping a fairly old (3.1.1) and problematic distribution repo install on a Linux VM and moving to a venv install in a native SmartOS Zone. I've gotten everything running well and integrated with our (fairly complex) mail server. I'd like to begin migration of the lists on the old mm3 instance. Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc? Thanks, -Dave -- Dave McGuire, AK4HZ New Kensington, PA

6 9

search this list searches more then just this list
by Marco van Tol Feb. 6, 2024

Feb. 6, 2024

Hi there, We run the following versions: mailman-core: 3.3.9 mailman-core-api: 3.1 mailman-core-python: 3.11.6 hyperkitty: 1.3.8 We have several lists on our server, I will write down a list of example list names that are good enough for the purpose of my question. xx(a)domain.tld <mailto:xx@domain.tld> yy-xx(a)domain.tld <mailto:yy-xx@domain.tld> zz-xx(a)domain.tld <mailto:zz-xx@domain.tld> When I now visit the archives for xx(a)domain.tld <mailto:xx@domain.tld> and enter a search phrase in “search this list”, the results include the other 2 lists. It looks like “search this list” searches for keyword in any listname that matches the one from "this list". Is there a way to really only search “this list” when I go to the archives for xx(a)domain.tld <mailto:xx@domain.tld>? I have tried to change the url for the results to have a ‘^’ in front of the listname, but that gives an error :-) Thanks in advance! Marco van Tol

4 23

after upgrade: OperationalError: no such column: hyperkitty_mailinglist.archive_rendering_mode
by Hans Mayer Jan. 31, 2024

Jan. 31, 2024

Dear All, recently I did a system os upgrade debian 11.8 -> 12.4 / bullseye -> bookworm This did also upgrade Mailman3 version 3.3.3 -> 3.3.8 as a used the packages which came with Debian OS. Initially after the upgrade I had some troubles with the MM3 services to start, but I fixed this. Now almost everything is running. Only the Archives are not working. The browser shows a server error, when I look at the mailman-web.log I see ERROR 2024-01-31 15:14:14,403 2699563 django.request Internal Server Error: /mailman3/hyperkitty/ Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/sqlite3/base.py", line 423, in execute return Database.Cursor.execute(self, query, params) sqlite3.OperationalError: no such column: hyperkitty_mailinglist.archive_rendering_mode When I check the schema I find such a column in table "mailinglist" I assume there is only one solution, to uninstall and delete everything and start from scratch. Or is there a way the check all tables with a script and do an table update for missing columns ? Or a partial installation of hyperkitty, as I don't care about the archives. I am using sqlite3 as DB backend. Kind regards Hans --

2 1

transport map (postfix_lmtp) does not update - new lists not working
by Nils Jan. 29, 2024

Jan. 29, 2024

End of last year, I've re-installed mailman3 using the virtualenv installation method on ubuntu 20.04. Since then, the transport map (/opt/mailman/mm/var/data/postfix_lmtp) has not been updated even though, new mailing lists were created. The old mailing lists, that existed before the re-installation work perfectly fine but those created after are not accessible. Because the entries for those new lists are missing in the transport map, postfix replies with a "mail for ... loops back to myself" error. I'm quite sure, that I did all the steps in the documentation (except that I use a slightly different apache2 vhost configuration). I double checked on the cronjobs. crontab -u mailman -e @daily /opt/mailman/venv/bin/mailman digests --periodic @daily /opt/mailman/venv/bin/mailman notify * * * * * /opt/mailman/venv/bin/mailman-web runjobs minutely 0,15,30,45 * * * * /opt/mailman/venv/bin/mailman-web runjobs quarter_hourly @hourly /opt/mailman/venv/bin/mailman-web runjobs hourly @daily /opt/mailman/venv/bin/mailman-web runjobs daily @weekly /opt/mailman/venv/bin/mailman-web runjobs weekly @monthly /opt/mailman/venv/bin/mailman-web runjobs monthly @yearly /opt/mailman/venv/bin/mailman-web runjobs yearly Any idea, what I could have missed??? Kind regards, Nils

2 2

Fresh installation mailman3-suite via "pipx" - any experiences yet?
by Wikinaut Jan. 27, 2024

Jan. 27, 2024

I recently found the "pipx" wrapper very useful and wanted to ask you if this could be facilitate the fresh installation of the mailman3-suite. Has anyone in the list already experiences installing mailman and mailmanweg through "pipx"? Here are my general tipps for using pipx # https://pipx.pypa.io/stable/installation/#on-linux sudo apt install pipx python3-venv To install a package systemwide for all users, I recommend the tip from https://github.com/pypa/pipx/issues/754#issuecomment-951162846 . sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install <package>

4 6

Single Click Unsubscribe
by J D Jan. 26, 2024

Jan. 26, 2024

I know this has been discussed in the past: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… However, Google has recently announced that they will be enforcing one-click unsubscribe for bulk senders: https://techcrunch.com/2023/10/03/gmail-to-enforce-harsher-rules-in-2024-to… Thoughts?

5 5

mailman-web compress - Invalid template socialaccount/login_cancelled.html
by Odhiambo Washington Jan. 24, 2024

Jan. 24, 2024

Running an install on Debian12 VM. Getting a new error message that I haven't seen previously: (venv) mailman@debian12:~$ mailman-web compress Compressing... Invalid template socialaccount/login_cancelled.html: Invalid block tag on line 7: 'element', expected 'endblock'. Did you forget to register or load this tag? Invalid template account/verified_email_required.html: Invalid block tag on line 7: 'element', expected 'endblock'. Did you forget to register or load this tag? Error parsing template socialaccount/signup.html: socialaccount/base.html done Compressed 2 block(s) from 78 template(s) for 1 context(s). -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

7 13

share directories between postfix and mailman 3
by Thomas Schachtner Jan. 24, 2024

Jan. 24, 2024

Hi there, I think I have (finally) set up a working instance of mailman 3 on a host separate from the companion postfix mail server actually sending out the mails. One server (postfix) is directly in the internet and the other one is in an internal network not directly reachable from the internet (but connected using a reverse proxy). The services on the hosts are available to one another (i. e. the postfix server is reachable using 127.0.0.1:25 from mailman and mailman is reachable using 127.0.0.1:8024 from postfix using a permanent ssh tunnel betwenn the two hosts). I think that this solution could work quite reliably... (at least I hope so.) But that's not the full deal. Postfix needs access to some hashed configuration files containing the transport maps and the domains of the mailman installation, and mailman needs the contents of the postfix configuration file. As a quick and dirty solution, I've established a process which periodically copies the /var/lib/mailman3 directory from the mailman box to the postfix box using rsync. But I would like to have a more professional solution, like a directly shared directory between both. I did not yet find any suggestions on how to achieve this. Are there any best practices recommendations? I already tried sharing the directories using sshfs, but that did not work. I connected the remote directory postfix.example.com:/var/lib/mailman3 to my local directory /var/lib/mailman3 using sshfs root@postfix.example.com:/var/lib/mailman3 /var/lib/mailman3 This seemed to work pretty fine, as all files and directories seemed to be available, but when starting mailman 3, the following error message was shown: FileExistsError: A race condition might have happened. /var/lib/mailman3 actually exists and is not a directory. Any idea why this message is shown? So, is it a bad idea to use sshfs? Do I have to install an NFS server? But I am not sure if sharing directories using NFS across the Internet is a good idea... Any suggestions? Best regards Tom

3 6

Re: Process on migrating mailman
by Odhiambo Washington Jan. 24, 2024

Jan. 24, 2024

On Tue, Jan 23, 2024 at 4:22 AM Mark Labeste <marqueejen13(a)gmail.com> wrote: > About 500 list. Not sure how to check size of the archives. > > On Thu, Jan 18, 2024 at 4:44 PM Odhiambo Washington <odhiambo(a)gmail.com> > wrote: > >> >> >> On Thu, Jan 18, 2024 at 8:49 AM Mark Labeste <marqueejen13(a)gmail.com> >> wrote: >> >>> Any suggestion on how to migrate mailing list and archirve with >>> diffirent server and domain. For the mailing list from >>> sample.domain1.example.com to sample.domain2.example.com >>> >>> Old Mailing list >>> Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) >>> Mailman Core API Version 3.1 >>> Mailman Core Python Version 3.6.8 (default, Sep 26 2019, 11:57:09) >>> [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] >>> Domain Domain1.example.com >>> >>> New Mailing list >>> Mailman Core Version GNU Mailman 3.3.9 (Tom Sawyer) >>> Mailman Core API Version 3.1 >>> Mailman Core Python Version 3.9.18 (main, Sep 22 2023, 17:58:34) >>> [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] >>> Domain domain2.example.com >> >> >> >> How many lists? How big are the archives? >> >> Looking at >> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…, >> it seems like it's not a trivial exercise. >> >> However, I am wondering about the consequences of: >> 1. dumping the DBs and doing 's/DOMAIN1/DOMAIN2/g' in the dumps, then >> re-importing >> 2. editing settings.py and webserver config to reflect the DOMAIN2 >> 3. reconfiguring your MTA to handle list mails using the DOMAIN2 >> >> I am imagining a virtualenv type of installation. >> > The archives are in the database so nothing to worry about, Now that I think about this again, after dumping and importing the database as shown above, you need to rerun the 'mailman-web migrate` once again to import any DB schema changes between the versions. Actually, just run all the options from https://docs.mailman3.org/en/latest/install/virtualenv.html#run-database-mi… . -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

2 3

log rotation for bounce.log
by Quanah Gibson-Mount Jan. 22, 2024

Jan. 22, 2024

I noticed that our smtp.log and bounce.log files haven't been rotated in years. I see the reason for smtp.log not being rotated is due to <https://gitlab.com/mailman/mailman/-/issues/931>. Does this same issue affect bounce.log or is it safe to rotate similar to mailman.log?

2 1

Extract a list of the lists and the administrators who manage it
by pau.baiges＠csuc.cat Jan. 22, 2024

Jan. 22, 2024

Hello, Is there a way to extract all the lists and show the administrators who manage those lists? Thanks!

3 2

emails to list missing from archive after upgrade
by Quanah Gibson-Mount Jan. 19, 2024

Jan. 19, 2024

So next issue post upgrade: I sent three emails to one of the lists today. All of them went to the list, as I got copies of each one, and the headers clearly show them coming from the list. However, when I go look at the archives for the list, they don't show up at all. And it's been over 8 hours since I sent them. --Quanah

2 3

Delivery to mail@domain.tld failed with code 444, [Errno 111] Connection refused
by christopher.claus＠tgcamberg1848.de Jan. 19, 2024

Jan. 19, 2024

Hi together, I´m using Mailman3.3.9 and postfix in docker containers seperatly. Everything worked fine since this afternoon. The error occurs, after postfix transports an incoming mail to mailman. After processing the mail, sending to all recepients failed >> Delivery to mail(a)domain.tld failed with code 444, [Errno 111] Connection refused I checked the configuration of postfix and added a wide IP-range to mynetworks: postfix-main.cf mynetworks = mailman-core tgc_mailman_core t 127.0.0.0/8 [::1]/128 [fe80::]/64 172.18.0.0/16 172.21.0.0/16 [2001:8d8:1800:283]/64 172.0.0.0/24 relay_domains = $mydestination #smtpd_sender_restrictions = reject_unknown_sender_domain,reject_unknown_reverse_client_hostname smtpd_sender_restrictions = reject_unknown_sender_domain smtpd_relay_restrictions = permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination smtpd_recipient_restrictions = permit_sasl_authenticated check_recipient_access hash:/etc/postfix/filtered_domains permit_mynetworks reject_unauth_destination smtpd_data_restrictions = reject_unauth_pipelining Does anyone have a clue for me as to why delivery to Postfix is failing? My last change a couple of days ago was adding an public ipv6-address to the host. But docker still works without IPv6. I thought about client-identification issues between maiolman-core and postfix but did noch find anything to fix ... Best regards, Christopher

2 2

moving list from mm3 -> mm3
by Jens Günther Jan. 19, 2024

Jan. 19, 2024

Hey there, I'm still trying to figure out, how to move a bunch of lists from one mm3-instance to another. If somebody has a working solution, I would be really glad to hear from it. I came across this thread, but I don't think it tackles my question: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… (or maybe I missed the right parts) We have equal setups and versions: - maxxking docker install 0.4.5 - Core 3.3.8 - API 3.1 - Postorius 1.3.8 ## Config So I found the possibility to get the config of a list via REST-API: curl -u restadmin:restpass localhost:8001/3.1/lists/$short.$domain/config | jq > short.domain This is a nice collection of the configuration of a certain list. Does anybody know how I would get this data back to the newly created list in the second working mm3-instance? Shouldn't it be possible via POST command? I thought of something like: curl -u restadmin:restpass -d @short.domain -H "Content-Type: application/json" localhost:8001/3.1/lists/&short.&domain/config But it returns {"title": "405 Method Not Allowed"} and I don't know where to look in the docs, since it shows almost always a way through mailman shell. Here I found something about POST: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… -> but how would it work exactly? ## Members, moderators, owners To export all members I normally run within mm-core-container as user mailman the following command: mailman members &short.&domain > &short.txt To fill them back in, it can be done in Postorius (or is there a way through API?). What I would love to know is how to get out Owners of the corresponding lists and reimport them in the new one (which will be the same at the end :-D but different instance). ## registered users This part is somehow tricky, since I have no clue how to do it. Some of the mailinglist-admins don't want to bother their list-owners again with registering newly on the other system. Is there a way to export all users (I think I found them in the DB in the table auth_users, somehow in combination with account_emailaddress) and get them over to the new instance (actually I just need part of them, but that's another question that I might solve lateron on my own). Maybe the answer is already here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… -> but can somebody help to get this working (import/export)? Thank you in advance, I hope it's not too much or too nooby :-) Jens.

3 10

All users getting disabled after upgrade of mailman3
by Quanah Gibson-Mount Jan. 19, 2024

Jan. 19, 2024

Hello, We just upgraded our mailman to the latest provided from Debian bookworm (I know, not the recommended installation method). Now everyone's having their addresses disabled due to "bounce" errors. This is odd, because I'm disabled, even though I received the 2 emails I sent to the list today (and that's the only list traffic today). I used the mailman shell command from <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> to see who has been disabled. I had to make one fix to it, it should be m.list_name not mlist.list_name. Confirmed that dozens of valid users have had their sub disabled. In the logs I see: Jan 18 19:25:17 2024 (952) Member quanah(a)fast-mail.org on list openldap-technical.openldap.org, bounce score = 1. Jan 18 19:37:11 2024 (952) Member quanah(a)fast-mail.org on list openldap-technical.openldap.org, bounce score = 2. Jan 18 19:37:11 2024 (952) Disabling delivery for quanah(a)fast-mail.org on list openldap-technical.openldap.org by bounce Jan 18 19:37:11 2024 (952) Member quanah(a)fast-mail.org on list openldap-technical.openldap.org, bounce score 2 >= threshold 2, disabling delivery. Jan 18 10:05:42 2024 (938) Peer: ('127.0.0.1', 36100) Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) handling connection Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'LHLO hypatia.openldap.org' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'MAIL FROM:<openldap-commit-bounces(a)openldap.org>' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) sender: openldap-commit-bounces(a)openldap.org Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'RCPT TO:<openldap-commit-owner(a)openldap.org>' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) recip: openldap-commit-owner(a)openldap.org Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'DATA' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'QUIT' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) connection lost Jan 18 10:05:42 2024 (938) Connection lost during _handle_client() Jan 18 10:05:45 2024 (940) <170557233590.937.7274928299237969540(a)hypatia.openldap.org> post to openldap-commit(a)openldap.org from openldap-commit-owner(a)openldap.org, 20265 bytes Jan 18 10:42:20 2024 (803) <170557453971.798.17359529178123532376(a)hypatia.openldap.org> low level smtp error: [Errno 111] Connection refused Jan 18 10:42:20 2024 (803) <170557453971.798.17359529178123532376(a)hypatia.openldap.org> delivery to quanah(a)openldap.org failed with code 444, [Errno 111] Connection refused But... postfix is working fine, and I've had hundreds of emails generated *by* mailman about it disabling people due to bounces. Thoughts on how to proceed appreciated. Regards, Quanah

2 4

Splitting hosts: mta&core vs postorius
by Ruth Ivimey-Cook Jan. 18, 2024

Jan. 18, 2024

I'm trying to set up mailman3 on two hosts, one of which runs core + exim, the other runs postorius (&possibly hyperkitty). I have I think got core set up correctly, and postorius is sort of usable, but anything in the web ui that talks to core ends up with a white screen. I think what is happening is a postorius URL ends up with the core (or perhaps no) hostname, and as there's nothing there to answer it fails. I've checked the settings and I don't think I have misconfigured postorius. Is it possible that the bad url is being passed from core to postorius in some way and reused without changing the host? In the config at present, both hosts are visible to the browser, and each host can communicate with the other. Any ideas on how to pin down the problem better? Ruth

6 31

Process on migrating mailman
by Mark Labeste Jan. 18, 2024

Jan. 18, 2024

Any suggestion on how to migrate mailing list and archirve with diffirent server and domain. For the mailing list from sample.domain1.example.com to sample.domain2.example.com Old Mailing list Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.6.8 (default, Sep 26 2019, 11:57:09) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] Domain Domain1.example.com New Mailing list Mailman Core Version GNU Mailman 3.3.9 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] Domain domain2.example.com

3 3

how to migrate mailing lists to a new domain using mailman 3
by C. Souza Jan. 18, 2024

Jan. 18, 2024

In a few days my current domain will be disabled (it is out of my control) and I would like to know how to migrate the mailing lists to the new domain name. As a solution I added a new domain in mailman 3 and created a script that replicates the lists in this new domain. To forward the emails received by the old lists, the script removes the members and adds the email of the new list. Is there a fashionable way to do that? Also, one problem that I don't have a solution is the archive of the old lists. How to bring them to the new ones? Thanks for your help. My mailing list server is running on a CenOS-7x with postgreSQL and the following mailman packages: Package Version ------------------- ---------- 1 aiosmtpd 1.2.4 2 alembic 1.5.3 3 arrow 0.17.0 4 asgiref 3.3.1 5 atpublic 2.1.2 6 authheaders 0.13.0 7 authres 1.2.0 8 bleach 3.3.0 9 blessed 1.17.12 10 certifi 2020.12.5 11 cffi 1.14.4 12 chardet 4.0.0 13 click 7.1.2 14 cmarkgfm 0.5.2 15 cryptography 3.3.1 16 decorator 4.4.2 17 defusedxml 0.6.0 18 Django 3.0.12 19 django-allauth 0.44.0 20 django-appconf 1.0.4 21 django-compressor 2.4 22 django-extensions 3.1.0 23 django-gravatar2 1.4.4 24 django-haystack 3.0 25 django-mailman3 1.3.5 26 django-picklefield 3.0.1 27 django-q 1.3.4 28 djangorestframework 3.12.2 29 dkimpy 1.0.5 30 dnspython 2.1.0 31 docutils 0.16 32 falcon 2.0.0 33 flufl.bounce 3.0.1 34 flufl.i18n 2.0.2 35 flufl.lock 3.2 36 gunicorn 20.0.4 37 HyperKitty 1.3.3 38 idna 2.10 39 importlib-resources 5.1.0 40 lazr.config 2.2.3 41 lazr.delegates 2.0.4 42 mailman 3.3.2 43 mailman-hyperkitty 1.1.0 44 mailman-web 0.0.3 45 mailmanclient 3.3.2 46 Mako 1.1.4 47 MarkupSafe 1.1.1 48 networkx 2.5 49 nose 1.3.7 50 oauthlib 3.1.0 51 packaging 20.9 52 passlib 1.7.4 53 pip 21.0.1 54 postorius 1.3.3 55 psycopg2-binary 2.8.6 56 publicsuffix2 2.20191221 57 pycparser 2.20 58 Pygments 2.7.4 59 PyJWT 2.0.1 60 pyparsing 2.4.7 61 python-dateutil 2.8.1 62 python-editor 1.0.4 63 python3-openid 3.2.0 64 pytz 2021.1 65 rcssmin 1.0.6 66 readme-renderer 28.0 67 requests 2.25.1 68 requests-oauthlib 1.3.0 69 rjsmin 1.1.0 70 robot-detection 0.4 71 setuptools 39.2.0 72 six 1.15.0 73 SQLAlchemy 1.3.23 74 sqlparse 0.4.1 75 typing-extensions 3.7.4.3 76 urllib3 1.26.3 77 uWSGI 2.0.19.1 78 wcwidth 0.2.5 79 webencodings 0.5.1 80 wheel 0.36.2 81 Whoosh 2.7.4 82 zipp 3.4.0 83 zope.component 4.6.2 84 zope.configuration 4.4.0 85 zope.deferredimport 4.3.1 86 zope.deprecation 4.4.0 87 zope.event 4.5.0 88 zope.hookable 5.0.1 89 zope.i18nmessageid 5.0.1 90 zope.interface 5.2.0 91 zope.proxy 4.3.5 92 zope.schema 6.0.1 -- Carlos Souza

2 1

Postorius and some CSS issue?
by Odhiambo Washington Jan. 12, 2024

Jan. 12, 2024

Happy New Year to you too all. MM3 installed today using the virtualenv method. On my testbench, I am having this strange issue where the Postorius is not displaying correctly perhaps because of some CSS issue - or just my own hangovers! I have used the standard method as documented. The error I get: https://pastebin.ubuntu.com/p/dhvsHFG4jH/ -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

2 1

Loops back to myself
by David Krantz Jan. 12, 2024

Jan. 12, 2024

Hi, I have a problem with my Mailman 3-setup. The symptom is that postfix rejects non-existing addresses for my list domains with <adr(a)example.com>: mail for example.com loops back to myself What I guess happens happens is that postfix for some reason accepts the mail as it is for a relay domain and finds that it is not deliverable locally as the address does not exist locally and then tries to relay it on to the appropriate MX server, which is itself and then fails and generates a bounce message. This is a bit annoying as I have set up postfix to only accept valid addresses. I have set the transport_maps, local_recipient_maps and relay_domains to the files generated by MM3. I run an older version of MM3 (3.0) but prefer to fix this without upgrading as upgrading is a relatively big undertaking. This is mostly a backscatter problem for spam I guess. Any ideas of very simple and easy to forget things that I've probably missed? Cheers // David

5 6

Review requested for some merge requests
by Stephen J. Turnbull Jan. 10, 2024

Jan. 10, 2024

Reply-To set to Mailman Developers. Please check that your replies go there. Or you can comment on MRs directly if you have a GitLab login. I have three merge requests pending: [1] https://gitlab.com/mailman/mailman/-/merge_requests/1159 [2] https://gitlab.com/mailman/mailman/-/merge_requests/1160 [3] https://gitlab.com/mailman/mailman/-/merge_requests/1161 [1] fixes the logging initialization bug that prevented the `smtp` log from rotating properly, and maybe some others. I don't think it needs user review because it's a pure bugfix, but if you're curious and take a look, and you think it needs better documentation let me know. [2] adds a configuration variable `lmtp_max_size`. This is user-visible, so review of the documentation in docs/mta.rst and config/mailman.cfg for clarity and readability would be appreciated. [3] adds postfix-pgsql support for use of the PostgreSQL database to route Mailman traffic. It's really most useful in an edge case (migration of 10,000+ list Mailman 2 instances), but it's theoretically attractive on the principle of "single source of truth", and possibly for implementing a "microservices" style cloud instance (it eliminates the need for Mailman and Postfix to share a file system). Besides opinions on the value of the general idea I have two specific questions: - Postfix docs recommend not looking up domains in the database to reduce load on the database server. But on a dedicated Mailman system, you're going to be looking up pretty much everything twice anyway (once to check for local recipient and once to fetch the transport), and my experience so far is that this isn't a big load on Postgres, and it's not a drag on list traffic distribution. Should I implement domain lookup in Postgres? Should it be optional if the lists are being looked up in Postgres? - The current patch completely disables generation of the postfix_lmtp alias file, even for the `mailman aliases` command. Should I implement an option to force generation of the files for documentation (and maybe a backup in a very rare and contorted situation where Mailman can see Postgres but Postfix can't)? Thanks, Steve

3 3

Mailman 3 Core user assistance
by Tony Deshields Jan. 10, 2024

Jan. 10, 2024

I currently have a docker mailman stack that utilizes Django LDAP backend for authentication. I have my users attributes mapped as you see below in my settings.py file. Whenever a user logs into postorius with their 'uid', mailman-core uses their 'uid' as their email address instead of the 'email' attribute. Is there anyway to change this functionality? I would like my users to be able to login with their enterprise uid but be mapped to their email but I feel like there is a disconnect between Django and MM3 core. Any assistance is much appreciated. AUTH_LDAP_USER_ATTR_MAP = { "uid" : "userPrincipalName", "first_name": "givenName", "last_name": "sn", "email" : "mail", }

2 1

Docker mailman image can't access https://publicsuffix.org/
by bob B Jan. 9, 2024

Jan. 9, 2024

Thanks so it looks like Mailman can not get to publicsuffix. The wrinkle is I am running maxking's docker version. The host can get there, but not from the docker image. Any idea to to configure that? and this only comes into play when I set the dmarc settings.

3 8

The 'flufl.lock>=5.1' distribution was not found, but flufl.lock 8.0.2 was installed
by Nils Jan. 8, 2024

Jan. 8, 2024

3 2

The doubled entries issue
by Lars Bjørndal Jan. 7, 2024

Jan. 7, 2024

I've got the problem Multiple rows were found when one or none was required. I found the issue and a script here: https://gitlab.com/mailman/mailman/-/issues/657 I did su - mailman3 and ran: python scriptname.py and got: Traceback (most recent call last): File "/opt/mailman3/fix-doubled-entries.py", line 21, in <module> listmgr = get_list_manager() ^^^^^^^^^^^^^^^^^^ File "/opt/mailman3/fix-doubled-entries.py", line 6, in get_list_manager return getUtility(IListManager) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman3/venv/lib64/python3.11/site-packages/zope/component/_api.py ", line 180, in getUtility raise ComponentLookupError(interface, name) zope.interface.interfaces.ComponentLookupError: (<InterfaceClass mailman.interfa ces.listmanager.IListManager>, '') (venv) 1[mailman3@dalen ~]$ Do I miss something? How can I fix the problem? Thanks, Lars

4 10

Archive import fails: "IndexError: list index out of range"
by Johannes Rohr Jan. 6, 2024

Jan. 6, 2024

I am now importing the archives of my old mailman2 installation. Unfortunately, with some of the larger and older lists, import fails halfway through, with an exception, the last message of which is IndexError: list index out of range Below is an example. This means that the last few years are missing from the archive in several lists. Are there any suggestions what I could do to still get them importet? Thanks! Johannes File "/usr/lib/python3.8/email/_header_value_parser.py", line 2069, in get_msg_id token, value = get_dot_atom_text(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1334, in get_dot_atom_text raise errors.HeaderParseError("expected atom at a start of " email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found '@localhost>' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/opt/mailman/venv/lib/python3.8/site-packages/mailman_web/manage.py", line 37, in main execute_from_command_line(sys.argv) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/venv/lib/python3.8/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) File "/opt/mailman/venv/lib/python3.8/site-packages/hyperkitty/management/commands/hyperkitty_import.py", line 387, in handle importer.from_mbox(mbfile, report_name) File "/opt/mailman/venv/lib/python3.8/site-packages/hyperkitty/management/commands/hyperkitty_import.py", line 212, in from_mbox progress_marker.tick(unquote(message.get("message-id", 'n/a'))) File "/usr/lib/python3.8/email/message.py", line 471, in get return self.policy.header_fetch_parse(k, v) File "/usr/lib/python3.8/email/policy.py", line 163, in header_fetch_parse return self.header_factory(name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 607, in __call__ return self[name](name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 202, in __new__ cls.parse(value, kwds) File "/usr/lib/python3.8/email/headerregistry.py", line 535, in parse kwds['parse_tree'] = parse_tree = cls.value_parser(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2126, in parse_message_id token, value = get_msg_id(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2073, in get_msg_id token, value = get_obs_local_part(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1516, in get_obs_local_part if (obs_local_part[0].token_type == 'dot' or IndexError: list index out of range

6 10

Not receiving email from outlook or gmail.
by Mark Labeste Jan. 5, 2024

Jan. 5, 2024

From mailman web it was able to deliver the email to outlook and gmail but when replying or confirming subcription it can't reach the mailman. This error is bounce back from outlook. Server at destination.email.com (192.10.10.10) returned '400 4.4.7 Message delayed' 1/4/2024 12:16:13 PM - Server at destination.email.com (192.10.10.10) returned '451 4.4.397 Error communicating with target host. -> 421 4.2.1 Unable to connect -> SocketTimedout: Socket error code 10060 Thanks, mARK

2 1

Programmatically modify Thread-Topic *before* message is sent out the door?
by Thomas Ward Jan. 3, 2024

Jan. 3, 2024

We use a custom plugin to do numeric thread stamping, which has been working well in the code system. However, it was discovered by Microsoft that Microsoft Outlook and OWA prioritize the Thread-Topic header over the Subject header, which results in the oddness we've experienced in MS365 clients in testing. During our tasks though a custom handler goes through during the pipeline and modifies the Subject field accordingly, adding a thread tag to it. While this is 'fragile' it does work; the only problem is that Thread-Topic doesn't get altered. Programmatically, how would we alter the Thread-Topic during the handler's processing to prepend the corresponding prefix/tag so that when the message actually goes out the door it is properly sending with the modified Thread-Topic? Thomas

2 6

REST API for list owners/moderators
by mathias＠koerber.org Jan. 3, 2024

Jan. 3, 2024

Does Mailman 3 have an REST API that allows list-owners/moderators to perform list-management functions without having access to the whole server/other lists etc? ie, an API that uses the same credentials as Postorious but allows mass-subscription/un-subscription/membership-export etc and changes to list settings, membership settings (mod etc) via REST or python? The documentation seems to only indicate a single one-admin-user REST API that should not even be accessible from outside?

2 1

Message getting shunted with enabling Dmarc replace with list address
by bob B Jan. 3, 2024

Jan. 3, 2024

So we host a mailing list for domain X, the MX record points to us, we are domain y. If we have Dmarc set to "no dmarc mitigations" I can send a message from my email in domain Y and it goes to the list in domain x and is processed and sent out. If I set the dmarc to "replace with list address". the same message gets shunted. I included the logs below Not sure why it is trying to contact publicsuffix.org? ************* /mailman/core/var/logs/mailman.log Jan 02 20:41:50 2024 (29) No cached copy of the public suffix list found Jan 02 20:41:55 2024 (29) Uncaught runner exception: HTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/public_suffix_list.dat (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)')) Jan 02 20:41:55 2024 (29) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn conn = connection.create_connection( File "/usr/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection raise err File "/usr/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection sock.connect(sa) TimeoutError: timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen httplib_response = self._make_request( File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn conn.connect() File "/usr/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect self.sock = conn = self._new_conn() File "/usr/lib/python3.10/site-packages/urllib3/connection.py", line 179, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)') During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 489, in send resp = conn.urlopen( File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen retries = retries.increment( File "/usr/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/public_suffix_list.dat (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/incoming.py", line 80, in _dispose process(mlist, msg, msgdata, start_chain) File "/usr/lib/python3.10/site-packages/mailman/core/chains.py", line 57, in process if link.rule.check(mlist, msg, msgdata): File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 316, in check if maybe_mitigate(mlist, address): File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 292, in maybe_mitigate org_dom = get_organizational_domain(from_domain) File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 142, in get_organizational_domain parse_suffix_list() File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 109, in parse_suffix_list filename = ensure_current_suffix_list() File "/usr/lib/python3.10/site-packages/mailman/rules/dmarc.py", line 77, in ensure_current_suffix_list content = get(config.dmarc.org_domain_data_url) File "/usr/lib/python3.10/site-packages/mailman/utilities/protocols.py", line 39, in get response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws) File "/usr/lib/python3.10/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/usr/lib/python3.10/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 553, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='publicsuffix.org', port=443): Max retries exceeded with url: /list/public_suffix_list.dat (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f8b38d318a0>, 'Connection to publicsuffix.org timed out. (connect timeout=5)')) Jan 02 20:41:55 2024 (29) SHUNTING: 1704228115.0417743+619bfacfe7e6d030991468aa2707d065418c56e5

2 1