January 2024 - Mailman-users - lists.mailman3.org

moving list from mm3 -> mm3
by Jens Günther 19 Jan '24

19 Jan '24

Hey there, I'm still trying to figure out, how to move a bunch of lists from one mm3-instance to another. If somebody has a working solution, I would be really glad to hear from it. I came across this thread, but I don't think it tackles my question: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… (or maybe I missed the right parts) We have equal setups and versions: - maxxking docker install 0.4.5 - Core 3.3.8 - API 3.1 - Postorius 1.3.8 ## Config So I found the possibility to get the config of a list via REST-API: curl -u restadmin:restpass localhost:8001/3.1/lists/$short.$domain/config | jq > short.domain This is a nice collection of the configuration of a certain list. Does anybody know how I would get this data back to the newly created list in the second working mm3-instance? Shouldn't it be possible via POST command? I thought of something like: curl -u restadmin:restpass -d @short.domain -H "Content-Type: application/json" localhost:8001/3.1/lists/&short.&domain/config But it returns {"title": "405 Method Not Allowed"} and I don't know where to look in the docs, since it shows almost always a way through mailman shell. Here I found something about POST: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… -> but how would it work exactly? ## Members, moderators, owners To export all members I normally run within mm-core-container as user mailman the following command: mailman members &short.&domain > &short.txt To fill them back in, it can be done in Postorius (or is there a way through API?). What I would love to know is how to get out Owners of the corresponding lists and reimport them in the new one (which will be the same at the end :-D but different instance). ## registered users This part is somehow tricky, since I have no clue how to do it. Some of the mailinglist-admins don't want to bother their list-owners again with registering newly on the other system. Is there a way to export all users (I think I found them in the DB in the table auth_users, somehow in combination with account_emailaddress) and get them over to the new instance (actually I just need part of them, but that's another question that I might solve lateron on my own). Maybe the answer is already here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… -> but can somebody help to get this working (import/export)? Thank you in advance, I hope it's not too much or too nooby :-) Jens.

3 10

All users getting disabled after upgrade of mailman3
by Quanah Gibson-Mount 19 Jan '24

19 Jan '24

Hello, We just upgraded our mailman to the latest provided from Debian bookworm (I know, not the recommended installation method). Now everyone's having their addresses disabled due to "bounce" errors. This is odd, because I'm disabled, even though I received the 2 emails I sent to the list today (and that's the only list traffic today). I used the mailman shell command from <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…> to see who has been disabled. I had to make one fix to it, it should be m.list_name not mlist.list_name. Confirmed that dozens of valid users have had their sub disabled. In the logs I see: Jan 18 19:25:17 2024 (952) Member quanah(a)fast-mail.org on list openldap-technical.openldap.org, bounce score = 1. Jan 18 19:37:11 2024 (952) Member quanah(a)fast-mail.org on list openldap-technical.openldap.org, bounce score = 2. Jan 18 19:37:11 2024 (952) Disabling delivery for quanah(a)fast-mail.org on list openldap-technical.openldap.org by bounce Jan 18 19:37:11 2024 (952) Member quanah(a)fast-mail.org on list openldap-technical.openldap.org, bounce score 2 >= threshold 2, disabling delivery. Jan 18 10:05:42 2024 (938) Peer: ('127.0.0.1', 36100) Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) handling connection Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'LHLO hypatia.openldap.org' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'MAIL FROM:<openldap-commit-bounces(a)openldap.org>' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) sender: openldap-commit-bounces(a)openldap.org Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'RCPT TO:<openldap-commit-owner(a)openldap.org>' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) recip: openldap-commit-owner(a)openldap.org Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'DATA' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) Data: b'QUIT' Jan 18 10:05:42 2024 (938) ('127.0.0.1', 36100) connection lost Jan 18 10:05:42 2024 (938) Connection lost during _handle_client() Jan 18 10:05:45 2024 (940) <170557233590.937.7274928299237969540(a)hypatia.openldap.org> post to openldap-commit(a)openldap.org from openldap-commit-owner(a)openldap.org, 20265 bytes Jan 18 10:42:20 2024 (803) <170557453971.798.17359529178123532376(a)hypatia.openldap.org> low level smtp error: [Errno 111] Connection refused Jan 18 10:42:20 2024 (803) <170557453971.798.17359529178123532376(a)hypatia.openldap.org> delivery to quanah(a)openldap.org failed with code 444, [Errno 111] Connection refused But... postfix is working fine, and I've had hundreds of emails generated *by* mailman about it disabling people due to bounces. Thoughts on how to proceed appreciated. Regards, Quanah

2 4

Splitting hosts: mta&core vs postorius
by Ruth Ivimey-Cook 18 Jan '24

18 Jan '24

I'm trying to set up mailman3 on two hosts, one of which runs core + exim, the other runs postorius (&possibly hyperkitty). I have I think got core set up correctly, and postorius is sort of usable, but anything in the web ui that talks to core ends up with a white screen. I think what is happening is a postorius URL ends up with the core (or perhaps no) hostname, and as there's nothing there to answer it fails. I've checked the settings and I don't think I have misconfigured postorius. Is it possible that the bad url is being passed from core to postorius in some way and reused without changing the host? In the config at present, both hosts are visible to the browser, and each host can communicate with the other. Any ideas on how to pin down the problem better? Ruth

6 31

Process on migrating mailman
by Mark Labeste 18 Jan '24

18 Jan '24

Any suggestion on how to migrate mailing list and archirve with diffirent server and domain. For the mailing list from sample.domain1.example.com to sample.domain2.example.com Old Mailing list Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.6.8 (default, Sep 26 2019, 11:57:09) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] Domain Domain1.example.com New Mailing list Mailman Core Version GNU Mailman 3.3.9 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.9.18 (main, Sep 22 2023, 17:58:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] Domain domain2.example.com

3 3

how to migrate mailing lists to a new domain using mailman 3
by C. Souza 18 Jan '24

18 Jan '24

In a few days my current domain will be disabled (it is out of my control) and I would like to know how to migrate the mailing lists to the new domain name. As a solution I added a new domain in mailman 3 and created a script that replicates the lists in this new domain. To forward the emails received by the old lists, the script removes the members and adds the email of the new list. Is there a fashionable way to do that? Also, one problem that I don't have a solution is the archive of the old lists. How to bring them to the new ones? Thanks for your help. My mailing list server is running on a CenOS-7x with postgreSQL and the following mailman packages: Package Version ------------------- ---------- 1 aiosmtpd 1.2.4 2 alembic 1.5.3 3 arrow 0.17.0 4 asgiref 3.3.1 5 atpublic 2.1.2 6 authheaders 0.13.0 7 authres 1.2.0 8 bleach 3.3.0 9 blessed 1.17.12 10 certifi 2020.12.5 11 cffi 1.14.4 12 chardet 4.0.0 13 click 7.1.2 14 cmarkgfm 0.5.2 15 cryptography 3.3.1 16 decorator 4.4.2 17 defusedxml 0.6.0 18 Django 3.0.12 19 django-allauth 0.44.0 20 django-appconf 1.0.4 21 django-compressor 2.4 22 django-extensions 3.1.0 23 django-gravatar2 1.4.4 24 django-haystack 3.0 25 django-mailman3 1.3.5 26 django-picklefield 3.0.1 27 django-q 1.3.4 28 djangorestframework 3.12.2 29 dkimpy 1.0.5 30 dnspython 2.1.0 31 docutils 0.16 32 falcon 2.0.0 33 flufl.bounce 3.0.1 34 flufl.i18n 2.0.2 35 flufl.lock 3.2 36 gunicorn 20.0.4 37 HyperKitty 1.3.3 38 idna 2.10 39 importlib-resources 5.1.0 40 lazr.config 2.2.3 41 lazr.delegates 2.0.4 42 mailman 3.3.2 43 mailman-hyperkitty 1.1.0 44 mailman-web 0.0.3 45 mailmanclient 3.3.2 46 Mako 1.1.4 47 MarkupSafe 1.1.1 48 networkx 2.5 49 nose 1.3.7 50 oauthlib 3.1.0 51 packaging 20.9 52 passlib 1.7.4 53 pip 21.0.1 54 postorius 1.3.3 55 psycopg2-binary 2.8.6 56 publicsuffix2 2.20191221 57 pycparser 2.20 58 Pygments 2.7.4 59 PyJWT 2.0.1 60 pyparsing 2.4.7 61 python-dateutil 2.8.1 62 python-editor 1.0.4 63 python3-openid 3.2.0 64 pytz 2021.1 65 rcssmin 1.0.6 66 readme-renderer 28.0 67 requests 2.25.1 68 requests-oauthlib 1.3.0 69 rjsmin 1.1.0 70 robot-detection 0.4 71 setuptools 39.2.0 72 six 1.15.0 73 SQLAlchemy 1.3.23 74 sqlparse 0.4.1 75 typing-extensions 3.7.4.3 76 urllib3 1.26.3 77 uWSGI 2.0.19.1 78 wcwidth 0.2.5 79 webencodings 0.5.1 80 wheel 0.36.2 81 Whoosh 2.7.4 82 zipp 3.4.0 83 zope.component 4.6.2 84 zope.configuration 4.4.0 85 zope.deferredimport 4.3.1 86 zope.deprecation 4.4.0 87 zope.event 4.5.0 88 zope.hookable 5.0.1 89 zope.i18nmessageid 5.0.1 90 zope.interface 5.2.0 91 zope.proxy 4.3.5 92 zope.schema 6.0.1 -- Carlos Souza

2 1

Issues with a DMARC record leading to message being shunted
by Ralf Hildebrandt 13 Jan '24

13 Jan '24

Hi! Recently a mail was not being distributed via our MM3 instance; I searched the logs and found: Dec 22 04:54:27 2023 (24) ACCEPT: <1d431542-b386-467b-8bf2-305f19ee7eb4(a)somenet.org> Dec 22 04:54:28 2023 (28) Uncaught runner exception: list index out of range Dec 22 04:54:28 2023 (28) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/lib/python3.10/site-packages/mailman/core/pipelines.py", line 53, in process handler.process(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 125, in process authenticate(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/utilities/retry.py", line 44, in f_retry return f(*args, **kwargs) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 93, in authenticate auth_result = authenticate_message( File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 395, in authenticate_message dmarc_result = check_dmarc(msg, spf_result, dkim_result, dnsfunc=dnsfunc, psddmarc=psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 343, in check_dmarc result, result_comment, from_domain, policy = dmarc_per_from(from_domain, spf_result, dkim_result, dnsfunc, psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 90, in dmarc_per_from record, orgdomain = receiver_record(from_domain) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 117, in receiver_record retval = lookup_receiver_record(hostSansDmarc, dnsfunc) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 92, in lookup_receiver_record tags = answer_to_dict(str(result)) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in answer_to_dict retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in <dictcomp> retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} IndexError: list index out of range Dec 22 04:54:28 2023 (28) SHUNTING: 1703220868.1418982+b27f5db154375fcd2c418d467172e4aad0d3d57a So the message has been shunted, due to this error. Something with DMARC and tags. So I checked the DMARC record for the sending domain somenet.org. It is "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;mailto:postmaster@somenet.org;ri=3600;fo=1;" which is syntactically incorrect. The extra ";mailto:postmaster@somenet.org" is wrong, I guess the tag "ruf=" is missing here! It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;" So, should mm3 somehow "catch" this error somehow? According to the DMARC specs, an syntactically incorrect DMARC record should be ignored / handled as if none were present. -- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin Tel. +49 30 450 570 155 ralf.hildebrandt(a)charite.de https://www.charite.de

3 6

Postorius and some CSS issue?
by Odhiambo Washington 12 Jan '24

12 Jan '24

Happy New Year to you too all. MM3 installed today using the virtualenv method. On my testbench, I am having this strange issue where the Postorius is not displaying correctly perhaps because of some CSS issue - or just my own hangovers! I have used the standard method as documented. The error I get: https://pastebin.ubuntu.com/p/dhvsHFG4jH/ -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]

2 1

Loops back to myself
by David Krantz 12 Jan '24

12 Jan '24

Hi, I have a problem with my Mailman 3-setup. The symptom is that postfix rejects non-existing addresses for my list domains with <adr(a)example.com>: mail for example.com loops back to myself What I guess happens happens is that postfix for some reason accepts the mail as it is for a relay domain and finds that it is not deliverable locally as the address does not exist locally and then tries to relay it on to the appropriate MX server, which is itself and then fails and generates a bounce message. This is a bit annoying as I have set up postfix to only accept valid addresses. I have set the transport_maps, local_recipient_maps and relay_domains to the files generated by MM3. I run an older version of MM3 (3.0) but prefer to fix this without upgrading as upgrading is a relatively big undertaking. This is mostly a backscatter problem for spam I guess. Any ideas of very simple and easy to forget things that I've probably missed? Cheers // David

5 6

Review requested for some merge requests
by Stephen J. Turnbull 10 Jan '24

10 Jan '24

Reply-To set to Mailman Developers. Please check that your replies go there. Or you can comment on MRs directly if you have a GitLab login. I have three merge requests pending: [1] https://gitlab.com/mailman/mailman/-/merge_requests/1159 [2] https://gitlab.com/mailman/mailman/-/merge_requests/1160 [3] https://gitlab.com/mailman/mailman/-/merge_requests/1161 [1] fixes the logging initialization bug that prevented the `smtp` log from rotating properly, and maybe some others. I don't think it needs user review because it's a pure bugfix, but if you're curious and take a look, and you think it needs better documentation let me know. [2] adds a configuration variable `lmtp_max_size`. This is user-visible, so review of the documentation in docs/mta.rst and config/mailman.cfg for clarity and readability would be appreciated. [3] adds postfix-pgsql support for use of the PostgreSQL database to route Mailman traffic. It's really most useful in an edge case (migration of 10,000+ list Mailman 2 instances), but it's theoretically attractive on the principle of "single source of truth", and possibly for implementing a "microservices" style cloud instance (it eliminates the need for Mailman and Postfix to share a file system). Besides opinions on the value of the general idea I have two specific questions: - Postfix docs recommend not looking up domains in the database to reduce load on the database server. But on a dedicated Mailman system, you're going to be looking up pretty much everything twice anyway (once to check for local recipient and once to fetch the transport), and my experience so far is that this isn't a big load on Postgres, and it's not a drag on list traffic distribution. Should I implement domain lookup in Postgres? Should it be optional if the lists are being looked up in Postgres? - The current patch completely disables generation of the postfix_lmtp alias file, even for the `mailman aliases` command. Should I implement an option to force generation of the files for documentation (and maybe a backup in a very rare and contorted situation where Mailman can see Postgres but Postfix can't)? Thanks, Steve

3 3

Mailman 3 Core user assistance
by Tony Deshields 10 Jan '24

10 Jan '24

I currently have a docker mailman stack that utilizes Django LDAP backend for authentication. I have my users attributes mapped as you see below in my settings.py file. Whenever a user logs into postorius with their 'uid', mailman-core uses their 'uid' as their email address instead of the 'email' attribute. Is there anyway to change this functionality? I would like my users to be able to login with their enterprise uid but be mapped to their email but I feel like there is a disconnect between Django and MM3 core. Any assistance is much appreciated. AUTH_LDAP_USER_ATTR_MAP = { "uid" : "userPrincipalName", "first_name": "givenName", "last_name": "sn", "email" : "mail", }

2 1