September 2024 - Mailman-users - lists.mailman3.org

Issues with a DMARC record leading to message being shunted
by Ralf Hildebrandt March 4, 2025

March 4, 2025

Hi! Recently a mail was not being distributed via our MM3 instance; I searched the logs and found: Dec 22 04:54:27 2023 (24) ACCEPT: <1d431542-b386-467b-8bf2-305f19ee7eb4(a)somenet.org> Dec 22 04:54:28 2023 (28) Uncaught runner exception: list index out of range Dec 22 04:54:28 2023 (28) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/lib/python3.10/site-packages/mailman/core/pipelines.py", line 53, in process handler.process(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 125, in process authenticate(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/utilities/retry.py", line 44, in f_retry return f(*args, **kwargs) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 93, in authenticate auth_result = authenticate_message( File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 395, in authenticate_message dmarc_result = check_dmarc(msg, spf_result, dkim_result, dnsfunc=dnsfunc, psddmarc=psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 343, in check_dmarc result, result_comment, from_domain, policy = dmarc_per_from(from_domain, spf_result, dkim_result, dnsfunc, psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 90, in dmarc_per_from record, orgdomain = receiver_record(from_domain) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 117, in receiver_record retval = lookup_receiver_record(hostSansDmarc, dnsfunc) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 92, in lookup_receiver_record tags = answer_to_dict(str(result)) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in answer_to_dict retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in <dictcomp> retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} IndexError: list index out of range Dec 22 04:54:28 2023 (28) SHUNTING: 1703220868.1418982+b27f5db154375fcd2c418d467172e4aad0d3d57a So the message has been shunted, due to this error. Something with DMARC and tags. So I checked the DMARC record for the sending domain somenet.org. It is "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;mailto:postmaster@somenet.org;ri=3600;fo=1;" which is syntactically incorrect. The extra ";mailto:postmaster@somenet.org" is wrong, I guess the tag "ruf=" is missing here! It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;" So, should mm3 somehow "catch" this error somehow? According to the DMARC specs, an syntactically incorrect DMARC record should be ignored / handled as if none were present. -- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin Tel. +49 30 450 570 155 ralf.hildebrandt(a)charite.de https://www.charite.de

5 12

How To Install Mailman 3 on Debian 10 (Complete Guide)
by Brian Carpenter Feb. 25, 2025

Feb. 25, 2025

I have finished my how-to install Mailman 3 on Debian 10 completed to the point where you can use it to get a fully functional Mailman 3 environment up and running. This is a comprehensive guide that walks you through installing EVERYTHING (currently!) that Mailman 3 needs to run. It also makes updating Mailman 3/Postorius/Hyperkitty very easy. I have not added the update directions yet but will very soon. I also plan on adding a section on setting up DKIM and Xapian with Mailman 3. I hope this helps those who wish to get a Mailman 3 server up and running quickly and brings a sense of sanity to the confusion regarding the various documentation out there concerning the installation of Mailman 3/Postorius/Hyperkitty. I intend to add a How-to for Ubuntu 20 at some point. Without further ado: https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 I am sure the above guide can use improvements. Feel free to contact me off-list to point out mistakes/improvements, etc. -- Brian Carpenter Harmonylists.com Emwd.com

13 38

Find a smtp server to send out emails
by Leon Jan. 15, 2025

Jan. 15, 2025

Hi, This problem is not caused by mailman, but I still want to give it a shot here. I'm hosting a mailing list on top of mailman3. Emails are supposed to be sent out by AWS ses. However, ses requires that sender address must be verified, which leads to a problem that emails sent by subscribers to mailing list cannot be sent to other subcirbers, since their addresses are not verified. And it is impossible to verified every subscriber. Are there smtp service providers allowing unverified email address to send out emails, or do I have other solutions? Thanks. -- Best wishes, Leon

3 2

How to remove Held Unsubscriptions
by Mark Jan. 6, 2025

Jan. 6, 2025

Hi As a moderator I receive an email each day for "Held Unsubscriptions" for a subscriber that was unsubscribed by the Owner sometime ago (using "Mass operations"). +++++++++++++++++++++++++++++++ The listname(a)domain.com list has 1 moderation requests waiting. Held Unsubscriptions: User: bart(a)example.com Please attend to this at your earliest convenience. +++++++++++++++++++++++++++++++ There is nothing showing under "Subscription requests > Pending approval". Could anyone tell me how I can stop these email notifications? I have access to the mailman shell and the databases. Thanks, Mark

4 7

Cache mailman3
by pau.baiges＠csuc.cat Dec. 22, 2024

Dec. 22, 2024

Is there a way to delete the mailman3 cache? It's because it gives us this error and we think it's because of the cache. Internal Server Error: /mailman3/accounts/login/ TemplateDoesNotExist at /accounts/login/ django_saml2_auth/error.html Request Method: GET Request URL: http://llistes.csuc.cat/mailman3/accounts/login/?next=/mailman3/postorius/l… Django Version: 2.2.12 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.10 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: dv., 15 Març 2024 10:33:00 +0100 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Template loader postmortem Django tried loading these templates, in this order: Using engine django: This engine did not provide a list of tried templates. Traceback: File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/utils.py" in wrapper 179. result = function(request) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/views.py" in signin 271. saml_client = get_saml_client(get_assertion_url(request), acs) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/saml.py" in get_saml_client 169. metadata = get_metadata(user_id) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/saml.py" in get_metadata 131. raise SAMLAuthError("Invalid metadata URL.", extra={ During handling of the above exception (Invalid metadata URL.), another exception occurred: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/utils.py" in wrapper 181. result = handle_exception(exc, request) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/utils.py" in handle_exception 162. return render(request, File "/usr/lib/python3/dist-packages/django/shortcuts.py" in render 36. content = loader.render_to_string(template_name, context, request, using=using) File "/usr/lib/python3/dist-packages/django/template/loader.py" in render_to_string 61. template = get_template(template_name, using=using) File "/usr/lib/python3/dist-packages/django/template/loader.py" in get_template 19. raise TemplateDoesNotExist(template_name, chain=chain) Exception Type: TemplateDoesNotExist at /accounts/login/ Exception Value: django_saml2_auth/error.html Request information: USER: AnonymousUser

3 2

Re: mail stuck in "out" queue
by Dmitry Makovey Dec. 18, 2024

Dec. 18, 2024

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

6 17

How to Increase outgoing mail volume for Mailman 3
by Brian Carpenter Dec. 9, 2024

Dec. 9, 2024

Hello everyone, We had a new client come on board that brought a list that had the largest out-going mail volume that I have ever dealt with. So I did what any sane server administrator would do that is running Mailman 3: I ran to Mark Sapiro for help! Here is what the Sage of all things Mailman counseled me to do: 1. Disabled VERP delivery. VERP can help improve mail delivery by delivering 1 message per SMTP transaction (I think I got that right). Using VERP is counter-productive as it can seriously delay outgoing mail delivery for a high-volume list. So you can use something like the following in the MTA section of your mailman.cfg: [mta] verp_confirmations: no verp_personalized_deliveries: no verp_delivery_interval: 0 2. I configured Mailman 3 to run 4 outgoing runners. The default is 1. This action was the game changer. It allows Mailman to handle 4 incoming messages at a time instead of 1. My client's incoming list volume was high enough where it was overwhelming the single outgoing runner. Here is what I added to the mailman.cfg file: [runner.out] instances: 4 Doing this step immediately solved the backlog in var/queue/out. 3. If you start up with a new server for a list that is high volume then keep an eye out for Proofpoint and Vade Security as they will block your new IP address quicker than Mark Sapiro can say "This is a bug. See <some Gitlab.com url>" Mark, thank you! -- Brian Carpenter Harmonylists.com Emwd.com

5 10

Aborted connection message in log in every 5 minutes
by zoltan＠circle-interactive.co.uk Dec. 3, 2024

Dec. 3, 2024

Hi Everyone, we have mailman 3.3.8 venv test installation which in general works fine however our mysql error log is filled up with messages below because they are so frequent. 2023-02-24 0:31:30 199567 [Warning] Aborted connection 199567 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:36:30 201063 [Warning] Aborted connection 201063 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:41:31 202559 [Warning] Aborted connection 202559 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:46:31 204056 [Warning] Aborted connection 204056 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) As suggested here https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… I tried to correlate with error logs and /opt/mailman/web/logs/mailmanweb.log shows some hyperkitty.lib.utils Failed to update the fulltext index: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (111)" /opt/mailman/web/logs/uwsgi-error.log Message: OperationalError(2013, "Lost connection to MySQL server at 'handshake: reading initial communication packet', sy stem error: 104") but they have not such frequency so unsure if they are related. Thanks for the suggestions

6 16

Moving existing lists to a new domain
by Jeremy Stanley Nov. 21, 2024

Nov. 21, 2024

I've moved lists and archives between domains in Mailman 2.1 and Pipermail, but not yet with Mailman 3 and Hyperkitty. I have a working multi-domain deployment, and one of the organizations using it is working on changing their domain name. Forwarding incoming messages at the MTA and setting up redirects on the Web server are simple enough, but internally within MM3/Django reside the bits I'm less certain about. Since all the lists for the given domain will be moving to the same unused new domain on the existing server, and can move at the same time, I suppose this could be done through (offline if necessary) backend database update queries. Is there anything outside the DB which would need updating or regenerating for this too? Is this how others have tackled similar tasks in the past, or is it recommended to export/import lists one at a time out of one domain and into another? Any tips are greatly appreciated! -- Jeremy Stanley

2 6

Migrating from old V2 server to new V3 server. Questions.
by jkoral＠luc.edu Nov. 14, 2024

Nov. 14, 2024

We have figured out how to migrate from one server to another, but after removing the list from the old server, the e-mails are still trying to go there. Is there something that needs to be done in mailman so that the e-mail list will hit the new server instead? Does it just take time? Thank you, Joe Koral

2 26