- Mailman-users - lists.mailman3.org

Weird startup problem after upgrade
by Odhiambo Washington 03 Dec '22

03 Dec '22

I am facing an odd situation in which MM3 doesn't start successfully on reboot, but I can start it manually. This is a venv install. This started after I did an upgrade (pip install --upgrade mailman postorius hyperkitty mailman-hyperkitty) followed by the requisite collectstatic|compress|compilemessages|migrate post-update commands. Secondly, it appears that the upgrade did NOT change the versions for the upgraded components - postorius=1.3.6, hyperkitty=1.3.5, Mailman Core Version=3.3.5. I was expecting the version to all bump up to the latest. Thanks in advance for any insights into what could be causing this. wash@lists:~$ systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: failed (Result: timeout) since Fri 2022-12-02 18:58:13 EAT; 9s ago CPU: 25.241s Dec 02 18:58:12 lists mailman[1637]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1639]: for atexit_call in reversed(_threading_atexits): Dec 02 18:58:12 lists mailman[1639]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1637]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1639]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1637]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:12 lists mailman[1639]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:13 lists systemd[1]: mailman3.service: Failed with result 'timeout'. Dec 02 18:58:13 lists systemd[1]: Failed to start GNU Mailing List Manager. Dec 02 18:58:13 lists systemd[1]: mailman3.service: Consumed 25.241s CPU time. root@lists:/home/wash# systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: *failed (Result: timeout) since Fri 2022-12-02 18:58:13 EAT; 53s ago* CPU: 25.241s Dec 02 18:58:12 lists mailman[1637]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1639]: for atexit_call in reversed(_threading_atexits): Dec 02 18:58:12 lists mailman[1639]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1637]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1639]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1637]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:12 lists mailman[1639]: mailman.interfaces.runner.RunnerInterrupt: *Dec 02 18:58:13 lists systemd[1]: mailman3.service: Failed with result 'timeout'.Dec 02 18:58:13 lists systemd[1]: Failed to start GNU Mailing List Manager.Dec 02 18:58:13 lists systemd[1]: mailman3.service: Consumed 25.241s CPU time.* root@lists:/home/wash# cd /opt/mailman/mm/var/logs/ root@lists:/opt/mailman/mm/var/logs# ls -al total 104948 drwxr-xr-x 2 mailman mailman 4096 Dec 2 18:56 . drwxr-xr-x 12 mailman mailman 4096 Dec 2 18:58 .. -rw-r--r-- 1 mailman mailman 7720631 Dec 2 18:52 access.log -rw-rw---- 1 mailman mailman 1514 Nov 28 05:58 bounce.log -rw-rw---- 1 mailman mailman 0 Aug 15 16:56 debug.log -rw-r--r-- 1 mailman mailman 497 Dec 2 18:56 error.log -rw-rw---- 1 mailman mailman 99339919 Dec 2 18:58 mailman.log -rw-r--r-- 1 mailman mailman 0 Dec 2 18:53 mailmansuite.log -rw-rw---- 1 mailman mailman 0 Aug 15 16:56 plugins.log -rw-rw---- 1 mailman mailman 374961 Dec 2 18:56 smtp.log root@lists:/opt/mailman/mm/var/logs# less error.log root@lists:/opt/mailman/mm/var/logs# cd ../locks/ root@lists:/opt/mailman/mm/var/locks# ls -al total 8 drwxr-xr-x 2 mailman mailman 4096 Dec 2 18:58 . drwxr-xr-x 12 mailman mailman 4096 Dec 2 18:58 .. root@lists:/opt/mailman/mm/var/locks# systemctl start mailman3 root@lists:/opt/mailman/mm/var/locks# systemctl status mailman3 ● mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2022-12-02 19:00:15 EAT; 11s ago Process: 2395 ExecStart=/opt/mailman/mm/bin/mailman start (code=exited, status=0/SUCCESS) Main PID: 2397 (python3) Tasks: 18 (limit: 4550) Memory: 791.9M CPU: 21.440s CGroup: /system.slice/mailman3.service ├─2397 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/master -C /opt/mailman/mm/mailman.cfg ├─2398 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=archive:0:1 ├─2399 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=bounces:0:1 ├─2400 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=command:0:1 ├─2401 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=in:0:1 ├─2402 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=lmtp:0:1 ├─2403 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=nntp:0:1 ├─2404 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=out:0:1 ├─2405 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=pipeline:0:1 ├─2406 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 ├─2407 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=retry:0:1 ├─2408 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=task:0:1 ├─2409 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=virgin:0:1 ├─2410 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=digest:0:1 ├─2423 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 └─2424 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 Dec 02 19:00:13 lists systemd[1]: Starting GNU Mailing List Manager... Dec 02 19:00:14 lists mailman[2395]: Starting Mailman's master runner Dec 02 19:00:14 lists mailman[2395]: Generating MTA alias maps *Dec 02 19:00:14 lists systemd[1]: mailman3.service: Can't open PID file /opt/mailman/mm/var/master.pid (yet?) after start: Operation not permitted* Dec 02 19:00:15 lists systemd[1]: Started GNU Mailing List Manager. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 4

Permission denied for fulltext_index
by Jan Eden 01 Dec '22

01 Dec '22

Hi, when trying to reattach a message thread in the Hyperkitty web interface, the following error occurs: > Internal Server Error: /archives/list/testlist(a)lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > > PermissionError at /archives/list/testlist(a)lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > [Errno 13] Permission denied: 'fulltext_index' The error occurs while haystack attempts to create a directory – > File "/opt/mailman/venv/lib/python3.10/site-packages/haystack/backends/whoosh_backend.py", line 412, in search > self.setup() > File "/opt/mailman/venv/lib/python3.10/site-packages/haystack/backends/whoosh_backend.py", line 133, in setup > os.makedirs(self.path) > File "/usr/lib/python3.10/os.py", line 225, in makedirs > mkdir(name, mode) > > Exception Type: PermissionError at /archives/list/testlist(a)lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > Exception Value: [Errno 13] Permission denied: 'fulltext_index' > Raised during: hyperkitty.views.thread.reattach_suggest > Request information: > USER: mailman – but the existing directory /opt/mailman/fulltext_index (and its contents) are readable and writeable by mailman:mailman: > drwxrwxr-x 2 mailman mailman 4096 Nov 27 08:00 fulltext_index/ > > -rw-rw-r-- 1 mailman mailman 4473 Nov 27 08:00 _MAIN_65.toc > -rw-rw-r-- 1 mailman mailman 15456 Nov 27 08:00 MAIN_bc0cs2aohovftat6.seg > -rw-rw-r-- 1 mailman mailman 15454 Nov 27 06:00 MAIN_brbo8i8edmlz0za4.seg > -rw-rw-r-- 1 mailman mailman 15462 Nov 27 07:00 MAIN_ewnemc7lsnodd3y7.seg > -rw-rw-r-- 1 mailman mailman 157324 Nov 27 04:00 MAIN_qag60g1rrumshtc2.seg > -rw-rw-r-- 1 mailman mailman 15459 Nov 27 05:00 MAIN_w6e0ohurasusy61d.seg > -rwxrwxr-x 1 mailman mailman 0 Nov 24 16:00 MAIN_WRITELOCK* I fail to see where haystack tries (and fails) to create the directory. FWIW, deleting threads is not a problem. Thanks, Jan

2 10

Mailman web interface / uwsgi
by Jan Eden 28 Nov '22

28 Nov '22

Hi, I followed the installation instructions at https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst…, which thankfully shielded me from configuring the Django applications for the web interface manually. All Django applications (postorius, hyperkitty, haystack etc) are loaded via mailman_web/settings/base.py, which is in turn referenced by the top-level settings file in /etc/mailman3. uwsgi is linked to mailman_web via /etc/mailman3/uwsgi.ini. So far, so good. I do not want to dive into the interaction between the various components and just want to switch from a http socket (http-socket = 0.0.0.0:8000) to a file socket (socket = /opt/mailman/mm/var/mailman.sock). After making this change in /etc/mailman3/uwsgi.ini, I applied it in the nginx proxy configuration proxy_pass http://127.0.0.1:8000; → proxy_pass http://unix:/opt/mailman/mm/var/mailman.sock; and received an error message (as expected): 2022/11/25 18:39:12 [error] 2978606#2978606: *3 upstream prematurely closed connection while reading response header from upstream, client: 37.201.155.65, server: lists.eden.one, request: "GET /mailman3/lists/ HTTP/2.0", upstream: "http://unix:/opt/mailman/mm/var/mailman.sock:/mailman3/lists/", host: "lists.eden.one" There are obviously a couple of places where the http-socket on port 8000 is referenced (e.g. mailman_web/settings/mailman.py, /opt/mailman/mm/hyperkitty.cfg). Is there an overview of the required changes when using a different (file) socket? Related question: My own (much simpler) Django applications use the following (e.g. in mysite_nginx.conf): upstream django { server unix:///private/tmp/mysite.sock; } location / { uwsgi_pass django; } instead of the "http://unix:/..." notation I tried for Mailman above. Is there any difference between the upstream/uwsgi_pass combination and the proxy_pass variant? Thanks, Jan

3 8

Default list styles
by Jan Eden 27 Nov '22

27 Nov '22

Hi, the list creation web interface offers three list styles (announce, ordinary, ordinary w/ private archive). When selecting the announce style, I would expect a different default message acceptance setting for members (i.e. reject), because this is how an announce list should work. In practice, a list created with the announce style is configured just as an ordinary mailing list (as far as I can see in the web interface, and according to the handling of a test message sent by a regular member of an announce list). What am I missing here? - Jan

2 1

Mailman3 not able to deliver emails sent through list, but moderation notifications
by Daniel Böhmer 27 Nov '22

27 Nov '22

Hello Mailman users, I've been using Mailman2 for 10+ years and have recently upgraded to Debian Buster with Mailman3. I've tried to setup Mailman3 similarly but it fails to deliver emails sent through a list. I do receive all moderation notifications at the same time. During startup /var/log/mailman3/mailman.log receives: > Cannot connect to SMTP server localhost on port 25 but I've made sure postfix is listening on *:25 for both IPv4 and IPv6. When sending a message to a list the same file receives: > Nov 25 11:38:14 2022 (70371) ACCEPT: <MESSAGEID@HOST> (message ID and host redacted) and nothing else happens: no more log messages, Postfix logs the email sent TO Mailman but doesn't log anything about emails sent BY Mailman. If Mailman fails to send emails via Postfix I'd expect it to log error messages but it doesn't seem to do so. I found this article stating the the SMTP error message might be misleading: https://blog.tfiu.de/mailman3-cannot-connect-to-smtp-server-localhost-on-po… How to further debug this? What could be wrong with my Mailman3 instance and how to fix it? Kind regards Daniel Böhmer

3 5

MTA authentication and smtpd_sender_login_maps
by Jan Eden 25 Nov '22

25 Nov '22

Hi, I installed Mailman3 successfully and would like to change the default smtp settings to an authenticated connection (smtp_user, smtp_pass). But my Postfix configuration restricts sending messages via the smtpd_sender_login_maps directive (which points to a hashed file I created years ago) and correctly refuses connections for messages with the test mailing list's From header. I obviously do not want to add an entry to the map file for each mailing list created by someone on the server. Is there a way to programmatically create a sender login map with Mailman? Thanks, Jan

2 2

Hyperkitty no longer archiving
by Hirayama, Pat 23 Nov '22

23 Nov '22

Greetings, A couple of months ago, I needed to move the Mailman3 server I inherited from one AWS account to another AWS account. This necessitated a change in the IP address of instance hosting the docker containers that comprise my mailman3 server. I thought I had updated the IP address wherever it needed to be, but I suppose I've missed something, because hyperkitty has not been archiving any messages. (Most of my lists don't archive, so it took a while before someone noticed). When I look at /opt/mailman/var/archives/hyperkitty/spool, I see messages dating back over the past couple of months waiting to be archived. So, they don't appear to be lost (and things are being mailed out normally). The thing is, the IP addresses of the docker containers haven't changed with respect to docker networking. docker networking ------------------------ docker host 172.31.0.1 mailman-core 172.31.0.3 mailman-nginx 172.31.0.5 mailman-postfix 172.31.0.4 mailman-django-uwsgi 172.31.0.2 settings_local.py: MAILMAN_ARCHIVER_FROM = ('172.18.8.14', '172.31.0.1', '172.31.0.2', '172.31.0.3', '172.31.0.4', '172.31.0.5', '127.0.0.1', '::1') During troubleshooting, I added 172.18.8.14 (new IP of the webserver), but that didn't seem to help. It seems like I added one of the docker addresses that was missing .... but again, that doesn't seem to have fixed the issue. mailmansuite.log is full of: ERROR 2022-11-22 00:25:40,365 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured ERROR 2022-11-22 00:25:40,399 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured Any suggestions as to where I might look for a solution, or what setting I need to change? Thanks! -p Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam(a)fredhutch.org<mailto:phirayam@fredhutch.org>

4 12

Upgrading a Ubuntu Mailman 3 Installation
by Allan Hansen 21 Nov '22

21 Nov '22

Dear Mailman 3 wizards, Several years ago I tried upgrading to Mailman 3 (I did install Mailman 2), but quickly ran into a thicket too complex for me to get out of. Two others - way more experienced than me at system-level manipulations - tried as well but did not succeed. Brian Carpenter of EMWD installed the system seemingly effortlessly, and seemingly overnight. We had a good relationship, and I was a beta site for his new product, SAAS. Brian also did the system MM3 upgrades as they appeared and would check that all was well a couple of times a year. Unfortunately, and I respect that, the current management at EMWD "don't provide working hands service for foreign Mailman 3 installations." I do not dare touch it, based on my earlier attempts with the installation. With Brian's passing I'm thus faced with a system that has no capable system support (I manage the lists, not the core). I have 50+ lists with 10000+ subscription total, and a lot of people depend on this tool. (It's a non-profit organization). So, I'm interesting in hiring someone for part-time, occasional work (a few times a year). Update MM3 as new versions are made available. Diagnose and fix the system if anything goes awry. I will pay you, as I did Brian, out of my own pocket, for your time. Yours, Allan Hansen

2 1

Migrating to mailman-web and Ubuntu 22.04
by Andrew Hodgson 16 Nov '22

16 Nov '22

Hi. For various reasons I want to migrate to a larger host running Ubuntu 22.04. I have set up the various packages ok and as part of the migration I want to use mailman-web instead of mailman-suite which is an older setup. I have instructions for Mailman-web for setting up Uwsgi but I would prefer to keep Gunicorn working as per the instructions here: DOC/Mailman 3 installation experience - Mailman Wiki (list.org)<https://wiki.list.org/DOC/Mailman%203%20installation%20experience> Has anyone done this before? Thanks. Andrew.

3 5

New Mailman Core Release
by Mark Sapiro 15 Nov '22

15 Nov '22

We are pleased to announce the release of Mailman core 3.3.7. It is published to PyPI and you can find links to all the PyPI projects: https://docs.mailman3.org/en/latest/ The change log is at https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… - please note the `BREAKING CHANGE` documented there and due to our now using SQLAlchemy >-1.4. Finally, the minimum Python version required is 3.7 although, 3.9 is recommended if you are doing a new install for longest support. 3.10 is supported for Django apps, but Mailman Core is missing that due to one of the core dependencies (aiosmtpd) not supporting 3.10 yet. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0