Weird startup problem after upgrade
I am facing an odd situation in which MM3 doesn't start successfully on reboot, but I can start it manually. This is a venv install. This started after I did an upgrade (pip install --upgrade mailman postorius hyperkitty mailman-hyperkitty) followed by the requisite collectstatic|compress|compilemessages|migrate post-update commands.
Secondly, it appears that the upgrade did NOT change the versions for the upgraded components - postorius=1.3.6, hyperkitty=1.3.5, Mailman Core Version=3.3.5. I was expecting the version to all bump up to the latest.
Thanks in advance for any insights into what could be causing this.
wash@lists:~$ systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: failed (Result: timeout) since Fri 2022-12-02 18:58:13 EAT; 9s ago CPU: 25.241s
Dec 02 18:58:12 lists mailman[1637]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1639]: for atexit_call in reversed(_threading_atexits): Dec 02 18:58:12 lists mailman[1639]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1637]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1639]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1637]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:12 lists mailman[1639]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:13 lists systemd[1]: mailman3.service: Failed with result 'timeout'. Dec 02 18:58:13 lists systemd[1]: Failed to start GNU Mailing List Manager. Dec 02 18:58:13 lists systemd[1]: mailman3.service: Consumed 25.241s CPU time.
root@lists:/home/wash# systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: *failed (Result: timeout) since Fri 2022-12-02 18:58:13 EAT; 53s ago* CPU: 25.241s
Dec 02 18:58:12 lists mailman[1637]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1639]: for atexit_call in reversed(_threading_atexits): Dec 02 18:58:12 lists mailman[1639]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1637]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1639]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1637]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:12 lists mailman[1639]: mailman.interfaces.runner.RunnerInterrupt:
*Dec 02 18:58:13 lists systemd[1]: mailman3.service: Failed with result 'timeout'.Dec 02 18:58:13 lists systemd[1]: Failed to start GNU Mailing List Manager.Dec 02 18:58:13 lists systemd[1]: mailman3.service: Consumed 25.241s CPU time.* root@lists:/home/wash# cd /opt/mailman/mm/var/logs/ root@lists:/opt/mailman/mm/var/logs# ls -al total 104948 drwxr-xr-x 2 mailman mailman 4096 Dec 2 18:56 . drwxr-xr-x 12 mailman mailman 4096 Dec 2 18:58 .. -rw-r--r-- 1 mailman mailman 7720631 Dec 2 18:52 access.log -rw-rw---- 1 mailman mailman 1514 Nov 28 05:58 bounce.log -rw-rw---- 1 mailman mailman 0 Aug 15 16:56 debug.log -rw-r--r-- 1 mailman mailman 497 Dec 2 18:56 error.log -rw-rw---- 1 mailman mailman 99339919 Dec 2 18:58 mailman.log -rw-r--r-- 1 mailman mailman 0 Dec 2 18:53 mailmansuite.log -rw-rw---- 1 mailman mailman 0 Aug 15 16:56 plugins.log -rw-rw---- 1 mailman mailman 374961 Dec 2 18:56 smtp.log root@lists:/opt/mailman/mm/var/logs# less error.log root@lists:/opt/mailman/mm/var/logs# cd ../locks/ root@lists:/opt/mailman/mm/var/locks# ls -al total 8 drwxr-xr-x 2 mailman mailman 4096 Dec 2 18:58 . drwxr-xr-x 12 mailman mailman 4096 Dec 2 18:58 .. root@lists:/opt/mailman/mm/var/locks# systemctl start mailman3 root@lists:/opt/mailman/mm/var/locks# systemctl status mailman3 ● mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2022-12-02 19:00:15 EAT; 11s ago Process: 2395 ExecStart=/opt/mailman/mm/bin/mailman start (code=exited, status=0/SUCCESS) Main PID: 2397 (python3) Tasks: 18 (limit: 4550) Memory: 791.9M CPU: 21.440s CGroup: /system.slice/mailman3.service ├─2397 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/master -C /opt/mailman/mm/mailman.cfg ├─2398 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=archive:0:1 ├─2399 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=bounces:0:1 ├─2400 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=command:0:1 ├─2401 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=in:0:1 ├─2402 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=lmtp:0:1 ├─2403 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=nntp:0:1 ├─2404 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=out:0:1 ├─2405 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=pipeline:0:1 ├─2406 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 ├─2407 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=retry:0:1 ├─2408 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=task:0:1 ├─2409 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=virgin:0:1 ├─2410 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=digest:0:1 ├─2423 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 └─2424 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1
Dec 02 19:00:13 lists systemd[1]: Starting GNU Mailing List Manager... Dec 02 19:00:14 lists mailman[2395]: Starting Mailman's master runner Dec 02 19:00:14 lists mailman[2395]: Generating MTA alias maps *Dec 02 19:00:14 lists systemd[1]: mailman3.service: Can't open PID file /opt/mailman/mm/var/master.pid (yet?) after start: Operation not permitted* Dec 02 19:00:15 lists systemd[1]: Started GNU Mailing List Manager.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 12/2/22 08:21, Odhiambo Washington wrote:
I am facing an odd situation in which MM3 doesn't start successfully on reboot, but I can start it manually. This is a venv install. This started after I did an upgrade (pip install --upgrade mailman postorius hyperkitty mailman-hyperkitty) followed by the requisite collectstatic|compress|compilemessages|migrate post-update commands.
Was the venv activated when you ran pip?
You should also include django-mailman3 and mailmanclient in the packages to be upgraded although they are dependencies and "should" upgrade anyway.
Also, at least one person has noted that --force-reinstall was required to get all dependencies upgraded <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/OLISJTCOMZKBHDCRR3XET7E5RZIQCMYI/>
Secondly, it appears that the upgrade did NOT change the versions for the upgraded components - postorius=1.3.6, hyperkitty=1.3.5, Mailman Core Version=3.3.5. I was expecting the version to all bump up to the latest.
What does pip freeze in the venv show for versions?
Thanks in advance for any insights into what could be causing this.
How do you start it manually? mailman start or systemctl start mailman3? What's the content of /etc/systemd/system/mailman3.service?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, Dec 2, 2022 at 8:58 PM Mark Sapiro <mark@msapiro.net> wrote:
On 12/2/22 08:21, Odhiambo Washington wrote:
I am facing an odd situation in which MM3 doesn't start successfully on reboot, but I can start it manually. This is a venv install. This started after I did an upgrade (pip install --upgrade mailman postorius hyperkitty mailman-hyperkitty) followed by the requisite collectstatic|compress|compilemessages|migrate post-update commands.
Was the venv activated when you ran pip?
Yes.
You should also include django-mailman3 and mailmanclient in the packages to be upgraded although they are dependencies and "should" upgrade anyway.
I did not include them in the list of packages to be upgraded because I did not explicitly install them when I installed my server. In BC's HOWTO <https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10> which is what my MM3 server is based, they were never part of the modules (app?) installed explicitly.
Also, at least one person has noted that --force-reinstall was required to get all dependencies upgraded < https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
I can try this as well and report back.
Secondly, it appears that the upgrade did NOT change the versions for the upgraded components - postorius=1.3.6, hyperkitty=1.3.5, Mailman Core Version=3.3.5. I was expecting the version to all bump up to the latest.
What does
pip freezein the venv show for versions?
Here is the listing: https://pastebin.ubuntu.com/p/tMGDxjKwJ8/
Thanks in advance for any insights into what could be causing this.
How do you start it manually?
mailman startorsystemctl start mailman3?
systemctl start mailman3
What's the content of /etc/systemd/system/mailman3.service?
root@lists:/home/wash# cat /etc/systemd/system/mailman3.service [Unit] Description=GNU Mailing List Manager After=syslog.target network.target mysql.service postgresql.service
[Service] Type=forking PIDFile=/opt/mailman/mm/var/master.pid User=mailman Group=mailman ExecStart=/opt/mailman/mm/bin/mailman start ExecReload=/opt/mailman/mm/bin/mailman restart ExecStop=/opt/mailman/mm/bin/mailman stop
[Install] WantedBy=multi-user.target
A special note that before I did this upgrade, the startup wasn't an issue. I can actually go back to where I was, because I have complete backup of my venv before the update, but it would be great to figure out why things are the way they are now.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 12/3/22 00:03, Odhiambo Washington wrote:
On Fri, Dec 2, 2022 at 8:58 PM Mark Sapiro <mark@msapiro.net> wrote:
How do you start it manually?
mailman startorsystemctl start mailman3?systemctl start mailman3
Presumably you do this as root or via sudo so I don't understand
what would be different between doing it manually vs. via systemd at
startup. I can't explain that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 12/2/22 08:21, Odhiambo Washington wrote:
Secondly, it appears that the upgrade did NOT change the versions for the upgraded components - postorius=1.3.6, hyperkitty=1.3.5, Mailman Core Version=3.3.5. I was expecting the version to all bump up to the latest.
Your pip freeze at https://pastebin.ubuntu.com/p/tMGDxjKwJ8/ shows
django-mailman3==1.3.8 HyperKitty==1.3.6 mailman==3.3.7 mailman-hyperkitty==1.2.0 mailmanclient==3.3.4 postorius==1.3.7
which are the latest. Perhaps you have older versions installed outside the venv and perhaps this confusion is the cause of your issue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro -
Odhiambo Washington