Sometimes when I reboot the machine, I experience that mailman doesn't
start because the files /opt/mailman3/mm/var/locks/master.lck* isn't
deleted during shutdown. Maybe one way to solve this could be to use
ExecStartPre directive in the systemd service to remove the lock files.
What's the best practice to solve this problem? BTW I've noticed that
doing systemctl stop mailman3 takes a very long time, 45 secs. or
something like that.
Mailman 3.3.10
Python 3.12.13
[Unit]
Description=GNU Mailing List Manager
After=syslog.target network.target postgresql.service
[Service]
Type=forking
PIDFile=/opt/mailman3/mm/var/master.pid
User=mailman3
Group=mailman3
Environment="MAILMAN_CONFIG_FILE=/etc/mailman3/mailman.cfg"
ExecStart=/opt/mailman3/venv/bin/mailman start
ExecReload=/opt/mailman3/venv/bin/mailman restart
ExecStop=/opt/mailman3/venv/bin/mailman stop
# Tested, but have no effect on deletion of the lock file.
#Restart=on-failure
#RestartSec=15
[Install]
WantedBy=multi-user.target
Lars
On 6/21/26 4:16 AM, Lars Bjørndal wrote:
Sometimes when I reboot the machine, I experience that mailman doesn't start because the files /opt/mailman3/mm/var/locks/master.lck* isn't deleted during shutdown. Maybe one way to solve this could be to use ExecStartPre directive in the systemd service to remove the lock files. What's the best practice to solve this problem?
You can add --force to the ExecStart command in your service file.
BTW I've noticed that doing
systemctl stop mailman3takes a very long time, 45 secs. or something like that.
This is because it waits for all the runners to exit and bounce runner in particular sleeps for (default) 15 seconds.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 6/21/26 8:50 AM, Mark Sapiro wrote:
This is because it waits for all the runners to exit and bounce runner in particular sleeps for (default) 15 seconds.
Actually, the default sleep time for the bounce runner is 2 minutes, and the retry runner sleeps even longer <https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/config/mailman....>.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thank you! You wrote:
On 6/21/26 4:16 AM, Lars Bjørndal wrote:
Sometimes when I reboot the machine, I experience that mailman doesn't start because the files /opt/mailman3/mm/var/locks/master.lck* isn't deleted during shutdown. Maybe one way to solve this could be to use ExecStartPre directive in the systemd service to remove the lock files. What's the best practice to solve this problem?
You can add
--forceto theExecStartcommand in your service file.
At least, with the --force option, Mailman started successfully this morning when I rebooted the machine.
[...]
Lars
Le 2026-06-21 11:16, Lars Bjørndal a écrit :
Sometimes when I reboot the machine, I experience that mailman doesn't start because the files /opt/mailman3/mm/var/locks/master.lck* isn't deleted during shutdown. Maybe one way to solve this could be to use ExecStartPre directive in the systemd service to remove the lock files. What's the best practice to solve this problem? BTW I've noticed that doing
systemctl stop mailman3takes a very long time, 45 secs. or something like that.
How about this ...
rm /opt/mailman/mm/var/locks/* systemctl start mailman3 systemctl start mailmanweb systemctl daemon-reload systemctl start qcluster
What is in the logs about why the locks remain? There should be something in the system log about it if systemd kills a service process because it doesn't stop in time during shutdown... -Gerald
On 21.06.26 13:16, Lars Bjørndal wrote:
Sometimes when I reboot the machine, I experience that mailman doesn't start because the files /opt/mailman3/mm/var/locks/master.lck* isn't deleted during shutdown. Maybe one way to solve this could be to use ExecStartPre directive in the systemd service to remove the lock files. What's the best practice to solve this problem? BTW I've noticed that doing
systemctl stop mailman3takes a very long time, 45 secs. or something like that.
Mailman 3.3.10
Python 3.12.13
[Unit] Description=GNU Mailing List Manager After=syslog.target network.target postgresql.service [Service] Type=forking PIDFile=/opt/mailman3/mm/var/master.pid User=mailman3 Group=mailman3 Environment="MAILMAN_CONFIG_FILE=/etc/mailman3/mailman.cfg" ExecStart=/opt/mailman3/venv/bin/mailman start ExecReload=/opt/mailman3/venv/bin/mailman restart ExecStop=/opt/mailman3/venv/bin/mailman stop # Tested, but have no effect on deletion of the lock file. #Restart=on-failure #RestartSec=15 [Install] WantedBy=multi-user.targetLars
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to vogt@spamcop.net
[Gerald]
What is in the logs about why the locks remain? There should be something in the system log about it if systemd kills a service process because it doesn't stop in time during shutdown... -Gerald
I got this:
juni 12 08:13:28 dalen.lamasti.net systemd[1]: Stopping mailman3.service - GNU Mailing List Manager...
juni 12 08:13:49 dalen.lamasti.net mailman[497907]: Shutting down Mailman's master runner
juni 12 08:13:51 dalen.lamasti.net mailman[1851]: Traceback (most recent call last):
juni 12 08:13:51 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1363, in main
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: Traceback (most recent call last):
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: rv = self.invoke(ctx)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: Traceback (most recent call last):
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1226, in invoke
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return ctx.invoke(self.callback, **ctx.params)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return callback(*args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return f(get_current_context(), *args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(runner.status)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: SystemExit: 15
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: During handling of the above exception, another exception occurred:
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: Traceback (most recent call last):
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/bin/runner", line 8, in <module>
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(main())
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return callback(*args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1442, in __call__
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return self.main(*args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1362, in main
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: with self.make_context(prog_name, args, **extra) as ctx:
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 480, in __exit__
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: def __exit__(
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/core/runner.py", line 117, in signal_handler
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return callback(*args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return f(get_current_context(), *args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return f(get_current_context(), *args, **kwargs)
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: sys.exit(runner.status)
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: sys.exit(runner.status)
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: SystemExit: 15
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: During handling of the above exception, another exception occurred:
...
Lars
On 21.06.26 13:16, Lars Bjørndal wrote:
Sometimes when I reboot the machine, I experience that mailman doesn't start because the files /opt/mailman3/mm/var/locks/master.lck* isn't deleted during shutdown. Maybe one way to solve this could be to use ExecStartPre directive in the systemd service to remove the lock files. What's the best practice to solve this problem? BTW I've noticed that doing
systemctl stop mailman3takes a very long time, 45 secs. or something like that.
Mailman 3.3.10
Python 3.12.13
[Unit] Description=GNU Mailing List Manager After=syslog.target network.target postgresql.service [Service] Type=forking PIDFile=/opt/mailman3/mm/var/master.pid User=mailman3 Group=mailman3 Environment="MAILMAN_CONFIG_FILE=/etc/mailman3/mailman.cfg" ExecStart=/opt/mailman3/venv/bin/mailman start ExecReload=/opt/mailman3/venv/bin/mailman restart ExecStop=/opt/mailman3/venv/bin/mailman stop # Tested, but have no effect on deletion of the lock file. #Restart=on-failure #RestartSec=15 [Install] WantedBy=multi-user.targetLars
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to vogt@spamcop.net
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to lars@lamasti.net
Well, then fixing this exception would be the solution to your problem. Discarding the lock files or using the force option doesn't solve the real problem. -Gerald
On 22.06.26 08:56, Lars Bjørndal wrote:
[Gerald]
What is in the logs about why the locks remain? There should be something in the system log about it if systemd kills a service process because it doesn't stop in time during shutdown... -Gerald
I got this:
juni 12 08:13:28 dalen.lamasti.net systemd[1]: Stopping mailman3.service - GNU Mailing List Manager... juni 12 08:13:49 dalen.lamasti.net mailman[497907]: Shutting down Mailman's master runner juni 12 08:13:51 dalen.lamasti.net mailman[1851]: Traceback (most recent call last): juni 12 08:13:51 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1363, in main juni 12 08:13:52 dalen.lamasti.net mailman[1855]: Traceback (most recent call last): juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: rv = self.invoke(ctx) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: Traceback (most recent call last): juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1226, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return ctx.invoke(self.callback, **ctx.params) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: SystemExit: 15 juni 12 08:13:52 dalen.lamasti.net mailman[1851]: During handling of the above exception, another exception occurred: juni 12 08:13:52 dalen.lamasti.net mailman[1851]: Traceback (most recent call last): juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/bin/runner", line 8, in <module> juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(main()) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1442, in __call__ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return self.main(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1362, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: with self.make_context(prog_name, args, **extra) as ctx: juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 480, in __exit__ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: def __exit__( juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/core/runner.py", line 117, in signal_handler juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1860]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1855]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: SystemExit: 15 juni 12 08:13:52 dalen.lamasti.net mailman[1855]: During handling of the above exception, another exception occurred: ...Lars
On 21.06.26 13:16, Lars Bjørndal wrote:
Sometimes when I reboot the machine, I experience that mailman doesn't start because the files /opt/mailman3/mm/var/locks/master.lck* isn't deleted during shutdown. Maybe one way to solve this could be to use ExecStartPre directive in the systemd service to remove the lock files. What's the best practice to solve this problem? BTW I've noticed that doing
systemctl stop mailman3takes a very long time, 45 secs. or something like that.
Mailman 3.3.10
Python 3.12.13
[Unit] Description=GNU Mailing List Manager After=syslog.target network.target postgresql.service [Service] Type=forking PIDFile=/opt/mailman3/mm/var/master.pid User=mailman3 Group=mailman3 Environment="MAILMAN_CONFIG_FILE=/etc/mailman3/mailman.cfg" ExecStart=/opt/mailman3/venv/bin/mailman start ExecReload=/opt/mailman3/venv/bin/mailman restart ExecStop=/opt/mailman3/venv/bin/mailman stop # Tested, but have no effect on deletion of the lock file. #Restart=on-failure #RestartSec=15 [Install] WantedBy=multi-user.targetLars
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to vogt@spamcop.net
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to lars@lamasti.net
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to vogt@spamcop.net
Thank you, Gerald! You wrote:
Well, then fixing this exception would be the solution to your problem. Discarding the lock files or using the force option doesn't solve the real problem. -Gerald
That sounds reasonable. Do you have any hint on why this exception happens now and then when rebooting?
Thanks, Lars
Lars Bjørndal writes:
[Gerald]
What is in the logs about why the locks remain? There should be something in the system log about it if systemd kills a service process because it doesn't stop in time during shutdown... -Gerald
I got this:
It's hard to say what's happening because your traceback(s) are truncated. The fact that the timestamps are all the same suggests that the problem isn't that systemd gets tired of waiting and eventually force-kills (signal 9) Mailman. Rather, systemd times out because Mailman aborts from an internal Exception (not in the log you reported) and never removes the pidfile, so systemd doesn't know that Mailman exited. The locks remain because Mailman aborted.
Normally you don't clean up locks because you don't want whatever happened to destroy even more data, but experience has shown that these events "never" (as far as we know) destroy data. The store-and-forward discipline for the message-handling queues is very robust. I would say it's nearly mathematically certain that Mailman exceptions cannot destroy messages. Apparently we get the granularity of database transactions right, but I'm not as sure of that.
I'm guessing that all of the runners eventually raise the same exception, but the first mention is likely to be quite a way down in the log file.
We need to see the tracebacks all the way to the end or we can't help. It's likely to be *very* long if all 15 or so runners raise the exception.
For the terminally curious, a few comments are below the lines being analyzed. Since the timestamps are all the same, I'm going to collect the logs for each process for clarity.
juni 12 08:13:28 dalen.lamasti.net systemd[1]: Stopping mailman3.service - GNU Mailing List Manager... juni 12 08:13:49 dalen.lamasti.net mailman[497907]: Shutting down Mailman's master runner
This process is the mailman command line utility to shut down the Mailman
system. (Mailman appears to have run quite a while before being stopped.)
juni 12 08:13:51 dalen.lamasti.net mailman[1851]: Traceback (most recent call last):
This appears to be a runner process (queue handling worker), forked from the master process mentioned above. But it might be the master itself. This is probably in the boot process for the OS, invoked automatically by init (systemd).
juni 12 08:13:51 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1363, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: rv = self.invoke(ctx) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1226, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return ctx.invoke(self.callback, **ctx.params) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: SystemExit: 15
This is a "shut down gracefully" signal received presumably from the master runner process. This looks normal.
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: During handling of the above exception, another exception occurred:
"Above exception" == SystemExit, raised when receiving the "shut down" signal.
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: Traceback (most recent call last): juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/bin/runner", line 8, in <module> juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(main()) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1442, in __call__ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return self.main(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1362, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: with self.make_context(prog_name, args, **extra) as ctx: juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 480, in __exit__ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: def __exit__( juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/core/runner.py", line 117, in signal_handler
This is not the end of the log for this process, the line of code should be quoted later in the log, and probably the log continues.
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: Traceback (most recent call last):
Another runner process. Note that the traceback is the same as the traceback for process 1851, as far as it goes.
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1855]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: SystemExit: 15 juni 12 08:13:52 dalen.lamasti.net mailman[1855]: During handling of the above exception, another exception occurred:
Again, the log is truncated for this process.
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: Traceback (most recent call last):
Another runner process. Note that the traceback is the same as the traceback for process 1851, as far as it goes.
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1860]: sys.exit(runner.status)
The exception is almost certainly another SystemExit. It looks like all of the runners are going to have the same logs, interleaved.
Steve
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
Thank you, Stephen! Your information is interesting. I think, next time it happens, even though I now use the --force to the start command, I will post the complete log.
Thanks, Lars
On Tue, Jun 23, 2026 at 01:31:23AM +0900, Stephen J. Turnbull wrote:
Lars Bjørndal writes:
[Gerald]
What is in the logs about why the locks remain? There should be something in the system log about it if systemd kills a service process because it doesn't stop in time during shutdown... -Gerald
I got this:
It's hard to say what's happening because your traceback(s) are truncated. The fact that the timestamps are all the same suggests that the problem isn't that systemd gets tired of waiting and eventually force-kills (signal 9) Mailman. Rather, systemd times out because Mailman aborts from an internal Exception (not in the log you reported) and never removes the pidfile, so systemd doesn't know that Mailman exited. The locks remain because Mailman aborted.
Normally you don't clean up locks because you don't want whatever happened to destroy even more data, but experience has shown that these events "never" (as far as we know) destroy data. The store-and-forward discipline for the message-handling queues is very robust. I would say it's nearly mathematically certain that Mailman exceptions cannot destroy messages. Apparently we get the granularity of database transactions right, but I'm not as sure of that.
I'm guessing that all of the runners eventually raise the same exception, but the first mention is likely to be quite a way down in the log file.
We need to see the tracebacks all the way to the end or we can't help. It's likely to be *very* long if all 15 or so runners raise the exception.
For the terminally curious, a few comments are below the lines being analyzed. Since the timestamps are all the same, I'm going to collect the logs for each process for clarity.
juni 12 08:13:28 dalen.lamasti.net systemd[1]: Stopping mailman3.service - GNU Mailing List Manager... juni 12 08:13:49 dalen.lamasti.net mailman[497907]: Shutting down Mailman's master runner
This process is the
mailmancommand line utility to shut down the Mailman system. (Mailman appears to have run quite a while before being stopped.)juni 12 08:13:51 dalen.lamasti.net mailman[1851]: Traceback (most recent call last):
This appears to be a runner process (queue handling worker), forked from the master process mentioned above. But it might be the master itself. This is probably in the boot process for the OS, invoked automatically by init (systemd).
juni 12 08:13:51 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1363, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: rv = self.invoke(ctx) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1226, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return ctx.invoke(self.callback, **ctx.params) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: SystemExit: 15
This is a "shut down gracefully" signal received presumably from the master runner process. This looks normal.
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: During handling of the above exception, another exception occurred:
"Above exception" == SystemExit, raised when receiving the "shut down" signal.
juni 12 08:13:52 dalen.lamasti.net mailman[1851]: Traceback (most recent call last): juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/bin/runner", line 8, in <module> juni 12 08:13:52 dalen.lamasti.net mailman[1851]: sys.exit(main()) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1442, in __call__ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: return self.main(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 1362, in main juni 12 08:13:52 dalen.lamasti.net mailman[1851]: with self.make_context(prog_name, args, **extra) as ctx: juni 12 08:13:52 dalen.lamasti.net mailman[1851]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 480, in __exit__ juni 12 08:13:52 dalen.lamasti.net mailman[1851]: def __exit__( juni 12 08:13:52 dalen.lamasti.net mailman[1851]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/core/runner.py", line 117, in signal_handler
This is not the end of the log for this process, the line of code should be quoted later in the log, and probably the log continues.
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: Traceback (most recent call last):
Another runner process. Note that the traceback is the same as the traceback for process 1851, as far as it goes.
juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1855]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1855]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1855]: sys.exit(runner.status) juni 12 08:13:52 dalen.lamasti.net mailman[1855]: SystemExit: 15 juni 12 08:13:52 dalen.lamasti.net mailman[1855]: During handling of the above exception, another exception occurred:
Again, the log is truncated for this process.
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: Traceback (most recent call last):
Another runner process. Note that the traceback is the same as the traceback for process 1851, as far as it goes.
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/core.py", line 794, in invoke
juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return callback(*args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/click/decorators.py", line 34, in new_func juni 12 08:13:52 dalen.lamasti.net mailman[1860]: return f(get_current_context(), *args, **kwargs) juni 12 08:13:52 dalen.lamasti.net mailman[1860]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ juni 12 08:13:52 dalen.lamasti.net mailman[1860]: File "/opt/mailman3/venv/lib64/python3.12/site-packages/mailman/bin/runner.py", line 198, in main juni 12 08:13:52 dalen.lamasti.net mailman[1860]: sys.exit(runner.status)
The exception is almost certainly another SystemExit. It looks like all of the runners are going to have the same logs, interleaved.
Steve
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
participants (5)
-
Gerald Vogt -
iMark -
Lars Bjørndal -
Mark Sapiro -
Stephen J. Turnbull