I am very sorry to you folks for not getting back on the problems I am having to install on Debian testing. There are two reasons. One is that I have been much offline for personal reasons. The other is that I have another machine called tagol where I have been running Mailman3 for about three years, using the Debian packet on Debian testing. As testing move to 3.13 this installation could not run. I replaced mailman-core with a 3.12 venv install, but left the mailman3-web package of Debian. This half-arsed job held up for a long time but two weeks ago collapsed. No email went through. I have been trying to switch to a full venv version, still staying with 3.12.
Tagol hosts a single list
mailman@tagol:~$ mailman lists 1 matching mailing lists found: bibnez@folks.email
I have some bad queue files. Let's look at the oldest
mailman@tagol:~$ ls -lt /var/lib/mailman3/queue/bad/ | tail -1 -rw-rw---- 1 mailman mailman 8579 Mar 11 14:32 1773239548.2417333+451b61b8707218b4dc3d0864eb4ee0172bd21d71.psv
Write a script "test_bad" to submit this back.
mailman@tagol:~$ cat test_bad #!/bin/bash
base=/var/lib/mailman3/queue/bad/1773239548.2417333+451b61b8707218b4dc3d0864eb4ee0172bd21d71 psv=$base.psv pck=$base.pck
cp $psv $pck mailman unshunt mailman@tagol:~$
When I run this script, I get in my log
Mar 15 05:40:46 2026 (2658932) <a930211e778a4aa0829cee5b2a9e7db9@kbs-mg.de> runner "in" shunting message for missing list: <object object at 0x7fb1ad1ad730>
I suspect mailman does not see the database. But
mailman@tagol:~$ mailman info | grep -v credent GNU Mailman 3.3.10 (Tom Sawyer) Python 3.12.9 (main, Feb 5 2025, 01:31:18) [GCC 14.2.0] config file: /etc/mailman3/mailman.cfg db url: sqlite:////var/lib/mailman3/data/mailman.db devmode: DISABLED REST root url: http://localhost:8001/3.1/
The file exists
mailman@tagol:~$ ls -l /var/lib/mailman3/data/mailman.db -rw-rw---- 1 mailman mailman 4825088 Mar 15 05:20 /var/lib/mailman3/data/mailman.db
I have no idea where to look next.
Thanks and cheers!
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22199th day.
On Sun, Mar 15, 2026 at 10:31 AM Thomas Krichel <krichel@openlib.org> wrote:
I am very sorry to you folks for not getting back on the problems I am having to install on Debian testing. There are two reasons. One is that I have been much offline for personal reasons. The other is that I have another machine called tagol where I have been running Mailman3 for about three years, using the Debian packet on Debian testing. As testing move to 3.13 this installation could not run. I replaced mailman-core with a 3.12 venv install, but left the mailman3-web package of Debian. This half-arsed job held up for a long time but two weeks ago collapsed. No email went through. I have been trying to switch to a full venv version, still staying with 3.12.
Tagol hosts a single list
mailman@tagol:~$ mailman lists 1 matching mailing lists found: bibnez@folks.email
I have some bad queue files. Let's look at the oldest
mailman@tagol:~$ ls -lt /var/lib/mailman3/queue/bad/ | tail -1 -rw-rw---- 1 mailman mailman 8579 Mar 11 14:32 1773239548.2417333+451b61b8707218b4dc3d0864eb4ee0172bd21d71.psv
Write a script "test_bad" to submit this back.
mailman@tagol:~$ cat test_bad #!/bin/bash
base=/var/lib/mailman3/queue/bad/1773239548.2417333+451b61b8707218b4dc3d0864eb4ee0172bd21d71 psv=$base.psv pck=$base.pck
cp $psv $pck mailman unshunt mailman@tagol:~$
When I run this script, I get in my log
Mar 15 05:40:46 2026 (2658932) <a930211e778a4aa0829cee5b2a9e7db9@kbs-mg.de
runner "in" shunting message for missing list: <object object at 0x7fb1ad1ad730>
I suspect mailman does not see the database. But
mailman@tagol:~$ mailman info | grep -v credent GNU Mailman 3.3.10 (Tom Sawyer) Python 3.12.9 (main, Feb 5 2025, 01:31:18) [GCC 14.2.0] config file: /etc/mailman3/mailman.cfg db url: sqlite:////var/lib/mailman3/data/mailman.db devmode: DISABLED REST root url: http://localhost:8001/3.1/
The file exists
mailman@tagol:~$ ls -l /var/lib/mailman3/data/mailman.db -rw-rw---- 1 mailman mailman 4825088 Mar 15 05:20 /var/lib/mailman3/data/mailman.db
I have no idea where to look next.
Thanks and cheers!
MM3 with Sqlite is not recommended for production. The virtualenv HOWTO even doesn't mention it. So ideally, you should be using PostgreSQL or MariaDB. In a production setup, the MM3 core and Django config files should ideally reside in /etc/mailman3/ The output of the command 'mailman info' should show "config file: /etc/mailman3/mailman.cfg". with db_url pointing to an officially supported RDBM. That will tell you that MM3 is reading the expected config. Mixing-and-match of virtualenv and OS package install will lead to lots of pain chasing ghosts - as you can already see. Having said that, let's say your choice is sqlite as the DB, does your list exist in the DB?
Run this command from the DB and show output: select id, list_id from mailinglist;
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Washington Odhiambo writes
MM3 with Sqlite is not recommended for production. The virtualenv HOWTO even doesn't mention it.
I know now, but I have it in production now. It was the default in the Debian package.
So ideally, you should be using PostgreSQL or MariaDB.
But I don't know how to convert. I know very little about RDMS, have been avoiding them since my earliest childhood. ;-)
In a production setup, the MM3 core and Django config files should ideally reside in /etc/mailman3/
mailman@tagol:~$ mailman info | grep -v creden GNU Mailman 3.3.10 (Tom Sawyer) Python 3.12.9 (main, Feb 5 2025, 01:31:18) [GCC 14.2.0] config file: /etc/mailman3/mailman.cfg db url: sqlite:////var/lib/mailman3/data/mailman.db devmode: DISABLED REST root url: http://localhost:8001/3.1/
The output of the command 'mailman info' should show "config file: /etc/mailman3/mailman.cfg". with db_url pointing to an officially supported RDBM.
Well the list has been running off sqlite for years, so I believe all the data is there and it is correct.
That will tell you that MM3 is reading the expected config.
I think it does.
Having said that, let's say your choice is sqlite as the DB, does your list exist in the DB?
I think so.
Run this command from the DB and show output: select id, list_id from mailinglist;
mailman@tagol:~$ sqlite3 /var/lib/mailman3/data/mailman.db SQLite version 3.46.1 2024-08-13 09:16:08 Enter ".help" for usage hints. sqlite> select id, list_id from mailinglist; 1|bibnez.folks.email sqlite>
So it is in the database. Thank you so much for your help!
So presumably whatver is running "mailman shunt" is looking for the database in a different location.
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22199th day.
On Sun, Mar 15, 2026 at 12:51 PM Thomas Krichel <krichel@openlib.org> wrote:
Washington Odhiambo writes
MM3 with Sqlite is not recommended for production. The virtualenv HOWTO even doesn't mention it.
I know now, but I have it in production now. It was the default in the Debian package.
So ideally, you should be using PostgreSQL or MariaDB.
But I don't know how to convert. I know very little about RDMS, have been avoiding them since my earliest childhood. ;-)
There are tools that can help you do it, if my Google-foo serves me right, but I suppose you have feared those since childhood as well.
In a production setup, the MM3 core and Django config files should ideally reside in /etc/mailman3/
mailman@tagol:~$ mailman info | grep -v creden GNU Mailman 3.3.10 (Tom Sawyer) Python 3.12.9 (main, Feb 5 2025, 01:31:18) [GCC 14.2.0] config file: /etc/mailman3/mailman.cfg db url: sqlite:////var/lib/mailman3/data/mailman.db devmode: DISABLED REST root url: http://localhost:8001/3.1/
The output of the command 'mailman info' should show "config file: /etc/mailman3/mailman.cfg". with db_url pointing to an officially supported RDBM.
Well the list has been running off sqlite for years, so I believe all the data is there and it is correct.
OK.
That will tell you that MM3 is reading the expected config.
I think it does.
Having said that, let's say your choice is sqlite as the DB, does your list exist in the DB?
I think so.
Run this command from the DB and show output: select id, list_id from mailinglist;
mailman@tagol:~$ sqlite3 /var/lib/mailman3/data/mailman.db SQLite version 3.46.1 2024-08-13 09:16:08 Enter ".help" for usage hints. sqlite> select id, list_id from mailinglist; 1|bibnez.folks.email sqlite>
So it is in the database. Thank you so much for your help!
Great. The list is in the DB. Now, let's see if MM3 is using that DB: What is the output of the command 'mailman lists'?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Washington Odhiambo writes
Great. The list is in the DB. Now, let's see if MM3 is using that DB: What is the output of the command 'mailman lists'?
mailman@tagol:~$ mailman lists 1 matching mailing lists found: bibnez@folks.email
And manual inspection of the pck file suggests that this is the list the email was sent to.
Thanks and cheers!
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22199th day.
Thomas Krichel wrote:
Washington Odhiambo writes
MM3 with Sqlite is not recommended for production. The virtualenv HOWTO even doesn't mention it.
I know now, but I have it in production now. It was the default in the Debian package.
| Default database backend is SQLite3 in order to not break automated | installations. For productive setups, PostgreSQL or MySQL are much | better options though. See README.Debian for further information. <https://packages.debian.org/trixie/mailman3>
-thh
Thomas Hochstein writes
| Default database backend is SQLite3 in order to not break automated | installations. For productive setups, PostgreSQL or MySQL are much | better options though. See README.Debian for further information. <https://packages.debian.org/trixie/mailman3>
It's what I read at the time. But it's neither here nor there now. the sqlite has the list information, the question is why "mailman shunt" does not see it, when "mailman lists" does.
But if you go back to the error message, it says
| Mar 15 05:40:46 2026 (2658932) <a930211e778a4aa0829cee5b2a9e7db9@kbs-mg.de> | runner "in" shunting message for missing list: <object object at 0x7fb1ad1ad730>
it does appear that "<object object at 0x7fb1ad1ad730>" is the list_id, when it should in fact be "bibnez@folks.email". Ok. In the runner.py, line 237
| list_id = msgdata.get('listid', missing)
if I add right after line 237
| elog.error('%s is the list_id from the message', list_id)
I see log entry
| Mar 15 12:40:11 2026 (3262760) list_id is <object object at 0x7fbdd6175730>
So maybe just renaming .psv to .pck and moving to shunt does not make a file in the bad queue fit for unshunting.
Thanks and cheers!
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22199th day.
Thomas Krichel writes
So maybe just renaming .psv to .pck and moving to shunt does not make a file in the bad queue fit for unshunting.
It turns out this is correct. I just did submit the messages badly. The messages need to be copied from the 'bad' directory into the 'in' directory. There is no need to unshunt. Here is a bash script for this
#!/bin/bash
qdir=/var/lib/mailman3/queue for fufi in $qdir/bad/*.psv; do if [ -f "$fufi" ]; then psv_bana=$(basename "$fufi") pck_bana="${psv_bana%.*}.pck" cp $qdir/bad/$psv_bana $qdir/in/$pck_bana fi done
I hope this will be useful!
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22199th day.
participants (3)
-
Thomas Hochstein -
Thomas Krichel -
Washington Odhiambo