No emails distributed after mail received for a new list following https://docs.list.org/en/latest/install/virtualenv.html on new mailman3 Debian 12 installation
Hello to all:
I require a Debian 12 VM that supports:
- Subversion
- Let's Encrypt and required challenges
- IMAP server
- mailman3
- DKIM server
I'm currently running same on a CentOS 7.x VM. So that needs to be retired/migrated sooner rather than later because CentOS is out-of-support. On the Debian 12 VM other services appear to be setup and working. mailman3 is the 2nd last service that I'm standing up.
In this moment, I've been able to follow https://docs.list.org/en/latest/install/virtualenv.html and https://docs.list.org/en/latest/config-web.html
I have mailman3 and mailmanweb services running successfully. I have not yet been able to integrate mailmanweb into Apache (probably because of the Subversion and Let's Encrypt requirements). But I'm able to access local Postorius instance and configure a superuser and a list with web browser running locally on Debian 12 VM.
My list is named ebill. An email was received for the DL, but not distributed. Can you advise how to troubleshoot from here?
Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: connect from smtprelay0015.hostedemail.com[216.40.44.15] Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: Anonymous TLS connection established from smtprelay0015.hostedemail.com[216.40.44.15]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256 Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: 3CD47189B: client=smtprelay0015.hostedemail.com[216.40.44.15] Mar 25 15:23:04 shackleton12 postfix/cleanup[247722]: 3CD47189B: message-id=<a7aee014-f200-4bd4-bbc4-bc9f37abb6df@systemdatabase.com> Mar 25 15:23:04 shackleton12 postfix/qmgr[241822]: 3CD47189B: from=<pjbondi@systemdatabase.com>, size=3238, nrcpt=1 (queue active) Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: disconnect from smtprelay0015.hostedemail.com[216.40.44.15] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7 Mar 25 15:23:04 shackleton12 postfix/lmtp[247723]: 3CD47189B: to=<ebill@systemdatabase.ca>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.29, delays=0.06/0.03/0.02/0.19, dsn=2.0.0, status=sent (250 Ok) Mar 25 15:23:04 shackleton12 postfix/qmgr[241822]: 3CD47189B: removed
root@shackleton12:~# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no compatibility_level = 3.6 inet_protocols = ipv4 local_recipient_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp mailbox_size_limit = 0 mailbox_transport = lmtp:unix:private/dovecot-lmtp masquerade_domains = systemdatabase.ca mydestination = $myhostname, systemdatabase.ca, localhost.systemdatabase.ca, localhost, systemdatabase.homelinux.com mydomain = systemdatabase.ca myhostname = systemdatabase.ca mynetworks = 127.0.0.0/8 myorigin = systemdatabase.ca owner_request_special = no proxy_interfaces = 192.168.1.8 readme_directory = no recipient_delimiter = + relay_domains = hash:/opt/mailman/mm/var/data/postfix_domains relayhost = [smtp.bellnet.ca]:25 smtp_tls_loglevel = 1 smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_security_level = may smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination permit_inet_interfaces smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination permit_inet_interfaces smtpd_tls_cert_file = /etc/letsencrypt/live/systemdatabase.ca-0001/fullchain.pem smtpd_tls_key_file = /etc/letsencrypt/live/systemdatabase.ca-0001/privkey.pem smtpd_tls_loglevel = 1 smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtputf8_enable = no transport_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp unknown_local_recipient_reject_code = 550
On March 26, 2025 3:28:27 AM GMT+07:00, Philip Bondi <pjbondi@systemdatabase.com> wrote:
My list is named ebill. An email was received for the DL, but not distributed. Can you advise how to troubleshoot from here?
Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: connect from smtprelay0015.hostedemail.com[216.40.44.15] Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: Anonymous TLS connection established from smtprelay0015.hostedemail.com[216.40.44.15]: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256 Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: 3CD47189B: client=smtprelay0015.hostedemail.com[216.40.44.15] Mar 25 15:23:04 shackleton12 postfix/cleanup[247722]: 3CD47189B: message-id=<a7aee014-f200-4bd4-bbc4-bc9f37abb6df@systemdatabase.com> Mar 25 15:23:04 shackleton12 postfix/qmgr[241822]: 3CD47189B: from=<pjbondi@systemdatabase.com>, size=3238, nrcpt=1 (queue active) Mar 25 15:23:04 shackleton12 postfix/smtpd[247708]: disconnect from smtprelay0015.hostedemail.com[216.40.44.15] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7 Mar 25 15:23:04 shackleton12 postfix/lmtp[247723]: 3CD47189B: to=<ebill@systemdatabase.ca>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.29, delays=0.06/0.03/0.02/0.19, dsn=2.0.0, status=sent (250 Ok) Mar 25 15:23:04 shackleton12 postfix/qmgr[241822]: 3CD47189B: removed
The above says the message was successfully delivered to Mailman's LMTP runner. Is it queued in one of Mailman's queue/* queued? Check mailman.log for info.
-- Mark Sapiro <mark@msapiro.net> Sent from my Not_an_iThing with standards compliant, open source software.
Thx very much for the assistance.
I've never used the REST api before. I don't think it's working, yet, for me. So I don't know how to dump my queues.
Here's my first try:
(venv) mailman@shackleton12:~$ mailman info GNU Mailman 3.3.10 (Tom Sawyer) Python 3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0] config file: /etc/mailman3/mailman.cfg db url: postgresql://mailman:PASSWORDSNIPPED@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:restpass (venv) mailman@shackleton12:~$ python Python 3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.
from mailmanclient import Client client = Client('http://localhost:8001/3.1', 'restadmin', 'restpass') dump(client.system) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'dump' is not defined
I can see messages from members in Posting Activity. But Non-Members do not yet show-up in posting activity. There are 3 non-member posts that I cannot find.
List Members (2)
CSV Export Remove ALL members Address Delivery Mode Moderation Action Bounce Score maha@systemdatabase.com Regular List default 0 pjbondi@systemdatabase.com Regular List default 0
List Nonmembers (3)
Email Address: Display Name: Address Delivery Mode Moderation Action 01000195d3585cf6-feabd0f1-2f9c-4197-86f7-222af724c2f6-000000@amazonses.com Regular List default Delete noreply@fromyourdentalprovider.com Regular Accept immediately (bypass other rules) Delete notifications@goldencaredentalservices.com Regular Accept immediately (bypass other rules) Delete
mailman.log
Mar 26 12:21:19 2025 (2942) ACCEPT: <c176607f-8892-4a59-82bc-1f97c72d578a@systemdatabase.com> [26/Mar/2025:12:21:32 -0400] "GET /3.1/lists/ebill@systemdatabase.ca HTTP/1.1" 200 368 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:21:32 -0400] "GET /3.1/lists/ebill@systemdatabase.ca/config HTTP/1.1" 200 3055 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:21:32 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/roster/owner HTTP/1.1" 200 748 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:21:32 -0400] "GET /3.1/addresses/pjbondi@systemdatabase.com HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:21:32 -0400] "GET /3.1/users/pjbondi@systemdatabase.com HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" Mar 26 12:21:34 2025 (2939) HyperKitty archived message <c176607f-8892-4a59-82bc-1f97c72d578a@systemdatabase.com> to https://systemdatabase.ca/archives/list/ebill@systemdatabase.ca/message/H5GJ... [26/Mar/2025:12:23:43 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:43 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/addresses HTTP/1.1" 200 500 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:43 -0400] "POST /3.1/members/find HTTP/1.1" 200 1363 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:43 -0400] "GET /3.1/addresses/pjbondi@systemdatabase.com HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:43 -0400] "GET /3.1/lists/ebill@systemdatabase.ca HTTP/1.1" 200 368 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:43 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/roster/owner HTTP/1.1" 200 748 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:43 -0400] "GET /3.1/addresses/pjbondi@systemdatabase.com HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:23:49 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:02 -0400] "GET /3.1/domains HTTP/1.1" 200 342 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:02 -0400] "GET /3.1/domains/systemdatabase.ca HTTP/1.1" 200 237 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:02 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:03 -0400] "POST /3.1/lists/find HTTP/1.1" 200 843 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:07 -0400] "GET /3.1/lists/ebill.systemdatabase.ca HTTP/1.1" 200 368 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/roster/owner?fields=email&fields=address&fields=self_link HTTP/1.1" 200 365 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/roster/moderator?fields=email&fields=address&fields=self_link HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill@systemdatabase.ca/config HTTP/1.1" 200 3055 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/archivers HTTP/1.1" 200 100 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill@systemdatabase.ca/requests?request_type=subscription HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/member/pjbondi%40systemdatabase.com HTTP/1.1" 200 613 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/members/9773dcac5e9744ddbcf42ef0a83a9f1d/preferences HTTP/1.1" 200 271 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/preferred_address HTTP/1.1" 404 58 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill@systemdatabase.ca/requests/count?token_owner=moderator HTTP/1.1" 200 73 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:08 -0400] "GET /3.1/lists/ebill@systemdatabase.ca/held/count HTTP/1.1" 200 73 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/addresses HTTP/1.1" 200 500 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "POST /3.1/members/find HTTP/1.1" 200 1363 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "GET /3.1/lists/ebill.systemdatabase.ca HTTP/1.1" 200 368 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/preferred_address HTTP/1.1" 404 58 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:18 -0400] "GET /3.1/members/9773dcac5e9744ddbcf42ef0a83a9f1d/preferences HTTP/1.1" 200 271 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:30 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:30 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/addresses HTTP/1.1" 200 500 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:30 -0400] "POST /3.1/members/find HTTP/1.1" 200 1363 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:30 -0400] "GET /3.1/members/9773dcac5e9744ddbcf42ef0a83a9f1d/preferences HTTP/1.1" 200 271 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:30 -0400] "GET /3.1/addresses/pjbondi@systemdatabase.com HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/addresses HTTP/1.1" 200 500 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "POST /3.1/members/find HTTP/1.1" 200 1363 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "GET /3.1/lists/ebill.systemdatabase.ca HTTP/1.1" 200 368 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/preferred_address HTTP/1.1" 404 58 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:49 -0400] "GET /3.1/members/9773dcac5e9744ddbcf42ef0a83a9f1d/preferences HTTP/1.1" 200 271 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/lists/ebill.systemdatabase.ca HTTP/1.1" 200 368 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/roster/owner?fields=email&fields=address&fields=self_link HTTP/1.1" 200 365 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/lists/ebill.systemdatabase.ca/roster/moderator?fields=email&fields=address&fields=self_link HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/lists/ebill@systemdatabase.ca/config HTTP/1.1" 200 3055 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146 HTTP/1.1" 200 395 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/preferred_address HTTP/1.1" 404 58 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "GET /3.1/users/befd42e8ea844204b074d39041de0146/addresses HTTP/1.1" 200 500 "-" "GNU Mailman REST client v3.3.5" [26/Mar/2025:12:24:55 -0400] "POST /3.1/members/find HTTP/1.1" 200 1363 "-" "GNU Mailman REST client v3.3.5"
Philip Bondi writes:
I've never used the REST api before. I don't think it's working, yet, for me.
It pretty clearly is running and basically working.
So I don't know how to dump my queues.
"Queues" in Mailman usually refers to email messages being processed. As the name suggests, they are processed as soon as possible and in order. You appear to be looking at the request pools, which aren't queues: they are processed at any time and in any order an administrator chooses to.
I don't think you need to be using mailmanclient for this problem. First you should recheck the "held posts" on that list. Setting a delivery option on the address does not release the held post, only future posts are affected. You must *also* accept that post explicitly to release it for delivery.
Second, you should check smtp.log. I believe it by default records the LMTP connection setup and close for incoming messages (at least it does on my own site), it records the number of recipients of the distribution, and certain other events (if a subscriber bounces the message).
If that is not the situation, what Mark suggessted is to look in the message queues, which are stored in the queue directories under /opt/mailman/mm/var/queue. Files being currently processed by the associated queue runner have the extension .bak. You can ignore those, they will disappear within a few seconds (unless the list has hundreds of subscribers). Files waiting to be processed have the extension .pck. You can examine their content with "mailman qfile path/to/queue/file.pck".
You're looking for files in queue/shunt or queue/bad (which indicates some problem that makes that message require operator intervention before they can be delivered) and files older than a few minutes in queue/out or queue/in (which Mailman believes it can deliver but for some reason are "stuck" in the queue).
In case none of that helps, some advice on mailmanclient usage:
Here's my first try:
(venv) mailman@shackleton12:~$ mailman info GNU Mailman 3.3.10 (Tom Sawyer) Python 3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0] config file: /etc/mailman3/mailman.cfg db url: postgresql://mailman:PASSWORDSNIPPED@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:restpass
This all looks normal, except you should change the REST credentials.
(venv) mailman@shackleton12:~$ python Python 3.11.2 (main, Nov 30 2024, 21:22:50) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.
from mailmanclient import Client client = Client('http://localhost:8001/3.1', 'restadmin', 'restpass')
You can do this, but I find 'mailman shell' more convenient most of the time. See
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/instal...
dump(client.system)
If you have omitted nothing you did, there is no "dump" function in that environment. (I don't think one exists at all.) You need to import such helpers. See
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/r...
Many examples of usage are here:
https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclient...
All of the information you present from the site or logs looks normal. So you should look at the more superficial data (on-disk queues, the held messages, etc) first, and worry about poking around with the internals (mailman shell, mailmanclient, REST API) later.
Steve
participants (3)
-
Mark Sapiro -
Philip Bondi -
Stephen J. Turnbull