dedicated mailman3 list server on Ubuntu 18.04 LTS - questions after installation
My goal is to build a stable dedicated list server using mailman3. It should use all current email authentication techniques to clearly show that it is an authentic non-spam mail list server. In the future, I may want to use the REST interface, so that’s why mailman2 is not an option.
My installation:
Installed Ubuntu 18.04 LTS server, with postfix, mysql and postgres. I then installed mail-stack-delivery, postfix-policyd-spf-python, OpenDKIM, and OpenDMARC. I configured SPF, DKIM, and DMARC records on my DNS server, and then configured postfix and dovecot to DKIM sign outgoing mail from local users, and check SPF, DMARC, and DKIM on incoming mail.
I then installed the package mailman3-full, which provides version 3.1.1-9, using postgres 10 as the database. I have looked in Ubuntu bionic-backports and bionic-updates and do not see a more recent version. I see that version 3.2 was released in July, are debian/ubuntu mailman 3.2 packages available somewhere?
I set up the /etc/mailman3 files:
I added this to the end of mailman.cfg [archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /etc/mailman3/mailman-hyperkitty.cfg
In mailman-hyperkitty.cfg I copied the api_key: value from mailman-web.py MAILMAN_ARCHIVER_KEY, but removed the single quotes surrounding the value in the mailman-web.py file when writing the key in the mailman-hyperkitty.cfg file.
I modified /etc/postfix/main.cf with this appended to the end: # Mailman-postfix integration owner_request_special = no # Support the default VERP delimiter. recipient_delimiter = + unknown_local_recipient_reject_code = 550 owner_request_special = no transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp local_recipient_maps = proxy:unix:passwd.byname, $alias_maps, hash:/var/lib/mailman3/data/postfix_lmtp relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} hash:/var/lib/mailman3/data/postfix_domains
I ran /usr/share/mailman3-web/manage.py createsuperuser I ran a2enmod proxy_uwsgi I made a symbolic link from /etc/mailman3/apache.conf to /etc/apache2/conf-available/mailman3.conf, then ran a2enconf mailman3
I have read the documentation in /usr/share/docs/mailman3* along with what I could find concerning mailman3 on the web.
The installation allows me to sign into the web interface, create a host, create mailing lists, and mass subscribe members. When members send email to the list address, the server sends the messages normally to the other members of the list.
However, when I look at the list archives, there are no posts. I deliberately mass-subscribed an invalid address, and I can see in /var/log/mail.log that the message bounces, but I get no notification, nor do I see any bounce info when I look at the member info.
My questions:
- logging
A lot of my questions would probably be answered if I could figure out hot to activate logging correctly. Currently, I see:
root@mailman:/var/log/mailman3# ll -R -rw-rw---- 1 list list 0 Jul 29 20:06 bounce.log -rw-rw---- 1 list list 0 Jul 29 20:06 debug.log -rw-r----- 1 list list 3298 Aug 4 00:00 mailman.log -rw-rw---- 1 list list 34606 Jul 30 17:44 smtp.log drwxr-xr-x 2 www-data www-data 4096 Aug 3 06:25 web/ ./web: -rw-rw-r-- 1 www-data www-data 1520 Aug 3 06:25 mailman-web.log
The bounce log and debug log are empty. I’ve set propagate: yes in the [logging.debian] section
From /etc/mailman3/mailman.cfg # In this section, you can define defaults for all loggers, which will be # prefixed by 'mailman.'. Use subsections to override settings for specific # loggers. The names of the available loggers are: # # - archiver -- All archiver output # - bounce -- All bounce processing logs go here # - config -- Configuration issues <…snip…>
Can anyone post an example of a ‘subsection’ to set archiver and bounce to debug? Do I have to create /var/log/mailman3/mailman.archiver.log and set ownership to list:list?
- Where are posts stored?
For me, this is the biggest mystery in mailman3. Local mail is delivered to ~/Maildir by dovecot. In the $var_dir hierarchy the directory messages contains just 2 messages dating to my initial tests. Anything I send to a list now, and which seems to be sent properly to the member addresses, I cannot find on the filesystem.
Any clues as to what may be going on here?
- Archival - Hyperkitty
How does this work? All the parts seem to be in place, and I can see from ps /usr/bin/python3 /usr/lib/mailman3/bin/runner --runner=archive:0:1 -C /etc/mailman3/mailman.cfg
How can I figure out why this is not working? In the web interface, I see that each of my lists has zero posts. But posts mailed to the list address do appear in the list members’ mailboxes!
- Bounces
I’ve deliberately put an erroneous email address as a list member. In postfix’s /var/log/mail.log I see the bounces, but get no notification, and /var/log/mailman3/bounce.log remains empty.
- DKIM and DMARC
I’ve got OpenDKIM running on inet:12345@localhost and OpenDMARC running on net:54321@localhost.
How can I get mailman to use these milters and sign messages? How should DMARC munging be configured when signing?
- Is there an overview of how mailman3 works?
Is there documentation which explains the path of an incoming post through the system, including initial reception, list processing, sending to members, processing bounces, and archiving posts?
Thanks in advance for help and pointers to any available documentation!
John Seifarth Words & Wires SPRL Computer Consulting & Language Services Brussels, Belgium
On 08/05/2018 03:37 PM, John Seifarth via Mailman-users wrote:
The installation allows me to sign into the web interface, create a host, create mailing lists, and mass subscribe members. When members send email to the list address, the server sends the messages normally to the other members of the list.
That's the good news. That's cool.
You have raised a lot of questions and I don't have time now to deal with them all. Maybe others can help. I will say a few things now and maybe more to follow.
However, when I look at the list archives, there are no posts. I deliberately mass-subscribed an invalid address, and I can see in /var/log/mail.log that the message bounces, but I get no notification, nor do I see any bounce info when I look at the member info.
About Archiving: Are you running django qcluster http://docs.list.org/en/latest/config-web.html#running-the-task-queue-requir... and all the scheduled tasks http://docs.list.org/en/latest/config-web.html#scheduled-tasks-required?
About Bounce processing: Bounce processing is not functional in Mailman 3 yet.
- DKIM and DMARC
I’ve got OpenDKIM running on inet:12345@localhost and OpenDMARC running on net:54321@localhost.
How can I get mailman to use these milters and sign messages? How should DMARC munging be configured when signing?
Your outgoing MTA should be doing the signing, not Mailman. Your signing outgoing messages won't help with DMARC if Mailman alters the message in any way, e.g. by subject prefixing, content filtering or addition of list headers and/or footers.
Minimum DMARC settings would be DMARC mitigation action -> Replace From: with list address and DMARC Mitigate unconditionally -> No
- Is there an overview of how mailman3 works?
Is there documentation which explains the path of an incoming post through the system, including initial reception, list processing, sending to members, processing bounces, and archiving posts?
All the docs are at http://docs.list.org/en/latest/ You may want to see http://mailman.readthedocs.io/en/latest/src/mailman/docs/8-miles-high.html amongst the many other things you'll find there.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, Aug 5, 2018, at 3:37 PM, John Seifarth via Mailman-users wrote:
My goal is to build a stable dedicated list server using mailman3. It should use all current email authentication techniques to clearly show that it is an authentic non-spam mail list server. In the future, I may want to use the REST interface, so that’s why mailman2 is not an option.
My installation:
Installed Ubuntu 18.04 LTS server, with postfix, mysql and postgres. I then installed mail-stack-delivery, postfix-policyd-spf-python, OpenDKIM, and OpenDMARC. I configured SPF, DKIM, and DMARC records on my DNS server, and then configured postfix and dovecot to DKIM sign outgoing mail from local users, and check SPF, DMARC, and DKIM on incoming mail.
I then installed the package mailman3-full, which provides version 3.1.1-9, using postgres 10 as the database. I have looked in Ubuntu bionic-backports and bionic-updates and do not see a more recent version. I see that version 3.2 was released in July, are debian/ubuntu mailman 3.2 packages available somewhere?
I set up the /etc/mailman3 files:
I added this to the end of mailman.cfg [archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /etc/mailman3/mailman-hyperkitty.cfg
In mailman-hyperkitty.cfg I copied the api_key: value from mailman- web.py MAILMAN_ARCHIVER_KEY, but removed the single quotes surrounding the value in the mailman-web.py file when writing the key in the mailman-hyperkitty.cfg file.
I modified /etc/postfix/main.cf with this appended to the end: # Mailman-postfix integration owner_request_special = no # Support the default VERP delimiter. recipient_delimiter = + unknown_local_recipient_reject_code = 550 owner_request_special = no transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp local_recipient_maps = proxy:unix:passwd.byname, $alias_maps, hash:/var/lib/mailman3/data/postfix_lmtp relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} hash:/var/lib/mailman3/data/postfix_domains
I ran /usr/share/mailman3-web/manage.py createsuperuser I ran a2enmod proxy_uwsgi I made a symbolic link from /etc/mailman3/apache.conf to /etc/apache2/ conf-available/mailman3.conf, then ran a2enconf mailman3
I have read the documentation in /usr/share/docs/mailman3* along with what I could find concerning mailman3 on the web.
The installation allows me to sign into the web interface, create a host, create mailing lists, and mass subscribe members. When members send email to the list address, the server sends the messages normally to the other members of the list.
However, when I look at the list archives, there are no posts. I deliberately mass-subscribed an invalid address, and I can see in /var/ log/mail.log that the message bounces, but I get no notification, nor do I see any bounce info when I look at the member info.
My questions:
- logging
A lot of my questions would probably be answered if I could figure out hot to activate logging correctly. Currently, I see:
root@mailman:/var/log/mailman3# ll -R -rw-rw---- 1 list list 0 Jul 29 20:06 bounce.log -rw-rw---- 1 list list 0 Jul 29 20:06 debug.log -rw-r----- 1 list list 3298 Aug 4 00:00 mailman.log -rw-rw---- 1 list list 34606 Jul 30 17:44 smtp.log drwxr-xr-x 2 www-data www-data 4096 Aug 3 06:25 web/ ./web: -rw-rw-r-- 1 www-data www-data 1520 Aug 3 06:25 mailman-web.log
The bounce log and debug log are empty. I’ve set propagate: yes in the [logging.debian] section
From /etc/mailman3/mailman.cfg # In this section, you can define defaults for all loggers, which will be # prefixed by 'mailman.'. Use subsections to override settings for specific # loggers. The names of the available loggers are: # # - archiver -- All archiver output # - bounce -- All bounce processing logs go here # - config -- Configuration issues <…snip…>
Can anyone post an example of a ‘subsection’ to set archiver and bounce to debug? Do I have to create /var/log/mailman3/mailman.archiver.log and set ownership to list:list?
[logging.archiver] level: debug
[logging.bounce] level: debug
- Where are posts stored?
For me, this is the biggest mystery in mailman3. Local mail is delivered to ~/Maildir by dovecot. In the $var_dir hierarchy the directory messages contains just 2 messages dating to my initial tests. Anything I send to a list now, and which seems to be sent properly to the member addresses, I cannot find on the filesystem.
Any clues as to what may be going on here?
Mailman by itself doesn't store anything, it just sends and receives. $var/queue directory should have all the posts that are currently in queue and are waiting to be processed.
If you are talking about Archived messages, they should be in the database.
- Archival - Hyperkitty
How does this work? All the parts seem to be in place, and I can see from ps /usr/bin/python3 /usr/lib/mailman3/bin/runner --runner=archive:0:1 -C / etc/mailman3/mailman.cfg
How can I figure out why this is not working? In the web interface, I see that each of my lists has zero posts. But posts mailed to the list address do appear in the list members’ mailboxes!
The runner above sends posts to Hyperkitty, which is a running Django app. You should check the logs if the messages are being successfully delivered to Hyperkitty.
- Bounces
I’ve deliberately put an erroneous email address as a list member. In postfix’s /var/log/mail.log I see the bounces, but get no notification, and /var/log/mailman3/bounce.log remains empty.
- DKIM and DMARC
I’ve got OpenDKIM running on inet:12345@localhost and OpenDMARC running on net:54321@localhost.
How can I get mailman to use these milters and sign messages? How should DMARC munging be configured when signing?
- Is there an overview of how mailman3 works?
Is there documentation which explains the path of an incoming post through the system, including initial reception, list processing, sending to members, processing bounces, and archiving posts?
Thanks in advance for help and pointers to any available documentation!
John Seifarth Words & Wires SPRL Computer Consulting & Language Services Brussels, Belgium
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
participants (3)
-
Abhilash Raj -
John Seifarth -
Mark Sapiro