March 2022 - Mailman-users - lists.mailman3.org

Lessons learned and a couple little gotcha for newbies
by William Oliver 13 Apr '22

13 Apr '22

I just finished, with great help from Mark Sapiro on the list, a new installation of mailman3. I used the instructions at https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/insta… and https://docs.mailman3.org/en/latest/install/virtualenv.html There were a couple of gotchas that I didn't see well documented. 1) I got the error django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/venv/lib/python3.9/site- packages/mistune/scanner.py) when I ran 'mailman-web migrate' The fix was trivial: I just added the lines: from mistune.util import escape_html from mistune.util import escape_url to /opt/mailman/venv/lib/python3.9/site-packages/mistune/scanner.py It's not hard to search for those definitions, but if someone doesn't know a smattering of python, it might not be obvious. 2) Similarly, migrate (using postgres) also gave the error: File "/opt/mailman/venv/lib/python3.9/site- packages/django/db/backends/postgresql/utils.py", line 6, in utc_tzinfo_factory raise AssertionError("database connection isn't set to UTC") I was able to find a discusson of this one at https://stackoverflow.com/questions/68024060/assertionerror-database-connec… which had me downgrade to an older version of psycopg2-binary: pip install psycopg2-binary==2.8.6 There might be a different fix, but this worked. 3) When I ran the test "uwsgi --ini /etc/mailman3/uwsgi.ini" I got the error: ModuleNotFoundError: No module named 'settings' This was also trivially fixed by adding the path: export PYTHONPATH=$PYTHONPATH:/etc/mailman3 in /opt/mailman/.bashrc 4) Sometimes when I rebooted the machine, when I ran 'systemctl start mailman3' I'd get the error that mailman wouldn't start because the lock was stll there. I never could figure out where the lock was, so I worked around it by replacing ExecStart=/opt/mailman/venv/bin/mailman start with ExecStart=/opt/mailman/venv/bin/mailman start --force in /etc/systemd/system/mailman3.service I'm sure there's something bad about doing that, but it works for me, so far. 5) I had a problem with mail to outside people not being delivered while mail to the local host was delivered. It turned out that this was because in my mailman.cfg I had "smtp_host:my.listserver.tld when it should have been localhost. The error was "relay access denied." To fix this, I had to add my.listserver.tld to "mynetworks" in /etc/postfix/main.cf, ie change: mynetworks = 127.0.0.0/8 10.0.0.0/24 to mynetworks = 127.0.0.0/8 10.0.0.0/24 111.222.333.444 <--- my ip address However, Mark Sapiro (a God) told me this was because I had changed a line in the [mta] section of mailman.cfg. I had changed smtp_host: localhost to smtp_host: mydomain.tld Here's the thing. The reason I did this was because the instructions at https://docs.mailman3.org/en/latest/config-core.html give this as their model: [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: mail.example.com lmtp_port: 8024 smtp_host: mail.example.com <---- not localhost smtp_port: 25 The instructions are good, but there are a few places where one size does not fit all, I guess. 6) A second error in delivery to outside addresses resulted in an "Access denied" error. A search on the internet revealed that this had to do with (for me) smtpd_recipient_restrictions in postfix. See, for instance: https://know.mailsbestfriend.com/postfix_error_relay_access_denied_noqueue_… Changing this fixed things. I'm not sure, but I think this shows up mostly with installations that include dovecot. 7) If you are using virtual domains in postfix, you need to use the "unusual postfix configuration" in https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h… virtual_alias_maps = hash:/path-to-mailman/var/data/postfix_vmap 8) You need to manually set the local virtual host for mailman3. Somehow, I got the impression from the installation instructions that this was done automagically to the default x.mydomain.tld if your domain is mydomain.tld. It doesn't. 9) Follow the instructions verbatim, and follow them in the order provided. I installed postfix and nginx out of order, and it did OK, but once you get into the python virtual environment, it is easy to miss something if you don't do it in order. 10) Jot down lots of notes. This is one of the more complicated installations I've done, mostly because it involved configuring a zillion packages to talk to each other: Postfix, nginx, postorius, hyperkitty, django, and others (I'd never heard of "whoosh" or "sass" for instance). All of these can be unforgiving with configuration errors. The default installation instructions assumed a number of packages I'd never used. I'm an apache/dovecot/mysql kind of person, and I had to switch to nginx/postgres to use the instructions as the written defaults. I'm sure I could have used apache and mysql, but it would mean trying to figure out what didn't work without following the installations verbatim. I'd never even heard of postorius, hyperkitty, or django. So, it was easy to get confused a bit. Taking notes of changes I made was important in being able to back out of disastrous choices. Plus, once I'd installed it on one machine, I sat down and installed it on another machine, and it was very useful to have written down how I fixed an issue when it came up again -- and I didn't remember exactly what I had done. 11) Make multiple sequential backups of configuration files. I went down a couple of paths I shouldn't have gone down. For instance, with the vmap issue, I got it in my head that it was a postfix configuration issue, not a mailman issue. So I kept changing things in postfix. Once I was told that it was in *mailman* I had to set the virtual host, I needed to back out of about 8 tweaks I'd made to main.cf in /etc/postfix. If I hadn't been making main.cf.backup.01, main.cf.backup.02, etc. I would have had to spend another few hours trying to remember what inappropriate changes I had made. This is particularly true for me because I am not a great typist, and when I try to retype something to back out, there's a reasonable chance I'll mistype something and it still won't work. With these backups, I don't have both manually back out of things *and* proof my typing again and again. It's a lot easier to delete backups once things are working than to try to remember what you did late at night after a couple of bourbons. 12) Choose one set of installation instructions and follow it. Don't try to mix instructions from different tutorials unless you really know what you are doing. I didn't. For instance, there's a different set of mailman3 installation instructions at: https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 It does things a little differently. It's useful to look at that when you are debugging why something's not working, but you can get into some trouble trying to do both installations at the same time -- which I found myself doing when something wasn't working. It didn't help. billo

3 6

Re: Mailman3 reinstall fail, need some help
by Yvan ♞ 11 Apr '22

11 Apr '22

Yes it's debian and I have root access but it's a more "user friendly" server release. I tryed this but it block https://github.com/YunoHost-Apps/mailman3_ynh/blob/master/README.md https://yunohost.org/en/administrate/whatisyunohost Sincerely Yvan Le 12/03/2022 à 15:28, Eggert Ehmke a écrit : > Hello Yvan, > > I would suggest to install mailman3 following this guide: > https://docs.mailman3.org/en/latest/install/virtualenv.html > > but at first check, which operating system is on your yunohost? Is it Debian/ > Ubuntu? In this case the guide is fine, else there will be some adaptions. > > Tell me some details of your system, and if you have root access and can > install software packages. > > Regards > Eggert > > Am Samstag, 12. März 2022, 15:16:11 CET schrieb Yvan ♞: >> Hello, just made a try to install mailman 3 on yunohost unsuccessfully >> Sad... >> If some body can help me... >> Thanks >> https://paste.yunohost.org/raw/hujesebita >> >> Le 07/02/2022 à 15:57, Stephen J. Turnbull a écrit : >> >>> Yvan ♞ writes: >>> >>> >>> >>> > Just to know if I have a chance, do I have to unistall mailman before >>> > installing mailman 3 ? >>> >>> >>> >>> No. In fact, you can run both at the same time. Not for the same >>> list, obviously. And it requires a fair amount of planning and careful >>> attention to your web and MTA configurations. As far as I know there >>> are no naming conflicts in the default installations, Mailman 2 >>> installs into directory trees under directories named "mailman", while >>> Mailman 3 installs under "mailman3". >> _______________________________________________ >> Mailman-users mailing list --mailman-users(a)mailman3.org >> To unsubscribe send an email tomailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > >

4 9

out queue backlog and delivery rate
by dancab＠caltech.edu 11 Apr '22

11 Apr '22

We experienced an issue where our upstream mail protection gateway had an issue for a few hours. As a result out Mailman system was inundated with hundreds of messages within half an hour. Mail delivery is continuing at a snails pace and it is now 4 hours behind. I noticed that the system isn't overloaded either via CPU or Memory. It's just slowly "working on it". Are there rate limits that can be tuned for Mailman queues such as the "out" queue? Could this be IO related somehow? I'd expect there to be a CPU spike in that case. Please advise. Thanks in advance. ---- Dan Caballero Systems Administrator Academic Computing Solutions IMSS - Caltech https://imss.caltech.edu Is there anyway to

2 7

Active archivers and archiving and thread question.
by bob B 31 Mar '22

31 Mar '22

Digging into the mailman3 world, and want to check my logic... We normally don't use archives, so I have "archive policy" set to "Do not archive this list" for our lists. However, the "archives" button is still there on the list info page, and if users go into it, they don't see anything, But they can click "start a new thread" which sends a message to the list, but since archiving is turned off, that message does not show up there. Am I understanding that correctly? if I have "Active archivers" disabled, that hides that area from the users, correct? Any reason why these are separated? why have an active archiver, if archiving is turned off -OR- archives are turned on, but no active archiver?

2 3

New install - The Message have no valid senders
by kenneth.karlsson＠eco.cept.org 31 Mar '22

31 Mar '22

I have installed a brand spanking new Ubuntu 21.10, with postfix as SMTP server and mailman3-full apt package.. All checks out fine except when i deliver mails into MM. Mailman Core Version GNU Mailman 3.3.3 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.9.7 (default, Sep 10 2021, 14:59:43) [GCC 11.2.0] I have a very simple mail command going into Postfix, and it receives it fine and relay it to MM (email address obscured below but of course a proper email address was used !): The command: EHLO contoso.com mail from: <xxx(a)xxx.com> rcpt to: <mhtest(a)list.cept.org> DATA This is a test message . Accepted by Postfix: Mar 29 11:17:17 lits01a postfix/smtpd[4088]: B86DD6A04B9: client=unknown[192.168.200.157] Mar 29 11:17:30 lits01a postfix/cleanup[4091]: B86DD6A04B9: message-id=<> Mar 29 11:17:30 lits01a postfix/qmgr[3952]: B86DD6A04B9: from=<xxx(a)xxx.com>, size=200, nrcpt=1 (queue active) Mar 29 11:17:30 lits01a postfix/lmtp[4109]: B86DD6A04B9: to=<mhtest(a)list.cept.org>, relay=127.0.0.1[127.0.0.1]:8024, delay=24, delays=24/0.01/0/0.01, dsn=2.0.0, status=sent (250 Ok) Mar 29 11:17:30 lits01a postfix/qmgr[3952]: B86DD6A04B9: removed MM receives it: Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) handling connection Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) Data: b'LHLO list.cept.org' Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) Data: b'MAIL FROM:<xxx(a)xxx.com>' Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) sender: xxx(a)xxx.com Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) Data: b'RCPT TO:<mhtest(a)list.cept.org>' Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) recip: mhtest(a)list.cept.org Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) Data: b'DATA' Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) Data: b'QUIT' Mar 29 11:17:30 2022 (3940) ('127.0.0.1', 37146) connection lost Mar 29 11:17:30 2022 (3940) Connection lost during _handle_client() But MM3 discards it: Mar 29 10:08:03 2022 (1767) DISCARD: <164854848338.1768.3954079936807939297@lits01a>; ['The message has no valid senders'] I have no idea where its goes wrong, but fear its something in the source code. Kind regards Kenneth Karlsson

2 3

Problem with redirection when entering alias without a / at the end
by Jens Günther 31 Mar '22

31 Mar '22

Hey there, I'm using Apache2 as the webserver, configured https://mailman.xyz.de as Web host. When I want to get to Mailman Starting Page now I need to Enter the following URL: https://mailman.xyz.de/mailman3/ to be directly redirected as expected to https://mailman.xyz.de/mailman3/postorius/lists/ -> works. Problem: When leave out the slash at the end and I ONLY enter the following URL: https://mailman.xyz.de/mailman3 -> then I'm wrongly redirected to a page, taht does not exist: https://mailman.xyz.de/mailman/postorius/lists/ -> 404 Where can I change this misbehaviour? In Apache-conf everything seems as it'S in the docs, for the vHost I do not have any other custom settings. Thank you in advance. By the way: Is there an easy way to redirect directly from https://mailman.xyz.de to https://mailman.xyz.de/mailman3/postorius/lists/ -> what would be a possibility for that?

4 12

Mails are stuck in Mailman Pipeline Queue
by Shashikanth Komandoor 30 Mar '22

30 Mar '22

Hi Team, I am currently running Mailman 3 with version 3.3.1 and in-built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version with default values. I am almost running 2000 lists with 64 GB RAM and 8 Cores on a single VM and on which all the mailman 3 components are running. I have been running this setup in production for the past 2 years and 6 months. I have been seeing a special scenario suddenly: most of the mails that are sent to this server are going to the pipeline queue and not getting released from there. No configuration changes are done and I'm not sure why the mails are getting stuck there. I have recreated the list but no luck. Could not identify the root cause. Please help in this regard. -- Thanks & Regards, Shashi Kanth.K 9052671936

2 5

changing default preferred language of list
by Paul Paulchen 29 Mar '22

29 Mar '22

Hi MM community, need a little advise. We're running mailman in a Docker Container (maxking) Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.10 (default, May 6 2021, 06:30:44) [GCC 9.3.0] As far as I understood, it should be pretty easy. My wish: every new list we create has a default value for the preferred language: de (instead of us) Like described here: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… We put the values in: [mailman] default_language: de "/etc/mailman.cfg" and "/opt/mailman/mailman-extra.cfg" Nothing changes. When I create a new list via Postorius, is see the preferred language is US. Am I missing something? Is the default always US? And I have to change it manually? Cheers in advance! Paul

2 1

Docker-Setup: example.com in test mails
by Bjoern Franke 29 Mar '22

29 Mar '22

Hi, we are running a docker setup with mailman-web and mailman-core. To our confusion, test mails testing for bounces are sent from example.com From: Mailman <mailman(a)example.com> To: Mailman Bounces <mailman-bounces(a)example.com> instead of the list domain and are therefore often considered as spam. The list domain should be lists.ffnw.de. Did I miss some setting? Best Regards Bjoern

3 7

users leaking across virtual hosts
by Stanisław Findeisen 25 Mar '22

25 Mar '22

Hi We are hosting several virtual hosts (mail host : web host is a 1:1 mapping with identical names), works fine, but it looks user accounts are somehow global, i.e. registering a new user in a new virtual host doesn't work if the user name or e-mail address are already in use in another virtual host. This happens on the Hyperkitty sign-up page: /accounts/signup/ , with the errors being "A user is already registered with this e-mail address." and "A user with that username already exists.". I am assuming this is a well-known limitation, which would require a serious internal redesign to fix; correct? I was able to successfully login in the browser to a vhost using my user credentials from the original vhost (the one where the account was created) (the password was not auto-filled, of course, because of a different domain name). But how safe is this? How about subscribing to lists, posting etc.? Is this going to mess things up somehow? Thanks Stanisław

2 1