April 2022 - Mailman-users - lists.mailman3.org

Problems with shunted Mails
by Stephan Krinetzki April 25, 2022

April 25, 2022

Hi all, we currently have a problem sending to an email list. The mailing list is an announcement list and is therefore moderated. The mail also arrives for moderation. However, when the mail is released, it is not delivered, but moved to the shunt queue. The error after an unshunt attempt in the log: Apr 19 12:31:43 2022 (17967) Uncaught runner exception: 'NoneType' object has no attribute 'email' Apr 19 12:31:43 2022 (17967) Traceback (most recent call last): File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/core/runner.py", line 176, in _one_iteration self._process_one_file(msg, msgdata) File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/core/runner.py", line 269, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/core/pipelines.py", line 50, in process handler.process(mlist, msg, msgdata) File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/handlers/member_recipients.py", line 84, in process for member in mlist.regular_members.members File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/handlers/member_recipients.py", line 85, in <genexpr> if member.delivery_status == DeliveryStatus.enabled) AttributeError: 'NoneType' object has no attribute 'email' Apr 19 12:31:43 2022 (17967) SHUNTING: 1650364303.8361175+08bb48ae877019e8a4c9dff8024c1270782fceb3 Any hints how we can solve the problem here?

2 4

set a new default for max_message_size
by Paul Paulchen April 24, 2022

April 24, 2022

Hi MM Community, I guess Mark can answer this question in a split second. We want another default than 40kb for the max_message_size. I know you can change it with the mailman shell or with postorius. But we want to skip this part an set the default to 4000kb for instance. I read about the max_message_size in this thread from 2020 https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… According to the answer of Mark you can change the "/usr/lib/python3/dist-packages/mailman/styles/base.py" - mlist.max_message_size. Now time went by, is there a possibility to use the mailman.cfg to set a new default? Cheers and thanks in advance!

3 2

How to delete large numbers of held messages
by lists＠g1fef.co.uk April 22, 2022

April 22, 2022

We had a mail loop on one of our lists and ended up with over 51k messages in pending (held due to post not form a list member) Trying to delete them from the web UI will take forever, as the largest pagination you can view is 200 at a time. Tried to delete them from the Python interactive shell, but it looks like the API times out after a while. Is there another way to delete all these unwanted held posts? The problem comes after executing: <block> reqs = list(adminlist.held) </block> After a couple of minutes it comes back with: <block> Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.9/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.9/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 718, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.9/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.9/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 107, in call response = request( File "/usr/lib/python3/dist-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py", line 123, in held response, content = self._connection.call( File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 135, in call raise MailmanConnectionError( mailmanclient.restbase.connection.MailmanConnectionError: ('Could not connect to Mailman API: ', "ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))") </block>

2 4

Apple Mail encoding again
by David Newman April 22, 2022

April 22, 2022

Starting a new thread because I don't know how to reply to this one from November 2021: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… A subscriber's post to a list bounced with a 501 bad encoding error. The subscriber uses Apple Mail on MacOS and PGP-signed his message. The earlier post above suggests the problem could be a bug in either Apple Mail encoding or python's smtp module. To rule out the smtp module, I sent a PGP-signed message using a different MUA (Thunderbird, in my case). Mailman3 accepted this post without complaint, and I reported back to list users that this appears to be an Apple Mail encoding issue. Normally, that'd be the end of it. But this is a list for a bunch of Internet graybeards, including one of the authors of the SMTP protocols and the subscriber is a former IETF chair. He's grumbling that if this really were an Apple Mail issue, wouldn't it be far more widespread? I don't think it would be, because (a) Mailman2 is still far more pervasive than Mailman3 and (b) the combination of PGP signing and Apple Mail on MacOS and Mailman3 is still pretty unusual. Has anyone else hit this encoding issue, other than the post above? And did Mailman2 silently accept this because it was less strict about checking for bad encoding? Thanks. dn

6 23

About mailing list configuration problem
by liutianhua68＠163.com April 20, 2022

April 20, 2022

Hello: I installed mailman3 through docker, and I created a mailing list for testing, which can subscribe, unsubscribe, receive emails, and send emails normally. Now there are two problems that have not been resolved. I would like to ask: 1. When a user subscribes, how to configure it like "lists.mailman3.org" to send a link to confirm the subscription instead of just replying to the confirmation email. 2. I used the mailling list I created to test the sending and receiving of emails, but I didn't see any historical messages in "Archives",Where is the configuration incorrect? And the url of my mailing list is "https://list.example.org/hyperkitty/list/" instead of "https://lists.mailman3.org/archives/list/", is this a problem? Also ask about the configuration, where to find the relevant documentation. thank you very much. liutianhua68(a)163.com

3 2

Prometheus mailman_exporter
by Ruth Ivimey-Cook April 19, 2022

April 19, 2022

Folks, I have recently been playing with Prometheus metrics, and having now coded a mailman3 Prometheus exporter in python I thought I'd mention here that it exists. It's not fancy and there is more to do. It works using the rest api. Source available at https://github.com/rivimey/mailman3_exporter Metrics exposed: 'mailman_up' (does the API /system/versions return http 200) and 'mailman_queues' (queue lengths for each queue, plus 'status' for the http status code of that query) Suggestions for other metrics would be interesting if accompanied with info on how to extract it. I'm currently contemplating including the number of 'to be moderated' messages. It would be possible to include the number of lists or domains but not sure it's useful enough to be worth the cost of fetching it. It is passed the api auth data separately. It would be better to read it from mm config: is there existing code I can call? Regards, Ruth -- Software Manager & Engineer Tel: 01223 414180 Blog: http://www.ivimey.org/blog LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/

2 1

How to create my first mailing list
by liutianhua68＠163.com April 19, 2022

April 19, 2022

Hello: I have installed mailman3 with docker, configured "MAILMAN_ADMIN" in docker-compose.yaml, but after logging into web with this user, there is nowhere to create mailing list. How should I create a super admin account now or where to create the first mailing list, is there any reference documentation, please help, thanks. liutianhua68(a)163.com

2 1

MAILMAN_ARCHIVER_KEY - 401 error
by Torsten R. April 14, 2022

April 14, 2022

Hi together, I just updated one of my servers to Debian 11 and that also introduced mailman3 to the system as I was running mailman 2.1 for years before on that server. It took me a while to configure everything but I am convinced that I now have everything done properly. I am able to access the web interface of mailman3 (postorius) and also the archive (hyperkitty). I configured exim to deal with messages send to the lists. But when I try to send a message (as a member of a list) to a list I always get a 401 in mailman.log and the message that I should check if I provided MAILMAN_ARCHIVER_KEY and if it is correct. So far, I can say that the key is the same in mailman-web.py (with single quotes) and mailman-hyperkitty.cfg (without quotes). in mailman-web.py I have: MAILMAN_ARCHIVER_KEY = 'somesecretkeywithlotsofcharacters' and in mailman-hyperkitty I have this: api_key: somesecretkeywithlotsofcharacters I have some messages in $var_dir/archives/hyperkitty/spool but mailman qfile <messagefile> doesn't show anything that would catch my eye. So, maybe I need another pair of eyes that could take a look at my configs or someone that is turning me in the right direction and might give some hints where to look and what to check. Thanks, Torsten

2 8

Lessons learned and a couple little gotcha for newbies
by William Oliver April 13, 2022

April 13, 2022

I just finished, with great help from Mark Sapiro on the list, a new installation of mailman3. I used the instructions at https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/insta… and https://docs.mailman3.org/en/latest/install/virtualenv.html There were a couple of gotchas that I didn't see well documented. 1) I got the error django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/venv/lib/python3.9/site- packages/mistune/scanner.py) when I ran 'mailman-web migrate' The fix was trivial: I just added the lines: from mistune.util import escape_html from mistune.util import escape_url to /opt/mailman/venv/lib/python3.9/site-packages/mistune/scanner.py It's not hard to search for those definitions, but if someone doesn't know a smattering of python, it might not be obvious. 2) Similarly, migrate (using postgres) also gave the error: File "/opt/mailman/venv/lib/python3.9/site- packages/django/db/backends/postgresql/utils.py", line 6, in utc_tzinfo_factory raise AssertionError("database connection isn't set to UTC") I was able to find a discusson of this one at https://stackoverflow.com/questions/68024060/assertionerror-database-connec… which had me downgrade to an older version of psycopg2-binary: pip install psycopg2-binary==2.8.6 There might be a different fix, but this worked. 3) When I ran the test "uwsgi --ini /etc/mailman3/uwsgi.ini" I got the error: ModuleNotFoundError: No module named 'settings' This was also trivially fixed by adding the path: export PYTHONPATH=$PYTHONPATH:/etc/mailman3 in /opt/mailman/.bashrc 4) Sometimes when I rebooted the machine, when I ran 'systemctl start mailman3' I'd get the error that mailman wouldn't start because the lock was stll there. I never could figure out where the lock was, so I worked around it by replacing ExecStart=/opt/mailman/venv/bin/mailman start with ExecStart=/opt/mailman/venv/bin/mailman start --force in /etc/systemd/system/mailman3.service I'm sure there's something bad about doing that, but it works for me, so far. 5) I had a problem with mail to outside people not being delivered while mail to the local host was delivered. It turned out that this was because in my mailman.cfg I had "smtp_host:my.listserver.tld when it should have been localhost. The error was "relay access denied." To fix this, I had to add my.listserver.tld to "mynetworks" in /etc/postfix/main.cf, ie change: mynetworks = 127.0.0.0/8 10.0.0.0/24 to mynetworks = 127.0.0.0/8 10.0.0.0/24 111.222.333.444 <--- my ip address However, Mark Sapiro (a God) told me this was because I had changed a line in the [mta] section of mailman.cfg. I had changed smtp_host: localhost to smtp_host: mydomain.tld Here's the thing. The reason I did this was because the instructions at https://docs.mailman3.org/en/latest/config-core.html give this as their model: [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: mail.example.com lmtp_port: 8024 smtp_host: mail.example.com <---- not localhost smtp_port: 25 The instructions are good, but there are a few places where one size does not fit all, I guess. 6) A second error in delivery to outside addresses resulted in an "Access denied" error. A search on the internet revealed that this had to do with (for me) smtpd_recipient_restrictions in postfix. See, for instance: https://know.mailsbestfriend.com/postfix_error_relay_access_denied_noqueue_… Changing this fixed things. I'm not sure, but I think this shows up mostly with installations that include dovecot. 7) If you are using virtual domains in postfix, you need to use the "unusual postfix configuration" in https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h… virtual_alias_maps = hash:/path-to-mailman/var/data/postfix_vmap 8) You need to manually set the local virtual host for mailman3. Somehow, I got the impression from the installation instructions that this was done automagically to the default x.mydomain.tld if your domain is mydomain.tld. It doesn't. 9) Follow the instructions verbatim, and follow them in the order provided. I installed postfix and nginx out of order, and it did OK, but once you get into the python virtual environment, it is easy to miss something if you don't do it in order. 10) Jot down lots of notes. This is one of the more complicated installations I've done, mostly because it involved configuring a zillion packages to talk to each other: Postfix, nginx, postorius, hyperkitty, django, and others (I'd never heard of "whoosh" or "sass" for instance). All of these can be unforgiving with configuration errors. The default installation instructions assumed a number of packages I'd never used. I'm an apache/dovecot/mysql kind of person, and I had to switch to nginx/postgres to use the instructions as the written defaults. I'm sure I could have used apache and mysql, but it would mean trying to figure out what didn't work without following the installations verbatim. I'd never even heard of postorius, hyperkitty, or django. So, it was easy to get confused a bit. Taking notes of changes I made was important in being able to back out of disastrous choices. Plus, once I'd installed it on one machine, I sat down and installed it on another machine, and it was very useful to have written down how I fixed an issue when it came up again -- and I didn't remember exactly what I had done. 11) Make multiple sequential backups of configuration files. I went down a couple of paths I shouldn't have gone down. For instance, with the vmap issue, I got it in my head that it was a postfix configuration issue, not a mailman issue. So I kept changing things in postfix. Once I was told that it was in *mailman* I had to set the virtual host, I needed to back out of about 8 tweaks I'd made to main.cf in /etc/postfix. If I hadn't been making main.cf.backup.01, main.cf.backup.02, etc. I would have had to spend another few hours trying to remember what inappropriate changes I had made. This is particularly true for me because I am not a great typist, and when I try to retype something to back out, there's a reasonable chance I'll mistype something and it still won't work. With these backups, I don't have both manually back out of things *and* proof my typing again and again. It's a lot easier to delete backups once things are working than to try to remember what you did late at night after a couple of bourbons. 12) Choose one set of installation instructions and follow it. Don't try to mix instructions from different tutorials unless you really know what you are doing. I didn't. For instance, there's a different set of mailman3 installation instructions at: https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 It does things a little differently. It's useful to look at that when you are debugging why something's not working, but you can get into some trouble trying to do both installations at the same time -- which I found myself doing when something wasn't working. It didn't help. billo

3 6

Issue with send mail to my domain in mailman3
by Paul Ssekamatte April 12, 2022

April 12, 2022

I get this error in mail.log when I try to send mail to my domain Recipient address rejected: SMTP AUTH is required for users under this sender domain (in reply to RCPT TO command)). My setup is such that the server hosting the mailman3 (lists@tld) lists is different from the mail server (mail@tld) Please advise.

2 3