July 2022 - Mailman-users - lists.mailman3.org

Mailman 3 stops delivering to a domain after too many bounces
by Julian Kippels 12 Jul '22

12 Jul '22

Hi, We have recently upgraded from Mailman 2 to Mailman 3. Since then I have observed many user reports that mails to mailing lists don't reach all members. My suspicion is that there are too many "dead" members on those lists. This was not a problem on Mailman 2, but has become a problem on Mailman 3. It seems to me that for example Mailman 3 sends mails to the first 5 existing member addresses from domain A, then encounters several (20 or so seems to be the threshold) "dead" member addresses from domain A that generate a bounce and then does not even attempt to reach the remaining 800 member adresses from domain A and only sends to members on domain B and domain C. Is there a way to globaly ignore bounces and send out mails to all members of a list, no matter what? Thanks, Julian

4 7

Notes about successfuly upgrade from python 3.7 to 3.9
by Guillermo Hernandez (Oldno7) 12 Jul '22

12 Jul '22

I've successfuly upgrade servers with mailman3 lists from FreeBSD 12.3 with python 3.7 to FreeBSD 13.1 with python 3.9 After doing the O.S. and all the python related apps upgraded I did the following steps: (note that my mailman3 installation is system-wide, pip drived and apache24/uwsgi served. The user that owns mailman3 is "mailman3" and all the stuff lies in "/usr/local/mailman3") -> su -> cd /usr/local/mailman3 -> pip install --upgrade django-mailman3 mailman mailman-hyperkitty mailmanclient postorius django hyperkitty uwsgi mod-wsgi -> mod_wsgi-express module-config (it returns the path for the apache24 httpd.conf adjustments. I edited it) -> su -m mailman3 -c "python3 manage.py collectstatic" -> su -m mailman3 -c "python3 manage.py migrate" -> su -m mailman3 -c "python3 manage.py compress" -> cd /usr/local/lib/python3.9/site-packages/ -> cd postorius -> django-admin compilemessages -> cd ../hyperkitty -> django-admin compilemessages -> cd ../django_mailman3 -> django-admin compilemessages I had to edit again /usr/local/lib/python3.9/site-packages/mistune/scanner.py and add this two lines (this is a little bit annoying) from mistune.util import escape_html from mistune.util import escape_url And that was all. Restarted the servers and all is going smoothly All of this steps have been documented in this list by Mark & Stephen whom I'm very grateful. P.S.: As I did some upgrades ago, I didn't have to modify settings.py in the Hyperkitty area to add DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' But if you come from an old mailman/Hyperkitty version, you'll have to include in it. And the it's same if you are using UWSGI, you have to include in your apache config: WSGIPassAuthorization On -- ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: firma-GHP-emails.png Replaced multipart/alternative part with first alternative.

2 2

Walk-thru for Ubuntu Bionic Mailman 3 Full setup
by kelly.close＠lrewater.com 09 Jul '22

09 Jul '22

I hope this is helpful to the Ubuntu folks out there. Once some critical small things were figured out, turns out it's pretty easy to get Mailman 3 running successfully under Ubuntu 18.04. The linked google doc includes my own walkthru, which includes using a cloud server image for Ubuntu 18.04 and a LAMP setup with Apache 2 and PHP, as well as Python and PostgreSQL, and the mailman3-full package that comes with that Ubuntu distribution. The walk-thru uses Postfix for mail and PostgreSQL for the back end database. https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plan… If you use this and see needed corrections or additions, please let me know! ~Kelly

6 21

Sorry to repeat a question, but how do I permanently delete messages, again?
by William Oliver 08 Jul '22

08 Jul '22

I askled this question some months ago, but for the life of me cannot find the response (though there is one). I want to purge some specific emails from the list archives. A user posted what he thought was a personal message to the list, and is embarrased by it, so I told him I couldn't delete emails that were already downloaded onto client machines, but I can delete it so that new users won't be able to see it. But... I can't figure out how to do it. I asked this once before, some months ago, and as I remember there is a command line way of doing it, but not a GUI way. But i can't remember what it was. Thanks! billo

2 2

Problem migrating from Ubuntu packaged Mailman to Virtualenv, including Hyperkitty
by tim＠thecutts.org 07 Jul '22

07 Jul '22

Hi everyone, As we all know, mailman3 got dropped from Ubuntu 22. So I updated the operating system anyway, and decided to switch to a virtualenv setup, keeping the old Debian/Ubuntu database and log locations etc. With some lumps and bumps, this has gone OK; the web interface appears to be working, and I can use the mailman CLI to manipulate the lists. So far so good. However, there do seem to be two problems, which might be linked: 1. If I send a message to my test list, it does not get sent on to any members. I can see the lmtp delivery to mailman. But nothing comes out again. I can see the two test messages in the “shunt” directory 2. Trying to look in the archives for my mailing lists does not work. On the web, I get "An error occurred while processing your request.”, and the log file seems to show a schema error in the hyper kitty table: ERROR 2022-07-07 13:18:41,634 116894 django.request Internal Server Error: /mailman3/archives/list/fairhaven-members(a)thecutts.org/ Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 86, in _execute return self.cursor.execute(sql, params) psycopg2.errors.UndefinedColumn: column hyperkitty_mailinglist.archive_rendering_mode does not exist LINE 1: ...e_policy", "hyperkitty_mailinglist"."created_at", "hyperkitt... ^ Do I need to run a database migration manually? If so, how? I’m not very familiar with Django, it seems a bit hideously complex to me. Are these issues likely to be related, or do I have two things going on here? Regards Tim

3 2

Problem with database change sqlite -> mysql
by moritz both 07 Jul '22

07 Jul '22

Greetings, I am totally new to this list and relatively new to mailman3 and came here because I have a problem I would like to solve soon. I hope somebody has a clue: I am running a really small list site with 10 or so lists, all of them have low traffic and no more than ~70 members. The lists are running on a shared server at uberspace. I followed their guide at https://lab.uberspace.de/guide_mailman-3/ Mailman Core Version GNU Mailman 3.3.5 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.13 (default, May 19 2022, 15:06:19) [GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] Since my migration from mm2 to mm3 I see what I consider high loads and excessive i/o and memory usage. My guess is that the main problem is the database backend for mailman, posterious and hyperkitty, which is sqlite. The database is >200MB at the moment (I have not found out why it is so huge, I tried to delete the hyperkitty archives from there and no longer archive messages, but maybe I have not managed to delete all). I take any recommandation for this situation. My current idea is changing the database backend to something else, i.e. mysql. I tried to follow this guide https://www.rockandnull.com/django-dumpdata/ but the "loaddata" command fails. The stack trace is below. Unfortunatelly, I have close to no knowledge of django and don't know why the Email matching query not exists. Will postgres make things easier? Thanks in advance for any help. Moritz Here is the stacktrace: Traceback (most recent call last): File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 173, in __get__ rel_obj = self.field.get_cached_value(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/mixins.py", line 15, in get_cached_value return instance._state.fields_cache[cache_name] KeyError: 'starting_email' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 78, in handle self.loaddata(fixture_labels) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 123, in loaddata self.load_label(fixture_label) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 190, in load_label obj.save(using=self.using) File "/home/daneben/.local/lib/python3.8/site-packages/django/core/serializers/base.py", line 223, in save models.Model.save_base(self.object, using=using, raw=True, **kwargs) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/base.py", line 763, in save_base pre_save.send( File "/home/daneben/.local/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 180, in send return [ File "/home/daneben/.local/lib/python3.8/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp> (receiver, receiver(signal=self, sender=sender, **named)) File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/signals.py", line 96, in Thread_on_pre_save kwargs["instance"].on_pre_save() File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/models/thread.py", line 157, in on_pre_save self.find_starting_email() File "/home/daneben/.local/lib/python3.8/site-packages/hyperkitty/models/thread.py", line 149, in find_starting_email if self.starting_email is not None: File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 187, in __get__ rel_obj = self.get_object(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 307, in get_object return super().get_object(instance) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 154, in get_object return qs.get(self.field.get_reverse_related_filter(instance)) File "/home/daneben/.local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get raise self.model.DoesNotExist( hyperkitty.models.email.DoesNotExist: Problem installing fixture '/home/daneben/mailman-suite/datadump.json': Email matching query does not exist.

3 5

Django errors after deleting a thread
by peter＠chubb.wattle.id.au 06 Jul '22

06 Jul '22

After deleting a thread via the web interface (because it's spam), I get lots of messages from hyperkitty E.g. Internal Server Error: /hyperkitty/list/list@domain/top-threads DoesNotExist at /hyperkitty/list/list@domain/top-threads Thread matching query does not exist. Request Method: GET Request URL: https://example.com/hyperkitty/list/list@domain/top-threads Django Version: 2.2.17 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.6 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Wed, 25 Nov 2020 07:12:00 +1100 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/lib/view_helpers.py" in inner 135. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mlist.py" in overview_top_threads 220. 'threads': mlist.top_threads, File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py" in top_threads 168. return self.cached_values["top_threads"]() File "/usr/lib/python3/dist-packages/hyperkitty/models/common.py" in __call__ 59. return self.get_or_set(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py" in get_or_set 376. return [Thread.objects.get(pk=pk) for pk in thread_ids] File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py" in <listcomp> 376. return [Thread.objects.get(pk=pk) for pk in thread_ids] File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 82. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in get 406. raise self.model.DoesNotExist( Exception Type: DoesNotExist at /hyperkitty/list/devel(a)sel4.systems/top-threads Exception Value: Thread matching query does not exist. Request information: USER: AnonymousUser GET: No GET data POST: No POST data FILES: No FILES data COOKIES: No cookie data

3 4

mailman3 suddenly stopped working after routine package upgrade with "apt update" and "apt upgrade".
by gordon＠dickens.com 05 Jul '22

05 Jul '22

My mailman3 installation is installed in Debian using the Mailman Suite's Virtualenv (venv) instructions (not the Debian mailman3 packages) and has been working great for several months until I recently executed a routine package upgrade with "apt update" and "apt upgrade" after which the mailman3 service quit working with the following error dialog in response to "systemctl start mailman3" # systemctl start mailman3 Job for mailman3.service failed because the control process exited with error code. See "systemctl status mailman3.service" and "journalctl -xe" for details. root@slc:/opt/mailman/venv/bin# systemctl status mailman3 ● mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/mailman3/mailman3.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2022-07-05 10:31:55 EDT; 4s ago Process: 3011826 ExecStart=/opt/mailman/venv/bin/mailman start (code=exited, status=2) CPU: 822ms Jul 05 10:31:54 slc.mailhub4u.com systemd[1]: Starting GNU Mailing List Manager... Jul 05 10:31:55 slc.mailhub4u.com mailman[3011826]: Usage: mailman start [OPTIONS] Jul 05 10:31:55 slc.mailhub4u.com mailman[3011826]: Try 'mailman start -h' for help. Jul 05 10:31:55 slc.mailhub4u.com mailman[3011826]: Error: A previous run of GNU Mailman did not exit cleanly (stale_lock). Try using --force Jul 05 10:31:55 slc.mailhub4u.com systemd[1]: mailman3.service: Control process exited, code=exited, status=2/INVALIDARGUMENT Jul 05 10:31:55 slc.mailhub4u.com systemd[1]: mailman3.service: Failed with result 'exit-code'. Jul 05 10:31:55 slc.mailhub4u.com systemd[1]: Failed to start GNU Mailing List Manager. Then, the command, "systemctl status mailman3" results in: # systemctl status mailman3 ● mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/mailman3/mailman3.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2022-07-05 10:31:55 EDT; 19min ago Process: 3011826 ExecStart=/opt/mailman/venv/bin/mailman start (code=exited, status=2) CPU: 822ms Jul 05 10:31:54 slc.mailhub4u.com systemd[1]: Starting GNU Mailing List Manager... Jul 05 10:31:55 slc.mailhub4u.com mailman[3011826]: Usage: mailman start [OPTIONS] Jul 05 10:31:55 slc.mailhub4u.com mailman[3011826]: Try 'mailman start -h' for help. Jul 05 10:31:55 slc.mailhub4u.com mailman[3011826]: Error: A previous run of GNU Mailman did not exit cleanly (stale_lock). Try using --force Jul 05 10:31:55 slc.mailhub4u.com systemd[1]: mailman3.service: Control process exited, code=exited, status=2/INVALIDARGUMENT Jul 05 10:31:55 slc.mailhub4u.com systemd[1]: mailman3.service: Failed with result 'exit-code'. Jul 05 10:31:55 slc.mailhub4u.com systemd[1]: Failed to start GNU Mailing List Manager. The only reason that I believe that the apt package upgrade caused the problem is because everything stopped working immediately after the package upgrade commands, "apt update" and "apt upgrade" were executed. Looking at the apt logs, the following packages were apparently upgraded: Start-Date: 2022-07-01 13:18:17 Commandline: apt upgrade Requested-By: gordon (1000) Install: linux-image-5.10.0-15-amd64:amd64 (5.10.120-1, automatic) Upgrade: tzdata:amd64 (2021a-1+deb11u3, 2021a-1+deb11u4), linux-image-amd64:amd64 (5.10.113-1, 5.10.120-1), libssl-dev:amd64 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), libssl-doc:amd64 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), libssl1.1:amd64 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), openssl:amd64 (1.1.1n-0+deb11u2, 1.1.1n-0+deb11u3), linux-libc-dev:amd64 (5.10.113-1, 5.10.120-1) End-Date: 2022-07-01 13:19:04 Start-Date: 2022-07-01 13:21:18 Commandline: apt autoremove Requested-By: gordon (1000) Remove: linux-image-5.10.0-13-amd64:amd64 (5.10.106-1) End-Date: 2022-07-01 13:21:22 Otherwise, other than this package upgrade command, nothing changed on my system between the time that mailman3 was working properly and when it stopped working. Does anyone see any potential problems with any of the packages that were upgraded? The logfile, /var/log/syslog, has the following lines immediately after the "systemctl start mailman3" command was executed: Jul 5 11:07:41 slc mailman[3030034]: Usage: mailman start [OPTIONS] Jul 5 11:07:41 slc mailman[3030034]: Try 'mailman start -h' for help. Jul 5 11:07:41 slc mailman[3030034]: Error: A previous run of GNU Mailman did not exit cleanly (stale_lock). Try using --force Jul 5 11:07:41 slc mariadbd[859]: 2022-07-05 11:07:41 1744540 [Warning] Aborted connection 1744540 to db: 'mailman3' user: 'mailman' host: 'localhost' (Got an error reading communication packets) Jul 5 11:07:41 slc systemd[1]: mailman3.service: Control process exited, code=exited, status=2/INVALIDARGUMENT Jul 5 11:07:41 slc systemd[1]: mailman3.service: Failed with result 'exit-code'. As an FYI, I have attempted to start the service using --force, however, it made no difference. BTW, I don't know if this is significant or not but the following error message (or similar message) is displayed often in the logs: slc mariadbd[859]: 2022-07-05 11:07:41 1744540 [Warning] Aborted connection 1744540 to db: 'mailman3' user: 'mailman' host: 'localhost' (Got an error reading communication packets) In any event, does anyone see any other potential causes for why the mailman3 service will not run? Thanks very much! Gordon

2 2

Monthly mailman-web job using a lot of memory
by Andy Smith 04 Jul '22

04 Jul '22

Hi, Is it normal for the monthly job: @monthly www-data [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.monthly /usr/share/mailman3-web/manage.py runjobs monthly to use nearly 3½GiB of memory? PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 770203 www-data 20 0 4516756 3.4g 1.6g R 99.7 43.5 12:08.00 python3 /usr/share/mailman3-web/manage.py runjo+ I assume it's updating the full text indexes or something. There are around 250k messages in our archives. In the job list I see: appname - jobname - when - help -------------------------------------------------------------------------------- hyperkitty - update_and_clean_index - monthly - Update the full-text index and clean old entries so I'm guessing it's that, but there is also: hyperkitty - update_index - hourly - Update the full-text index which hasn't been noitceably a problem so I wonder why the "and clean old entries" bit is so much more heavyweight, what exactly it dopes and if there's any way to make it lighter on resources? Thanks, Andy

3 7

mailman3 + postfix + dovecot + postfixadmin
by Geoff Nordli 03 Jul '22

03 Jul '22

Hi. Did anyone figure out the issue or have a working configuration with this: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… I am getting the same error, with the same configuration. Issue: Postfix LMTP does not pass messages onto MM3:LMTP. Instead the behavior summary is as follows: Jul 3 11:56:30 mail1 postfix/lmtp[824956]: 5168B20017A: to=<lmtp:[127.0.0.1]:8024@domain.com>, orig_to=<testing(a)domain.com>, relay=mail1.domain.com[private/dovecot-lmtp], delay=0.08, delays=0.03/0.01/0.02/0.01, dsn=5.1.1, status=bounced (host mail1.domain.com[private/dovecot-lmtp] said: 550 5.1.1 <"lmtp:[127.0.0.1]:8024"@domain.com> User doesn't exist: lmtp:[127.0.0.1]:8024@domain.com (in reply to RCPT TO command)) thanks, Geoff

2 2