December 2022 - Mailman-users - lists.mailman3.org

Strange messages after upgrading Mailman3
by Allan Hansen Dec. 6, 2022

Dec. 6, 2022

Hi all, I have been getting a few of these messages after my Mailman 3 was upgraded: "The content of this message was lost. It was probably cross-posted to multiple lists and previously handled on another list." The messages appear to come from list members and are distributed to the relevant lists, but with nothing of the original contents. They have appeared on different lists with, apparently different senders (unless spoofed). Other messages are going through the system unchanged. Anyone has any ideas? Yours, Allan

2 5

Issues encountered while upgrading to 3.3.7
by Odhiambo Washington Dec. 5, 2022

Dec. 5, 2022

Hello everyone, I am starting a new thread on this so that the issues I encountered do not get mixed up with the other discussions. Over the weekend I upgraded an MM3 installation to the latest versions of the components. I started off by creating a new venv, installing and then migrating the configurations and the scripts. In the end, I ended up with MM3 Core ==3.3.7, postorius==1.3.7, HyperKitty==1.3.6, django==4.1.3. 1. SQLAlchemy - At the stage where one runs 'mailman info', I encountered the first error: : sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:postgres This is solved by a change in mailman.cfg so that the URI starts with postgresql:// instead of postgres://. SQLAlchemy used to accept both but has removed support for the postgres name. <https://github.com/sqlalchemy/sqlalchemy/issues/6083#issuecomment-801478013> 2. Django==4.x At the point of running the post-update processes, particularly the "migrate" bit, I got the following error: : ImportError: cannot import name 'url' from 'django.conf.urls' (/opt/mailman/mm/venv/lib/python3.9/site-packages/django/conf/urls/__init__.py) I understand this is a consequence of the django.conf.urls.url() having been deprecated starting version 3 and has been removed in version 4 of Django. https://docs.djangoproject.com/en/4.1/releases/4.0/#features-removed-in-4-0 The upgrade process seems to have brought in the newer version of Django because now I have 4.1.3 up from 3.x If anyone wants, I found an elaborate explanation at https://bobbyhadz.com/blog/python-importerror-cannot-import-name-url-from-d… . My Google-fu found suggestions that in urls.py, I should import re_path and use it, or alias it for url for the migrate to work. The solution I used was to alias url to re_path, but I am considering just doing away with url altogether. Not sure which is the better option between using re_path or path. So here is the patch I have used: (venv) mailman@lists:~/mm$ diff -u urls.py.bak urls.py --- urls.py.bak 2022-12-03 12:20:28.028090659 +0300 +++ urls.py 2022-12-03 12:36:20.298875624 +0300 @@ -17,7 +17,8 @@ # Postorius. If not, see <http://www.gnu.org/licenses/>. -from django.conf.urls import include, url +from django.conf.urls import include +from django.urls import re_path as url from django.contrib import admin from django.urls import reverse_lazy from django.views.generic import RedirectView 3. The last issue that I encountered is one I kind of solved, but completely don't understand the consequences of the solution I applied: /opt/mailman/mm/bin/django-admin migrate <CUT> System check identified some issues: WARNINGS: django_mailman3.MailDomain: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the DjangoMailman3Config.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. django_mailman3.Profile: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the DjangoMailman3Config.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Attachment: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Email: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Favorite: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.LastView: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.MailingList: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Profile: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Tag: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Tagging: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Thread: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.ThreadCategory: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. hyperkitty.Vote: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the HyperKittyConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. postorius.EmailTemplate: (models.W042) Auto-created primary key used when not defining a primary key type, by default 'django.db.models.AutoField'. HINT: Configure the DEFAULT_AUTO_FIELD setting or the PostoriusConfig.default_auto_field attribute to point to a subclass of AutoField, e.g. 'django.db.models.BigAutoField'. Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount </CUT> Here again, Google-fu came to the rescue. I added the following line to settings_local.py and the warnings were suppressed. DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' I need to understand how this affects everything and how this is likely to impact the functionality of MM3. What is the proper solution for this? And should it be django.db.models.BigAutoField or django.db.models.AutoField? Which is which? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

3 5

Handling of spam
by Eggert Ehmke Dec. 5, 2022

Dec. 5, 2022

I have a mailing list that is open to nonmembers. Sometimes we get spam on that list. I have a working spamassassin/amavis spam filter on the system. Now comes the strange part: sometimes a spam message is put into a multi-part message and forwarded to the list owner (me). This multi-part message contains the message marked as spam, but is not marked as spam itself. The origin of those message is the bounces address of the list. So my first question is: where can those messages be generated? I can provide the message header, but it would contain a lot of details of my server. Cheers, Eggert

2 1

Is there an easy way to set Reply-To to a lists owner?
by Martin Lorenz Dec. 4, 2022

Dec. 4, 2022

Hi again, now, that Mailman3 is running smoothly I have a new question. We have a bunch of lists, which are only used for spreading news. Now the admin of theses lists (who happens to be my wife) asked me, if it was possible to change the settings such that all replys to one of these lists can only be sent to herself (i.e. the owner address) instead of being held for moderation. I thought of setting Reply-To to something like <list-owner> by "simply" entering a Variable into this field, instead of using the admins/owners literal email-address. Is this possible? Or is there another way of achieving the goal? TNX Greetings Martin

2 1

Weird startup problem after upgrade
by Odhiambo Washington Dec. 3, 2022

Dec. 3, 2022

I am facing an odd situation in which MM3 doesn't start successfully on reboot, but I can start it manually. This is a venv install. This started after I did an upgrade (pip install --upgrade mailman postorius hyperkitty mailman-hyperkitty) followed by the requisite collectstatic|compress|compilemessages|migrate post-update commands. Secondly, it appears that the upgrade did NOT change the versions for the upgraded components - postorius=1.3.6, hyperkitty=1.3.5, Mailman Core Version=3.3.5. I was expecting the version to all bump up to the latest. Thanks in advance for any insights into what could be causing this. wash@lists:~$ systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: failed (Result: timeout) since Fri 2022-12-02 18:58:13 EAT; 9s ago CPU: 25.241s Dec 02 18:58:12 lists mailman[1637]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1639]: for atexit_call in reversed(_threading_atexits): Dec 02 18:58:12 lists mailman[1639]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1637]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1639]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1637]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:12 lists mailman[1639]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:13 lists systemd[1]: mailman3.service: Failed with result 'timeout'. Dec 02 18:58:13 lists systemd[1]: Failed to start GNU Mailing List Manager. Dec 02 18:58:13 lists systemd[1]: mailman3.service: Consumed 25.241s CPU time. root@lists:/home/wash# systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: *failed (Result: timeout) since Fri 2022-12-02 18:58:13 EAT; 53s ago* CPU: 25.241s Dec 02 18:58:12 lists mailman[1637]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1639]: for atexit_call in reversed(_threading_atexits): Dec 02 18:58:12 lists mailman[1639]: File "/opt/mailman/mm/venv/lib/python3.10/site-packages/mailman/core/runner.py", line 114, in signal_handler Dec 02 18:58:12 lists mailman[1637]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1639]: raise RunnerInterrupt Dec 02 18:58:12 lists mailman[1637]: mailman.interfaces.runner.RunnerInterrupt: Dec 02 18:58:12 lists mailman[1639]: mailman.interfaces.runner.RunnerInterrupt: *Dec 02 18:58:13 lists systemd[1]: mailman3.service: Failed with result 'timeout'.Dec 02 18:58:13 lists systemd[1]: Failed to start GNU Mailing List Manager.Dec 02 18:58:13 lists systemd[1]: mailman3.service: Consumed 25.241s CPU time.* root@lists:/home/wash# cd /opt/mailman/mm/var/logs/ root@lists:/opt/mailman/mm/var/logs# ls -al total 104948 drwxr-xr-x 2 mailman mailman 4096 Dec 2 18:56 . drwxr-xr-x 12 mailman mailman 4096 Dec 2 18:58 .. -rw-r--r-- 1 mailman mailman 7720631 Dec 2 18:52 access.log -rw-rw---- 1 mailman mailman 1514 Nov 28 05:58 bounce.log -rw-rw---- 1 mailman mailman 0 Aug 15 16:56 debug.log -rw-r--r-- 1 mailman mailman 497 Dec 2 18:56 error.log -rw-rw---- 1 mailman mailman 99339919 Dec 2 18:58 mailman.log -rw-r--r-- 1 mailman mailman 0 Dec 2 18:53 mailmansuite.log -rw-rw---- 1 mailman mailman 0 Aug 15 16:56 plugins.log -rw-rw---- 1 mailman mailman 374961 Dec 2 18:56 smtp.log root@lists:/opt/mailman/mm/var/logs# less error.log root@lists:/opt/mailman/mm/var/logs# cd ../locks/ root@lists:/opt/mailman/mm/var/locks# ls -al total 8 drwxr-xr-x 2 mailman mailman 4096 Dec 2 18:58 . drwxr-xr-x 12 mailman mailman 4096 Dec 2 18:58 .. root@lists:/opt/mailman/mm/var/locks# systemctl start mailman3 root@lists:/opt/mailman/mm/var/locks# systemctl status mailman3 ● mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2022-12-02 19:00:15 EAT; 11s ago Process: 2395 ExecStart=/opt/mailman/mm/bin/mailman start (code=exited, status=0/SUCCESS) Main PID: 2397 (python3) Tasks: 18 (limit: 4550) Memory: 791.9M CPU: 21.440s CGroup: /system.slice/mailman3.service ├─2397 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/master -C /opt/mailman/mm/mailman.cfg ├─2398 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=archive:0:1 ├─2399 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=bounces:0:1 ├─2400 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=command:0:1 ├─2401 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=in:0:1 ├─2402 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=lmtp:0:1 ├─2403 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=nntp:0:1 ├─2404 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=out:0:1 ├─2405 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=pipeline:0:1 ├─2406 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 ├─2407 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=retry:0:1 ├─2408 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=task:0:1 ├─2409 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=virgin:0:1 ├─2410 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=digest:0:1 ├─2423 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 └─2424 /opt/mailman/mm/venv/bin/python3 /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 Dec 02 19:00:13 lists systemd[1]: Starting GNU Mailing List Manager... Dec 02 19:00:14 lists mailman[2395]: Starting Mailman's master runner Dec 02 19:00:14 lists mailman[2395]: Generating MTA alias maps *Dec 02 19:00:14 lists systemd[1]: mailman3.service: Can't open PID file /opt/mailman/mm/var/master.pid (yet?) after start: Operation not permitted* Dec 02 19:00:15 lists systemd[1]: Started GNU Mailing List Manager. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)

2 4

Permission denied for fulltext_index
by Jan Eden Dec. 1, 2022

Dec. 1, 2022

Hi, when trying to reattach a message thread in the Hyperkitty web interface, the following error occurs: > Internal Server Error: /archives/list/testlist@lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > > PermissionError at /archives/list/testlist@lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > [Errno 13] Permission denied: 'fulltext_index' The error occurs while haystack attempts to create a directory – > File "/opt/mailman/venv/lib/python3.10/site-packages/haystack/backends/whoosh_backend.py", line 412, in search > self.setup() > File "/opt/mailman/venv/lib/python3.10/site-packages/haystack/backends/whoosh_backend.py", line 133, in setup > os.makedirs(self.path) > File "/usr/lib/python3.10/os.py", line 225, in makedirs > mkdir(name, mode) > > Exception Type: PermissionError at /archives/list/testlist@lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > Exception Value: [Errno 13] Permission denied: 'fulltext_index' > Raised during: hyperkitty.views.thread.reattach_suggest > Request information: > USER: mailman – but the existing directory /opt/mailman/fulltext_index (and its contents) are readable and writeable by mailman:mailman: > drwxrwxr-x 2 mailman mailman 4096 Nov 27 08:00 fulltext_index/ > > -rw-rw-r-- 1 mailman mailman 4473 Nov 27 08:00 _MAIN_65.toc > -rw-rw-r-- 1 mailman mailman 15456 Nov 27 08:00 MAIN_bc0cs2aohovftat6.seg > -rw-rw-r-- 1 mailman mailman 15454 Nov 27 06:00 MAIN_brbo8i8edmlz0za4.seg > -rw-rw-r-- 1 mailman mailman 15462 Nov 27 07:00 MAIN_ewnemc7lsnodd3y7.seg > -rw-rw-r-- 1 mailman mailman 157324 Nov 27 04:00 MAIN_qag60g1rrumshtc2.seg > -rw-rw-r-- 1 mailman mailman 15459 Nov 27 05:00 MAIN_w6e0ohurasusy61d.seg > -rwxrwxr-x 1 mailman mailman 0 Nov 24 16:00 MAIN_WRITELOCK* I fail to see where haystack tries (and fails) to create the directory. FWIW, deleting threads is not a problem. Thanks, Jan

2 10