June 2021 - Mailman-users - lists.mailman3.org

How to identify the exact reason behind the shunting of the mails ?
by Shashikanth Komandoor June 20, 2021

June 20, 2021

Hi Team, I am currently running Mailman 3 with version 3.3.1 and in-built postfix version 2.10.1-6 on RHEL 7.5 with PostgreSQL 11.7 version with default values. I am almost running 2000 lists with 128 GB RAM and 24 Cores on a single VM and on which all the mailman 3 components are running. I have been running this setup in production for the past 2 years and 6 months. From the beginning of the above setup until recent days, I observed a few mails moving to the shunt queue now and then. But from recent days it is observed that many mails are moving to the shunt queue. I browsed many websites but I could not get a proper resolution for identifying the exact reason behind the shunting of the mails. Because of which I had been asking the user to resend the mails instead of suggesting the user regarding the issue properly. I in fact used the command "mailman qfile <filename.pck>" even then I don't understand what might be wrong with the mail composition. Most of the time I see *unknown-8bit* in To or CC fields and I don't know if it is linked with the reason for shunting or not and I don't get any clue from it either. Is there any chance to find out the reason for shunting of the mails through any means ? -- Thanks & Regards, Shashi Kanth.K 9052671936

2 6

ModuleNotFoundError: No module named 'settings'
by Michela June 18, 2021

June 18, 2021

Hello! I've just set up Mailman 3 with Hyperkitty and Postorius in venv via pip by using the guide at https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst… on Oracle Linux 8.3 with Python 3.6.8. The process seemed to go smoothly and without errors. The log file, /opt/mailman/web/logs/uwsgi-error.log, contains the following stack trace, however. Note the bit toward the bottom: "ModuleNotFoundError: No module named 'settings'" Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 224, in fetch_command app_name = commands[subcommand] KeyError: 'qcluster' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman-web", line 8, in <module> sys.exit(main()) File "/opt/mailman/venv/lib64/python3.6/site-packages/mailman_web/manage.py", line 30, in main execute_from_command_line(sys.argv) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line utility.execute() File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 395, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 231, in fetch_command settings.INSTALLED_APPS File "/opt/mailman/venv/lib64/python3.6/site-packages/django/conf/__init__.py", line 76, in __getattr__ self._setup(name) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/conf/__init__.py", line 63, in _setup self._wrapped = Settings(settings_module) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/conf/__init__.py", line 142, in __init__ mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 994, in _gcd_import File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'settings' [uwsgi-daemons] throttling "/opt/mailman/venv/bin/mailman-web qcluster" for 245 seconds [uwsgi-daemons] respawning "/opt/mailman/venv/bin/mailman-web qcluster" (uid: 1002 gid: 1002) Mailman and mailman-web do start without errors when I run them via their systemd unit files and Postorius serves up pages (without styles and such, which I'll fix later). SELinux is disabled, at least while I work through this issue. I've gone over the configuration files, and everything appears to be in order as written in the installation instructions, but that's likely not the case. :) My configuration files outside of the Mailman virtual environment are in /etc/mailman/: (venv) [mailman@host ~]$ ls /etc/mailman3/ mailman.cfg __pycache__ settings.py uwsgi.ini When searching the web about this, I only found one result, on this very list, but it didn't have quite enough information for me to solve my issue. It might also not quite be the same problem: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… Might you be able to point me in the right direction on solving this issue? Please tell me if there is any more information I can provide to that end. Best, Michela

7 9

Mailman - delete or backup old archived messages
by olenkoigetublim＠gmail.com June 18, 2021

June 18, 2021

Hello Mailman team! I have been using Mailman-Suite (Hyperkitty + Postorius) quite a long time now and now i was wondering if there is any way to erase or export old archived messages and back them up elsewhere (for example messages, that are older then 1 year). Didnt find any suitable option under "python3 manage.py" command and also didnt find any up to date post which related to mailman3 and python3. Additional info: *Database: Postgresql13 *Mailman version: 3.3.4 *Python version: 3.6 Br, Dim

2 1

Mailman archive issues (Hyperkitty)
by klillipuu＠gmail.com June 16, 2021

June 16, 2021

Hello All! I hope its a right place to write my concern about Mailman Hyperkitty. I have same quite odd issues lately and since I'm less experienced with Mailman, any help is welcome. So i have like hybrid version pip installation, where mailman v3 is using python3 and hyperkitty is using python2 (if im correct). I also mention, that hyperkitty is a pretty old version - 1.1.4. The main issue is that if i try to open some large-scale archived list, then i get "504 Gateway Time-out" error on webUI. Any ideas, where could i start improving the situation - should i try to upgrade some packages or do i need to give more resources to host ? Extra information: * I have also noticed some memory violation issues which has caused by uwsgi. * Segfaulting process - kernel: python[25870]: segfault at 24 ip 00007f9decb9a6ea sp 00007ffc413bb3d0 error 6 in libpython2.7.so.1.0[7f9decb10000+17e000] Output from mailman.log: HyperKitty failure on http://localhost/hyperkitty/api/mailman/archive: <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx/1.16.1</center> </body> </html> (504) Exception in the HyperKitty archiver: <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx/1.16.1</center> </body> </html> (1808) Traceback (most recent call last): File "/usr/lib/python3.5/site-packages/mailman_hyperkitty/__init__.py", line 154, in _archive_message url = self._send_message(mlist, msg) File "/usr/lib/python3.5/site-packages/mailman_hyperkitty/__init__.py", line 210, in _send_message raise ValueError(result.text) ValueError: <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx/1.16.1</center> </body> </html>

5 13

#mailman irc channel moving to libera.chat
by Abhilash Raj June 15, 2021

June 15, 2021

Hey Everyone, As some of you might already know that Freenode IRC server now has new owners and most of the volunteer staff have decided to move away from it to create a new IRC server libera.chat[1] due to the policies of the new owners. Along with many other open source projects, we have also decided to move away from freenode to use libera.chat instead. We have registered #mailman there and will be using that as primary IRC channel for Mailman things. Freenode has also decided to drop all registrations, so, #mailman channel there is no longer registered to any member of Mailman Core team as of some time earlier today (Tue June 15th). [1]: https://libera.chat/ -- thanks, Abhilash Raj (maxking)

1 0

docker-mailman Bump to 3.3.4
by matthew＠alberti.us June 11, 2021

June 11, 2021

Hi Mailman Docker Maintainer(s) (maxking 😊 ), Just wanted to see if the docker-mailman will be bumped to 3.3.4. I believe docker-mailman release 0.3.11 is the latest release, and that gives us Mailman 3.3.3. Thank you. - Matt Alberti

1 0

OS update 18.04 -> 20.04 with Python 3.6 -> 3.8
by Torge Riedel June 6, 2021

June 6, 2021

Hi, just want to report that updating my server's OS from Ubuntu 18.04 to 20.04 including an upgrade from Python 3.6 to 3.8 worked 75% on the mailman3 installation. Stopping and disabling the service before prevented most of the problems, but I forget to disable the cron jobs. **sigh** So whenever you have to upgrade your OS including python, don't forget the cron jobs. After OS upgrade I renamed both "venv" directories and rebuild them from my notes. Worked. Starting up mailman-core was easy. Getting mailman-web back to work was a little tricky: 1. After rebuild I forgot to run the post-update script, overseen that in my notes. **sigh #2** 2. My uwsgi-emperor was totally confused and needed some guidance and manual remove of .sock file 3. Finally I got this in the logs: /opt/mailman/web/venv/lib/python3.8/site-packages/django_q/conf.py:139: UserWarning: Retry and timeout are misconfigured. Set retry larger than timeout, failure to do so will cause the tasks to be retriggered before completion. Seehttps://django-q.readthedocs.io/en/latest/configure.html#retry for details. warn( I checked the reference file from here: |https://gitlab.com/mailman/mailman-suite/raw/master/mailman-suite_project/settings.py?inline=false| and it misses the "retry" value. After adding it to Q_CLUSTER everything works fine. Kind regards Torge

1 0

Long-shot cry for help
by David Partain June 3, 2021

June 3, 2021

Hi, I'm having severe operational issues with my mailman3 site. My site is built on standard Ubuntu packages (18.04), and they're ancient. I'm having two big issues... 1. I have the old smiley problem that I've had from the very beginning and never solved. I have added 'charset': 'utf8mb4' to the relevant config file (/etc/mailman3/mailman-web.py), but that hasn't "taken", in the sense that the calls to mysql do not include the charset stuff. /usr/share/mailman3-web/settings_local.py links to /etc/mailman3/mailman-web.py and there is no file /usr/share/mailman3-web/settings_local.pyc. I moved it out of the way hoping it'd get re-built, but it has not been in the months since. The result of this that I get hundreds of mail messages to root (thousands, really), rapidly filling / with mail in /var/mail/root and logging in /var/log/mail.log. The mail has something like "Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive" and "DataError at /hyperkitty/api/mailman/archive (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0D\\x0A...' for column 'content' at row 1")" I have tried to trace to figure out what file is _actually_ controlling the calls to mysql. There's a file called /var/lib/ucf/cache/:etc:mailman3:mailman-web.py as well, but I have no idea what that is or if it might be used. Is there something like 'mailman info' for mailman-web so I can see what files are being used? I'd love to fix this... 2. There's this one curious cron job... * * * * * www-data [ -f /usr/bin/django-admin ] && python2.7 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings (yes, I know python2.7 is a problem... see the last paragraph below). These jobs start piling up. They run & run & run, and eventually the machine screeches to halt when it runs out of memory and goes belly up. This is a relatively recent development. After a while, I'll have 5 or 6 or more of these all running, looking something like this: www-data 6366 6364 99 15:00 ? 00:12:27 python2.7 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings What this means in practice right now is that I'm forced to go in and kill these on a regular basis or the machine _will_ die and people _will_ shout at me. Does anyone have any idea why these might not be finishing? Can I comment this out completely & live happier? Any help whatsoever? ----------------------------------- Finally, I should add that I know my current installation is fundamentally flawed because I chose to go with distro packages. I simply cannot fix it. My strong recommendation to others who follow me is that they do NOT do this. Bite the bullet and install via pip... I'm now in the process of setting up a new server that is completely based on the latest via pip installation. I have to get away from what I have or lose my sanity. Cheers, David

4 3

Send MySQL data to mailman list for distribution
by Zden Pos June 2, 2021

June 2, 2021

I'm trying to connect existing MySQL data to the Mailman distribution list. Actually, a message is sent from the web page. It remains on this page and is also stored in the MySQL table on the provider server. This is working fine. I have also existing Mailman distribution list generated from the cPanel on the provider server as well. This list is working fine when used for a direct mailing. I need to link the MySQL table to the Mailman distribution list. When the message is posted on the web page and stored in the MySQL table, it is also to be sent to members as per the Mailman distribution list. I don't know where to configure it. In the phpMyAdmn page of the MySQL or in the Malman menu ? Thank you for your help

3 2

Getting connection timed out
by blackout69 June 1, 2021

June 1, 2021

Hello everybody, for a couple of days I have been seeing several lines calling these on /var/log/mail.log May 22 18:22:57 glugct postfix/qmgr[21205]: 0ACE641C1B: from=<lug-bounces(a)catania.linux.it>, size=3698, nrcpt=9 (queue active) May 22 18:23:06 glugct postfix/qmgr[27175]: 0ACE641C1B: from=<lug-bounces(a)catania.linux.it>, size=3698, nrcpt=9 (queue active) May 22 18:23:36 glugct postfix/smtp[27177]: 0ACE641C1B: to=<myaddress(a)simail.it>, relay=none, delay=55152, delays=55122/0.07/30/0, dsn=4.4.1, status=deferred (connect to mail.simail.it[217.27.113.30]:25: Connection timed out) May 22 19:33:06 glugct postfix/qmgr[27175]: 0ACE641C1B: from=<lug-bounces(a)catania.linux.it>, size=3698, nrcpt=9 (queue active) May 22 19:33:36 glugct postfix/smtp[28757]: 0ACE641C1B: to=<myaddress(a)simail.it>, relay=none, delay=59352, delays=59322/0.04/30/0, dsn=4.4.1, status=deferred (connect to mail.simail.it[217.27.113.30]:25: Connection timed out) May 22 18:33:06 glugct postfix/qmgr[27175]: 955D241C15: from=<direttivo-bounces(a)catania.linux.it>, size=11026, nrcpt=3 (queue active) May 22 18:33:36 glugct postfix/smtp[27474]: 955D241C15: to=<myaddress(a)simail.it>, relay=none, delay=124379, delays=124349/0.02/30/0, dsn=4.4.1, status=deferred (connect to mail.simail.it[217.27.113.30]:25: Connection timed out) May 22 19:43:06 glugct postfix/qmgr[27175]: 955D241C15: from=<direttivo-bounces(a)catania.linux.it>, size=11026, nrcpt=3 (queue active) May 22 19:43:36 glugct postfix/smtp[28975]: 955D241C15: to=<myaddress(a)simail.it>, relay=none, delay=128579, delays=128549/0.04/30/0, dsn=4.4.1, status=deferred (connect to mail.simail.it[217.27.113.30]:25: Connection timed out) But always on a single email address.What could it depend on?Probable that the mail.simail.it server is blacklisted? Thank you in advanc -- blackout69

4 11