July 2020 - Mailman-users - lists.mailman3.org

Anyone integrated Single Sign On like Azure SSO with Mailman3?
by Jacob Bargery 29 Jul '20

29 Jul '20

We are looking at whether it is possible to integrate Azure Single Sign On with Mailman3 to streamline use for our users, has anyone had any luck with doing this? If so, how did you go about doing so? Thanks

3 2

mailman2 import not working
by Abhijith PA 29 Jul '20

29 Jul '20

Hello. Recently I migrated a mailman2 mailing list to mailmam3. There was one problem[1] I faced. Currently I am running Debian packaged version (3.2.2-1) which don't have this fix. But it was manually done with this[2] method. Basically this is what I did for migration: ------------------------------------------- mailman create -o <foo(a)example.com> foo-bar(a)lists.example.com mailman import 21 foo-bar(a)lists.example.com path/to/config.pck ------------------------------------------- Then imported archives with ------------------------------------------ python3 manage.py hyperkitty_import --pythonpath /usr/share/mailman3-web --settings settings -l foo-list(a)list.example.com <path to>/foo-list.mbox python3 manage.py update_index_one_list foo-list(a)example.com --pythonpath /usr/share/mailman3-web --settings settings ------------------------------------------ After everything done tried a test mail, but didn't reach any of the inbox though mail is archived. mailman.log says message is accepted ACCEPT: <FEE2E515-C054-41E6-9853-744D3312A137(a)xxxxxx.org> Nothing after this. On smtp.log we have Jul 29 16:11:52 2020 (16161) Peer: ('127.0.0.1', 54792) Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) handling connection Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) Data: b'LHLO lists.xxxxxxx.in' Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) Data: b'MAIL FROM:<abhijith(a)xxxxxxxxx.org>' Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) sender: abhijith(a)xxxxxxx.org Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) Data: b'RCPT TO:<temptest(a)lists.xxxxxxxxxx.in>' Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) recip: temptest(a)lists.xxxxxxxxxx.in Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) Data: b'DATA' Jul 29 16:11:52 2020 (16161) ('127.0.0.1', 54792) Data: b'QUIT' Please help. Abhijith [1] https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… [2] - https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…

2 3

Could not locate column in row for column 'user.id'
by Shashikanth Komandoor 28 Jul '20

28 Jul '20

Hi Team, I am currently running Mailman 3 with version 3.3.1 and in built postfix version 2.10.1-6 on RHEL 7.5 in production environment from the past 2 months almost. From the past few days, I found my server is responding to the users very slowly and throwing the Server Error page (of course getting the required page after multiple refreshes) for almost every click on the postorius pages (not yet observed on hyperkitty front end which in fact is not so important for me as of now). During that time, I observed in the mailman.log an error message saying "Could not locate column in row for column 'user.id'". The complete traceback log is attached with the mail. The error message is generated for almost all the lists and all the clicks. And despite this error message, I don't know how the required page is loaded after a few refreshes of the webpage. I don't know how to resolve this issue. Please advise. Also, please let me know if anything I need to share with you. -- Thanks & Regards, Shashi Kanth.K 9052671936

2 5

Creating superuser - problem with documentation or it's just me?
by Odhiambo Washington 28 Jul '20

28 Jul '20

Hi, In my quest to create a superuser, I followed another HOWTO - https://docs.mailman3.org/en/latest/devsetup.html (venv) [root@gw /opt/mailman/git/mailman-suite/mailman-suite_project]# python manage.py migrate Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/management/__init__.py", line 401, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/management/__init__.py", line 377, in execute django.setup() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/apps/registry.py", line 114, in populate app_config.import_models() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/git/hyperkitty/hyperkitty/models/__init__.py", line 25, in <module> from .category import ThreadCategory File "/opt/mailman/git/hyperkitty/hyperkitty/models/category.py", line 61, in <module> class ThreadCategory(models.Model): File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/models/base.py", line 121, in __new__ new_class.add_to_class('_meta', Options(meta, app_label)) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/models/base.py", line 325, in add_to_class value.contribute_to_class(cls, name) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/models/options.py", line 208, in contribute_to_class self.db_table = truncate_name(self.db_table, connection.ops.max_name_length()) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/__init__.py", line 28, in __getattr__ return getattr(connections[DEFAULT_DB_ALIAS], item) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/utils.py", line 207, in __getitem__ backend = load_backend(db['ENGINE']) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/utils.py", line 111, in load_backend return import_module('%s.base' % backend_name) File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/db/backends/sqlite3/base.py", line 14, in <module> from sqlite3 import dbapi2 as Database File "/usr/local/lib/python3.7/sqlite3/__init__.py", line 23, in <module> from sqlite3.dbapi2 import * File "/usr/local/lib/python3.7/sqlite3/dbapi2.py", line 27, in <module> from _sqlite3 import * ModuleNotFoundError: No module named '_sqlite3' *<==============================================================* (venv) [root@gw /opt/mailman/git/mailman-suite/mailman-suite_project]# pip install pysqlite3 Collecting pysqlite3 Using cached pysqlite3-0.4.3.tar.gz (40 kB) Building wheels for collected packages: pysqlite3 Building wheel for pysqlite3 (setup.py) ... done WARNING: Legacy build of wheel for 'pysqlite3' created no files. Command arguments: /opt/mailman/mm/venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ge24cqof/pysqlite3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ge24cqof/pysqlite3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ou7p1uhx Command output: [use --verbose to show] Running setup.py clean for pysqlite3 Failed to build pysqlite3 Installing collected packages: pysqlite3 Running setup.py install for pysqlite3 ... done Successfully installed pysqlite3 <============================================================== But even after that, the error about *'ModuleNotFoundError: No module named '_sqlite3'* still persists.. How does one solve this issue?? So I figured out that I need to put my own settings_local.py into the directory and let the command rely on MySQL as the database backend since by default, it uses sqlite3 backend. But the question then is WHY is this not documented on the above page??? How can I delete the superuser after creating it?? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

2 5

# comment allowed in a template file?
by skenny＠scss.tcd.ie 28 Jul '20

28 Jul '20

I've found that dropping files in to the templates directory is a simple and convenient way to make certain things happen. E.g. I can set a default footer for all posts by putting the desired content in a file called var/templates/site/en/list:member:generic:footer.txt. I'm running Mailman 3 in maxking's Docker container. I'm also using Puppet to do as much configuration as possible, so I'd like to add a comment to the file that reads something like: # This file is managed by Puppet. Is it ok to do this? I may not have searched the docs enough, but I couldn't find anything relating to comments. One other observation in case it's any use: Initially, I created the template files with owner set to "root". This resulted in postings to lists getting stuck in mailman-core. It's vital that these files are owned by user "mailman". Once I changed the owner of the file to the latter, everything worked. Stephen

3 4

MM3 not starting afte an upgrade
by Gelpi Andrea 28 Jul '20

28 Jul '20

Hi, to day in a Debian system I upgraded some packages. After the MM3 doesn't start anymore. These are the packages I upgraded: [UPGRADE] clamav:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamav-base:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamav-daemon:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamav-freshclam:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamdscan:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] gcc-10-base:amd64 10.1.0-4 -> 10.1.0-6 [UPGRADE] libc-l10n:amd64 2.30-8 -> 2.31-1 [UPGRADE] libclamav9:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] libgcc-s1:amd64 10.1.0-4 -> 10.1.0-6 [UPGRADE] libnss3:amd64 2:3.42.1-1+deb10u2 -> 2:3.42.1-1+deb10u3 [UPGRADE] python3-django-hyperkitty:amd64 1.3.2-1 -> 1.3.2-1.1 [UPGRADE] python3-django-postorius:amd64 1.3.2-2 -> 1.3.2-2.1 [UPGRADE] qemu-guest-agent:amd64 1:3.1+dfsg-8+deb10u5 -> 1:3.1+dfsg-8+deb10u6 MM3 on start up says: Jul 23 13:29:39 posta mailman3[2608]: Traceback (most recent call last): Jul 23 13:29:39 posta mailman3[2608]: File "/usr/bin/mailman", line 11, in <module> Jul 23 13:29:39 posta mailman3[2608]: load_entry_point('mailman==3.2.1', 'console_scripts', 'mailman')() Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ Jul 23 13:29:39 posta mailman3[2608]: return self.main(*args, **kwargs) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main Jul 23 13:29:39 posta mailman3[2608]: rv = self.invoke(ctx) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 69, in invoke Jul 23 13:29:39 posta mailman3[2608]: return super().invoke(ctx) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke Jul 23 13:29:39 posta mailman3[2608]: return _process_result(sub_ctx.command.invoke(sub_ctx)) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke Jul 23 13:29:39 posta mailman3[2608]: return ctx.invoke(self.callback, **ctx.params) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke Jul 23 13:29:39 posta mailman3[2608]: return callback(*args, **kwargs) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func Jul 23 13:29:39 posta mailman3[2608]: return f(get_current_context(), *args, **kwargs) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/mailman/commands/cli_control.py", line 81, in start Jul 23 13:29:39 posta mailman3[2608]: status, lock = master_state() Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/mailman/bin/master.py", line 104, in master_state Jul 23 13:29:39 posta mailman3[2608]: os.kill(pid, 0) Jul 23 13:29:39 posta mailman3[2608]: PermissionError: [Errno 1] Operation not permitted There is a permission error, but on what? Thanks for any help. Regards -- Gelpi ing. Andrea -------------------------------------------------------------- It took the computing power of three C-64s to fly to the Moon. It takes a 486 to run Windows 95. Something is wrong here. --------------------------------------------------------------

3 7

sqlalchemy.orm.exc.MultipleResultsFound: Multiple rows were found for one_or_none()
by Shashikanth Komandoor 28 Jul '20

28 Jul '20

Hi Team, Today, I found an error message like "Multiple rows were found for one_or_none()" when a hold message is approved for a list. Until yesterday the mails were accepted and delivered fine. But today the above said error message is thrown. Apart from that, I saw the mail had gone into the shunt queue. The "mailman qfile" output of the shunt message and the respective traceback log from mailman.log are attached. You may please go through them and help me out. Please also let me know if I have to share any more details. -- Thanks & Regards, Shashi Kanth.K 9052671936

2 5

Running Mailman3 under supervisord
by Odhiambo Washington 27 Jul '20

27 Jul '20

Hello world, Is it possible to run MM3 under supervisord? What would I use for command= ?? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

3 4

mailman3 nightly script(s) are leaking memory + mitigation
by Mark Dadgar 24 Jul '20

24 Jul '20

So as I mentioned in another thread, I have a shiny new install of mailman3 latest using venv (thanks, Brian!). It’s running on ubuntu 20.04 using python 3.8.2, django 2.2.12, nginx, postgres, and memcached. The only functions this server runs are 1) postfix and dovecot for a small number of personal email accounts, and 2) mailman3 for a couple of mailing lists (one of which is fairly high traffic). Not long after install, I saw a couple of instances of the OOM killer activating in the logs. I set up a 1GB swapfile (discouraged on Digital Ocean instances because they don’t like their flash drives to take that kind of pounding) as a safety net. Then I saw the free swap space eroding over time, so I started collecting some data. Initial evening stats: top - 23:29:16 up 15 days, 9:20, 1 user, load average: 0.07, 0.16, 0.17 Tasks: 200 total, 1 running, 199 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 1.7 sy, 0.0 ni, 96.3 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 3936.2 total, 264.9 free, 1651.4 used, 2019.9 buff/cache MiB Swap: 1000.0 total, 811.2 free, 188.8 used. 1854.5 avail Mem Stats the next morning: top - 08:42:15 up 13 days, 18:33, 1 user, load average: 0.29, 0.27, 0.19 Tasks: 204 total, 1 running, 203 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.5 us, 1.4 sy, 0.0 ni, 96.0 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 3936.2 total, 214.7 free, 2244.8 used, 1476.7 buff/cache MiB Swap: 1000.0 total, 500.3 free, 499.7 used. 1318.8 avail Mem Stats the next evening: top - 23:26:00 up 14 days, 9:17, 1 user, load average: 0.10, 0.06, 0.08 Tasks: 209 total, 1 running, 208 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.6 us, 1.7 sy, 0.0 ni, 95.2 id, 0.3 wa, 0.0 hi, 0.3 si, 0.1 st MiB Mem : 3936.2 total, 206.0 free, 2253.6 used, 1476.6 buff/cache MiB Swap: 1000.0 total, 485.9 free, 514.1 used. 1305.5 avail Mem Stats the following morning: top - 08:09:32 up 14 days, 18:01, 1 user, load average: 0.01, 0.07, 0.14 Tasks: 194 total, 1 running, 193 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 1.4 sy, 0.0 ni, 96.6 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 3936.2 total, 270.6 free, 1974.5 used, 1691.2 buff/cache MiB Swap: 1000.0 total, 266.5 free, 733.5 used. 1545.1 avail Mem As you can see, something during the night is eating memory and not letting it go. If I restart gunicorn, I get the memory back: sudo systemctl restart gunicorn top - 08:52:29 up 14 days, 18:44, 1 user, load average: 0.71, 0.49, 0.25 Tasks: 243 total, 1 running, 242 sleeping, 0 stopped, 0 zombie %Cpu(s): 6.1 us, 3.0 sy, 0.0 ni, 90.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 3936.2 total, 936.3 free, 1277.1 used, 1722.9 buff/cache MiB Swap: 1000.0 total, 804.8 free, 195.2 used. 2242.5 avail Mem I did a little googling and a mitigation that seems to mostly work is to have gunicorn retire its workers periodically to free up memory. gunicorn.cf: max_requests = 1000 max_requests_jitter = 50 That restarts a worker after 1000 requests and adds some jitter so they don’t all get restarted at the same time. I am still seeing the erosion of a couple of MB of swap each day, but it’s WAY better than losing 250MB-350MB overnight. FYI for others who may see this. - Mark ----- mark(a)pdc-racing.net | 408-348-2878

2 1

Re: Mailman REST API not available. Please start Mailman Core.
by Shashikanth Komandoor 23 Jul '20

23 Jul '20

Thank you Mark for your response. I understood that gunicorn is a front end that is used like web server like nginx. But I don't think I am using it instead I am using the httpd to redirect requests to postorius. And I am starting the mailman-suite with the command "python manage.py runserver 0.0.0.0:8000 --insecure &". Hope this does not use gunicorn (which of course I don't know what it is until now). Please correct me if any point I am mentioning is wrong. So, in this case, do you think configuring gunicorn as below would work: *calling gunicorn.cfg in mailman.cfg as below * *[webservice]* *configuration: /var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg* *and content of gunicorn.cfg as below* *(venv3) [root@lsmgr mailman]# cat /var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg[gunicorn]# All the options that can be put here are available at# http://docs.gunicorn.org/en/stable/settings.html#settings <http://docs.gunicorn.org/en/stable/settings.html#settings># No of seconds to wait before killing the worker processes.* *workers = 4graceful_timeout = 90* Also what is the difference between timeout and graceful_timeout ? Please help me if I can increase the management better. Any guide is also helpful for me. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Thu, Jul 23, 2020 at 4:10 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 7/22/20 9:21 AM, Shashikanth Komandoor wrote: > > Hi Mark, > > > > Never mind please I am sending to you only instead of group as I am > > sending you the production content along with the mail. > > > See my other reply. > > > > I have configured the workers count as 4 and timeout as 900 in > > mailman.cfg under [webservice]. This is FYI. > > You can't configure Mailman's REST gunicorn workers directly in the > [webservice] section of mailman.cfg. The only thing you can configure > there is: > > configuration: /path/to/gunicorn.cfg > > That file, which is separate from your web server gunicorn.cfg, is where > you put things like > > [gunicorn] > timeout = 900 > > It also will not work to put a gunicorn section in mailman.cfg. It must > be in a separate file pointed to by webservice.configuration. > > Increasing the number of workers won't help this issue. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > -- Thanks & Regards, Shashi Kanth.K 9052671936

3 2