May 2019 - Mailman-users - lists.mailman3.org

How to separate bans for sending emails to lists and subscribing to them
by Nathan Dixon 29 May '19

29 May '19

Hi, Our requirements for using Mailman are that we can have non-members send an email to a list from any domain name, but only people who's email addresses match a list of specific domains should be able to actually subscribe to those lists. I can't seem to find any way of doing this. Any ideas? I have tried using the bans (global and on each list) but these appear to stop all subscriptions and emails being sent to or from a list if the address matches a ban. Having moderated list subscriptions seems to be the only workaround, but this wouldn't be great if we could automate it (we are currently using Listserv which does allow us to do this) Thank you, -- *Nathan Dixon* MEng Senior Software Architect

2 1

update_index_one_list in parallel?
by Szymon Sokół 29 May '19

29 May '19

Hello, I am new to this list. Last week I upgraded my old Mailman 2.1 site (running since 2011) to Mailman 3.2.1; wasn't quite straightforward, and part of the blame goes to people who made the choice of using sqlite as default database backend for mailman3-full Debian package, but finally I have it running with PostgreSQL, which I should've been using from the beginning. Now, however, I have noticed that indexing my archives in hyperkitty (using "manage.py update_index_one_list" for each list in sequence) takes quite a lot of time while using only one CPU (out of 4). Hence my question - is it safe to run several such processes in parallel, and if so, will it have any benefits? I have no idea how locking on the database is done in this code, maybe I'll end up running the jobs in sequence anyway due to exclusive locks… Any informed answers will be appreciated.

3 2

What to backup?
by Alain Kohli 28 May '19

28 May '19

Hi What exactly are we supposed to include in our mailman backups? Obviously the database, but are there any other files that should be included to make sure we can properly restore? Regards, Alain

2 8

Hyperkitty CPU usage
by Alain Kohli 24 May '19

24 May '19

Hi I have hyperkitty running and noticed that the process 'python manage.py runjobs minutely' is running for over 70h now and is constantly using 100% of a CPU core. That can hardly be intended, but I don't know why it's doing that. As far as I could see, this job mainly updates the search index. Running 'python manage.py rebuild_index' only takes about an hour, though, so I don't see why this would take so much longer. The text search is not working in the first place, so that might be connected as well. I'm currently running the v1.2.2. Regards, Alain

3 9

Changing Mailman's "from" domain - redux
by roger＠reasonal.com 23 May '19

23 May '19

(this is a posting directly into this web ui as mails to this list just vanish without a trace) -- Anyone? Anyone? With reference to the original mail here https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… It seems that the "From" domain field is set by Postorius, but I can't find anything in its documentation on how to change it. I've done obvious things like change the boxes hostname to what I want, have fiddled around in the various Postfix settings for mydomain, etc etc. But all admin messages, listserver replies etc come from the partial domain "blah.blah", not "listman.blah.blah". Any ideas welcome.

2 1

Changing Mailman's "from" domain
by Roger Hislop (IS) 23 May '19

23 May '19

Hi guys -- I'm a little baffled here... I'd got my Mailman instance running, then went on leave, and then tried to use it. No dice. If I subscribed an address, no confirmation email would come through. Checked the Postfix log and see this: Roger Hislop (IS) Executive Head - IOT T +27 11 5751600 M+27 78 4595952 May 16 14:34:50 ip-172-31-23-250 postfix/smtp[5025]: 15B87A35FB: to=<iotic-za(a)reasondigital.co.za>, relay=email-smtp.eu-west-1.amazonaws.com[52.51.155.83]:587, delay=0.22, delays=0.01/0.02/0.1/0.1, dsn=5.0.0, status=bounced (host email-smtp.eu-west-1.amazonaws.com[52.51.155.83] said: 554 Message rejected: Email address is not verified. The following identities failed the check in region EU-WEST-1: test-list-request(a)iotcouncil.org.za, test-list-bounces(a)iotcouncil.org.za (in reply to end of DATA command)) I've verified the domain listman.iotcouncil.org.za, so it would seem the obvious solution is to change the domain the mails come from. It appeared that I just change the postfix config to specify domain list so: # INTERNET HOST AND DOMAIN NAMES # # The myhostname parameter specifies the internet hostname of this # mail system. The default is to use the fully-qualified domain name # from gethostname(). $myhostname is used as a default value for many # other configuration parameters. # myhostname = listman.iotcouncil.org.za <<< HERE #myhostname = virtual.domain.tld # The mydomain parameter specifies the local internet domain name. # The default is to use $myhostname minus the first component. # $mydomain is used as a default value for many other configuration # parameters. # mydomain = listman.iotcouncil.org.za <<< and HERE # SENDING MAIL # # The myorigin parameter specifies the domain that locally-posted # mail appears to come from. The default is to append $myhostname, # which is fine for small sites. If you run a domain with multiple # machines, you should (1) change this to $mydomain and (2) set up # a domain-wide alias database that aliases each user to # user(a)that.users.mailhost. # # For the sake of consistency between sender and recipient addresses, # myorigin also specifies the default domain name that is appended # to recipient addresses that have no @domain part. # myorigin = listman.iotcouncil.org.za <<< and HERE #myorigin = $mydomain But when I look in the log: May 16 15:13:05 ip-172-31-23-250 postfix/qmgr[5700]: D58F9A35FB: from=<test-list-bounces(a)iotcouncil.org.za>, size=1224, nrcpt=1 (queue active) May 16 15:13:05 ip-172-31-23-250 postfix/smtpd[5710]: disconnect from ip-172-19-199-2.eu-west-1.compute.internal[172.19.199.2] May 16 15:13:06 ip-172-31-23-250 postfix/smtp[5713]: D58F9A35FB: to=<iotic-za(a)reasondigital.co.za>, relay=email-smtp.eu-west-1.amazonaws.com[52.51.155.83]:587, delay=0.25, delays=0.01/0.03/0.1/0.12, dsn=5.0.0, status=bounced (host email-smtp.eu-west-1.amazonaws.com[52.51.155.83] said: 554 Message rejected: Email address is not verified. The following identities failed the check in region EU-WEST-1: test-list-request(a)iotcouncil.org.za, test-list-bounces(a)iotcouncil.org.za (in reply to end of DATA command)) But the "from" is not changing. I see that messages TO the mailman server are also not being delivered ("-- 5.1.1 Requested action not taken: mailbox unavailable"). So there's another problem. Should I just verify iotcouncil.org.za and add DKIM to its MX record? Seems like a fail...

2 1

Upgrade
by David Krantz 23 May '19

23 May '19

Hi all, I am running a relatively old version of mailman 3, namely 3.2.0a1 from late september 2017. This is installed using the old mailman-bundler method. I use postfix and postgres. I really have two questions: First: What is the normal way to upgrade such a setup? I guess that I just can get the updated versions of all packages through git, set up new venv's to simplify a rollback, back up the database and then just restarting with the new version hoping for the best. Second: Another way would be to set up a new virtual server and reinstall everything and then just switching IPs on the servers. Then: what is the preferred way to install mailman3 today, docker only? I will also have to move all config and data to the new server, this should be mainly about restoring a database backup and copying relevant config, is there anything I miss here? cheers // David

2 1

How to remove global ban using API?
by Nathan Dixon 22 May '19

22 May '19

I am using a global ban to stop anyone not in a list of valid domains from being able to subscribe to any of our lists. This ban needs to be maintained using the Mailman3 api as it is being fed from another source. I can create the ban fine by the following: from mailmanclient import Client client = Client(URI,apiuser,apipass) client.bans.add("^(?!.*[(a)\.](validdomain1\.com|validdomain2\.org|validdomain3\.co\.uk)") However, I cannot seem to be able to remove it using the api: >>> client.bans[0].delete() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 137, in delete self._connection.call(self._url, method='DELETE') File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 404: b'' Or >>> client.bans.remove(str(client.bans[0])) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restobjects/ban.py", line 83, in remove ban.delete() File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 137, in delete self._connection.call(self._url, method='DELETE') File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 404: b'' Or >>> client.bans.remove("^(?!.*[(a)\.](validdomain1\.com|validdomain2\.org|validdomain3\.co\.uk))") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restobjects/ban.py", line 83, in remove ban.delete() File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 137, in delete self._connection.call(self._url, method='DELETE') File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 404: b'' And also, when I now try and go into the mailman-core docker image to use the mailman.interfaces.bans, it will not let me see the global bans: >>> from mailman.interfaces.bans import IBanManager >>> global_bans = IBanManager(None) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: ('Could not adapt', None, <InterfaceClass mailman.interfaces.bans.IBanManager>) I have opened an issue in Gitlab for this as I feel it may be a bug related to regex and global bans (there was a similar one for list banned addresses): https://gitlab.com/mailman/mailman/issues/598 Many thanks for your time, -- *Nathan Dixon* MEng Senior Software Architect m: +44 (0)7402 690311 t: +44 (0)20 81231252 e: nathan.dixon(a)evadon.com w: www.evadon.com This e-mail and any attachments are confidential and may be protected by legal, professional or other privilege. If you are not the intended recipient you should not store it, copy it, re-transmit it, use it or disclose its contents, but should return it to the sender immediately and delete your copy from your system. The views expressed are those of the sender and not necessarily those of Evadon. Please note that whilst we scan all e-mails for viruses we cannot guarantee that any e-mail is virus-free. Please be advised that we expressly reserve the right to monitor email content for the purposes of ensuring compliance with legal requirements and company policies and your sending to, or receiving from, us of any email constitutes your agreement to these terms. Evadon Dynamics Limited, registered in England No. 09494097. Registered Office: Electric Works, Sheffield Digital Campus, 3 Concourse Way, Sheffield, S1 2BJ

2 2

Uncaught runner exception with out runner process
by Alan So 18 May '19

18 May '19

The out runner process shutdown with the following error. Any clue? May 16 17:32:12 2019 (31602) Uncaught runner exception: (pymysql.err.InternalError) (1053, 'Server shutdown in progress') [SQL: 'SELECT anon_1.member_id AS anon_1_member_id, anon_1.member__member_id AS anon_1_member__member_id, anon_1.member_role AS anon_1_member_role, anon_1.member_list_id AS anon_1_member_list_id, anon_1.member_moderation_action AS anon_1_member_moderation_action, anon_1.member_address_id AS anon_1_member_address_id, anon_1.member_preferences_id AS anon_1_member_preferences_id, anon_1.member_user_id AS anon_1_member_user_id \nFROM (SELECT member.id AS member_id, member._member_id AS member__member_id, member.`role` AS member_role, member.list_id AS member_list_id, member.moderation_action AS member_moderation_action, member.address_id AS member_address_id, member.preferences_id AS member_preferences_id, member.user_id AS member_user_id \nFROM member, address \nWHERE member.list_id = %(list_id_1)s AND member.`role` = %(role_1)s AND address.email = %(email_1)s AND member.address_id = address.id UNION SELECT member.id AS member_id, member._member_id AS member__member_id, member.`role` AS member_role, member.list_id AS member_list_id, member.moderation_action AS member_moderation_action, member.address_id AS member_address_id, member.preferences_id AS member_preferences_id, member.user_id AS member_user_id \nFROM member, address, user \nWHERE member.list_id = %(list_id_2)s AND member.`role` = %(role_2)s AND address.email = %(email_2)s AND member.user_id = user.id AND user._preferred_address_id = address.id) AS anon_1'] [parameters: {'list_id_1': 'listname.list.company.domain', 'role_1': 1, 'email_1': 'usera(a)company.domain', 'list_id_2': 'listname.list.company.domain', 'role_2': 1, 'email_2': 'usera(a)company.domain'}] (Background on this error at: http://sqlalche.me/e/2j85) File "/usr/local/lib/python3.6/site-packages/mailman/core/runner.py", line 173, in _one_iteration File "/usr/local/lib/python3.6/site-packages/mailman/core/runner.py", line 266, in _process_one_file File "/usr/local/lib/python3.6/site-packages/mailman/runners/outgoing.py", line 93, in _dispose

2 1

installation choices for mm3
by John Griessen 17 May '19

17 May '19

I've been reading https://wiki.list.org/DOC/Mailman%203%20installation%20experience and see that a virtual is recommended, which is not something I have done yet, and wonder if just putting the mm3 on a different server is easier? I have a server, mail1.cibolo.us that has postfix, postgresql, dovecot, mm2, nsd on it. Another server has just web pages and nsd doing backup DNS for mail1.cibolo.us DNS server. I could do a straight debian install of mailman3-full on the 2nd server and keep it simple, let debian setup a blank database installation, right? Has anyone seen a good write up on how to do that on a debian server step by step?

2 2