December 2019 - Mailman-users - lists.mailman3.org

Mass subscribe question
by Brian Carpenter 17 Dec '19

17 Dec '19

Does mass subscribing bypass a list's subscription policy? I had a list owner mass subscribe a large number of invalid mailboxes which resulted in us terminating their service since Mailman 3 has no bounce processing that helps to clean a list up nor a mass invite function that a user can use instead of mass subscribing. If mass subscribing does bypass the subscription policy then I beg you to make mass inviting and bounce processing a HIGH priority. I am really disappointed that Mailman 3 did not come out with a bounce process feature right out of the gates. But the fact it still doesn't have one on Mailman 3.3 seems scandalous. Brian

1 0

UnicodeEncodeError: 'ascii' codec can't encode character
by Marvin Gülker 16 Dec '19

16 Dec '19

Hi, I've accidentally sent from an unregisted address to one of my lists. Mailman correctly held the message back for moderation, but when I go to the page for held messages in postorios, I receive a 500 error. I've included an except from the log below. To me, it looks as if it errors on processing my last name (which has a non-ascii character in it). This is Postorius 1.1.2 on Ubuntu 18.04 LTS from the repos. I've searched on the tracker for the problem and found one ticket[1], but it has something with logging so I suppose this is a different problem. Ideas? And how do I discard the message without access to the held messages page? Python version is 3.6.8. Log excerpt: Dec 09 13:21:04 2019 (3090) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 215, in falcon.api.API.__call__ (falcon/api.c:2872) File "falcon/api.py", line 189, in falcon.api.API.__call__ (falcon/api.c:2419) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 78, in _make_resource resource['msg'] = msg.as_string() File "/usr/lib/python3.6/email/message.py", line 158, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/lib/python3.6/email/generator.py", line 243, in _handle_text msg.set_payload(payload, charset) File "/usr/lib/python3.6/email/message.py", line 315, in set_payload payload = payload.encode(charset.output_charset) UnicodeEncodeError: 'ascii' codec can't encode character '\ufffd' in position 269: ordinal not in range(128) Dec 09 13:21:04 2019 (3090) 127.0.0.1 - - "GET /3.0/lists/tsc-devel(a)lists.secretchronicles.org/held?count=10&page=1 HTTP/1.1" 500 59 -- Blog: https://mg.guelker.eu

3 15

admin/login/ cannot be accessed
by jean-christophe manciot 14 Dec '19

14 Dec '19

Ubuntu 20.04 python3-django 2:2.2.6-1ubuntu1 python3-django-hyperkitty 1.3.1 (built from sources) mailman3-full 3.2.2-1 Nginx server configuration: ``` ... ######## # Static ######## location /favicon.ico { alias <mysite_dir>/static/hyperkitty/img/favicon.ico; } location /static/favicon.ico { alias <mysite_dir>/static/postorius/img/favicon.ico; } location /static/ { alias <mysite_dir>/static/; } ####################### # Upstream uwsgi server ####################### location / { include /etc/nginx/uwsgi_params; uwsgi_pass 127.0.0.1:<uwsgi_server_port>; } ... ``` where: - <mysite_dir> is a symlink to <django_dir>/static - <uwsgi_server_port> matches the one defined in ```/etc/mailman3/uwsgi.ini```: ``` [uwsgi] # Port on which uwsgi will be listening. uwsgi-socket = 127.0.0.1:<uwsgi_server_port> ``` All 3 systemd services run fine: - mailman3 - mailman3-web - qcluster I'm trying to login to the django administration pages. I get the django administration login page at: https://mysite/admin/login/ Logging in with the admin credentials leads to: ``` This site can’t be reached The webpage at https://mysite/admin/login/ might be temporarily down or it may have moved permanently to a new web address. ERR_HTTP2_PROTOCOL_ERROR ``` This is very strange because it is the URL which I used to get the login page in the first place. If I launch a test web server at another port with: ``` <django_dir># python3 manage.py runserver <mysite_ip_address>:8080 Performing system checks... System check identified no issues (0 silenced). December 11, 2019 - 17:50:48 Django version 2.2.6, using settings 'settings' Starting development server at http://<mysite_ip_address>:8080/ Quit the server with CONTROL-C. ``` and access it at ```http://<mysite_ip_address>:8080/admin/login/``` to login with the same credentials as before, I get through and all the django administration lines appear, although in a degraded layout: ``` Site administration Accounts Email addresses Add Change Authentication and Authorization Groups Add Change Users Add Change Django Mailman 3 Mail domains Add Change Profiles Add Change ... ``` Any idea what could be happening here?

2 6

bounces with empty message-id
by Peter Münster 12 Dec '19

12 Dec '19

Hi, It seems, that mailman bounces the bounces, so that the list-owner never sees them: --8<---------------cut here---------------start------------->8--- postfix/smtpd[12002]: connect from out04.smtpout.orange.fr[193.252.22.213] postfix/smtpd[12002]: 879D6F12E8: client=out04.smtpout.orange.fr[193.252.22.213] postfix/cleanup[11841]: 879D6F12E8: message-id=<> postfix/qmgr[10092]: 879D6F12E8: from=<>, size=4048, nrcpt=1 (queue active) postfix/smtpd[12002]: disconnect from out04.smtpout.orange.fr[193.252.22.213] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5 postfix/lmtp[11846]: 879D6F12E8: to=<test-bounces(a)my-domain.org>, relay=127.0.0.1[127.0.0.1]:8024, delay=0.05, delays=0.04/0/0/0.01, dsn=5.0.0, status=bounced (host 127.0.0.1[127.0.0.1] said: 550 No Message-ID header provided (in reply to end of DATA command)) postfix/qmgr[10092]: 879D6F12E8: removed --8<---------------cut here---------------end--------------->8--- What could I do please, to get the bounces? I would like to use VERP to identify wrong addresses... TIA for any hints, -- Peter

3 8

Hyperkitty Error
by Brian Carpenter 11 Dec '19

11 Dec '19

I am seeing the following error being repeated every 2 seconds in the uwsgi-error.log: 01:22:05 [Q] INFO Process-1:19112 processing [compute_thread_positions] 01:22:05 [Q] ERROR Failed [compute_thread_positions] - 'DiGraph' object has no attribute 'node' : Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/django_q/cluster.py", line 381, in worker res = f(*task['args'], **task['kwargs']) File "/usr/lib/python3.6/site-packages/hyperkitty/tasks.py", line 127, in _compute_thread_positions compute_thread_order_and_depth(thread) File "/usr/lib/python3.6/site-packages/hyperkitty/lib/analysis.py", line 55, in compute_thread_order_and_depth walk_successors(thread.starting_email.id) File "/usr/lib/python3.6/site-packages/hyperkitty/lib/analysis.py", line 37, in walk_successors obj = graph.node[msgid]["obj"] AttributeError: 'DiGraph' object has no attribute 'node' I am running the docker version of Mailman, version 1.3.0. Any suggestions? Thanks, Brian

3 4

Default moderation for new members
by Seth Seeger 11 Dec '19

11 Dec '19

Hello everyone, I’m loving Mailman3 so far! Great work on it. I’d like to have new members who join my lists have their posting ability set to moderated. (After their first legitimate post, they would set to be unmoderated.) I haven’t been able to find this, though I’m suspecting the language has changed. Thanks, Seth

2 3

HyperKitty async tasks runner: systemd qcluster service fails eventually
by actionmystique＠gmail.com 09 Dec '19

09 Dec '19

Ubuntu 20.04 mailman3 3.2.2-1 python3-django-hyperkitty 1.3.0-1ubuntu1 According to [Hyperkitty Asynchronous tasks paragraph](https://hyperkitty.readthedocs.io/en/latest/install.html#asynchronous-tasks), we can start a qcluster systemd service file: ``` [Unit] Description=HyperKitty async tasks runner After=network.target remote-fs.target [Service] ExecStart=/usr/bin/django-admin qcluster --pythonpath <django_project_path> --settings settings User=<user> Group=<group> Restart=always [Install] WantedBy=multi-user.target ``` Starting it works fine: ``` # systemctl restart qcluster # systemctl status qcluster ● qcluster.service - HyperKitty async tasks runner Loaded: loaded (/etc/systemd/system/qcluster.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2019-12-04 15:09:46 CET; 2ms ago Main PID: 1055959 ((go-admin)) Tasks: 0 (limit: 9284) Memory: 0B CGroup: /system.slice/qcluster.service └─1055959 (go-admin) Dec 04 15:09:46 samsung4-ubuntu systemd[1]: Started HyperKitty async tasks runner. ``` However, the service fails after a few seconds: ``` # systemctl status qcluster ● qcluster.service - HyperKitty async tasks runner Loaded: loaded (/etc/systemd/system/qcluster.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2019-12-04 15:09:48 CET; 1min 1s ago Process: 1055961 ExecStart=/usr/bin/django-admin qcluster --pythonpath <django_project_path> --settings settings (code=exited, status=1/FAILURE) Main PID: 1055961 (code=exited, status=1/FAILURE) Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Service RestartSec=100ms expired, scheduling restart. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Scheduled restart job, restart counter is at 5. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: Stopped HyperKitty async tasks runner. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Start request repeated too quickly. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Failed with result 'exit-code'. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: Failed to start HyperKitty async tasks runner. ``` The corresponding log file is empty and I cannot find any issue in the other logs. Any suggestion? Also, another doc paragraph [Running the task queue (required)](http://docs.mailman3.org/en/latest/config-web.html#running-the-t…) talks about another required command "python manage.py qcluster"; so which one is it?

3 9

Source Installation Assistance
by Brian Carpenter 06 Dec '19

06 Dec '19

I am trying to install Mailman 3 from source using: https://docs.mailman3.org/en/latest/index.html My first question is where is the best place to get assistance or ask questions regarding the installation process? I am not sure if mailman-users is that place anymore. Thanks, Brian

4 6

Cannot start lmtp runner due to error while attempting to bind on address ('127.0.0.1', 8024): address already in use
by actionmystique＠gmail.com 04 Dec '19

04 Dec '19

Ubuntu focal 20.04 mailman3 3.2.2-1 In ```/etc/mailman3/mailman.cfg```: ``` # Where the LMTP server listens for connections. Use 127.0.0.1 instead of # localhost for Postfix integration, because Postfix only consults DNS # (e.g. not /etc/hosts). # Configuration is for a system where Mailman and the MTA are on the same host lmtp_host: 127.0.0.1 lmtp_port: 8024 ``` Stopping mailman3 and checking that no lmtp runner is active: ``` # systemctl stop mailman3 # ps -aux |grep mailman|grep -Pv "(postgres|grep)" www-data 855395 0.0 0.4 82856 37904 ? Ss 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 855411 0.0 0.3 156588 25724 ? Sl 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 858518 0.0 0.2 82856 22532 ? Ss 13:52 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini ``` Starting mailman3 and checking all active mailman3 runners: ``` # ps -aux |grep mailman|grep -Pv "(postgres|grep)" www-data 855395 0.0 0.4 82856 37904 ? Ss 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 855411 0.0 0.3 156588 25724 ? Sl 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 858612 0.0 0.2 82856 22532 ? Ss 13:54 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini list 858626 17.8 0.9 103248 74680 ? Ss 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/master --force -C /etc/mailman3/mailman.cfg list 858628 27.2 0.7 87176 62244 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 list 858629 28.6 0.8 89116 64044 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 list 858630 29.2 0.8 90184 64972 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 list 858631 27.0 0.7 87560 62420 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 list 858632 28.8 0.8 89620 64508 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=lmtp:0:1 list 858633 26.6 0.7 84464 60800 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 list 858634 26.2 0.7 81964 58580 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 list 858635 28.4 0.7 88692 63648 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 list 858636 26.4 0.7 83308 59820 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 list 858637 26.8 0.7 84076 60828 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 list 858638 29.2 0.8 90176 64964 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 list 858639 27.4 0.7 87564 62528 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 ``` We clearly see that lmtp runner is active; then it exits after a few seconds: ``` # ps -aux |grep mailman|grep -Pv "(postgres|grep)" www-data 855395 0.0 0.4 82856 37904 ? Ss 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 855411 0.0 0.3 156588 25724 ? Sl 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 858612 0.0 0.2 82856 22532 ? Ss 13:54 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini list 858626 2.7 0.9 103248 74680 ? Ss 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/master --force -C /etc/mailman3/mailman.cfg list 858628 4.3 0.9 103260 74704 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 list 858629 4.3 0.9 103256 74964 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 list 858630 4.3 0.9 103256 74920 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 list 858631 4.3 0.9 103256 74804 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 list 858633 4.4 0.9 103924 75444 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 list 858634 4.3 0.9 103256 74392 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 list 858635 4.3 0.9 103260 74876 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 list 858636 4.4 0.9 182412 79472 ? Sl 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 list 858637 4.4 0.9 103256 74964 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 list 858638 4.5 0.9 103256 74808 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 list 858639 4.4 0.9 103256 74776 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 ``` The systemd service status gives a clue: ``` # systemctl status mailman3 ● mailman3.service - Mailman3 server Loaded: loaded (/lib/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2019-12-02 13:54:45 CET; 2min 51s ago Docs: man:mailman(1) https://mailman.readthedocs.io/ Process: 858617 ExecStart=/usr/bin/mailman -C /etc/mailman3/mailman.cfg start --force (code=exited, status=0/SUCCESS) Main PID: 858626 (python3) Tasks: 14 (limit: 9284) Memory: 688.5M CGroup: /system.slice/mailman3.service ├─858626 /usr/bin/python3 /usr/lib/mailman3/bin/master --force -C /etc/mailman3/mailman.cfg ├─858628 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 ├─858629 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 ├─858630 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 ├─858631 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 ├─858633 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 ├─858634 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 ├─858635 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 ├─858636 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 ├─858637 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 ├─858638 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 └─858639 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: self.lmtp.start() Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3/dist-packages/aiosmtpd/controller.py", line 60, in start Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: raise self._thread_exception Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3/dist-packages/aiosmtpd/controller.py", line 40, in _run Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: ssl=self.ssl_context)) Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: return future.result() Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3.7/asyncio/base_events.py", line 1374, in create_server Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: % (sa, err.strerror.lower())) from None Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: OSError: [Errno 98] error while attempting to bind on address ('127.0.0.1', 8024): address already in use ``` Of course the port is already in use: it is used by dovecot. ``` tcp6 0 0 ::1:8024 :::* LISTEN 0 1317360 129620/dovecot tcp 0 0 127.0.0.1:8024 0.0.0.0:* LISTEN 0 1316324 129620/dovecot ``` I must be missing something.

3 5