October 2021 - Mailman-users - lists.mailman3.org

Postorius fails to show list because of a large message being held
by Gilles Filippini Dec. 22, 2021

Dec. 22, 2021

Hi, Recently a member of one of our private lists sent a message with a picture attached. Because of the large message size (23MB) it was put into the moderation queue. But since then postorius fails to open the list's page with this error: Something went wrong Mailman REST API not available. Please start Mailman core. The database logs show that it processed a request related to the held message just before failing: LOG: statement: SELECT message.id AS message_id_1, message.message_id AS message_message_id, message.message_id_hash AS message_message_id_hash, message.path AS message_path FROM message WHERE message.message_id = '<CAEcCghAdp+20jiG19jNGX06xhJ-iYvZGyyV8b-toW7LqCSv5JQ(a)mail.gmail.com>' LIMIT 1 Mailman Core logs: [2020-07-14 10:41:21 +0000] [36] [CRITICAL] WORKER TIMEOUT (pid:49) [2020-07-14 10:41:21 +0000] [49] [INFO] Worker exiting (pid: 49) [2020-07-14 10:41:22 +0000] [50] [INFO] Booting worker with pid: 50 How should I proceed to unblock the situation? Thanks, _g.

4 7

mass subscribers disablement!?
by Christian Stalberg Nov. 26, 2021

Nov. 26, 2021

I added the additional hosts to ALLOWED HOSTS in settings_local.py and mailman is distributing mail to subscribers again. However, I then received many emails of valid users subscriptions being disabled. What would cause that? Can subscriber management be done via command line?

2 5

Negative number for digest size
by Gila Halpern Nov. 19, 2021

Nov. 19, 2021

If I set the digest size threshold to -1, will that make the digest size unlimited, and it will send once per period specified in the digest frequency setting?

3 4

Race condition when mass subscribing same email addresses to two lists?
by Alan So Nov. 14, 2021

Nov. 14, 2021

We are testing the latest Mailman core 3.3.5 and Postorius 1.3.6. My colleagues reported that some email addresses cannot subscribe to the lists when two of them are using mass subscribe feature to add email addresses to the list. Then I try whether I can repeat the same issue. I did this: 1) Create two lists with default setting 2) Prepare a list of 1000 fresh email addresses not added to the system before 3) Mass subscribe both lists of these 1000 email addresses around the same time with all three Pre (confirm, approved, verified) checked There are quite a lot of similar errors in mailman.log. Did not study the code yet. Which part is causing the issue? SQLAlchemy 1.3.24 psycopg2-binary 2.8.6 [2021-10-28 08:38:30 +0000] [20141] [ERROR] Error handling request /3.1/members Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 136, in handle self.handle_request(listener, req, client, addr) File "/opt/mailman/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 179, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/opt/mailman/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 51, in wrapper config.db.commit() File "/opt/mailman/venv/lib/python3.6/site-packages/mailman/database/base.py", line 52, in commit self.store.commit() File "/opt/mailman/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1046, in commit self.transaction.commit() File "/opt/mailman/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 502, in commit self._assert_active(prepared_ok=True) File "/opt/mailman/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 296, in _assert_active code="7s2a", sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ix_address_email" DETAIL: Key (email)=(testemail999(a)example.com) already exists.

4 9

Moderation Action Changed After Import
by Brian Carpenter Nov. 12, 2021

Nov. 12, 2021

I migrated a Mailman 2 list using import21 and the list's config.pck into a Mailman 3 list. The list owner reported back that about 98% of the list members had their moderation setting changed to Reject (with notification). He said the previous moderation was the list's default which was hold for moderation. Is such a thing possible with an import job? Is this perhaps a bug that has been reported previously? -- Brian Carpenter Harmonylists.com Emwd.com

3 10

mm3 floods syslog
by Wolfgang Bock Oct. 31, 2021

Oct. 31, 2021

Dear friends, I got another problem: mailman floods my syslog. I deleted a domain/webdomain (further mentioned as deleted.org) in postorius, django web. No entries are now in /var/lib/mailman3/data/postfix_domains or postfix_lmtp or /var/lib/mailman3/lists. Here an extract of my syslog (anonymized): Oct 31 10:49:02 myserver mailman3[1284]: Oct 31 10:49:02 2021 (1284) Certificate did not match expected hostname: localhost. Certificate: {'subject': ((('commonName', 'deleteted.org'),),), 'issuer': ((('countryName', 'US'),), (('or ganizationName', "Let's Encrypt"),), (('commonName', 'R3'),)), 'version': 3, 'serialNumber': '0326A733CA7B16D13C830EE707EA614E25F9', 'notBefore': 'Oct 30 20:21:08 2021 GMT', 'notAfter': 'Jan 28 20:21:07 2022 GMT', 'subjectAltName': (('DN S', 'deleteted.org'), ('DNS', 'lists.deleteted.org'), ('DNS', 'mail.deleteted.org'), ('DNS', 'ns.deleteted.org'), ('DNS', 'www.deleteted.org')), 'OCSP': ('http://r3.o.lencr.org',), 'caIssuers': ('http:/ /r3.i.lencr.org/',)} This appears nonstop. I now stopped the cronjob fort he moment: "[ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.minutely /usr/share/mailman3-web/manage.py runjobs minutely" and others Does mailman or django store any information about domains in another place/file. If yes, how this can be deleted?? Thanks for your reply in advance. Regards Wolfgang

2 1

mailman3 postorius cannot retrieve template at ... (<no authorization>)
by Wolfgang Bock Oct. 31, 2021

Oct. 31, 2021

Hello, the question is: postgres - mailman3web-postorius_emailtemplate table vs mailman3 - templates table I installed a brand new mailman3 with mailman Core 3.3.3, Api 3.1, Core Python 3.9.2 postgres12 nginx on debian bullseye. I created a template via postorius -> lists -> (testliste01) -> templates -> new template -> [list:member:regular:footer] and [list:member:digest:footer] They are shown in the postgres database mailman3 table templates: id name context uri username password 14 list:member:digest:footer testliste01.mydomain.de http://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:m ember:digest:footer 15 list:member:regular:footer testliste01.mydomain.de http://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:m ember:regular:footer First there where no templates in the /var/lib/mailman3/templates folder, which was empty after installation. Later I created in /var/lib/mailman3/templates theses folders and the files /list/testliste01.mydomain.de/list:member:digest:footer + list:member:regular:footer owned by list:list 644 A testmail went through but instead of the default footer no footer was shown. In the sys.log I found the following error message: Oct 29 11:28:04 myserver mailman3[33015]: Message: 'Cannot retrieve template at http://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:m ember:regular:footer (<no authorization>)' How can I solve this problem? What kind of authorisation is meant? list? restadmin? I read in https://docs.mailman3.org/en/latest/config-core.html#configure-templates ... list specific templates invar/templates/lists/LIST-ID/LC/ ... I guess the LC means Language Code. But in the database table templates no LC subdirectory is mentioned (in my case must be de). And I read (dont know where) .... The templates created in Postorius are created in the postorius_emailtemplate table in Mailman's database and referenced via URLs like 'https://example.com/mailman3/api/templates/list/<list-id>/<template-name>'. I.e., they are not stored in the file system. ... That's true. I found in postgres database mailman3web in the table mailman3web postorius_emailtemplate my id name data language created_at modified_at context identifier 15 list:member:digest:footer postorius footer digest 1x linefeed 2021-10-29 11:17:40.664689+02 2021-10-29 11:26:48.692628+02 list testliste01.mydomain.de 16 list:member:regular:footer postorius footer non-digest 1x linefeed 2021-10-29 11:18:29.166241+02 2021-10-29 11:27:24.519024+02 list testliste01.mydomain.de But my question is how to integrate this into the mailman3-system. The language isn't set either. Thanks in advance! Regards Wolfgang

2 2

Unable to get mailman3 working on Ubuntu 20.04 LTS
by Michael Heseltine Oct. 29, 2021

Oct. 29, 2021

Hi all, after upgrading an older server to Ubuntu 20.04, I tried to first upgrade my old mailman 2 installation to mailman 3. I've been on it for three weeks now, but unable to get anything working. I have long forgone the idea of upgrading and thus purged everything from mailman2 on my machine, and installed and removed mailman3 several times to no avail. My most recent try (it is currently an empty mailman installation) was done by calling apt install mailman3 mailman3-full mailman3-web and saying "yes" to every question. I'm using Mariadb as the database, and two databases called mailman3 and mailman3web (plus two users with the same names) have been created. I have configured a virtual host for apache (see below) and changed the mailman.cfg and mailman-weg.cfg to match (also see below). I have restarted both mailman3 and mailman3-web several times now, but when I try to access the site, I just get | Something went wrong | Mailman REST API not available. Please start Mailman core. plus an e-mail with subject | [Django] ERROR (EXTERNAL IP): Service Unavailable: /mailman3/postorius/lists/ I'm running | GNU Mailman 3.3.3b1 (Tom Sawyer) | Python 3.8.10 (default, Sep 28 2021, 16:10:42) Can anybody tell me what is going on? Also, the confirmation e-mail I got for registering a user says "example.com" in its subject line, even though I replaced every mention in the config files below with my actual URL. Any help is appreaciated, I can provide logs or other data on request. Cheers, michael /etc/mailman.mailman3.cfg ================== site_owner: postmaster@<site> noreply_address: noreply default_language: en sender_headers: from from_ reply-to sender email_commands_max_lines: 10 pending_request_life: 3d cache_life: 7d pre_hook: post_hook: layout: debian filtered_messages_are_preservable: no html_to_plain_text_command: /usr/bin/lynx -dump $filename listname_chars: [-_.0-9a-z] [shell] prompt: >>> banner: Welcome to the GNU Mailman shell use_ipython: no history_file: [paths.debian] var_dir: /var/lib/mailman3 queue_dir: $var_dir/queue bin_dir: /usr/lib/mailman3/bin list_data_dir: $var_dir/lists log_dir: /var/log/mailman3 lock_dir: $var_dir/locks data_dir: $var_dir/data cache_dir: $var_dir/cache etc_dir: /etc/mailman3 ext_dir: $var_dir/ext messages_dir: $var_dir/messages archive_dir: $var_dir/archives template_dir: $var_dir/templates pid_file: /run/mailman3/master.pid lock_file: $lock_dir/master.lck [database] class: mailman.database.mysql.MySQLDatabase url: mysql+pymysql://mailman3:<password>@localhost/mailman3?charset=utf8&use_unicode=1 debug: no [logging.debian] format: %(asctime)s (%(process)d) %(message)s datefmt: %b %d %H:%M:%S %Y propagate: no level: info path: mailman.log [webservice] hostname: <site> port: 8001 use_https: yes show_tracebacks: yes api_version: 3.1 admin_user: restadmin admin_pass: <pass> [mta] incoming: mailman.mta.exim4.LMTP outgoing: mailman.mta.deliver.deliver smtp_host: localhost smtp_port: 25 smtp_user: smtp_pass: lmtp_host: 127.0.0.1 lmtp_port: 8024 configuration: python:mailman.config.exim4 virtual host in Apache =============== <VirtualHost <site>:443> AssignUserID www-data www-data ServerAdmin webmaster@<site> ServerName <site> SSLEngine on DocumentRoot /webspace/mailman/www ErrorLog /var/log/apache2/mm3_error_log TransferLog /var/log/apache2/mm3_access_log Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico Alias /mailman3/static /var/lib/mailman3/web/static <Directory "/var/lib/mailman3/web/static"> Require all granted </Directory> <IfModule mod_proxy_uwsgi.c> ProxyPass /mailman3/favicon.ico ! ProxyPass /mailman3/static ! ProxyPass /mailman3 unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost </IfModule> Include /etc/letsencrypt/options-ssl-apache.conf SSLCertificateFile /etc/letsencrypt/live/<site>/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/<site>/privkey.pem </VirtualHost> /etc/mailman3/mailman-web.cfg ========================= SECRET_KEY = <secret> ADMINS = ( ('Mailman Suite Admin', 'root@localhost'), ) ALLOWED_HOSTS = [ #"localhost", # Archiving API from Mailman, keep it. # "lists.your-domain.org", # Add here all production URLs you may have. '*' ] MAILMAN_REST_API_URL = 'http://localhost:8001' MAILMAN_REST_API_USER = 'restadmin' MAILMAN_REST_API_PASS = '' MAILMAN_ARCHIVER_KEY = <secret> MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1') INSTALLED_APPS = ( 'hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', # 'django.contrib.admindocs', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora', ) DATABASES = { 'default': { # Use 'sqlite3', 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'. 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': <passwd>, 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See # https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", }, } } USE_X_FORWARDED_HOST = True LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = True USE_L10N = True USE_TZ = True EMAILNAME = '<site>' DEFAULT_FROM_EMAIL = 'postorius@{}'.format(EMAILNAME) SERVER_EMAIL = 'root@{}'.format(EMAILNAME) ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https" SOCIALACCOUNT_PROVIDERS = { } COMPRESS_OFFLINE = True POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/'

5 7

Internal server errors after upgrade
by Seth Seeger Oct. 29, 2021

Oct. 29, 2021

Hello community, I'm running the Docker version of MM3. I've just run: docker-compose stop docker-compose pull docker-compose up -d I am now at versions: mailman-core: 3.3.4 mailman-web: 2.2.20 I am running a 2-core, 4GB RAM VM on Linode. The web interface mostly works. However, sometimes/regularly I get a 500 Internal Server Error. In /opt/mailman/core/var/logs/mailman.log I see the errors, but they are usually different. The common thread seems they are always about some sort of database call. Reloading the page is usually successful. Did anything change with the latest upgrade in terms of database timeouts? Before this upgrade, I was never getting it. (htop does not show anything unusual.) Any leads? Thank you, Seth

2 2

Upgrade commands under Docker
by Seth Seeger Oct. 28, 2021

Oct. 28, 2021

Another docker upgrade question... according to the upgrade docs https://docs.mailman3.org/en/latest/upgrade-3.2.html#post-upgrade you are supposed to stop the web service and run a series of commands (compress/collectstatic/etc). How do you stop the service under Docker? If I stop the container, then I cannot access the manage.py command. Thanks, Seth

2 1