October 2021 - Mailman-users - lists.mailman3.org

Race condition when mass subscribing same email addresses to two lists?
by Alan So Nov. 14, 2021

Nov. 14, 2021

We are testing the latest Mailman core 3.3.5 and Postorius 1.3.6. My colleagues reported that some email addresses cannot subscribe to the lists when two of them are using mass subscribe feature to add email addresses to the list. Then I try whether I can repeat the same issue. I did this: 1) Create two lists with default setting 2) Prepare a list of 1000 fresh email addresses not added to the system before 3) Mass subscribe both lists of these 1000 email addresses around the same time with all three Pre (confirm, approved, verified) checked There are quite a lot of similar errors in mailman.log. Did not study the code yet. Which part is causing the issue? SQLAlchemy 1.3.24 psycopg2-binary 2.8.6 [2021-10-28 08:38:30 +0000] [20141] [ERROR] Error handling request /3.1/members Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 136, in handle self.handle_request(listener, req, client, addr) File "/opt/mailman/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 179, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/opt/mailman/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 51, in wrapper config.db.commit() File "/opt/mailman/venv/lib/python3.6/site-packages/mailman/database/base.py", line 52, in commit self.store.commit() File "/opt/mailman/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1046, in commit self.transaction.commit() File "/opt/mailman/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 502, in commit self._assert_active(prepared_ok=True) File "/opt/mailman/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 296, in _assert_active code="7s2a", sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ix_address_email" DETAIL: Key (email)=(testemail999(a)example.com) already exists.

4 9

Moderation Action Changed After Import
by Brian Carpenter Nov. 12, 2021

Nov. 12, 2021

I migrated a Mailman 2 list using import21 and the list's config.pck into a Mailman 3 list. The list owner reported back that about 98% of the list members had their moderation setting changed to Reject (with notification). He said the previous moderation was the list's default which was hold for moderation. Is such a thing possible with an import job? Is this perhaps a bug that has been reported previously? -- Brian Carpenter Harmonylists.com Emwd.com

3 10

mm3 floods syslog
by Wolfgang Bock Oct. 31, 2021

Oct. 31, 2021

Dear friends, I got another problem: mailman floods my syslog. I deleted a domain/webdomain (further mentioned as deleted.org) in postorius, django web. No entries are now in /var/lib/mailman3/data/postfix_domains or postfix_lmtp or /var/lib/mailman3/lists. Here an extract of my syslog (anonymized): Oct 31 10:49:02 myserver mailman3[1284]: Oct 31 10:49:02 2021 (1284) Certificate did not match expected hostname: localhost. Certificate: {'subject': ((('commonName', 'deleteted.org'),),), 'issuer': ((('countryName', 'US'),), (('or ganizationName', "Let's Encrypt"),), (('commonName', 'R3'),)), 'version': 3, 'serialNumber': '0326A733CA7B16D13C830EE707EA614E25F9', 'notBefore': 'Oct 30 20:21:08 2021 GMT', 'notAfter': 'Jan 28 20:21:07 2022 GMT', 'subjectAltName': (('DN S', 'deleteted.org'), ('DNS', 'lists.deleteted.org'), ('DNS', 'mail.deleteted.org'), ('DNS', 'ns.deleteted.org'), ('DNS', 'www.deleteted.org')), 'OCSP': ('http://r3.o.lencr.org',), 'caIssuers': ('http:/ /r3.i.lencr.org/',)} This appears nonstop. I now stopped the cronjob fort he moment: "[ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.minutely /usr/share/mailman3-web/manage.py runjobs minutely" and others Does mailman or django store any information about domains in another place/file. If yes, how this can be deleted?? Thanks for your reply in advance. Regards Wolfgang

2 1

mailman3 postorius cannot retrieve template at ... (<no authorization>)
by Wolfgang Bock Oct. 31, 2021

Oct. 31, 2021

Hello, the question is: postgres - mailman3web-postorius_emailtemplate table vs mailman3 - templates table I installed a brand new mailman3 with mailman Core 3.3.3, Api 3.1, Core Python 3.9.2 postgres12 nginx on debian bullseye. I created a template via postorius -> lists -> (testliste01) -> templates -> new template -> [list:member:regular:footer] and [list:member:digest:footer] They are shown in the postgres database mailman3 table templates: id name context uri username password 14 list:member:digest:footer testliste01.mydomain.de http://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:m ember:digest:footer 15 list:member:regular:footer testliste01.mydomain.de http://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:m ember:regular:footer First there where no templates in the /var/lib/mailman3/templates folder, which was empty after installation. Later I created in /var/lib/mailman3/templates theses folders and the files /list/testliste01.mydomain.de/list:member:digest:footer + list:member:regular:footer owned by list:list 644 A testmail went through but instead of the default footer no footer was shown. In the sys.log I found the following error message: Oct 29 11:28:04 myserver mailman3[33015]: Message: 'Cannot retrieve template at http://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:m ember:regular:footer (<no authorization>)' How can I solve this problem? What kind of authorisation is meant? list? restadmin? I read in https://docs.mailman3.org/en/latest/config-core.html#configure-templates ... list specific templates invar/templates/lists/LIST-ID/LC/ ... I guess the LC means Language Code. But in the database table templates no LC subdirectory is mentioned (in my case must be de). And I read (dont know where) .... The templates created in Postorius are created in the postorius_emailtemplate table in Mailman's database and referenced via URLs like 'https://example.com/mailman3/api/templates/list/<list-id>/<template-name>'. I.e., they are not stored in the file system. ... That's true. I found in postgres database mailman3web in the table mailman3web postorius_emailtemplate my id name data language created_at modified_at context identifier 15 list:member:digest:footer postorius footer digest 1x linefeed 2021-10-29 11:17:40.664689+02 2021-10-29 11:26:48.692628+02 list testliste01.mydomain.de 16 list:member:regular:footer postorius footer non-digest 1x linefeed 2021-10-29 11:18:29.166241+02 2021-10-29 11:27:24.519024+02 list testliste01.mydomain.de But my question is how to integrate this into the mailman3-system. The language isn't set either. Thanks in advance! Regards Wolfgang

2 2

Unable to get mailman3 working on Ubuntu 20.04 LTS
by Michael Heseltine Oct. 29, 2021

Oct. 29, 2021

Hi all, after upgrading an older server to Ubuntu 20.04, I tried to first upgrade my old mailman 2 installation to mailman 3. I've been on it for three weeks now, but unable to get anything working. I have long forgone the idea of upgrading and thus purged everything from mailman2 on my machine, and installed and removed mailman3 several times to no avail. My most recent try (it is currently an empty mailman installation) was done by calling apt install mailman3 mailman3-full mailman3-web and saying "yes" to every question. I'm using Mariadb as the database, and two databases called mailman3 and mailman3web (plus two users with the same names) have been created. I have configured a virtual host for apache (see below) and changed the mailman.cfg and mailman-weg.cfg to match (also see below). I have restarted both mailman3 and mailman3-web several times now, but when I try to access the site, I just get | Something went wrong | Mailman REST API not available. Please start Mailman core. plus an e-mail with subject | [Django] ERROR (EXTERNAL IP): Service Unavailable: /mailman3/postorius/lists/ I'm running | GNU Mailman 3.3.3b1 (Tom Sawyer) | Python 3.8.10 (default, Sep 28 2021, 16:10:42) Can anybody tell me what is going on? Also, the confirmation e-mail I got for registering a user says "example.com" in its subject line, even though I replaced every mention in the config files below with my actual URL. Any help is appreaciated, I can provide logs or other data on request. Cheers, michael /etc/mailman.mailman3.cfg ================== site_owner: postmaster@<site> noreply_address: noreply default_language: en sender_headers: from from_ reply-to sender email_commands_max_lines: 10 pending_request_life: 3d cache_life: 7d pre_hook: post_hook: layout: debian filtered_messages_are_preservable: no html_to_plain_text_command: /usr/bin/lynx -dump $filename listname_chars: [-_.0-9a-z] [shell] prompt: >>> banner: Welcome to the GNU Mailman shell use_ipython: no history_file: [paths.debian] var_dir: /var/lib/mailman3 queue_dir: $var_dir/queue bin_dir: /usr/lib/mailman3/bin list_data_dir: $var_dir/lists log_dir: /var/log/mailman3 lock_dir: $var_dir/locks data_dir: $var_dir/data cache_dir: $var_dir/cache etc_dir: /etc/mailman3 ext_dir: $var_dir/ext messages_dir: $var_dir/messages archive_dir: $var_dir/archives template_dir: $var_dir/templates pid_file: /run/mailman3/master.pid lock_file: $lock_dir/master.lck [database] class: mailman.database.mysql.MySQLDatabase url: mysql+pymysql://mailman3:<password>@localhost/mailman3?charset=utf8&use_unicode=1 debug: no [logging.debian] format: %(asctime)s (%(process)d) %(message)s datefmt: %b %d %H:%M:%S %Y propagate: no level: info path: mailman.log [webservice] hostname: <site> port: 8001 use_https: yes show_tracebacks: yes api_version: 3.1 admin_user: restadmin admin_pass: <pass> [mta] incoming: mailman.mta.exim4.LMTP outgoing: mailman.mta.deliver.deliver smtp_host: localhost smtp_port: 25 smtp_user: smtp_pass: lmtp_host: 127.0.0.1 lmtp_port: 8024 configuration: python:mailman.config.exim4 virtual host in Apache =============== <VirtualHost <site>:443> AssignUserID www-data www-data ServerAdmin webmaster@<site> ServerName <site> SSLEngine on DocumentRoot /webspace/mailman/www ErrorLog /var/log/apache2/mm3_error_log TransferLog /var/log/apache2/mm3_access_log Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico Alias /mailman3/static /var/lib/mailman3/web/static <Directory "/var/lib/mailman3/web/static"> Require all granted </Directory> <IfModule mod_proxy_uwsgi.c> ProxyPass /mailman3/favicon.ico ! ProxyPass /mailman3/static ! ProxyPass /mailman3 unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost </IfModule> Include /etc/letsencrypt/options-ssl-apache.conf SSLCertificateFile /etc/letsencrypt/live/<site>/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/<site>/privkey.pem </VirtualHost> /etc/mailman3/mailman-web.cfg ========================= SECRET_KEY = <secret> ADMINS = ( ('Mailman Suite Admin', 'root@localhost'), ) ALLOWED_HOSTS = [ #"localhost", # Archiving API from Mailman, keep it. # "lists.your-domain.org", # Add here all production URLs you may have. '*' ] MAILMAN_REST_API_URL = 'http://localhost:8001' MAILMAN_REST_API_USER = 'restadmin' MAILMAN_REST_API_PASS = '' MAILMAN_ARCHIVER_KEY = <secret> MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1') INSTALLED_APPS = ( 'hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', # 'django.contrib.admindocs', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora', ) DATABASES = { 'default': { # Use 'sqlite3', 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'. 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': <passwd>, 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See # https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", }, } } USE_X_FORWARDED_HOST = True LANGUAGE_CODE = 'en-us' TIME_ZONE = 'UTC' USE_I18N = True USE_L10N = True USE_TZ = True EMAILNAME = '<site>' DEFAULT_FROM_EMAIL = 'postorius@{}'.format(EMAILNAME) SERVER_EMAIL = 'root@{}'.format(EMAILNAME) ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https" SOCIALACCOUNT_PROVIDERS = { } COMPRESS_OFFLINE = True POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/'

5 7

Internal server errors after upgrade
by Seth Seeger Oct. 29, 2021

Oct. 29, 2021

Hello community, I'm running the Docker version of MM3. I've just run: docker-compose stop docker-compose pull docker-compose up -d I am now at versions: mailman-core: 3.3.4 mailman-web: 2.2.20 I am running a 2-core, 4GB RAM VM on Linode. The web interface mostly works. However, sometimes/regularly I get a 500 Internal Server Error. In /opt/mailman/core/var/logs/mailman.log I see the errors, but they are usually different. The common thread seems they are always about some sort of database call. Reloading the page is usually successful. Did anything change with the latest upgrade in terms of database timeouts? Before this upgrade, I was never getting it. (htop does not show anything unusual.) Any leads? Thank you, Seth

2 2

Upgrade commands under Docker
by Seth Seeger Oct. 28, 2021

Oct. 28, 2021

Another docker upgrade question... according to the upgrade docs https://docs.mailman3.org/en/latest/upgrade-3.2.html#post-upgrade you are supposed to stop the web service and run a series of commands (compress/collectstatic/etc). How do you stop the service under Docker? If I stop the container, then I cannot access the manage.py command. Thanks, Seth

2 1

How to change rest APi password
by bob B Oct. 28, 2021

Oct. 28, 2021

This is probably s stupid question but I have been searching for an answer. How do you set a different rest api password? The instructions say : The credentials are set in the [webservice] section of the configuration using the admin_user and admin_pass properties. But what is that referring to? I am running docker, so is this in the docker.yaml file (I tried it there it did not work) or in the Django config file or someplace else?

3 7

list admin via email
by Christian Stalberg Oct. 28, 2021

Oct. 28, 2021

I'm unclear on how to approve via email a message sent to a list. As list owner you receive a 'requires approval' email. It then says: "If you reply to this message, keeping the Subject: header intact, Mailman will discard the held message. Do this if the message is spam. If you reply to this message and include an Approved: header with the list password in it, the message will be approved for posting to the list. The Approved: header can also appear in the first line of the body of the reply." Question: if I want to approve the message, where do I "...include an Approved: header with the list password in it" ? TY!

2 1

Mailman import of 2000 lists
by dancab＠caltech.edu Oct. 27, 2021

Oct. 27, 2021

I'm working on importing nearly 2000 lists and have been doing some testing prior to migrating from Mailman2. I'm using the mailman create command via for loop. It takes quite a while to do all the work including updating MySQL and re-generating the LMTP files for each list. As the number of lists increases the process gets slower. Do you have any recommendations for speeding up this process? I am running the system in AWS using EFS for storage of the LMTP files. Thanks! -- Dan Caballero Caltech

4 6