- Mailman-users - lists.mailman3.org

Waht are all these messages
by andrew.bernard＠gmail.com June 30, 2019

June 30, 2019

My log is flooded with messages like this, with apparent IP address in them: [pid: 1014|app: 0|req: 228/228] 148.251.78.18 () {38 vars in 865 bytes} [Sun Jun 30 13:26:10 2019] GET /accounts/login/?next=/archives/list/jackrail(a)jackrail.org/thread/E5WNO25XQV74CIV4QS7MQ32EZ5EWKLQ7/ => generated 7658 bytes in 9 msecs (HTTP/1.1 200) 6 headers in 331 bytes (1 switches on core 1) What are these? The flow is continuous, many different IP addresses? Andrew

1 1

Reset digest number and volume
by andrew.bernard＠gmail.com June 29, 2019

June 29, 2019

How to do this? After lots of testing on my pre-release beta testing list, I want the first production digest to be reset toVolume 1, Number 1. Andrew

3 3

Monthly job failure
by andrew.bernard＠gmail.com June 26, 2019

June 26, 2019

I see this error oin startup: 3:22:11 [Q] INFO Process-1:1 processing [rebuild_mailinglist_cache_for_month] 13:22:11 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. : Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.7/site-packages/django_q-1.0.1-py3.7.egg/django_q/cluster.py", line 377, in worker res = f(*task['args'], **task['kwargs']) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.2.3-py3.7.egg/hyperkitty/tasks.py", line 79, in _rebuild_mailinglist_cache_for_month mlist = MailingList.objects.get(name=mlist_name) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-2.1.9-py3.7.egg/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-2.1.9-py3.7.egg/django/db/models/query.py", line 399, in get self.model._meta.object_name hyperkitty.models.mailinglist.MailingList.DoesNotExist: MailingList matching query does not exist. Latest build versions of everything. Any pointers? Andrew

3 7

SECRET_KEY is not defined. Aborting.
by Steven Jones June 26, 2019

June 26, 2019

8><--- root@debian-mailman-001:/opt/mailman/docker-mailman# docker run maxking/mailman-web:0.2 SECRET_KEY is not defined. Aborting. root@debian-mailman-001:/opt/mailman/docker-mailman# 8><--- I keep getting this, I have attempted to set a SECRET_KEY in /opt/mailman/web/settings_local.py and, docker-compose.yaml No joy, can you tell me how I set this variable? or where I get it from? is it pre-generated somewhere? your documentation just does not explain this that I can find, despite even a lot of time googling. PS I have assumed the hyprekitty key is something I set myself, just like a manual password sort of thing, again there is no explanation I can find that explains how to do this properly either. Can this documentation be improved please. To get anywhere at all I finally found this, https://xiaoxing.us/2018/01/01/deploy-mailman-3-on-aws-using-docker/ So I have core running OK I think, ====== root@debian-mailman-001:/opt/mailman/docker-mailman# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 99f434ef3664 maxking/mailman-core:0.2 "docker-entrypoint.s…" 2 hours ago Up About an hour 8001/tcp, 8024/tcp mailman-core 4c0e165d3a90 postgres:9.6-alpine "docker-entrypoint.s…" 2 hours ago Up About an hour 5432/tcp dockermailman_database_1 root@debian-mailman-001:/opt/mailman/docker-mailman# ====== regards Steven

2 2

Confirmation for subscriptions
by andrew.bernard＠gmail.com June 24, 2019

June 24, 2019

Suddenly stopped working. For all the lists I have created, suddenly the confirm setting is ignored for subscriptions and users are subscribed right away. This only happened after I added another domain - surely that can't be the cause? Where to look to troubleshoot this? Andrew

3 5

Unable to download attachments from HyperKitty
by Derek Lambert June 24, 2019

June 24, 2019

I’m having an issue downloading attachments from HyperKitty. The download begins as expected, but never progresses. Eventually it times out. I imagine the browser is expecting data matching the content-length header, but never receives it. Inspecting the contents of the download file reveals (address varies): <memory at 0x7f148fcb9588> I’ve been able to work around the issue with the following patch: --- env/lib/python3.7/site-packages/hyperkitty/models/email.py.orig 2018-12-06 21:38:39.544448735 +0000 +++ env/lib/python3.7/site-packages/hyperkitty/models/email.py 2018-12-06 23:01:39.923239421 +0000 @@ -309,7 +309,7 @@ def get_content(self): folder = self._get_folder() if folder is None: - return self.content + return self.content.tobytes() filepath = os.path.join(folder, str(self.counter)) if not os.path.exists(filepath): logger.error("Could not find local attachment %s for email %s", I haven't been able to replicate the issue with the HyperKitty tests, presumably the issue is somewhere in the Django <-> uwsgi <-> nginx stack. My configs are based on mailman-suite and maxking/docker-mailman. I've tried stripping out all the performance tweaks from my uwsgi and nginx configs, but no change. Versions: HyperKitty 1.2.1 Postorius 1.2.3 Django 2.1.7 nginx 1.14.1 uwsgi 2.0.18 Python 3.7.2 Has anyone come across this before? Derek Lambert

5 6

Unwanted indent on first line of post in hyperkiity
by andrew.bernard＠gmail.com June 23, 2019

June 23, 2019

I don't want hyperkiitty adding an unasked for indent on the first line of all posts. How can one turn that off? Andrew

2 1

Mail not going out
by andrew.bernard＠gmail.com June 22, 2019

June 22, 2019

What process handles the sending of the mail out to subscribers? My new installation suddenly stopped sending. Apologies for so many newbie questions. Andrew

2 1

How to download attachments from hyperkitty
by andrew.bernard＠gmail.com June 22, 2019

June 22, 2019

Not working - download just waits forever after a few bytes in the browser. Could it be that you have to set hyperkitty to store attachments in the filesystem, and configure uwsgi to serve them as static files? Is that what the hyperkitty setting HYPERKITYY_ATTACHMENT_FOLDER is for? If this is so, can anybody show the fragment of uwsgi configuration required? Andrew

1 1

change preferred language of list
by mailman3＠malevil.cc June 22, 2019

June 22, 2019

hey, I am not very experienced with the mailman shell and configuration. I configured mailman using the mailman.cnf and I know that I can use "mailman conf" to get all the parameters. But now I would like to change the preferred language of a list. Because despite I set [mailman] default_language: de even new lists will still have "en" as preferred language. I checked this just by reading the $language variable in a list template. How can I change the list parameters and how can I make mailman to use the german language files? I found the files, so I know they exist, but still need to make mailman use them...

3 8

join via webform
by mailman3＠malevil.cc June 21, 2019

June 21, 2019

Is there a simple possibility to join a mailinglist besider writing a mail to list-join(a)example.com? I know you can register and then join a list, but thats only possible for public lists and I would like to have such a very simple webform for a specific list where you just enter your email address and then get the confirmation mail. I guess there was something like this with mailman2

2 5

Please help - Held message is crashing mailman.
by Alex King June 21, 2019

June 21, 2019

Hi, I've used mailman a lot in the past, and decided to install mailman3 in a recent install. I'm unimpressed so far, it seems buggy. Running on Ubuntu 18.04.2 LTS from packages, mailman3 3.1.1-9, mailman3-web 0+20170523-14, python-django-postorius 1.1.2-3. After a few messages, I now can't go to the "Held Messages" in postorius, I get a 500 error. I tried to interact on the command line, but could not work out how. Reading https://mailman.readthedocs.io/en/latest/src/mailman/docs/install.html and https://mailman.readthedocs.io/en/latest/src/mailman/commands/docs/shell.ht…, I tried: # mailman shell Welcome to the GNU Mailman shell >>> command = cli('mailman.commands.cli_withlist.shell') Traceback (most recent call last): File "/usr/lib/python3.6/code.py", line 91, in runcode exec(code, self.locals) File "<console>", line 1, in <module> NameError: name 'cli' is not defined I did a lot of reading of the manual and googled, but I have no idea where the cli object is supposed to come from.....? The documentation is confusing. Anyway, after some trial and error, i found: # mailman shell Welcome to the GNU Mailman shell >>> list_manager = getUtility(IListManager) >>> m=list_manager.get("committee(a)[redacted].org") >>> from mailman.interfaces.requests import IListRequests >>> requests = IListRequests(m) >>> [x for x in requests.held_requests][0].id 8 >>> requests.get_request(8) Traceback (most recent call last): File "/usr/lib/python3.6/code.py", line 91, in runcode exec(code, self.locals) File "<console>", line 1, in <module> File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0) It seems something was written out to disk that can't be read in for some reason (assuming a json object on disk?) How do I debug this further? How do I find the json being decoded? Any help would be appreciated. (I found mailman2 just worked, I was happy with that.) (My colleague posted a bug about this I believe but I don't have the link to that.) Thanks, Alex

3 3

mass invitation
by mailman3＠malevil.cc June 20, 2019

June 20, 2019

in postorius there is just the option to make a mass subscription, in mailman2 there was also the possibility to just invite users, so they need to click a link when they really want to join the list. Is this also available in mailman3/postorius?

3 2

Importing mbox files into hyperkitty since date
by andrew.bernard＠gmail.com June 20, 2019

June 20, 2019

What is the syntax for the data in the --since argument to hyperkitty_import> --since SINCE SINCE is not very informative. :-) Guessing, using '1/1/1970' does not seem to achieve anything, and gives no error.

2 3

Hyperkitty Archives - Sidebar years/months
by Enrique Terrazas June 20, 2019

June 20, 2019

Hello, I’m going through importing archives from mm2 to mm3. One of the lists I’ve imported has messages going back to 11/2013. I was able to successfully import the lists settings using: mailman import21 lists(a)domain.com<mailto:lists@domain.com> /path/to/list/config.pck This successfully imported lists owners, moderators, subscribers, & settings. I then imported the messages with the following command: python /path/to/manage.py hyperkitty_import -l list(a)domain.com<mailto:list@domain.com> /path/to/list.mbox —since 10/01/2013 I used that date because it is older than the first message sent to the list: 11/13/2013 And followed that up with the update index command: python /path/to/manage.py update_index_one_list list(a)domain.com<mailto:list@domain.com> This successfully imported all the messages from 11/13/2013 through 09/30/2018. Here is where I’m facing two issues: * Messages from October 2018 were not imported * The sidebar only shows the year 2018. Fdasfds If I modify the URL to go to https://my.domain.org/hyperkitty/list/list@domain.com/2013/11 the archived emails show up fine. October 2018 shows two test messages I’ve sent but not any of the messages from October 2018 that should have been imported. Is there something additional that needs to be done to show the previous years and months in the sidebar? Thanks, Enrique

4 5

Flushing/syncing hyperkitty with mailman
by Eric Abrahamsen June 20, 2019

June 20, 2019

I've got mailman running on Ubuntu 14.04, with systemd service files to govern the mailman and hyperkitty qcluster processes. I notice in the systemctl status call for the qcluster command that there are a bunch of errors like: [Q] ERROR Failed [rebuild_thread_cache_votes] - Thread matching query does not exist. [Q] INFO Process-1:1 processing [rebuild_email_cache_votes] [Q] ERROR Failed [rebuild_email_cache_votes] - Email matching query does not exist. [Q] INFO Process-1:2 processing [rebuild_thread_cache_votes] [Q] ERROR Failed [rebuild_thread_cache_votes] - Thread matching query does not exist. [Q] INFO Process-1:3 processing [rebuild_email_cache_votes] [Q] ERROR Failed [rebuild_email_cache_votes] - Email matching query does not exist. [Q] INFO Process-1:4 processing [rebuild_thread_cache_votes] [Q] ERROR Failed [rebuild_thread_cache_votes] - Thread matching query does not exist. Is there something more I should be doing to sync hyperkitty's database with mailman? I've got the crontab file in place, and that appears to be running without error. Anything else? Thanks, Eric

4 5

Django error - External IP
by andrew.bernard＠gmail.com June 20, 2019

June 20, 2019

I am seeing this error frequently: Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /archives/list/jackrail(a)list.jackrail.org/message/new Internal Server Error: /archives/list/jackrail(a)list.jackrail.org/message/new Error at /archives/list/jackrail(a)list.jackrail.org/message/new error 14 from memcached_get(:2:User:11:subscriptions) Request Method: GET Request URL: https://list.jackrail.org/archives/list/jackrail@list.jackrail.org/message/… ... followed by many many pages of settings information, none of which is informative to me. What does this signify? I am trying to bring my list to a go live state, but this does not bode well. I can send the full diagnostic (long) if need be. Andrew

2 2

problems in setting up mailman3 containers.
by Steven Jones June 20, 2019

June 20, 2019

Hi, Trying to setup the 2 mailman3 containers, they fail to start, config and failure messages are, ======== [root@rhel8-svr-001 mailman]# pwd /opt/mailman [root@rhel8-svr-001 mailman]# !269 more core/mailman-extra.cfg # mailman-extra.cfg [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: 172.19.199.2 lmtp_port: 8024 smtp_host: 172.19.199.1 smtp_port: 25 configuration: /etc/postfix-mailman.cfg [mailman] site_owner: steven(a)xx.co.nz SECRET_KEY: testing123 [root@rhel8-svr-001 mailman]# !271 more web/settings_local.py SECRET_KEY: testing123 [root@rhel8-svr-001 mailman]# !272 podman run mailman-core DATABASE_URL is not defined. Using sqlite database... No MTA environment variable found, defaulting to Exim HYPERKITTY_API_KEY not defined, please set this environment variable... exiting... [root@rhel8-svr-001 mailman]# !273 podman run mailman-web SECRET_KEY is not defined. Aborting. [root@rhel8-svr-001 mailman]# ========= Could really do with some examples of these files as well... regards Steven

2 1

join via webform
by mailman3＠malevil.cc June 19, 2019

June 19, 2019

Is there a simple possibility to join a mailinglist besider writing a mail to list-join(a)example.com? I know you can register and then join a list, but thats only possible for public lists and I would like to have such a very simple webform for a specific list where you just enter your email address and then get the confirmation mail. I guess there was something like this with mailman2

1 0

howto reallow a banned email ?
by Nicolas Pettiaux June 19, 2019

June 19, 2019

Hello, By mistake a user has been banned to post to mailman list. How can I allow him not only to post but also to become a member of a list ? I have not found any info on the web, or I have been unable to search correctly. Much thanks, Nicolas -- Nicolas Pettiaux - PhD - nicolas(a){educode,pettiaux}.be - Président de Educode asbl Informer, former et susciter la réflexion des acteurs du monde de l'enseignement face aux évolutions d'un monde numérique - http://2019.educode.be & http://2018.educode.be

2 4

Do not show archives to public
by andrew.bernard＠gmail.com June 19, 2019

June 19, 2019

Is there a way to have list archives only visible to list subscribers? Presently, they are world visible - unless I have configured my site incorrectly.

4 3

Domain owner functionality?
by Derek Lambert June 14, 2019

June 14, 2019

I noticed there's a "domain_owners" table in the mailman core database. If I add a user and domain id to this table will it grant the user owner permissions for all lists in a domain? Will they be able to create lists? Thanks! Derek

2 1

Invalid FilterAction: discard. Treating as discard
by Henrik Rasmussen June 12, 2019

June 12, 2019

Having set the Filter content to Yes - regardless of other setting in the "Alter Messages" tab, make a HTML message come out empty. I can replicate this on my test list. I have tried debugging by changing each setting one-by-one and sending the user's mail, but no change makes the mail come out with a content. However setting "Filter content" to "no" makes the mail come out okay. With "Filter content" set to "Yes", sending another HTML mail is okay, so something in the user's mail may not be right, but I wonder why setting "Filter content" to no "solves" the problem. Afte all this setting is switching all other settings on and off, so I can't just ask the user to turn it off. May 06 11:53:32 2019 (33) ACCEPT: <250E794B36C1E646AF3D81E1F15348C79DA8CA83(a)P2KITMBX06WC03.unicph.domain> May 06 11:53:32 2019 (37) [LISTNAME](a)lists.example.com invalid FilterAction: discard. Treating as discard May 06 11:53:32 2019 (37) <250E794B36C1E646AF3D81E1F15348C79DA8CA83(a)P2KITMBX06WC03.unicph.domain> discarded by "default-posting-pipeline" pipeline handler "mime-delete": After content filtering, the message was empty May 06 17:27:35 2019 (33) ACCEPT: <250E794B36C1E646AF3D81E1F15348C79DA8CAEC(a)P2KITMBX06WC03.unicph.domain> May 06 17:27:35 2019 (37) [LISTNAME](a)lists.example.com invalid FilterAction: discard. Treating as discard May 06 17:27:35 2019 (37) <250E794B36C1E646AF3D81E1F15348C79DA8CAEC(a)P2KITMBX06WC03.unicph.domain> discarded by "default-posting-pipeline" pipeline handler "mime-delete": After content filtering, the message was empty /Henrik Rasmussen

2 9

Postorius performance with over 1000 pending users
by Nick W June 11, 2019

June 11, 2019

I've got a couple things I need help with: The Symptom and The Problem. The Symptom: Postorius has gotten quite slow to load, taking more than 30s to load the Subscription Requests tab. Having zero pending subscription requests (I'm on v1.2.5), I started investigating unconfirmed subscriptions. I followed the commands from here[1] and got this: mailman shell -l your.list.id util = getUtility(IPendings) print(len(list(util.find(mlist=m,pend_type='subscription')))) 1816 Wow, 1816 pending subscription requests. I can say for certain that my list is not popular so these requests must have been submitted by bots. Let's get there in The Problem. How can I "purge" this list? I'm willing to assume that none of these requests are valid and would reject all unconfirmed user requests. The Problem: How might this activity be happening? Running cat /opt/mailman/core/var/logs/mailman.log | grep POST returns a lot of 202 and 409 responses for `POST /3.1/members`. Could it be that these requests are being submitted through Postorius? I'd assume there would be some sort of honeypot input to prevent abuse. I'll continue investigating and follow up with any insight I can on the cause of The Problem. Thanks, Nick [1]: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…>

3 4

WSGI: gunicorn and uwsgi
by andrew.bernard＠gmail.com June 9, 2019

June 9, 2019

Using the Ubuntu 19 packages for all the mailman suite, uwsgi is the Python WSGI server included. I have now switched over to a full build from current source from gitlab, following Mark Sapiro's excellent instructions. It would appear that the current code has the use of gunicorn built in. Is gunicorn the preferred way now to run the mailman Django apps? Is it worth spending time converting everything to use uwsgi, or should I go with the flow and use gunicorn? What are the advantages of gunicorn over uwsgi?

2 1

Help with installing Postorius and Hyperkitty on Ubuntu 19
by andrew.bernard＠gmail.com June 9, 2019

June 9, 2019

I'm new to Mailman. I have a Ubuntu 19 production server. I attempted to get Mailman 3 and the web components going with the system packages, but I encountered a few showstopper errors (such as indexing not working at all) which I know have been fixed in the codebase. To that end I erased my installation, and installed mailman from the gitlab source. That all works much more nicely now and I feel I have a lot more control of the layout of all the files (the various Ubuntu packages smear the parts all over the place, confusingly). Now I want to get Postorius and Hyperkitty going again, independently of Ubuntu packages. I would like to run Postorius and Hyperkitty under one Django project. So I downloaded the git source for mailman_web. I do not know if this is the right way to install. I am partially familiar with Django, but to come to the point, I have no idea once having obtained the source from git how to setup and install the project, where it lives in the filesystem, and so on. Are there any pointers to help in this area? The very nice readthedocs format files for Postorius and Hyperkitty are very silent on basic matters like this, which is not of much help! The git README files also define a new level of brevity. I need some initial hand holding here. If any help is forthcoming, such as links to more full documentation for newbies to Mailman, I would be most grateful.

4 6

Daily Cronjob fails
by Stefan Tatschner June 8, 2019

June 8, 2019

Hey hey, I applied a recent Update from Debian Buster, und this night a received an email about a failing cronjob. The traceback can be reproduced using: # sudo -u www-data flock -n /var/run/mailman3-web/cron.daily /usr/share/mailman3-web/manage.py runjobs daily ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) START TRACEBACK: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) psycopg2.IntegrityError: null value in column "description" violates not-null constraint DETAIL: Failing row contains (2, hb-events(a)hackbrettl.de, hb-events, null, [hb-events] , 1, 2015-10-22 09:54:53.684306+00, hb-events.hackbrettl.de). The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django_extensions/management/commands/runjobs.py", line 36, in runjobs job().execute() File "/usr/lib/python3/dist-packages/hyperkitty/jobs/sync_mailman.py", line 35, in execute sync_with_mailman() File "/usr/lib/python3/dist-packages/hyperkitty/lib/mailman.py", line 131, in sync_with_mailman mlist.update_from_mailman() File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py", line 192, in update_from_mailman self.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 808, in save force_update=force_update, update_fields=update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 838, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 905, in _save_table forced_update) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 955, in _do_update return filtered._update(values) > 0 File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 667, in _update return query.get_compiler(self.db).execute_sql(CURSOR) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql cursor = super(SQLUpdateCompiler, self).execute_sql(result_type) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 899, in execute_sql raise original_exception File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 889, in execute_sql cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python3/dist-packages/django/utils/six.py", line 685, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) django.db.utils.IntegrityError: null value in column "description" violates not-null constraint DETAIL: Failing row contains (2, hb-events(a)hackbrettl.de, hb-events, null, [hb-events] , 1, 2015-10-22 09:54:53.684306+00, hb-events.hackbrettl.de). END TRACEBACK Appearently there is something wrong with the database. Is there anything what I can do about this? Running the migrations doesn't help: # sudo -u www-data flock -n /var/run/mailman3-web/cron.daily /usr/share/mailman3-web/manage.py migrate Operations to perform: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount Running migrations: No migrations to apply. Thanks! Stefan

3 3

Import Archives using Docker Image
by brian＠emwd.com June 7, 2019

June 7, 2019

I want to test out importing a 2.1 archive into a Mailman 3.2 list. I am using maxking's docker images for MM3. How would I go about importing the mbox file? I don't think the following commands would work on a Docker installation of MM3: $ python manage.py hyperkitty_import -l foo-list(a)example.com $var_prefix/archives/private/foo-list.mbox/foo-list.mbox $ python manage.py update_index_one_list foo-list(a)example.com I am not sure how to go about doing this within Docker. Thanks for any assistance, Brian

2 2

Hyperkitty Authorization Required
by andrew.bernard＠gmail.com June 6, 2019

June 6, 2019

Nearly there configuring my first Mailman 3 installation with Postorius and Hyperkiity. This issue eludes me: When trying to start a new thread in the archive, we have: HyperKitty failure on https://list.jackrail.org/hyperkitty/api/mailman/archive: <html><title>Auth required</title><body> <h1>Authorization Required</h1></body></html> (401) I can access this URL just fine" https://list.jackrail.org/hyperkitty/api/ Where do we look to solve this issue?

4 3

Hope for Mailman 3.X in cPanel
by Cold Kayak June 6, 2019

June 6, 2019

Hi, it appears that cPanel is interested in supporting MailMan 3.X! See - https://features.cpanel.net/topic/upgrade-to-mailman-3-0#comment-54870 <https://features.cpanel.net/topic/upgrade-to-mailman-3-0#comment-54870> "We’re hoping to work with the Mailman folks at some point in the future to accomplish this goal." Hopefully this happens sooner than later. regards, Edmond

2 1

Changing default message limit (max_message_size)
by roger＠reasonal.com May 30, 2019

May 30, 2019

Right, me again. I'm so close to having what I need up and running ... but a showstopper is the max message size that triggers a moderate in it. 40kB these days is just way small ... unless you're a 1% using text-only email. So found this on list archive: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… $ bin/mailman shell -l list(a)example.com // Welcome to the GNU Mailman shell // The variable 'm' is the list(a)example.com // mailing list m.max_message_size = nn // commit() Unfortunately this does not work in the containerised mailman. So I get the container ID and run: docker exec 3373d426996d /usr/local/bin/mailman shell -i test-list(a)iotcouncil.org.za (in the post by Mark it says "-l" but I get "unrecognized argument" -- I assume it should be -i for "interactive"? Either way, it fires up the mailman shell and then throws me straight out back to the command line: Welcome to the GNU Mailman shell The variable 'm' is the test-list(a)iotcouncil.org.za mailing list >>> [root@listman ~]# So that doesn't help...

3 7

Bug - if you change primary email address, you're "semi-subscribed"
by roger＠reasonal.com May 30, 2019

May 30, 2019

Subscribed to the list, sending mails worked. Changed primary email (my company email system adds branding crap that exceeded the 40k limit). Then mails to list simply vanished. Sent mail via UI, suddenly get a "welcome to list" and held message says "The message comes from a moderated member".

3 4

Re: Bug - if you change primary email address, you're "semi-subscribed"
by Roger Hislop May 30, 2019

May 30, 2019

Testing if emails are being delivered to the list …

1 0

How to separate bans for sending emails to lists and subscribing to them
by Nathan Dixon May 29, 2019

May 29, 2019

Hi, Our requirements for using Mailman are that we can have non-members send an email to a list from any domain name, but only people who's email addresses match a list of specific domains should be able to actually subscribe to those lists. I can't seem to find any way of doing this. Any ideas? I have tried using the bans (global and on each list) but these appear to stop all subscriptions and emails being sent to or from a list if the address matches a ban. Having moderated list subscriptions seems to be the only workaround, but this wouldn't be great if we could automate it (we are currently using Listserv which does allow us to do this) Thank you, -- *Nathan Dixon* MEng Senior Software Architect

2 1

update_index_one_list in parallel?
by Szymon Sokół May 29, 2019

May 29, 2019

Hello, I am new to this list. Last week I upgraded my old Mailman 2.1 site (running since 2011) to Mailman 3.2.1; wasn't quite straightforward, and part of the blame goes to people who made the choice of using sqlite as default database backend for mailman3-full Debian package, but finally I have it running with PostgreSQL, which I should've been using from the beginning. Now, however, I have noticed that indexing my archives in hyperkitty (using "manage.py update_index_one_list" for each list in sequence) takes quite a lot of time while using only one CPU (out of 4). Hence my question - is it safe to run several such processes in parallel, and if so, will it have any benefits? I have no idea how locking on the database is done in this code, maybe I'll end up running the jobs in sequence anyway due to exclusive locks… Any informed answers will be appreciated.

3 2

What to backup?
by Alain Kohli May 28, 2019

May 28, 2019

Hi What exactly are we supposed to include in our mailman backups? Obviously the database, but are there any other files that should be included to make sure we can properly restore? Regards, Alain

2 8

Hyperkitty CPU usage
by Alain Kohli May 24, 2019

May 24, 2019

Hi I have hyperkitty running and noticed that the process 'python manage.py runjobs minutely' is running for over 70h now and is constantly using 100% of a CPU core. That can hardly be intended, but I don't know why it's doing that. As far as I could see, this job mainly updates the search index. Running 'python manage.py rebuild_index' only takes about an hour, though, so I don't see why this would take so much longer. The text search is not working in the first place, so that might be connected as well. I'm currently running the v1.2.2. Regards, Alain

3 9

Changing Mailman's "from" domain - redux
by roger＠reasonal.com May 23, 2019

May 23, 2019

(this is a posting directly into this web ui as mails to this list just vanish without a trace) -- Anyone? Anyone? With reference to the original mail here https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… It seems that the "From" domain field is set by Postorius, but I can't find anything in its documentation on how to change it. I've done obvious things like change the boxes hostname to what I want, have fiddled around in the various Postfix settings for mydomain, etc etc. But all admin messages, listserver replies etc come from the partial domain "blah.blah", not "listman.blah.blah". Any ideas welcome.

2 1

Changing Mailman's "from" domain
by Roger Hislop (IS) May 23, 2019

May 23, 2019

Hi guys -- I'm a little baffled here... I'd got my Mailman instance running, then went on leave, and then tried to use it. No dice. If I subscribed an address, no confirmation email would come through. Checked the Postfix log and see this: Roger Hislop (IS) Executive Head - IOT T +27 11 5751600 M+27 78 4595952 May 16 14:34:50 ip-172-31-23-250 postfix/smtp[5025]: 15B87A35FB: to=<iotic-za(a)reasondigital.co.za>, relay=email-smtp.eu-west-1.amazonaws.com[52.51.155.83]:587, delay=0.22, delays=0.01/0.02/0.1/0.1, dsn=5.0.0, status=bounced (host email-smtp.eu-west-1.amazonaws.com[52.51.155.83] said: 554 Message rejected: Email address is not verified. The following identities failed the check in region EU-WEST-1: test-list-request(a)iotcouncil.org.za, test-list-bounces(a)iotcouncil.org.za (in reply to end of DATA command)) I've verified the domain listman.iotcouncil.org.za, so it would seem the obvious solution is to change the domain the mails come from. It appeared that I just change the postfix config to specify domain list so: # INTERNET HOST AND DOMAIN NAMES # # The myhostname parameter specifies the internet hostname of this # mail system. The default is to use the fully-qualified domain name # from gethostname(). $myhostname is used as a default value for many # other configuration parameters. # myhostname = listman.iotcouncil.org.za <<< HERE #myhostname = virtual.domain.tld # The mydomain parameter specifies the local internet domain name. # The default is to use $myhostname minus the first component. # $mydomain is used as a default value for many other configuration # parameters. # mydomain = listman.iotcouncil.org.za <<< and HERE # SENDING MAIL # # The myorigin parameter specifies the domain that locally-posted # mail appears to come from. The default is to append $myhostname, # which is fine for small sites. If you run a domain with multiple # machines, you should (1) change this to $mydomain and (2) set up # a domain-wide alias database that aliases each user to # user(a)that.users.mailhost. # # For the sake of consistency between sender and recipient addresses, # myorigin also specifies the default domain name that is appended # to recipient addresses that have no @domain part. # myorigin = listman.iotcouncil.org.za <<< and HERE #myorigin = $mydomain But when I look in the log: May 16 15:13:05 ip-172-31-23-250 postfix/qmgr[5700]: D58F9A35FB: from=<test-list-bounces(a)iotcouncil.org.za>, size=1224, nrcpt=1 (queue active) May 16 15:13:05 ip-172-31-23-250 postfix/smtpd[5710]: disconnect from ip-172-19-199-2.eu-west-1.compute.internal[172.19.199.2] May 16 15:13:06 ip-172-31-23-250 postfix/smtp[5713]: D58F9A35FB: to=<iotic-za(a)reasondigital.co.za>, relay=email-smtp.eu-west-1.amazonaws.com[52.51.155.83]:587, delay=0.25, delays=0.01/0.03/0.1/0.12, dsn=5.0.0, status=bounced (host email-smtp.eu-west-1.amazonaws.com[52.51.155.83] said: 554 Message rejected: Email address is not verified. The following identities failed the check in region EU-WEST-1: test-list-request(a)iotcouncil.org.za, test-list-bounces(a)iotcouncil.org.za (in reply to end of DATA command)) But the "from" is not changing. I see that messages TO the mailman server are also not being delivered ("-- 5.1.1 Requested action not taken: mailbox unavailable"). So there's another problem. Should I just verify iotcouncil.org.za and add DKIM to its MX record? Seems like a fail...

2 1

Upgrade
by David Krantz May 23, 2019

May 23, 2019

Hi all, I am running a relatively old version of mailman 3, namely 3.2.0a1 from late september 2017. This is installed using the old mailman-bundler method. I use postfix and postgres. I really have two questions: First: What is the normal way to upgrade such a setup? I guess that I just can get the updated versions of all packages through git, set up new venv's to simplify a rollback, back up the database and then just restarting with the new version hoping for the best. Second: Another way would be to set up a new virtual server and reinstall everything and then just switching IPs on the servers. Then: what is the preferred way to install mailman3 today, docker only? I will also have to move all config and data to the new server, this should be mainly about restoring a database backup and copying relevant config, is there anything I miss here? cheers // David

2 1

How to remove global ban using API?
by Nathan Dixon May 22, 2019

May 22, 2019

I am using a global ban to stop anyone not in a list of valid domains from being able to subscribe to any of our lists. This ban needs to be maintained using the Mailman3 api as it is being fed from another source. I can create the ban fine by the following: from mailmanclient import Client client = Client(URI,apiuser,apipass) client.bans.add("^(?!.*[(a)\.](validdomain1\.com|validdomain2\.org|validdomain3\.co\.uk)") However, I cannot seem to be able to remove it using the api: >>> client.bans[0].delete() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 137, in delete self._connection.call(self._url, method='DELETE') File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 404: b'' Or >>> client.bans.remove(str(client.bans[0])) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restobjects/ban.py", line 83, in remove ban.delete() File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 137, in delete self._connection.call(self._url, method='DELETE') File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 404: b'' Or >>> client.bans.remove("^(?!.*[(a)\.](validdomain1\.com|validdomain2\.org|validdomain3\.co\.uk))") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restobjects/ban.py", line 83, in remove ban.delete() File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/base.py", line 137, in delete self._connection.call(self._url, method='DELETE') File "/home/wanout/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 404: b'' And also, when I now try and go into the mailman-core docker image to use the mailman.interfaces.bans, it will not let me see the global bans: >>> from mailman.interfaces.bans import IBanManager >>> global_bans = IBanManager(None) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: ('Could not adapt', None, <InterfaceClass mailman.interfaces.bans.IBanManager>) I have opened an issue in Gitlab for this as I feel it may be a bug related to regex and global bans (there was a similar one for list banned addresses): https://gitlab.com/mailman/mailman/issues/598 Many thanks for your time, -- *Nathan Dixon* MEng Senior Software Architect m: +44 (0)7402 690311 t: +44 (0)20 81231252 e: nathan.dixon(a)evadon.com w: www.evadon.com This e-mail and any attachments are confidential and may be protected by legal, professional or other privilege. If you are not the intended recipient you should not store it, copy it, re-transmit it, use it or disclose its contents, but should return it to the sender immediately and delete your copy from your system. The views expressed are those of the sender and not necessarily those of Evadon. Please note that whilst we scan all e-mails for viruses we cannot guarantee that any e-mail is virus-free. Please be advised that we expressly reserve the right to monitor email content for the purposes of ensuring compliance with legal requirements and company policies and your sending to, or receiving from, us of any email constitutes your agreement to these terms. Evadon Dynamics Limited, registered in England No. 09494097. Registered Office: Electric Works, Sheffield Digital Campus, 3 Concourse Way, Sheffield, S1 2BJ

2 2

Uncaught runner exception with out runner process
by Alan So May 18, 2019

May 18, 2019

The out runner process shutdown with the following error. Any clue? May 16 17:32:12 2019 (31602) Uncaught runner exception: (pymysql.err.InternalError) (1053, 'Server shutdown in progress') [SQL: 'SELECT anon_1.member_id AS anon_1_member_id, anon_1.member__member_id AS anon_1_member__member_id, anon_1.member_role AS anon_1_member_role, anon_1.member_list_id AS anon_1_member_list_id, anon_1.member_moderation_action AS anon_1_member_moderation_action, anon_1.member_address_id AS anon_1_member_address_id, anon_1.member_preferences_id AS anon_1_member_preferences_id, anon_1.member_user_id AS anon_1_member_user_id \nFROM (SELECT member.id AS member_id, member._member_id AS member__member_id, member.`role` AS member_role, member.list_id AS member_list_id, member.moderation_action AS member_moderation_action, member.address_id AS member_address_id, member.preferences_id AS member_preferences_id, member.user_id AS member_user_id \nFROM member, address \nWHERE member.list_id = %(list_id_1)s AND member.`role` = %(role_1)s AND address.email = %(email_1)s AND member.address_id = address.id UNION SELECT member.id AS member_id, member._member_id AS member__member_id, member.`role` AS member_role, member.list_id AS member_list_id, member.moderation_action AS member_moderation_action, member.address_id AS member_address_id, member.preferences_id AS member_preferences_id, member.user_id AS member_user_id \nFROM member, address, user \nWHERE member.list_id = %(list_id_2)s AND member.`role` = %(role_2)s AND address.email = %(email_2)s AND member.user_id = user.id AND user._preferred_address_id = address.id) AS anon_1'] [parameters: {'list_id_1': 'listname.list.company.domain', 'role_1': 1, 'email_1': 'usera(a)company.domain', 'list_id_2': 'listname.list.company.domain', 'role_2': 1, 'email_2': 'usera(a)company.domain'}] (Background on this error at: http://sqlalche.me/e/2j85) File "/usr/local/lib/python3.6/site-packages/mailman/core/runner.py", line 173, in _one_iteration File "/usr/local/lib/python3.6/site-packages/mailman/core/runner.py", line 266, in _process_one_file File "/usr/local/lib/python3.6/site-packages/mailman/runners/outgoing.py", line 93, in _dispose

2 1

installation choices for mm3
by John Griessen May 17, 2019

May 17, 2019

I've been reading https://wiki.list.org/DOC/Mailman%203%20installation%20experience and see that a virtual is recommended, which is not something I have done yet, and wonder if just putting the mm3 on a different server is easier? I have a server, mail1.cibolo.us that has postfix, postgresql, dovecot, mm2, nsd on it. Another server has just web pages and nsd doing backup DNS for mail1.cibolo.us DNS server. I could do a straight debian install of mailman3-full on the 2nd server and keep it simple, let debian setup a blank database installation, right? Has anyone seen a good write up on how to do that on a debian server step by step?

2 2

Any known limit of number of lists that one can be owner/member of?
by Alan So May 15, 2019

May 15, 2019

We have just migrated hundreds of lists from Mailman 2 to Mailman 3. It was smooth in most of the cases. However, we just notice that there are some abnormal behaviour of Postorius for particular users who are owner/member of a large number of lists. We further look into a particular user who is owner of 84 lists. When the user login Postorius, only 50 lists are displayed no matter what the number of "Results per page" is chosen. Prev/Next page cannot be clicked as there is only 1 page. When I use the following mailman client snippet, it also displays the same 50 lists: lists = client.find_lists('xxx(a)yyy.com', 'owner') for m_list in lists: print(m_list.fqdn_listname) Did we hit any limit of find_lists? Is it a bug or are something wrong during the migration? When the user use "Mailman settings", the page can still display all lists though (it's slow as it probably did a lot of SQL queries!).

2 7

Re: Archive Policy not reflecting in Hyperkitty
by Abhilash Raj May 14, 2019

May 14, 2019

Are you using images with "rolling" tags? If yes, you should try to switch to "0.2" tag and the new release will be tagged with the same one. To update, you can do: $ docker pull ... $ docker pull .. $ docker-compose stop $ docker-compose rm $ docker-compose up -d This should basically get you up and running with the new image you pulled. -- thanks, Abhilash Raj (maxking)

2 2

Archive Policy not reflecting in Hyperkitty
by Nathan Dixon May 7, 2019

May 7, 2019

So the issue I am seeing is that when I set a lists archive policy to prviate, it isn't reflecting that in Hyperkitty. I am running the docker installation of Mailman, postorius and hyperkitty. So I go into the docker instance of mailman-core and run this: python3 manage.py shell from hyperkitty.models.mailinglist import MailingList MailingList.objects.all().values_list('name', 'archive_policy') And all my lists are showing '2', regardless of how I set the mailing list archive policy. This might be a duplicate of https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… Am I missing something in my install? Or is this a bug? Many thanks for your attention, -- *Nathan Dixon* MEng Senior Software Architect m: +44 (0)7402 690311 t: +44 (0)20 81231252 e: nathan.dixon(a)evadon.com w: www.evadon.com This e-mail and any attachments are confidential and may be protected by legal, professional or other privilege. If you are not the intended recipient you should not store it, copy it, re-transmit it, use it or disclose its contents, but should return it to the sender immediately and delete your copy from your system. The views expressed are those of the sender and not necessarily those of Evadon. Please note that whilst we scan all e-mails for viruses we cannot guarantee that any e-mail is virus-free. Please be advised that we expressly reserve the right to monitor email content for the purposes of ensuring compliance with legal requirements and company policies and your sending to, or receiving from, us of any email constitutes your agreement to these terms. Evadon Dynamics Limited, registered in England No. 09494097. Registered Office: Electric Works, Sheffield Digital Campus, 3 Concourse Way, Sheffield, S1 2BJ

3 5

Moderate mails from owner
by Henrik Rasmussen May 7, 2019

May 7, 2019

I assume this is by design, but is it correct that a list owner, who are not member, will always be allowed to post to the list, even though "Default action to take when a member posts to the list" and "Default action to take when a non-member posts to thelist" are both set to "Hold for moderation"? The owner is set as also moderator, by the way, which seems redundant. I have a list administrator who would like to be moderated when sending to the list, but there is no Member Options on Owners (nor Moderators). A workaround may be to create a new account for the owner and let the current owner be member instead of owner, as she will then be moderated. /Henrik Rasmussen

4 16

Creating and placing custom templates
by kelly.close＠lrewater.com May 2, 2019

May 2, 2019

I know there are previous threads related to this topic but none were very recent so I thought I'd start a new one. I've referenced two of the old ones below. I am attempting to implement a custom footer on one of my lists. This seems like it should be straightforward but I'm clearly doing something wrong, since the default footer keeps coming through despite my efforts. I've referenced the docs as well as these two posts, to no avail. https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… Here's what I'm doing: I found my templates folder and verified it matched what is set in the mailman.cfg file - on my Ubuntu Bionic system it is /var/lib/mailman3/templates/ In that folder I created the following folders and made sure they were properly owned (by list:list) /var/lib/mailman3/templates/lists/f2p2-test.udfcd-lists.org/en/ Into that en/ folder I placed a file called "list:member:regular:footer" It is a text file with just one line of text in it (a link to a website). Also, because there was a mention in one of the earlier posts that this file should have a .txt extension, I also put the same file in there named "list:member:regular:footer.txt". I have two of my own emails subscribed t my "f2p2-test" list at the domain udfcd-lists.org. When I post to the list, and check my other email for the disseminated post, it still has the default footer. I have restarted the mailman3 service. Can anyone spot an obvious error here or offer troubleshooting suggestions? Thank You! ~Kelly

2 1

We're in Season of Docs!
by Stephen J. Turnbull May 1, 2019

May 1, 2019

So, we're in Season of Docs! Currently, Abhilash and I are registered as mentor/admins. I don't think we need more admins, but more mentors is always a good thing. Hey, users! You don't need to be a programmer to mentor docs, of course, and it may be a *good* thing to *not* be a programmer and even *not* a Mailman developer. :-) Suggestions as to doc projects we *really oughtta do!* are very welcome. (I can't believe we got in with the thin doc proposal I put together literally at the last minute, but here we are. ;-) Feel free to get in touch with us on mailman-developers (reply-to set; please respect) if you're interested in mentoring. Or if you prefer you can contact me or Abhilash directly. We can presumably add mentors up to July 23 (deadline for technical writer selection by orgs). For more information: https://wiki.list.org/DEV/SeasonOfDocs2019 https://developers.google.com/season-of-docs/ https://developers.google.com/season-of-docs/docs/participants/ https://developers.google.com/season-of-docs/docs/timeline Regards, Steve -- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull(a)sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN

1 0

GDPR - save subscriptions confirmation emails
by contact＠maadix.net May 1, 2019

May 1, 2019

Hi, I am new in the list. I have been looking for this topic in the archives in order to avoid duplicate, but I could not find any thread. According to the new GDPR rules, lists owners need to have a confirmation from subscribers, and this is easy to do with Mailman 3. What I can't figure out is how to save all the confirmation emails received from users. I only see the confirmations commands emails in the logs, Is there another place/way to store them? For GDPR compliance it is important to be able to demonstrate that you received the confirmation, in case you may need it in the future. Bests Maddish -- *MaadiX* Tu nube en tus manos https://maadix.net https://twitter.com/MaadiXnet

3 4