- Mailman-users - lists.mailman3.org

Huge performance issue: Mailman doesn't (seem to) receive any mails.
by Henrik Rasmussen Sept. 20, 2019

Sept. 20, 2019

I am running maxking/mailman-*:0.2 (GNU Mailman 3.2.0 (La Villa Strangiato)). It looks like Mailman is not receiving (enough) mails, but are keps in the Postfix queue and I can't figure out why. I get a lot of "mail for [172.19.199.2]:8024 is using up 7841 of 7841 active queue entries" in the postfix container log (more postfix log at the buttom. I have restarted all Mailman containers this morning, but this didn't help anything. Last entries in Mailman's smtp.log were when we restarted the containers: Sep 18 08:30:02 2019 (36) Peer: ('172.19.199.1', 35454) Sep 18 08:30:02 2019 (36) ('172.19.199.1', 35454) handling connection Sep 18 08:30:13 2019 (36) ('172.19.199.1', 35454) EOF received Sep 18 08:30:13 2019 (36) Connection lost during _handle_client() Sep 18 08:30:13 2019 (36) ('172.19.199.1', 35454) connection lost Sep 18 08:30:18 2019 (36) Peer: ('172.19.199.1', 37164) Sep 18 08:30:18 2019 (36) ('172.19.199.1', 37164) handling connection Sep 18 08:30:20 2019 (36) ('172.19.199.1', 37164) EOF received Sep 18 08:30:20 2019 (36) Connection lost during _handle_client() Sep 18 08:30:20 2019 (36) ('172.19.199.1', 37164) connection lost The runners last ran at Sep 18 08:42:21 2019 The mailman.log updates constantly, but with "Sep 18 09:45:16 2019 (39) 172.19.199.3 - - "GET /3.0/users/" something something only or other http gets. A lot of them is 404 errors, but also a lot of "GET /3.0/lists/LISTNAME HTTP 200 responces, which should not be related. In "uwsgi-error.log" I get some tracebacks, but since I don't think should be related to incoming mails, I'll leave the traceback for now. Sep 18 10:07:47 postfix postfix/smtpd[5426]: connect from gateway[172.19.199.1] Sep 18 10:07:47 postfix postfix/smtpd[5426]: 45DAA7E97BE94: client=gateway[172.19.199.1] Sep 18 10:07:47 postfix postfix/cleanup[5429]: 45DAA7E97BE94: message-id=<A29DAD01610ED744A9D6F14880665668011C8A2C@P 1KITMBX01WC01.unicph.domain> Sep 18 10:07:47 postfix opendkim[19]: 45DAA7E97BE94: DKIM-Signature field added (s=default, d=listsgw.ku.dk) Sep 18 10:07:47 postfix postfix/smtpd[5426]: disconnect from gateway[172.19.199.1] Sep 18 10:07:47 postfix postfix/qmgr[98]: 45DAA7E97BE94: from=<sender(a)example.com>, size=7988, nrcpt=1 (queue active) Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: mail for [172.19.199.2]:8024 is using up 7844 of 7844 active queue entries Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: you may need to reduce lmtp connect and helo timeouts Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: so that Postfix quickly skips unavailable hosts Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: you may need to increase the main.cf minimal_backoff_time and maximal_backoff_time Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: so that Postfix wastes less time on undeliverable mail Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: you may need to increase the master.cf lmtp process limit Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: please avoid flushing the whole queue when you have Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: lots of deferred mail, that is bad for performance Sep 18 10:07:47 postfix postfix/qmgr[98]: warning: to turn off these warnings specify: qmgr_clog_warn_time = 0 [SLIP] Sep 18 10:26:34 postfix postfix/qmgr[98]: B34C07E980CA3: from=<LISTNAME(a)lists.example.com>, size=1219, nrcpt=1 (queue active) Sep 18 10:26:34 postfix postfix/smtp[6559]: connect to sst-dk.mx1-dk.mailanyone.net[130.226.237.91]:25: Connection refused Sep 18 10:26:34 postfix postfix/smtp[6559]: connect to sst-dk.mx1-dk.mailanyone.net[::ffff:130.226.237.91]:25: Connection refused Sep 18 10:26:34 postfix postfix/smtp[6559]: connect to sst-dk.mx2-dk.mailanyone.net[::ffff:130.226.237.91]:25: Connection refused Sep 18 10:26:34 postfix postfix/smtp[6559]: connect to sst-dk.mx2-dk.mailanyone.net[130.226.237.91]:25: Connection refused Sep 18 10:26:34 postfix postfix/smtp[6559]: connect to sst-dk.mx3-dk.mailanyone.net[130.226.237.91]:25: Connection refused Sep 18 10:26:34 postfix postfix/smtp[6559]: B34C07E980CA3: to=<RECIPIENT(a)foo.bar>, relay=none, delay=420673, delays=420673/0.05/0.2/0, dsn=4.4.1, status=deferred (connect to sst-dk.mx3-dk.mailanyone.net[130.226.237.91]:25: Connection refused) / Henrik Rasmussen

3 10

My list has been switched from public to private archives, but they are still visible
by Paul Arenson Sept. 20, 2019

Sept. 20, 2019

Can anyone help with this? I set my list to private archives today. Up until then, it was pubic. And yet, if you go to the list page, you will see that if you click archives (top left), you can see them. (Filled with a bunch of tests posts for now) https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… Am I misunderstanding what a private archive is? Is something wrong with my setup (which is hosted)? Thank you.

4 8

Mailing List Description that can't be modified by list owners
by chilfer＠unr.edu Sept. 20, 2019

Sept. 20, 2019

Is there a way to create an additional mailing list field or lock down the current Description or Information fields so they are not modifiable by a list owner? The goal is to have a field that identifies the department or area a list belongs to so that it's easier to find if an owner leaves the organization and the owner needs to be updated on multiple lists.

2 1

Re: hosted signup is producing unexpected results
by tokyoprogressive＠mailbox.org Sept. 17, 2019

Sept. 17, 2019

I mistakenly posted twice. Please see: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… Sent from my iPhone

1 0

Execution of msgfmt failed when performing post upgrade steps
by Torge Riedel Sept. 17, 2019

Sept. 17, 2019

Hi, I'm currently running a test setup of MM3 using Mark Sapiros approach with two venv. I also adopted his post-upgrade-script for the web part (django). Just some minutes ago I updated the packages of my test setup with pip3 where hyperkitty is now 1.3.0 When runnin the post-upgrade-script I get the following error message: Execution of msgfmt failed: /opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/locale/nb_NO/LC_MESSAGES/django.po:630: duplicate message definition... /opt/mailman/web/venv/lib/python3.6/site-packages/hyperkitty/locale/nb_NO/LC_MESSAGES/django.po:273: ...this is the location of the first definition msgfmt: found 1 fatal error Not sure whether this is an error of MM3 or one of its dependency packages. Best regards Torge

2 1

hosted signup is producing unexpected results
by tokyoprogressive＠mac.com Sept. 17, 2019

Sept. 17, 2019

In my test of the two forms of signup, I am getting these unexpected results. Is it my settings or is something weird within the program itself? 1) https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… 2) Choose "You can also subscribe without creating an account. If you wish to do so, please use the form below" 3) Then I have to approve it. They get "Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org 4) But I have to approve it.I assume this is because they do not yet have a real account. And they should be listed as a NON MEMBER? 5) But even when they sign up here: https://list.tokyoprogressive.org/accounts/signup/?next=%2Fpostorius%2Flist… they get an email from my provider (NOT MY LIST????) as follows: "Hello from mailman3.emwd.com! You're receiving this e-mail because user iCloud has given yours as an e-mail address to connect their account. To confirm this is correct, go to https://list.tokyoprogressive.org/accounts/confirm-email/MTE3:1iA44k:E7w7xh… Thank you from mailman3.emwd.com! mailman3.emwd.com" 6) And this is where it gets weird. If they send from their email program and get the welcome (Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org You can unsubscribe or make adjustments to your options via email by sending a message to....... I then get a notice: "Your mail to 'discuss(a)list.tokyoprogressive.org' with the subject test notice Is being held until the list moderator can review it for approval. The message is being held because: The message is not from a list member" 7) While if they send FROM THE interface it goes through. 8) What I am confused about. Signing up for an account should make them a member, right? Subscribing should make them a non member, no? But here it seems that both make them a non member. And worse, someone with an account sending from the Interface has their message get through while the same person sending from an email program has me get a notice that a non member is trying to post. Before I can move my actual members over from another mailing list provider, I want to make sure what works and what doesn't. Thanks. paul!!!

1 0

HyperKitty and duplicate emails across maillists
by Derek Lambert Sept. 17, 2019

Sept. 17, 2019

How does HyperKitty handle duplicate messages/message-ids across maillists? We have quite a few lists that are members of other lists. I'm in the process of importing our old Mailman 2.1 archives and seeing a lot of duplicate message errors, and wondering if this is why. Would new messages coming into Mailman be handled the same way? Thanks, Derek

5 8

Template's encoding setup
by Ugnius Sept. 15, 2019

Sept. 15, 2019

I use Lithuanian language (it has special characters like ąčįū...) for regular message template feature. I noticed that some posts come correct, but some have question marks (?) in places of these characters. Looks like there is sittings trick. But footers adds mailman, so it should be always correct, or it depends on senders environment setup? Kind regards Ugnius

3 6

Weird recipient
by Andrew Bernard Sept. 12, 2019

Sept. 12, 2019

Both before and after upgrading tonight, my lists send out bizarre recipients that I cannot understand. It usually happens at the end of the distribution to other valid recipients. Sep 5 10:05:54 www postfix/smtp[3344]: A5DF9FA14E: to=<0101016d00e302ec-4db56da5-fd69-4b5e-9327-292c55e13ea8-000000(a)us-west-2.amazonses.com>, relay=email-smtp.us-west-2.amazonaws.com[52.36.120.66]:587, delay=2.4, delays=0/0/2.2/0.22, dsn=5.0.0, status=bounced (host email-smtp.us-west-2.amazonaws.com[52.36.120.66] said: 501 Invalid MAIL FROM address provided (in reply to MAIL FROM command)) Help! This behaviour is able to be replicated - it happens on every post. Andrew

2 1

Alter Messages: Reply to confusion
by paul＠arenson.org Sept. 12, 2019

Sept. 12, 2019

This is one thing I am confused about. The explanations in the settings about No Munging, etc. are not completely clear. MY GOAL What I want to make sure, since this is a discussion list, is that most messages go to the mailing list, not to individuals. I plan to put something in one of the templates to remind people that a reply goes to the discussion list. If they do not want to send it to everyone, they should send to the original poster. 1) I understand there are plusses and minuses but I am not sure I understand the explanation given. If it IS preferable to use NO MUNGING, how do I make sure that this is easy to accomplish? 2) I can put something in the footer of every message to remind people, but first I need confirmation of what I should choose in this section. Thanks in advance.

4 3

Mailman 3 installation, no mails in hyperkitty, unable to add footer.
by ugniusviln＠gmail.com Sept. 12, 2019

Sept. 12, 2019

I have read documentation, but still have stupid questions, could You please help, because I can't make it working. I do not understand something important. I have no experience with python at all. I have mailman 3, python 3.5 in my debian OS system, and additional packages like uwsgi and ect. using https://buildmedia.readthedocs.org/media/pdf/gnu-mailman/latest/gnu-mailman… about till chapter 6. I understand mailman suite has hyperkitty already. But I am not sure about: 1. Does mailman-suite have hyper-kitty inside? Because if I run $python manage.py runserver - it runs. In installed it under dedicated OS user home and fonfigured to run. I can create lists, event add lists to hyperkitty page using: $django-admin runjob new_lists_from_mailman list server sends messages to list members as well. But nothing happens more. No emails come to hyperkitty, arciver log 0 size. $ python manage.py runjobs -l shows the lists of the jobs, so seems nothing to add. Except uwsgi.ini, I have only 2 configuration files in mailman-suite project directory: settings.py and mailman.cfg So I do not understand what did I configured badly. 2. If I (using Postorius web interface) add footer template to my list under regular message, something strange happens. Mailman stops to send messages for list members. No any entries in the logs. Just Mailman log writes: "unable to connect localhost port 25" and no more explanation. If I delete list's footer template (web interface), mailman starts working again. What is bad here? How to add custom footer? In mailman 2 if was working so good and was easy to setup.

4 7

Suspend a list temporarily
by Andrew Bernard Sept. 12, 2019

Sept. 12, 2019

Is there any way to do this, while keeping other lists running?

3 3

Re: HYPERKITTY_ATTACHMENT_FOLDER
by Ugnius S Sept. 11, 2019

Sept. 11, 2019

Could You please confirm that this parameter should be placed in mailman-hyperkitty.cfg file? I have checked mailman/postorius/hyperkitty owner user can write files there and web user group has privileges to write and to read, but *no any files appear there*. Mailman works, it is possible to download attachments from hyperkitty. Mailman restarted, uwsgi restarted, runjobs restarted. No files in /full/path/to/folder/ File looks like: [general] base_url: https://flistserv.example.com/hyperkitty/ api_key: BlaBlabla HYPERKITTY_ATTACHMENT_FOLDER = /full/path/to/folder/ BR Ugnius 2019-09-11, tr, 21:21 Ugnius S <ugniusviln(a)gmail.com> rašė: > Thank You so much. > > 1) https://hyperkitty.readthedocs.io/en/latest/install.html > Close to the end before the "UPDATE" topic, chapter "Customization". > > 2) > https://buildmedia.readthedocs.org/media/pdf/hyperkitty/stable/hyperkitty.p… > Page 10, chapter 2.8 "Customization" > > BR > Ugnius > > 2019-09-11, tr, 20:16 Abhilash Raj <maxking(a)asynchronous.in> rašė: > >> >> >> On Wed, Sep 11, 2019, at 9:52 AM, Ugnius S wrote: >> > Hi. >> > >> > Can anybody send example how to set it? It is mentioned in >> documentation, >> > but very vaguely. >> >> If you can point out where did you find it in docs, I can update it. >> Information is kind of duplicated in a few places. >> >> > HYPERKITTY_ATTACHMENT_FOLDER = /full/unix/path >> >> This is the right value. >> >> It is a path on the filesystem, where the user running Hyperkitty is >> allowed to write. >> >> > >> > HYPERKITTY_ATTACHMENT_FOLDER = http://localhost:8000/something >> > >> > HYPERKITTY_ATTACHMENT_FOLDER = ./var/data/something/ >> > >> > Has anybody experience of using this, maybe have recommendations of >> using >> > it? Better under project user or better under web content owner where >> the >> > static content is. I would like to store attachments separately to avoid >> > growing database. >> > Hyperkitty documentation: >> > By default, HyperKitty stores the email attachments in the database. >> If >> > you would rather have them stored on the filesystem, you can set the >> > HYPERKITTY_ATTACHMENT_FOLDER configuration value to a directory. >> > Make sure that the user running the Django process (for example, apache >> or >> > www-data) has the permissions to write in this directory. >> > >> > ------------- >> > _______________________________________________ >> > Mailman-users mailing list -- mailman-users(a)mailman3.org >> > To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > >> >> -- >> thanks, >> Abhilash Raj (maxking) >> _______________________________________________ >> Mailman-users mailing list -- mailman-users(a)mailman3.org >> To unsubscribe send an email to mailman-users-leave(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> >

2 2

HYPERKITTY_ATTACHMENT_FOLDER
by Ugnius S Sept. 11, 2019

Sept. 11, 2019

Hi. Can anybody send example how to set it? It is mentioned in documentation, but very vaguely. HYPERKITTY_ATTACHMENT_FOLDER = /full/unix/path HYPERKITTY_ATTACHMENT_FOLDER = http://localhost:8000/something HYPERKITTY_ATTACHMENT_FOLDER = ./var/data/something/ Has anybody experience of using this, maybe have recommendations of using it? Better under project user or better under web content owner where the static content is. I would like to store attachments separately to avoid growing database. Hyperkitty documentation: By default, HyperKitty stores the email attachments in the database. If you would rather have them stored on the filesystem, you can set the HYPERKITTY_ATTACHMENT_FOLDER configuration value to a directory. Make sure that the user running the Django process (for example, apache or www-data) has the permissions to write in this directory. -------------

2 1

Custom templates error and archive link
by Ugnius S Sept. 11, 2019

Sept. 11, 2019

My server uses https connection. According this error seems that certificate of course has not localhost written in it: requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /mailman3/api/ templates/list/example.com/list:user:notice:welcome (Caused by SSLError(CertificateError("hostname 'localhost' doesn't match either of 'listserv.example.com', 'example.com', 'www.example.com ",),)) ------ Second question, is archiver template example: $hyperkitty_url: Permalink to archived message in Hyperkitty If to set as in the example, then it add " : " to the end of the line, and link does not work. I would recommend to update it. My question is: My server's address is listserv.example.com, but mailman to email message variable $hyperkitty replaces like https://example.com/.... What does then happen? https://example.com does not point to my web interface address. I have resolved it with the Apache2 redirection tools, but is it possible to set it correct in mailman's configuration? Kind regards Ugnius

1 0

mailman-users archive search
by Mark Dale Sept. 10, 2019

Sept. 10, 2019

Is this list's search function kaput, or is it an issue with MM3, or is it me? *** https://lists.mailman3.org/archives/search?mlist=mailman-users%40mailman3.o… *** Mailman-users search results for query "Postorius" mailman-users(a)mailman3.org 0 messages Sorry no email could be found for this query. /Mark

2 2

django 2.2.5 which is incompatible
by Mark Dale Sept. 10, 2019

Sept. 10, 2019

Hi, On Debian, in the process of setting up Mailman 3.3.0 I'm getting: $ pip3 install postorius ... django-mailman3 1.3.0 has requirement Django<2.2,>=1.11, but you'll have django 2.2.5 which is incompatible. ... Successfully installed Django-2.2.5 ... Which may or may not be causing a later error when creating superuser: ImportError: No module named django.core.management Am I missing the obvious somewhere? Thanks, Mark

2 1

Google auth issue redirect_uri_mismatch
by Ugnius S Sept. 9, 2019

Sept. 9, 2019

Hi. New question about configuration, maybe somebody has experience and know what I should change it. I have tried to implement Google authorisation. I have created Google credentials keys and successfully added using Django admin page. I use UWSGI proxy for apache setup like "ProxyPass / http://127.0.0.1:8002", UWSGI runs on 0.0.0.0:8002. (So Appache from external address redirects to internal) What happens when I try Google auth... If I set Authorised redirect URIs: http://lists.example.com/accounts/google/login/callback/ , Google gives me error 400 Error: redirect_uri_mismatch and requests for: http://127.0.0.1:8002/accounts/google/login/callback/ If I set that requested address to Google, then Google authorises me, but and after authorisation redirects to http://127.0.0.1:8002/accounts/google....../.... (very long line) Of course this internal address does not open, but... If I just manually replace part of address just: http://127.0.0.1:8002 replace to https://lists.example.com/.../...(long line) this makes me get in as authorised user to the page. How to make django to send external request to Google? Because Google should not point back to internal address... This seems simple, but I do not find what I should change in the configuration: Apache or Django or uwsgi.ini I hope anybody of You have experience and made it to run. Kind regards Ugnius

2 2

Downloads corrupt
by Andrew Bernard Sept. 8, 2019

Sept. 8, 2019

Latest upgrade version has not improved matters. When I download, the file is generated but it is ten bytes long and obviously therefore corrupt. Why am I the only one seeing this error? What can I do to diagnose it? Andrew

3 9

Dmarc settings and
by paul＠arenson.org Sept. 8, 2019

Sept. 8, 2019

In a long thread I asked this but have not gotten a clear enough explanation. Presumably, to insure fewer problems with users of Yahoo etc,, there seem to be two settings: 1) Alter messages 2) Dmarc mitigations I don’t understand how my choice in one affect the other, if they do even, Two experts have tried to help, but I think I need a little hand holding. I understand there is no consumer ready manual and all I have found us this: https://mailman.readthedocs.io/en/latest/src/mailman/handlers/docs/dmarc-mi… My settings; REPLY GOES TO LIST (alter messages)--current settings REPLACE FROM: with list (DMARC mitigations) -> current settings DMARCH Mitigation YES OR NO (DMARC mitigations)--> currently NO I was told that Dmarc mitigations YES might prevent people from not getting mail How about Replace from with list? How about reply goes to list? In other words, what is the relationship between these two different menu items and the effect of the settings in one on the settings of the other? Can anyone help a confused would be happy user? Thanks? If so, what about the Alter Messages settings:

3 5

sqlite3 Database "read only" error on mailman_sync job
by ugniusviln＠gmail.com Sept. 6, 2019

Sept. 6, 2019

Hi. All the project installation is under the same user / path and it works. Archiving working as well, (just one remark: no matter what to set in postorius, hyperkitty shows list as public) I have found work around - it is possible to change list as private in "admin" page. I started to get issues with runjobs after I have created couple of new lists with "no archive" option. If as owner user from project home I run: django-admin mailman_sync Traceback (most recent call last): File "/home/user/.local/lib/python3.5/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/user/.local/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py", line 383, in execute return Database.Cursor.execute(self, query, params) sqlite3.OperationalError: attempt to write a readonly database "django-admin runjob sync_mailman " - makes read only errors as well. "django-admin runjob new_lists_from_mailman" django.db.utils.OperationalError: attempt to write a readonly database END TRACEBACK I did not find information how to fix this. It was worked without errors before. I noticed, that if I stop mailman, "django-admin mailman_sync" starts to work. If I start "mailman start", then "django-admin mailman_sync" failing again. I did not check other jobs, but I think what is related to database, will get errors. Seems mailman locking the database. It is not a file permissions issue. Mailman db location is: /project_directory/var/data/mailman.db if to rename /project_directory/var/data/mailman.db to something, mailman creates database file size 0, django-admin runjobs start working, but of course postorius stops working. I did it for testing. I have returned this database file back, postorius started to work again, but django-admin runjobs does not work, error: "database read only". Please help me how to fix this issue.

4 10

Mass subscribe bug - if you delete and re-add users it does not use updated info
by roger＠reasonal.com Sept. 5, 2019

Sept. 5, 2019

Hi all -- think this is a bug. I had added some users as a mass subscribe. Some were Name Surname <name(a)foo.bar> format, some were just email addresses. I wanted to update the member list with complete name & email addresses. I unsubscribed the partials, and re-added them as full versions. However the updated member list keeps showing the partial address only. This bug also works the other way. I added "Bob Smith <bob(a)bob.com>", checked it was there in list. Unsubscribed the address, and then re-added as "bob(a)bob.com" -- now in the members list it appears as the full "Bob Smith <bob(a)bob.com>.

2 1

Posting replies vs starting a thread via email
by paul＠arenson.org Sept. 5, 2019

Sept. 5, 2019

In my tests where replies go to the list, when a person uses the email client? So far a reply to the original message and a reply to a reply to a reply all go in the same thread. Is this standard? Do I have to warn users not to have endless Re re re re? And the other question is what happens if a user sends a message via email with a new subject? Does the list accept it as a new thread? This should be one of my last big questions. Thank you. Paul

3 6

Mailman Suite release announcement
by Abhilash Raj Sept. 5, 2019

Sept. 5, 2019

Hi All, I am pleased to announce new releases for Mailman Suite, including - Mailman Core 3.3.0 (Tom Swayer) - Mailmanclient 3.3.0 - django-mailman3 1.3.0 - Postorius 1.3.0 - Hyperkitty 1.3.0 This is a feature release and also includes a bunch of bugfixes. I am really really thankful to a lot of contributors who have helped out with this release. It is really nice to see Mailman 3 gaining more users and contributors. Python 3.5 is the minimum supported version, but all the releases have been tested to work with Python 3.5-3.7. This is the last release to support Python 3.5 since it has been in security only maintenance mode for a while now. Some of the highlights of this release includes: Mailman Core 3.3.0 - Support for ARC signing. Thanks to Gene Shuman and Stephen Turnbull. - Many many improvements to "mailman import21" command. Thanks to Mark Sapiro. - Addition of French templates. Thanks to Stéphane Parunakian. - A better and more scalable REST API server using Gunicorn. - Support for SMTPS/STARTTLS. Thanks to J08ny. A full copy of change log is available at: https://mailman.readthedocs.io/en/latest/src/mailman/docs/NEWS.html Postorius 1.3.0 - Many more settings for MailingList configuration exposed, including preferred language, digest related settings. - Support for FILTER_VHOST to filter MailingLists based on the serving domain. Thanks to Nate Coraor. - Ability to write MailingList info in Markdown to render info in HTML instead of Plaintext. A full copy of change log is available at: https://postorius.readthedocs.io/en/latest/news.html Hyperkitty 1.3.0 - Unread messages, when logged in, are more easily recognizable with blue envelope icon. - Improved thread view with colored background for quoted parts of messages. - Improved readability on small screens by removing the indentation in thread view. A full copy of the change log is available at: https://hyperkitty.readthedocs.io/en/latest/news.html All the packages are available to download from PyPi (https://pypi.org) https://pypi.org/project/postorius/ https://pypi.org/project/hyperkitty/ https://pypi.org/project/django-mailman3/ https://pypi.org/project/mailman/ https://pypi.org/project/mailmanclient/ -- thanks, Abhilash Raj (maxking)

1 0

Re: Amazon SES
by Andrew Bernard Sept. 5, 2019

Sept. 5, 2019

No, out of the sandbox, and fully verified. Andrew On 5/9/19 12:18 pm, S. Selby Mats Mats wrote: > Are you still in the “Amazon SES sandbox”?

1 0

Amazon SES
by Andrew Bernard Sept. 5, 2019

Sept. 5, 2019

Is it possible to use Amazon SES with Mailman? Despite my best efforts, I have conclusively verified my sending domain, but Amazon SES rejects mails with the 'sender email not verified' message. If you verify a domain, you do not have to verift individual senders. The help from amazon SES mentions it looks at several headers, but I am not sure what is what. Has anybody had success with this? Andrew'

1 0

My email address appears in quoted test in my list. Hoot change that.
by paul＠arenson.org Sept. 5, 2019

Sept. 5, 2019

Hi. I thought I had the settings right, but I do not see one to get rid of this. My name appears as "On Tue, Sep 3, 2019, at 9:26 PM, paul(a)domainname.org wrote. How can I hide that?

2 3

How to delete an archive
by Andrew Bernard Sept. 5, 2019

Sept. 5, 2019

I deleted a list but I want to delete the archive as well. How do you do that? Also, I had a list called Test. I deleted it. I made a new list called Testing, with a different email address, bu the archive points to the test list. This seems to be messed up. Why? I am sorry to say I have had continuous trouble with many aspects of Hyperkitty from the beginning several months ago. I know that is not helpful, but I am starting to wonder about all this. Andrew

2 1

Editing templates from settings
by paul＠arenson.org Sept. 5, 2019

Sept. 5, 2019

Hello again. Gradually coming along thanks to the help of other members. One thing I want to do is edit my headers or footers to have key info about my site. But it seems even if I skip spaces, when a message is received the spaces disappear. Likewise, is there any way to add basic formatting like BOLD? For example, if I make a footer like this... Skipping a space here __/__/__/__/__/_/__/__/__/__/ Our List _/__/__/__/__/_/__/__/__/__/ Skipping a space here ...the centering disappears, as do the skipped spaces, meaning the message after a header or before a footer looks cluttered, Is there a fix? Thank you.

1 2

Editing templates from settings
by paul＠arenson.org Sept. 5, 2019

Sept. 5, 2019

Hello again. Gradually coming along thanks to the help of other members. One thing I want to do is edit my headers or footers to have key info about my site. But it seems even if I skip spaces, when a message is received the spaces disappear. Likewise, is there any way to add basic formatting like BOLD? For example, if I make a footer like this... Skipping a space here __/__/__/__/__/_/__/__/__/__/ Our List _/__/__/__/__/_/__/__/__/__/ Skipping a space here ...the centering disappears, as do the skipped spaces, meaning the message after a header or before a footer looks cluttered, Is there a fix? Thank you.

2 2

Save original message/headers in HyperKitty?
by Derek Lambert Sept. 5, 2019

Sept. 5, 2019

Is it possible to configure HyperKitty to store the original messages, or the headers? Has this ever been discussed? Thanks, Derek Lambert

2 1

Basic settings for a discussion list
by paul＠arenson.org Sept. 5, 2019

Sept. 5, 2019

Over the last 20 hours I have been trying to figure out setup for list members to sign up. One person has kindly helped out, though I am still unsure of the difference between the several signups. That thread is here and I welcome more advice. https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… But it occurs to me that my problem is that I have not actually configured my list. So this thread is for help on that. I have found a guide for general users here: http://docs.mailman3.org/en/latest/userguide.html But I have only found a highly technical guide for administrators here: https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclien… It is written for coders and I am looking for a guide similar to the one for users. It seems not to exist yet. WHAT I WANT Discussion only list. Ideally anyone who signs up via the various ways should be able to get the archives and post via email and the interface if they want. The list should be closed to others. In the absence of any guide, all I can do is cross check with you guys and hope I am on the right track. I am non technical, although I managed a Mailman 2 list without difficulty. 1) FIRST ANOMALY Under https://list.tokyoprogressive.org/postorius/accounts/subscriptions/ I see myself twice: List Name Subscription Address Delivery Mode Role discuss.list.tokyoprogressive.org paul(a)tokyoprogressive.org None member discuss.list.tokyoprogressive.org paul(a)tokyoprogressive.org regular owner Is this correct? I added My name in one field somewhere and this is the result. I even got a welcome message. I think it is because I subscribed to my own list.If this was not necessary, what should I do? Global based preferences are listed. Address and email based ones are not showing either enabled or disabled. 2) MANAGE LISTS For myself I have done the following in my MAILING LIST SETTINGS Subscribe to this list My email and name (Maybe that is why i am listed twice) Now I see 3) Users For now only me- I am listed as member and owner. There will be no other moderators or owners I have no idea what a non member is (if they register aren't they a member?) 4) Templates This is showing nothing at this point. Before it was. I am not sure what i did to stop it from showing. Perhaps because I made it a private list? 5) Settings a) List identity I have it set for NOT SHOWING on the Index page because I will link to it from my news magazine website as a way to comment on articles. Is this ok? Other data will be filled in later. I think I am confusing this with private list. I also will have no other list. b) AUTO RESPOND LIST OWNER-- I added a short note thanking people. Respond and Continue Processing c) AUTO RESPOND POSTER--Your message has been been received. If you are a member, it should be posted soon. Respond and Continue Processing d) AUTO RESPOND REQUESTS--No auto response OTHER THINGS DEFAULT e) URL WELCOME MESSAGE--NOT SURE. I want people to subscribe themselves but I also have 500 people from another list from DADA I will import. I want them automatically be able to post. Is it possible? If so, I hope to add them manually. DO I compose a text on my website and welcome them by adding the URL, or enter it in my list? f) NOTIFY OF MEMBERSHIP CHANGES--Yes g) ALTER MESSAGES ---I have left most things as they are. However, as this is a discussion list, I want replies ONLY to go to the list. I chose that. h) DMARC --I have left as is i) DIGEST--I have left as is 6) MASS OPERATIONS--I have left as is 7) BAN--left as is 8) MESSAGE ACCEPTANCE--I have left as is 9) HEADER FILTERS--left as is 10) ARCHIVES--Private Archives (Maybe this is why templates do not show???) 11) SUBSCRIPTION POLICY--MODERATE-- I intend to send them an email to confirm their reasons for subscribing to avoid spammers. LAST THING. My Gravity does not work. I edited it online as per instructions but it does not appear. That is all I can think of. Thanks in advance. paul

2 4

Trying to configure a list (hosted)
by paul＠arenson.org Sept. 4, 2019

Sept. 4, 2019

Is there a how to on how to configure a list? For example, using a hosted option, I see this menu: Subscription requests Held messages Users Templates Settings Mass operations Banned addresses Header filters And cling on settings, for example. I have already removed my list from the list of lists on the server since I want my website users to only see my list. They can either do that here https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… or here https://list.tokyoprogressive.org/accounts/signup/ I actually do not WANT people to join without being able to use the web interface because it is intended to be a discussion list. So the option to subscribe without creating an account. is confusing to me, which is why I prefer the latter. BUT...using a test of signing up myself (using 3 other email addresses I have), some using the subscription option and some using the bottom one (no account), all the email addresses end up in my members list and non end up in my non members list. (So I am wondering what a non member is. I had thought it was someone who signed up without registering (just name, no password), but if so i wonder why all 3 of my addresses end up in the members place. Lastly, I want to configure it as a discussion list. However, I do not see that option, which was clear in MAILMAN 2. (And I understand that if someone registers without making an account, I cannot set them myself to have an account enabling them to see the archives and post from the web? If it IS possible, that would be great, because the purpose of the archives in my case is to enable better discussions than if one has only to refer to their past emails.) That is about it for now, but I am stuck on this right now and a user manual for non techie people, if it exists, would be nice. I see one for list members but not administrators. Maybe I am looking in the wrong place? Thanks paul

3 8

Settings: No templates are visible in settings
by paul＠arenson.org Sept. 3, 2019

Sept. 3, 2019

I am struggling to find out why I do not see TEMPLATES in the Settings. When I first logged they were there. Many of them. Then, toying with the settings, they disappeared. I cannot find them now. Originally i thought I would not list the mailing list but changed my mind. Likewise, archives are now listed as public. I am still thinking about that. In any case, I want to see if there are templates to allow me to change headers or footers in messages, such as give the address of the list. Thank you for your advice. P.S> Sorry i asked too many questions the other day. Trying to keep this brief.

1 1

still up to date?
by mailman3＠malevil.cc Aug. 27, 2019

Aug. 27, 2019

I have subscribed to the following rss feeds to keep track of updates: https://libraries.io/pypi/mailman/versions.atom https://libraries.io/pypi/mailman-hyperkitty/versions.atom https://libraries.io/pypi/HyperKitty/versions.atom https://libraries.io/pypi/postorius/versions.atom https://gitlab.com/mailman/mailman-suite/commits/master?feed_token=yitiVdhX… but none of them was updated for half a year, is that actually the status? No current releases or updates for mailman3?

2 2

Status of hosting options after a year for non techies
by paul＠arenson.org Aug. 26, 2019

Aug. 26, 2019

Hi all This is Paul Arenson in Japan. I temporarily dropped off the list year ago when it was clear there was no Cpanel Solution forthcoming. I thought I would check in again. I used to be on CPanel and I was advised it might be a wait. 1) I am now on Plesk. I heard Plesk might allow Docker and installation may be possible. Am I misinformed or a possibility? --->Assuming no, I guess jump to question 4. 2) Assuming yes, I see a list of people who can possibly help install. https://wiki.list.org/COM/Mailman%20consulting%20services Only a few people on it. Has anyone used any or can recommend someone. 3) There is also a Python list, but before I submit a request, is this something that works on general. For example, when a new version or bug release comes up, will I need to call on the person or persons again? Is it still better to wait? 4) Hosting services: Only a few listed at https://wiki.list.org/COM/Mailman%20hosting%20services are or Mailman 3. Has anyone used this: https://mailman3.com The site seems incomplete. Or this: https://maadix.net/en/servers Looks good. Might be a bit expensive for my use. Or this: https://www.mailmanhost.com/mailman-3-hosting/ Also looks good. A long time ago we corresponded. They seem reasonable. My list is about 700 people. These seem to be the same people: https://www.emwd.com/mailman-3-shared-list-hosting/ Any others I should know about? Thanks in advance. paul arenson japan

2 3

MM3 + Postfix + Dovecot + PostfixAdmin
by maztec＠gmail.com Aug. 20, 2019

Aug. 20, 2019

System: Debian Buster Install: Functioning Postfix w/ Dovecot using PostfixAdmin for virtual domain and mailbox management Definitions: Host.TLD = my server's domain name; List.TLD = listserv's domain name; ListName = list's name. Issue: Postfix LMTP does not pass messages onto MM3:LMTP. Instead the behavior summary is as follows: postfix/lmtp[...]: ###: to=<mailman3:[127.0.0.1]:8024@Host.TLD>, orig_to=<ListName-confirm+###(a)ListServer.TLD>, relay=Host.TLD[private/dovecot-lmtp], delay=439, delays=439/0.03/0.01/0.01, dsn=5.1.1, status=bounced (host Host.TLD[private/dovecot-lmtp] said: 550 5.1.1 <"mailman3:[127.0.0.1]:8024"@Host.TLD> User doesn't exist: mailman3:[127.0.0.1]:8024@Host.TLD (in reply to RCPT TO command)) The problem appears to be that Postfix defaults the relay to being private/dovecot-lmtp and not the general lmtp protocol or the mailman3 dedicated lmtp protocol. ---------------------------------------- ---------------------------------------- Details: # cat /proc/version Linux version 4.19.0-5-amd64 (debian-kernel(a)lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.37-5+deb10u1 (2019-07-19) ---------------------------------------- # postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no broken_sasl_auth_clients = yes compatibility_level = 2 debug_peer_level = 4 html_directory = /usr/share/doc/postfix/html inet_interfaces = all inet_protocols = ipv4 local_recipient_maps = $virtual_mailbox_maps,regexp:/var/lib/mailman3/data/postfix_lmtp local_transport = virtual mailbox_size_limit = 0 mydestination = localhost myhostname = rijr.com mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 myorigin = /etc/mailname owner_request_special = no policy-spf_time_limit = 3600s readme_directory = /usr/share/doc/postfix recipient_delimiter = + relay_domains = $mydestination,regexp:/var/lib/mailman3/data/postfix_domains relayhost = smtp_tls_loglevel = 1 smtp_tls_security_level = may smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = $mydomain smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_tls_security_options = $smtpd_sasl_security_options smtpd_sasl_type = dovecot smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/ssl/private/vmail.crt smtpd_tls_key_file = /etc/ssl/private/vmail.key smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtpd_use_tls = yes transport_maps = hash:/etc/postfix/transport,regexp:/var/lib/mailman3/data/postfix_lmtp virtual_alias_domains = proxy:mysql:/etc/postfix/virtual_alias_domains.cf virtual_alias_maps = proxy:mysql:/etc/postfix/virtual_alias_maps.cf,proxy:mysql:/etc/postfix/virtual_alias_domains_maps.cf,regexp:/var/lib/mailman3/data/postfix_lmtp virtual_gid_maps = static:5000 virtual_mailbox_base = /home/vmail virtual_mailbox_domains = proxy:mysql:/etc/postfix/virtual_mailbox_domains.cf virtual_mailbox_limit = 512000000 virtual_mailbox_maps = regexp:/var/lib/mailman3/data/postfix_lmtp,proxy:mysql:/etc/postfix/virtual_mailbox_maps.cf virtual_minimum_uid = 5000 virtual_transport = lmtp:unix:private/dovecot-lmtp virtual_uid_maps = static:5000 # cat master.cf [grep'd for lmtp and mailman3] lmtp unix - - y - - lmtp mailman3 unix - - - - - lmtp -o lmtp_send_xforward_command=yes -o disable_dns_lookups=yes ---------------------------------------- # cat mailman.cfg [mta section only] [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver smtp_host: localhost smtp_port: 25 lmtp_host: 127.0.0.1 lmtp_port: 8024 configuration: /etc/mailman3/postfix-mailman.cfg ---------------------------------------- # cat /var/lib/mailman3/data/postfix_lmtp # AUTOMATICALLY GENERATED BY MAILMAN ON 2019-08-06 06:39:21 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Aliases which are visible only in the @List.TLD domain. /^List(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-bounces(\+.*)?(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-confirm(\+.*)?(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-join(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-leave(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-owner(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-request(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-subscribe(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-unsubscribe(a)List\.TLD$/ mailman3:[127.0.0.1]:8024 ---------------------------------------- # cat postfix-mailman.cfg [postfix] # if I don't have postmap_command set, mailman errors out, even though transport_file_type is regex. postmap_command: /usr/sbin/postmap transport_file_type: regex ---------------------------------------- # cat mail.debug Aug 6 01:03:28 Host dovecot: lmtp(23055): Connect from local Aug 6 01:03:28 Host dovecot: auth: Debug: master in: USER#0112#011mailman3:[127.0.0.1]:8024@Host.TLD#011service=lmtp Aug 6 01:03:28 Host dovecot: auth: userdb(?): Username character disallowed by auth_username_chars: 0x3a (username: mailman3:[127.0.0.1]:8024@Host.TLD) Aug 6 01:03:28 Host dovecot: auth: Debug: userdb out: NOTFOUND#0112 Aug 6 01:03:28 Host postfix/cleanup[22970]: 2937FB0: message-id=<20190806080328.2937FB0(a)rijr.com> Aug 6 01:03:28 Host postfix/lmtp[23054]: CA030AA: to=<mailman3:[127.0.0.1]:8024@rijr.com>, orig_to=<gilman-clan-confirm+51c0564a95aa31a8840305bec77e9023baa46477(a)List.TLD>, relay=Host.TLD[private/dovecot-lmtp], delay=439, delays=439/0.03/0.01/0.01, dsn=5.1.1, status=bounced (host Host.TLD[private/dovecot-lmtp] said: 550 5.1.1 <"mailman3:[127.0.0.1]:8024"@Host.TLD> User doesn't exist: mailman3:[127.0.0.1]:8024@Host.TLD (in reply to RCPT TO command)) Aug 6 01:03:28 Host dovecot: lmtp(23050): Disconnect from local: Client has quit the connection (state=READY) Aug 6 01:03:28 Host postfix/lmtp[22981]: C0EEEA8: to=<mailman3:[127.0.0.1]:8024@Host.TLD>, orig_to=<ListName-confirm+51c0564a95aa31a8840305bec77e9023baa46477(a)List.TLD>, relay=Host.TLD[private/dovecot-lmtp], delay=566, delays=566/0.04/0.01/0.01, dsn=5.1.1, status=bounced (host Host.com[private/dovecot-lmtp] said: 550 5.1.1 <"mailman3:[127.0.0.1]:8024"@Host.TLD> User doesn't exist: mailman3:[127.0.0.1]:8024@Host.TLD (in reply to RCPT TO command)) ---------------------------------------- I have played with dozens of variations on the configuration files and tracing what is going on. The only reason I see that relay=Host.TLD[private/dovecot-lmtp] is because of: virtual_transport = lmtp:unix:private/dovecot-lmtp Hopefully I can get this figured out .... - Jay

4 6

Some threads donw download
by Andrew Bernard Aug. 18, 2019

Aug. 18, 2019

Further to my previos queries about download failures - the generated file is corrupt, I turned up debugging and I now see this on the threads that don't download. [pid: 25261|app: 0|req: 27/27] 59.167.198.231 () {48 vars in 1198 bytes} [Sat Aug 17 08:55:15 2019] GET /archives/list/jackrail@jackrail.org/export/jackrail@jackrail.org-CKOK7BEELMHIFF2MTPUZXQXQYZZGWF3D.mbox.gz?thread=CKOK7BEELMHIFF2MTPUZXQXQYZZGWF3D => generated 0 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 241 bytes (0 switches on core 0) Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.2.3-py3.7.egg/hyperkitty/views/mlist.py", line 327, in stream_mbox msg = email.as_message() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.2.3-py3.7.egg/hyperkitty/models/email.py", line 188, in as_message subtype=mimetype[1], filename=attachment.name) File "/usr/lib/python3.7/email/message.py", line 1147, in add_attachment self._add_multipart('mixed', *args, _disp='attachment', **kw) File "/usr/lib/python3.7/email/message.py", line 1135, in _add_multipart part.set_content(*args, **kw) File "/usr/lib/python3.7/email/message.py", line 1162, in set_content super().set_content(*args, **kw) File "/usr/lib/python3.7/email/message.py", line 1092, in set_content content_manager.set_content(self, *args, **kw) File "/usr/lib/python3.7/email/contentmanager.py", line 35, in set_content handler = self._find_set_handler(msg, obj) File "/usr/lib/python3.7/email/contentmanager.py", line 58, in _find_set_handler raise KeyError(full_path_for_error) KeyError: 'builtins.NoneType' What is this saying? Andrew [Somewhat perplexed at why Mailman 3 is behaving so badly for me...]

2 2

Idle connections to Postgres
by Andrew Bernard Aug. 17, 2019

Aug. 17, 2019

I have 20 idle connections to Postgres with Mailman 3. What are these for, and what are they doing (apart from apparently nothing)? main: mailman3 mailman3 ::1(46058) idle Andrew

1 0

Compression error
by Andrew Bernard Aug. 17, 2019

Aug. 17, 2019

I did a git pull today. Now I get the following error and the archives wont display: compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "b5ae52a1ebb585c556edceb45a148cb948b2a2673d133a340a90cd02fdd34828" is missing from offline manifest. You may need to run "python manage.py compress". Here is the original content: Any clues?

2 2

Download errors
by Andrew Bernard Aug. 17, 2019

Aug. 17, 2019

Whenever i try to download an archive, the file seems to be generated and it downloads, but it will never open, with errors like 'an error occurred while extracting files'. What can be causing this?

2 2

qcluster CPU usage
by Andrew Bernard Aug. 17, 2019

Aug. 17, 2019

Recently the qcluster program is engaging the CPU continuously at about 5% CPU. While that may not be much, previously it was not doing this. What could be the cause of this? I cant see any warnings in the logs.

1 0

Hyperkitty reply indent
by Andrew Bernard Aug. 16, 2019

Aug. 16, 2019

Is there any way the indent in the thread view can be reduced or set to zero? My users are complaining that by the time you have a reply four or five deep the view on a mobile phone leaves only one or two words, because it does not wrap responsively correctly (have I set something wrongly?) Andrew

3 8

Question after upgrade to mm 3.2.3 and latest postorius
by David Krantz Aug. 13, 2019

Aug. 13, 2019

Hi all, I have just upgraded to mailman 3.2.3. My installation is from the git sources and I use two different venv:s, one for the web part as I did not want to compile the web server plugin to a newer python version than 3.5, and one venv for the rest running python 3.7. Apart from some missing dependencies and some changes that affected the django config I just have two issues with postorius: 1. The index page with all lists is not sorted in any useful way although it is based on something as the order seems to be stable. Prior to the upgrade the lists were sorted in alphabetical order after list name which is a lot more usable for me as admin. If I click the link at the bottom with "click to view all lists" I get the old behaviour. 2. The template menu simply does not work, it seems as if some database table is missing even after running migrations. Django logs the following: ERROR 2019-08-11 20:58:55,038 exception 11837 139756347221760 Internal Server Error: /mailman3/lists/TESTLIST/templates Traceback (most recent call last): File "/...VENV3.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute return self.cursor.execute(sql, params) psycopg2.errors.UndefinedTable: relation "postorius_emailtemplate" does not exist LINE 1: ...ext", "postorius_emailtemplate"."identifier" FROM "postorius.. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/...VENV3.5/lib/python3.5/site-packages/django/core/handlers/exception.py", line 35, in inner response = get_response(request) File "/...VENV3.5/lib/python3.5/site-packages/django/core/handlers/base.py", line 158, in _get_response response = self.process_exception_by_middleware(e, request) File "/...VENV3.5/lib/python3.5/site-packages/django/core/handlers/base.py", line 156, in _get_response response = response.render() File "/...VENV3.5/lib/python3.5/site-packages/django/template/response.py", line 106, in render self.content = self.rendered_content File "/...VENV3.5/lib/python3.5/site-packages/django/template/response.py", line 83, in rendered_content content = template.render(context, self._request) File "/...VENV3.5/lib/python3.5/site-packages/django/template/backends/django.py", line 61, in render return self.template.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 175, in render return self._render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 167, in _render return self.nodelist.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 943, in render bit = node.render_annotated(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 910, in render_annotated return self.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/loader_tags.py", line 155, in render return compiled_parent._render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 167, in _render return self.nodelist.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 943, in render bit = node.render_annotated(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 910, in render_annotated return self.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/loader_tags.py", line 67, in render result = block.nodelist.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 943, in render bit = node.render_annotated(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/base.py", line 910, in render_annotated return self.render(context) File "/...VENV3.5/lib/python3.5/site-packages/django/template/defaulttags.py", line 313, in render if match: File "/...VENV3.5/lib/python3.5/site-packages/django/db/models/query.py", line 276, in __bool__ self._fetch_all() File "/...VENV3.5/lib/python3.5/site-packages/django/db/models/query.py", line 1179, in _fetch_all self._result_cache = list(self._iterable_class(self)) File "/...VENV3.5/lib/python3.5/site-packages/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File "/...VENV3.5/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1063, in execute_sql cursor.execute(sql, params) File "/...VENV3.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 68, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/...VENV3.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers return executor(sql, params, many, context) File "/...VENV3.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute return self.cursor.execute(sql, params) File "/...VENV3.5/lib/python3.5/site-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/...VENV3.5/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute return self.cursor.execute(sql, params) django.db.utils.ProgrammingError: relation "postorius_emailtemplate" does not exist LINE 1: ...ext", "postorius_emailtemplate"."identifier" FROM "postorius... Cheers // David

3 4

Internal server error
by Andrew Bernard Aug. 9, 2019

Aug. 9, 2019

My mailbox is filling up with dozens of notification of an internal server error. i wont post the whole mail as it is long, but here is the top: Internal Server Error: /archives/list/jackrail-meta(a)jackrail.org/recent-threads TypeError at /archives/list/jackrail-meta(a)jackrail.org/recent-threads 'datetime.date' object is not iterable Request Method: GET Request URL: https://list.jackrail.org/archives/list/jackrail-meta@jackrail.org/recent-t… Django Version: 2.1.9 Python Executable: /opt/mailman/mm/venv/bin/uwsgi Python Version: 3.7.3 What is causing this? [Assuming that knowing the cause will lead to me fixing it.] Andrew

2 1

Error in mailman3 debian package, python error
by Lars Schimmer Aug. 8, 2019

Aug. 8, 2019

Hi! Mailman Core Version GNU Mailman 3.2.1 (La Villa Strangiato) Mailman Core API Version 3.0 Mailman Core Python Version 3.7.3 (default, Apr 3 2019, 05:39:12) [GCC 8.3.0] Since yesterday our mailman3 (debian package) throws this error: Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x8D\\x8E' for column `mailman3web`.`hyperkitty_email`.`subject` at row 1") Request Method: POST Request URL: http://localhost/mailman3/hyperkitty/api/mailman/archive?key=zMxTGlteq4tqkL… Django Version: 1.11.22 Python Executable: /usr/bin/uwsgi-core Python Version: 3.7.3 Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Wed, 7 Aug 2019 08:03:47 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x8D\\x8E' for column `mailman3web`.`hyperkitty_email`.`subject` at row 1")) was the direct cause of the following exception: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator 69. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view 58. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 115. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list 151. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 808. force_update=force_update, update_fields=update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 838. updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 924. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 963. using=using, raw=raw) File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 85. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1079. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1112. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 94. six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python3/dist-packages/django/utils/six.py" in reraise 685. raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x8D\\x8E' for column `mailman3web`.`hyperkitty_email`.`subject` at row 1") Request information: USER: AnonymousUser Has anyone a idea what went wrong or howto fix it? MfG, Lars Schimmer -- ------------------------------------------------------------- TU Graz, Institut für ComputerGraphik & WissensVisualisierung Tel: +43 316 873-5405 E-Mail: l.schimmer(a)cgv.tugraz.at Fax: +43 316 873-5402 PGP-Key-ID: 0x4A9B1723

3 4

exposing Mailman3 API securely
by dancab＠caltech.edu Aug. 5, 2019

Aug. 5, 2019

I'm working on a container based Mailman3 solution for my university. We have a help desk tool which needs to access the API remotely for creating and deleting Mailman lists. The API will be restricted to a private network, behind a firewall, etc. What is the simplest way to expose the API to an external host? At the moment, I seem to have this partially working via an Apache virtual host configuration which simply proxies via api.example.edu:80 to localhost:8001 on the container. I'm able to remotely use the mailmanclient module for the client.lists, client.system and client.members commands. However running client.domains generates an API connection error. So it seems unstable. Any thoughts or suggestions? Thanks. ---- Dan Caballero Systems Administrator Academic Computing Solutions IMSS - Caltech https://imss.caltech.edu

4 10

Replies unthreading
by Andrew Bernard Aug. 1, 2019

Aug. 1, 2019

In my new list, I have a user - one of the foundation members, so an important use case - who will not be talked out of using a very old version of Eudora (!) for mail. Replies to threads from that mailer keep sometimes being properly threaded in Hyperkitty and then randomly jumping out into a new thread. This messes up the web view of the threads, and it's not right. Yet the threading appears to be correct in mail programs such as Thunderbird, using the email based interface to the list. Have others seen such behaviour with ancient mailers? Is it the mailer, or is there a known issue with threading the web interface? Looking carefully at message ID's and how they are threaded to examine the behaviour, I can't see what is causing this. Possibly some interaction with other RFC-822 headers? Andrew

3 3

Authenticating against Hyperkitty's DB
by Marvin Gülker July 29, 2019

July 29, 2019

Hi, inspired by the question about LDAP, I want to ask a related question. We're currently thinking about adding a wiki to our small OSS project, and requiring users to have multiple accounts for the same project is nonsense. We don't have LDAP set up yet, but we do have a running Mailman 3 installation, including Hyperkitty. Thus I thought about whether it's possible to just write some glue script that authenticates against the user information in Hyperkitty's database. Creating an account for Hyperkitty would then be sufficient to get access to the wiki. Could someone give me a hint where and in what format Hyperkitty stores the user information into its database? Marvin -- Blog: https://mg.guelker.eu

2 6

Exim, Mailman3 and missing Message-ID:
by Robert Lister July 29, 2019

July 29, 2019

Just a note that may help some people. After a bit of a learning curve, managed to get Mailman3 working in Debian 10 using the Debian package. I have mailman3-full + exim4 + apache2 setup: I did quite a lot of testing and getting it working, then when all seemed well, declared success and made it live. As (bad) luck would have it, the very first real message bounced: :-( 2019-07-25 20:53:09 1hqjnY-0005ms-RD <= xxxxxxx(a)ntlworld.com H=cloudi.spamtitan.com [52.26.55.167] P=esmtps X=TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256 CV=no S=7854 DKIM=ntlworld.com 2019-07-25 20:53:09 1hqjnY-0005ms-RD ** mylist(a)lentil.org R=mailman3_router T=mailman3_t H=127.0.0.1 [127.0.0.1]: LMTP error after DATA: 550 No Message-ID header provided 2019-07-25 20:53:09 1hqjnZ-0005mx-3F <= <> R=1hqjnY-0005ms-RD U=Debian-exim P=local S=9222 The user forwarded me the bounce, which contained the original message and headers, and indeed the original message did not contain a Message-ID: header. (Oddly, his message to me reporting the error using the same mail client DID contain a Message-ID: header.) Mail client: X-Mailer: Microsoft Outlook 16.0 So it seems in certain circumstances, Outlook doesn't put a Message-ID: header in its messages. The change from Mailman2.x to Mailman3 means it's now using LMTP to deliver to mailman instead of a local submission seems to have exposed this issue. Exim tries to be helpful for "locally submitted" messages: <https://www.exim.org/exim-html-current/doc/html/spec_html/ch-message_proces…> "12. The Message-ID: header line If a locally-generated or submission-mode incoming message does not contain a Message-ID: or Resent-Message-ID: header line, and the suppress_local_fixups control is not set, Exim adds a suitable header line to the message. If there are any Resent-: headers in the message, it creates Resent-Message-ID:. The id is constructed from Exim's internal message id, preceded by the letter E to ensure it starts with a letter, and followed by @ and the primary host name." However: "Note: Messages received over TCP/IP on the loopback interface (127.0.0.1 or ::1) are not considered to be locally-originated. Exim does not treat the loopback interface specially in any way." Therefore, Mailman3 using LMTP means exim will NOT "fix up" this problem before mailman3 receives the message. My solution for now (apart from telling people not to use old broken Outlook!) is to modify the exim mailman3_transport so that any message with a missing Message-ID: will have one generated by exim before it's passed to LMTP. This reproduces the same behaviour exim uses for locally submitted messages. (headers_remove ensures that we end up only with one Message-ID: header and not two, then we put back either the original Message-ID, or if that's not there, use Exim's ID to generate a new Message-ID header.) Here's an updated version of the example at: https://mailman.readthedocs.io/en/latest/src/mailman/docs/mta.html#exim # /etc/exim4/conf.d/main/25_mm3_macros # The colon-separated list of domains served by Mailman. domainlist mm_domains=list.example.net MM3_LMTP_PORT=8024 # MM3_HOME must be set to mailman's var directory, wherever it is # according to your installation. # debian default is /var/lib/mailman3 MM3_HOME=/opt/mailman/var MM3_UID=list MM3_GID=list ################################################################ # The configuration below is boilerplate: # you should not need to change it. # The path to the list receipt (used as the required file when # matching list addresses) MM3_LISTCHK=MM3_HOME/lists/${local_part}.${domain} # /etc/exim4/conf.d/router/455_mm3_router mailman3_router: driver = accept domains = +mm_domains require_files = MM3_LISTCHK local_part_suffix_optional local_part_suffix = \ -bounces : -bounces+* : \ -confirm : -confirm+* : \ -join : -leave : \ -owner : -request : \ -subscribe : -unsubscribe transport = mailman3_transport # /etc/exim4/conf.d/transport/55_mm3_transport mailman3_transport: driver = smtp protocol = lmtp allow_localhost hosts = localhost port = MM3_LMTP_PORT rcpt_include_affixes = true mailman3_transport: driver = smtp protocol = lmtp allow_localhost hosts = localhost port = MM3_LMTP_PORT rcpt_include_affixes = true headers_remove = message-id headers_add = "Message-ID: ${if def:header_message-id:{$h_message-id:}{<E${message_exim_id}@${qualify_domain}>}}"

4 6