- Mailman-users - lists.mailman3.org

HyperKitty async tasks runner: systemd qcluster service fails eventually
by actionmystique＠gmail.com Dec. 9, 2019

Dec. 9, 2019

Ubuntu 20.04 mailman3 3.2.2-1 python3-django-hyperkitty 1.3.0-1ubuntu1 According to [Hyperkitty Asynchronous tasks paragraph](https://hyperkitty.readthedocs.io/en/latest/install.html#asynchronous-tasks), we can start a qcluster systemd service file: ``` [Unit] Description=HyperKitty async tasks runner After=network.target remote-fs.target [Service] ExecStart=/usr/bin/django-admin qcluster --pythonpath <django_project_path> --settings settings User=<user> Group=<group> Restart=always [Install] WantedBy=multi-user.target ``` Starting it works fine: ``` # systemctl restart qcluster # systemctl status qcluster ● qcluster.service - HyperKitty async tasks runner Loaded: loaded (/etc/systemd/system/qcluster.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2019-12-04 15:09:46 CET; 2ms ago Main PID: 1055959 ((go-admin)) Tasks: 0 (limit: 9284) Memory: 0B CGroup: /system.slice/qcluster.service └─1055959 (go-admin) Dec 04 15:09:46 samsung4-ubuntu systemd[1]: Started HyperKitty async tasks runner. ``` However, the service fails after a few seconds: ``` # systemctl status qcluster ● qcluster.service - HyperKitty async tasks runner Loaded: loaded (/etc/systemd/system/qcluster.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2019-12-04 15:09:48 CET; 1min 1s ago Process: 1055961 ExecStart=/usr/bin/django-admin qcluster --pythonpath <django_project_path> --settings settings (code=exited, status=1/FAILURE) Main PID: 1055961 (code=exited, status=1/FAILURE) Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Service RestartSec=100ms expired, scheduling restart. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Scheduled restart job, restart counter is at 5. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: Stopped HyperKitty async tasks runner. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Start request repeated too quickly. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: qcluster.service: Failed with result 'exit-code'. Dec 04 15:09:48 samsung4-ubuntu systemd[1]: Failed to start HyperKitty async tasks runner. ``` The corresponding log file is empty and I cannot find any issue in the other logs. Any suggestion? Also, another doc paragraph [Running the task queue (required)](http://docs.mailman3.org/en/latest/config-web.html#running-the-t…) talks about another required command "python manage.py qcluster"; so which one is it?

3 9

Source Installation Assistance
by Brian Carpenter Dec. 6, 2019

Dec. 6, 2019

I am trying to install Mailman 3 from source using: https://docs.mailman3.org/en/latest/index.html My first question is where is the best place to get assistance or ask questions regarding the installation process? I am not sure if mailman-users is that place anymore. Thanks, Brian

4 6

Cannot start lmtp runner due to error while attempting to bind on address ('127.0.0.1', 8024): address already in use
by actionmystique＠gmail.com Dec. 4, 2019

Dec. 4, 2019

Ubuntu focal 20.04 mailman3 3.2.2-1 In ```/etc/mailman3/mailman.cfg```: ``` # Where the LMTP server listens for connections. Use 127.0.0.1 instead of # localhost for Postfix integration, because Postfix only consults DNS # (e.g. not /etc/hosts). # Configuration is for a system where Mailman and the MTA are on the same host lmtp_host: 127.0.0.1 lmtp_port: 8024 ``` Stopping mailman3 and checking that no lmtp runner is active: ``` # systemctl stop mailman3 # ps -aux |grep mailman|grep -Pv "(postgres|grep)" www-data 855395 0.0 0.4 82856 37904 ? Ss 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 855411 0.0 0.3 156588 25724 ? Sl 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 858518 0.0 0.2 82856 22532 ? Ss 13:52 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini ``` Starting mailman3 and checking all active mailman3 runners: ``` # ps -aux |grep mailman|grep -Pv "(postgres|grep)" www-data 855395 0.0 0.4 82856 37904 ? Ss 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 855411 0.0 0.3 156588 25724 ? Sl 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 858612 0.0 0.2 82856 22532 ? Ss 13:54 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini list 858626 17.8 0.9 103248 74680 ? Ss 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/master --force -C /etc/mailman3/mailman.cfg list 858628 27.2 0.7 87176 62244 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 list 858629 28.6 0.8 89116 64044 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 list 858630 29.2 0.8 90184 64972 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 list 858631 27.0 0.7 87560 62420 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 list 858632 28.8 0.8 89620 64508 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=lmtp:0:1 list 858633 26.6 0.7 84464 60800 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 list 858634 26.2 0.7 81964 58580 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 list 858635 28.4 0.7 88692 63648 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 list 858636 26.4 0.7 83308 59820 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 list 858637 26.8 0.7 84076 60828 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 list 858638 29.2 0.8 90176 64964 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 list 858639 27.4 0.7 87564 62528 ? R 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 ``` We clearly see that lmtp runner is active; then it exits after a few seconds: ``` # ps -aux |grep mailman|grep -Pv "(postgres|grep)" www-data 855395 0.0 0.4 82856 37904 ? Ss 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 855411 0.0 0.3 156588 25724 ? Sl 13:01 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini www-data 858612 0.0 0.2 82856 22532 ? Ss 13:54 0:00 /usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini list 858626 2.7 0.9 103248 74680 ? Ss 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/master --force -C /etc/mailman3/mailman.cfg list 858628 4.3 0.9 103260 74704 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 list 858629 4.3 0.9 103256 74964 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 list 858630 4.3 0.9 103256 74920 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 list 858631 4.3 0.9 103256 74804 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 list 858633 4.4 0.9 103924 75444 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 list 858634 4.3 0.9 103256 74392 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 list 858635 4.3 0.9 103260 74876 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 list 858636 4.4 0.9 182412 79472 ? Sl 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 list 858637 4.4 0.9 103256 74964 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 list 858638 4.5 0.9 103256 74808 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 list 858639 4.4 0.9 103256 74776 ? S 13:54 0:01 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 ``` The systemd service status gives a clue: ``` # systemctl status mailman3 ● mailman3.service - Mailman3 server Loaded: loaded (/lib/systemd/system/mailman3.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2019-12-02 13:54:45 CET; 2min 51s ago Docs: man:mailman(1) https://mailman.readthedocs.io/ Process: 858617 ExecStart=/usr/bin/mailman -C /etc/mailman3/mailman.cfg start --force (code=exited, status=0/SUCCESS) Main PID: 858626 (python3) Tasks: 14 (limit: 9284) Memory: 688.5M CGroup: /system.slice/mailman3.service ├─858626 /usr/bin/python3 /usr/lib/mailman3/bin/master --force -C /etc/mailman3/mailman.cfg ├─858628 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 ├─858629 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 ├─858630 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 ├─858631 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 ├─858633 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 ├─858634 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 ├─858635 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 ├─858636 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 ├─858637 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 ├─858638 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 └─858639 /usr/bin/python3 /usr/lib/mailman3/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: self.lmtp.start() Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3/dist-packages/aiosmtpd/controller.py", line 60, in start Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: raise self._thread_exception Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3/dist-packages/aiosmtpd/controller.py", line 40, in _run Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: ssl=self.ssl_context)) Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: return future.result() Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: File "/usr/lib/python3.7/asyncio/base_events.py", line 1374, in create_server Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: % (sa, err.strerror.lower())) from None Dec 02 13:54:52 samsung4-ubuntu mailman3[858617]: OSError: [Errno 98] error while attempting to bind on address ('127.0.0.1', 8024): address already in use ``` Of course the port is already in use: it is used by dovecot. ``` tcp6 0 0 ::1:8024 :::* LISTEN 0 1317360 129620/dovecot tcp 0 0 127.0.0.1:8024 0.0.0.0:* LISTEN 0 1316324 129620/dovecot ``` I must be missing something.

3 5

Changelog and/or Release Notes
by Brian Carpenter Nov. 28, 2019

Nov. 28, 2019

Now that I am using Mailman 3.3, where can I find the release notes and/or changelog for core, postorius and hyperkitty under Mailman 3.3? Thanks! Brian

2 2

Re: [Mailman-Users] Mailman 3 Ready?
by Stephen J. Turnbull Nov. 27, 2019

Nov. 27, 2019

Allan Hansen writes: > What is the best, easiest, guaranteed-to-work way to get this > Mailman 3 up and running (OS, add-ons, options, etc.). When I tried > myself, I saw a lot of options that I really did not really care to > have because I did not know the consequences of each. I’m an > application programmer, not a systems programmer. It's email, there are no guarantees. I hate to say that, but that's the way it is. In particular, I'm concerned about the "iRedMail" integration, which may be a complete wildcard from my point of view. With respect to the relevant iRedMail components, I can say that Mailman 3 is perfectly happy with Postfix as MTA (it's the recommended configuration, although I use Exim4 for historical reasons), and I've been satisfied with a small-scale set of lists (about 10 active ones, with at most 20 subscribers total) using Apache+mod_swgi to host Postorius and HyperKitty. The problem would be if your friend (is that the Frank in the CC? hi, Frank!) really really wants to manage Mailman 3 through iRedAdmin -- that will get messy I suspect, and pragmatically speaking, we can't support it, we don't have the cycles. So he'd be on his own, especially since the iRedMail folks have their own bespoke mailing list manager. I would recommend avoiding iRedAdmin integration, and just using Postorius since the subscribers and non-site-admin moderators (if any) will need to use Postorius anyway. Ignoring the iRedAdmin issue, the absolute simplest is 1. Install any Linux distribution with a Docker package. 2. Install the distribution's Docker package. 3. Download and install Mailman Project's Mailman 3 container. 4. Upgrade the container manually (ie, download and reinstall) as appropriate. For reference, what I would recommend to my friends, who would give me hands-on access in emergencies: 1. Install the Debian distribution. (What I use.) 2. Pin the Mailman 3 packages to "experimental". The idea would be to get Mailman 3 upgrades as fast as possible, even at the risk of packaging issues. (My experience with default-to-experimental system for several years was that this happened at most once a year, and the three or four cases all got fixed within days.) 3. Install the Mailman 3 packages. 4. Adjust any dependencies that need non-"stable" versions. 5. Upgrade via apt or whatever occasionally. 6. When Mailman 3 is working well, downgrade the pins to a more stable level. Since you do have such a friend, there's probably a similar process that works for his distribution-of-choice. > The problem is that my MacOS users are getting tripped up by the > mangling, as it is hidden by the mail clients. I have instructed > them to remove previous recipients regularly from the list of such, > if they contain the ‘via list’ string. But they don’t read my > emails, and we see many embarrassing email ssent to the lists that > are meant for one-on-one communication. I guess we could provide a setting where reply-to goes *only* to author, instead of to author *and* list as we currently recommend. But that is going to make you (or list owners) the focus of user ire because their intended replies to list go only to author. What do you think? tl;dr The rest is me blowing off steam. :-) > The fact that the mangling will continue is a disappointment. It's a disappointment to the sane people at the IETF, too, including the rep from Yahoo!, who's a very smart lady (I may be biased, she gave me a kitten once. ;-) Yahoo! is violating the original concept (DMARC "p=reject" was supposed to be used *only* for "transactional" mail, i.e., direct private communication between a business and its clients), but I see her point: when you've had a hundred million or so address books hacked and sold to spammers, and the spammers are sending literally millions of "recommendation from a friend" spams per hour for days on end, you use the tools to hand. > I’m inclined to tell all my AOL and Yahoo users and others in the same > boat to find another email service. I know the feeling. I'm lucky in that respect: the Ministry of Education prohibits use of yahoo mailboxes at Japanese universities. (Which is a WTF of a different kind: yahoo.co.jp is a franchise, not controlled by yahoo.com, and doesn't use p=reject at least at the time MoE issued that policy.) Mail is just hard. It was designed for a "friendly network of users", not for today's rather hostile open Internet, and even if they had envisioned this in the 70s when the message format and SMTP mail transmission protocol were designed, the fundamental problem is that pretty much everyone has a genuine need, even if only occasional, to receive messages from untrusted and even unknown senders.

1 0

Setting up From: field as list address
by Steven Maddox Nov. 25, 2019

Nov. 25, 2019

Hi folks, I'm trying to configure our mailman3 lists (via postorious) in a way that'll mean they'll behave in the way way that I've witnessed on another 3rd party list (one running mailman2). This is where (when receiving a message from the list) the From: is the list itself, and the person who actually sent it bizarrely appears as a CC:, for example... From: Steven Madox via Shropshire LUG <shropshire(a)mailman.lug.org.uk> Subject: [Shropshire LUG] Whatever To: shropshire(a)mailman.lug.org.uk CC: s.maddox(a)lantizia.me.uk At the moment our settings are the default/recommended... I *was* just going to set munging to 'Reply goes to list' which would just set a 'Reply-To' header... until I realised that... a) that won't help Thunderbird users as it just ignores Reply-To if it came from a list b) doesn't fix the issue with people having their own SPF/DKIM set on their mail domains So ultimately I'm starting to see why this 3rd party mailing list (another linux user group, a neighbouring one just south of us) does it this way... if anyone knows the mailman3 equivalent setting? p.s. I've read both the mailman2 and 3 notes related to why munging is not recommended, but I'd still like to know how this is done - to keep our options open more than anything. -- Steven Maddox Lantizia

2 3

[Mailman-Users] Mailman 3 Ready?
by Stephen J. Turnbull Nov. 25, 2019

Nov. 25, 2019

Reply-To set to mailman-users(a)mailman3.org. Allan Hansen writes: > I need to upgrade from Mailman 2.1 - my hardware needs upgrading > (badly) and the address mangling is having my subscribers rip their > hair out. That's not going to go away. We have ARC support in Mailman 3, but (a) like most envelope manipulation, that should be done in boundary MTAs, not in Mailman, and (b) ARC implementation is not universal. *Very likely* some of your subscribers are going to lose mail if you depend on ARC, rather than munge from. > I need to get this done before they start on my already thinning > hair! I sympathize. I tell the little kids I meet that my hair grows out of the bottom of my face, not the top (it's very alliterative in Japanese[1], the kids frequently don't get it but the accompanying adults usually laugh). > I’m ready and eager to move on to MM3 on a Linux box. The obvious "turn-key" installation is to use the Docker containers, which Abhilash keeps fairly up-to-date. But see comment below about universal working solutions. Which distro do you have? Are you using the distro's packages of Mailman 3 (not yet a good idea, unfortunately), or installing from source? > I have looked into it a few times now and I keep running into the > same blocks. I don't see much moving either. I was hoping someone > (Mailman developing community) would come up with a better working > solution. It's email. There are no universal working solutions. > [Some third party wrote:] My concern is that the Mailman3 is not > ready. There are too many dead ends and undocumented stuff. Who is this guy and has he consulted us? I'd like to do an archive search if he's done so to see what the issues are. > For Mailman2 I have a working integration with iRedMail, that does > not seem to work with 3. What's iRedMail? How does it fit into the community your lists serve? > So, is MM3 ready for the big screen? Works for us, for values of us including python.org. > If so, is there some way/how/where/when that my friend can get help > helping me with the installation other than the published > installation docs, which did not work for us? mailman-users(a)mailman3.org is the Mailman 3 version of this list. If your friend is the type to get his hands dirty with code, and has the time for development activity, mailman-developers(a)python.org would also be useful. You might be better off getting bids from consultants. There's a page listing them on wiki.list.org. > I have looked for both free and commercial alternatives to MM, There are sites that support MM commercially. I think the prices are reasonable, of course YMMV on that. Again, there's a page for those services on wiki.list.org. Footnotes: [1] "hage dakara hige o" -- the vowels are pronounced as Spanish or Italian, the consonants as in English.

1 0

Create new list styles via Postorius
by Torge Riedel Nov. 23, 2019

Nov. 23, 2019

Dear all, I have found this documentation on list styles: https://mailman.readthedocs.io/en/latest/src/mailman/styles/docs/styles.html Is there any way to create and configure list styles via Postorius web interface? Thanks in advance Torge

2 1

Advice
by jk＠ascii.com Nov. 20, 2019

Nov. 20, 2019

We are seeking a consultant to help migrate us from Mailman 2.1.9 to 3 > Mailing list (Core + Postorius) + web-forum (Hyperkitty). Any recommendations is appreciated - Jerry

3 2

Adding / manipulating users
by Seth Seeger Nov. 19, 2019

Nov. 19, 2019

Hello, Thank you for your help with the moderation status. My next question - I have two lists from Yahoo to import. I’ve migrated them through groups.io and have a convenient export. I now have all the data in formats I can handle. (json file for members, mbox for archives.) What’s the best way to import hundreds of users, setting their mail delivery status and moderation status at the same time? (There doesn’t seem to be a command line option for adding members anymore where you can also set their delivery status.) I’ve looked at the documentation, but couldn’t figure out what to do. Thanks, Seth

2 3