- Mailman-users - lists.mailman3.org

# comment allowed in a template file?
by skenny＠scss.tcd.ie July 28, 2020

July 28, 2020

I've found that dropping files in to the templates directory is a simple and convenient way to make certain things happen. E.g. I can set a default footer for all posts by putting the desired content in a file called var/templates/site/en/list:member:generic:footer.txt. I'm running Mailman 3 in maxking's Docker container. I'm also using Puppet to do as much configuration as possible, so I'd like to add a comment to the file that reads something like: # This file is managed by Puppet. Is it ok to do this? I may not have searched the docs enough, but I couldn't find anything relating to comments. One other observation in case it's any use: Initially, I created the template files with owner set to "root". This resulted in postings to lists getting stuck in mailman-core. It's vital that these files are owned by user "mailman". Once I changed the owner of the file to the latter, everything worked. Stephen

3 4

MM3 not starting afte an upgrade
by Gelpi Andrea July 28, 2020

July 28, 2020

Hi, to day in a Debian system I upgraded some packages. After the MM3 doesn't start anymore. These are the packages I upgraded: [UPGRADE] clamav:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamav-base:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamav-daemon:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamav-freshclam:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] clamdscan:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] gcc-10-base:amd64 10.1.0-4 -> 10.1.0-6 [UPGRADE] libc-l10n:amd64 2.30-8 -> 2.31-1 [UPGRADE] libclamav9:amd64 0.102.3+dfsg-1 -> 0.102.4+dfsg-1 [UPGRADE] libgcc-s1:amd64 10.1.0-4 -> 10.1.0-6 [UPGRADE] libnss3:amd64 2:3.42.1-1+deb10u2 -> 2:3.42.1-1+deb10u3 [UPGRADE] python3-django-hyperkitty:amd64 1.3.2-1 -> 1.3.2-1.1 [UPGRADE] python3-django-postorius:amd64 1.3.2-2 -> 1.3.2-2.1 [UPGRADE] qemu-guest-agent:amd64 1:3.1+dfsg-8+deb10u5 -> 1:3.1+dfsg-8+deb10u6 MM3 on start up says: Jul 23 13:29:39 posta mailman3[2608]: Traceback (most recent call last): Jul 23 13:29:39 posta mailman3[2608]: File "/usr/bin/mailman", line 11, in <module> Jul 23 13:29:39 posta mailman3[2608]: load_entry_point('mailman==3.2.1', 'console_scripts', 'mailman')() Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__ Jul 23 13:29:39 posta mailman3[2608]: return self.main(*args, **kwargs) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main Jul 23 13:29:39 posta mailman3[2608]: rv = self.invoke(ctx) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 69, in invoke Jul 23 13:29:39 posta mailman3[2608]: return super().invoke(ctx) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke Jul 23 13:29:39 posta mailman3[2608]: return _process_result(sub_ctx.command.invoke(sub_ctx)) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke Jul 23 13:29:39 posta mailman3[2608]: return ctx.invoke(self.callback, **ctx.params) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke Jul 23 13:29:39 posta mailman3[2608]: return callback(*args, **kwargs) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func Jul 23 13:29:39 posta mailman3[2608]: return f(get_current_context(), *args, **kwargs) Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/mailman/commands/cli_control.py", line 81, in start Jul 23 13:29:39 posta mailman3[2608]: status, lock = master_state() Jul 23 13:29:39 posta mailman3[2608]: File "/usr/lib/python3/dist-packages/mailman/bin/master.py", line 104, in master_state Jul 23 13:29:39 posta mailman3[2608]: os.kill(pid, 0) Jul 23 13:29:39 posta mailman3[2608]: PermissionError: [Errno 1] Operation not permitted There is a permission error, but on what? Thanks for any help. Regards -- Gelpi ing. Andrea -------------------------------------------------------------- It took the computing power of three C-64s to fly to the Moon. It takes a 486 to run Windows 95. Something is wrong here. --------------------------------------------------------------

3 7

sqlalchemy.orm.exc.MultipleResultsFound: Multiple rows were found for one_or_none()
by Shashikanth Komandoor July 28, 2020

July 28, 2020

Hi Team, Today, I found an error message like "Multiple rows were found for one_or_none()" when a hold message is approved for a list. Until yesterday the mails were accepted and delivered fine. But today the above said error message is thrown. Apart from that, I saw the mail had gone into the shunt queue. The "mailman qfile" output of the shunt message and the respective traceback log from mailman.log are attached. You may please go through them and help me out. Please also let me know if I have to share any more details. -- Thanks & Regards, Shashi Kanth.K 9052671936

2 5

Running Mailman3 under supervisord
by Odhiambo Washington July 27, 2020

July 27, 2020

Hello world, Is it possible to run MM3 under supervisord? What would I use for command= ?? -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

3 4

mailman3 nightly script(s) are leaking memory + mitigation
by Mark Dadgar July 24, 2020

July 24, 2020

So as I mentioned in another thread, I have a shiny new install of mailman3 latest using venv (thanks, Brian!). It’s running on ubuntu 20.04 using python 3.8.2, django 2.2.12, nginx, postgres, and memcached. The only functions this server runs are 1) postfix and dovecot for a small number of personal email accounts, and 2) mailman3 for a couple of mailing lists (one of which is fairly high traffic). Not long after install, I saw a couple of instances of the OOM killer activating in the logs. I set up a 1GB swapfile (discouraged on Digital Ocean instances because they don’t like their flash drives to take that kind of pounding) as a safety net. Then I saw the free swap space eroding over time, so I started collecting some data. Initial evening stats: top - 23:29:16 up 15 days, 9:20, 1 user, load average: 0.07, 0.16, 0.17 Tasks: 200 total, 1 running, 199 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 1.7 sy, 0.0 ni, 96.3 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 3936.2 total, 264.9 free, 1651.4 used, 2019.9 buff/cache MiB Swap: 1000.0 total, 811.2 free, 188.8 used. 1854.5 avail Mem Stats the next morning: top - 08:42:15 up 13 days, 18:33, 1 user, load average: 0.29, 0.27, 0.19 Tasks: 204 total, 1 running, 203 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.5 us, 1.4 sy, 0.0 ni, 96.0 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 3936.2 total, 214.7 free, 2244.8 used, 1476.7 buff/cache MiB Swap: 1000.0 total, 500.3 free, 499.7 used. 1318.8 avail Mem Stats the next evening: top - 23:26:00 up 14 days, 9:17, 1 user, load average: 0.10, 0.06, 0.08 Tasks: 209 total, 1 running, 208 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.6 us, 1.7 sy, 0.0 ni, 95.2 id, 0.3 wa, 0.0 hi, 0.3 si, 0.1 st MiB Mem : 3936.2 total, 206.0 free, 2253.6 used, 1476.6 buff/cache MiB Swap: 1000.0 total, 485.9 free, 514.1 used. 1305.5 avail Mem Stats the following morning: top - 08:09:32 up 14 days, 18:01, 1 user, load average: 0.01, 0.07, 0.14 Tasks: 194 total, 1 running, 193 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 1.4 sy, 0.0 ni, 96.6 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st MiB Mem : 3936.2 total, 270.6 free, 1974.5 used, 1691.2 buff/cache MiB Swap: 1000.0 total, 266.5 free, 733.5 used. 1545.1 avail Mem As you can see, something during the night is eating memory and not letting it go. If I restart gunicorn, I get the memory back: sudo systemctl restart gunicorn top - 08:52:29 up 14 days, 18:44, 1 user, load average: 0.71, 0.49, 0.25 Tasks: 243 total, 1 running, 242 sleeping, 0 stopped, 0 zombie %Cpu(s): 6.1 us, 3.0 sy, 0.0 ni, 90.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 3936.2 total, 936.3 free, 1277.1 used, 1722.9 buff/cache MiB Swap: 1000.0 total, 804.8 free, 195.2 used. 2242.5 avail Mem I did a little googling and a mitigation that seems to mostly work is to have gunicorn retire its workers periodically to free up memory. gunicorn.cf: max_requests = 1000 max_requests_jitter = 50 That restarts a worker after 1000 requests and adds some jitter so they don’t all get restarted at the same time. I am still seeing the erosion of a couple of MB of swap each day, but it’s WAY better than losing 250MB-350MB overnight. FYI for others who may see this. - Mark ----- mark(a)pdc-racing.net | 408-348-2878

2 1

Re: Mailman REST API not available. Please start Mailman Core.
by Shashikanth Komandoor July 23, 2020

July 23, 2020

Thank you Mark for your response. I understood that gunicorn is a front end that is used like web server like nginx. But I don't think I am using it instead I am using the httpd to redirect requests to postorius. And I am starting the mailman-suite with the command "python manage.py runserver 0.0.0.0:8000 --insecure &". Hope this does not use gunicorn (which of course I don't know what it is until now). Please correct me if any point I am mentioning is wrong. So, in this case, do you think configuring gunicorn as below would work: *calling gunicorn.cfg in mailman.cfg as below * *[webservice]* *configuration: /var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg* *and content of gunicorn.cfg as below* *(venv3) [root@lsmgr mailman]# cat /var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg[gunicorn]# All the options that can be put here are available at# http://docs.gunicorn.org/en/stable/settings.html#settings <http://docs.gunicorn.org/en/stable/settings.html#settings># No of seconds to wait before killing the worker processes.* *workers = 4graceful_timeout = 90* Also what is the difference between timeout and graceful_timeout ? Please help me if I can increase the management better. Any guide is also helpful for me. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Thu, Jul 23, 2020 at 4:10 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 7/22/20 9:21 AM, Shashikanth Komandoor wrote: > > Hi Mark, > > > > Never mind please I am sending to you only instead of group as I am > > sending you the production content along with the mail. > > > See my other reply. > > > > I have configured the workers count as 4 and timeout as 900 in > > mailman.cfg under [webservice]. This is FYI. > > You can't configure Mailman's REST gunicorn workers directly in the > [webservice] section of mailman.cfg. The only thing you can configure > there is: > > configuration: /path/to/gunicorn.cfg > > That file, which is separate from your web server gunicorn.cfg, is where > you put things like > > [gunicorn] > timeout = 900 > > It also will not work to put a gunicorn section in mailman.cfg. It must > be in a separate file pointed to by webservice.configuration. > > Increasing the number of workers won't help this issue. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > -- Thanks & Regards, Shashi Kanth.K 9052671936

3 2

Re: [ERROR] Error handling request List archivers
by Shashikanth Komandoor July 23, 2020

July 23, 2020

Thank you Mark for guiding me to the benefit of putting the list and so I have added the list. Along with the [archiver.hyperkitty], [mta] and [database], few lines of my mailman.cfg look like below: *[webservice]workers = 4timeout = 900* The above block of webservice is added yesterday for testing or trail as I am getting the above issue. And there is no line that is calling *gunicorn* in my mailman.cfg. Apart from that I tried to locate and see the content of the gunicorn.cfg file as below : *(venv3) [root@lsmgr mailman]# cat /var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg[gunicorn]# All the options that can be put here are available at# http://docs.gunicorn.org/en/stable/settings.html#settings <http://docs.gunicorn.org/en/stable/settings.html#settings># No of seconds to wait before killing the worker processes.graceful_timeout = 30* Please confirm the time I have to change is *graceful_timeout *in *gunicorn.cfg* and if I have changed how do I restart the *gunicorn*. I have never used or never known about this *gunicorn* but still running the service in production. Please help me and also suggest if I could provide you any more data from giving resolution. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#m_5710631741815235215_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> On Thu, Jul 23, 2020 at 3:47 AM Mark Sapiro <mark(a)msapiro.net> wrote: > On 7/22/20 9:13 AM, Shashikanth Komandoor wrote: > > > Never mind please I am sending to you only instead of group as I am > sending you the production content along with the mail. > > I don't see anything there that can't be posted to the > mailman-users(a)mailman3.org list. If your concern is revealing the > support-tnvc.lsmgr.nic.in list id, just replace it with <list-id> or > list.example.com or similar. > > Messages sent to me personally rather than the list suffer many > problems. They are my absolute last priority, and I am only one person, > the collective knowledge of the list is greater. Also, at times I am off > line for several days or longer. > > Finally, there is no opportunity for other members of the community to > benefit from an off-list exchange. > > > When I am trying to browse one of my lists, I am getting "[ERROR] > > Error handling request /3.1/lists/support-tnvc.lsmgr.nic.in/archivers > > <http://support-tnvc.lsmgr.nic.in/archivers>" in the mailman.log and the > > full error message is in attachment. > > Try increasing your gunicorn worker timeout from the default 30 seconds. > I.e., add > > timeout = 90 > > to your gunicorn.cfg and restart gunicorn. > > -- > Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > -- Thanks & Regards, Shashi Kanth.K 9052671936 <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 1

migration plan and auto responder
by Mohsen Masoudfar July 23, 2020

July 23, 2020

Hi, I am writing this in the hope to get some good advises for migration lists from Mailman2 to Mailman3. Thanks a lot in advance. Here is the requirements: * During the migration the old system shouldn't process the emails for the list under migration in the normal way. It should send an email, and only one to the member who sent the original email to the list saying the list is under migration. I know I could do this by auto responder, but the original email would be processed and sent to all members, which I want to block . * After migration the old system should send, per auto responder again, an email to the member who sent to the list, mentioning the new address (the domain is a new one, the list name the same), but again not sending the original email to the list members. * No hackers, robots, spammers should get the new list address. The lists are all hidden. Best regards Mohsen Masoudfar Lead System Analyst, IT Operation American Association for the Advancement of Science 1200 New York Ave, NW, Washington, DC 20005 202-326-7087

2 4

Python 3.5 support?
by Joel Lord July 22, 2020

July 22, 2020

I'm looking through the recent changes to Core and Postorius and I keep spotting what looks like removal of the test cases for Python 3.5. Is the next release going to be dropping support for 3.5? Is this because 3.5 is going EOL shortly or because there are changes in the upcoming release that actively will not work on 3.5? -- Joel Lord

2 1

Docker Support Sucks
by Brian Carpenter July 22, 2020

July 22, 2020

Ok, now that I got that out of the way. Maybe I can get some help here for a Docker Installation of Mailman 3. I updated a Maxking's Docker installation to Postorius 1.3.3. It made my client's list unusable to a continual stream of removals due to ghost bounce processes. So, I want to totally move away from the use of Docker (mainly because I have found it buggy and the support is simply lacking) and install Mailman 3 for this client from source. However I need to get an export of their list members. But of course, I can't. Rest API times out and I don't know how to apply the instructions I was given to fix it and I don't have days to way for one. So please help. I increased the gunicorn workers to 4, hoping that would allow me to do the CSV export. No. So was given these instructions: ### The issues seems to be due to timeout not due to high load. You want to increase the timeout for gunicorn and you can do that by specifying a gunicorn.ini config file: [webservice] configuration: /path/to/gunicorn.cfg and then in the gunicorn.cfg you can add [gunicorn] timeout = 90 graceful_timeout = 30 ### I applied the above changes to /opt/mailman/core/mailman-extra.cfg and created the gunicorn.cfg file. Restarted the docker container (mailman-core) and no access to Postorius due to no REST api. After restarting docker-mailman, I can't access Postorius due to the following error: ERROR 2020-07-20 20:16:27,351 15 postorius Mailman REST API not available Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/usr/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection raise err File "/usr/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen chunked=chunked, File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/lib/python3.6/http/client.py", line 1254, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.6/http/client.py", line 1300, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.6/http/client.py", line 1249, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.6/http/client.py", line 1036, in _send_output self.send(msg) File "/usr/lib/python3.6/http/client.py", line 974, in send self.connect() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect conn = self._new_conn() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f96340a2e10>: Failed to establish a new con$ During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen Commenting out "configuration: /opt/mailman/gunicorn.cfg" brings Postorius back up. I am really frustrated here. All I want to do is export a 3000 member list and I can't. Updating Mailman 3 using new Docker images shouldn't break a stable installation either. Please help. Brian

3 4