Mailman3 stopped working after Upgrade to Ubuntu 21.04
Hey,
I have set up mailman3 with mailman3-web on my mailserver and after upgrading to Ubuntu 21.04 my setup stopped working. I can see repeating log entries of mailman trying to reach something within hyperkitty and getting an error. The same constant repeating cycle of log entries is visible in my log files.
I have attached my config files as well as the logfiles for mailman3 and mailman-web.
As you can see, I'm not really getting many errors, yet within 5 seconds there are almost 2000 lines of log entries where mailman tries to reach something within hyperkitty over and over and over again. One thing I did to create an error message is start up both services, then edit a test list, that I created, in the web interface. This is the error you can find in the logs then:
ERROR 2021-08-07 12:50:26,932 2343625 postorius Un-handled exception: HTTP Error 404: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/generic.py", line 74, in dispatch return super(MailingListView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 97, in dispatch return handler(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 301, in get primary_email = set_preferred(request.user, mm_user) File "/usr/lib/python3/dist-packages/postorius/utils.py", line 77, in set_preferred mm_user.preferred_address = primary_email.email File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/user.py", line 57, in __setattr__ super(User, self).__setattr__(name, value) File "/usr/lib/python3/dist-packages/mailmanclient/restbase/base.py", line 131, in __setattr__ return super(RESTObject, self).__setattr__(name, value) File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/user.py", line 145, in preferred_address response, content = self._connection.call(url, {'email': email}) File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 127, in call raise HTTPError(url, response.status_code, urllib.error.HTTPError: HTTP Error 404: WARNING 2021-08-07 12:50:26,938 2343625 django.request Not Found: /mailman3/postorius/lists/test.lists.example.com/
Can anyone here help me debug this? I'm working on this problem for days now and can't seem to find a solution.
Thanks so much in advance, Thomas
On 8/7/21 4:35 AM, Thomas Möhle via Mailman-users wrote:
Hey,
I have set up mailman3 with mailman3-web on my mailserver and after upgrading to Ubuntu 21.04 my setup stopped working.
In what way? Do list posts get delivered? What works and what doesn't? What are the observable symptoms for things that don't work?
I can see repeating log entries of mailman trying to reach something within hyperkitty and getting an error. The same constant repeating cycle of log entries is visible in my log files.
It appears from your logs that Mailman core's attempts to reach the hyperkitty api may be failing. Mailman core uses the mailman_hyperkitty module to communicate with the hyperkitty api to get the url at which a message will be stored and to deliver a message to hyperkitty. This may not be working. There are probably unarchived messages in Mailman's var/archives/hyperkitty/spool/ directory.
What do you get for example if you go in a web browser with hyperkitty running to
And please don't post secret keys and change the value of api_key: in mailman-hyperkitty.cfg and the corresponding value in MAILMAN_ARCHIVER_KEY in your Django settings.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hey Mark,
thanks for your reply!> In what way? Do list posts get delivered? What works and what doesn't?
What are the observable symptoms for things that don't work? Yes, mails are not delivered. You can send a mail to the mailinglist without getting a bounce, but you will not receive the mail back through mailman. In addition to that, I'm unable to edit mailing lists in the web interface. After clicking on a list, I'll get a 404. Plus, I noticed that mailman is constantly writing files to /var/lib/mailman3/queue/out/ but they seem to get deleted right away again. I'm getting alot of IO there though.
What do you get for example if you go in a web browser with hyperkitty running to
I get a small gziped json file back. Key is "url", value is a link to a message.
And please don't post secret keys and change the value of api_key: in mailman-hyperkitty.cfg and the corresponding value in MAILMAN_ARCHIVER_KEY in your Django settings. I actually tried to redact the logs and config files, but I must have failed to save my edits on that one file. I'll change it right away.
On 8/8/21 4:41 AM, Thomas Möhle via Mailman-users wrote:
Hey Mark,
thanks for your reply!> In what way? Do list posts get delivered? What works and what doesn't?
What are the observable symptoms for things that don't work? Yes, mails are not delivered. You can send a mail to the mailinglist without getting a bounce, but you will not receive the mail back through mailman. In addition to that, I'm unable to edit mailing lists in the web interface. After clicking on a list, I'll get a 404. Plus, I noticed that mailman is constantly writing files to /var/lib/mailman3/queue/out/ but they seem to get deleted right away again. I'm getting alot of IO there though.
So messages are being queued for the outgoing runner and then something is done with them. If there are no errors in mailman.log about this and nothing in the shunt queue, they are being delivered to the MTA. What's in the MTAs logs?
What do you get for example if you go in a web browser with hyperkitty running to
I get a small gziped json file back. Key is "url", value is a link to a message.
OK. That's correct and it seems that Mailman core can communicate with the HyperKitty API, at least when HyperKitty is running.
And please don't post secret keys and change the value of api_key: in mailman-hyperkitty.cfg and the corresponding value in MAILMAN_ARCHIVER_KEY in your Django settings. I actually tried to redact the logs and config files, but I must have failed to save my edits on that one file. I'll change it right away.
The key was in the mailman-hyperkitty.cfg file you posted and was in multiple entries in mailman-web.log.
There is some strangeness in the logs you posted. Mailman.log contains many of these:
Aug 07 12:49:55 2021 (2343598) HyperKitty failure on https://mail.example.com/mailman3/hyperkitty/api/mailman/urls: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>503 Service Unavailable</title> </head><body> <h1>Service Unavailable</h1> <p>The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.</p> </body></html> (503)
and mailman-web log contains many of these
[pid: 2343625|app: 0|req: 1/1] 10.133.3.2 () {60 vars in 1350 bytes} [Sat Aug 7 12:49:58 2021] GET /mailman3/hyperkitty/api/mailman/urls?mlist=aussen%40lists.example2.com&msgid=002601d777e2%244c34c540%24e49e4fc0%24%40example2.com&key=g8nOyDVbCNsi18L%2FjgTj%2BWDVKUQ2XeQM => generated 131 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 160 bytes (1 switches on core 0)
which actually seem normal except for the quantity, but the time stamps don't correlate. There's nothing in mailman-web.log for the times of the 'HyperKitty failure' messages in mailman.log. Was the web service even up during those times.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro -
Thomas Möhle