Hello,

tried to upgrade ubuntu 18.04 to 20.04 but the packages mailman3 (full,web) bringing a lot of problems.

If someone have the same problems like me, better don't try to upgrade cause it brings a lot of trouble, maybe it is better to use a new installation of ubuntu 20.04 and install the mailman3 packages after with apt.

Am 04.09.20 um 15:32 schrieb Stephen J. Turnbull:

Perhaps someone else can provide more concrete help, but you're running a version of HyperKitty that is at least 2 years old (we stopped supporting Python 2 around second quarter 2018). I strongly recommend upgrading. I understand that this may be extremely inconvenient since you're running an older Ubuntu LTS, but not only have a lot of features been added since then, a lot of bugs have been fixed.

r.woithe@callassoftware.com writes:

...

every night at 00:00 i get an email by mailman3:

Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/django_extensions/management/commands/runjobs.py", line 34, in runjobs job().execute() File "/usr/lib/python2.7/dist-packages/hyperkitty/jobs/sync_mailman.py", line 37, in execute sync_with_mailman()

I don't recognize this issue, but this looks dangerous. I believe that this process keeps HyperKitty's database synced with Mailman's, and if this process is consistently failing, they could become unsynced and HyperKitty's archive could become corrupt or report incorrect information.

-- Roberto Woithe | IT-Systemadministrator callas software GmbH | Schoenhauser Allee 6/7 | 10119 Berlin | Germany Tel +49.30.4439031-0 | Fax +49.30.4416402 | www.callassoftware.com Amtsgericht Charlottenburg, HRB 59615 Geschäftsführung: Olaf Drümmer, Ulrich Frotscher, Dietrich von Seggern

UPDATE

The command u can see above realy did bring mailman3 to wok again.

My question now is, should i set COMPRESS_OFFLINE to false again?

How u can see above: the file client.py is referenced about /usr/lib/python3.8/http but all other files descripted in the email are referenced about /usr/lib/python3/dist-packages/... But in /usr/lib/python3/ the directory and the file client.py is missing. Can this be the problem?

On 9/10/20 12:12 AM, r.woithe@callassoftware.com wrote:

UPDATE mailman3 woks now under python3 but i still get the nightly email, now with an other file but with the same result:

File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse raise ResponseNotReady(self.__state) http.client.ResponseNotReady: Request-started ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty)

anybody can help ?

See my reply at <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/HO6OFEXVQELVVI5JFSM6IRQ62KP7C37P/>

additionally should i set COMPRESS_OFFLINE to false again?

No.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Hello, Version: Mailman Core Version GNU Mailman 3.2.2 (La Villa Strangiato) Mailman Core API Version 3.0 Mailman Core Python Version 3.8.2 (default, Jul 16 2020, 14:00:26) [GCC 9.3.0]

But the Error was there allready before the update have been done. That means the Mailman-Core changed but the error persisted. mfG

Hello, this is the complete email we get every night:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django_extensions/management/commands/runjobs.py", line 36, in runjobs job().execute() File "/usr/lib/python3/dist-packages/hyperkitty/jobs/sync_mailman.py", line 36, in execute sync_with_mailman() File "/usr/lib/python3/dist-packages/hyperkitty/lib/mailman.py", line 146, in sync_with_mailman sender.set_mailman_id() File "/usr/lib/python3/dist-packages/hyperkitty/models/sender.py", line 55, in set_mailman_id mm_user = client.get_user(self.address) File "/usr/lib/python3/dist-packages/mailmanclient/client.py", line 320, in get_user response, content = self._connection.call( File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 95, in call response, content = Http().request(url, method, data_str, headers) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1948, in request (response, content) = self._request( File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1621, in _request (response, content) = self._conn_request( File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1560, in _conn_request response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1322, in getresponse raise ResponseNotReady(self.__state) http.client.ResponseNotReady: Request-started ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty) START TRACEBACK: END TRACEBACK

Thank u for the hint ! The only one is that Ubuntu uses uWSGI and not gunicorn. I set the para socket-timeout = XXX. I hope thats the right parameter.

On 9/23/20 7:02 AM, Brian Carpenter wrote:

On 9/23/20 7:33 AM, r.woithe@callassoftware.com wrote:

...

Thank u for the hint ! The only one is that Ubuntu uses uWSGI and not gunicorn. I set the para socket-timeout = XXX. I hope thats the right parameter.

---

I think Mailman 3 core still has its own gunicorn process that it runs.

Brian is correct. The REST API does not use the web server and uWSGI is not involved. Since Mailman core 3.3.0, the REST API is served by Gunicorn processes that have nothing to do with the web server. The time limit for these processes needs to be set as described at <https://wiki.list.org/x/17892072>.

Prior to 3.3.0, the REST API used the standard library wsgiref module which doesn't have timeouts. Since you said you have Mailman core 3.2.2, this may not be a REST timeout at all.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 9/28/20 12:58 AM, r.woithe@callassoftware.com wrote:

Hello, i set the necessary config params like descriped at https://wiki.list.org/x/17892072 with timeout = 9000. But we still get the nigthly email.

If you are still using Mailman core 3.2.2, those settings have no effect because the REST server doesn't use gunicorn prior to 3.3.0.

See <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...

I guess the best way would to upgrade to mailman-core 3.3.3 to avoid this error. Unfortunatly i can not use the ubuntu package-manager... Or is there a second way to avoid this error?

I don't know if upgrading Mailman core would avoid the error or not. As I indicated, it's almost certainly not a REST timeout after all.

We need to understand what causes the error. As I've said, the sync_mailman goes through the message senders and tries to set the senders mailman_id.

If you apply the attached patch to hyperkitty/models/sender.py, it will log the error and ignore it. Then, we'll get a bit more info from the log message.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

I'm migrating to Debian 10 and moving our lists from mailman 2.1 to mailman 3:Mailman Core Version GNU Mailman 3.2.1 (La Villa Strangiato)Mailman Core API Version 3.0Mailman Core Python Version 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]

While the list configurations moved over, after I started importing the archives yesterday we got a similar error, and now see the same error on our daily cron. It comes down to a mailman_sync: /usr/share/mailman3-web/manage.py mailman_sync --verbosity 3 -- tracebackTraceback (most recent call last): File "/usr/share/mailman3- web/manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 364, in execute_from_command_line utility.execute() File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 356, in execute self.fetch_command(subcommand).run_from_argv(self.argv) Fil e "/usr/lib/python3/dist-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/usr/lib/python3/dist- packages/hyperkitty/management/commands/mailman_sync.py", line 44, in handle sync_with_mailman(overwrite=options.get("overwrite", False)) File "/usr/lib/python3/dist- packages/hyperkitty/lib/mailman.py", line 145, in sync_with_mailman sender.set_mailman_id() File "/usr/lib/python3/dist-packages/hyperkitty/models/sender.py", line 54, in set_mailman_id mm_user = client.get_user(self.address) File "/usr/lib/python3/dist-packages/mailmanclient/client.py", line 321, in get_user 'users/{0}'.format(address)) File "/usr/lib/python3/dist- packages/mailmanclient/restbase/connection.py", line 95, in call response, content = Http().request(url, method, data_str, headers) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1513, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1263, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1216, in _conn_request response = conn.getresponse() File "/usr/lib/python3.7/http/client.py", line 1326, in getresponse raise ResponseNotReady(self.__state)http.client.ResponseNotReady: Request- started

I added a bit of debugging and it comes down to this. The email address being looked up is: Firstname Surname@Company-UK ^ which is obviously invalid (note the space). The mailing list archive imported that started this off does NOT have that email address anywhere within the headers of any email in the .mbox archive. However, the body content of a couple of emails contains quotes of this format: ....blah blah blahSigned -----Original Message-----From: Another User [mailto:user@example.com]S ent: 09 June 2020 12:50To: Firstname Surname at Company-UKCc: original.list@example.comSubject: topic blah blah blah

So something in the import has decided that the original message quote is a header and the To: field has been modified to protect the sender so has created the email <Firstname Surname@Company-UK> (note the space). Not only that, it has decided this is a valid email address. So a bug, filed as https://gitlab.com/mailman/mailman/-/issues/777 Unfortunately I've been moving the lists over a clutch at a time, and they are active, so I cannot just drop everything and start over. I can munge the future .mbox files being imported to remove the quoted email to stop this happening again, but ideally I also need to find and fix the errors in the database. I'm very new to mm3, but handy with SQL (I'm running on mariadb) so suggestions appreciated. Thanks-- Alex

On Fri, 2020-10-02 at 12:00 -0700, Mark Sapiro wrote:

[...]There's much more detail in my reply at< https://gitlab.com/mailman/hyperkitty/-/issues/320#note_422894486>;, butyou can fix the ResponseNotReady() issue from the sync_mailman job byfinding the entries in the hyperkitty_sender table with invalidaddresses in the address column and deleting them.

As per comment https://gitlab.com/mailman/hyperkitty/-/issues/320#note_422958260 , it was not a simple delete, but an insert/update/delete

We can try to address this specific issue in hyperkitty_import, butthere are in general various mbox defects that we can't detect. mboxesneed to be checked for unescaped From lines in message bodies andother defects before importing them. The Mailman 2.1 bin/cleanarchscript and the new hyperkitty/contrib/check_hk_import script can helpwith some of this.

I ran both on the original archive, cleanarch made no changes and check_hk_import just retported the number of messages.

As it turns out, the message was imported, but with the dodgy email as sender which was fixed by the insert/update as per my comment.

Hope you have some success with the redacted mbox example.

Cheers -- Alex

On Fri, 2020-10-02 at 20:28 -0700, Mark Sapiro wrote:

[...]Part of what's going on is some MM 2.1 archives has email addressesobfuscated by replacing @ with at so the import process reversesthis and changes the From: toheader isFrom: Firstname Surname@example-UK <user@example.com>which now contains two email addresses albeit not separated by a commaand picks the first. The message then gets properly archived withsender Firstname Surname@example-UK which is a syntactically validemail address and doesn't cause the ResponseNotReady() exception insync_mailman so I don't see that issue.

I saw the message archived also, but in the sync_mailman I did pick up something.

I'm a bit old school so my first goto is normally a strace. From that I saw the sync connect to the rest qrunner, querying each email address for their mailman id. However, when it came to querying the dodgy email, it did not get that far. The connection was made and closed. Initially I thought the rest closed its connection before the sync could even get it's query out (i.e. GET /3.0/users/Firstname Surname@example-UK) but an strace on the rest and the sync shows the connection made and closed immediately afterwards at both ends. Twice in fact (IPv4 and IPv6). Previous calls were as you would expected with the GET and HTTP/1.0 response.

The traceback ResponseNotReady() exception within the client.get_user(self.address) call of Sender.set_mailman_id in hyperkitty/models/sender.py only makes sense if sync was attempting to talk on a closed socket.

So I didn't get it - I still dont. Why did the sync not get as for as the rest "GET /3.0/users/Firstname Surname@example-UK" and "HTTP/1.0 404 Not Found" exchange, as it had with the previous (valid) email? Why was the rest socket closed without a query made? My only thought last night was that the rest connection was established by the sync, when then decided to close the connection because the email was invalid, but it continued attempting to get the mailman_id hence the exception.

Anyway, you then stepped up with your suggestion, I added the correct email to hyperkitty_sender, updated the faulty email record in hyperkitty_email to point to the correct hyperkitty_sender record and deleted the dodgy hyperkitty_sender record, and the sync passed. All good for me anyway.

The difference is either the fact that my test runs with the HEAD of theGitLab branch which may not have the issue or the difference between thereal From: header and the sanitized one is significant.

If the sync works for you in the trunk now, there seems little point figuring out why this does not work in the current debian package. Hopefully there will be an update by debain soon, and for now I can carry on my import and fix the issue if it happens again.

However, it would be interesting to know what ends up in your archive? I suspect it will have the dodgy "Firstname Surname@example- UK" address as the sender. If that is the case then certainly as you mention in https://gitlab.com/mailman/hyperkitty/-/issues/320#note_423030205, HyperKitty's wrapper should be smarter and call email.utils.parseaddr() first and only try replacing at if the returned address doesn't contain @. Otherwise you end up with a bad import and dodgy email in your archives.

Thanks -- Alex

Hallo, cause of we import our lists from mailman 2.1 to mailman3 as Alex Schuilenburg did, i did take a look in hyperkitty_sender and find an address like it is discribed above ( "firstname surename"@... ); i did insert,update and delete this address. Now the sync is working and all is fine.

Well, excuse my ignorance; cause of i have import the patch and set the logging level to debug but i could not find any more detailed error messages into the log files. Therefor im glad about to figured out the problem and for the solution. Thank you all

On Wed, 2020-10-07 at 06:03 -0700, Mark Sapiro wrote:

I don't know why the patch didn't work. It should have produced warning

level messages in the Django log as configured in settings(_local).py.

The patch did not give the warnings in the exception, because it was the wrong exception. The exception was not being caught.

I just modified the warning to log email addresses before looking up their id (where the exception occurred), moving it up 3 lines or so, on the basis the last email shown would be the faulty one. And it was

-- Alex