Listing/Exporting member list error: Mailman REST API not available. Please start Mailman core.
Our MailMan3 server is extremely slow and when trying to list or export members for a larger list it is throwing 'Mailman REST API not available. Please start Mailman core'. In Mailman core logs I am seeing
[27] [CRITICAL] WORKER TIMEOUT (pid:40) [43] [INFO] Booting worker with pid: 43
Has anyone seen this issue? Is there any way to increase 'timeouts'?
Thanks for any help!
Tom Mailman Novice
16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/somelist@somelist.com/requests HTTP/1.1" 200 982 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/somelist@somelist.com/held?count=50&page=1 HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/list.somelist.com HTTP/1.1" 200 347 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/list.somelist.com/roster/owner HTTP/1.1" 200 634 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:01 +0000] "GET /3.1/lists/list.somelist.com/roster/moderator HTTP/1.1" 200 638 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:01 +0000] "GET /3.1/lists/somelist@somelist.com/config HTTP/1.1" 200 2257 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:05 +0000] "GET /3.1/lists/list.somelist.com/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:08 +0000] "GET /3.1/lists/list.somelist.com/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:08 +0000] "GET /3.1/lists/somelist@somelist.com/requests HTTP/1.1" 200 982 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:10 +0000] "GET /3.1/lists/list.somelist.com/member/somelist@somelist HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:16 +0000] "GET /3.1/lists/somelist@somelist.com/requests HTTP/1.1" 200 982 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:19 +0000] "GET /3.1/lists/somelist@somelist.com/held?count=50&page=1 HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:26 +0000] "GET /3.1/lists/list.somelist.com HTTP/1.1" 200 347 "-" "GNU Mailman REST client v3.3.0" [2020-01-16 16:56:58 +0000] [27] [CRITICAL] WORKER TIMEOUT (pid:40) [2020-01-16 16:57:00 +0000] [43] [INFO] Booting worker with pid: 43
DISCLAIMER: The information contained in this message is privileged and confidential. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please reply to the message and delete/destroy any copy of this message. Information in this message that does not relate to the official business of the firm shall be understood as neither given nor endorsed by it.
On Thu, Jan 16, 2020, at 9:18 AM, Thomas Lehker wrote:
Our MailMan3 server is extremely slow and when trying to list or export members for a larger list it is throwing 'Mailman REST API not available. Please start Mailman core'. In Mailman core logs I am seeing
[27] [CRITICAL] WORKER TIMEOUT (pid:40) [43] [INFO] Booting worker with pid: 43
Has anyone seen this issue? Is there any way to increase 'timeouts'?
How did you install Mailman3 and which version are you on?
Mailman 3, since I think 3.3.0 uses gunicorn for REST API, which does have a setting to increase the timeout for the worker to get killed1, the default I think is 30seconds. You can add a new gunicorn.cfg next to your mailman.cfg and add the additional options.
You can also increase the total number of workers, but that wouldn't help speed up queries, which I think is what you are actually looking for.
The slower queries could be because of too many database calls, can you specifically tell which endpoint is taking over 30seconds for you? I see that you are using mailmanclient to call the API.
Thanks for any help!
Tom Mailman Novice
16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/somelist@somelist.com/requests HTTP/1.1" 200 982 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/somelist@somelist.com/held?count=50&page=1 HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/list.somelist.com HTTP/1.1" 200 347 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:55:58 +0000] "GET /3.1/lists/list.somelist.com/roster/owner HTTP/1.1" 200 634 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:01 +0000] "GET /3.1/lists/list.somelist.com/roster/moderator HTTP/1.1" 200 638 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:01 +0000] "GET /3.1/lists/somelist@somelist.com/config HTTP/1.1" 200 2257 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:05 +0000] "GET /3.1/lists/list.somelist.com/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:08 +0000] "GET /3.1/lists/list.somelist.com/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:08 +0000] "GET /3.1/lists/somelist@somelist.com/requests HTTP/1.1" 200 982 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:10 +0000] "GET /3.1/lists/list.somelist.com/member/somelist@somelist HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:16 +0000] "GET /3.1/lists/somelist@somelist.com/requests HTTP/1.1" 200 982 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:19 +0000] "GET /3.1/lists/somelist@somelist.com/held?count=50&page=1 HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.0" [16/Jan/2020:16:56:26 +0000] "GET /3.1/lists/list.somelist.com HTTP/1.1" 200 347 "-" "GNU Mailman REST client v3.3.0" [2020-01-16 16:56:58 +0000] [27] [CRITICAL] WORKER TIMEOUT (pid:40) [2020-01-16 16:57:00 +0000] [43] [INFO] Booting worker with pid: 43
DISCLAIMER: The information contained in this message is privileged and confidential. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please reply to the message and delete/destroy any copy of this message. Information in this message that does not relate to the official business of the firm shall be understood as neither given nor endorsed by it.
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
Hi Abhilash,
Thomas is one of my clients and I helped install the Mailman 3 setup he is using. I installed MM3 using your docker images. He is currently running version 3.1. Lists with a smaller amount of members are exporting fine. However on a list with around 1100 members, the request is timing out.
Brian
Can confirm the problem is reproducible on mailman 3.3.0 installed via PIP.
Postorius hangs on exporting CSV subscribers [1] (~6k in this case), with error "Mailman REST API not available. Please start Mailman core" (which is otherwise running fine).
mailman3.log also shows WORKER TIMEOUT log entry, exactly the same as OP's message.
You can add a new |gunicorn.cfg| next to your |mailman.cfg| and add the additional options. For further reference if anyone reads this thread, see [2].
I had to look that up to figure out where the file name "gunicorn.cfg" came from. It's not immediately obvious to the reader why a file named "guincorn.cfg" should be specially interpreted by mailman3.
The suggested fix - increasing timeout of worker process - did work perfectly for me. Thanks :)
Regards, Daniel
[1] http://www.<redacted>.org/postorius/lists/<redacted>/csv_view/ [2] https://mailman.readthedocs.io/en/latest/src/mailman/runners/docs/rest.html#...
On Mon, Feb 17, 2020, at 2:24 PM, Daniel Armengod wrote:
Can confirm the problem is reproducible on mailman 3.3.0 installed via PIP.
Postorius hangs on exporting CSV subscribers [1] (~6k in this case), with error "Mailman REST API not available. Please start Mailman core" (which is otherwise running fine).
mailman3.log also shows WORKER TIMEOUT log entry, exactly the same as OP's message.
You can add a new |gunicorn.cfg| next to your |mailman.cfg| and add the additional options. For further reference if anyone reads this thread, see [2].
I had to look that up to figure out where the file name "gunicorn.cfg" came from. It's not immediately obvious to the reader why a file named "guincorn.cfg" should be specially interpreted by mailman3.
There are several 3rd party libraries/CLIs that collectively makes Mailman Core work. Each of which, have their own configs and Mailman does not wrap each of their configs in it's own, a design decision made by the initial authors which I agree with.
Wrapping 3rd party tool's configs in Mailman's configs only makes the job of maintaining much harder.
The suggested fix - increasing timeout of worker process - did work perfectly for me. Thanks :)
Regards, Daniel
[1] http://www.<redacted>.org/postorius/lists/<redacted>/csv_view/ [2] https://mailman.readthedocs.io/en/latest/src/mailman/runners/docs/rest.html#...
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
participants (4)
-
Abhilash Raj -
Brian Carpenter -
Daniel Armengod -
Thomas Lehker