"Something went wrong | Mailman REST API not available. Please restart Mailman core.
Greetings,
Some of my lists have started displaying this behavior — when you try to access the list page(s), you get a page with this message:
Something went wrong Mailman REST API not available. Please restart Mailman core.
My mailman-core docker container is running, as are mailman-django-uwsgi, mailman-nginx and mailman-postfix. Restarting the containers doesn't change this behavior.
Google suggests restarting mailman core. Or that we might not be using the correct account name, password or port.
The weird thing is that it is not all lists. Most of them seem to be fine. Which suggests to me that the issue is with a few lists, but not systemwide, so I think my settings in mailman.cfg, settings.py, settings_local.py are likely correct.
For the problem lists, I see this in the log:
ERROR 2025-10-29 18:38:56,784 7 postorius Mailman REST API not available ERROR 2025-10-29 18:38:56,790 7 django.request Service Unavailable: /postorius/lists/obfuscated.name.lists.fhcrc.org/ ERROR 2025-10-29 18:38:56,790 7 django.request Service Unavailable: /postorius/lists/obfuscated.name.lists.fhcrc.org/
So, Postorius can't communicate with the API, I guess?
I look at settings_local.py:
MAILMAN_REST_API_URL = 'http://mailman-core:8001'
I believe 8001 is the default, and since I haven't specified one in the [webservice] section of mailman.cfg, I'm assuming 8001 is correct (plus, it works for other lists).
There are a few messages waiting for moderator approval — otherwise, I'd probably try just deleting the list and recreating it.
Anyone have any suggestions on what else I should look at?
Thanks!
-p
Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering - Infrastructure Fred Hutch Cancer Center O 206.667.4856
phirayam@fredhutch.org<mailto:phirayam@fredhutch.org>
On 10/29/25 14:31, Hirayama, Pat wrote:
The weird thing is that it is not all lists. Most of them seem to be fine. Which suggests to me that the issue is with a few lists, but not systemwide, so I think my settings in mailman.cfg, settings.py, settings_local.py are likely correct.
So Postorius can connect to Mailman core's API on behalf of some, but not all lists. This is very strange.
For the problem lists, I see this in the log:
ERROR 2025-10-29 18:38:56,784 7 postorius Mailman REST API not available ERROR 2025-10-29 18:38:56,790 7 django.request Service Unavailable: /postorius/lists/obfuscated.name.lists.fhcrc.org/ ERROR 2025-10-29 18:38:56,790 7 django.request Service Unavailable: /postorius/lists/obfuscated.name.lists.fhcrc.org/
So, Postorius can't communicate with the API, I guess?
These are the normal messages when the REST API is not available.
But it apparently is available for at least some API endpoints.
What is the traceback from the
ERROR 2025-10-29 18:38:56,784 7 postorius Mailman REST API not available
error and what is in the web server logs?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Ah, thanks for the reply, Mark. Forgot to include Traceback:
Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request httplib_response = conn.getresponse() File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse response.begin() File "/usr/local/lib/python3.8/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/local/lib/python3.8/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen retries = retries.increment( File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 734, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request httplib_response = conn.getresponse() File "/usr/local/lib/python3.8/http/client.py", line 1347, in getresponse response.begin() File "/usr/local/lib/python3.8/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/local/lib/python3.8/http/client.py", line 276, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/mailmanclient/restbase/connection.py", line 91, in call response = request( File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.8/site-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/local/lib/python3.8/site-packages/postorius/views/generic.py", line 74, in dispatch return super(MailingListView, self).dispatch(request, *args, **kwargs) File "/usr/local/lib/python3.8/site-packages/django/views/generic/base.py", line 97, in dispatch return handler(request, *args, **kwargs) File "/usr/local/lib/python3.8/site-packages/postorius/views/list.py", line 288, in get return render(request, 'postorius/lists/summary.html', data) File "/usr/local/lib/python3.8/site-packages/django/shortcuts.py", line 19, in render content = loader.render_to_string(template_name, context, request, using=using) File "/usr/local/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string return template.render(context, request) File "/usr/local/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render return self.template.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 171, in render return self._render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 163, in _render return self.nodelist.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/loader_tags.py", line 150, in render return compiled_parent._render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 163, in _render return self.nodelist.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/loader_tags.py", line 62, in render result = block.nodelist.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/library.py", line 234, in render return t.render(new_context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 173, in render return self._render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 163, in _render return self.nodelist.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/defaulttags.py", line 309, in render return nodelist.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/defaulttags.py", line 309, in render return nodelist.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 986, in render output = self.filter_expression.resolve(context) File "/usr/local/lib/python3.8/site-packages/django/template/base.py", line 697, in resolve new_obj = func(obj, *arg_vals) File "/usr/local/lib/python3.8/site-packages/postorius/templatetags/nav_helpers.py", line 74, in held_count return mlist.get_held_page().total_size File "/usr/local/lib/python3.8/site-packages/mailmanclient/restobjects/mailinglist.py", line 132, in get_held_page return Page(self._connection, url, HeldMessage, count, page) File "/usr/local/lib/python3.8/site-packages/mailmanclient/restbase/page.py", line 37, in __init__ self._create_page() File "/usr/local/lib/python3.8/site-packages/mailmanclient/restbase/page.py", line 62, in _create_page response, content = self._connection.call(self._build_url()) File "/usr/local/lib/python3.8/site-packages/mailmanclient/restbase/connection.py", line 119, in call raise MailmanConnectionError( mailmanclient.restbase.connection.MailmanConnectionError: ('Could not connect to Mailman API: ', "ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))")
web logs:
10.9.253.147 - - [29/Oct/2025:18:54:41 +0000] "GET /hyperkitty/list/obfuscated.name@lists.fhcrc.org/ HTTP/1.1" 404 6305 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "-" 10.9.253.147 - - [29/Oct/2025:21:03:52 +0000] "GET /postorius/lists/obfuscated.name.lists.fhcrc.org/ HTTP/1.1" 503 6310 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "-" 10.9.253.147 - - [29/Oct/2025:21:06:37 +0000] "GET /postorius/lists/obfuscated.name.lists.fhcrc.org/ HTTP/1.1" 503 6310 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "-" 10.9.253.147 - - [29/Oct/2025:21:24:25 +0000] "GET /postorius/lists/obfuscated.name.lists.fhcrc.org/ HTTP/1.1" 503 6310 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "-" 10.9.253.147 - - [29/Oct/2025:22:11:56 +0000] "GET /postorius/lists/obfuscated.name.pattest.lists.fhcrc.org/ HTTP/1.1" 404 6305 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "-" 10.9.253.147 - - [29/Oct/2025:22:12:03 +0000] "GET /postorius/lists/obfuscated.name.lists.fhcrc.org/ HTTP/1.1" 503 6310 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36" "-"
Thanks,
-p
Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam@fredhutch.org
From: Mark Sapiro <mark@msapiro.net> Sent: Wednesday, October 29, 2025 15:12 To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: "Something went wrong | Mailman REST API not available. Please restart Mailman core.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On 10/29/25 14:31, Hirayama, Pat wrote:
The weird thing is that it is not all lists. Most of them seem to be fine. Which suggests to me that the issue is with a few lists, but not systemwide, so I think my settings in mailman.cfg, settings.py, settings_local.py are likely correct.
So Postorius can connect to Mailman core's API on behalf of some, but not all lists. This is very strange.
For the problem lists, I see this in the log:
ERROR 2025-10-29 18:38:56,784 7 postorius Mailman REST API not available ERROR 2025-10-29 18:38:56,790 7 django.request Service Unavailable: /postorius/lists/obfuscated.name.lists.fhcrc.org/ ERROR 2025-10-29 18:38:56,790 7 django.request Service Unavailable: /postorius/lists/obfuscated.name.lists.fhcrc.org/
So, Postorius can't communicate with the API, I guess?
These are the normal messages when the REST API is not available.
But it apparently is available for at least some API endpoints.
What is the traceback from the
ERROR 2025-10-29 18:38:56,784 7 postorius Mailman REST API not available
error and what is in the web server logs?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://urldefense.com/v3/__https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/__;!!GuAItXPztq0!g8e5xt3CEbVj22YPHZnsW7OQGYMBisPagx2Kxcm4_So0MZaHmihE-9NIUBiN7MzDnS6CMzew0NSDVX8$<https://urldefense.com/v3/__https:/lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/__;!!GuAItXPztq0!g8e5xt3CEbVj22YPHZnsW7OQGYMBisPagx2Kxcm4_So0MZaHmihE-9NIUBiN7MzDnS6CMzew0NSDVX8$> Archived at: https://urldefense.com/v3/__https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/LR77KHMNYFTPDHGLI4KH5XKUKCASIUMX/__;!!GuAItXPztq0!g8e5xt3CEbVj22YPHZnsW7OQGYMBisPagx2Kxcm4_So0MZaHmihE-9NIUBiN7MzDnS6CMzewhr_U-cM$<https://urldefense.com/v3/__https:/lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/LR77KHMNYFTPDHGLI4KH5XKUKCASIUMX/__;!!GuAItXPztq0!g8e5xt3CEbVj22YPHZnsW7OQGYMBisPagx2Kxcm4_So0MZaHmihE-9NIUBiN7MzDnS6CMzewhr_U-cM$>
This message sent to phirayam@fredhutch.org
On 10/29/25 15:40, Hirayama, Pat wrote:
Ah, thanks for the reply, Mark. Forgot to include Traceback:
Traceback (most recent call last): ...
Thanks. These are not the normal 'core isn't running' tracebacks which
are connection refused. These seem to say that core is dropping the
connection without a response. What's in mailman.log from these times?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thanks, Mark. Yeah, it's an odd one. These are from around 22:10-22:14 in mailman.log within the mailman-core container.
[29/Oct/2025:22:10:23 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:23 +0000] "POST /3.1/lists/find HTTP/1.1" 200 13020 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:25 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org HTTP/1.1" 200 433 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/roster/owner HTTP/1.1" 200 1175 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/roster/moderator HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100@lists.fhcrc.org/config HTTP/1.1" 200 3016 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100@lists.fhcrc.org/requests HTTP/1.1" 200 1307 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/member/aismael%40fredhutch.org HTTP/1.1" 200 509 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100@lists.fhcrc.org/requests HTTP/1.1" 200 1307 "-" "GNU Mailman REST client v3.3.1" [2025-10-29 22:10:26 +0000] [31] [ERROR] Socket error processing request. Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 134, in handle self.handle_request(listener, req, client, addr) File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 190, in handle_request util.reraise(*sys.exc_info()) File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 625, in reraise raise value File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 175, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 163, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in _make_collection entries = [self._resource_as_dict(resource) File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in <listcomp> entries = [self._resource_as_dict(resource) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 153, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 76, in _make_resource msg = getUtility(IMessageStore).get_message_by_id(key) File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 103, in get_message_by_id return self._get_message(row) File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 95, in _get_message with open(path, 'rb') as fp: FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/WD/BX/WDBXUFG34LR6QJUQXUWXRHZ4OO4GMQ52' [29/Oct/2025:22:11:49 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org HTTP/1.1" 200 373 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/roster/owner HTTP/1.1" 200 624 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/roster/moderator HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/config HTTP/1.1" 200 3262 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/member/cit-sys-engineering%40fredhutch.org HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/held?count=50&page=1 HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:11:56 +0000] "GET /3.1/lists/vtn.fiscal.pattest.lists.fhcrc.org HTTP/1.1" 404 58 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org HTTP/1.1" 200 389 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/roster/owner HTTP/1.1" 200 628 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/roster/moderator HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal@lists.fhcrc.org/config HTTP/1.1" 200 46422 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:03 +0000] "GET /3.1/lists/vtn.fiscal@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:03 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/member/cit-sys-engineering%40fredhutch.org HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:03 +0000] "GET /3.1/lists/vtn.fiscal@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1" [2025-10-29 22:12:03 +0000] [31] [ERROR] Socket error processing request. Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 134, in handle self.handle_request(listener, req, client, addr) File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 190, in handle_request util.reraise(*sys.exc_info()) File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 625, in reraise raise value File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 175, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 163, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in _make_collection entries = [self._resource_as_dict(resource) File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in <listcomp> entries = [self._resource_as_dict(resource) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 153, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 76, in _make_resource msg = getUtility(IMessageStore).get_message_by_id(key) File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 103, in get_message_by_id return self._get_message(row) File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 95, in _get_message with open(path, 'rb') as fp: FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/LZ/OC/LZOCY4BX6FBKMLJZQKVED6AVXOUBDFXQ' [29/Oct/2025:22:12:15 +0000] "GET /3.1/lists?advertised=true&count=0&page=1 HTTP/1.1" 200 92 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:15 +0000] "GET /3.1/lists?advertised=true&count=10&page=1 HTTP/1.1" 200 4353 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:15 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" Oct 29 22:13:20 2025 (10) ACCEPT: <DM6PR11MB4692672CB4AB8CA1AB739567D5FAA@DM6PR11MB4692.namprd11.prod.outlook.com> [29/Oct/2025:22:13:33 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:34 +0000] "POST /3.1/lists/find HTTP/1.1" 200 2569 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists?count=0&page=1 HTTP/1.1" 200 93 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists?count=10&page=1 HTTP/1.1" 200 4441 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/015.site.257.cergo015.site.257.cer@lists.fhcrc.org/config HTTP/1.1" 200 46497 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/1100_labs@lists.fhcrc.org/config HTTP/1.1" 200 3053 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/9228.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/9292.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/a2223dsmbdl@lists.fhcrc.org/config HTTP/1.1" 200 3145 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/a244.ct@lists.fhcrc.org/config HTTP/1.1" 200 46388 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/a5397.vtn603.pi.coord@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/ab.mop@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/accessdev@lists.fhcrc.org/config HTTP/1.1" 200 3173 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/actg.site.31422.coord@lists.fhcrc.org/config HTTP/1.1" 200 46429 "-" "GNU Mailman REST client v3.3.1" Oct 29 22:13:47 2025 (10) ACCEPT: <MN2PR11MB4333B8F87B77ED1E7F83422DDDFAA@MN2PR11MB4333.namprd11.prod.outlook.com> [29/Oct/2025:22:14:23 +0000] "GET /3.1/lists?count=0&page=1 HTTP/1.1" 200 93 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists?count=200&page=1 HTTP/1.1" 200 83390 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/015.site.257.cergo015.site.257.cer@lists.fhcrc.org/config HTTP/1.1" 200 46497 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/1100_labs@lists.fhcrc.org/config HTTP/1.1" 200 3053 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/9228.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/9292.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/a2223dsmbdl@lists.fhcrc.org/config HTTP/1.1" 200 3145 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/a244.ct@lists.fhcrc.org/config HTTP/1.1" 200 46388 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/a5397.vtn603.pi.coord@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/ab.mop@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/accessdev@lists.fhcrc.org/config HTTP/1.1" 200 3173 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.coord@lists.fhcrc.org/config HTTP/1.1" 200 46429 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.dataqc@lists.fhcrc.org/config HTTP/1.1" 200 46389 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.pharm@lists.fhcrc.org/config HTTP/1.1" 200 46428 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.pi@lists.fhcrc.org/config HTTP/1.1" 200 46402 "-" "GNU Mailman REST client v3.3.1" [29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.reg@lists.fhcrc.org/config HTTP/1.1" 200 46410 "-" "GNU Mailman REST client v3.3.1"
Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam@fredhutch.org
From: Mark Sapiro <mark@msapiro.net> Sent: Wednesday, October 29, 2025 16:07 To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: "Something went wrong | Mailman REST API not available. Please restart Mailman core.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On 10/29/25 15:40, Hirayama, Pat wrote:
Ah, thanks for the reply, Mark. Forgot to include Traceback:
Traceback (most recent call last): ...
Thanks. These are not the normal 'core isn't running' tracebacks which
are connection refused. These seem to say that core is dropping the
connection without a response. What's in mailman.log from these times?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://urldefense.com/v3/__https://lists.mailman3.org/mailman3/lists/mailma... Archived at: https://urldefense.com/v3/__https://lists.mailman3.org/archives/list/mailman...
This message sent to phirayam@fredhutch.org
On 10/29/25 16:25, Hirayama, Pat wrote:
Thanks, Mark. Yeah, it's an odd one. These are from around 22:10-22:14 in mailman.log within the mailman-core container.
[2025-10-29 22:10:26 +0000] [31] [ERROR] Socket error processing request. Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 134, in handle self.handle_request(listener, req, client, addr) File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 190, in handle_request util.reraise(*sys.exc_info()) File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 625, in reraise raise value File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 175, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 163, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in _make_collection entries = [self._resource_as_dict(resource) File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in <listcomp> entries = [self._resource_as_dict(resource) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 153, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 76, in _make_resource msg = getUtility(IMessageStore).get_message_by_id(key) File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 103, in get_message_by_id return self._get_message(row) File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 95, in _get_message with open(path, 'rb') as fp: FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/WD/BX/WDBXUFG34LR6QJUQXUWXRHZ4OO4GMQ52'
This is https://gitlab.com/mailman/mailman/-/issues/877 and https://gitlab.com/mailman/mailman/-/issues/924, fixed in mailman core 3.3.5 by https://gitlab.com/mailman/mailman/-/merge_requests/836
How old is your mailman core?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Hirayama, Pat -
Mark Sapiro