[MM3-users] Re: "Something went wrong | Mailman REST API not available. Please restart Mailman core.

Oct. 29, 2025

      Thanks, Mark. Yeah, it's an odd one. These are from around 22:10-22:14 in mailman.log within the mailman-core container.
[29/Oct/2025:22:10:23 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:23 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:23 +0000] "POST /3.1/lists/find HTTP/1.1" 200 13020 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:25 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org HTTP/1.1" 200 433 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/roster/owner HTTP/1.1" 200 1175 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/roster/moderator HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100@lists.fhcrc.org/config HTTP/1.1" 200 3016 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100@lists.fhcrc.org/requests HTTP/1.1" 200 1307 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100.lists.fhcrc.org/member/aismael%40fredhutch.org HTTP/1.1" 200 509 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:10:26 +0000] "GET /3.1/lists/cancerbiology100@lists.fhcrc.org/requests HTTP/1.1" 200 1307 "-" "GNU Mailman REST client v3.3.1"
[2025-10-29 22:10:26 +0000] [31] [ERROR] Socket error processing request.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 134, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 190, in handle_request
util.reraise(*sys.exc_info())
File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 625, in reraise
raise value
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 175, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper
rtn = function(*args, **kws)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__
return super().__call__(environ, start_response)
File "/usr/local/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__
responder(req, resp, **params)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 163, in on_get
resource = self._make_collection(request)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in _make_collection
entries = [self._resource_as_dict(resource)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in <listcomp>
entries = [self._resource_as_dict(resource)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 153, in _resource_as_dict
resource = self._make_resource(request.id)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 76, in _make_resource
msg = getUtility(IMessageStore).get_message_by_id(key)
File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 103, in get_message_by_id
return self._get_message(row)
File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 95, in _get_message
with open(path, 'rb') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/WD/BX/WDBXUFG34LR6QJUQXUWXRHZ4OO4GMQ52'
[29/Oct/2025:22:11:49 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org HTTP/1.1" 200 373 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/roster/owner HTTP/1.1" 200 624 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/roster/moderator HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/config HTTP/1.1" 200 3262 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest.lists.fhcrc.org/member/cit-sys-engineering%40fredhutch.org HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:50 +0000] "GET /3.1/lists/pattest@lists.fhcrc.org/held?count=50&page=1 HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:11:56 +0000] "GET /3.1/lists/vtn.fiscal.pattest.lists.fhcrc.org HTTP/1.1" 404 58 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org HTTP/1.1" 200 389 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/roster/owner HTTP/1.1" 200 628 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/roster/moderator HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal@lists.fhcrc.org/config HTTP/1.1" 200 46422 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:02 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/archivers HTTP/1.1" 200 81 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:03 +0000] "GET /3.1/lists/vtn.fiscal@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:03 +0000] "GET /3.1/lists/vtn.fiscal.lists.fhcrc.org/member/cit-sys-engineering%40fredhutch.org HTTP/1.1" 404 26 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:03 +0000] "GET /3.1/lists/vtn.fiscal@lists.fhcrc.org/requests HTTP/1.1" 200 90 "-" "GNU Mailman REST client v3.3.1"
[2025-10-29 22:12:03 +0000] [31] [ERROR] Socket error processing request.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 134, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 190, in handle_request
util.reraise(*sys.exc_info())
File "/usr/local/lib/python3.8/site-packages/gunicorn/util.py", line 625, in reraise
raise value
File "/usr/local/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 175, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper
rtn = function(*args, **kws)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__
return super().__call__(environ, start_response)
File "/usr/local/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__
responder(req, resp, **params)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 163, in on_get
resource = self._make_collection(request)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in _make_collection
entries = [self._resource_as_dict(resource)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/helpers.py", line 172, in <listcomp>
entries = [self._resource_as_dict(resource)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 153, in _resource_as_dict
resource = self._make_resource(request.id)
File "/usr/local/lib/python3.8/site-packages/mailman/rest/post_moderation.py", line 76, in _make_resource
msg = getUtility(IMessageStore).get_message_by_id(key)
File "/usr/local/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 103, in get_message_by_id
return self._get_message(row)
File "/usr/local/lib/python3.8/site-packages/mailman/model/messagestore.py", line 95, in _get_message
with open(path, 'rb') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/var/messages/LZ/OC/LZOCY4BX6FBKMLJZQKVED6AVXOUBDFXQ'
[29/Oct/2025:22:12:15 +0000] "GET /3.1/lists?advertised=true&count=0&page=1 HTTP/1.1" 200 92 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:15 +0000] "GET /3.1/lists?advertised=true&count=10&page=1 HTTP/1.1" 200 4353 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:15 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:12:15 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
Oct 29 22:13:20 2025 (10) ACCEPT: <DM6PR11MB4692672CB4AB8CA1AB739567D5FAA@DM6PR11MB4692.namprd11.prod.outlook.com>
[29/Oct/2025:22:13:33 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:34 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:34 +0000] "POST /3.1/lists/find HTTP/1.1" 200 2569 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists?count=0&page=1 HTTP/1.1" 200 93 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists?count=10&page=1 HTTP/1.1" 200 4441 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/015.site.257.cergo015.site.257.cer@lists.fhcrc.org/config HTTP/1.1" 200 46497 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/1100_labs@lists.fhcrc.org/config HTTP/1.1" 200 3053 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/9228.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/9292.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/a2223dsmbdl@lists.fhcrc.org/config HTTP/1.1" 200 3145 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/a244.ct@lists.fhcrc.org/config HTTP/1.1" 200 46388 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/a5397.vtn603.pi.coord@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/ab.mop@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/accessdev@lists.fhcrc.org/config HTTP/1.1" 200 3173 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:13:37 +0000] "GET /3.1/lists/actg.site.31422.coord@lists.fhcrc.org/config HTTP/1.1" 200 46429 "-" "GNU Mailman REST client v3.3.1"
Oct 29 22:13:47 2025 (10) ACCEPT:
<MN2PR11MB4333B8F87B77ED1E7F83422DDDFAA@MN2PR11MB4333.namprd11.prod.outlook.com>
[29/Oct/2025:22:14:23 +0000] "GET /3.1/lists?count=0&page=1 HTTP/1.1" 200 93 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists?count=200&page=1 HTTP/1.1" 200 83390 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/domains HTTP/1.1" 200 926 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/fhcrc.org HTTP/1.1" 200 193 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/fredhutch.org HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/lists.fhcrc.org HTTP/1.1" 200 220 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/domains/lists.hanc.info HTTP/1.1" 200 201 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/015.site.257.cergo015.site.257.cer@lists.fhcrc.org/config HTTP/1.1" 200 46497 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/1100_labs@lists.fhcrc.org/config HTTP/1.1" 200 3053 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/9228.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/9292.dsmb.members.only@lists.fhcrc.org/config HTTP/1.1" 200 3082 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/a2223dsmbdl@lists.fhcrc.org/config HTTP/1.1" 200 3145 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/a244.ct@lists.fhcrc.org/config HTTP/1.1" 200 46388 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/a5397.vtn603.pi.coord@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/ab.mop@lists.fhcrc.org/config HTTP/1.1" 200 46380 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/accessdev@lists.fhcrc.org/config HTTP/1.1" 200 3173 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.coord@lists.fhcrc.org/config HTTP/1.1" 200 46429 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.dataqc@lists.fhcrc.org/config HTTP/1.1" 200 46389 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.pharm@lists.fhcrc.org/config HTTP/1.1" 200 46428 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.pi@lists.fhcrc.org/config HTTP/1.1" 200 46402 "-" "GNU Mailman REST client v3.3.1"
[29/Oct/2025:22:14:24 +0000] "GET /3.1/lists/actg.site.31422.reg@lists.fhcrc.org/config HTTP/1.1" 200 46410 "-" "GNU Mailman REST client v3.3.1"
Pat Hirayama
Pronouns: he/him/his
Systems Engineer
IT | Systems Engineering
Fred Hutchinson Cancer Center
O 206.667.4856
phirayam@fredhutch.org

From: Mark Sapiro <mark@msapiro.net>
Sent: Wednesday, October 29, 2025 16:07
To: mailman-users@mailman3.org <mailman-users@mailman3.org>
Subject: [MM3-users] Re: "Something went wrong | Mailman REST API not available. Please restart Mailman core.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On 10/29/25 15:40, Hirayama, Pat wrote:
...
Ah, thanks for the reply, Mark.  Forgot to include Traceback:
Traceback (most recent call last):
...
Thanks. These are not the normal 'core isn't running' tracebacks which
are connection refused. These seem to say that core is dropping the
connection without a response. What's in mailman.log from these times?
--
Mark Sapiro <mark@msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan

Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-leave@mailman3.org
https://urldefense.com/v3/__https://lists.mailman3.org/mailman3/lists/mailma...
Archived at: https://urldefense.com/v3/__https://lists.mailman3.org/archives/list/mailman...
This message sent to phirayam@fredhutch.org