On 03/14/2018 03:04 AM, Aymeric Mansoux wrote:
So far so good, however, for some reasons, a couple of our lists have started to accumulate held messages that cannot be manipulated via the REST API. Regardless if we use the web interface or mailmanclient, it ends up with:
Mar 14 10:58:32 2018 (24287) REST request handler error: Traceback (most recent call last):
File "/usr/lib/python3.5/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response)
File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "/usr/local/lib/python3.5/dist-packages/falcon/api.py", line 244, in __call__ responder(req, resp, **params) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 70, in _make_resource resource = super()._make_resource(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/rest/post_moderation.py", line 39, in _make_resource results = requests.get_request(request_id) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/local/lib/python3.5/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/local/lib/python3.5/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.5/json/__init__.py", line 319, in loads return _default_decoder.decode(s) File "/usr/lib/python3.5/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.5/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0) Mar 14 10:58:32 2018 (24287) 127.0.0.1 - - "GET /3.1/lists/xx@xx.xx/held HTTP/1.1" 500 59Is there a way to manipulate/see/remove these mails differently? Right now this prevent moderation of any held messages (note: some lists do not suffer from this and the held panel or mailmanclient works perfectly fine).
We've seen this issue before with a different underlying cause. See <https://gitlab.com/mailman/mailman/issues/256>.
There are a couple of problems here. The serious problem is if there is a held message which somehow throws an exception in processing in mailman/rest/post_moderation.py, that totally breaks the Postorius held messages view for that list. We need to fix that.
Also it would be good to see the problem message.
Here are some things you can do in 'mailman shell' (aka mailman withlist). It is not clear what might not work, but you can try
mailman shell -l list@domain
which will respond
Welcome to the GNU Mailman shell The variable 'm' is the list@domain mailing list
Then you should be able to do at the >>> prompts:
req_db = IListRequests(m) reqs = list(req_db.held_requests)
reqs is now a list of the held requests and you can do something like
for req in reqs: ... print('{}: {}'.format(req.id, req.request_type)) ...
or since req_db.held_requests is a generator
for req in rdb.held_requests: ... print('{}: {}'.format(req.id, req.request_type)) ...
either of which will produce a list like
1: RequestType.held_message
with the IDs and types of each request.
You can handle these requests with something like
from mailman.app.moderator import handle_message handle_message(m, 1, Action.defer)
where the second argument is the ID of the message and the action is one of Action.defer, Action.hold, Action.discard, Action.reject or Action.accept, the first two of which do nothing, but the others will do the indicated action.
You can see the actual messages like this
msg_db = getUtility(IMessageStore)
then
key = req[0].key msg_db.get_message_by_id(key)
which will return the actual held message that is in this case index 0 in the list of held_requests.
You may have to do
commit()
when done, although if you don't do abort(), just exiting with control-D should do a commit().
I hope you find this helpful, and if you do determine the content of the offending message(s), it will be helpful if you post that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan