Thanks for your help Mark, it's ideal having the pointer to the issue in gitlab.

I'll raise a bug with Ubuntu because this makes mailman3 in their LTS version unusable for me.  Perhaps they can backport a fix for their version in LTS, otherwise they may have a backport or PPA source where I can get a 3.2 version, or just pin some packages from cosmic or disco.  I think documenting this will be useful to people who hit the issue in future (Bionic/18.04 is supported until 2023, and likely the most used until 20.04 is released).

Thanks, Alex

On 7/04/19 4:29 AM, Mark Sapiro wrote:

On 4/5/19 10:27 PM, Alex King via Mailman-users wrote:

...

Hi,

I've used mailman a lot in the past, and decided to install mailman3 in a recent install.  I'm unimpressed so far, it seems buggy.

Running on Ubuntu 18.04.2 LTS from packages, mailman3 3.1.1-9, mailman3-web 0+20170523-14, python-django-postorius 1.1.2-3.

After a few messages, I now can't go to the "Held Messages" in postorius, I get a 500 error.

I tried to interact on the command line, but could not work out how. Reading https://mailman.readthedocs.io/en/latest/src/mailman/docs/install.html and https://mailman.readthedocs.io/en/latest/src/mailman/commands/docs/shell.htm..., I tried:

# mailman shell Welcome to the GNU Mailman shell

...

...

...

command = cli('mailman.commands.cli_withlist.shell') Traceback (most recent call last): File "/usr/lib/python3.6/code.py", line 91, in runcode exec(code, self.locals) File "<console>", line 1, in <module> NameError: name 'cli' is not defined

I did a lot of reading of the manual and googled, but I have no idea where the cli object is supposed to come from.....?  The documentation is confusing.

The line

command = cli('mailman.commands.cli_withlist.shell')

in <https://mailman.readthedocs.io/en/latest/src/mailman/commands/docs/shell.htm...> is dependent upon some behind the scenes setup to make the doctests work. Granted this is confusing, but basically all that does is set things up for the doctests to run the 'mailman shell' command. It is not intended to be run from within 'mailman shell'.

...

Anyway, after some trial and error, i found:

...

...

...

list_manager = getUtility(IListManager) m=list_manager.get("committee@[redacted].org") from mailman.interfaces.requests import IListRequests requests = IListRequests(m) [x for x in requests.held_requests][0].id 8 requests.get_request(8) Traceback (most recent call last): File "/usr/lib/python3.6/code.py", line 91, in runcode exec(code, self.locals) File "<console>", line 1, in <module> File "/usr/lib/python3/dist-packages/mailman/database/transaction.py",

# mailman shell Welcome to the GNU Mailman shell line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line 138, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0)

It seems something was written out to disk that can't be read in for some reason (assuming a json object on disk?)

This is <https://gitlab.com/mailman/mailman/issues/385> which should be fixed by <https://gitlab.com/mailman/mailman/merge_requests/333>. The fix in in Mailman core version 3.2.0

...

How do I debug this further?  How do I find the json being decoded?  Any help would be appreciated.  (I found mailman2 just worked, I was happy with that.)

The issue is in storing a pickled representation of the json encoding of 'rule_misses'. This gets very long if you have lots of header filters, and with at least the mysql backend gets truncated in the database which causes the Unterminated string error.

I'm not sure what your options are for upgrading. The current Debian packages are all >= 3.2.0 [1], but Ubuntu doesn't get 3.2.0 until cosmic (18.10) [2].

[1] https://packages.debian.org/search?keywords=mailman3 [2] https://packages.ubuntu.com/search?keywords=mailman3

...

(My colleague posted a bug about this I believe but I don't have the link to that.)

I haven't seen it.