- Mailman-users - lists.mailman3.org

Problem with list creation
by Pawel Grzywaczewski Sept. 22, 2020

Sept. 22, 2020

Hi, We want to create thousand of lists. We use a simple curl command curl --user restadmin:restpass -X POST http://localhost:8001/3.1/lists -d fqdn_listname=nameOfTheList(a)example.com After creating ~2k of them we started getting errors in Mailman logs: ``` [2020-09-16 17:11:10 +0200] [4564] [DEBUG] POST /3.1/lists [2020-09-16 17:11:41 +0200] [1339] [CRITICAL] WORKER TIMEOUT (pid:4564) [2020-09-16 17:11:41 +0200] [4564] [INFO] Worker exiting (pid: 4564) [2020-09-16 17:11:41 +0200] [5019] [INFO] Booting worker with pid: 5019 ``` Since then creation fails in most cases. We increased timeout in mailman.cfg ``` [gunicorn] workers: 4 timeout: 900 ``` And we increased timeout in nginx ``` http { # MODIF UNICORN proxy_connect_timeout 300s; proxy_read_timeout 300s; # MODIF UNICORN ``` Where might be the problem? How to start investigating and fix it? Thanks, Paweł

4 7

cc'd e-mail addresses rewritten with @invalid.domain
by peterbw＠gmail.com Sept. 22, 2020

Sept. 22, 2020

Hi, We've recently noticed an issue where users who are cc'd in a message where a mailing list is also within the cc list are getting rewritten a user(a)invalid.domain. This issue looks like it may be related to Outlook and how user names are displayed specifically. Here is an example: To: test_user(a)gmail.com cc: listserv(a)lists.test.org; Williams, Peter; "Williams, Peter" maps to a user e-mail in the Outlook Global Address book. When the message is sent the list will send the message with the first name and last name separated, for example the message will appear: to: test_user(a)gmail.com cc: listserv(a)lists.test.org, Williams(a)invalid.domain, Peter(a)invalid.domain This only seems to happen with users who are cc'd who are also members of the list that is being cc'd. For example in the scenario above my e-mail that is cc'd is also a member of listserv(a)lists.test.org. This issue does not occur if the user e-mail address is listed before the mailing list in the cc field. So for example, if the cc in the scenario above contained Williams, Peter; listserv(a)lists.test.org; it would route properly. Do you know what may be causing this? Please let me know if there is any additional information I may provide regarding this and I will try to respond as soon as possible. Thanks and have a nice day! Peter

2 1

Template path
by Seth Seeger Sept. 21, 2020

Sept. 21, 2020

Hello everyone, I’m running Mailman from Docker containers, without much customization. (Postorius 1.3.3, Core 3.3.1, API 3.1, Python 3.8.5). I’ve created a list:user:notice:welcome template, and when I try to confirm a subscription, I get an error. I followed previous emails on this list which told me to set POSTORIUS_TEMPLATE_BASE_URL in /opt/mailman/web/settings_local.py, but that doesn’t seem to make any difference. (I set it to a different value and the error message didn’t change. I also tried setting it in /opt/mailman/core/settings_local.py to no effect. After each change, I restarted mailman-web and mailman-core containers.) From within mailman-web and mailman-core containers, I can reach “mailman-web:8000”. In the webpage, I see: Mailman REST API not available. Please start Mailman core. In /opt/mailman/core/var/logs/mailman.log, I get: [17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1" 200 416 "-" "GNU Mailman REST client v3.3.1" Sep 17 16:23:35 2020 (48) deque: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 268, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.8/socket.py", line 669, in readinto return self._sock.recv_into(b) socket.timeout: timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen retries = retries.increment( File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise raise value File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen httplib_response = self._make_request( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line 69, in __next__ return step() File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line 332, in _step_do_subscription self.member = self.mlist.subscribe( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py", line 505, in subscribe notify(SubscriptionEvent( File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line 32, in notify subscriber(event) File "/usr/lib/python3.8/site-packages/mailman/app/membership.py", line 176, in handle_SubscriptionEvent send_welcome_message(mlist, member, member.preferred_language) File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py", line 51, in send_welcome_message welcome_message = wrap(getUtility(ITemplateLoader).get( File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188, in get contents = getUtility(ITemplateManager).get( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110, in get contents = protocols.get(actual_uri, **auth) File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line 38, in get response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws) File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5) [2020-09-17 16:23:35 +0000] [48] [ERROR] Socket error processing request. Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 268, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.8/socket.py", line 669, in readinto return self._sock.recv_into(b) socket.timeout: timed out Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen retries = retries.increment( File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise raise value File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen httplib_response = self._make_request( File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 428, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 335, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 135, in handle self.handle_request(listener, req, client, addr) File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 191, in handle_request six.reraise(*sys.exc_info()) File "/usr/lib/python3.8/site-packages/gunicorn/six.py", line 625, in reraise raise value File "/usr/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 176, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3.8/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "/usr/lib/python3.8/site-packages/falcon/api.py", line 269, in __call__ responder(req, resp, **params) File "/usr/lib/python3.8/site-packages/mailman/rest/sub_moderation.py", line 88, in on_post self._registrar.confirm(self._token) File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line 559, in confirm list(workflow) File "/usr/lib/python3.8/site-packages/mailman/app/workflow.py", line 69, in __next__ return step() File "/usr/lib/python3.8/site-packages/mailman/app/subscriptions.py", line 332, in _step_do_subscription self.member = self.mlist.subscribe( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/mailinglist.py", line 505, in subscribe notify(SubscriptionEvent( File "/usr/lib/python3.8/site-packages/zope/event/__init__.py", line 32, in notify subscriber(event) File "/usr/lib/python3.8/site-packages/mailman/app/membership.py", line 176, in handle_SubscriptionEvent send_welcome_message(mlist, member, member.preferred_language) File "/usr/lib/python3.8/site-packages/mailman/app/notifications.py", line 51, in send_welcome_message welcome_message = wrap(getUtility(ITemplateLoader).get( File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188, in get contents = getUtility(ITemplateManager).get( File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110, in get contents = protocols.get(actual_uri, **auth) File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line 38, in get response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws) File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5) What’s the next step to get templates working? Thank you, Seth

3 8

Redundant and load balanced Mailman3
by Pawel Grzywaczewski Sept. 20, 2020

Sept. 20, 2020

Hi, Can someone share some hints about how to deploy redundant and load balanced Mailman3? What are your recommendations and best practices? For example what components to put on the same VM? Can we have 2VM with Mailman core/Postorious/Hyperkitty/django-mailman/mailmanclient connecting to one (the same) database? How to load balance mail routing (incoming)? Thanks! Paweł

3 3

Bad Signature with Gmail
by Massimo Zappalà Sept. 19, 2020

Sept. 19, 2020

Hi everyone, if I write a my lists from other email address I get the correct message: opendkim[23516]: 8EEE440FFD: DKIM-Signature field added (s=mailselector, d=catania.linux.it). While if I write to a my lists from email address Gmail, on opendkim I get the following messages: opendkim[23516]: BA95241952: no signing table match for 'maszap69(a)gmail.com' opendkim[23516]: BA95241952: s=20161025 d=gmail.com SSL error:04091068:rsa routines:int_rsa_verify:bad signature opendkim[23516]: BA95241952: bad signature data The message is received regularly by all members of the list. I also receive an email at superuser: [Django] ERROR (EXTERNAL IP): Internal Server Error: /hyperkitty/api/mailman/archive Internal Server Error: /hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1") ... If I check with mail-tester I get 10/10 If I check with dmarcanalyzer I get "This seems to be a valid DKIM Record." I don't understand where the problem is. Could anyone tell me how to solve? Thank you in advance Max

2 4

Can creating too many Django accounts crash Mailman3 server?
by Mohsen Masoudfar Sept. 19, 2020

Sept. 19, 2020

Hi, It seems anybody can go to the site: https://LISTSERVNAME/mailman3/postorius/lists/ and click on the [Sign up] on the top-right corner and create an account. This can easily be automated, even though the next step, confirming the email address, being ignored. I believe it can be used as a target for a DOS attack by creating so many accounts, that eventually causes an 'out of space' error. Is this a justified concern? Is there a way to manage this feature in a secure way? Best regards Mohsen Masoudfar Lead System Analyst, IT Operation American Association for the Advancement of Science 1200 New York Ave, NW, Washington, DC 20005 202-326-7087

4 3

Office365 posts shunted (authres.core.SyntaxError)
by Bryan Loniewski Sept. 19, 2020

Sept. 19, 2020

Hi again -- Besides my compress issue described earlier today here: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… I've been dealing with another issue (and why I upgraded to the latest code) when posting messages to my test list from our schools Office365 infrastructure (OWA). The post gets shunted; mailman.log has the following error: Sep 11 11:29:44 2020 (31396) ACCEPT: <BL0PR14MB3681E636E905E9D34588C85088240(a)BL0PR14MB3681.namprd14.prod.outlook.com> Sep 11 11:29:45 2020 (31401) Uncaught runner exception: Syntax error: Expected "=" at: ; dmarc=none action=none header.from=o... Sep 11 11:29:45 2020 (31401) Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/core/runner.py", line 173, in _one_iteration self._process_one_file(msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/core/runner.py", line 266, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/core/pipelines.py", line 50, in process handler.process(mlist, msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity.py", line 118, in process authenticate(msg, msgdata) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/utilities/retry.py", line 44, in f_retry return f(*args, **kwargs) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity.py", line 85, in authenticate prev = trusted_auth_res(msg) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/handlers/validate_authenticity.py", line 73, in trusted_auth_res authserv_id = AuthenticationResultsHeader.parse(header).authserv_id File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/__init__.py", line 206, in parse return authres.core.AuthenticationResultsHeader.parse(core_features(), string) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", line 442, in parse return self.parse_value(feature_context, string) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", line 454, in parse_value header._parse() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", line 517, in _parse result = self._parse_resinfo() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", line 550, in _parse_resinfo method, version, result = self._parse_methodspec() File "/opt/mailman/mm/venv/lib/python3.6/site-packages/authres/core.py", line 573, in _parse_methodspec raise SyntaxError('Expected "="', self._parse_text) authres.core.SyntaxError: Syntax error: Expected "=" at: ; dmarc=none action=none header.from=o... Sep 11 11:29:45 2020 (31401) SHUNTING: 1599838185.4911082+9dde922f77a1d66e3511f2709ec4b436ab6a3ecc Debugging the issue further shows this header to be the problem: authentication-results: lists-test.rutgers.edu; dkim=none (message not signed) header.d=none;lists-test.rutgers.edu; dmarc=none action=none header.from=oit.rutgers.edu; What is syntactically wrong with it and/or is it more than that? Note1: If I remove that header from a copy of the message and inject directly into Mailman it will deliver as expected. Note2: Posting messages from my personal Gmail account or cmdline mail program to the same test list will also deliver as expected. ________________________ Bryan Loniewski Rutgers University OIT:EAS:Messaging and Collaboration Services System Administrator

4 8

Re-send subscription response and file permission error in queue
by accounts-mailman＠holbrook.no Sept. 15, 2020

Sept. 15, 2020

A user sent subscription request to the `..-join@` address of a list. Email domain wasn't set up properly, so response got rejected. Now, if the user tries to send another request, no reponse comes back. In the log I find a python stack trace saying subscription already pending. ``` Sep 08 10:32:44 2020 (19918) Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 196, in _dispose mlist, msg, msgdata, parts, results) File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py", line 104, in process ISubscriptionManager(mlist).register(subscriber) File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 520, in register list(workflow) File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, in __next__ return step() File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 227, in _step_sanity_checks raise SubscriptionPendingError(self.mlist, self.address.email) mailman.interfaces.subscriptions.SubscriptionPendingError ``` Furthermore I am getting a permission denied error in the attempt of shunting the message, which I don't quite understand because the folder has user `list` group `list` and `700` mode: ``` Sep 08 10:25:29 2020 (19918) SHUNTING: 1599553529.9889646+88be5afa0bf3111a83e68cc97ce2000723ad93fd Sep 08 10:29:49 2020 (19918) Uncaught runner exception: [Errno 13] Permission denied: '/var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck' Sep 08 10:29:49 2020 (19918) Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 156, in _one_iteration msg, msgdata = self.switchboard.dequeue(filebase) File "/usr/lib/python3/dist-packages/mailman/core/switchboard.py", line 150, in dequeue with open(filename, 'rb') as fp: PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck' ``` How can I remove the pending subscription, or re-send the response email? And what could be the cause of the permission error? Using mailman 3.1.1

2 4

Held Message view, JSONDecodeError: Invalid control character
by mm3＠hovo.id.au Sept. 15, 2020

Sept. 15, 2020

I am running the Debian 10 mailman3 and mailman3-web packages with mariadb. I received an email to the list from an invoicing software and now I am unable to view or accept this email from the list. All other lists seem to be still working fine. I'm guessing there is a piece of data in my database that is corrupt or unable to be decoded, but I am at a loss on how to find and fix it. Any help would be greatly appreciated ``` Sep 11 12:29:23 2020 (11577) 127.0.0.1 - - "GET /3.1/lists/redacted(a)redacted.com/held?count=0&page=1 HTTP/1.1" 200 90 Sep 11 12:29:23 2020 (11577) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3.7/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 216, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 215, in falcon.api.API.__call__ File "falcon/api.py", line 189, in falcon.api.API.__call__ File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 160, in on_get resource = self._make_collection(request) File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 150, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 70, in _make_resource resource = super()._make_resource(request_id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 39, in _make_resource results = requests.get_request(request_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line 120, in get_request result.data_hash, expunge=False) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line 142, in confirm value = json.loads(keyvalue.value) File "/usr/lib/python3.7/json/__init__.py", line 348, in loads return _default_decoder.decode(s) File "/usr/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.7/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Invalid control character at: line 1 column 21 (char 20) ```

2 2

Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius
by Simon Handfield Sept. 14, 2020

Sept. 14, 2020

Hi, I have been working through other threads that indicate the same symptom (500 error when loading held_messages for a list). These include potential truncation of values in pendedkeyvalue table (don't this is our problem) and error when trying to access a message in /var/lib/mailman3/messages (ran the suggested code to loop through the stored files but no error was thrown.) Environment: Ubuntu 18.04 mailman3/bionic,now 3.1.1-9 all [installed,automatic] mailman3-full/bionic,now 3.1.1-9 all [installed] mailman3-web/bionic,now 0+20170523-14 all [installed,automatic] python-django-mailman3/bionic,now 1.1.0-4 all [installed,automatic] python-mailmanclient/bionic,now 3.1.1-5 all [installed,automatic] python3-mailman-hyperkitty/bionic,now 1.1.0-4 all [installed,automatic] Error reported in log with debug enabled: Traceback (most recent call last): File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 214, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 215, in falcon.api.API.__call__ (falcon/api.c:2872) File "falcon/api.py", line 189, in falcon.api.API.__call__ (falcon/api.c:2419) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 167, in on_get resource = self._make_collection(request) File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in _make_collection for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/helpers.py", line 159, in <listcomp> for resource in collection] File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 157, in _resource_as_dict resource = self._make_resource(request.id) File "/usr/lib/python3/dist-packages/mailman/rest/post_moderation.py", line 78, in _make_resource resource['msg'] = msg.as_string() File "/usr/lib/python3.6/email/message.py", line 158, in as_string g.flatten(self, unixfrom=unixfrom) File "/usr/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/lib/python3.6/email/generator.py", line 272, in _handle_multipart g.flatten(part, unixfrom=False, linesep=self._NL) File "/usr/lib/python3.6/email/generator.py", line 116, in flatten self._write(msg) File "/usr/lib/python3.6/email/generator.py", line 181, in _write self._dispatch(msg) File "/usr/lib/python3.6/email/generator.py", line 214, in _dispatch meth(msg) File "/usr/lib/python3.6/email/generator.py", line 243, in _handle_text msg.set_payload(payload, charset) File "/usr/lib/python3.6/email/message.py", line 315, in set_payload payload = payload.encode(charset.output_charset) UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-9: ordinal not in range(128) I'm sorry if this is a query that's answered already but I have not been able to find matching thread that seems to relate to my error. Thanks for any advice on where to look next. Cheers, Simon. _ [State Library of NSW]<https://www.sl.nsw.gov.au/> Simon Handfield Desktop and Infrastructure Leader simon.handfield(a)sl.nsw.gov.au<mailto:simon.handfield@sl.nsw.gov.au> +61 2 9273 1535 Macquarie Street, Sydney NSW 2000, Australia www.sl.nsw.gov.au<https://www.sl.nsw.gov.au/> This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of their organisation.

3 7