May 2021 - Mailman-users - lists.mailman3.org

trying to clean up the "held messages" mess
by Bjoern Franke 28 May '21

28 May '21

Hi, I migrated a Mailman3 installation on Debian10 yesterday evening from Debian packages to PIP (thanks to [1]). I think some bugs which existed before are now cleaned out, but mailman still threw exceptions when trying to get the "held messages". I went through the pendedkeyvalue table of the MySQL DB to get all request ids of the held messages and their lists and discarded the messages via curl and the REST API. Mailman also logged the discarding was successful. The amount of held messages is now reduced, but e.g. a list with 120 held messages before has till 59 held messages. And now I'm confused: where are they? Best Regards Bjoern [1]https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10

2 3

non-member moderation: list default vs default processing
by dancab＠caltech.edu 24 May '21

24 May '21

I've started testing and importing lists from Mailman2 to Mailman3. I've found that there's an inconsistency with the non-members moderation setting. As a result we have non-members unable to email a list via Mailman3 that had no problem on Mailman2. Each time this problem has come up I've had to change the moderation setting from "List Default" to "Default Processing". But I've also noticed that some non-members were imported and already have "Default Processing" configured. Can you help me understand what determines this setting in Mailman3? It seems this would be something addressed via the mailman import21 option I've been using to import the lists. Thanks. ---- Dan Caballero Systems Administrator Academic Computing Solutions IMSS - Caltech https://imss.caltech.edu

2 4

Intermittent: pymysql.err.InternalError: Packet sequence number wrong - got 102 expected 77
by Torge Riedel 24 May '21

24 May '21

|Hi list, sometimes I get the following error in mailman core's log: "pymysql.err.InternalError: Packet sequence number wrong - got 102 expected 77". I attached the full log after this mail. I check the provided link in the log (|||http://sqlalche.me/e/2j85|) and also searched for it on the internet and found the following possible reasons: 1. DB user does not have permissions at all -> I doubt that, because in general mailman is working ok 2. DB user is not allowed to connect from remote -> I doubt that too, because everything is on one host and DB user is allowed to connect from localhost 3. Multi-Threading -> pymysql is mentioned to be not thread-safe and each thread needs its own connection May this be caused by multi-threading? What do I have to do (is there anything I can do?) to solve this on my own?||||| Kind regards Torge || |Full log follows: [2021-05-23 07:31:45 +0200] [8244] [ERROR] Error handling request /3.1/lists?advertised=true&count=10&page=1 Traceback (most recent call last): File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1284, in _execute_context cursor, statement, parameters, context File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute cursor.execute(statement, parameters) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute result = self._query(query) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query conn.query(q) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 517, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 732, in _read_query_result result.read() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1082, in read self._read_result_packet(first_packet) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1151, in _read_result_packet self._get_descriptions() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1222, in _get_descriptions field = self.connection._read_packet(FieldDescriptorPacket) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 671, in _read_packet % (packet_number, self._next_seq_id)) pymysql.err.InternalError: Packet sequence number wrong - got 102 expected 77 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 274, in falcon.api.API.__call__ File "falcon/api.py", line 269, in falcon.api.API.__call__ File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/lists.py", line 322, in on_get resource = self._make_collection(request) File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/helpers.py", line 172, in _make_collection request, self._get_collection(request)) File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/helpers.py", line 167, in _paginate return list_start, total_size, collection[list_start:list_end] File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/utilities/queries.py", line 42, in __getitem__ return self._query[index] File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3149, in __getitem__ return list(res) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3481, in __iter__ return self._execute_and_instances(context) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3506, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1020, in execute return meth(self, multiparams, params) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_clauseelement distilled_params, File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1324, in _execute_context e, statement, parameters, cursor, context File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1518, in _handle_dbapi_exception sqlalchemy_exception, with_traceback=exc_info[2], from_=e File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_ raise exception File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1284, in _execute_context cursor, statement, parameters, context File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute cursor.execute(statement, parameters) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute result = self._query(query) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query conn.query(q) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 517, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 732, in _read_query_result result.read() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1082, in read self._read_result_packet(first_packet) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1151, in _read_result_packet self._get_descriptions() File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1222, in _get_descriptions field = self.connection._read_packet(FieldDescriptorPacket) File "/opt/mailman/core/venv/lib/python3.6/site-packages/pymysql/connections.py", line 671, in _read_packet % (packet_number, self._next_seq_id)) sqlalchemy.exc.InternalError: (pymysql.err.InternalError) Packet sequence number wrong - got 102 expected 77 [SQL: SELECT mailinglist.list_id AS mailinglist_list_id, mailinglist.preferred_language AS mailinglist_preferred_language, mailingli$ FROM mailinglist WHERE mailinglist.advertised = true ORDER BY mailinglist.list_id LIMIT %(param_1)s] [parameters: {'param_1': 10}] (Background on this error at: http://sqlalche.me/e/2j85) |

3 5

Unknown attribute: process_bounces
by xlefloch＠alwaysdata.com 21 May '21

21 May '21

Hello, I've upgraded my Mailman installation from those versions: mailman==3.2.2 postorius==1.2.4 mailmanclient==3.2.2 To those ones: mailman==3.3.0 postorius==1.3.3 mailmanclient==3.3.2 I'm having issues in Postorius in the settings section: Unknown attribute: process_bounces. When i'm printing the settings list for the concerned list retrieved by the REST API, i'am missing all the parameters related to the management of bounces. In the "mailinglist" table of the Mailman database, i have the columns managing these parameters. Anyone have any ideas? Regards,

2 3

New Mailman 3 installation issues
by sarah.hartt＠ionica.ca 21 May '21

21 May '21

I have been trying to get a mailman 3 installation working for longer than I'd like to admit. Core appears to be installed and working as desired but Postorius (and hyperkitty, but I haven't even started troubleshooting it while trying to get the web interface working) is only loading the main page with no lists shown and all remaining links return a server error (specific text is: "Server error An error occurred while processing your request." with the error text highlighted). I think I followed all of the installation instructions, including the updated pythonpath line in the systemd module, but it's entirely possible that there is an error somewhere along the way. I have had to make two minor tweaks, the /static/ location was moved from /opt/mailman/web/static to /var/www/mailman/ (owner mailman:nginx) as nginx was giving permissions errors (even if it owned the files or with the mod set to 777) with the default location and wouldn't load the css, and installed postgres directly from their repo as the method in the guide doesn't create the user. Aside from that everything should be exactly as laid out in the installation guide. The logs are showing everything running and the DB logs show all the queries going through successfully, there are no errors in the web logs corresponding with the errors being served on the site aside an entry for a 500 in the access logs [18/May/2021:12:13:01] "GET /archives/ HTTP/1.1" 500 1102 "http://mailman3/mailman3/lists/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.91 Safari/537.36" "-" when I took a look in the DB tables many of them appear to be empty, I have a few lists in core that were imported from 2.1 but no new lists created. Mailman: 3.3.4 Postorius: 1.3.4 OS: Oracle Linux 8.3 Python: 3.6 postgres: 13.2

4 11

timezone in hyperkitty
by erlonmz＠hotmail.com 21 May '21

21 May '21

How do I change the hyperkitty timezone?

7 8

Re: [EXTERNAL] [Mailman-Users] Re: Help with 'The message has no valid senders'
by Mark Sapiro 21 May '21

21 May '21

On 5/19/21 10:37 AM, McGuire, Tommy M. (MSFC-IS90)[EAST2] wrote: > > Here are (what I think are :-)) the anonymized relevant parts of the smtp.log. > > May 19 08:52:29 2021 (126270) _handle_client readline: b'MAIL FROM:<aaaaaaaaaa.a.aaaaa-1(a)nasa.gov> SIZE=11726\r\n' > May 19 08:52:29 2021 (126270) ('127.0.0.1', 52130) >> b'MAIL FROM:<aaaaaaaaaa.a.aaaaa-1(a)nasa.gov> SIZE=11726' > May 19 08:52:29 2021 (126270) ('127.0.0.1', 52130) sender: aaaaaaaaaa.a.aaaaa-1(a)nasa.gov > May 19 08:52:29 2021 (126270) ('127.0.0.1', 52130) << b'250 OK' > ... > May 19 08:52:29 2021 (126270) DATA readline: b'From: "Aaaaa, Aaaaaaaaaa A. (location)[employer,\r\n' > May 19 08:52:29 2021 (126270) DATA readline: b' Inc.]" <aaaaaaaaaa.a.aaaaa-1(a)nasa.gov>\r\n' That all looks good. I don't understand what's wrong. Can you apply the attached patch to mailman/rules/no_senders.py and see what you get in Mailman's debug.log. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 1

Can't deliver mails in queue
by Nathanael Schneider 20 May '21

20 May '21

Hello, I have stumbled upon some weird issues with my migration to mailman3.. I've followed the step by step instructions for installing mm3 manually in a venv with postfix and virtual aliases (if that is of any significance). Now the Problem is, that some lists don't deliver EMails at all. The list was one which I migrated with the mailman21 command. But the Mail I sent did not get through to no one and the mailman logs did not show anything. I then created a completely new List which did not exists in mm2.1, subscribed two addresses and sent myself an EMail. That one worked as intended. I then deleted the whole broken list and created it cleanly, added the same two people, sent an email and nothing. Did not show up in the Moderation Queue or anywhere. it also doesn't get shunted. It just never arrives. I then turned the log level to debug on the smtp module. And I only get the following statement, ever repeating, never ending: [...] May 19 18:54:40 2021 (8280) Using agent: <mailman.mta.deliver.Deliver object at 0x7fb49b143780> May 19 18:54:40 2021 (8280) IndividualDelivery to: nathanael(a)lumenliquid.net May 19 18:54:40 2021 (8280) Using agent: <mailman.mta.deliver.Deliver object at 0x7fb4974541d0> May 19 18:54:40 2021 (8280) IndividualDelivery to: nathanael(a)lumenliquid.net May 19 18:54:40 2021 (8280) Using agent: <mailman.mta.deliver.Deliver object at 0x7fb4978db438> May 19 18:54:40 2021 (8280) IndividualDelivery to: nathanael(a)lumenliquid.net May 19 18:54:40 2021 (8280) Using agent: <mailman.mta.deliver.Deliver object at 0x7fb497f05f28> May 19 18:54:40 2021 (8280) IndividualDelivery to: nathanael(a)lumenliquid.net May 19 18:54:40 2021 (8280) Using agent: <mailman.mta.deliver.Deliver object at 0x7fb4978adb38> May 19 18:54:40 2021 (8280) IndividualDelivery to: nathanael(a)lumenliquid.net [...] I have no clue what to do with this. I then turned the debug module loglevel to debug. Also nothing useful. Just an endless list of [...] May 19 18:17:29 2021 (7209) [outgoing] <function deliver at 0x7f2faf384c80>: <B9531076-2480-4925-9A2B-57802B53D5ED(a)gmx.de> May 19 18:17:29 2021 (7203) [ArchiveRunner] starting oneloop May 19 18:17:29 2021 (7203) [ArchiveRunner] ending oneloop: 0 May 19 18:17:29 2021 (7209) [OutgoingRunner] finishing filebase: 1621441049.4089153+6dff56673dc64b890dd352941267782c264586dd May 19 18:17:29 2021 (7209) [OutgoingRunner] doing periodic May 19 18:17:29 2021 (7209) [OutgoingRunner] committing transaction May 19 18:17:29 2021 (7209) [OutgoingRunner] checking short circuit May 19 18:17:29 2021 (7209) [OutgoingRunner] ending oneloop: 1 May 19 18:17:29 2021 (7209) [OutgoingRunner] starting oneloop May 19 18:17:29 2021 (7209) [OutgoingRunner] processing filebase: 1621441049.4628382+6a0437ea387a2508ad947c0a51712075cbdb6d2e May 19 18:17:29 2021 (7209) [OutgoingRunner] processing onefile May 19 18:17:29 2021 (7209) [outgoing] <function deliver at 0x7f2faf384c80>: <B9531076-2480-4925-9A2B-57802B53D5ED(a)gmx.de> [...] Which also tells me nothing I can understand. I have no clue what's going wrong. (I've turned on debugging both on the original member list and the reduced member list. Both hang on the first delivery they do). If I restart mailman, the message gets correctly shunted as it is interrupted mid-delivery. My maillogs don't register any activity beside the listpost. Any ideas what I need to do to figure out what's not working here or where the culprit could be? Not sure if it is related to the mailinglist being in mailman2.1 before. -- Nathanael Schneider

2 5

Bounce counter visibility for List Admins
by smorytko＠gmail.com 19 May '21

19 May '21

As you may know, we use MM3 but the Affinity/Empathy flavor and we only have List Administrator using the Postorious/Affinity. Unlike many here, we have no access to any server-based files. Can someone tell me if Postorious gives list administrators visibility into the bounce counters? While List Admins get to set various bounce counters we have no visibility of those counters until, all of a sudden, an entire source of subscribers hits the bounce threshold and gets bounced off the list. It would be really nice to be able to see the bounce counters and possibly head off trouble before users/subscriber mail hosts. It can serve as an effective early warning system that something is wrong. Assuming Postorious cannot, is there another way List Admins can inspect bounce counters? Email command? Thanks - Steve

3 3

Re: [EXTERNAL] [Mailman-Users] Re: Help with 'The message has no valid senders'
by Mark Sapiro 19 May '21

19 May '21

Putting this back on the list. On 5/19/21 9:46 AM, McGuire, Tommy M. (MSFC-IS90)[EAST2] wrote: >> -----Original Message----- >> From: Mark Sapiro <mark(a)msapiro.net> ... >> Have you set >> ``` >> [mailman] >> sender_headers: ... >> ``` >> in mailman.cfg?, and if so, to what? > > We have not changed it: > > [mailman] sender_headers: from from_ reply-to sender > > According to the smtp.log, there was an envelope sender ("MAIL FROM:.."), which matched the From header. In the example I'm looking at right now, the sender was not a member of the list, but the Accept these non-members box in the list settings has a regexp that matches the sender's domain. In another example from this morning, to a different list, the envelope sender and From header match, and is a member of the list. > > I don't see any obvious reason or configuration setting that would make either address invalid. First of all, the discard has nothing to do with whether the sender has any connection to the list. It only says the message had no sender at all. Assuming the message had a From: with a syntactically valid email address, this discard should not occur. Also, if the envelope (MAIL FROM:) was a syntactically valid email address, this discard should not occur. The address is validated by mailman/email/validate.py by first splitting it into `local_part`, `@`, `[domain_parts]` where domain_parts are the `between dots` parts of the domain and then validating the local_part and domain with regexps defined per > # What other characters should be allowed? > _valid_local = re.compile("[-0-9a-z!#$%&'*+./=?@_`{}~]", re.IGNORECASE) > # Strictly speaking, both ^ and | are allowed and others are allowed in quoted > # local parts, but this can open the door to certain web exploits so we don't > # allow them. > _valid_domain = re.compile('[-a-z0-9]', re.IGNORECASE) > # These are the only characters allowed in domain parts. Does the From: local part or the MAIL FROM: contain any characters not in the [-0-9a-z!#$%&'*+./=?@_`{}~] class? -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0