January 2024 - Mailman-users - lists.mailman3.org

Issues with a DMARC record leading to message being shunted
by Ralf Hildebrandt 27 Jul '24

27 Jul '24

Hi! Recently a mail was not being distributed via our MM3 instance; I searched the logs and found: Dec 22 04:54:27 2023 (24) ACCEPT: <1d431542-b386-467b-8bf2-305f19ee7eb4(a)somenet.org> Dec 22 04:54:28 2023 (28) Uncaught runner exception: list index out of range Dec 22 04:54:28 2023 (28) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 179, in _one_iteration self._process_one_file(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/core/runner.py", line 272, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/runners/pipeline.py", line 37, in _dispose process(mlist, msg, msgdata, pipeline) File "/usr/lib/python3.10/site-packages/mailman/core/pipelines.py", line 53, in process handler.process(mlist, msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 125, in process authenticate(msg, msgdata) File "/usr/lib/python3.10/site-packages/mailman/utilities/retry.py", line 44, in f_retry return f(*args, **kwargs) File "/usr/lib/python3.10/site-packages/mailman/handlers/validate_authenticity.py", line 93, in authenticate auth_result = authenticate_message( File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 395, in authenticate_message dmarc_result = check_dmarc(msg, spf_result, dkim_result, dnsfunc=dnsfunc, psddmarc=psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 343, in check_dmarc result, result_comment, from_domain, policy = dmarc_per_from(from_domain, spf_result, dkim_result, dnsfunc, psddmarc) File "/usr/lib/python3.10/site-packages/authheaders/__init__.py", line 90, in dmarc_per_from record, orgdomain = receiver_record(from_domain) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 117, in receiver_record retval = lookup_receiver_record(hostSansDmarc, dnsfunc) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 92, in lookup_receiver_record tags = answer_to_dict(str(result)) File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in answer_to_dict retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} File "/usr/lib/python3.10/site-packages/authheaders/dmarc_lookup.py", line 42, in <dictcomp> retval = {t[0].strip().lower(): t[1].strip().lower() for t in rawTags} IndexError: list index out of range Dec 22 04:54:28 2023 (28) SHUNTING: 1703220868.1418982+b27f5db154375fcd2c418d467172e4aad0d3d57a So the message has been shunted, due to this error. Something with DMARC and tags. So I checked the DMARC record for the sending domain somenet.org. It is "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;mailto:postmaster@somenet.org;ri=3600;fo=1;" which is syntactically incorrect. The extra ";mailto:postmaster@somenet.org" is wrong, I guess the tag "ruf=" is missing here! It should probably read "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@somenet.org;ruf=mailto:postmaster@somenet.org;ri=3600;fo=1;" So, should mm3 somehow "catch" this error somehow? According to the DMARC specs, an syntactically incorrect DMARC record should be ignored / handled as if none were present. -- Ralf Hildebrandt Charité - Universitätsmedizin Berlin Geschäftsbereich IT | Abteilung Netz | Netzwerk-Administration Invalidenstraße 120/121 | D-10115 Berlin Tel. +49 30 450 570 155 ralf.hildebrandt(a)charite.de https://www.charite.de

4 11

Digests not working correctly
by Joel Lord 08 Jun '24

08 Jun '24

So I seem to have problems with both periodic AND size-triggered digests. Size first, this is simpler. Just got one sent out, clearly triggered by a new email getting added to the queued up messages, but the setting for the list is the default 30KB and the digest came out at 23.2 KB. Is it measuring the accumulated queue as size on disk including metadata and filesystem block sizes or something, rather than the size of what the emailed digest will be? Periodic is much more complicated, even to explain. I have the cron job running nightly at 11:00 PM and it certainly seems to be working, except for one really odd thing. The list also definitely has periodic digests enabled in the web UI. This is a very bursty list, so it will go mad with traffic for a few weeks and then will go mostly idle for a month or so. The first digest of a burst is definitely size-triggered, but pretty reliably it has the last few messages from the previous burst in there. So the one I got today had three messages from today and one message from May 4th. But that May 4th message should have gone out at 11:00 PM on May 4th with the daily digest and then gotten flushed from the pending digest queue, right? -- Joel Lord

3 14

Change config to create a new list with all archives off
by Tibor Molnar 21 Apr '24

21 Apr '24

Can archiving be changed anywhere in mailman3 configs, so then all new lists created do not archive by default?

5 8

Mailman 3 hosting query
by Bill Robertson 18 Apr '24

18 Apr '24

Hi I'm investigating the possibility of transferring a fairly small mailing list I own from Google Groups to MM3 via a hosting service. I'd appreciate any pointers regarding what I should be looking out for with companies that offer this service It also seems very hard to find out which hosting companies are worth considering (In particular, reliability and support are important) Any help much is much appreciated Thanks

4 5

mailman3 digest sending questions and also adding docker to the mix
by bob B 17 Apr '24

17 Apr '24

So I have been down the digest rabbit hole. So to confirm my assumptions/findings: Digest size threshold---- means when the total size of held messages hits this setting, a digest is sent Digest Volume Frequency. ---- specifies when the digest volume number gets incremented for digest messages, IT DOES NOT send out digests based on the setting, daily, weekly ... So what does the GUI setting "Send Digest Periodically yes no" do? how periodically does it send digests, if this is set to yes? If I want to send a digest out for a list every day, or every week, how do I set that, especially in the docker environment? Is this something that can be configured in the GUI and configured by each of my list admins? I found that ”'mailman digests --periodic' via cron” should be set in googling, but does that mean I have to do a cron job for any list that wants a scheduled digest sent? My use case is, that I have a list admin who set a size threshold, but regardless of that, they also want their list digest sent out daily, they thought it was the "Digest Volume Frequency" setting but that is not correct. However in their testing they are not getting daily digest, they are getting digests only when the threshold is reached, BUT they have "Send Digest Periodically" set to "yes" and "Enable Digests" is set to "yes"

7 17

hyperkitty.lib.utils Failed to update the fulltext index
by Ben Harris 13 Apr '24

13 Apr '24

hello there, i migrated to a new host last night and i've been getting this error from a cronjob mail every minute. anyone have any ideas what would cause this? ERROR 2019-04-17 14:26:33,056 27205 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/hyperkitty/search_indexes.py", line 87, in update_index update_cmd.update_backend("hyperkitty", "default") File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 276, in update_backend last_max_pk=max_pk) File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/lib/python2.7/dist-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/lib/python2.7/dist-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1)) thanks, ben

5 8

Error -- postorius Un-handled exception
by Mark 12 Apr '24

12 Apr '24

Hi All, I've just installed MM3 on a fresh Debian 12 using the Virtualenv method. (venv)...$ mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] Postorius Version 1.3.10 After creating a couple of lists, the lists are shown in the table on the "Mailing lists" webpage as they should, but clicking on any of the listnames returns an error page: Something went wrong HTTP Error 500: {"title": "500 Internal Server Error"} And the Mailman log is a mystery to me, and no error is reported in the Nginx log ... # tail -f /opt/mailman/web/logs/mailmanweb.log ERROR 2024-01-24 05:41:46,327 51799 postorius Un-handled exception: HTTP Error 500: {"title": "500 Internal Server Error"} Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 103, in view return self.dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/postorius/views/generic.py", line 76, in dispatch return super(MailingListView, self).dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 142, in dispatch return handler(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/postorius/views/list.py", line 359, in get and 'hyperkitty' in archivers # noqa: W504 ^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen _collections_abc>", line 780, in __contains__ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/base.py", line 146, in __getitem__ return self._get(key) ^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/base.py", line 88, in _get return self.rest_data[key] ^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/base.py", line 74, in rest_data response, content = self._connection.call(self._url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/connection.py", line 160, in call raise HTTPError(params.get('url'), response.status_code, urllib.error.HTTPError: HTTP Error 500: {"title": "500 Internal Server Error"} ERROR 2024-01-24 05:41:46,333 51799 django.request Internal Server Error: /postorius/lists/testing.example.com/ ERROR 2024-01-24 05:41:46,333 51799 django.request Internal Server Error: /postorius/lists/testing.example.com/ Could anyone steer me in the right direction for fixing this? Thanks, Mark

6 20

export/import list?
by Dave McGuire 01 Mar '24

01 Mar '24

I'm migrating to a new Mailman3 instance, finally escaping a fairly old (3.1.1) and problematic distribution repo install on a Linux VM and moving to a venv install in a native SmartOS Zone. I've gotten everything running well and integrated with our (fairly complex) mail server. I'd like to begin migration of the lists on the old mm3 instance. Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc? Thanks, -Dave -- Dave McGuire, AK4HZ New Kensington, PA

6 9

search this list searches more then just this list
by Marco van Tol 06 Feb '24

06 Feb '24

Hi there, We run the following versions: mailman-core: 3.3.9 mailman-core-api: 3.1 mailman-core-python: 3.11.6 hyperkitty: 1.3.8 We have several lists on our server, I will write down a list of example list names that are good enough for the purpose of my question. xx(a)domain.tld <mailto:xx@domain.tld> yy-xx(a)domain.tld <mailto:yy-xx@domain.tld> zz-xx(a)domain.tld <mailto:zz-xx@domain.tld> When I now visit the archives for xx(a)domain.tld <mailto:xx@domain.tld> and enter a search phrase in “search this list”, the results include the other 2 lists. It looks like “search this list” searches for keyword in any listname that matches the one from "this list". Is there a way to really only search “this list” when I go to the archives for xx(a)domain.tld <mailto:xx@domain.tld>? I have tried to change the url for the results to have a ‘^’ in front of the listname, but that gives an error :-) Thanks in advance! Marco van Tol

4 23

after upgrade: OperationalError: no such column: hyperkitty_mailinglist.archive_rendering_mode
by Hans Mayer 31 Jan '24

31 Jan '24

Dear All, recently I did a system os upgrade debian 11.8 -> 12.4 / bullseye -> bookworm This did also upgrade Mailman3 version 3.3.3 -> 3.3.8 as a used the packages which came with Debian OS. Initially after the upgrade I had some troubles with the MM3 services to start, but I fixed this. Now almost everything is running. Only the Archives are not working. The browser shows a server error, when I look at the mailman-web.log I see ERROR 2024-01-31 15:14:14,403 2699563 django.request Internal Server Error: /mailman3/hyperkitty/ Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/sqlite3/base.py", line 423, in execute return Database.Cursor.execute(self, query, params) sqlite3.OperationalError: no such column: hyperkitty_mailinglist.archive_rendering_mode When I check the schema I find such a column in table "mailinglist" I assume there is only one solution, to uninstall and delete everything and start from scratch. Or is there a way the check all tables with a script and do an table update for missing columns ? Or a partial installation of hyperkitty, as I don't care about the archives. I am using sqlite3 as DB backend. Kind regards Hans --

2 1