- Mailman-users - lists.mailman3.org

Mailman 3 install questions: HAYSTACK_CONNECTIONS
by heller＠deepsoft.com 07 Jul '24

07 Jul '24

I am using the debian 12 repo versions of mailman3, django, posterious, hyperkitty, and xapian: sharky5% dpkg-query -l python3-{django,django-mailman3,django-hyperkitty,django-postorius,xapian} mailman3\* | grep '^ii' ii mailman3 3.3.8-2~deb12u2 all Mailing list management system ii mailman3-doc 3.3.8-2~deb12u2 all Mailing list management system documentation ii mailman3-web 0+20200530-2.1 all Django project integrating Mailman3 Postorius and HyperKitty ii python3-django 3:3.2.19-1+deb12u1 all High-level Python web development framework ii python3-django-hyperkitty 1.3.7-1 all Web user interface to access GNU Mailman3 archives ii python3-django-mailman3 1.3.9-1 all Django library to help interaction with Mailman3 (Python 3 version) ii python3-django-postorius 1.3.8-3 all Web user interface to access GNU Mailman3 ii python3-xapian 1.4.22-1 amd64 Xapian search engine interface for Python3 Before adding the HAYSTACK_CONNECTIONS options, I have this: sharky5% sudo mailman-web haystack_info --no-color --skip-checks Number of handled 1 index(es). - Model: Email by Index: <hyperkitty.search_indexes.EmailIndex object at 0x7fd0de1b0470> When I add this to /etc/mailman3/maymail_web.py (adapted from the section titled: "Setting up Fulltext search (xapian)" on https://docs.mailman3.org/en/latest/install/virtualenv.html#set-up-xapian HAYSTACK_CONNECTIONS = { 'default': { 'PATH': "/var/lib/mailman3/web/fulltext_index", 'ENGINE': 'xapian_backend.XapianEngine' }, } I get: File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked ModuleNotFoundError: No module named 'xapian_backend' So, do I use Xapian or not? Is something missing somewhere? Is this a documentation bug or something else? -- Robert Heller -- Cell: 413-658-7953 GV: 978-633-5364 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller(a)deepsoft.com -- Webhosting Services

2 1

Subject Munging
by Sam Darwin 06 Jul '24

06 Jul '24

Hi, We have a user who is proposing that reply messages from a mailing list should be in the following format: Re: [MM3-users] Ubuntu 24.04 upgrade notes and not [MM3-users] Re: Ubuntu 24.04 upgrade notes Their reason... is that mailman2, and other lists, used to do it that way. I myself am agnostic about the difference, and probably prefer to keep things the way mailman3 recommends. The questions are: 1. What could be explained to the user to convince them the new method is preferable. 2. Can this be customized with a Setting? (Not that we will end up modifying it. Just curious). Thanks.

3 3

Mailman 3, postfix domains, and postgresql cannot authenticate user
by Keith Christian 05 Jul '24

05 Jul '24

--------------------------------------------------------------------------- First issue:Postfix v3.5.8 on RHEL 8. The below is from journalctl -xe: --------------------------------------------------------------------------- Jul 03 08:12:04 somehost.somedomain postfix/qmgr[947499]: error: open database /opt/mailman3/mm/var/data/postfix_domains.db: No such file or directory Jul 03 08:12:42 somehost.somedomain postfix/smtpd[947510]: error: open database /opt/mailman3/mm/var/data/postfix_domains.db: No such file or directory Jul 03 08:12:42 somehost.somedomain postfix/smtpd[947510]: error: open database /opt/mailman3/mm/var/data/postfix_lmtp.db: No such file or directory I manually created the var_dir: /opt/mailman3/mm/var directory specified in the mailman.cfg file. How do I find out how to populate the postfix_domains.db files, and the postfix_lmtp.db file? --------------------------------------------------------------------------- Second issue: Postgresql authentication. PostgreSQL v10.23. --------------------------------------------------------------------------- I have this in the active mailman3.cfg file, where xxxxxxxxxx is the same complex password in the mailman3 config file: url: postgresql://mailman3:xxxxxxxxxx@localhost/mailman3/mailman3 Same password when I created the mailman3 and mailman3web users: create role mailman3 with encrypted password 'xxxxxxxxxx '; create role mailman3web with encrypted password 'xxxxxxxxxx '; psycopg2.OperationalError: connection to server at "localhost" (127.0.0.1), port 5432 failed: FATAL: Ident authentication failed for user "mailman3" --------------------------------------------------------------------------- Contents of the "live" mailman.cfg file, from "mailman info" --------------------------------------------------------------------------- (venv) [mailman3@someserver ~]$ cat -n /opt/mailman3/var/etc/mailman.cfg 1 # /opt/mailman3/mailman.cfg 2 [paths.here] 3 var_dir: /opt/mailman3/mm/var 4 5 [mailman] 6 layout: here 7 # This address is the "site owner" address. Certain messages which must be 8 # delivered to a human, but which can't be delivered to a list owner (e.g. a 9 # bounce from a list owner), will be sent to this address. It should point to 10 # a human. 11 site_owner: user(a)example.com 12 13 [database] 14 class: mailman.database.postgresql.PostgreSQLDatabase 15 ##### url: postgresql://mailman:MYPASSWORD@localhost/mailman 16 # ---------------------------------------------------------------------------------- 17 # Changed: Fri Jun 28 11:56:02 EDT 2024 18 url: postgresql://mailman3:xxxxxxxx@localhost/mailman3/mailman3 19 # Changed: Fri Jun 28 11:56:02 EDT 2024 20 # ---------------------------------------------------------------------------------- 21 22 [archiver.prototype] 23 enable: yes 24 25 # For the HyperKitty archiver. 26 [archiver.hyperkitty] 27 class: mailman_hyperkitty.Archiver 28 enable: yes 29 configuration: /opt/mailman3/mailman-hyperkitty.cfg 30 31 [shell] 32 history_file: $var_dir/history.py 33 34 [mta] 35 incoming: mailman.mta.postfix.LMTP 36 outgoing: mailman.mta.deliver.deliver 37 lmtp_host: 127.0.0.1 38 lmtp_port: 8024 39 smtp_host: localhost 40 smtp_port: 25 41 configuration: python:mailman.config.postfix 42 verp_confirmations: yes 43 verp_personalized_deliveries: yes 44 verp_delivery_interval: 1 Suggestions appreciated for how to fix the postfix_domains files, and the postgres authentication isssue. ------------- Keith

3 5

Ubuntu 24.04 upgrade notes
by Andrew Hodgson 03 Jul '24

03 Jul '24

Hi. Just thought I would write up my installation experiences on Ubuntu 24.04 following this guide as much as possible: Virtualenv Installation - Mailman Suite 3.3 documentation (mailman3.org)<https://docs.mailman3.org/en/latest/install/virtualenv.html> Ubuntu packages: I'm trying not to install dev packages on the server if possible and managed to get away with the following packages: python3-psycopg2 sassc lynx python3-venv python3-certbot-nginx postgresql exim4-daemon-light Creating the venv: python3 -m venv --system-site-packages --symlinks venv This creates the venv referencing the system packages as well which is useful as some of the dependencies for Mailman are met by the Certbot and python3-psycopg2 packages. Using Pip I installed the following packages: pip install mailman mailman-hyperkitty mailman-web Postgres: I ran into an issue with the database after running `mailman-web migrate`. This was because a newer version of Django pulled in from the above command needed some tables creating in the public schema, and from Postgres 15 (Ubuntu ships with Postgres 16) that isn't allowed by default. I solved this by making the Mailman role owner of the mailman database. I am aware of discussions around whether its best to do this or give explicit perms on the databases to allow all on the public schema, but I decided to go with this option. My Postgres commands are now: sudo -u postgres psql create database mailman; create user mailman with encrypted password '[...]'; ALTER DATABASE mailman OWNER TO mailman; Django: I ran into an issue with a newer Django installation not allowing any form submission, was getting a 403 returned due to CSRF protection failure. I resolved this by adding this line to my local Settings.py: CSRF_TRUSTED_ORIGINS=['https://lists.domain.com'] I moved back to using Gunicorn for the backend server, the instructions on this were spot on. I've had this error before but the mailman-web compilemessages command failed because of missing msgfmt. I think I can pull this in from Ubuntu packages but may need to update installation docs with this extra package requirement. Search: I continued to use Woosh as I'm running a very low traffic site and that is what comes with Mailman-Web as part of package requirements. Thanks for the good support as always guys. Andrew.

1 0

Mailman Ubuntu 24.04 install notes
by Andrew Hodgson 01 Jul '24

01 Jul '24

Hi. Just thought I would write up my installation experiences on Ubuntu 24.04 following this guide as much as possible: Virtualenv Installation - Mailman Suite 3.3 documentation (mailman3.org)<https://docs.mailman3.org/en/latest/install/virtualenv.html> Ubuntu packages: I'm trying not to install dev packages on the server if possible and managed to get away with the following packages: python3-psycopg2 sassc lynx python3-venv python3-certbot-nginx postgresql exim4-daemon-light Creating the venv: python3 -m venv --system-site-packages --symlinks venv This creates the venv referencing the system packages as well which is useful as some of the dependencies for Mailman are met by the Certbot and python3-psycopg2 packages. Using Pip I installed the following packages: pip install mailman mailman-hyperkitty mailman-web Postgres: I ran into an issue with the database after running `mailman-web migrate`. This was because a newer version of Django pulled in from the above command needed some tables creating in the public schema, and from Postgres 15 (Ubuntu ships with Postgres 16) that isn't allowed by default. I solved this by making the Mailman role owner of the mailman database. I am aware of discussions around whether its best to do this or give explicit perms on the databases to allow all on the public schema, but I decided to go with this option. My Postgres commands are now: sudo -u postgres psql create database mailman; create user mailman with encrypted password '[...]'; ALTER DATABASE mailman OWNER TO mailman; Django: I ran into an issue with a newer Django installation not allowing any form submission, was getting a 403 returned due to CSRF protection failure. I resolved this by adding this line to my local Settings.py: CSRF_TRUSTED_ORIGINS=['https://lists.domain.com'] I moved back to using Gunicorn for the backend server, the instructions on this were spot on. I've had this error before but the mailman-web compilemessages command failed because of missing msgfmt. I think I can pull this in from Ubuntu packages but may need to update installation docs with this extra package requirement. Search: I continued to use Woosh as I'm running a very low traffic site and that is what comes with Mailman-Web as part of package requirements. Thanks for the good support as always guys. Andrew.

1 0

Held messages showing a reason not in English on the Postorious
by Alan KT So (ITSC) 27 Jun '24

27 Jun '24

We just notice a held message from a user's list showing the held reason in Ukrainian, but the interface is in US English and the list's preferred language is English (USA). We could not repeat the issue. What can be the reason behind? [cid:image001.png@01DAC7B9.AF2B0400] ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: image001.png Replaced multipart/alternative part with first alternative.

3 2

A poll: "Delete this thread"
by Sam Darwin 26 Jun '24

26 Jun '24

Hi, If you have a locally installed mailman3 instance, could you try (as an administrator) to visit a thread in Hyperkitty and click "Delete this thread"? Does the thread get deleted, and you return to "Recently active discussions" and everything is fine? Even just a few replies would indicate a pattern. That it's usually ok, for example. I get a spinning wheel, and "Thread matching query does not exist." HyperKitty==1.3.9, PostgreSQL, Ubuntu 22.04, memcached, gunicorn.

5 7

Problem with digests not being sent daily
by Mark 24 Jun '24

24 Jun '24

Hi, I have a problem with digests not being sent daily (periodically). Mailman 3.3.9 installed in virtualenv Enable Digests = YES Send Digest Periodically = YES Digest size threshold = 0 Cron is set and is running. An error report (below) is generated and emailed. The last line in the error report says "FileNotFoundError: [Errno 2] No such file or directory ...". Setting the list's folder to 777 didn't help. Any clues on where to start looking for a solution would be greatly appreciated. Thanks, Mark /////////////////////////////////////// CRON IS SET: /////////////////////////////////////// su mailman (venv) mailman:/$ crontab -e ... 5 6 * * * /opt/mailman/venv/bin/mailman digests --periodic ... /////////////////////////////////////// CRON RUNS: /////////////////////////////////////// # tail -100 /var/log/cron.log ... 2024-06-21T06:05:01.815948+00:00 deu CRON[3105464]: (mailman) CMD (/opt/mailman/venv/bin/mailman digests --periodic) ... /////////////////////////////////////// ERROR REPORT EMAILED BY CRON: /////////////////////////////////////// Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 8, in <module> sys.exit(main()) ^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/commands/cli_digests.py", line 126, in digests maybe_send_digest_now(mlist, force=True) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/app/digests.py", line 106, in maybe_send_digest_now os.rename(mailbox_path, mailbox_dest) FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/mm/var/lists/LISTNAME.LISTDOMAIN/digest.mmdf' -> '/opt/mailman/mm/var/lists/LISTNAME.LISTDOMAIN/digest.252.14.mmdf' ////////////////////////////////////////////////////////////////////////////////

3 14

unique key violation because of case sensitivity of email address
by Jörg Schulz 16 Jun '24

16 Jun '24

After updating to the most recent version of mailman-web, the migrations script issued an error: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount mailman-web | Running migrations: mailman-web | Traceback (most recent call last): mailman-web | File "/usr/lib/python3.12/site-packages/django/db/backends/utils.py", line 89, in _execute mailman-web | return self.cursor.execute(sql, params) mailman-web | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mailman-web | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "account_emailaddress_user_id_email_987c8728_uniq" mailman-web | DETAIL: Key (user_id, email)=(27, John.Doe(a)mynicedomain.de) already exists. mailman-web |(...) Digging into the db, I found the duplicate to result from a duplicate entry using john.doe(a)mynicedomain.de Obviously, the process that originally created the entry, took case sensitivity literally which doesn't seem to be the case any longer for the migration script. I fixed this by removing one of the duplicates hoping no further errors will come up.... What would have been the better approach?

2 1

running a script for the mailman-shell (to discard held messages) gives errors about a package argument for relative import
by stephan.edel＠gmx.net 16 Jun '24

16 Jun '24

Our organization has been been communicating through about a dozen mailing lists, running on mailman, with ~200 subscribers, for 15-odd years. After Debian 10 LTS ran out, I had to upgrade our organisations Debian server, forcing me into mailman3. I haven't managed, yet, to get the web interface running, so I need to do the most urgent tasks in the shell for now. I have found out that, for example, to discard all held messages from the list "motor", I need to > sudo -u list mailman shell -l motor.our-server.de and in the shell, >>> from mailman.app.moderator import handle_message >>> rdb = IListRequest(m) >>> for req in rdb.held_requests: ... if req.request_type == RequestType.held_message: ... handle_message(m, req.id, Action.discard) > commit() Now, obviously, I put these exact lines in a script, so I don't have to type it out every single time, and try to run it using > sudo -u list mailman shell -l testlist.our-server.de -r ./mm_discard_held.py but that invariably gives me ------------------------ Traceback (most recent call last): File "/usr/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.8', 'console_scripts', 'mailman')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/commands/cli_withlist.py", line 294, in shell r = call_name(dotted_name, m, *run_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 69, in call_name named_callable = find_name(dotted_name) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 52, in find_name module = import_module(module_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/__init__.py", line 121, in import_module raise TypeError(msg.format(name)) TypeError: the 'package' argument is required to perform a relative import for './mm_discard_held' ------------- I've tried a PYTHONPATH=/root/bin , where mm_discard_held.py lives, as suggested. I've tried moving it to the path where the mailman binary lives, as suggested. I always get the same result. What else can I do? Thank you.

2 1