- Mailman-users - lists.mailman3.org

Mailman 3, postfix domains, and postgresql cannot authenticate user
by Keith Christian 03 Jul '24

03 Jul '24

--------------------------------------------------------------------------- First issue:Postfix v3.5.8 on RHEL 8. The below is from journalctl -xe: --------------------------------------------------------------------------- Jul 03 08:12:04 somehost.somedomain postfix/qmgr[947499]: error: open database /opt/mailman3/mm/var/data/postfix_domains.db: No such file or directory Jul 03 08:12:42 somehost.somedomain postfix/smtpd[947510]: error: open database /opt/mailman3/mm/var/data/postfix_domains.db: No such file or directory Jul 03 08:12:42 somehost.somedomain postfix/smtpd[947510]: error: open database /opt/mailman3/mm/var/data/postfix_lmtp.db: No such file or directory I manually created the var_dir: /opt/mailman3/mm/var directory specified in the mailman.cfg file. How do I find out how to populate the postfix_domains.db files, and the postfix_lmtp.db file? --------------------------------------------------------------------------- Second issue: Postgresql authentication. PostgreSQL v10.23. --------------------------------------------------------------------------- I have this in the active mailman3.cfg file, where xxxxxxxxxx is the same complex password in the mailman3 config file: url: postgresql://mailman3:xxxxxxxxxx@localhost/mailman3/mailman3 Same password when I created the mailman3 and mailman3web users: create role mailman3 with encrypted password 'xxxxxxxxxx '; create role mailman3web with encrypted password 'xxxxxxxxxx '; psycopg2.OperationalError: connection to server at "localhost" (127.0.0.1), port 5432 failed: FATAL: Ident authentication failed for user "mailman3" --------------------------------------------------------------------------- Contents of the "live" mailman.cfg file, from "mailman info" --------------------------------------------------------------------------- (venv) [mailman3@someserver ~]$ cat -n /opt/mailman3/var/etc/mailman.cfg 1 # /opt/mailman3/mailman.cfg 2 [paths.here] 3 var_dir: /opt/mailman3/mm/var 4 5 [mailman] 6 layout: here 7 # This address is the "site owner" address. Certain messages which must be 8 # delivered to a human, but which can't be delivered to a list owner (e.g. a 9 # bounce from a list owner), will be sent to this address. It should point to 10 # a human. 11 site_owner: user(a)example.com 12 13 [database] 14 class: mailman.database.postgresql.PostgreSQLDatabase 15 ##### url: postgresql://mailman:MYPASSWORD@localhost/mailman 16 # ---------------------------------------------------------------------------------- 17 # Changed: Fri Jun 28 11:56:02 EDT 2024 18 url: postgresql://mailman3:xxxxxxxx@localhost/mailman3/mailman3 19 # Changed: Fri Jun 28 11:56:02 EDT 2024 20 # ---------------------------------------------------------------------------------- 21 22 [archiver.prototype] 23 enable: yes 24 25 # For the HyperKitty archiver. 26 [archiver.hyperkitty] 27 class: mailman_hyperkitty.Archiver 28 enable: yes 29 configuration: /opt/mailman3/mailman-hyperkitty.cfg 30 31 [shell] 32 history_file: $var_dir/history.py 33 34 [mta] 35 incoming: mailman.mta.postfix.LMTP 36 outgoing: mailman.mta.deliver.deliver 37 lmtp_host: 127.0.0.1 38 lmtp_port: 8024 39 smtp_host: localhost 40 smtp_port: 25 41 configuration: python:mailman.config.postfix 42 verp_confirmations: yes 43 verp_personalized_deliveries: yes 44 verp_delivery_interval: 1 Suggestions appreciated for how to fix the postfix_domains files, and the postgres authentication isssue. ------------- Keith

3 2

Mailman3 on AlmaLinux 9 - Python3.9 errors
by willie.castillo＠caemilusa.com 01 Jul '24

01 Jul '24

Hi, I'm trying to configure mailman3 on my AlmaLinux 9 server and when checking the mailman3 status, I get the following: # mailman3 status Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 227, in _catch_revision_errors yield File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 439, in _upgrade_revs for script in reversed(list(revs)) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 793, in iterate_revisions revisions, heads = fn( File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 1420, in _collect_upgrade_revisions current_revisions = self.get_revisions(lower) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 525, in get_revisions return sum([self.get_revisions(id_elem) for id_elem in id_], ()) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 525, in <listcomp> return sum([self.get_revisions(id_elem) for id_elem in id_], ()) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 549, in get_revisions return tuple( File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 550, in <genexpr> self._revision_for_ident(rev_id, branch_label) File "/usr/lib/python3.9/site-packages/alembic/script/revision.py", line 619, in _revision_for_ident raise ResolutionError( alembic.script.revision.ResolutionError: No such revision or branch '2156fc3f6f7d' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/libexec/mailman3/mailman", line 8, in <module> sys.exit(main()) File "/usr/lib/python3.9/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3.9/site-packages/click/core.py", line 1052, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/lib/python3.9/site-packages/click/core.py", line 914, in make_context self.parse_args(ctx, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 1615, in parse_args rest = super().parse_args(ctx, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 1370, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/lib/python3.9/site-packages/click/core.py", line 2347, in handle_parse_result value = self.process_value(ctx, value) File "/usr/lib/python3.9/site-packages/click/core.py", line 2309, in process_value value = self.callback(ctx, self, value) File "/usr/lib/python3.9/site-packages/mailman/bin/mailman.py", line 95, in initialize_config initialize(value) File "/usr/lib/python3.9/site-packages/mailman/core/initialize.py", line 229, in initialize initialize_2(propagate_logs=propagate_logs) File "/usr/lib/python3.9/site-packages/mailman/core/initialize.py", line 187, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() File "/usr/lib/python3.9/site-packages/mailman/database/factory.py", line 58, in create SchemaManager(database).setup_database() File "/usr/lib/python3.9/site-packages/mailman/database/factory.py", line 109, in setup_database alembic.command.upgrade(alembic_cfg, 'head') File "/usr/lib/python3.9/site-packages/alembic/command.py", line 320, in upgrade script.run_env() File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 563, in run_env util.load_python_file(self.dir, "env.py") File "/usr/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file module = load_module_py(module_id, path) File "/usr/lib/python3.9/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py spec.loader.exec_module(module) # type: ignore File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/usr/lib/python3.9/site-packages/mailman/database/alembic/env.py", line 77, in <module> run_migrations_online() File "/usr/lib/python3.9/site-packages/mailman/database/alembic/env.py", line 71, in run_migrations_online context.run_migrations() File "<string>", line 8, in run_migrations File "/usr/lib/python3.9/site-packages/alembic/runtime/environment.py", line 851, in run_migrations self.get_context().run_migrations(**kw) File "/usr/lib/python3.9/site-packages/alembic/runtime/migration.py", line 608, in run_migrations for step in self._migrations_fn(heads, self): File "/usr/lib/python3.9/site-packages/alembic/command.py", line 309, in upgrade return script._upgrade_revs(revision, rev) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 435, in _upgrade_revs return [ File "/usr/lib64/python3.9/contextlib.py", line 137, in __exit__ self.gen.throw(typ, value, traceback) File "/usr/lib/python3.9/site-packages/alembic/script/base.py", line 259, in _catch_revision_errors raise util.CommandError(resolution) from re alembic.util.exc.CommandError: Can't locate revision identified by '2156fc3f6f7d' Any ideas why?

2 1

Mailman Ubuntu 24.04 install notes
by Andrew Hodgson 01 Jul '24

01 Jul '24

Hi. Just thought I would write up my installation experiences on Ubuntu 24.04 following this guide as much as possible: Virtualenv Installation - Mailman Suite 3.3 documentation (mailman3.org)<https://docs.mailman3.org/en/latest/install/virtualenv.html> Ubuntu packages: I'm trying not to install dev packages on the server if possible and managed to get away with the following packages: python3-psycopg2 sassc lynx python3-venv python3-certbot-nginx postgresql exim4-daemon-light Creating the venv: python3 -m venv --system-site-packages --symlinks venv This creates the venv referencing the system packages as well which is useful as some of the dependencies for Mailman are met by the Certbot and python3-psycopg2 packages. Using Pip I installed the following packages: pip install mailman mailman-hyperkitty mailman-web Postgres: I ran into an issue with the database after running `mailman-web migrate`. This was because a newer version of Django pulled in from the above command needed some tables creating in the public schema, and from Postgres 15 (Ubuntu ships with Postgres 16) that isn't allowed by default. I solved this by making the Mailman role owner of the mailman database. I am aware of discussions around whether its best to do this or give explicit perms on the databases to allow all on the public schema, but I decided to go with this option. My Postgres commands are now: sudo -u postgres psql create database mailman; create user mailman with encrypted password '[...]'; ALTER DATABASE mailman OWNER TO mailman; Django: I ran into an issue with a newer Django installation not allowing any form submission, was getting a 403 returned due to CSRF protection failure. I resolved this by adding this line to my local Settings.py: CSRF_TRUSTED_ORIGINS=['https://lists.domain.com'] I moved back to using Gunicorn for the backend server, the instructions on this were spot on. I've had this error before but the mailman-web compilemessages command failed because of missing msgfmt. I think I can pull this in from Ubuntu packages but may need to update installation docs with this extra package requirement. Search: I continued to use Woosh as I'm running a very low traffic site and that is what comes with Mailman-Web as part of package requirements. Thanks for the good support as always guys. Andrew.

1 0

security app causing django Internal Server Error: /archives/search emails
by sun zi 28 Jun '24

28 Jun '24

Greetings, I have set up a few lists and all seems to be working as expected, however I get a large number of emails due to internal server errors received when my security application is testing exploits to list URLs such as the following: https://<mylistdomain>/archives/search?sort=177120&cauft=&page=1&q=../../../../../../../../../../tmb/wditeteet424465304.txt. Is there any django logging filter changes that can be made to avoid receiving these emails? Any assistance would be greatly appreciated. Thank you!

3 2

Held messages showing a reason not in English on the Postorious
by Alan KT So (ITSC) 27 Jun '24

27 Jun '24

We just notice a held message from a user's list showing the held reason in Ukrainian, but the interface is in US English and the list's preferred language is English (USA). We could not repeat the issue. What can be the reason behind? [cid:image001.png@01DAC7B9.AF2B0400] ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: image001.png Replaced multipart/alternative part with first alternative.

3 2

A poll: "Delete this thread"
by Sam Darwin 26 Jun '24

26 Jun '24

Hi, If you have a locally installed mailman3 instance, could you try (as an administrator) to visit a thread in Hyperkitty and click "Delete this thread"? Does the thread get deleted, and you return to "Recently active discussions" and everything is fine? Even just a few replies would indicate a pattern. That it's usually ok, for example. I get a spinning wheel, and "Thread matching query does not exist." HyperKitty==1.3.9, PostgreSQL, Ubuntu 22.04, memcached, gunicorn.

5 7

Problem with digests not being sent daily
by Mark 24 Jun '24

24 Jun '24

Hi, I have a problem with digests not being sent daily (periodically). Mailman 3.3.9 installed in virtualenv Enable Digests = YES Send Digest Periodically = YES Digest size threshold = 0 Cron is set and is running. An error report (below) is generated and emailed. The last line in the error report says "FileNotFoundError: [Errno 2] No such file or directory ...". Setting the list's folder to 777 didn't help. Any clues on where to start looking for a solution would be greatly appreciated. Thanks, Mark /////////////////////////////////////// CRON IS SET: /////////////////////////////////////// su mailman (venv) mailman:/$ crontab -e ... 5 6 * * * /opt/mailman/venv/bin/mailman digests --periodic ... /////////////////////////////////////// CRON RUNS: /////////////////////////////////////// # tail -100 /var/log/cron.log ... 2024-06-21T06:05:01.815948+00:00 deu CRON[3105464]: (mailman) CMD (/opt/mailman/venv/bin/mailman digests --periodic) ... /////////////////////////////////////// ERROR REPORT EMAILED BY CRON: /////////////////////////////////////// Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 8, in <module> sys.exit(main()) ^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/commands/cli_digests.py", line 126, in digests maybe_send_digest_now(mlist, force=True) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/app/digests.py", line 106, in maybe_send_digest_now os.rename(mailbox_path, mailbox_dest) FileNotFoundError: [Errno 2] No such file or directory: '/opt/mailman/mm/var/lists/LISTNAME.LISTDOMAIN/digest.mmdf' -> '/opt/mailman/mm/var/lists/LISTNAME.LISTDOMAIN/digest.252.14.mmdf' ////////////////////////////////////////////////////////////////////////////////

3 14

unique key violation because of case sensitivity of email address
by Jörg Schulz 16 Jun '24

16 Jun '24

After updating to the most recent version of mailman-web, the migrations script issued an error: Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount mailman-web | Running migrations: mailman-web | Traceback (most recent call last): mailman-web | File "/usr/lib/python3.12/site-packages/django/db/backends/utils.py", line 89, in _execute mailman-web | return self.cursor.execute(sql, params) mailman-web | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mailman-web | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "account_emailaddress_user_id_email_987c8728_uniq" mailman-web | DETAIL: Key (user_id, email)=(27, John.Doe(a)mynicedomain.de) already exists. mailman-web |(...) Digging into the db, I found the duplicate to result from a duplicate entry using john.doe(a)mynicedomain.de Obviously, the process that originally created the entry, took case sensitivity literally which doesn't seem to be the case any longer for the migration script. I fixed this by removing one of the duplicates hoping no further errors will come up.... What would have been the better approach?

2 1

running a script for the mailman-shell (to discard held messages) gives errors about a package argument for relative import
by stephan.edel＠gmx.net 16 Jun '24

16 Jun '24

Our organization has been been communicating through about a dozen mailing lists, running on mailman, with ~200 subscribers, for 15-odd years. After Debian 10 LTS ran out, I had to upgrade our organisations Debian server, forcing me into mailman3. I haven't managed, yet, to get the web interface running, so I need to do the most urgent tasks in the shell for now. I have found out that, for example, to discard all held messages from the list "motor", I need to > sudo -u list mailman shell -l motor.our-server.de and in the shell, >>> from mailman.app.moderator import handle_message >>> rdb = IListRequest(m) >>> for req in rdb.held_requests: ... if req.request_type == RequestType.held_message: ... handle_message(m, req.id, Action.discard) > commit() Now, obviously, I put these exact lines in a script, so I don't have to type it out every single time, and try to run it using > sudo -u list mailman shell -l testlist.our-server.de -r ./mm_discard_held.py but that invariably gives me ------------------------ Traceback (most recent call last): File "/usr/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.8', 'console_scripts', 'mailman')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/commands/cli_withlist.py", line 294, in shell r = call_name(dotted_name, m, *run_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 69, in call_name named_callable = find_name(dotted_name) ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 52, in find_name module = import_module(module_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/__init__.py", line 121, in import_module raise TypeError(msg.format(name)) TypeError: the 'package' argument is required to perform a relative import for './mm_discard_held' ------------- I've tried a PYTHONPATH=/root/bin , where mm_discard_held.py lives, as suggested. I've tried moving it to the path where the mailman binary lives, as suggested. I always get the same result. What else can I do? Thank you.

2 1

mailman3 on Rocky 9 is not starting after an OS update and reboot
by dgoldsmith＠sans.org 12 Jun '24

12 Jun '24

We are running mailman 3.x in a venv setup using Postgresql. The mailman service hangs during startup and eventually fails with a timeout: # systemctl status mailman3 × mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; preset: disabled) Active: failed (Result: timeout) since Wed 2024-06-12 13:35:23 UTC; 11s ago Process: 1205 ExecStart=/opt/mailman/venv/bin/mailman start (code=killed, signal=TERM) CPU: 1.019s Jun 12 13:33:53 <hostname> systemd[1]: Starting GNU Mailing List Manager... Jun 12 13:35:23 <hostname>systemd[1]: mailman3.service: start operation timed out. Terminating. Jun 12 13:35:23 <hostname> systemd[1]: mailman3.service: Failed with result 'timeout'. Jun 12 13:35:23 <hostname> systemd[1]: Failed to start GNU Mailing List Manager. Jun 12 13:35:23 <hostname> systemd[1]: mailman3.service: Consumed 1.019s CPU time. Searching I found some similar reports where it seemed to be an issue with a stale .lck lock file. But on our system: - /var/lib/mailman is empty - /opt/mailman/mm/var/locks is empty The last entries in /opt/mailman/mm/var/logs/mailman.log is: 2024-06-12 09:23:44 -0400] [2101411] [INFO] Handling signal: term Jun 12 13:23:44 2024 (2101395) Master watcher caught SIGTERM. Exiting. [2024-06-12 13:23:44 +0000] [2101451] [INFO] Worker exiting (pid: 2101451) Jun 12 09:23:44 2024 (2101412) retry runner caught SIGTERM. Stopping. [2024-06-12 13:23:44 +0000] [2101450] [INFO] Worker exiting (pid: 2101450) Jun 12 09:23:44 2024 (2101408) nntp runner caught SIGTERM. Stopping. [2024-06-12 09:23:48 -0400] [2101411] [INFO] Shutting down: Master Jun 12 13:23:50 2024 (2101395) Master stopped Any help would be appreciated. Postgres appears to be up., Postfix is up, nginx is up. The web UI gives me the error: Something went wrong Mailman REST API not available. Please start Mailman core. Running "mailman version" hangs for a long time and then returns: $ mailman version GNU Mailman 3.3.9 (Tom Sawyer) Running "mailman status" hangs for a long time and then returns: $ mailman status GNU Mailman is not running What other logs can be checked for more detail? What possible services or file/dir permissions may have had something changed by the OS updates? Thanks, David Goldsmith

3 2