November 2022 - Mailman-users - lists.mailman3.org

MTA authentication and smtpd_sender_login_maps
by Jan Eden 25 Nov '22

25 Nov '22

Hi, I installed Mailman3 successfully and would like to change the default smtp settings to an authenticated connection (smtp_user, smtp_pass). But my Postfix configuration restricts sending messages via the smtpd_sender_login_maps directive (which points to a hashed file I created years ago) and correctly refuses connections for messages with the test mailing list's From header. I obviously do not want to add an entry to the map file for each mailing list created by someone on the server. Is there a way to programmatically create a sender login map with Mailman? Thanks, Jan

2 2

Hyperkitty no longer archiving
by Hirayama, Pat 23 Nov '22

23 Nov '22

Greetings, A couple of months ago, I needed to move the Mailman3 server I inherited from one AWS account to another AWS account. This necessitated a change in the IP address of instance hosting the docker containers that comprise my mailman3 server. I thought I had updated the IP address wherever it needed to be, but I suppose I've missed something, because hyperkitty has not been archiving any messages. (Most of my lists don't archive, so it took a while before someone noticed). When I look at /opt/mailman/var/archives/hyperkitty/spool, I see messages dating back over the past couple of months waiting to be archived. So, they don't appear to be lost (and things are being mailed out normally). The thing is, the IP addresses of the docker containers haven't changed with respect to docker networking. docker networking ------------------------ docker host 172.31.0.1 mailman-core 172.31.0.3 mailman-nginx 172.31.0.5 mailman-postfix 172.31.0.4 mailman-django-uwsgi 172.31.0.2 settings_local.py: MAILMAN_ARCHIVER_FROM = ('172.18.8.14', '172.31.0.1', '172.31.0.2', '172.31.0.3', '172.31.0.4', '172.31.0.5', '127.0.0.1', '::1') During troubleshooting, I added 172.18.8.14 (new IP of the webserver), but that didn't seem to help. It seems like I added one of the docker addresses that was missing .... but again, that doesn't seem to have fixed the issue. mailmansuite.log is full of: ERROR 2022-11-22 00:25:40,365 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured ERROR 2022-11-22 00:25:40,399 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured Any suggestions as to where I might look for a solution, or what setting I need to change? Thanks! -p Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam(a)fredhutch.org<mailto:phirayam@fredhutch.org>

4 12

Upgrading a Ubuntu Mailman 3 Installation
by Allan Hansen 21 Nov '22

21 Nov '22

Dear Mailman 3 wizards, Several years ago I tried upgrading to Mailman 3 (I did install Mailman 2), but quickly ran into a thicket too complex for me to get out of. Two others - way more experienced than me at system-level manipulations - tried as well but did not succeed. Brian Carpenter of EMWD installed the system seemingly effortlessly, and seemingly overnight. We had a good relationship, and I was a beta site for his new product, SAAS. Brian also did the system MM3 upgrades as they appeared and would check that all was well a couple of times a year. Unfortunately, and I respect that, the current management at EMWD "don't provide working hands service for foreign Mailman 3 installations." I do not dare touch it, based on my earlier attempts with the installation. With Brian's passing I'm thus faced with a system that has no capable system support (I manage the lists, not the core). I have 50+ lists with 10000+ subscription total, and a lot of people depend on this tool. (It's a non-profit organization). So, I'm interesting in hiring someone for part-time, occasional work (a few times a year). Update MM3 as new versions are made available. Diagnose and fix the system if anything goes awry. I will pay you, as I did Brian, out of my own pocket, for your time. Yours, Allan Hansen

2 1

Migrating to mailman-web and Ubuntu 22.04
by Andrew Hodgson 16 Nov '22

16 Nov '22

Hi. For various reasons I want to migrate to a larger host running Ubuntu 22.04. I have set up the various packages ok and as part of the migration I want to use mailman-web instead of mailman-suite which is an older setup. I have instructions for Mailman-web for setting up Uwsgi but I would prefer to keep Gunicorn working as per the instructions here: DOC/Mailman 3 installation experience - Mailman Wiki (list.org)<https://wiki.list.org/DOC/Mailman%203%20installation%20experience> Has anyone done this before? Thanks. Andrew.

3 5

New Mailman Core Release
by Mark Sapiro 15 Nov '22

15 Nov '22

We are pleased to announce the release of Mailman core 3.3.7. It is published to PyPI and you can find links to all the PyPI projects: https://docs.mailman3.org/en/latest/ The change log is at https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… - please note the `BREAKING CHANGE` documented there and due to our now using SQLAlchemy >-1.4. Finally, the minimum Python version required is 3.7 although, 3.9 is recommended if you are doing a new install for longest support. 3.10 is supported for Django apps, but Mailman Core is missing that due to one of the core dependencies (aiosmtpd) not supporting 3.10 yet. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0

Do not display any lists on index the page (/postorius/lists/) by default (when you create the list it defaults to Hide) - Debian package install
by daniel.botting＠codethink.co.uk 11 Nov '22

11 Nov '22

Hi all, I am currently in the process of moving from mailman 2.1 to mailman 3, the basic functions are completed (mailing list and archiving). On our current mailman 2.1 install it advises there are no publicly advertised lists, when I visit mailman3-list.domain/postorius/lists/ I see all the lists that have been created, I require them not to be displayed. I know when you create a list you can on the 'Advertise this list' line select 'Hide this list in list index', but I would like it to default to this in case somebody forgets to select it. I am not a Python expert, though from the description at the top of the below file and the contents further down it looks like you are defining in base.py and then in default.py you are using what was defined, a class of 'Private' which should cause the lists not to be advertised, but they are? File: /usr/ib/python3/dist-packages/mailman/styles/base.py class Private: """Settings for private mailing lists.""" def apply(self, mailing_list): mlist = mailing_list mlist.advertised = False <-------------- mlist.archive_policy = ArchivePolicy.private mlist.subscription_policy = SubscriptionPolicy.confirm_then_moderate File: /usr/lib/python3/dist-packages/mailman/styles/default.py class PrivateDefaultStyle( Identity, BasicOperation, Bounces, Private, Discussion, Moderation): """Style for mailing-lists with private archives.""" name = 'private-default' description = _('Discussion mailing list style with private archives.') def apply(self, mailing_list): """See `IStyle`.""" Identity.apply(self, mailing_list) BasicOperation.apply(self, mailing_list) Bounces.apply(self, mailing_list) Private.apply(self, mailing_list) Discussion.apply(self, mailing_list) Moderation.apply(self, mailing_list) I added the below to /etc/mailman3/mailman.cfg: [styles] # The default style to apply if nothing else was requested. The value is the # name of an existing style. If no such style exists, no style will be # applied. default: private-default I took 'private-default' from /usr/lib/python3/dist-packages/mailman/styles/default.py, this means now when a new list is created the List style is 'Discussion mail list style with private archives so it is not displayed when you are logged out at /hyperkitty. Many thanks Daniel

1 0

Re: MM3.3 ignores
by Axel Rau 08 Nov '22

08 Nov '22

Am 08.11.2022 um 09:14 schrieb Axel Rau <Axel.Rau(a)chaos1.de>: > Am 08.11.2022 um 08:11 schrieb Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>>: > > > What is /var/templates now? I.e., is it a symlink to /usr/local/mailman3/var/templates? what does `ls -l /var/templates` (no slash at the end) show? > It does not exist. No directory, no link. Maybe I remember it wrong. Axel PS: Can I send posts with attachments to this list? My last posting didn’t make it. --- PGP-Key: CDE74120 ☀ computing @ chaos claudius --- PGP-Key: CDE74120 ☀ computing @ chaos claudius

2 1

MM3.3 ignores
by Axel Rau 08 Nov '22

08 Nov '22

Hi all, mm 3.3.5 started to complain: - - - File "/usr/local/py_venv/mailman_33/lib/python3.9/site-packages/mailman/config/config.py", line 142, in load self.push(filename, user_config.read()) File "/usr/local/py_venv/mailman_33/lib/python3.9/site-packages/mailman/config/config.py", line 148, in push self._post_process() File "/usr/local/py_venv/mailman_33/lib/python3.9/site-packages/mailman/config/config.py", line 160, in _post_process self.ensure_directories_exist() File "/usr/local/py_venv/mailman_33/lib/python3.9/site-packages/mailman/config/config.py", line 253, in ensure_directories_exist makedirs(directory) File "/usr/local/py_venv/mailman_33/lib/python3.9/site-packages/mailman/utilities/filesystem.py", line 69, in makedirs os.makedirs(path, mode) File "/usr/local/py_venv/mailman_33/lib/python3.9/os.py", line 225, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '//var/templates‘ - - - part from mailman.cfg: - - - [paths.dev] var_dir: $cfg_file/../.. [paths.here] var_dir: $cwd/var [paths.fhs] bin_dir: /usr/local/bin var_dir: /var/lib/mailman queue_dir: /var/spool/mailman log_dir: /var/log/mailman lock_dir: /var/spool/lock/mailman etc_dir: /usr/local/etc ext_dir: /usr/local/etc/mailman.d pid_file: /var/run/mailman/master.pid [paths.home] var_dir: /usr/local/mailman3 - - - var subdirectory: - - - (mailman_33) [root@lists5 /usr/local/py_venv/mailman_33]# ls -l /usr/local/mailman3/var/ total 62 drwxr-xr-x 3 mailman mailman 3 18 Jan. 2022 archives drwxr-xr-x 2 mailman mailman 2 18 Jan. 2022 cache drwxr-xr-x 2 mailman mailman 2 18 Jan. 2022 data drwxrwx--- 2 mailman mailman 4 2 Feb. 2022 etc drwxr-xr-x 6 mailman mailman 6 19 Jan. 2022 lists drwxr-xr-x 2 mailman mailman 2 5 Nov. 17:48 locks drwxr-xr-x 2 mailman mailman 7 18 Jan. 2022 logs drwxr-xr-x 155 mailman mailman 155 24 Okt. 09:09 messages drwxr-xr-x 3 root mailman 3 14 Juni 17:57 old_templates drwxr-xr-x 14 mailman mailman 14 18 Jan. 2022 queue drwxr-xr-x 3 mailman mailman 3 18 Jan. 2022 templates # ls -ld /usr/local/mailman3 drwxr-xr-x 11 mailman mailman 24 24 März 2022 /usr/local/mailman3 - - - I’m confused with the layout (and I dont like var stuff not in var). I have no layout set. Why does it not see /usr/local/mailman3/var/templates ? Any help apprecaiated Axel --- PGP-Key: CDE74120 ☀ computing @ chaos claudius

5 5

Imported archives MM2 to MM3
by Mark Dale 04 Nov '22

04 Nov '22

Hi I've just migrated a list from MM 2.1.39 to MM 3.3 (Debian). Following the guide at: https://docs.mailman3.org/en/latest/migration.html Importing the config went fine and all is well with that. The import of the archive appeared to go well (notes below) as did the rebuilding of the index. However, the archives show up empty on the list's webpage. I don't know what I should be looking for to fix this, so any pointers would be great. Thanks. //////////////////////////////////////// Migrate the list archives ========================= # python3 /usr/share/mailman3-web/manage.py hyperkitty_import -l LISTNAME@DOMAIN ./LISTNAME.mbox Importing from mbox file ./LISTNAME.mbox to LISTNAME@DOMAIN Computing thread structure Synchronizing properties with Mailman 33 emails left to refresh, checked 0 Warming up cache The full-text search index is not updated for this list. It will not be updated by the 'minutely' incremental update job. To update the index for this list, run the Django admin command with arguments 'update_index_one_list leden(a)vereniging.tropischebossen.nl'. //////////////////////////////////////// Rebuild the index: ======================== # python3 /usr/share/mailman3-web/manage.py update_index_one_list LISTNAME@DOMAIN Indexing 646 emails

1 1

Re: No DBA password when installing Mailman3 with remote database server
by Markus Ludwig Grandpré 04 Nov '22

04 Nov '22

Dear Mr. Turnbull, dear blackout69, thank you very much for your answers. Below this text you will find a brief documentation how I finally managed to install mailman3 and mailman3web packages under Debian 11 with a remote PostGreSQL server v10.22. Maybe there is a more elegant way to install both packages correctly, but AFAIK any solution will be only a workaround for the main problem, that the DBA's password is usually unknown during the whole installation process if you use a remote database server that is run by another staff team. Best regards, Markus ------------ 8< ------------ Mailman3 installation ===================== 1. Installation mailman3 2. Installation mailman3web 3. Integration in boot sequence 1. Installation mailman3 ------------------------ Once the remote database server has been set up with databases mailman3 and mailman3web and with a database user mailman3 that has access to both databases, create database mailman3; create database mailman3web; create user mailman3 with encrypted password 'streng_geheim'; grant all privileges on database mailman3 to mailman3; grant all privileges on database mailman3web to mailman3; the package mailman3 can be installed: # apt install mailman3 To avoid the DBA password prompt that ususally cannot be answered if the database server is run by another team, the installation tool dbconfig-common should be omitted: * Configure database for mailman3 with dbconfig-common? No The mailman3 package will be installed without errors. The configuration of the database connection can be done in file /etc/mailman3/mailman.cfg: ... class: mailman.database.postgresql.PostgreSQLDatabase ... url: postgres://mailman3:<secret>@<host>:<port>/mailman3 At this point the required tables have not yet been created in the database. Therefore, the mailman3 package should be configured again: # dpkg-reconfigure mailman3 Because the database mailman3 already exists on the database server, it does not need to be recreated using dbconfig-common: * Reinstall database for mailman3? No * Add the HyperKitty configuration to mailman.cfg? No Also moving the HyperKitty configuration to /etc/mailman3/mailman.cfg is not necessary at this time. After dpkg-reconfigure has finished, all necessary tables do now exist in the database mailman3: mailman3=> \dt List of relations Schema | Name | Type | Owner --------+--------------------+-------+---------- public | _request | table | mailman3 public | acceptablealias | table | mailman3 public | address | table | mailman3 public | alembic_version | table | mailman3 ... An initial start of the service mailman3 shows no errors: # systemctl start mailman3 # systemctl status mailman3 ● mailman3.service - mailman3 server. Loaded: loaded (/lib/system/system/mailman3.service; enabled; vendor default: enabled) Active: active (running) since Thu 2022-11-03 10:10:29 CET; 11min ago. ... 2. Installing mailman3-web -------------------------- Once the mailman3 package is installed, package mailman3-web can be installed: # apt install mailman3-web This time you should not do the installation process without the help of the installation tool dbconfig-common: * Configure database for mailman3 with dbconfig-common? Yes * Database type to be used by mailman3-web: pgsql * Host name of the PostgreSQL database server for mailman3-web: new host * Host running the PostgreSQL server for mailman3-web: <remote host> * PostgreSQL application password for mailman3-web: <secret> * Password confirmation: <secret> Even if the DBA password prompt cannot be answered: Create configuration file /etc/dbconfig-common/mailman3-web.conf with new version Password for user postgres: ?????? the dbconfig-common tool should be used again: * Next step for database installation: retry * Configure database for mailman3-web with dbconfig-common? Yes * Connection method for PostgreSQL database of mailman3-web: TCP/IP * Host name of the PostgreSQL database server for mailman3-web: <remote host> * Port number for the PostgreSQL service: 5432 * Method for authenticating the PostgreSQL administrator: password * Method for authenticating PostgreSQL user: password * PostgreSQL database name for mailman3-web: mailman3web * PostgreSQL username for mailman3-web: mailman3 * PostgreSQL application password for mailman3-web: <secret> * Password confirmation: <secret> * Name of the database's administrative user: postgres * Password of your database's administrative user: sldjkfhskldfjhyxlökcjy The prompt for the DBA password cannot be answered correctly again, therefore "ignore" should now be the best choice for the further course of the installation process: * Next step for database installation: ignore * Password of your database's administrative user: sldjkfhskldfjhyxlkcylö An error does occur now because the database user mailman3 can not be created: ... Password for user postgres: unable to connect to postgresql server. error encountered creating user: psql: error: fe_sendauth: no password supplied ... But this error doesn't matter, because the database user mailman3 does already exist and the installation process of the package mailman3web will nevertheless be finished completely. A look into file /etc/mailman3/mailman-web.py shows that dbconfig-common has entered the database connection to the database server correctly, too: DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'mailman3web', 'USER': 'mailman3', 'PASSWORD': '<secret>', 'HOST': '<remote host>', 'PORT': '', <--- missing? 'OPTIONS': { }, } } Also all required tables have been created in the database mailman3web during the installation process: mailman3web=> \dt List of relations Schema | Name | Type | Owner --------+-----------------------------------+-------+---------- public | account_emailaddress | table | mailman3 public | account_emailconfirmation | table | mailman3 public | auth_group | table | mailman3 public | auth_group_permissions | table | mailman3 public | auth_permission | table | mailman3 ... The initial start of the service mailman3-web shows no errors: # systemctl start mailman3-web # systemctl status mailman3-web ● mailman3-web.service - Mailman3-web uWSGI service. Loaded: loaded (/lib/system/system/mailman3-web.service; enabled; vendor default: enabled). Active: active (running) since Thu 2022-11-03 10:42:30 CET; 11min ago ... 3. Intergration in boot sequence -------------------------------- Finally, the two services mailman3 and mailman3-web can be included in the boot sequence: # systemctl enable mailman3 Synchronize the state of mailman3.service with the SysV service script with /lib/systemd/systemd-sysv-install. Runs: /lib/systemd/systemd-sysv-install enable mailman3 # systemctl enable mailman3-web Synchronize the status of mailman3-web.service with the SysV service script using /lib/systemd/systemd-sysv-install. Runs: /lib/systemd/systemd-sysv-install enable mailman3-web ------------ >8 ------------ On 01.11.22 08:10, Stephen J. Turnbull wrote: > blackout69 writes: > > > For mailman3 > > sudo dpkg-reconfigure mailman3 > > > > For mailman3-web > > sudo dpkg-reconfigure mailman3-web > > I guess those will work too as long as the users and databases have > been created. But they'll still fail at the first step without that > (and they might fail anyway if checking for the existence of those > users and databases involves logging in to the database as the admin > user). > > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ -- Markus Ludwig Grandpré Universität Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Abteilung IT-Dienste Forschung und Lehre, Tel: ++49 7531 88 4342

3 2