- Mailman-users - lists.mailman3.org

Hyperkitty performance problem
by Tobias Diekershoff 06 Nov '24

06 Nov '24

Hey all, last week we caught some AI bots causing high LOAD on our mailman3 servers archive, that we blocked with nginx rules. I'm not sure if it is related to this incident, but we noticed this when analyzing the log files at the time. In the mailman.log hyperkitty is listed 10k times and more to archive a single mail over the time of 2h, seemingly all with the same process ID > (1186) HyperKitty archived message [...] The mailman3-web.log has matching log entries that the mail was archived > hyperkitty.views.mailman Archived message every 0.5 seconds. In the web interface to the archive, the mail is only listed once though. My suspicion is, that these archiving processes are now overloading the system, preventing a timely catching up work of the queued messages. I could not find anything in the Mailman3 nor Hyperkitty documentation to better distribute the system resources to the Mailman runner or prevent Hyperkitty to restart archiving a mail over and over again. Any hint about where to look for the reason of this is highly appreciated. Best, Tobias -- Tobias Diekershoff >>> System Hacker Free Software Foundation Europe Schönhauser Allee 6/7, 10119 Berlin, Germany | t +49-30-27595290 Registered at Amtsgericht Hamburg, VR 17030 | fsfe.org/support OpenPGP-Key ID ... 0x25FE376FF17694A1 Fingerprint ...... 23EE F484 FDF8 291C BA09 A406 25FE 376F F176 94A1

1 0

Where to find disabled members?
by Bill Oliver 05 Nov '24

05 Nov '24

Now that I have mailman running smoothly, I'm having to deal with a few other issues. When I changed vendors for my vps, my vendor gave me a static IP address that was apparently run by Satan himself. The ip address is -- or was -- on a number of blacklists. I have spent the past couple of weeks going through various appeals processes to get whitelisted, and I think I'm in good shape. In the meantime, a fair number of subscribers have been disabled. When I look at Postorius -> Settings -> Bounce processing, I see all of the policies I can play with, but I don't see how to a) See who is disabled, and b) un-disable them by hand or... do I just have to wait seven days for them to automatically get un-disabled? Thanks! billo

2 2

member appears twice in list - how?
by Jose Bernal 05 Nov '24

05 Nov '24

I've got a list member showing up twice in a list (same email address). Can't use the GUI to remove the member from this list (get a 503 error) Screenshots attached, underlying database query reveals no duplicates. Wondering how this is possible or where else I should be looking to resolve this duplicate before I simply remove their entry from the members table altogether. Thank you, --------------------- José Bernal Web/Application Engineer Cell 541-228-8481 jbernal.web.dev(a)gmail.com ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: Screen Shot 2024-11-05 at 10.53.55 AM.png Content-Type: image/png Name: Screen Shot 2024-11-05 at 10.59.12 AM.png Replaced multipart/alternative part with first alternative.

2 1

"held for moderation" - alias domain details sought
by Greg Newby 05 Nov '24

05 Nov '24

Hi friends. I've viewed this old thread: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… and documentation here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h… (and the Postfix docs it links to). Those documents don't seem to guide my situation, or maybe I just need a more specific example. The problem is that when messages are delivered to the alias domain, they are held for moderation. I.e., mail to the alias domain like testlist(a)domain.tld is held for moderation, but mail from the same subscribed, non-moderated user to testlist(a)lists.domain.tld is posted immediately without moderation. I wonder whether my main domain needs to be @domain.tld and my alias domain needs to be @lists.domain.tld - is that what I should try? Or, am I not setting up the alias domain correctly? Details: I'm writing to ask for advice so that mailman messages to address(a)domain.tld can also be sent to address(a)lists.domain.tld. There are various other non-Mailman addresses (local Unix accounts and aliases) handled by @domain.tld as well as @otherdomain.tld on the same server. I have a running 3.3.10 server (venv, using Postfix and PostgreSQL under Ubuntu 24.04LTS). The configured domain is of the form: lists.domain.tld and it's working ok. I put this in /etc/aliases and it seems to work, to forward messages from testlist(a)domain.tld to testlist(a)lists.domain.tld: testlist: testlist(a)lists.domain.tld I know it works because I see it in MM3's smtp.log and mailman.log, including a line like this in mailman.log: Oct 12 21:12:27 2024 (2560960) HOLD: testlist(a)lists.domain.tld post from gbnewby(a)petascale.org held, message-id=< CANLuyJZd8NNn5rOLe3XUhooG9F33U_7VOQXrS-7Z241koAV_Ww(a)mail.otherdomain.com>: Message has implicit destination And, indeed, the message is held for moderation. In the MM3 configuration panel for Domains ( https://lists.domain.tld/mailman3/domains/), I have put "domain.tld" in the "Alias domain" field. Thanks for any guidance. I appreciated seeing the recent threat about "redirecting -bounces replies" which also addresses alias domains. ~ Greg

3 4

HSTS Missing from HTTPS Server
by jkoral＠luc.edu 05 Nov '24

05 Nov '24

In my configuration, I am using Apache to reverse proxy the requests to a uwsgi. After our security office scanned the server, it was shown that HSTS is missing from the HTTPS server. The instructions say to add some lines to /etc/apache2/sites-enabled/. The only file in there is 000-default.conf that holds the configuration, but the VirtualHost is port 80. I am using an F5 to to automatically redirect port 80 to port 443, but it is still showing this vulnerability. My question is, how can enable the Virtual Host for 443 on the server? Or can that configuration be located somewhere else? Or can I just create a new Virtual Host with the added configuration for mailman? Thanks, Joe Koral

2 1

The dread "relay access denied" error
by Bill Oliver 05 Nov '24

05 Nov '24

I've been trying to avoid asking this question since "relay access denied" problems are usually pretty silly configuration errors that I should pick up. But I've been staring at this for three days and I'm stumped. Once again here's the scenario. I set up mailman3 on Ubuntu 24.04 and it's running without a hitch, particularly now that I added the appropriate archival commands. I decided to create a backup on an older server of mine running Ubuntu 22.04. The setup is almost identical except for domain names. Unfortunately, in the second setup, I'm getting "relay access denied" to senders/recipients outside the host domain. Things go fine to list members on the local machine. Mailman is successfully sending admin mail to mailman(a)bill-oliver.com Mail not using mailman works fine. The domain where everything works is billoblog.com. The domain I'm setting up is bill-oliver.com Here's the error in syslog (mirrored in mail.log) on bill-oliver.com trying to talk to list members oliver(a)billoblog.com and billo(a)billoblog.com:: Nov 1 21:24:52 mail postfix/smtpd[10193]: NOQUEUE: reject: RCPT from 162-144-108-19.bluehost.com[162.144.108.19]: 454 4.7.1 <oliver(a)billoblog.com>: Relay access denied; from=<tradfp-bounces+oliver=billoblog.com(a)bill-oliver.com> to=<oliver(a)billoblog.com> proto=ESMTP helo=<162-144-108- 19.bluehost.com> Nov 1 21:24:52 mail postfix/smtpd[10193]: disconnect from 162-144-108- 19.bluehost.com[162.144.108.19] ehlo=1 mail=2 rcpt=1/2 data=1 rset=1 quit=1 commands=7/8 Nov 1 21:24:52 mail postfix/smtpd[10193]: connect from 162-144-108- 19.bluehost.com[162.144.108.19] Nov 1 21:24:52 mail postfix/smtpd[10193]: NOQUEUE: reject: RCPT from 162-144-108-19.bluehost.com[162.144.108.19]: 454 4.7.1 <billo(a)billoblog.com>: Relay access denied; from=<tradfp-bounces+billo=billoblog.com(a)bill-oliver.com> to=<billo(a)billoblog.com> proto=ESMTP helo=<162-144-108-19.bluehost.com> Nov 1 21:24:52 mail postfix/smtpd[10193]: disconnect from 162-144-108- 19.bluehost.com[162.144.108.19] ehlo=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=4/5 Nov 1 21:25:48 mail dovecot: auth-worker(10954): Debug: conn unix:auth- worker (pid=10903,uid=116): Disconnected: Connection closed (fd=-1) Nov 1 21:25:48 mail dovecot: auth-worker(10954): Debug: mysql(localhost): Connection finished (queries=1, slow queries=0) Nov 1 21:27:28 mail dovecot: auth: Debug: mysql: Connection finished (queries=0, slow queries=0) FYI, 162-144-108-19.bluehost.com is my ip address and bluehost.com is the VPS vendor. I don't know why that pops up, since the domain is bill-oliver.com. I assume it's VPS-related magic. Here are similar errors in /opt/mailman/mm/var/logs/smtp.log: Nov 01 21:22:46 2024 (1573) Available AUTH mechanisms: LOGIN(builtin) PLAIN(builtin) Nov 01 21:22:46 2024 (1573) Peer: ('162.144.108.19', 51476) Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) handling connection Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) >> b'LHLO mail.bill-oliver.com' Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) >> b'MAIL FROM:<billo(a)billoblog.com> SIZE=1486 BODY=8BITMIME' Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) sender: billo(a)billoblog.com Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) >> b'RCPT TO:<tradfp(a)lists.bill-oliver.com>' Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) recip: tradfp(a)lists.bill-oliver.com Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) >> b'DATA' Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) >> b'QUIT' Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) connection lost Nov 01 21:22:46 2024 (1573) ('162.144.108.19', 51476) Connection lost during _handle_client() Nov 01 21:22:48 2024 (1574) <cbacd3866ebdfeeb706993996ba84e67f7db5ad9.camel(a)billoblog.com> recipients refused: {'oliver(a)billoblog.com': (454, b'4.7.1 <oliver(a)billoblog.com>: Relay access denied')} Nov 01 21:22:48 2024 (1574) <cbacd3866ebdfeeb706993996ba84e67f7db5ad9.camel(a)billoblog.com> recipients refused: {'billo(a)billoblog.com': (454, b'4.7.1 <billo(a)billoblog.com>: Relay access denied')} Nov 01 21:22:48 2024 (1574) <cbacd3866ebdfeeb706993996ba84e67f7db5ad9.camel(a)billoblog.com> smtp to tradfp(a)bill-oliver.com for 2 recips, completed in 0.27154111862182617 seconds Nov 01 21:22:48 2024 (1574) <cbacd3866ebdfeeb706993996ba84e67f7db5ad9.camel(a)billoblog.com> post to tradfp(a)bill-oliver.com from billo(a)billoblog.com, 1448 bytes, 2 failures Nov 01 21:22:48 2024 (1574) <cbacd3866ebdfeeb706993996ba84e67f7db5ad9.camel(a)billoblog.com> delivery to oliver(a)billoblog.com failed with code 454, b'4.7.1 <oliver(a)billoblog.com>: Relay access denied' Nov 01 21:22:48 2024 (1574) <cbacd3866ebdfeeb706993996ba84e67f7db5ad9.camel(a)billoblog.com> delivery to billo(a)billoblog.com failed with code 454, b'4.7.1 <billo(a)billoblog.com>: Relay access denied' Nov 01 21:24:11 2024 (1574) <173051065026.1640.13931339026208387550(a)162-144-108-19.bluehost.com> smtp to tradfp(a)bill-oliver.com for 1 recips, completed in 0.24669885635375977 seconds Nov 01 21:24:11 2024 (1574) <173051065026.1640.13931339026208387550(a)162-144-108-19.bluehost.com> post to tradfp(a)bill-oliver.com from tradfp-request(a)bill-oliver.com, 1222 bytes I have gone over my main.cf and master.cf a zillion times. Worse, I've compared them with the same files in the setup that works, and I can't find a significant error) though things are in a slightly different order. I don't know if I'm missing something big or am just a poor proofreader. Any ideas would be appreciated. Here's the obligatory main.cf and master.cf in /etc/postfix. I use dovecot and virtual domains and virtual mailboxex administered using postfixadmin. Here's the main.cf: # See /usr/share/postfix/main.cf.dist for a commented, more complete version # Debian specific: Specifying a file name will cause the first # line of that file to be used as the name. The Debian default # is /etc/mailname. #myorigin = /etc/mailname smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h readme_directory = no # See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on # fresh installs. compatibility_level = 3.6 #Enable TLS Encryption when Postfix receives incoming emails smtpd_tls_cert_file=/etc/letsencrypt/live/mail.bill- oliver.com/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/mail.bill- oliver.com/privkey.pem smtpd_tls_security_level=may smtpd_tls_loglevel = 1 smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache #Enable TLS Encryption when Postfix sends outgoing emails smtp_tls_CApath=/etc/ssl/certs smtp_tls_security_level = may smtp_tls_loglevel = 1 smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache transport_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp local_recipient_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp relay_domains = hash:/opt/mailman/mm/var/data/postfix_domains myhostname = mail.bill-oliver.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = /etc/mailname mydestination = $myhostname, localhost.$mydomain, localhost relayhost = mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 mail.bill- oliver.com bill-oliver.com lists.bill-oliver.com virtual_mailbox_limit = 0 mailbox_size_limit = 0 message_size_limit = 0 recipient_delimiter = + inet_interfaces = all inet_protocols = ipv4 #Enforce TLSv1.3 or TLSv1.2 smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 mailbox_transport = lmtp:unix:private/dovecot-lmtp smtputf8_enable = no virtual_mailbox_domains = proxy:mysql:/etc/postfix/sql/mysql_virtual_domains_maps.cf virtual_mailbox_maps = proxy:mysql:/etc/postfix/sql/mysql_virtual_mailbox_maps.cf, proxy:mysql:/etc/postfix/sql/mysql_virtual_alias_domain_mailbox_maps.cf virtual_alias_maps = proxy:mysql:/etc/postfix/sql/mysql_virtual_alias_maps.cf, proxy:mysql:/etc/postfix/sql/mysql_virtual_alias_domain_maps.cf, proxy:mysql:/etc/postfix/sql/mysql_virtual_alias_domain_catchall_maps.c f hash:/opt/mailman/mm/var/data/postfix_vmap #virtual_transport = lmtp:unix:private/dovecot-lmtp virtual_mailbox_base = /var/vmail virtual_minimum_uid = 2000 virtual_uid_maps = static:2000 virtual_gid_maps = static:2000 policyd-spf_time_limit = 3600 smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policyd-spf smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination # Milter configuration milter_default_action = accept milter_protocol = 6 smtpd_milters = local:opendkim/opendkim.sock non_smtpd_milters = $smtpd_milters unknown_local_recipient_reject_code = 550 ########################################### Here's the master.cf (comments deleted) : smtp inet n - y - - smtpd submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_tls_wrappermode=no -o smtpd_sasl_auth_enable=yes -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticate d,reject -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth smtps inet n - y - - smtpd -o syslog_name=postfix/smtps -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticate d,reject -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth pickup unix n - y 60 1 pickup cleanup unix n - y - 0 cleanup qmgr unix n - n 300 1 qmgr tlsmgr unix - - y 1000? 1 tlsmgr rewrite unix - - y - - trivial-rewrite bounce unix - - y - 0 bounce defer unix - - y - 0 bounce trace unix - - y - 0 bounce verify unix - - y - 1 verify flush unix n - y 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - y - - smtp relay unix - - y - - smtp -o syslog_name=postfix/$service_name showq unix n - y - - showq error unix - - y - - error retry unix - - y - - error discard unix - - y - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - y - - lmtp anvil unix - - y - 1 anvil scache unix - - y - 1 scache postlog unix-dgram n - n - 1 postlogd uucp unix - n n - - pipe flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient) policyd-spf unix - n n - 0 spawn user=policyd-spf argv=/usr/bin/policyd-spf Thanks, billo

2 2

How to fix: [Django] ERROR (EXTERNAL IP)
by Jigme Datse 02 Nov '24

02 Nov '24

This feels like something that I should have working. I know I've not been attending to things all that well, so the thing that has happened, that's got us from, "possibly doing nothing," to throwing this error is that I went to update my mail server, and in the update, Mailman3 stopped working entirely (it was a variety of things), but now it's launching, and will load the main web interface... but Going to any of the list pages, gives the above Error. Well, not exactly the above error, Django is in square brackets. OK, that will work better for me with the how to fix. How to fix, probably isn't quite what I'm looking for, it's how to diagnose. The error tells me that the IP address is 127.0.0.1 and that 127.0.0.1 is in the ALLOWED_HOSTS. I *mostly* see this coming from Google, but when I go and look at the webpage, I get: normal page at /mailman3/lists/ But my lists, all give me... Something went wrong HTTP error 500: {"title": "500 Internal Server Error"} I've tried to figure out where to go from here, and right now I'm not making any progress, so I thought I'd ask here.

2 6

redirecting -bounces replies
by Hirayama, Pat 02 Nov '24

02 Nov '24

Greetings, Is it possible to redirect the -bounces alias of a specific list so that it also goes to a person, in addition to the normal bounces processing for that list? In Mailman2, I know that I could do that by modifying the aliases table. I've been poking around in Mailman3, but haven't managed to figure out which container I need to be looking at. And rather than keep spinning, I figured that I'd ask the hive mind, since I'm sure one of you can quickly point me to the correct location. Thanks! -p Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering - Infrastructure Fred Hutch Cancer Center O 206.667.4856 phirayam(a)fredhutch.org<mailto:phirayam@fredhutch.org>

3 3

Do i need a mailman-hyperkitty.cfg file?
by Bill Oliver 30 Oct '24

30 Oct '24

While trying to set up my new mailman installation in Ubuntu 22.04, which doesn't quite work yet, I decided to compare my config files with my Ubuntu 24.04 LTS installation, which does seem to work to the degree I've tested it. I noticed this little snippet in the settings.py file: # Set this to match the api_key setting in # /opt/mailman/mm/mailman-hyperkitty.cfg (quoted here, not there). MAILMAN_ARCHIVER_KEY = 'Secret_Hyperkitty_API_Key' Aha, I thought. I should have changed that. Then I noticed that I didn't even have a mailman-hyperkitty.cfg file, and I can't find mention of it in the venv instructions. I *have* installed mailman- hyperkitty during the "pip install mailman-web mailman-hyperkitty" step. I did a "find" on /opt/mailman, and it doesn't seem the file is there at all. Do I need this file? I found a template here: https://gitlab.com/mailman/mailman-hyperkitty/blob/master/mailman-hyperkitt… Do I need to copy and paste that in my installation? Thanks! billo

2 2

Welcome message issue
by Mohsen Masoudfar 29 Oct '24

29 Oct '24

Good afternoon I have an issue with welcome message. A list owner is requesting to update the welcome message for only one of their lists. They are requesting to change the default text and insert additional URLs pointing to sites related to their list. I know there is a URI field that you may include a web page. But this doesn't work and gets ignored. - is there a mailman cli option to update the default template for a single list? I couldn't find it in WUI. I reviewed all parameters and variables of the list and couldn't find one that might be related to it. We are running an old version of Mailman 3. Thanks Mohsen

2 2