- Mailman-users - lists.mailman3.org

Migrating mailman3 to latest ubuntu lts
by Helio Loureiro 19 Dec '23

19 Dec '23

Hi, I'm trying to migrate several lists from an ubuntu 18.04, where it was installed via packages (mailman 3.1.1), to ubuntu 22.04. Since ubuntu 22.04 is broken in terms of packages for mailman3, I'm trying to install manually following the site instructions. I just dumped the mysql DB we have nowadays, mailman3 and mailman3-web, and loaded in the new server. I also copied the configurations and fixed the paths since now mailman3 lives on /local/mailman and it uses venv. I also copied /var/lib/mailman3 into /local/mailman/var/lib/mailman3. mailman seems to accept pretty much of the work done. (venv) mailman@new-server ~ (v3.3.9)> mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] config file: /etc/mailman3/mailman.cfg db url: mysql+pymysql://mailman3:********@localhost/mailman3?charset=utf8mb4&use_unicode=1 devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:*********** Even configuration is working fine. (venv) mailman@new-server ~ (v3.3.9)> mailman conf [ARC] authserv_id: [ARC] dkim: yes [ARC] dmarc: yes [ARC] domain: [ARC] enabled: no [ARC] privkey: [ARC] selector: [ - a lot of stuff - ] [shell] banner: Welcome to the GNU Mailman shell [shell] history_file: [shell] prompt: >>> [shell] use_ipython: no [styles] default: legacy-default [webservice] admin_pass: ************************* [webservice] admin_user: restadmin [webservice] api_version: 3.1 [webservice] configuration: python:mailman.config.gunicorn [webservice] hostname: localhost [webservice] port: 8001 [webservice] show_tracebacks: yes [webservice] use_https: no [webservice] workers: 2 But it is failing to start displaying the domain name from the previous machine. (venv) mailman@new-server ~ (v3.3.9)> mailman start Starting Mailman's master runner Generating MTA alias maps Traceback (most recent call last): File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.OperationalError: (1054, "Unknown column 'domain.alias_domain' in 'field list'") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/local/mailman/venv/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts', 'mailman')()) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 69, in invoke return super().invoke(ctx) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/local/mailman/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/commands/cli_control.py", line 109, in start call_name(config.mta.incoming).regenerate() File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 109, in regenerate self._generate_lmtp_file(fp) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 149, in _generate_lmtp_file mlist = _FakeList(list_name, mail_host) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 75, in __init__ self.mail_host = _get_alias_domain(mail_host) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/mta/postfix.py", line 63, in _get_alias_domain d = domain_manager.get(domain) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/database/transaction.py", line 106, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/local/mailman/venv/lib/python3.10/site-packages/mailman/model/domain.py", line 162, in get if domains.count() < 1: File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 3132, in count self._legacy_from_self(col).enable_eagerloads(False).scalar() File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2825, in scalar ret = self.one() File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2798, in one return self._iter().one() # type: ignore File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2847, in _iter result: Union[ScalarResult[_T], Result[_T]] = self.session.execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2308, in execute return self._execute_internal( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 2190, in _execute_internal result: Result[Any] = compile_state_cls.orm_execute_statement( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement result = conn.execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1416, in execute return meth( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 516, in _execute_on_connection return connection._execute_clauseelement( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1639, in _execute_clauseelement ret = self._execute_context( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context return self._exec_single_context( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context self._handle_dbapi_exception( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2343, in _handle_dbapi_exception raise sqlalchemy_exception.with_traceback(exc_info[2]) from e File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context self.dialect.do_execute( File "/local/mailman/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute cursor.execute(statement, parameters) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute result = self._query(query) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query conn.query(q) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result result.read() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read first_packet = self.connection._read_packet() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet packet.raise_for_error() File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/local/mailman/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1054, "Unknown column 'domain.alias_domain' in 'field list'") [SQL: SELECT count(*) AS count_1 FROM (SELECT domain.id AS domain_id, domain.mail_host AS domain_mail_host, domain.description AS domain_description, domain.alias_domain AS domain_alias_domain FROM domain WHERE domain.mail_host = %(mail_host_1)s) AS anon_1] [parameters: {'mail_host_1': 'old-server.mydomain.com'}] (Background on this error at: https://sqlalche.me/e/20/e3q8) But I can see some processes are starting. (venv) mailman@new-server ~ (v3.3.9)> ps auxwww | grep "[p]ython" | grep mailman mailman 1411605 0.0 0.0 32180 22212 ? Ss 10:44 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1411607 0.0 0.0 44244 32412 ? S 10:44 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1411671 1.0 0.2 98584 83004 ? Ss 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/master -C /etc/mailman3/mailman.cfg mailman 1411681 1.0 0.2 98392 82504 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=archive:0:1 mailman 1411682 97.7 10.1 3396160 3346868 ? R 10:45 1:58 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=bounces:0:1 mailman 1411683 1.0 0.2 98588 82816 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=command:0:1 mailman 1411684 1.0 0.2 98580 82904 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=in:0:1 mailman 1411685 1.0 0.2 172312 82772 ? Sl 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=lmtp:0:1 mailman 1411686 1.1 0.2 98588 82652 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=nntp:0:1 mailman 1411687 1.1 0.2 98608 83016 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=out:0:1 mailman 1411688 1.0 0.2 98584 82936 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=pipeline:0:1 mailman 1411689 1.1 0.2 107040 90972 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1411690 1.0 0.2 98592 82784 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=retry:0:1 mailman 1411691 41.3 0.5 187380 171012 ? S 10:45 0:50 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=task:0:1 mailman 1411692 1.0 0.2 98472 83028 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=virgin:0:1 mailman 1411693 1.0 0.2 98448 82608 ? S 10:45 0:01 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=digest:0:1 mailman 1411705 0.0 0.2 107040 75968 ? S 10:45 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1411709 0.0 0.2 107040 75968 ? S 10:45 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/runner -C /etc/mailman3/mailman.cfg --runner=rest:0:1 mailman 1413629 0.0 0.0 17644 8812 ? Rs 10:47 0:00 /local/mailman/venv/bin/python /local/mailman/venv/bin/mailman start Best Regards, Helio Loureiro http://helio.loureiro.eng.br https://github.com/helioloureiro

4 17

Spam sent to list owner causes bounce increment for list owner
by Allan Hansen 19 Dec '23

19 Dec '23

Hi all, Someone sent a spam message to the list owner (me) of one of my lists. Then I got booted from the list. The scenario in more detail: The list server forwards the message to me, as I’m the list owner. This message is sent from <list>- <mailto:artists-bounces+hansen=rc.org@list.rc.org>bounces+hansen=rc.org(a)list.rc.org <mailto:bounces+hansen=rc.org@list.rc.org> to hansen(a)rc.org <mailto:hansen@rc.org>. My service provider’s spam filter rejects the spam (correctly) in a message and informs the list server. The list server sends a message to <list>- <mailto:artists-bounces+hansen=rc.org@list.rc.org>bounces+hansen=rc.org(a)list.rc.org <mailto:bounces+hansen=rc.org@list.rc.org> (the fake sender) saying that the message to hansen(a)rc.org <mailto:hansen@rc.org> (the recipient) could not be delivered because it was spam. The list server also blames hansen(a)rc.org <mailto:hansen@rc.org> (me) for the bounce and increments my bounce score, which, to me, is unfair. So I wonder: Is the Mailman host at list.rc.org <http://list.rc.org/> at fault for not catching the spam before it reaches the list server? (One could argue that) Is my rc.org <http://rc.org/> service provider at fault for sending the bounce message to the server instead of the to the original sender? Is Mailman at fault for thinking that the bounce message was caused by a list message FROM me, instead of a bounced message TO the list owner? Yours, Allan

2 2

MM3 Postfix FROM header issue
by eboltz＠lhtservices.com 18 Dec '23

18 Dec '23

Installed MM3 on Debian 11, followed the steps from the MM3 website. I'm running into issues with Postfix headers. I can only get postfix to send emails to/from the list if I set the sender_canonical_maps in the main.cf file. I don't want to have to use that as I'd like the recipient to receive the following in their email header: <Subscriber> via <List Name> <FROM:LIST@DOMAIN>. But if I do the sender_canonical_maps, then the <FROM:LIST@DOMAIN> and all email headers are overwritten by the maps email address. When I check the /var/log/mail.log without the sender_canonical_maps setting enabled, then the initial email sender is either rejected because 'Recipient address rejected: User unknown in virtual alias table' or '550-From header sender domain not verified (example.com), 550-On your sending > Verified senders page 550 verify the sender domain or email to be allowed to send' I have smtp_sasl_passwd_maps disabled, since I'm trying a different SMTP relay Here is my main.cf setup # See /usr/share/postfix/main.cf.dist for a commented, more complete version # Debian specific: Specifying a file name will cause the first # line of that file to be used as the name. The Debian default # is /etc/mailname. #myorigin = /etc/mailname smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) biff = no # appending .domain is the MUA's job. append_dot_mydomain = no # Uncomment the next line to generate "delayed mail" warnings #delay_warning_time = 4h readme_directory = no # See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on # fresh installs. compatibility_level = 2 # Mailman related settings #owner_request_special = no transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp local_recipient_maps = hash:/var/lib/mailman3/data/postfix_lmtp relay_domains = hash:/var/lib/mailman3/data/postfix_domains #relay_domains = ${{$compatibility_level} < {2} ? {$mydestination} : {}} hash:/var/lib/mailman3/data/postfix_domains # TLS parameters smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key smtpd_tls_security_level=may #smtp_tls_CApath=/etc/postfix/o365_smtp_cert.pem smtp_tls_security_level=may smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination myhostname = localhost alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = /etc/mailname mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain relayhost = [mail.smtp2go.com]:587 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all inet_protocols = ipv4 sender_canonical_maps = regexp:/etc/postfix/sender_canonical smtp_use_tls = yes smtp_always_send_ehlo = yes unknown_local_recipient_reject_code = 550 always_add_missing_headers = yes #smtp_sasl_auth_enable = yes #smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd smtp_sasl_security_options = noanonymous header_size_limit = 409600 default_destination_recipient_limit = 30 default_destination_concurrency_limit = 15 header_checks = regexp:/etc/postfix/header_checks mynetworks = 127.0.0.0/8 [::fff:127.0.0.0]/104 [::1]/128

4 18

Enable Notifications on Mailman3 Threads
by jdamos＠iu12.org 18 Dec '23

18 Dec '23

In my environment, I am trying to enable notifications on a thread just like this one. When someone responds, I want the list to get a notification and or the person that is getting responded to. Any thoughts? Thank you!

2 2

List users from different domains getting blocked.
by bob B 14 Dec '23

14 Dec '23

I am running MM3 and have a list that has users from different organizations and email domains. So let's say I am orga.com and hosting mailman Bill is borg.com Cindy is mew.com When Bill sends a message to my MM3 list it gets sent out, However, cindy's email system rejects it cause it sees the message is sent from Bill(a)borg.com but it came from my MM3 server which is under orga.com, so it rejects it. I have been told more organizations are checking this and starting to reject these types of messages and also we could get blacklisted because it looks like we are spoofing. The only solutions I can think of are: -Have the receiving organizations whitelist my server, but that is probably not feasible. -set the list to anonymous..... But then you don't know who sent the email. Anyway around this? or is there a way to have the sender's email included in the list message but keep the list anonymous? or am I missing something else? and if I send amessage from my organaxztion to MM# it will go out to the peopl n the list, and if and some of those people are at other organizations. When the recive the email, sometoimes their mail system reject it with a DMAR message. I have been told thisis beacue the other organazatione dmeail system sees I snet the message

3 2

customized welcome message
by Nils 07 Dec '23

07 Dec '23

2 6

templates/socialaccount/login.html - any reason not to include?
by Jered Floyd 07 Dec '23

07 Dec '23

When using a social login button, there is a completely unstyled page as an interstitial before login. I copied the upstream allauth/templates/socialaccount/login.html and changed the first line from: {% extends "socialaccount/base_entrance.html" %} to {% extends "django_mailman3/profile/base.html" %} This seems to work properly and look correct. Is there a reason not to PR this? --Jered

1 0

mailman3-web language preference and translation settings
by Nils 07 Dec '23

07 Dec '23

Hi, I've tested mailman3 and now installed it on productive server. The customer complains that the translation to German is incomplete on the productive server while it was complete on the test server. I can't find a way to confirm this because I only see the English frontend of mailman3-web and I don't know how to switch language in the frontend (I have my OS set to English). Kind regards

2 4

no bounce-processing with verp_probes: yes
by Stefan Bauer 07 Dec '23

07 Dec '23

Dear Users, during one of our upgrades to 3.3.x we set verp_probes: yes according to upgrade notes[1]. However it looks like this disabled bounce handling completely. Howto verify (MM 3.3.9): Set verp_probes: yes verp_delivery_interval: 1 restart mailman3 create test-list add an undeliverable recipient to the list that is known to bounce. send 6 mails to list. see, that all the bounce-messages are received by mailman according to postfix log[2] check bounce.log - no entry. get bounce_score from API[3] - shows bounce_score 0. Can anyone confirm? Any help is greatly appreciated. Thank you. Stefan [1] https://docs.mailman3.org/en/latest/upgrade-3.2.html [2] from=<>, size=8565, nrcpt=1 (queue active) ... to=<v-test-bounces+undeliverable-recipient=realdomain.tld(a)lists.lists.tld>, relay=127.0.0.1[127.0.0.1]:8024 status=sent [3] curl -s -q --user a:b http://localhost:8001/3.1/members/find?subscriber=undeliverable-recipient@r…

2 2

Odd behavior: "Invalid" list destinations result in mail routing loop w/ SMTP relay in line
by Thomas Ward 06 Dec '23

06 Dec '23

We use a multi-hop SMTP system to go from the inside of our network out to the Internet at the domain test.lists.example.com. The setup is this way for ingress: {EXTERNAL} -> MS365 (relay/antispam) -> MS On-Prem (via 365 Connector) -> Mailman for delivery to listserv Egress is this: Mailman -> Postfix SMTP Relay (smtpproxy) -> INTERNET -- When a message is sent an invalid list (we'll call it invalid(a)test.lists.example.com) we end up with a nasty side effect in the system. The "no such recipient" response gets sent to the smtpproxy machine and then *it gets redelivered back to Mailman* as a redelivery as if it isn't accepting a permanent-failure condition code. This results in a nasty mail loop that results with a "too many hops" error that doesn't state "invalid recipient" and I have no idea why this is the case. Can anyone provide any insight into how to prevent this type of routing loop caused by the egress of messages? Thomas

2 4