MM3 + Postfix + Dovecot + PostfixAdmin
System: Debian Buster Install: Functioning Postfix w/ Dovecot using PostfixAdmin for virtual domain and mailbox management
Definitions: Host.TLD = my server's domain name; List.TLD = listserv's domain name; ListName = list's name.
Issue: Postfix LMTP does not pass messages onto MM3:LMTP. Instead the behavior summary is as follows:
postfix/lmtp[...]: ###: to=mailman3:[127.0.0.1]:8024@Host.TLD, orig_to=ListName-confirm+###@ListServer.TLD, relay=Host.TLD[private/dovecot-lmtp], delay=439, delays=439/0.03/0.01/0.01, dsn=5.1.1, status=bounced (host Host.TLD[private/dovecot-lmtp] said: 550 5.1.1 <"mailman3:[127.0.0.1]:8024"@Host.TLD> User doesn't exist: mailman3:[127.0.0.1]:8024@Host.TLD (in reply to RCPT TO command))
The problem appears to be that Postfix defaults the relay to being private/dovecot-lmtp and not the general lmtp protocol or the mailman3 dedicated lmtp protocol.
Details:
# cat /proc/version Linux version 4.19.0-5-amd64 (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.37-5+deb10u1 (2019-07-19)
# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no broken_sasl_auth_clients = yes compatibility_level = 2 debug_peer_level = 4 html_directory = /usr/share/doc/postfix/html inet_interfaces = all inet_protocols = ipv4 local_recipient_maps = $virtual_mailbox_maps,regexp:/var/lib/mailman3/data/postfix_lmtp local_transport = virtual mailbox_size_limit = 0 mydestination = localhost myhostname = rijr.com mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 myorigin = /etc/mailname owner_request_special = no policy-spf_time_limit = 3600s readme_directory = /usr/share/doc/postfix recipient_delimiter = + relay_domains = $mydestination,regexp:/var/lib/mailman3/data/postfix_domains relayhost = smtp_tls_loglevel = 1 smtp_tls_security_level = may smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU) smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain = $mydomain smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_tls_security_options = $smtpd_sasl_security_options smtpd_sasl_type = dovecot smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/ssl/private/vmail.crt smtpd_tls_key_file = /etc/ssl/private/vmail.key smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtpd_use_tls = yes transport_maps = hash:/etc/postfix/transport,regexp:/var/lib/mailman3/data/postfix_lmtp virtual_alias_domains = proxy:mysql:/etc/postfix/virtual_alias_domains.cf virtual_alias_maps = proxy:mysql:/etc/postfix/virtual_alias_maps.cf,proxy:mysql:/etc/postfix/virtual_alias_domains_maps.cf,regexp:/var/lib/mailman3/data/postfix_lmtp virtual_gid_maps = static:5000 virtual_mailbox_base = /home/vmail virtual_mailbox_domains = proxy:mysql:/etc/postfix/virtual_mailbox_domains.cf virtual_mailbox_limit = 512000000 virtual_mailbox_maps = regexp:/var/lib/mailman3/data/postfix_lmtp,proxy:mysql:/etc/postfix/virtual_mailbox_maps.cf virtual_minimum_uid = 5000 virtual_transport = lmtp:unix:private/dovecot-lmtp virtual_uid_maps = static:5000
# cat master.cf [grep'd for lmtp and mailman3] lmtp unix - - y - - lmtp
mailman3 unix - - - - - lmtp -o lmtp_send_xforward_command=yes -o disable_dns_lookups=yes
# cat mailman.cfg [mta section only] [mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver smtp_host: localhost smtp_port: 25 lmtp_host: 127.0.0.1 lmtp_port: 8024 configuration: /etc/mailman3/postfix-mailman.cfg
# cat /var/lib/mailman3/data/postfix_lmtp # AUTOMATICALLY GENERATED BY MAILMAN ON 2019-08-06 06:39:21 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own.
# Aliases which are visible only in the @List.TLD domain. /^List@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-bounces(\+.*)?@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-confirm(\+.*)?@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-join@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-leave@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-owner@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-request@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-subscribe@List\.TLD$/ mailman3:[127.0.0.1]:8024 /^List-unsubscribe@List\.TLD$/ mailman3:[127.0.0.1]:8024
# cat postfix-mailman.cfg [postfix] # if I don't have postmap_command set, mailman errors out, even though transport_file_type is regex. postmap_command: /usr/sbin/postmap transport_file_type: regex
# cat mail.debug
Aug 6 01:03:28 Host dovecot: lmtp(23055): Connect from local Aug 6 01:03:28 Host dovecot: auth: Debug: master in: USER#0112#011mailman3:[127.0.0.1]:8024@Host.TLD#011service=lmtp Aug 6 01:03:28 Host dovecot: auth: userdb(?): Username character disallowed by auth_username_chars: 0x3a (username: mailman3:[127.0.0.1]:8024@Host.TLD) Aug 6 01:03:28 Host dovecot: auth: Debug: userdb out: NOTFOUND#0112 Aug 6 01:03:28 Host postfix/cleanup[22970]: 2937FB0: message-id=20190806080328.2937FB0@rijr.com Aug 6 01:03:28 Host postfix/lmtp[23054]: CA030AA: to=mailman3:[127.0.0.1]:8024@rijr.com, orig_to=gilman-clan-confirm+51c0564a95aa31a8840305bec77e9023baa46477@List.TLD, relay=Host.TLD[private/dovecot-lmtp], delay=439, delays=439/0.03/0.01/0.01, dsn=5.1.1, status=bounced (host Host.TLD[private/dovecot-lmtp] said: 550 5.1.1 <"mailman3:[127.0.0.1]:8024"@Host.TLD> User doesn't exist: mailman3:[127.0.0.1]:8024@Host.TLD (in reply to RCPT TO command)) Aug 6 01:03:28 Host dovecot: lmtp(23050): Disconnect from local: Client has quit the connection (state=READY) Aug 6 01:03:28 Host postfix/lmtp[22981]: C0EEEA8: to=mailman3:[127.0.0.1]:8024@Host.TLD, orig_to=ListName-confirm+51c0564a95aa31a8840305bec77e9023baa46477@List.TLD, relay=Host.TLD[private/dovecot-lmtp], delay=566, delays=566/0.04/0.01/0.01, dsn=5.1.1, status=bounced (host Host.com[private/dovecot-lmtp] said: 550 5.1.1 <"mailman3:[127.0.0.1]:8024"@Host.TLD> User doesn't exist: mailman3:[127.0.0.1]:8024@Host.TLD (in reply to RCPT TO command))
I have played with dozens of variations on the configuration files and tracing what is going on. The only reason I see that relay=Host.TLD[private/dovecot-lmtp] is because of: virtual_transport = lmtp:unix:private/dovecot-lmtp
Hopefully I can get this figured out ....
- Jay
maztec@gmail.com writes:
Issue: Postfix LMTP does not pass messages onto MM3:LMTP.
I don't understand why this is a Mailman problem. Seems like Postfix user lists would be a better venue to get expert help with this. Among other things, our most responsive Postfix guy is out of cell coverage for now.
I would post this To: the relevant Postfix list, Cc: this list, and set Reply-To and Mail-Followups-To to the Postfix list to get everybody who can contribute on the same page.
If it is a Mailman problem, you have my apologies and I'm listening.
Steve
On 8/8/19 10:14 PM, Stephen J. Turnbull wrote:
maztec@gmail.com writes:
Issue: Postfix LMTP does not pass messages onto MM3:LMTP.
I don't understand why this is a Mailman problem. Seems like Postfix user lists would be a better venue to get expert help with this. Among other things, our most responsive Postfix guy is out of cell coverage for now.
I have a window of availability - and it's longer than I thought because my second of three back-country trips has been canceled :(
I would post this To: the relevant Postfix list, Cc: this list, and set Reply-To and Mail-Followups-To to the Postfix list to get everybody who can contribute on the same page.
I don't have much to add to what Steve has said except our recommended config includes
transport_maps = hash:/path-to-mailman/var/data/postfix_lmtp
in Postfix main.cf. This should contain (per your log message) things like
ListName-confirm@ListServer.TLD lmtp:[127.0.0.1]:8024
to tell Postfix to deliver that address via lmtp to port 8024 on the local host. Dovecot should not be involved. There is something in your config that trumps this and delivers via lmtp to Dovecot for mailman3:[127.0.0.1]:8024. Does your transport_maps look like the above or perhaps like
ListName-confirm@ListServer.TLD mailman3:[127.0.0.1]:8024
If so, I don't know where mailman3 instead of lmtp: comes from. Also, even if it is lmtp: I don't know that Dovecot can deliver via lmtp.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Steve: Fair enough. The issue ultimately was with Postfix, but I only encountered it because I misread the Mailman3 configuration wiki as setting "virtual_alias_maps = hash:/path-to-mailman/var/data/postfix_vmap" to be /path-to-mailman/var/data/postfix_lmtp. The lmtp versus vmap distinction made a large difference. I'm not sure where I picked that error up or whether I just imagined it myself.
Fixed now!
Thank you.
Jay
On Thu, Aug 8, 2019 at 10:14 PM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
maztec@gmail.com writes:
Issue: Postfix LMTP does not pass messages onto MM3:LMTP.
I don't understand why this is a Mailman problem. Seems like Postfix user lists would be a better venue to get expert help with this. Among other things, our most responsive Postfix guy is out of cell coverage for now.
I would post this To: the relevant Postfix list, Cc: this list, and set Reply-To and Mail-Followups-To to the Postfix list to get everybody who can contribute on the same page.
If it is a Mailman problem, you have my apologies and I'm listening.
Steve
Jay Gairson writes:
Steve: Fair enough. The issue ultimately was with Postfix, but I only encountered it because I misread the Mailman3 configuration wiki as setting "virtual_alias_maps = hash:/path-to-mailman/var/data/postfix_vmap" to be /path-to-mailman/var/data/postfix_lmtp.
Thanks for the followup!
For future reference for those who are following along, while that's technically a Postfix configuration issue, it's the kind of thing that people like Mark and Abhilash are at least as likely to notice as most of the folks on Postfix lists. Bottom line: don't hesitate to report these issues to us, but if it could be the MTA, report to their users' list too. More eyes, etc.
Steve
Exactly as Steve said. I did report it to the Postfix lists and then had my running in circles for completely unrelated issues. They were convinced that specifying Mailman3 as an LMTP instead of an SMTP server was incorrect, because if it was LMTP I should be pointing at a socket instead of localhost. At the end of the day I figured it out by reading problems other people were having in their configurations that sounded like the same thing, and someone made a side comment about virtual_alias_maps. I double-checked, saw my configuration error, and fixed it. I think I made this mistake because I had started with a hash-based configuration and then realized I needed to use regexp. The hash-based configuration has a side case that requires postfix_vmap to be defined. When changing to regexp I simply changed the labels to regexp, saw my error log spitting out that there was no postfix_vmap file, looked at the regexp configuration saw that there was no vmap for it and instead of deleting the vmap changed it over to lmtp. My own fault all the way around, but worth noting at this point since others might be tired or in a hurry and make the same mistake. A mistake that doesn't really report an error in a way that makes it stand out.
On Mon, Aug 19, 2019 at 12:29 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Jay Gairson writes:
Steve: Fair enough. The issue ultimately was with Postfix, but I only encountered it because I misread the Mailman3 configuration wiki as setting "virtual_alias_maps = hash:/path-to-mailman/var/data/postfix_vmap" to be /path-to-mailman/var/data/postfix_lmtp.
Thanks for the followup!
For future reference for those who are following along, while that's technically a Postfix configuration issue, it's the kind of thing that people like Mark and Abhilash are at least as likely to notice as most of the folks on Postfix lists. Bottom line: don't hesitate to report these issues to us, but if it could be the MTA, report to their users' list too. More eyes, etc.
Steve
Jay Gairson writes:
They were convinced that specifying Mailman3 as an LMTP instead of an SMTP server was incorrect, because if it was LMTP I should be pointing at a socket instead of localhost.
Did they say why? I understand why a socket MAY be used, but is there a reason why localhost SHOULD NOT be used? (If it's complicated or unclear and this was all on their public lists, point me that-a-way.)
My own fault all the way around,
Well, maybe we can provide a better default, somehow.
A mistake that doesn't really report an error in a way that makes it stand out.
Reporting that error is pretty much out of Mailman's reach, correct? (Except by implementing config consistency checking that we should contribute to Postfix, I mean.)
Steve
-- Associate Professor Division of Policy and Planning Science http://turnbull.sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN
participants (4)
-
Jay Gairson -
Mark Sapiro -
maztec＠gmail.com
-
Stephen J. Turnbull