September 2023 - Mailman-users - lists.mailman3.org

Templates via Webinterface are i) not set, and ii) set templates (files) are not shown in the Webinterface
by Wikinaut 30 Sep '23

30 Sep '23

I use mailman3 since a while, for some of my lists I have been using - list:member:digest:footer.txt - list:member:regular:footer.txt When I go to "Vorlagen" (LC: DE) (templates) in the web interface, I do _not_ see these two templates! When I got o my newest mailing list and when I setup templates via the Webinterfaces, these are shown there, but are neither working nor seen as files. What's wrong here? Is that a problem of the localisation language (DE)? Bug, feature or misunderstanding of myself?

2 2

Is there a way of recording bounce messages?
by Andrew Hodgson 28 Sep '23

28 Sep '23

Hi. I recently had an incident of all gmail.com members being bounced off one email list (around 180 subscribers). The messages recording the bounce don't include the original DSN. Is there a way of getting the DSN recorded? I'm sure it will give me some nonsense Gmail refusal message but I would like to identify if messages are bouncing for specific users, or if I am harming my reputation by sending to disabled accounts etc. Thanks. Andrew.

5 17

Mailing list name with capital letters?
by mrl＠psfc.mit.edu 26 Sep '23

26 Sep '23

How can I create a mailing list with capital letters? Right now, only the first letter can be capitalized. Some of my mailing list names in mailman 2 had capital letters. When I migrated them to mailman 3, the capital letters remained. So it should be possible to create new ones? Thanks. - Mark

3 3

Mailman 3 - Header Filter
by Thomas Lehker 25 Sep '23

25 Sep '23

I am a Mailman neophyte and need help with filtering posts based on the subject field. We have a few list owners that want 'Out of Office' replies discarded. When using the below syntax in Postorius/Header filter the post is not discarded but ends up in 'Held Messages'. Is there any syntax I am missing or a setting in the lists that needs to be disabled/enabled to get these posts dropped? Any help is much appreciated. Header: ^subject: subject: subject Pattern: Automatic Reply 'Automatic Reply' "Automatic Reply" Action: Discard DISCLAIMER: The information contained in this message is privileged and confidential. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please reply to the message and delete/destroy any copy of this message. Information in this message that does not relate to the official business of the firm shall be understood as neither given nor endorsed by it.

4 4

Issues with @gmail.com addresses
by Mohsen Masoudfar 25 Sep '23

25 Sep '23

Hi, since 8/1/2023 I get complains from users with @gmail.com addresses, that they are not getting any emails. I checked the logs and I see that the emails are going out: to=<xxxx(a)gmail.com>, relay=email-smtp.us-east-1.amazonaws.com[35.169.171.20]:587, delay=0.46, delays=0.03/0.06/0.13/0.24, dsn=2.0.0, status=sent (250 Ok xxx ) But it seems the users are getting email delivered to them. I’ve advised them to check their spam, promotions, and social folders and to add the list address to their email safe-senders lists, but still no success. Is there any way to help this list members? Thanks

8 21

Strange password problem
by Eggert Ehmke 25 Sep '23

25 Sep '23

This happened several times recently with Postorius Version 1.3.8: -I try to log in as admin -rejected because of wrong password (password is stored in browser) -used forgotten password function -reset password to the stored value from browser -login works now. Works also when logged out/in again -however, the next day this behaviour repeats. Any hints?

6 22

how to administrate mailman3 in CLI ?
by testeur 23 Sep '23

23 Sep '23

Hello, I have installed mailman3 and i want to administrate it with rest api, i ve read https://mailman-cli.readthedocs.io/en/latest/src/mailmanclient/cli/docs/usi… and https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… and https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/commands/d… i tryied with mailman command, but it don't do some actions (eg. delete held messages...). i tryied mailman shell... i tryied with curl to manage it, i can see if there is a pending message with : curl --user restadmin:restpass http://localhost:8001/3.1/lists/ml@domain.tld/held/count|| python3 -m json.tool| but how to delete all held messages ? is it possible in one time ? And how to select a specific message to discard /accept / defer ? do i have to use a POST ? Regards, PatG

3 4

Message getting held due to max number of recipients and maximum message size
by Wong, Wesley 22 Sep '23

22 Sep '23

Hi all, I am running into an issue with messages getting held for a list with the following reasons: The message is being held because: Message has more than 10 recipients The message is larger than the 40 KB maximum size Either the message will get posted to the list, or you will receive notification of the moderator's decision. Currently, the settings are set to 0 for both settings on the list so I am not sure where mailman is getting the default settings. Is this default hard coded somewhere in the Mailman core? The version of Mailman running is 3.3.6. Any help for this issue is greatly appreciate. wesley Wesley Wong Senior Systems Engineer Anderson Computing & Information Services | ACIS 310-206-2571 [ucla-logo---new_smaller] ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: image001.png Replaced multipart/alternative part with first alternative.

2 3

Hyperkitty
by Massimiliano Paragnani 21 Sep '23

21 Sep '23

I set up the mailman3 services and lists, and we can send mail to the list successfully. However in the Archive section I couldn't see anything. My configuration: ubuntu, milman core 3.2.2, postresql, postorius 1.2.4, hyperkitty 1.3.2 This is my mailman conf result: [antispam] header_checks: [antispam] jump_chain: hold [archiver.mail_archive] class: mailman.archiving.mailarchive.MailArchive [archiver.mail_archive] clobber_date: maybe [archiver.mail_archive] clobber_skew: 1d [archiver.mail_archive] configuration: python:mailman.config.mail_archive [archiver.mail_archive] enable: no [archiver.mhonarc] class: mailman.archiving.mhonarc.MHonArc [archiver.mhonarc] clobber_date: maybe [archiver.mhonarc] clobber_skew: 1d [archiver.mhonarc] configuration: python:mailman.config.mhonarc [archiver.mhonarc] enable: no [archiver.prototype] class: mailman.archiving.prototype.Prototype [archiver.prototype] clobber_date: maybe [archiver.prototype] clobber_skew: 1d [archiver.prototype] configuration: changeme [archiver.prototype] enable: no [bounces] register_bounces_every: 15m [database] class: mailman.database.postgresql.PostgreSQLDatabase [database] debug: no [database] url: postgres://mailman3:--------------@localhost/mailman3 [devmode] enabled: no [devmode] recipient: [devmode] testing: no [devmode] wait: 60s [digests] mime_digest_keep_headers: Date From To Cc Subject Message-ID Keywords In-Reply-To References Content-Type MIME-Version Content-Transfer-Encoding Precedence Reply-To Message List-Post [digests] plain_digest_keep_headers: Message Date From Subject To Cc Message-ID Keywords Content-Type [dmarc] cache_lifetime: 7d [dmarc] org_domain_data_url: https://publicsuffix.org/list/public_suffix_list.dat [dmarc] resolver_lifetime: 5s [dmarc] resolver_timeout: 3s [language.ar] charset: utf-8 [language.ar] description: Arabic [language.ar] enabled: yes [language.ast] charset: iso-8859-1 [language.ast] description: Asturian [language.ast] enabled: yes [language.ca] charset: utf-8 [language.ca] description: Catalan [language.ca] enabled: yes [language.cs] charset: iso-8859-2 [language.cs] description: Czech [language.cs] enabled: yes [language.da] charset: iso-8859-1 [language.da] description: Danish [language.da] enabled: yes [language.de] charset: utf-8 [language.de] description: German [language.de] enabled: yes [language.el] charset: iso-8859-7 [language.el] description: Greek [language.el] enabled: yes [language.es] charset: iso-8859-1 [language.es] description: Spanish [language.es] enabled: yes [language.et] charset: iso-8859-15 [language.et] description: Estonian [language.et] enabled: yes [language.eu] charset: iso-8859-15 [language.eu] description: Euskara [language.eu] enabled: yes [language.fi] charset: iso-8859-1 [language.fi] description: Finnish [language.fi] enabled: yes [language.fr] charset: iso-8859-1 [language.fr] description: French [language.fr] enabled: yes [language.gl] charset: utf-8 [language.gl] description: Galician [language.gl] enabled: yes [language.he] charset: utf-8 [language.he] description: Hebrew [language.he] enabled: yes [language.hr] charset: iso-8859-2 [language.hr] description: Croatian [language.hr] enabled: yes [language.hu] charset: iso-8859-2 [language.hu] description: Hungarian [language.hu] enabled: yes [language.ia] charset: iso-8859-15 [language.ia] description: Interlingua [language.ia] enabled: yes [language.it] charset: iso-8859-1 [language.it] description: Italian [language.it] enabled: yes [language.ja] charset: euc-jp [language.ja] description: Japanese [language.ja] enabled: yes [language.ko] charset: euc-kr [language.ko] description: Korean [language.ko] enabled: yes [language.lt] charset: iso-8859-13 [language.lt] description: Lithuanian [language.lt] enabled: yes [language.nl] charset: iso-8859-1 [language.nl] description: Dutch [language.nl] enabled: yes [language.no] charset: iso-8859-1 [language.no] description: Norwegian [language.no] enabled: yes [language.pl] charset: iso-8859-2 [language.pl] description: Polish [language.pl] enabled: yes [language.pt] charset: iso-8859-1 [language.pt] description: Protuguese [language.pt] enabled: yes [language.pt_BR] charset: iso-8859-1 [language.pt_BR] description: Protuguese (Brazil) [language.pt_BR] enabled: yes [language.ro] charset: iso-8859-2 [language.ro] description: Romanian [language.ro] enabled: yes [language.ru] charset: koi8-r [language.ru] description: Russian [language.ru] enabled: yes [language.sk] charset: utf-8 [language.sk] description: Slovak [language.sk] enabled: yes [language.sl] charset: iso-8859-2 [language.sl] description: Slovenian [language.sl] enabled: yes [language.sr] charset: utf-8 [language.sr] description: Serbian [language.sr] enabled: yes [language.sv] charset: iso-8859-1 [language.sv] description: Swedish [language.sv] enabled: yes [language.tr] charset: iso-8859-9 [language.tr] description: Turkish [language.tr] enabled: yes [language.uk] charset: utf-8 [language.uk] description: Ukrainian [language.uk] enabled: yes [language.vi] charset: utf-8 [language.vi] description: Vietnamese [language.vi] enabled: yes [language.zh_CN] charset: utf-8 [language.zh_CN] description: Chinese [language.zh_CN] enabled: yes [language.zh_TW] charset: utf-8 [language.zh_TW] description: Chinese (Taiwan) [language.zh_TW] enabled: yes [logging.archiver] datefmt: %b %d %H:%M:%S %Y [logging.archiver] format: %(asctime)s (%(process)d) %(message)s [logging.archiver] level: info [logging.archiver] path: mailman.log [logging.archiver] propagate: no [logging.bounce] datefmt: %b %d %H:%M:%S %Y [logging.bounce] format: %(asctime)s (%(process)d) %(message)s [logging.bounce] level: info [logging.bounce] path: bounce.log [logging.bounce] propagate: no [logging.config] datefmt: %b %d %H:%M:%S %Y [logging.config] format: %(asctime)s (%(process)d) %(message)s [logging.config] level: info [logging.config] path: mailman.log [logging.config] propagate: no [logging.database] datefmt: %b %d %H:%M:%S %Y [logging.database] format: %(asctime)s (%(process)d) %(message)s [logging.database] level: warn [logging.database] path: mailman.log [logging.database] propagate: no [logging.debug] datefmt: %b %d %H:%M:%S %Y [logging.debug] format: %(asctime)s (%(process)d) %(message)s [logging.debug] level: info [logging.debug] path: debug.log [logging.debug] propagate: no [logging.error] datefmt: %b %d %H:%M:%S %Y [logging.error] format: %(asctime)s (%(process)d) %(message)s [logging.error] level: info [logging.error] path: mailman.log [logging.error] propagate: no [logging.fromusenet] datefmt: %b %d %H:%M:%S %Y [logging.fromusenet] format: %(asctime)s (%(process)d) %(message)s [logging.fromusenet] level: info [logging.fromusenet] path: mailman.log [logging.fromusenet] propagate: no [logging.http] datefmt: %b %d %H:%M:%S %Y [logging.http] format: %(asctime)s (%(process)d) %(message)s [logging.http] level: info [logging.http] path: mailman.log [logging.http] propagate: no [logging.locks] datefmt: %b %d %H:%M:%S %Y [logging.locks] format: %(asctime)s (%(process)d) %(message)s [logging.locks] level: info [logging.locks] path: mailman.log [logging.locks] propagate: no [logging.mischief] datefmt: %b %d %H:%M:%S %Y [logging.mischief] format: %(asctime)s (%(process)d) %(message)s [logging.mischief] level: info [logging.mischief] path: mailman.log [logging.mischief] propagate: no [logging.plugins] datefmt: %b %d %H:%M:%S %Y [logging.plugins] format: %(asctime)s (%(process)d) %(message)s [logging.plugins] level: info [logging.plugins] path: plugins.log [logging.plugins] propagate: no [logging.root] datefmt: %b %d %H:%M:%S %Y [logging.root] format: %(asctime)s (%(process)d) %(message)s [logging.root] level: info [logging.root] path: mailman.log [logging.root] propagate: no [logging.runner] datefmt: %b %d %H:%M:%S %Y [logging.runner] format: %(asctime)s (%(process)d) %(message)s [logging.runner] level: info [logging.runner] path: mailman.log [logging.runner] propagate: no [logging.smtp] datefmt: %b %d %H:%M:%S %Y [logging.smtp] every: $msgid smtp to $listname for $recip recips, completed in $time seconds [logging.smtp] failure: $msgid delivery to $recip failed with code $smtpcode, $smtpmsg [logging.smtp] format: %(asctime)s (%(process)d) %(message)s [logging.smtp] level: info [logging.smtp] path: smtp.log [logging.smtp] propagate: no [logging.smtp] refused: $msgid post to $listname from $sender, $size bytes, $refused failures [logging.smtp] success: $msgid post to $listname from $sender, $size bytes [logging.subscribe] datefmt: %b %d %H:%M:%S %Y [logging.subscribe] format: %(asctime)s (%(process)d) %(message)s [logging.subscribe] level: info [logging.subscribe] path: mailman.log [logging.subscribe] propagate: no [logging.vette] datefmt: %b %d %H:%M:%S %Y [logging.vette] format: %(asctime)s (%(process)d) %(message)s [logging.vette] level: info [logging.vette] path: mailman.log [logging.vette] propagate: no [mailman] cache_life: 7d [mailman] default_language: en [mailman] email_commands_max_lines: 10 [mailman] filtered_messages_are_preservable: no [mailman] html_to_plain_text_command: /usr/bin/lynx -dump $filename [mailman] layout: debian [mailman] listname_chars: [-_.0-9a-z] [mailman] noreply_address: noreply [mailman] pending_request_life: 3d [mailman] post_hook: [mailman] pre_hook: [mailman] sender_headers: from from_ reply-to sender [mailman] site_owner: max.paragnani(a)gmail.com [mta] configuration: python:mailman.config.postfix [mta] delivery_retry_period: 5d [mta] incoming: mailman.mta.postfix.LMTP [mta] lmtp_host: 127.0.0.1 [mta] lmtp_port: 8024 [mta] max_autoresponses_per_day: 10 [mta] max_delivery_threads: 0 [mta] max_recipients: 500 [mta] max_sessions_per_connection: 0 [mta] outgoing: mailman.mta.deliver.deliver [mta] remove_dkim_headers: no [mta] smtp_host: localhost [mta] smtp_pass: [mta] smtp_port: 25 [mta] smtp_user: [mta] verp_confirm_format: $address+$cookie [mta] verp_confirm_regexp: ^(.*<)?(?P<addr>[^+]+?)\+(?P<cookie>[^@]+)@.*$ [mta] verp_confirmations: no [mta] verp_delimiter: + [mta] verp_delivery_interval: 0 [mta] verp_format: ${bounces}+${local}=${domain} [mta] verp_personalized_deliveries: no [mta] verp_probe_format: $bounces+$token@$domain [mta] verp_probe_regexp: ^(?P<bounces>[^+]+?)\+(?P<token>[^@]+)@.*$ [mta] verp_probes: no [mta] verp_regexp: ^(?P<bounces>[^+]+?)\+(?P<local>[^=]+)=(?P<domain>[^@]+)@.*$ [nntp] host: [nntp] password: [nntp] port: [nntp] remove_headers: nntp-posting-host nntp-posting-date x-trace x-complaints-to xref date-received posted posting-version relay-version received [nntp] rewrite_duplicate_headers: To X-Original-To CC X-Original-CC Content-Transfer-Encoding X-Original-Content-Transfer-Encoding MIME-Version X-MIME-Version [nntp] user: [passwords] configuration: python:mailman.config.passlib [passwords] password_length: 8 [plugin.master] class: [plugin.master] component_package: [plugin.master] configuration: [plugin.master] enabled: no [shell] banner: Welcome to the GNU Mailman shell [shell] history_file: [shell] prompt: >>> [shell] use_ipython: no [styles] default: legacy-default [webservice] admin_pass: --------------------------------- [webservice] admin_user: restadmin [webservice] api_version: 3.1 [webservice] hostname: localhost [webservice] port: 8001 [webservice] show_tracebacks: yes [webservice] use_https: no

7 20

Re: Importing mbox files into archive defect with lines with From
by Alex Schuilenburg 21 Sep '23

21 Sep '23

Hi Stephen On 19/09/2023 19:59, Stephen J. Turnbull wrote: > Did you intend to send this to me only? Nope, apologies. I'm happy to add the list back in - I thought I hit reply-to-list. > Alex Schuilenburg writes: > > > Yes, I retested with from the git head as well. Same result. It > > sees the line > > > > From 1st April 2019, our turnover has exceeded the threshold for > > > > as the start of a new message and reports no anomolies although > > there is clearly one. > > If there is an empty line preceding that one, and the "F" is in the > first column (no marginal whitespace), that is considered correct > behavior for processing an mbox file. As an email separator agreed, provided the body has all "First " (preceded by a blank line and nothing preceding on the same line) suitably escaped. > If you don't like it, don't use the mbox format. The maildir > message-per-file format is widely supported, and if you want a > folder-per-file rather than message-per-file format, MMDF is fairly > widely supported, and does not suffer from the ambiguities of mbox. > All three (with variations) are supported by recent Python 3. Understood. My issue is the lists were imported from MM2.1 in 2020 into MM3.2.1 onto hyperkitty+mariadb/mysql. So the current installation has no mbox files - archives are stored in a mysql database. > > The old MM3 installation I am migrating away from is obviously > > broken in that its mbox archive when downloaded is not escaped, > > That's interesting. I presume that's because HyperKitty saves > messages in a database rather than as mbox files, and the old mbox > exporter just flattened and concatenated the messages. Or perhaps > that also uses mailbox.mbox and old enough versions of that didn't > From-stuff. Spot on. I have to move the lists onto a new Debian 12 server using the native mailman 3.3.8 & mailman-web 0+20200530-2 packages. I tried dropping in the old mailman3 database under the new software but that did not work. Instead I manually imported the old mailman3 data directly into the new mailman3 database as, after inspections, there were no new tables and only a couple of additional fields which have suitable defaults. So dumped the mailman3 data with mysqldump --no-create-info --no-create-db --disable-keys --complete-insert --ignore-table=mailman3.alembic_version mailman3 and simply imported the dump into the new schema. OK so far. The lists showed up in postorius, obviously without archives in hyperkitty. > The normal way to upgrade a HyperKitty archive is to do nothing, just > upgrade the software. I guess you moved to a new host and deleted the > database? The preferred way is to dump the database to SQL, and then > load it in to the new database directly rather than downloading the > mbox files and importing. No ambiguity and much faster. Thats what I thought initially, but that failed as per https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…. As my old installation appears to have the django_migrations table inconsistent with the state of the database, and Debian have been unresponsive so far. Unfortunately the same mailman3-style manual import of mailman3web from old mailman3web db into the new db was not possible. There were additional tables and fields that needed values rather than defaults. Instead I opted to download the mbox exports from the old installation via the web interface and import them into the new installation for expediency.This is where I was when I posted the message. However, the old installation's mbox archive export (from the database) was problematic (e.g. 5min timeout) but after getting around those issues I ended up with some broken mbox files (e.g. the "From 1st April ..." line within a message, resulting in hyperkitty_import failures). > > although I tested downloading this thread archive and it is clearly > > escaped. Unfortunately I cannot tell if hyperkitty_import works > > using the git head since I am restricted to using the version > > provided with Debian 12. > > In July it worked for me on a site with 5300 archived lists. I assume > some of them had From-stuffed lines. Can't be sure, most of the posts > are machine-generated, but lines beginning with "From " are pretty > common in natural English. I'd be surprised if there were not any. I have 2003 lists and several occurrences. > > > There are indeed two versions though of mailbox.py on the VM: > > > > /usr/lib/python3/dist-packages/mailman/utilities/mailbox.py: from > > mailman3 (3.3.8-2~deb12u1) > > /usr/lib/python3.11/mailbox.py from libpython3.11-stdlib (3.11.2-6) > > The first applies one small change to the second (a so-called > "monkey-patch") for use by Mailman core. It should only be visible > to Python by the name "mailman.utilities.mailbox". The HyperKitty > utilities import the name "mailbox". If overwriting the first with > the second changes HyperKitty's behavior, something is wrong with > sys.path. Then I guess that is the case in Debian 12. > > > the latter of which appears far more substantial , so I dropped > > that module over the former and reran hyperkitty_import over the > > '>' escaped mbox, and while it did import without error, it did > > leave the escape in place (i.e. the 'From ...' was quoted when > > viewed). > > This is considered correct behavior. It is not possible to determine > whether the escape was in the original, or added by a receiving MTA. > Better to leave it. I thought that ">From " would be escaped to ">>From ", and so on, so the escape could easily be reversed when imported. I tested exports from the mailman-users lists and lines beginning "From " (preceded by a blank line) are escaped to ">From ", so incorrectly figured this would be unescaped by hyperkitty_import. After all, I would expect that an export of the archive to mbox, followed by a delete of the archive, followed by a hyperkitty_import of the archive, should leave you at the same place. Not with ">From " escapes in the new archives. In fact I also had a number of messages with "Message-ID: <>" and worse: all messages with attachments had the text/plain content empty. So mbox exports from MM 3.2.1 on Debian 10 (using hyperkitty+mysql) are broken. > > > The unescaped mbox import died in the same way. > > As expected. > > > Anyhow, thanks for your suggestion. For now I can stick with a > > manual repair and spaced escape of From_. > > If the old database is still available, I recommend dumping that and > loading it into a fresh version of the DBMS. > ... Thanks for the pointer. As I had already done the same with mailman3, so repeated the excercise. The following dump and import worked. oldhost> mysqldump --no-create-info --no-create-db --disable-keys --complete-insert mailman3web > mailman3web.sql newhost> mysql MariaDB [(none]> use mailman3web MariaDB [mailman3web]> source mailman3web.sql Thanks again -- Alex

2 3