When creating a new list in Mailman 3, the Digest Size Threshold is set to 30KB, which is not much in my opnion.
- How do I set another default for Digest Size Threshold for new lists.
- How do I change the default for existing lists
When Digest Size Threshold is not reached, mails seems to be sent according to digest_volume_frequency': which is set to 'monthly'. The web interface doesn't seem to have a setting for this.
How do I change this setting, both on a single list and the default for new lists?
Henrik Rasmussen
On 9/21/18 6:21 AM, Henrik Rasmussen wrote:
When creating a new list in Mailman 3, the Digest Size Threshold is set to 30KB, which is not much in my opnion.
- How do I set another default for Digest Size Threshold for new lists.
It's hard coded in mailman/styles/base.py. It should be a config setting, but I think it's not.
- How do I change the default for existing lists
Postorius -> Settings -> Digest
When Digest Size Threshold is not reached, mails seems to be sent according to digest_volume_frequency': which is set to 'monthly'. The web interface doesn't seem to have a setting for this.
No. digest_volume_frequency only controls how often the digest volume number increments (and issue number resets).
In order to send periodic digests, you need to run 'mailman digests --periodic' via cron.
How do I change this setting, both on a single list and the default for new lists?
The only setting is the lists digest_send_periodic which again is hard coded (to True) in mailman/styles/base.py. For existing lists, it is exposed in REST, but not yet in Postorius.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
On 9/21/18 6:21 AM, Henrik Rasmussen wrote:
- How do I change the default for existing lists
Postorius -> Settings -> Digest
Sorry, I was unclear here. I was aware of changing the size in the web interface, but what I meant was changing the Digest Size Threshold value for all lists using REST API.
Is that possible?
-- Henrik Rasmussen
On 9/24/18 4:52 AM, Henrik Rasmussen wrote:
Mark Sapiro wrote:
On 9/21/18 6:21 AM, Henrik Rasmussen wrote:
- How do I change the default for existing lists
Postorius -> Settings -> Digest
Sorry, I was unclear here. I was aware of changing the size in the web interface, but what I meant was changing the Digest Size Threshold value for all lists using REST API.
Is that possible?
Probably not in a single REST interaction, but you could iterate over all lists.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
On 9/21/18 6:21 AM, Henrik Rasmussen wrote:
When Digest Size Threshold is not reached, mails seems to be sent according to digest_volume_frequency': which is set to 'monthly'. The web interface doesn't seem to have a setting for this.
No. digest_volume_frequency only controls how often the digest volume number increments (and issue number resets).
In order to send periodic digests, you need to run 'mailman digests --periodic' via cron.
Currently I am running Mailman version 3.2.0a1 which doesn't seem to have a --periodic option.
When running "docker exec mailman-core mailman digests --send --verbose -l listname@lists.example.com", output is "listname.lists.example.com sent volume 11, number 4" and this increases each time, but even I do approve held mails, none are sent (according to the smtp.log and docker logs posrfix).
As I understand, using --sent should send a digest immediately (except when used with --dry-run).
As the default Digest size threshold was 30KB, making very small digests being sent, I increased this (for test) to 5000 KB (or is it Kilo bits, as the text says "How big in Kb should a digest be before it gets sent out?"?
Could Digest size threshold have any impact on why no digests are sent even I run "docker exec mailman-core mailman digests --send --verbose -l listname@lists.example.com" or is this an issue in the version I am using?
-- Henrik Rasmussen
On 10/04/2018 12:19 AM, Henrik Rasmussen wrote:
Currently I am running Mailman version 3.2.0a1 which doesn't seem to have a --periodic option.
3.2.0a1 was never an 'official' release. It could be any HEAD version between June 4, 2017 and Jul 10, 2018. The --periodic option was added April 28, 2018.
When running "docker exec mailman-core mailman digests --send --verbose -l listname@lists.example.com", output is "listname.lists.example.com sent volume 11, number 4" and this increases each time, but even I do approve held mails, none are sent (according to the smtp.log and docker logs posrfix).
Are there accumulated messages for the digest when you run the command?. I.e., is there a var/lists/listname.lists.example.com/digest.mmdf file with messages in it? Are there list members receiving digests? If so, there should be something in Mailman's smtp.log.
As I understand, using --sent should send a digest immediately (except when used with --dry-run).
Correct,if there is one to send.
As the default Digest size threshold was 30KB, making very small digests being sent, I increased this (for test) to 5000 KB (or is it Kilo bits, as the text says "How big in Kb should a digest be before it gets sent out?"?
It is Kilobytes.
Could Digest size threshold have any impact on why no digests are sent even I run "docker exec mailman-core mailman digests --send --verbose -l listname@lists.example.com" or is this an issue in the version I am using?
The digest size threshold has no effect on whether or not a digest is sent by 'mailman digests'.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Having upgraded to Mailman version 3.2.0 (official version) on my testserver, I am still having problems with daily digests (not digests by file size).
My test list was set to 30 KB digests which worked fine and I received mails with (small) digests right after posting to the list. Having set the list to 500KB (to avoid immediate digests on posting) digests are piled up in the digest.mmdf file as expected.
But running docker exec mailman-core mailman digests --periodic --verbose still only renames the digest.mmdf to digest.[volume].[digest]. No file is sent according to the smtp.log (and my mail box) - see below.
Before running /usr/bin/docker exec mailman-core mailman digests --periodic --verbose
Every 2.0s: ls -l;grep "Test 20181011" digest.mmdf;tail /var/log/mailman/core/smtp.log Thu Oct 11 12:23:32 2018
-rw-rw---- 1 100 nogroup 103433 Oct 10 14:36 digest.1.7.mmdf -rw-rw---- 1 100 nogroup 155540 Oct 11 11:29 digest.1.8.mmdf -rw-rw---- 1 100 nogroup 104002 Oct 11 12:21 digest.mmdf Thread-Topic: Test 201810111220 Subject: [test] Test 201810111220 Thread-Topic: Test 201810111221 Subject: [test] Test 201810111221 Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'MAIL FROM:<me@foo.bar>' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) sender: me@foo.bar Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'RCPT TO:<test@lists.example.com>' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) recip: test@lists.example.com Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'DATA' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'QUIT' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) connection lost Oct 11 10:21:26 2018 (36) Connection lost during _handle_client() Oct 11 10:21:27 2018 (38) <6DCC3E5DA06FE346B4DE4876C4F2713D01E00C2C1B@P2KITMBX06WC03.unicph.domain> smtp to test @lists.example.com for 1 recips, completed in 0.16465544700622559 seconds Oct 11 10:21:27 2018 (38) <6DCC3E5DA06FE346B4DE4876C4F2713D01E00C2C1B@P2KITMBX06WC03.unicph.domain> post to test @lists.example.com from test@lists.example.com, 52588 bytes
After running /usr/bin/docker exec mailman-core mailman digests --periodic --verbose
Every 2.0s: ls -l;grep "Test 20181011" digest.mmdf;tail /var/log/mailman/core/smtp.log Thu Oct 11 12:44:45 2018
total 372 -rw-rw---- 1 100 nogroup 103433 Oct 10 14:36 digest.1.7.mmdf -rw-rw---- 1 100 nogroup 155540 Oct 11 11:29 digest.1.8.mmdf -rw-rw---- 1 100 nogroup 104002 Oct 11 12:21 digest.1.9.mmdf grep: digest.mmdf: No such file or directory Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'MAIL FROM:<her@adm.ku.dk>' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) sender: her@adm.ku.dk Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'RCPT TO:<hertest2@mm3test.ku.dk>' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) recip: hertest2@mm3test.ku.dk Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'DATA' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) Data: b'QUIT' Oct 11 10:21:26 2018 (36) ('172.19.199.5', 36282) connection lost Oct 11 10:21:26 2018 (36) Connection lost during _handle_client() Oct 11 10:21:27 2018 (38) <6DCC3E5DA06FE346B4DE4876C4F2713D01E00C2C1B@P2KITMBX06WC03.unicph.domain> smtp to hertest2 @mm3test.ku.dk for 1 recips, completed in 0.16465544700622559 seconds Oct 11 10:21:27 2018 (38) <6DCC3E5DA06FE346B4DE4876C4F2713D01E00C2C1B@P2KITMBX06WC03.unicph.domain> post to hertest2 @mm3test.ku.dk from hertest2@mm3test.ku.dk, 52588 bytes
Henrik Rasmussen
On 10/11/2018 03:48 AM, Henrik Rasmussen wrote:
Before running /usr/bin/docker exec mailman-core mailman digests --periodic --verbose
Every 2.0s: ls -l;grep "Test 20181011" digest.mmdf;tail /var/log/mailman/core/smtp.log Thu Oct 11 12:23:32 2018
-rw-rw---- 1 100 nogroup 103433 Oct 10 14:36 digest.1.7.mmdf -rw-rw---- 1 100 nogroup 155540 Oct 11 11:29 digest.1.8.mmdf -rw-rw---- 1 100 nogroup 104002 Oct 11 12:21 digest.mmdf
...
After running /usr/bin/docker exec mailman-core mailman digests --periodic --verbose
Every 2.0s: ls -l;grep "Test 20181011" digest.mmdf;tail /var/log/mailman/core/smtp.log Thu Oct 11 12:44:45 2018
total 372 -rw-rw---- 1 100 nogroup 103433 Oct 10 14:36 digest.1.7.mmdf -rw-rw---- 1 100 nogroup 155540 Oct 11 11:29 digest.1.8.mmdf -rw-rw---- 1 100 nogroup 104002 Oct 11 12:21 digest.1.9.mmdf
The mailman digests --periodic command renames the digest.mmdf file to (in this case) digest.1.9.mmdf and then queues an entry in the digest queue for DigestRunner to actually build the digests from the digest.1.9.mmdf file, send them and remove the digest.1.9.mmdf file.
Something is going wrong in this process in DigestRunner. Is DigestRunner running? Is there anything in Mailman's digest or shunt queues? Is there anything logged in Mailman's 'mailman' log?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thanks for the hints. It looks to be a file permission problem. I am not sure what the right persommions should be inside he container. Is there a command for checking file permissions, like the Mailman 2's command check_perms? I couldn't file one.
Mark Sapiro wrote:
Something is going wrong in this process in DigestRunner. Is DigestRunner running?
Yes: 43 mailman 2:05 /usr/local/bin/python /usr/local/bin/runner -C /etc/mailman.cfg --runner=digest:0:1
Is there anything in Mailman's digest or shunt queues?
As of /opt/mailman/core/var/queue/digest/ and /opt/mailman/core/var/queue/shunt/ no, but there is three files in /opt/mailman/core/var/queue/bad/
-rw-rw---- 1 root root 398 Oct 10 14:42 1539175326.8246331+4b7e80db0d22f01c2d4363e323320fcd34560657.psv -rw-rw---- 1 root root 398 Oct 11 12:11 1539252672.3947155+ae2631d9f6514d74c77a38f09f821ee71515f9d9.psv -rw-rw---- 1 root root 398 Oct 11 12:24 1539253469.5928977+15b0842462ddd3abcac8d688e5bbb195b07409bc.psv
They seems to be binary, buy strings reveal:
mailman.email.message Message policy email._policybase Compat32 _headers _unixfrom _payload _charset preamble epilogue defects _default_type text/plain listid test.lists.example.com digest_path =/opt/mailman/var/lists/test.lists.example.com/digest.1.9.mmdf volume digest_number version _parsemsg
If that reveals anything.
Is there anything logged in Mailman's 'mailman' log?
It seem to say PermissionError: [Errno 13] Permission denied, but inside the container, the directoried in /opt/mailman/var/queue is
drwxrwx--- 2 mailman root 4096 Mar 14 2018 archive drwxrwx--- 2 mailman root 4096 Oct 12 10:14 bad drwxrwx--- 2 mailman root 4096 Mar 14 2018 bounces drwxrwx--- 2 mailman root 4096 Mar 14 2018 command drwxrwx--- 2 mailman root 4096 Oct 11 10:24 digest drwxrwx--- 2 mailman root 4096 Oct 12 10:07 in drwxrwx--- 2 mailman root 4096 Mar 14 2018 nntp drwxrwx--- 2 mailman root 4096 Oct 12 10:07 out drwxrwx--- 2 mailman root 4096 Oct 12 10:07 pipeline drwxrwx--- 2 mailman root 4096 Mar 26 2018 retry drwxrwx--- 2 mailman root 4096 Mar 14 2018 shunt drwxrwx--- 2 mailman root 4096 Oct 10 12:32 virgin
Henrik Rasmussen
On 10/12/2018 03:46 AM, Henrik Rasmussen wrote:
As of /opt/mailman/core/var/queue/digest/ and /opt/mailman/core/var/queue/shunt/ no, but there is three files in /opt/mailman/core/var/queue/bad/
-rw-rw---- 1 root root 398 Oct 10 14:42 1539175326.8246331+4b7e80db0d22f01c2d4363e323320fcd34560657.psv -rw-rw---- 1 root root 398 Oct 11 12:11 1539252672.3947155+ae2631d9f6514d74c77a38f09f821ee71515f9d9.psv -rw-rw---- 1 root root 398 Oct 11 12:24 1539253469.5928977+15b0842462ddd3abcac8d688e5bbb195b07409bc.psv
They seems to be binary, ...
They are Python pickles. The fact that they are owned by 'root' probably means 'mailman digest' was run as 'root'. it should be run as 'mailman'. The files are preserved 'unparseable' queue files - probably unparseable because they can't be read because of the permissions error.
You can view them with 'mailman qfile'. I expect this will show an 'empty' message and metadata containing in this case
listid: 'test.lists.example.com' digest_path: '/opt/mailman/var/lists/test.lists.example.com/digest.1.9.mmdf' volume: 1 digest_number: 9
and a few other things like version and _parsemsg
This is the 'digest' queue entry that tells DigestRunner to produce digests from the file in digest_path.
It seem to say PermissionError: [Errno 13] Permission denied, but inside the container, the directoried in /opt/mailman/var/queue is
The permissions error is possibly on these files:
-rw-rw---- 1 100 nogroup 103433 Oct 10 14:36 digest.1.7.mmdf -rw-rw---- 1 100 nogroup 155540 Oct 11 11:29 digest.1.8.mmdf -rw-rw---- 1 100 nogroup 104002 Oct 11 12:21 digest.1.9.mmdf
or on the digest queue entries and it is because 'mailman digests' is not being run as the mailman user.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Henrik Rasmussen -
Mark Sapiro