On 6/4/23 11:45, Joel Lord wrote:
So I seem to have problems with both periodic AND size-triggered digests. Size first, this is simpler. Just got one sent out, clearly triggered by a new email getting added to the queued up messages, but the setting for the list is the default 30KB and the digest came out at 23.2 KB. Is it measuring the accumulated queue as size on disk including metadata and filesystem block sizes or something, rather than the size of what the emailed digest will be?
The threshold is determined by the size in bytes of the MMDF format mailbox file in which the the messages for the digest are saved. This does not include metadata and is not affected by block sizes. It contains the raw messages with all headers and 8 bytes of message separators per message and a 'From ' line per message.
Once the digest is triggered, the messages are configured into a digest message with boilerplate, a TOC and possibly a header and/or footer added. The messages themselves may be added as separate MIME message/rfc822 parts to a mime format digest or added as text to a plain format digest. Many headers are removed as the messages are added. See <https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/config/schema.cfg#L1002> for what is kept. In addition any non text/plain parts in the message are not included in the plain format digest, but are replaced by notes that they have been removed. All this says that a MIME format may be larger or smaller than the threshold depending on the size of the removed headers vs. the added parts, and a plain text digest will probably be smaller if the messages have non text/plain parts, e.g. if content filtering doesn't collapse alternatives and leaves the text/html alternative part, this part will not be in a plain format digest.
Periodic is much more complicated, even to explain. I have the cron job running nightly at 11:00 PM and it certainly seems to be working, except for one really odd thing. The list also definitely has periodic digests enabled in the web UI. This is a very bursty list, so it will go mad with traffic for a few weeks and then will go mostly idle for a month or so. The first digest of a burst is definitely size-triggered, but pretty reliably it has the last few messages from the previous burst in there. So the one I got today had three messages from today and one message from May 4th. But that May 4th message should have gone out at 11:00 PM on May 4th with the daily digest and then gotten flushed from the pending digest queue, right?
That's correct. Do you ever get periodic digests sent at 11:00 PM? If not, there is some issue either with configuration or something. If you get them sometimes, I don't know what could be wrong.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan