Remove old bounces in bouncedir
Hello all,
our Mailman3 has over 7GB of bounces in its bouncedir. I noticed that some messages there are already over 3 years old. Now my question: Is there a way to remove the old bounces?
On 3/9/22 00:58, Stephan Krinetzki wrote:
Hello all,
our Mailman3 has over 7GB of bounces in its bouncedir. I noticed that some messages there are already over 3 years old. Now my question: Is there a way to remove the old bounces?
I don't know what you mean by bouncedir.
If you are referring to the bounceevent table in the database, it would
be OK to remove all the entries for which processed is True.
If you are referring to messages in Mailman's bounces queue, if these are not processed and removed within 15 minutes of arrival, Mailman's bounce runner is not running.
If you are referring to bounce messages in the message store (var/messages/), there have been issues with those not being removed, but Mailman core >= 3.3.5 should clean those up.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
On 3/9/22 00:58, Stephan Krinetzki wrote:
Hello all, our Mailman3 has over 7GB of bounces in its bouncedir. I noticed that some messages there are already over 3 years old. Now my question: Is there a way to remove the old bounces? I don't know what you mean by bouncedir. If you are referring to the bounceevent table in the database, it would be OK to remove all the entries for which processed is True.
Ok, so i can remove there entries which are processed. For a smaller database ;)
If you are referring to messages in Mailman's bounces queue, if these are not processed and removed within 15 minutes of arrival, Mailman's bounce runner is not running.
Well, it is the queue.Our var directory is under /opt/mailman/var and the /opt/mailman/var/queue/bounces is over 8GB big. There are *.pck messages and some of them are very old (>3 Years). The bounce seems to be running:
Mar 10 09:24:53 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:00 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:06 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:12 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:18 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:24 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:30 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:36 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:43 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
Mar 10 09:25:49 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
The process it self is there
mailman 376 99.4 19.1 6578312 6279000 ? R 09:17 10:52 venv/bin/python3 venv/bin/runner -C mailman.cfg --runner=bounces:0:1
(Pathes are shorted)
So i guess, there is everything fine. But the files are not deleted.
If you are referring to bounce messages in the message store (var/messages/), there have been issues with those not being removed, but Mailman core >= 3.3.5 should clean those up.
SInce the messages are hashed or otherwise stored, i can't verify or deny this.
On 3/10/22 00:31, Stephan Krinetzki wrote:
Mark Sapiro wrote:
If you are referring to messages in Mailman's bounces queue, if these are not processed and removed within 15 minutes of arrival, Mailman's bounce runner is not running.
Well, it is the queue.Our var directory is under /opt/mailman/var and the /opt/mailman/var/queue/bounces is over 8GB big. There are *.pck messages and some of them are very old (>3 Years). The bounce seems to be running:
Mar 10 09:24:53 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:00 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:06 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:12 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:18 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:24 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:30 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:36 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:43 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:49 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.The process it self is there
mailman 376 99.4 19.1 6578312 6279000 ? R 09:17 10:52 venv/bin/python3 venv/bin/runner -C mailman.cfg --runner=bounces:0:1
I don't understand what's happening. If the bounces runner is processing it's queue, those .pck files should be processed and deleted.
Are there errors in /opt/mailman/var/logs/mailman?
If you examine (some of) those files with mailman qfile do you see
anything strange?
Is there an ownership/permissions issue? Sometimes people run Mailman or
some Mailman command as root and things get owned by root and then
when Mailman is later run as mailman it can't delete or maybe even
read them.
If you are referring to bounce messages in the message store (var/messages/), there have been issues with those not being removed, but Mailman core >= 3.3.5 should clean those up.
SInce the messages are hashed or otherwise stored, i can't verify or deny this.
Is the size of the /opt/mailman/var/messages directory very big?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
On 3/10/22 00:31, Stephan Krinetzki wrote:
Mark Sapiro wrote: If you are referring to messages in Mailman's bounces queue, if these are not processed and removed within 15 minutes of arrival, Mailman's bounce runner is not running. Well, it is the queue.Our var directory is under /opt/mailman/var and the /opt/mailman/var/queue/bounces is over 8GB big. There are *.pck messages and some of them are very old (>3 Years). The bounce seems to be running: Mar 10 09:24:53 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:00 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:06 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:12 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:18 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:24 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:30 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:36 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:43 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today. Mar 10 09:25:49 2022 (376) Member xxx@xxx already scored a bounce on list list.lists.example.com today.
The process it self is there mailman 376 99.4 19.1 6578312 6279000 ? R 09:17 10:52 venv/bin/python3 venv/bin/runner -C mailman.cfg --runner=bounces:0:1
I don't understand what's happening. If the bounces runner is processing it's queue, those .pck files should be processed and deleted. Are there errors in /opt/mailman/var/logs/mailman? Nope. Only the daily info, that it runs. For today:
Mar 14 03:15:34 2022 (21507) Task runner evicted 25336 expired pendings
Mar 14 03:15:43 2022 (21507) Task runner deleted 3696 orphaned workflows
Mar 14 03:16:09 2022 (21507) Task runner deleted 7965 orphaned requestsSeems to be okay
If you examine (some of) those files with mailman qfile do you see anything strange?
Nope. Just regular Bounce Files.
Is there an ownership/permissions issue? Sometimes people run Mailman or some Mailman command as root and things get owned by root and then when Mailman is later run as mailman it can't delete or maybe even read them.
Nope:
/opt/mailman/var/queue/bounces
ls -altrh | head
total 12G
-rw-rw---- 1 mailman mailman 37K Jan 5 09:51 1641372708.3212914+fc901998d5e2ace166db31ace6d77081b0c7650c.pck
-rw-rw---- 1 mailman mailman 42K Jan 5 09:52 1641372768.1659493+7afbcddc270547485732252874adcc27e04c9983.pck
-rw-rw---- 1 mailman mailman 42K Jan 5 09:52 1641372768.2633681+73c527870449492689fd30502b73f455a191701e.pck
-rw-rw---- 1 mailman mailman 7.7K Jan 5 09:58 1641373138.119037+2d5dd400e989db3303517b900d7f2b312d4cd7b5.pck
-rw-rw---- 1 mailman mailman 7.7K Jan 5 09:58 1641373139.585048+ee824dec757735ef8bd8139ee0cd858cc7c92606.pck
-rw-rw---- 1 mailman mailman 288K Jan 5 09:58 1641373139.846666+d75af26011eb1a8ef5673d9ee13c5425daec2bd4.pck
-rw-rw---- 1 mailman mailman 288K Jan 5 09:59 1641373140.024808+be60d1d97590d77eaec46751524d1bfe7a2f9ee7.pck
-rw-rw---- 1 mailman mailman 289K Jan 5 09:59 1641373140.8432257+678f545d176008a5483b8bc6747aebc2b84a7f4e.pck
-rw-rw---- 1 mailman mailman 288K Jan 5 09:59 1641373141.505824+c253072ebeb3f07b2b455725ace225abdd621b22.pckAs you can see, there are old messages (not 3 years old, but some months old)
If you are referring to bounce messages in the message store (var/messages/), there have been issues with those not being removed, but Mailman core >= 3.3.5 should clean those up. SInce the messages are hashed or otherwise stored, i can't verify or deny this. Is the size of the /opt/mailman/var/messages directory very big?
Nope, it's 1.9GB at the moment. I don't think that's big.
Is there anyway to run the bounce runner manually and get the debug output of it?
On 3/14/22 02:02, Stephan Krinetzki wrote:
Is there anyway to run the bounce runner manually and get the debug output of it?
Set
[logging.debug]
level: debug
in mailman.cfg. This will write lots of debug messages to /opt/mailman/var/logs/debug.log.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, hi everyone,
small update on the problem: it was not a problem at all, rather a temporary problem. The bounce runner was running and the last weekend has apparently led to the final processing of all the bounces. The directory is now as empty as it should be, the users were notified accordingly or removed from the lists and everything goes its way. It took Mailman over 6 months to process all the bounces!
TL;DR:
Just stay calm and wait, the system heals itself.
participants (2)
-
Mark Sapiro -
Stephan Krinetzki