Pck files stuck in out queue

older
Postorius extremely slow on some...

Lieuallen, Thomas Otis

Feb. 6, 2025

8:42 p.m.

Running Mailman 3.3.10

I have 11x pck files stuck in my out queue. There tends to always be a .bak file as well. If I run ‘ls -l’, the files are continually updated and rotated through, just as fast as I can look.

The out runner is consuming copious CPU time.

I was hoping this would help:

[logging.runner]

level: DEBUG

But, if so, I don’t see the logs anywhere.

I believe they’re all for one list. When I use ‘strings’ on one to get a hint of what’s inside, I see the list members at the bottom (recipients). One of them consistently has ‘!’ before their email address. Not sure if that’s a clue. Just no other reason to suspect that, though.

Ideas on where/how to track down the status of these files and fix whatever’s broken?

Thank you

Tom Lieuallen Oregon State University

Show replies by date

Mark Sapiro

February 2025

10:21 p.m.

On 2/6/25 12:42, Lieuallen, Thomas Otis via Mailman-users wrote:

...

Running Mailman 3.3.10

I have 11x pck files stuck in my out queue. There tends to always be a .bak file as well. If I run ‘ls -l’, the files are continually updated and rotated through, just as fast as I can look.

...

I believe they’re all for one list. When I use ‘strings’ on one to get a hint of what’s inside, I see the list members at the bottom (recipients). One of them consistently has ‘!’ before their email address. Not sure if that’s a clue. Just no other reason to suspect that, though.

The files are python pickles and strings shows lots of stuff. The ! may be something in the pickle that's not part of the email address. The way to examine these files is with the mailman qfile command.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Lieuallen, Thomas Otis

11:21 p.m.

Mark,

There were no apparent permissions problems on templates. I was getting the ‘Cannot connect to SMTP server’ message in mailman.log.

I did end up applying the patch to decorate.py and messages went through. I’ve since removed the patch, as I’m still looking for an answer and might as well let it queue up again.

The only thing that is making me ponder at the moment is that the list owners setup a template for list:member:regular:footer in postorius. It appears that each list has a copy of that file. The modifications I see in postorius are not in their …/templates/list/<listname>/en/list:member:regular:footer.txt file. Are their modifications stored somewhere else and causing the problem?

Thx

Tom Lieuallen

From: Mark Sapiro <mark@msapiro.net> Date: Thursday, February 6, 2025 at 2:22 PM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Pck files stuck in out queue [This email originated from outside of OSU. Use caution with links and attachments.]

On 2/6/25 12:42, Lieuallen, Thomas Otis via Mailman-users wrote:

...

Running Mailman 3.3.10

I have 11x pck files stuck in my out queue. There tends to always be a .bak file as well. If I run ‘ls -l’, the files are continually updated and rotated through, just as fast as I can look.

...

I believe they’re all for one list. When I use ‘strings’ on one to get a hint of what’s inside, I see the list members at the bottom (recipients). One of them consistently has ‘!’ before their email address. Not sure if that’s a clue. Just no other reason to suspect that, though.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

This message sent to lieualth@oregonstate.edu

Lieuallen, Thomas Otis

11:26 p.m.

I ended up emailing the list. That email got stuck as well. So, I removed their template – and the message went through. So, that appears to have been the problem. But, not sure why.

Thank you

Tom Lieuallen

From: Lieuallen, Thomas Otis via Mailman-users <mailman-users@mailman3.org> Date: Thursday, February 6, 2025 at 3:25 PM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Pck files stuck in out queue [This email originated from outside of OSU. Use caution with links and attachments.]

Mark,

There were no apparent permissions problems on templates. I was getting the ‘Cannot connect to SMTP server’ message in mailman.log.

I did end up applying the patch to decorate.py and messages went through. I’ve since removed the patch, as I’m still looking for an answer and might as well let it queue up again.

Thx

Tom Lieuallen

On 2/6/25 12:42, Lieuallen, Thomas Otis via Mailman-users wrote:

...

Running Mailman 3.3.10

I have 11x pck files stuck in my out queue. There tends to always be a .bak file as well. If I run ‘ls -l’, the files are continually updated and rotated through, just as fast as I can look.

Are old ones being delivered and new ones added or the same ones just being renamed? I suspect the latter and I suspect the issue in that case is probably due to the OUT runner trying to access a template file like list:member:{regular|generic}:{header|footer}.txt and not having permission. See the last two posts in the thread at https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Farchives%2Flist%2Fmailman-users%40mailman3.org%2Fthread%2FZFJ6WAXV536VXKOPISGLHCPIP6N4T3AZ%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7Cbc02eb34fa994402707808dd470516ea%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638744811157884212%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=aaaR4UnBA7cQfjoHOdOjTst17hjB3t7jfrWFOX2v9Iw%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Farchives%2Flist%2Fmailman-users%40mailman3.org%2Fthread%2FZFJ6WAXV536VXKOPISGLHCPIP6N4T3AZ%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7Cbc02eb34fa994402707808dd470516ea%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638744811157902439%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=aJVVpw2q72jVMKROIMHHDdJXBQHLmQIwehJS%2F4Taa9I%3D&reserved=0><https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/ZFJ6WAXV536VXKOPISGLHCPIP6N4T3AZ/>

...

I believe they’re all for one list. When I use ‘strings’ on one to get a hint of what’s inside, I see the list members at the bottom (recipients). One of them consistently has ‘!’ before their email address. Not sure if that’s a clue. Just no other reason to suspect that, though.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7Cbc02eb34fa994402707808dd470516ea%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638744811157914115%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=OroRtZTpDMM2LD8DI7FFDPNqQq3NOlp%2FldjLEUsZFZE%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7Cbc02eb34fa994402707808dd470516ea%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638744811157925587%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=p2WKIsm%2BmxaCWpFzYqN3PyZOaMUsa5wxQvPFR7a7Q00%3D&reserved=0><https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/> Archived at: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Farchives%2Flist%2Fmailman-users%40mailman3.org%2Fmessage%2FXU4GDRULTDG26A2LOOVACKWEYUADMTUB%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7Cbc02eb34fa994402707808dd470516ea%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638744811157937306%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=fI8rks%2FLXjONJs1vCSzU3jObtD3plWHJVOp7yagmztE%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Farchives%2Flist%2Fmailman-users%40mailman3.org%2Fmessage%2FXU4GDRULTDG26A2LOOVACKWEYUADMTUB%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7Cbc02eb34fa994402707808dd470516ea%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638744811157948788%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=vZobGt%2FPZ75yDnOqxmGiCoAMYSbva3hYCG8NgcP5U6Q%3D&reserved=0><https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/XU4GDRULTDG26A2LOOVACKWEYUADMTUB/>

This message sent to lieualth@oregonstate.edu

Mark Sapiro

11:46 p.m.

On 2/6/25 15:21, Lieuallen, Thomas Otis via Mailman-users wrote:

...

Mark,

There were no apparent permissions problems on templates. I was getting the ‘Cannot connect to SMTP server’ message in mailman.log.

I did end up applying the patch to decorate.py and messages went through. I’ve since removed the patch, as I’m still looking for an answer and might as well let it queue up again.

Per the above, template access is definitely the problem.

...

The only thing that is making me ponder at the moment is that the list owners setup a template for list:member:regular:footer in postorius. It appears that each list has a copy of that file. The modifications I see in postorius are not in their …/templates/list/<listname>/en/list:member:regular:footer.txt file. Are their modifications stored somewhere else and causing the problem?

Templates set in Postorius are stored in the database in the postorius_emailtemplate table. At the same time an entry is made in the template table with a uri to retrieve the template from a Postorius API. The uri is something like http(s)://host/mailman3/api/templates/list/LIST-ID/list:member:regular:footer where the http(s)://host part is from the setting for POSTORIUS_TEMPLATE_BASE_URL.

The issue is that uri is not retrievable probably because the setting for POSTORIUS_TEMPLATE_BASE_URL is wrong.

You should look at the uris in the template table in the database and figure out what they should be, probably by trying to retrieve one in a browser with the scheme and host what you actually use to access Postorius. Once you figure out what the correct uri is you can set POSTORIUS_TEMPLATE_BASE_URL appropriately, but this won't fix the existing template table entries. You can fix those manually in the database or you can delete and recreate the templates in Postorius after fixing POSTORIUS_TEMPLATE_BASE_URL and restarting the Mailman web service.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Lieuallen, Thomas Otis

6:08 p.m.

Mark,

You were (of course), absolutely on target, thank you!

I have a remaining issue that baffles me. With my test list, I created a template for ‘list:member:regular:header’. It just has text in it, keeping it simple. I create it and see the entries in the database tables and they are correct. However, when I send a test email, my template text is not inserted, but instead another simple text: “testing hello”. I do the same with list:member:regular:footer, and it works just fine. It is just the header that is working incorrectly. When I remove that postorius header template for the list, that ‘testing hello’ goes away. So, it’s only appearing when I have a postorius template setup for that list.

I have searched on the file system for any such templates and only see the default ones; all zero length files. There are no templates defined for the domain in postorius – but if there were, I’d expect to see that test header all the time.

I’m just baffled. Any clues?

I’m a relative newbie to mailman3. When I first set it up, I tried the postorius templates for the domain. They didn’t work (had to have been due to the POSTORIUS_TEMPLATE_BASE_URL being wrong. So, I scrapped that and went with templates in the file system.

Thank you

Tom Lieuallen

From: Mark Sapiro <mark@msapiro.net> Date: Thursday, February 6, 2025 at 3:46 PM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Pck files stuck in out queue [This email originated from outside of OSU. Use caution with links and attachments.]

On 2/6/25 15:21, Lieuallen, Thomas Otis via Mailman-users wrote:

...

Mark,

There were no apparent permissions problems on templates. I was getting the ‘Cannot connect to SMTP server’ message in mailman.log.

I did end up applying the patch to decorate.py and messages went through. I’ve since removed the patch, as I’m still looking for an answer and might as well let it queue up again.

Per the above, template access is definitely the problem.

...

The only thing that is making me ponder at the moment is that the list owners setup a template for list:member:regular:footer in postorius. It appears that each list has a copy of that file. The modifications I see in postorius are not in their …/templates/list/<listname>/en/list:member:regular:footer.txt file. Are their modifications stored somewhere else and causing the problem?

The issue is that uri is not retrievable probably because the setting for POSTORIUS_TEMPLATE_BASE_URL is wrong.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

This message sent to lieualth@oregonstate.edu

Mark Sapiro

6:36 p.m.

On 2/7/25 10:08, Lieuallen, Thomas Otis via Mailman-users wrote:

...

I have a remaining issue that baffles me. With my test list, I created a template for ‘list:member:regular:header’. It just has text in it, keeping it simple. I create it and see the entries in the database tables and they are correct. However, when I send a test email, my template text is not inserted, but instead another simple text: “testing hello”. I do the same with list:member:regular:footer, and it works just fine. It is just the header that is working incorrectly. When I remove that postorius header template for the list, that ‘testing hello’ goes away. So, it’s only appearing when I have a postorius template setup for that list.

What do you get from the following database queries?

SELECT * FROM template WHERE name = 'list:member:regular:header'; SELECT * FROM postorius_emailtemplate WHERE name = 'list:member:regular:header';

Depending on your installation, these tables may be in separate databases, i.e., template table in mailman db and postorius_emailtemplate in mailmanweb db.

And what do you get if you go to the uri in the result(s) from the first query?

The context in the first results and maybe the second results should also be a clue.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Lieuallen, Thomas Otis

9:42 p.m.

Mark,

I checked the database; nothing there. Note that there are only 3x entries and only one is list:member:regular:header (my test), so just nothing of interest. The URL works and produces the proper template information.

I finally did a find/grep in the mailman tree and found a match for this errant string. It is under the cache folder. This corresponds to the cache_dir value. I could wipe the file out, but what is the proper way of doing so and should they be cleaned out via a schedule? These don’t appear to be web related, as far as I can tell.

Thank you

Tom Lieuallen

From: Mark Sapiro <mark@msapiro.net> Date: Friday, February 7, 2025 at 10:37 AM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Pck files stuck in out queue [This email originated from outside of OSU. Use caution with links and attachments.]

On 2/7/25 10:08, Lieuallen, Thomas Otis via Mailman-users wrote:

...

I have a remaining issue that baffles me. With my test list, I created a template for ‘list:member:regular:header’. It just has text in it, keeping it simple. I create it and see the entries in the database tables and they are correct. However, when I send a test email, my template text is not inserted, but instead another simple text: “testing hello”. I do the same with list:member:regular:footer, and it works just fine. It is just the header that is working incorrectly. When I remove that postorius header template for the list, that ‘testing hello’ goes away. So, it’s only appearing when I have a postorius template setup for that list.

What do you get from the following database queries?

SELECT * FROM template WHERE name = 'list:member:regular:header'; SELECT * FROM postorius_emailtemplate WHERE name = 'list:member:regular:header';

Depending on your installation, these tables may be in separate databases, i.e., template table in mailman db and postorius_emailtemplate in mailmanweb db.

And what do you get if you go to the uri in the result(s) from the first query?

The context in the first results and maybe the second results should also be a clue.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

This message sent to lieualth@oregonstate.edu

Lieuallen, Thomas Otis

8:09 p.m.

I’m still wondering about mailman’s cache (cache_dir setting). I have 2x cache files under it that are 3-4 days old. These are mailman cache files, not mailman-web.

# pwd

/opt/mailman/mm

# find cache -type f -ls

38627053 4 -rw-rw---- 1 mailman mailman 13 Feb 6 16:42 cache/26/26/2626c32ded811be4e9631d2215f32d88522b67142bf0eea502362d2b15c4743e

118392 4 -rw-rw---- 1 mailman mailman 11 Feb 7 09:42 cache/98/14/9814e4457e2489e2ec77d19ce982d3643fac3ad849c9861a9198eebf53f1fcde

I haven’t figured out how to work with these cache files. I found some mention that a runner would clean up cache files after a week (default) and that one could use ‘cache_life’ to change that default. So, I set the following and restarted mailman. The cache files hadn’t been removed after > 4 hours, so I unset it and restarted again.

Cache_life: 1d

Without further information or direction, I’m tempted to just remove those cache files and move on with life.

Thank you

Tom Lieuallen

From: Lieuallen, Thomas Otis via Mailman-users <mailman-users@mailman3.org> Date: Friday, February 7, 2025 at 1:43 PM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Pck files stuck in out queue [This email originated from outside of OSU. Use caution with links and attachments.]

Mark,

Thank you

Tom Lieuallen

On 2/7/25 10:08, Lieuallen, Thomas Otis via Mailman-users wrote:

...

I have a remaining issue that baffles me. With my test list, I created a template for ‘list:member:regular:header’. It just has text in it, keeping it simple. I create it and see the entries in the database tables and they are correct. However, when I send a test email, my template text is not inserted, but instead another simple text: “testing hello”. I do the same with list:member:regular:footer, and it works just fine. It is just the header that is working incorrectly. When I remove that postorius header template for the list, that ‘testing hello’ goes away. So, it’s only appearing when I have a postorius template setup for that list.

What do you get from the following database queries?

SELECT * FROM template WHERE name = 'list:member:regular:header'; SELECT * FROM postorius_emailtemplate WHERE name = 'list:member:regular:header';

Depending on your installation, these tables may be in separate databases, i.e., template table in mailman db and postorius_emailtemplate in mailmanweb db.

And what do you get if you go to the uri in the result(s) from the first query?

The context in the first results and maybe the second results should also be a clue.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7C77d8ba7740ca48f3aae908dd47c07c83%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638745614241674450%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=XvnqWtxV2HXnNiEBr%2Bkkv%2FBO%2BJEsfL%2BKtLDrenZQoA8%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Fmailman3%2Flists%2Fmailman-users.mailman3.org%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7C77d8ba7740ca48f3aae908dd47c07c83%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638745614241694316%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=oAWbbLnlSsIak0z2joXU4qQarF5ic7DBOgwRDOtVlfY%3D&reserved=0><https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/> Archived at: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Farchives%2Flist%2Fmailman-users%40mailman3.org%2Fmessage%2FFVEPCQHV5N4M536OA42QCX7UMMR37H3X%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7C77d8ba7740ca48f3aae908dd47c07c83%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638745614241707007%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=JR%2FgWDWy20sD3021nvOyVA4xeSrpRuX0MTbcB3eesrg%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mailman3.org%2Farchives%2Flist%2Fmailman-users%40mailman3.org%2Fmessage%2FFVEPCQHV5N4M536OA42QCX7UMMR37H3X%2F&data=05%7C02%7CTom.Lieuallen%40oregonstate.edu%7C77d8ba7740ca48f3aae908dd47c07c83%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638745614241719500%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=dnDD7DBvbfH8tH3%2Fn2ivaJQsmaDNLlkG0r%2BmjdGIJGE%3D&reserved=0><https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/FVEPCQHV5N4M536OA42QCX7UMMR37H3X/>

This message sent to lieualth@oregonstate.edu

Mark Sapiro

8:45 p.m.

On 2/10/25 12:09, Lieuallen, Thomas Otis via Mailman-users wrote:

...

I haven’t figured out how to work with these cache files. I found some mention that a runner would clean up cache files after a week (default) and that one could use ‘cache_life’ to change that default. So, I set the following and restarted mailman. The cache files hadn’t been removed after > 4 hours, so I unset it and restarted again.

Cache_life: 1d

The setting is actually cache_lifetime in the [mailman] section of mailman.cfg, however when a file is cached, it's lifetime is saved in the database and won't be changed by changing that setting.

...

Without further information or direction, I’m tempted to just remove those cache files and move on with life.

Removing them is OK, but a database entry will still exist pointing to them and attempt to retrieve them will return Cache content lost. The way to properly remove them is in Mailman shell

$ mailman shell
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.

 >>> cm = getUtility(ICacheManager)
 >>> cm.clear()
 >>> commit()
 >>>

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Lieuallen, Thomas Otis

9:45 p.m.

Mark,

Actually, this is saying cache_life under mailman:

https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/config/docs...

There is a cache_lifetime under ‘dmarc’.

I used your clue about the database to concoct my own solution… 😊. I edited the ‘expires_on’ values in the file_cache table in the database so the files were expired.

It looks like the times in that table are UTC.

I’m going to ratchet down my cache_life so anyone that is playing with templates doesn’t go bonkers (like me) with why their changes aren’t honored. I have a small site, so not much performance impact.

Thank you for your help.

Tom Lieuallen

From: Mark Sapiro <mark@msapiro.net> Date: Monday, February 10, 2025 at 12:46 PM To: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Pck files stuck in out queue [This email originated from outside of OSU. Use caution with links and attachments.]

On 2/10/25 12:09, Lieuallen, Thomas Otis via Mailman-users wrote:

...

I haven’t figured out how to work with these cache files. I found some mention that a runner would clean up cache files after a week (default) and that one could use ‘cache_life’ to change that default. So, I set the following and restarted mailman. The cache files hadn’t been removed after > 4 hours, so I unset it and restarted again.

Cache_life: 1d

...

Without further information or direction, I’m tempted to just remove those cache files and move on with life.

Removing them is OK, but a database entry will still exist pointing to them and attempt to retrieve them will return Cache content lost. The way to properly remove them is in Mailman shell

$ mailman shell
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.

 >>> cm = getUtility(ICacheManager)
 >>> cm.clear()
 >>> commit()
 >>>

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

This message sent to lieualth@oregonstate.edu

Mark Sapiro

11:03 p.m.

On 2/10/25 13:45, Lieuallen, Thomas Otis via Mailman-users wrote:

...

Mark,

Actually, this is saying cache_life under mailman:

https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/config/docs...

There is a cache_lifetime under ‘dmarc’.

You are correct. I was confused by the dmarc one.

...

I used your clue about the database to concoct my own solution… 😊. I edited the ‘expires_on’ values in the file_cache table in the database so the files were expired.

It looks like the times in that table are UTC.

I’m going to ratchet down my cache_life so anyone that is playing with templates doesn’t go bonkers (like me) with why their changes aren’t honored. I have a small site, so not much performance impact.

That really shouldn't be necessary, but I think there is an issue in the code at https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/model/template.... which is intended to evict the cache entry for the previous template when a new one is set. If the previous template has a different uri from the one being set, e.g., a mailman:// uri is being replaced by a http(s):// Postorius uri the code will attempt to evict the Postorius uri which is not the cached value.

This should not be an issue when just updating a template already in Postorius and should not be an issue when setting a Postorius template to replace one previously the default or in Mailman's var/templates directory as these normally don't have template table entries, but prior to Mailman 3.3.6, the mailman import21 command would create template table entries for imported lists. This was https://gitlab.com/mailman/mailman/-/issues/988 fixed by https://gitlab.com/mailman/mailman/-/merge_requests/989

...

Thank you for your help.

You're welcome. I'm glad you solved it.

Stephen J. Turnbull

1:07 a.m.

Lieuallen, Thomas Otis via Mailman-users writes:

...

There were no apparent permissions problems on templates.

As Mark writes, there may be a mismatch with POSTORIUS_TEMPLATE_BASE_URL.

But a permissions problem need not be on the templates themselves. It can be directory search access for the 'mailman' user (often 'o-x' where the directory is owned by root) for one or more directories in the path to the template.

...

I was getting the ‘Cannot connect to SMTP server’ message in mailman.log.

I believe this is unrelated, but you should look into that.

Steve

Mark Sapiro

1:45 a.m.

On 2/6/25 17:07, Stephen J. Turnbull wrote:

...

Lieuallen, Thomas Otis via Mailman-users writes:

...
I was getting the ‘Cannot connect to SMTP server’ message in mailman.log.

I believe this is unrelated, but you should look into that.

It is definitely related. That (very uninformative) message is logged once only after the out runner is started the first time calls the decorate handler and the handler can't access a template.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Stephen J. Turnbull

6:48 a.m.

Mark Sapiro writes:

...

It is definitely related. That (very uninformative) message is logged once only after the out runner is started the first time calls the decorate handler and the handler can't access a template.

So this socket.error is an escape from template.py:TemplateLoader (most likely bubbling up from template.py:TemplateManager)? I'd like to catch this error and do something useful with it.

Steve

Age (days ago)

Last active (days ago)

List overview

Download

14 comments

3 participants

participants (3)

Lieuallen, Thomas Otis
Mark Sapiro
Stephen J. Turnbull

Pck files stuck in out queue

tags

participants (3)