On 12/14/22 03:31, Mark Sapiro wrote:
On 12/13/22 10:38, Dan Caballero wrote:
Hi Mark,
I ran the commands and it took about 35 seconds for the loop to run.
Here's the result.
count 4168
So that's at least part of the issue. How does 35 seconds compare to the length of time to process one moderated message through Postorius?
It seems it has to do with in-efficient lookup of the actual entry in the pendings table which needs to be purged after handling a held message. It scales linearly with how many entries are there in pendedkeyvalue and pendings tables.
I was actually browsing through the perf metrics on mail.python.org and found the slowest API response from Core is handling held messages and spent some time last night trying to improve it 1. The linked MR should significantly reduce the time, although there is more that can be done to make it even better.
I haven't had a chance to benchmark my solution yet, but I am hoping it will reduce the latency quite significantly. I'll report on the linked issue 2 after deploying 1 on m.p.o. how much it helped.
-- thanks, Abhilash Raj (maxking)