On Oct 19, 2021, at 7:37 PM, Stephen J. Turnbull stephenjturnbull@gmail.com wrote:
Bryan Loniewski writes:
We have noticed similar slowness when importing/creating/deleting lists. We have ~2600 lists and the bottleneck does seem to be the MTA alias code that reads from backend (sorts by domain and then list names?) and subsequently writes the (lmtp,vmap,etc) files.
Is there a GitLab issue for this? If not, will somebody file one for us? I'd do it but I don't really understand the issue and diagnosis except for S-L-O-W, that part I get.
I didn’t find any issue, so I created one
https://gitlab.com/mailman/mailman/-/issues/953
My understanding of the issue is that Mailman tries to create the LMTP files whenever a new list is created and in their setup, the LMTP files are on a remote volume. So the latency to write to LMTP file is significant. And when 2600 of those happen, it takes a lot of time.
I don’t think we can do anything about the MySQL latency though, since we have to create the lists. Although, these lists are separate rows in database, so it should be possible to do multiple imports in parallel by invoking the commands simultaneously.
We should be able to fix this by adding a flag that skips creation of LMTP files after creation/import of a list. Once the imports are complete, a single run of "mailman aliases” command should generate everything. Hyperkitty has a similar mechanism to avoid indexing emails when bulk importing emails from mboxes.
-- thanks, Abhilash Raj (maxking)