Thanks for some clarification.
I cleared out 2800 held messages via Python shell "mailman -l" option and it took about 28 hours. Anything done via Mailman shell is a connection from Mailman core to the database, correct?
Although I can't replicate that many messages via Postorius/DJango, I do see similar slowness there as well at a message per minute or more of processing.
I recreated the above Mailman shell clean up via Docker container running MariaDB and noticed similar processing time for discarding held messages. That leads me to conclude that the issue isn't network related or specific to AWS RDS.
I'm open to testing and switching over to Postgres if that is the straightforward solution.
Dan Caballero Systems Administrator Academic Computing Solutions IMSS - Caltech https://imss.caltech.edu
From: Stephen J. Turnbull <stephenjturnbull@gmail.com> Sent: Friday, November 11, 2022 10:23 AM To: Caballero, Danny (Dan) <dancab@caltech.edu> Cc: mailman-users@mailman3.org <mailman-users@mailman3.org> Subject: [MM3-users] Re: Emergency moderation and clearing out a lot of held messages
Caballero, Danny (Dan) writes:
I don't understand. We have a single database configured for everything. I don't recall reading anything in the set-up documentation about using more than 1 database for Mailman3.
I was trying to help localize the database request to one of the three major components: core, Postorius, HyperKitty. If you use the two- database scheme documented here: https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta... knowing that database would at least identify whether the slowness is associated with core or with the web apps.
The excruciating details:
Conceptually, there are three separate databases: the one used by Mailman core to manage everything but authentication and archives, the one used by Django to manage authentication for HyperKitty and Postorius, and the one used by HyperKitty to manage archives. They don't share tables, and there are no name collisions for the tables so it doesn't really matter how the backend handles them.
The two database configuration is not really a recommendation. I suppose the practice derives from an abundance of caution, as there are concepts like "User" that must be implemented in somewhat different ways across components (core doesn't authenticate users, while Postorius's primary function is authentication of users). I myself would worry that there might be name collisions among tables, which would cause mayhem, but that's probably excessive.