- On 4/20/25 11:24, Stephen J. Turnbull wrote:
This sounds to me like your runners are crashing, as Mark suggested. But even if that's resolved, you're still going to have a performance issue (lock contention for seconds or even minutes maybe[1]) even if you don't get stuck for days.
Fair enough. It was not obvious when installing that I should not use sqlite3.
Yeah, there's a sort of warning somewhere that we strongly recommend a full-featured RDBMS system, but we should probably make it a lot more prominent and explain why.
Strong ACK. Anecdotally, because I went the SQLite3 -> PostgreSQL route just recently:
I've initially set up mailman3 and Hyperkitty with the SQLite3 backend, since the system/lists is/are relatively small and the lists are mostly low-volume, so I figured it would be a good idea to go with the least complicated option.
Importing archives (which tend to be big in comparison to the incoming volume since they go back to 2011) took on the order of 10 hours or more for big lists, with django threads dying every now and then and locking messages appearing in the log.
I've since setup PostgreSQL and reconfigured both mailman and Hyperkitty to use it. Instead of migrating the old data from the SQLite3 database to the PostgreSQL database, I'm just re-importing the mailman 2.1 data again, since it didn't change and the whole sqlite->pgsql move process seems so finicky and error-prone.
Importing one of the biggest archives is orders of magnitude faster at 43 % completion after just half an hour. In comparison, the same progress would have taken 6 hours with the SQLite3 backend.
As far as I can see, using the SQLite3 backend should be strongly discouraged, even for small systems, and this should be properly mentioned in the documentation. It's certainly useful in an automated testing suite, but there's probably no production system that would benefit from using the SQLite3 backend.
Mihai