db migration from mailman3.2.2 to mailman3.3.9 (sqlite3 to postgres16)

Hello, I know this question comes up here periodically, but I am having some trouble performing a data migration from an older version of mailman3 to a more recent one.
We have a legacy environment with: mailman3.2.2 postorius 1.2.4 hyperkitty 1.2.2 there are two sqlite3 dbs for mailman core and mailman web.
I've staged a new environment using the docker-mailman project with the following:
mailman3.3.9 postorius 1.3.10 hyperkitty 1.3.8 there is a single postgres db here for both core and web.
I've tried migration a few different ways, using pgloader, and also with some manual python scripts. Data is definitely not my forte, but I am struggling to get a working instance. I am running into several issues such as foreignkeyviolations, datatype mismatches, unique PK errors, etc..
Does anyone out there have a prescriptive process for this migration, or a suggestion for workflow/tooling that has worked in a similar upgrade process?
Thanks! -Peter

Peter Thornton via Mailman-users writes:
We have a legacy environment with: there are two sqlite3 dbs for mailman core and mailman web.
I've staged a new environment there is a single postgres db here for both core and web.
This is known to be problematic. Try searching the archives https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/ for "sqlite postgres migration" (or maybe use "sqlite3" and/or "postgresql", and don't surround the phrase with quotes).
I would start by configuring the postgres db on the new host with separate databases. If you decide you want a single db, I guess it's easier to add the tables from the mailmanweb db to the mailman db than it is to get pgloader to do the right thing.
ISTR that the most straightforward approach was rather than using pgloader, you should create the databases in Postgres then dump the databases (text) and load them. You probably need to edit them. (This is very unreliable advice!)
It's almost 2am here, so I'm off to bed. You should consider all of the above advice except "known problematic" to be unreliable.
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
participantes (2)
-
Peter Thornton
-
Stephen J. Turnbull