I'm migrating to a new Mailman3 instance, finally escaping a fairly old (3.1.1) and problematic distribution repo install on a Linux VM and moving to a venv install in a native SmartOS Zone. I've gotten everything running well and integrated with our (fairly complex) mail server. I'd like to begin migration of the lists on the old mm3 instance.
Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc?
Thanks,
-Dave-- Dave McGuire, AK4HZ New Kensington, PA
Dave McGuire writes:
Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc?
All that is in an SQL database. Mailman still manages queues and digests "on disk", so I would
- shut down Mailman completely (but not the RDBMS)
- copy the queues (in $var_dir/queue) and digests (in $var_dir/lists/ *) over to the new installation
- dump the SQL database on the old host in SQL script format
- restore the SQL database on the new host
- restart Mailman
This shouldn't take more than 15-20 minutes, faster if you're handy with all the relevant utilities. The main forseeable issue is Mailman traffic queuing up on the old installation. Depending on your mail routing this may not be a problem:
- same MTA, just point it at the new host and everything will go through when Mailman starts)
- different MTA, before shutting down Mailman reroute Mailman traffic to new MTA where it will queue until Mailan starts
Of course if something is wonky somewhere, you could get an error in dumping or restoring the database. Also if you're changing databases eg from MySQL to Postgres you may need to edit the dump by hand. Don't forget to set your MySQL family databases to handle Unicode correctly (at least older versions would throw exceptions on 4-byte UTF-8 such as emoji).
Main thing is grab your dog-eared copy of the Hitchhiker's Guide to the Galaxy because it has the words "Don't Panic!" printed on the cover in large friendly letters. Everything in Internet mail including Mailman operates on the "store and forward" principle: there's alway a valid copy of your mail *somewhere*. If you've made "high availability" promises, then you'll need to plan harder to be more confident you can get everything done quickly and correctly the first time, but the above plan works.
On 8/21/23 00:31, Stephen J. Turnbull wrote:
Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc?
All that is in an SQL database. Mailman still manages queues and digests "on disk", so I would
- shut down Mailman completely (but not the RDBMS)
- copy the queues (in $var_dir/queue) and digests (in $var_dir/lists/ *) over to the new installation
- dump the SQL database on the old host in SQL script format
- restore the SQL database on the new host
- restart Mailman
This shouldn't take more than 15-20 minutes, faster if you're handy with all the relevant utilities. The main forseeable issue is Mailman traffic queuing up on the old installation. Depending on your mail routing this may not be a problem:
- same MTA, just point it at the new host and everything will go through when Mailman starts)
- different MTA, before shutting down Mailman reroute Mailman traffic to new MTA where it will queue until Mailan starts
Of course if something is wonky somewhere, you could get an error in dumping or restoring the database. Also if you're changing databases eg from MySQL to Postgres you may need to edit the dump by hand. Don't forget to set your MySQL family databases to handle Unicode correctly (at least older versions would throw exceptions on 4-byte UTF-8 such as emoji).
Main thing is grab your dog-eared copy of the Hitchhiker's Guide to the Galaxy because it has the words "Don't Panic!" printed on the cover in large friendly letters. Everything in Internet mail including Mailman operates on the "store and forward" principle: there's alway a valid copy of your mail *somewhere*. If you've made "high availability" promises, then you'll need to plan harder to be more confident you can get everything done quickly and correctly the first time, but the above plan works.
Thanks, that sounds reasonable to me. I'll give it a shot later this week. I appreciate the info.
-Dave-- Dave McGuire, AK4HZ New Kensington, PA
On 8/21/23 00:31, Stephen J. Turnbull wrote:
Is there a way to export a list from one Mailman 3 instance and import it into another, on a different machine? Ideally some way to capture its settings, members, members' settings, etc?
All that is in an SQL database. Mailman still manages queues and digests "on disk", so I would
- shut down Mailman completely (but not the RDBMS)
- copy the queues (in $var_dir/queue) and digests (in $var_dir/lists/ *) over to the new installation
- dump the SQL database on the old host in SQL script format
- restore the SQL database on the new host
- restart Mailman
This shouldn't take more than 15-20 minutes, faster if you're handy with all the relevant utilities. The main forseeable issue is Mailman traffic queuing up on the old installation. Depending on your mail routing this may not be a problem:
- same MTA, just point it at the new host and everything will go through when Mailman starts)
- different MTA, before shutting down Mailman reroute Mailman traffic to new MTA where it will queue until Mailan starts
Of course if something is wonky somewhere, you could get an error in dumping or restoring the database. Also if you're changing databases eg from MySQL to Postgres you may need to edit the dump by hand. Don't forget to set your MySQL family databases to handle Unicode correctly (at least older versions would throw exceptions on 4-byte UTF-8 such as emoji).
I'm moving from sqlite3 to postgres; I can handle that migration. But I'm also moving from Mailman 3.1.1 to 3.3.8; do you (or anyone else) know of any schema changes? I spot-checked the first few and they looked the same, but before going through all of them I figured I'd just ask here.
Thanks,
-Dave-- Dave McGuire, AK4HZ New Kensington, PA
On 8/22/23 2:38 PM, Dave McGuire wrote:
I'm moving from sqlite3 to postgres; I can handle that migration. But I'm also moving from Mailman 3.1.1 to 3.3.8; do you (or anyone else) know of any schema changes? I spot-checked the first few and they looked the same, but before going through all of them I figured I'd just ask here.
Schema changes are handled automatically by alembic. The migrations are defined in /mailman/database/alembic/versions/. Your current version is is stored in the database in the alembic_version table. When Mailman core is started, all migrations from your current version to the latest version will be applied to bring your database up to date. I.e., you don't need to be concerned about schema changes upon upgrade. They are handled automatically.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 8/22/23 17:59, Mark Sapiro wrote:
I'm moving from sqlite3 to postgres; I can handle that migration. But I'm also moving from Mailman 3.1.1 to 3.3.8; do you (or anyone else) know of any schema changes? I spot-checked the first few and they looked the same, but before going through all of them I figured I'd just ask here.
Schema changes are handled automatically by alembic. The migrations are defined in /mailman/database/alembic/versions/. Your current version is is stored in the database in the alembic_version table. When Mailman core is started, all migrations from your current version to the latest version will be applied to bring your database up to date. I.e., you don't need to be concerned about schema changes upon upgrade. They are handled automatically.
Brilliant! Thank you Mark.
-Dave-- Dave McGuire, AK4HZ New Kensington, PA
This appears to be practical for exporting/importing ALL the lists from one MM3 server to another MM3 server.
Is there a similar straightforward way of exporting/importing just ONE list?
On 2/29/24 14:44, imark@posteo.net wrote:
This appears to be practical for exporting/importing ALL the lists from one MM3 server to another MM3 server.
Is there a similar straightforward way of exporting/importing just ONE list?
No. It is a wishlist item. See https://gitlab.com/mailman/postorius/-/issues/13
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/1/24 00:11, Mark Sapiro wrote:
On 2/29/24 14:44, imark@posteo.net wrote:
This appears to be practical for exporting/importing ALL the lists from one MM3 server to another MM3 server.
Is there a similar straightforward way of exporting/importing just ONE list?
Why not export ALL list as indicated, then delete all unwanted lists on source resp. destination server ...
On 2024-03-01 10:56, Peter Adebahr via Mailman-users wrote:
On 3/1/24 00:11, Mark Sapiro wrote:
On 2/29/24 14:44, imark@posteo.net wrote:
This appears to be practical for exporting/importing ALL the lists from one MM3 server to another MM3 server.
Is there a similar straightforward way of exporting/importing just ONE list?
Why not export ALL list as indicated, then delete all unwanted lists on source resp. destination server ...
That's definitely Plan B.
And thank you Mark for the pointer to the Gitlab thread. Greater minds have clearly been giving this some thought for a while.
Cheers.
participants (6)
-
Dave McGuire -
imark＠posteo.net -
Mark
-
Mark Sapiro -
Peter Adebahr -
Stephen J. Turnbull