Hi gang,
I want to migrate mm21 to mm3, by changing the domain of the list, thus LIST@OLD_dom to LIST@NEW_DOM. I run on debian, with the debian testing version of mm3 installed.
Is there an obvious way to do that? Right now I think
- create new list
- read mm21 pickle to change the domains in it, write to temporary pickle
- run mailman import with that pickle
- read the archive mailbox and change the domain in all the headers
- change the domainname in bodies and attachments (?)
- run hyperkitty import
This may be the project of a madman... any advice?
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22110th day.
On Tue, Dec 16, 2025 at 7:15 AM Thomas Krichel <krichel@openlib.org> wrote:
Hi gang,
I want to migrate mm21 to mm3, by changing the domain of the list, thus LIST@OLD_dom to LIST@NEW_DOM. I run on debian, with the debian testing version of mm3 installed.
Is there an obvious way to do that? Right now I think
- create new list
- read mm21 pickle to change the domains in it, write to temporary pickle
- run mailman import with that pickle
- read the archive mailbox and change the domain in all the headers
- change the domainname in bodies and attachments (?)
- run hyperkitty import
This may be the project of a madman... any advice?
When you say 'read blah and change the domain', are you doing that programmatically or beforehand? I'd do it beforehand. And whichever way you do it, it will work.
In its simplest - computational power not taken into account:
sed -i.BAK 's/OLD_dom/NEW_dom/g' pickle archive_mailbox
- run the indexing.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Thomas Krichel writes:
I want to migrate mm21 to mm3, by changing the domain of the list, thus LIST@OLD_dom to LIST@NEW_DOM.
Are you changing the domain *because* you're upgrading Mailman? There's usually no need to do that. There are several ways to ensure continuity of operation during the switchover. Of course if you have other reasons to change the domain, that's your business and I don't need to know why (just that you do have reasons).
I run on debian, with the debian testing version of mm3 installed.
We do not recommend running any distro's version of Mailman 3 at this time. Mailman development is not of the "move fast and break things" ilk, but all of the major distros are out of date by the time they get Mailman into "testing" or the equivalent for non-Debian distros. It makes it harder for us to support you. See https://docs.mailman3.org/en/latest/install/virtualenv.html for the preferred installation approach. That makes it much easier for you to upgrade in the event of a security patch or new feature you want.
Is there an obvious way to do that?
First, think twice about Odhiambo's suggestion of 'sed -e s/OLD_dom/NEW_dom/', because that will change the List-ID and message-ids. Usually it is recommended to keep the List-ID. Among other things, keeping it will help establish the reputation of the migrated list with the more professionally-run freemail providers. Since message-ids as data are spread across subscribers' mailboxes, you really don't want to change those.
Right now I think 0. create new list
- read mm21 pickle to change the domains in it, write to temporary pickle
- run mailman import with that pickle
The issue of changing domains "forward" (= for future posts) came up earlier, and Mark recommended using the database's command line utility to change the list's mail_host entry:
I don't have much experience with this, but I think just something like this database query will do it UPDATE mailinglist SET mail_host = 'NEW_dom';
If so, the better strategy is to just import the old pickle, then change the associated mail_host. This leaves the list_id the same, which you probably want.
- read the archive mailbox and change the domain in all the headers
This will break threads and archive pointers in members' personal archives if you change the Message-ID, In-Reply-To, or References headers. If people are replying to messages in their own mailboxes after the switchover, the breakage will propagate to your HyperKitty archives.
Note that HyperKitty doesn't care what the domain name of the archive host is when serving a list. It's perfectly happy to serve any number of domains. If it was designed properly (I think it was but not 100% sure), continuity of each list's archive is based on the list_id, not the posting address. The same is true of threading, it's based on chaining Message-IDs via References and In-Reply-To headers, not the posting address.
- change the domainname in bodies and attachments (?)
- run hyperkitty import
This may be the project of a madman... any advice?
Before you go messing with the data of the old list configs and archives, consider that most of the mechanisms that provide continuity in the archives don't depend on the list posting address. There's nothing you can do about people who are replying to old messages in their local mailboxes. So if possible, you probably want to keep the old address as a CNAME or MX in DNS at least for a while, and perhaps have the MTA rewrite the list-post address of new posts to the list before distribution. That approach is much less likely to cause discontinuities in list operation.
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
Stephen J. Turnbull writes
Are you changing the domain *because* you're upgrading Mailman? There's usually no need to do that. There are several ways to ensure continuity of operation during the switchover.
I understand that.
Of course if you have other reasons to change the domain, that's your business and I don't need to know why (just that you do have reasons).
I do have other reasons, but I thought the once-in a lifetime change from Mailman2.1 will be the best time to do it.
We do not recommend running any distro's version of Mailman 3 at this time.
This argument is well-known and has been discussed here many times.
Mailman development is not of the "move fast and break things" ilk, but all of the major distros are out of date by the time they get Mailman into "testing" or the equivalent for non-Debian distros. It makes it harder for us to support you. See https://docs.mailman3.org/en/latest/install/virtualenv.html for the preferred installation approach. That makes it much easier for you to upgrade in the event of a security patch or new feature you want.
I have used that when Debian testing updated the default python to 3.13 and Mailman was not ready for it. So your idea that distros are lagging is not always completely accurate. But anyway this has been discussed here many times.
Is there an obvious way to do that?
First, think twice about Odhiambo's suggestion of 'sed -e s/OLD_dom/NEW_dom/',
I already thought 3 times about it ;-)
because that will change the List-ID and message-ids.
The most problematic aspect is that it may change the actual contents of the posts.
Usually it is recommended to keep the List-ID. Among other things, keeping it will help establish the reputation of the migrated list with the more professionally-run freemail providers.
I am astounded by this assertion. (1) As far as I understand, spam reputation is based on ip addresses, as they are the only thing in an email you can not fake. (2) the freemail providers' ways of running spam filtering appears very secretive. So while they may take account of a list-id, it's not usually known how and why.
Since message-ids as data are spread across subscribers' mailboxes, you really don't want to change those.
Right now I think 0. create new list
- read mm21 pickle to change the domains in it, write to temporary pickle
- run mailman import with that pickle
The issue of changing domains "forward" (= for future posts) came up earlier, and Mark recommended using the database's command line utility to change the list's mail_host entry:
I don't have much experience with this, but I think just something like this database query will do it UPDATE mailinglist SET mail_host = 'NEW_dom';
I am happy to read that what I think about here is not completely trivial.
If so, the better strategy is to just import the old pickle, then change the associated mail_host. This leaves the list_id the same, which you probably want.
I don't know what the mail_host is.
- read the archive mailbox and change the domain in all the headers
This will break threads and archive pointers in members' personal archives if you change the Message-ID, In-Reply-To, or References headers.
Hmm, you mean what users have in their archives on their computer? I have no write access to that.
If people are replying to messages in their own mailboxes after the switchover, the breakage will propagate to your HyperKitty archives.
I doubt it. I intend to remove the mm21 list at migration time, so list@old_dom email will become unroutable, I suspect. This break in service is tolerable in my circumstances. All folks on the list know that I run them. They can email me personally to read me the riot act ;-)
Note that HyperKitty doesn't care what the domain name of the archive host is when serving a list. It's perfectly happy to serve any number of domains. If it was designed properly (I think it was but not 100% sure), continuity of each list's archive is based on the list_id, not the posting address. The same is true of threading, it's based on chaining Message-IDs via References and In-Reply-To headers, not the posting address.
- change the domainname in bodies and attachments (?)
- run hyperkitty import
This may be the project of a madman... any advice?
Before you go messing with the data of the old list configs and archives, consider that most of the mechanisms that provide continuity in the archives don't depend on the list posting address. There's nothing you can do about people who are replying to old messages in their local mailboxes. So if possible, you probably want to keep the old address as a CNAME or MX in DNS at least for a while, and perhaps have the MTA rewrite the list-post address of new posts to the list before distribution. That approach is much less likely to cause discontinuities in list operation.
Thank you so much for your kind and detailed response. I ought to have specified that discontinuity of operation is not an issue in my specific circumstances.
I will report on my progress.
-- Written by Thomas Krichel http://openlib.org/home/krichel on his 22110th day.
Thomas Krichel writes:
Stephen J. Turnbull writes
I have used that when Debian testing updated the default python to 3.13 and Mailman was not ready for it. So your idea that distros are lagging is not always completely accurate.
I'm saying that when you come to this list, you will get better service if you have a reasonably fresh build from PyPI or source in a venv.
Usually it is recommended to keep the List-ID. Among other things, keeping it will help establish the reputation of the migrated list with the more professionally-run freemail providers.
I am astounded by this assertion.
It's based on conversations with the DMARC working group participants, including the head of email security at Yahoo at the time. In any case, it's well-known that spam-fighters use machine learning models that assess a wide variety of attributes of each message. I assume that claimed, and especially validated, sender identity would be among them.
(1) As far as I understand, spam reputation is based on ip addresses, as they are the only thing in an email you can not fake.
Not true. You can't fake valid DKIM signatures, you can't fake DMARC From alignment, and you can't fake valid ARC seals. If the List-ID is signed by either DKIM or attested to by ARC, you can't fake that, either. On the other hand, a useful IP is often not available, or is lumped in with a whole netblock (especially in the case of IPv6).
In any case, if you're DKIM-signing the RFC 2369 and RFC 2919 headers with the key of the list domain, recipients can generally trust that because normally the list host's signature arrives valid.
The issue of changing domains "forward" (= for future posts) came up earlier, and Mark recommended using the database's command line utility to change the list's mail_host entry:
I don't have much experience with this, but I think just something like this database query will do it UPDATE mailinglist SET mail_host = 'NEW_dom';
If so, the better strategy is to just import the old pickle, then change the associated mail_host. This leaves the list_id the same, which you probably want.
I don't know what the mail_host is.
It's the SQL database name of the field that contains the domain of the list. You "su mailman", invoke the command line utility such as psql or mysql which probably will connect to the 'mailman' database, and issue a simple SQL command. I think you need to be a little more careful, with something like
UPDATE mailinglist SET mail_host = 'NEW_dom' WHERE mail_host = 'OLD_dom';but that will effectively change the domain of all the mailing lists on 'OLD_dom' in the 'mailinglist' table. I guess you probably want to change the web host, too, so future messages would have the correct archive URLs in the message header and mailing list footer.
- read the archive mailbox and change the domain in all the headers
This will break threads and archive pointers in members' personal archives if you change the Message-ID, In-Reply-To, or References headers.
Hmm, you mean what users have in their archives on their computer? I have no write access to that.
Which is why I would be careful about making the list's own archive diverge from the content in their mailboxes, which their MUAs will be using to compose the headers in their posts to the list.
I'm willing to bet that some of your members will compose email by hitting reply-to-list to a message from OLD_dom, and aware of the change will fix up the To address to @NEW_dom by hand. But (unless they're real mail nerds) they will not update the References and Reply-To headers, breaking those threads in your archives because HyperKitty won't recognize the IDs in those posts' References.
Thank you so much for your kind and detailed response. I ought to have specified that discontinuity of operation is not an issue in my specific circumstances.
I will report on my progress.
Thanks! It's helpful to know what people are trying to do.
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
participants (3)
-
Odhiambo Washington -
Stephen J. Turnbull -
Thomas Krichel