deployment recommendation(?)
(sorry, if this comes in double; i sent the same mail yesterday, but it seems it didn't make it (i did check the archives); so i'm re-sending via the webinterface)
hi,
it's summer here in europe, and students have left the campus. so - like every summer - I ponder on the idea of migrating our old mailman2 instance that has served us for the last twenty years to a new and shiny mailman3.
and - like always - i'm a bit unsure about the best way to proceed to find the optimum deployment strategy for a long-running (decades) medium-sized (our largest list has ~1800 members) installation. one of our primary objectives is receiving *security updates*. we also would like to stay on top of development (that is: use the latest and greatest *stable* mailman3 release).
in my heart i am a Debian person, so my mailman2 installation just used the Debian packages. the new deployment machine, will be Debian based as well.
now https://docs.mailman3.org/en/latest/install/install.html lists three options:
# Debian packages this is what I would normally pick, but: my past experience with mailman3 in Debian was... rather suboptimal. (my current Debian box running mailman3 is frozen at Debian/buster ("old-old-stable") because of mailman3).
i see that the packages in current Debian/stable ("bookworm"), are lagging slightly (3.3.8, rather than 3.3.9), though that's to be expected from a distro with stable releases. otoh, Debian/unstable still ships 3.3.8, which is a bit disappointing.
in any case, the really nice thing about (Debian-official) Debian packages is that they are typically
- stable
- guarantee an upgrade path between releases
Docker packages
this is of course the simplest from a deployment perspective, but then i'm a bit worried about the security perspective. in my perception, docker deployments tend to get installed and then forgotten, and while the host OS keeps getting updates, the docker images are just left as is. so they tend to get stale.
i would like to know how this is handled
# virtualenv
this is the deployment method recommended in the docs. however, it doesn't say why this method is any better than the others. also, the website says that it was last updated in 2019, so: is this recommendation still relevant?
personally i do not have much experience with deployment of long-running software in virtualenv, and i'm esp worried a bit when it comes to switching python releases (which will happen multiple times in the next few years)
i use virtualenv all the time for my pet projects, but there it doesn't matter much, if the venv breaks: i just delete it and start from scratch. but those are not long-running services.
so: what are the drawbacks, limitations, advantages and benefits of the various deployment methods?
i'd really like to make the switch this year!
gmfasdr IOhannes
Hello,
On Thu, Jul 18, 2024 at 09:05:28AM -0000, noc@iem.at wrote:
(my current Debian box running mailman3 is frozen at Debian/buster ("old-old-stable") because of mailman3).
Which is no longer supported (unless you subscribed to ELTS) -- for more than two weeks now.
I upgraded from Mailman2 (buster) to Mailman3 (bookworm). I took the opportunity to isolate mailman in its own container, so I really imported the lists & archives. PostgreSQL is also in another container now. The main mail server also.
So far I have not seen any issue that I could not fix easily, in part with the help of this list, which is very efficient.
what are the drawbacks, limitations, advantages and benefits of the various deployment methods?
Some of the enhancements (e.g. password forgotten mail only sent to actual account holders) will not be backported: only real security fixes will. I might apply some of those by hands, using diversions, as I was doing for Mailman2.
I will also implement a IPS using mailman's log, looks useful since, opposite to some Perl web apps I run, it does not seem there are any built-in counters in Python/Django (e.g. max number of logins per minute, hour, day; max number of e-mails to an e-mail address per minute/hour/day, etc). Maybe I just don't know enough of Django though. It's handy eg. when people mass-create accounts, etc.
The main issue I had is the enormous amount of resources (disk, database, memory) that Mailman3 uses over Mailman2 (about 10x more RAM, and the mailmanweb DB backup, zstd compressed, is 2 GBytes, where the archives were AFAIR 500M in Mailman2). With some tweaking, I could reduce the 8 GB memory footprint to 2 GB.
But as you already run Mailman3 it should not be new.
On 2024-07-18 11:27:37 +0200 (+0200), Marc SCHAEFER wrote: [...]
The main issue I had is the enormous amount of resources (disk, database, memory) that Mailman3 uses over Mailman2 (about 10x more RAM, and the mailmanweb DB backup, zstd compressed, is 2 GBytes, where the archives were AFAIR 500M in Mailman2). With some tweaking, I could reduce the 8 GB memory footprint to 2 GB. [...]
Some of that (though not really the storage) depends on what kind of MM2 site(s) you were running as well. In my case I was maintaining a bunch of different mail domains with their own distinct sets of mailing lists, and that necessitated running multiple instances of MM2. When upgrading, I took advantage of MM3's proper multi-tenancy so I only need one set of services to take care of all those domains with their independent sets of lists. The new server is using less RAM and CPU than the old one as a result.
Jeremy Stanley
noc@iem.at wrote:
# Debian packages this is what I would normally pick, but: my past experience with mailman3 in Debian was... rather suboptimal. (my current Debian box running mailman3 is frozen at Debian/buster ("old-old-stable") because of mailman3).
Yep, I think a lot of people on this list have tried these packages and have got burnt. That is mor likely due to our own shortcomings rather than anything the package maintainers have done, but my impression is that most people on this list have gone either virtualenv or Docker due to the support that others have with these methods. For various reasons I now have to use Ubuntu, and when I migrated to Ubuntu 22.04 I wanted to go from Docker to use these packages, only to find that the packages were missing from Ubuntu due to the fact that Ubuntu shipped a newer Python version and that didn't work with the older version of Mailman in those packages. Since then I did an upgrade to Ubuntu 24.04 and briefly looked at the packages but my existing install was already on 3.3.9 so continued with the venv method.
# Docker packages this is of course the simplest from a deployment perspective, but then i'm a bit worried about the security perspective. in my perception, docker deployments tend to get installed and then forgotten, and while the host OS keeps getting updates, the docker images are just left as is. so they tend to get stale.
The Docker containers are well maintained by Maxking (Abhilash Raj) who is also the core Mailman maintainer. If you look at the images on Dockerhub they do get regular updates. I would say that your update strategy would need to take Docker image updates into account. I used these images for a while, I moved from Docker because I run software on the same machine as my small Mailman instance which takes over the Docker management, and I wanted Mailman to be installed alongside that.
# virtualenv this is the deployment method recommended in the docs. however, it doesn't say why this method is any better than the others. also, the website says that it was last updated in 2019, so: is this recommendation still relevant?
This is the method I use now and seems to be the most universally supported on this list. I've explained why I went this method earlier on in this post. The installation docs for virtualenv installation were very recently updated. Upgrading the distro (Python major version upgrade) does require a venv rebuild, but general updates throughout the year seem absolutely fine. All the persistent data is stored outside the venv of course.
Thanks. Andrew.
participants (4)
-
Andrew Hodgson -
Jeremy Stanley -
Marc SCHAEFER -
noc＠iem.at