Mailman 3 with AWS auto scaling and potentially load balancing
After doing a lot of googling I can't seem to find solid answers to a couple of basic mailman hosting questions so I thought I would ask here.
I'm looking to build an auto scaling group so that my mailman services can self heal if the EC2 instance goes away. What state if any is stored locally on the filesystem that would need to be backed up and restored when an instance is replaced?
I'm also interested in a high availability deployment so I would like to do some basic load balancing to ensure that this requirement is met but I haven't been able to find much on this topic for mailman 3. Can someone point me in the direction of a write up on this problem or comment on it?
On Thu, Mar 29, 2018, at 7:35 AM, Chris McMacken wrote:
After doing a lot of googling I can't seem to find solid answers to a couple of basic mailman hosting questions so I thought I would ask here.
I'm looking to build an auto scaling group so that my mailman services can self heal if the EC2 instance goes away. What state if any is stored locally on the filesystem that would need to be backed up and restored when an instance is replaced?
Maliman Core, the engine that sends and receives emails stores messages in queues on disk. Usually this is all inside "var" directory, but they may be scattered depending on where you install Mailman 3 from. So, you'd have to backup all those directories from the host.
Apart from that, Mailman Core and Django frontend all store most of their other data in a databases and you can backup-restore them using usual methods.
I'm also interested in a high availability deployment so I would like to do some basic load balancing to ensure that this requirement is met but I haven't been able to find much on this topic for mailman 3.
Since Core store some state on disk, it won't be possible to have multiple instances of Core serving the front-end. Front-end on the other-hand (based on Django) can easily be scaled and made to talk to a single Core as it stores everything in a Database that you can share across them.
Note that for now, Core expects to know the address for front-end (for archiving), and doesn't yet support multiple options there. So, you'd have to have one instance for receiving archived emails from Core and then others can just serve it.
-- Abhilash Raj maxking@asynchronous.in
Thanks for the prompt response.
Why does the application store messages on the local disk? I'm guessing it's for performance reasons but wanted to ask. Do you have any plans to add database storage for these at some point in the future?
On Fri, Mar 30, 2018, at 12:00 PM, Chris McMacken wrote:
Thanks for the prompt response.
Why does the application store messages on the local disk?
It is how our queuing system is built, every queue is stored on disk and messages in intermediate states like when being processed or decorated inside Mailman Core as stored on disk. *Most* of these isn't the problem, since the emails are sent out pretty fast, it's the held messages waiting for approval or broken messages in the shunt queue that would be lost and is what I mean by "State" mostly.
AFAIK API being served by different instances will affect what and how many held messages are reported.
I'm guessing it's for performance reasons but wanted to ask. Do you have any plans to add database storage for these at some point in the future?
Mailman-users mailing list mailman-users@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Abhilash Raj maxking@asynchronous.in
On Fri, Mar 30, 2018, at 1:13 PM, Abhilash Raj wrote:
On Fri, Mar 30, 2018, at 12:00 PM, Chris McMacken wrote:
Thanks for the prompt response.
Why does the application store messages on the local disk?
It is how our queuing system is built, every queue is stored on disk and messages in intermediate states like when being processed or decorated inside Mailman Core as stored on disk. *Most* of these isn't the problem, since the emails are sent out pretty fast, it's the held messages waiting for approval or broken messages in the shunt queue that would be lost and is what I mean by "State" mostly.
AFAIK API being served by different instances will affect what and how many held messages are reported.
I'm guessing it's for performance reasons but wanted to ask. Do you have any plans to add database storage for these at some point in the future?
And No, currently we don't have any plans to tinker with the queuing system in near future. It works very well right now and is quite fragile, one single bug in queuing/accounting of emails and you can bomb your subscribers with 1000s of emails or important emails being swallowed up.
Mailman-users mailing list mailman-users@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Abhilash Raj maxking@asynchronous.in
-- Abhilash Raj maxking@asynchronous.in
Yes, of course the queuing system needs to work correctly.
I only ask because when you store state in the filesystem it makes auto scaling and self healing very difficult and prone to errors. Even if the files in the queue are very short lived there is still plenty of opportunity for them to be lost, especially if the system is processing a lot of mail.
Thanks again for the informative and prompt response.
participants (2)
-
Abhilash Raj -
Chris McMacken