Hi.
Now running mailman 3 I am a bit surprised that the memory footprint is so large. I count to 13 processes with each a virtual space of 160-250MB and a reserved space of about 50MB each. Not much, but quite much to run on smaller cloud instances which generally have limited memory. This is (a lot) more than the corresponding postgres installation uses,
Is this normal?
regards // David
On 04/10/2017 02:25 PM, David Krantz wrote:
Hi.
Now running mailman 3 I am a bit surprised that the memory footprint is so large. I count to 13 processes with each a virtual space of 160-250MB and a reserved space of about 50MB each. Not much, but quite much to run on smaller cloud instances which generally have limited memory. This is (a lot) more than the corresponding postgres installation uses,
Is this normal?
Those numbers seem "normal" but you really have to look deeper to understand if you just have a bunch of processes sitting in memory because the memory is available and nothing else wants it, or if there's real memory contention going on.
The article at <https://wiki.list.org/x/4030711>, while written about Mailman 2.1, has some general information which may be of interest.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
some systems also support the sar command (system activity report). If you have it read the manpages to see all the options as there are many.
When looking for performance issues/problems that aren't real easy to find I like to just let 'vmstat 5' run in a window and take a look occasionally, particularly when things seem slow. I prefer the 5 second readings to see more about general trends.
On 4/11/2017 8:33 PM, Mark Sapiro wrote:
On 04/10/2017 02:25 PM, David Krantz wrote:
Hi.
Now running mailman 3 I am a bit surprised that the memory footprint is so large. I count to 13 processes with each a virtual space of 160-250MB and a reserved space of about 50MB each. Not much, but quite much to run on smaller cloud instances which generally have limited memory. This is (a lot) more than the corresponding postgres installation uses,
Is this normal?
Those numbers seem "normal" but you really have to look deeper to understand if you just have a bunch of processes sitting in memory because the memory is available and nothing else wants it, or if there's real memory contention going on.
The article at <https://wiki.list.org/x/4030711>, while written about Mailman 2.1, has some general information which may be of interest.
On Wed, Apr 12, 2017 at 3:44 PM, Richard Shetron <guest2@sgeinc.com> wrote:
some systems also support the sar command (system activity report). If you have it read the manpages to see all the options as there are many.
When looking for performance issues/problems that aren't real easy to find I like to just let 'vmstat 5' run in a window and take a look occasionally, particularly when things seem slow. I prefer the 5 second readings to see more about general trends.
Ok. Here are statistics from my case. I used top in this case (although with a custom view to show memory usage). As this is a small AWS machine there was only 1GB ram and no swap. I can't say that it all that much RAM used for each process, but the thread model with 13-14 python processes of the same size is relatively heavy if the mail lists are not high volume and a large number of mail lists.
But never mind, I upgraded the instance so now it works just fine. :-)
Tasks: 149 total, 1 running, 148 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0/0.0 0[ ] KiB Mem : 1014376 total, 114212 free, 751932 used, 148232 buff/cache KiB Swap: 0 total, 0 free, 0 used. 195452 avail Mem
PR VIRT RES SHR %MEM SWAP DATA CODE USED S %CPU TIME+ COMMAND 20 243404 58824 2216 5.8 0 128164 2944 58824 S 0.0 0:34.40 python 20 164580 52316 2372 5.2 0 49340 2944 52316 S 0.0 0:11.68 python 20 163980 51672 2400 5.1 0 48740 2944 51672 S 0.0 0:11.55 python 20 163588 49428 528 4.9 0 48348 2944 49428 S 0.0 0:11.32 python 20 162624 49180 1268 4.8 0 47384 2944 49180 S 0.0 0:11.47 python 20 162456 49080 1304 4.8 0 47216 2944 49080 S 0.0 0:11.19 python 20 234588 48880 1356 4.8 0 119348 2944 48880 S 0.3 0:08.54 python 20 160444 47308 1508 4.7 0 45204 2944 47308 S 0.0 0:01.20 python 20 160460 47100 1156 4.6 0 45220 2944 47100 S 0.0 0:11.04 python 20 160464 47052 1232 4.6 0 45224 2944 47052 S 0.0 0:11.09 python 20 160468 46884 1072 4.6 0 45228 2944 46884 S 0.0 0:11.20 python 20 160496 46732 876 4.6 0 45256 2944 46732 S 0.0 0:11.37 python 20 161976 46640 860 4.6 0 45076 2944 46640 S 0.0 0:01.08 python
On 4/17/2017 7:11 AM, David Krantz wrote:
On Wed, Apr 12, 2017 at 3:44 PM, Richard Shetron <guest2@sgeinc.com> wrote:
some systems also support the sar command (system activity report). If you have it read the manpages to see all the options as there are many.
When looking for performance issues/problems that aren't real easy to find I like to just let 'vmstat 5' run in a window and take a look occasionally, particularly when things seem slow. I prefer the 5 second readings to see more about general trends.
Ok. Here are statistics from my case. I used top in this case (although with a custom view to show memory usage). As this is a small AWS machine there was only 1GB ram and no swap. I can't say that it all that much RAM used for each process, but the thread model with 13-14 python processes of the same size is relatively heavy if the mail lists are not high volume and a large number of mail lists.
But never mind, I upgraded the instance so now it works just fine. :-)
Tasks: 149 total, 1 running, 148 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0/0.0 0[ ] KiB Mem : 1014376 total, 114212 free, 751932 used, 148232 buff/cache KiB Swap: 0 total, 0 free, 0 used. 195452 avail Mem
PR VIRT RES SHR %MEM SWAP DATA CODE USED S %CPU TIME+ COMMAND 20 243404 58824 2216 5.8 0 128164 2944 58824 S 0.0 0:34.40 python 20 164580 52316 2372 5.2 0 49340 2944 52316 S 0.0 0:11.68 python 20 163980 51672 2400 5.1 0 48740 2944 51672 S 0.0 0:11.55 python 20 163588 49428 528 4.9 0 48348 2944 49428 S 0.0 0:11.32 python 20 162624 49180 1268 4.8 0 47384 2944 49180 S 0.0 0:11.47 python 20 162456 49080 1304 4.8 0 47216 2944 49080 S 0.0 0:11.19 python 20 234588 48880 1356 4.8 0 119348 2944 48880 S 0.3 0:08.54 python 20 160444 47308 1508 4.7 0 45204 2944 47308 S 0.0 0:01.20 python 20 160460 47100 1156 4.6 0 45220 2944 47100 S 0.0 0:11.04 python 20 160464 47052 1232 4.6 0 45224 2944 47052 S 0.0 0:11.09 python 20 160468 46884 1072 4.6 0 45228 2944 46884 S 0.0 0:11.20 python 20 160496 46732 876 4.6 0 45256 2944 46732 S 0.0 0:11.37 python 20 161976 46640 860 4.6 0 45076 2944 46640 S 0.0 0:01.08 python
It looks like adding memory has helped.
At some point, you may just want to check and see what processes are running and what are automatically started. I've seen some odd things at times, like headless serves running an x-server and nothing using it.
On Apr 10, 2017, at 11:25 PM, David Krantz wrote:
Now running mailman 3 I am a bit surprised that the memory footprint is so large. I count to 13 processes
These correspond to the runners, most of which are long running processes that sleep for a while, then wake up and dequeue messages for handling. We model the REST API and LMTP servers as runners as well, but they are technically different since both listen on ports for connections. There's also a master process that controls the runner subprocesses.
This architecture is some of the oldest and most stable bits of Mailman, and while the details have changed, the essential nature of these processes are largely inherited from Mailman 2. Given the technology at the time, it was the best way of doing things. It's probably still the most generic way of doing it, since they don't rely on much more than standard POSIX semantics, so they run on just about any POSIX-compatible system right out of the box.
I can imagine different ways of doing it. For example, systemd socket activation on OSes that support that for the REST and LMTP servers. And maybe use inotify in the master watcher to spawn queue processing runners on demand. These would all be interesting things to explore, although they may not be universally appropriate (i.e. not all *nix systems supported by Mailman run systemd, nor support inotify).
Cheers, -Barry
After I wrote this, I see that the OP has a usecase where they've got a shrunk to fit AWS instance. So this is more useful than I thought, but I still don't think that's a huge audience. (The OP himself just upgraded the instance, after all.)
Barry Warsaw writes:
I can imagine different ways of doing it. For example, systemd socket activation on OSes that support that for the REST and LMTP servers. And maybe use inotify in the master watcher to spawn queue processing runners on demand. These would all be interesting things to explore, although they may not be universally appropriate (i.e. not all *nix systems supported by Mailman run systemd, nor support inotify).
Not to mention that spawning is typically a heavyweight action, so if you're heavily loaded enough to care about load, you'd need to add complex logic like that used by Apache to ensure you have same processes/threads hanging around to answer frequent calls, rather than one-shot "CGI" style. We do have thread and process pools in Python, but they wouldn't necessarily be compatible with this use case combined with systemd or inotify without additional logic in Mailman.
If somebody wants to mess with this specifically, it could be useful. But I think it's kind of a long shot given the kinds of scenarios Mailman is typically used in (lightweight Mom & Pop shops, virtual servers with Mailman-as-a-Service, and hard-core 5,000-list 100,000 message per day university settings be the not unusual extremes). Sort of like gilectomy efforts in Python itself (maybe //arry would like to work on this? ;-) ;-) nudge nudge say-no-more, eh?)
Regards,
Teilnehmer (6)
-
Barry Warsaw -
David Krantz -
Mark Sapiro -
Richard Shetron -
Richard Shetron -
Stephen J. Turnbull