Philip Colmer writes:
[Nothing about individual processes.] *Something* is using that memory. ps(1) should help identify it when your alarm is triggered, top(1) allows monitoring in realtime. For example, the information about the size of archives is interesting only if HyperKitty is using the memory. I guess it might be indicative of core usage since there's an awful lot of traffic, but core won't be reading archives into memory or anything like that, it doesn't even know or care where the archives are. It only knows where on the Internet to send messages so the archiver that lives there can add them. And I find it hard to imagine why Postorius would be consuming memory.
What is particularly curious for us is that we have four Mailman 3 servers.
Are these servers identically configured, or nearly so? If so, perhaps you can find "interesting" differences in memory usage across them.
Finally, and just FYI, we're using PostgreSQL for the database and Xapian for the indexing engine.
Either of those would be a weird cause. The point of an RDBMS is to store data in mass storage, although I suppose it's possible to write queries that would return huge quantites of data in memory. And the indexer is not a long-running process, as far as I know. So it should run, exit, and release memory. Again, if it's related to one of those it should turn up in ps.