Greetings,
Ok, I just updated the docker containers that are running my Mailman 3 installation over the weekend. mailman-core is now 3.3.10, with mailman-ngnix at 1.28.0, mailman-postfix at 3.10.5-r0 and mailman-django-uwsgi at "latest".
I think I have got everything working normally — messages seem to be processed correctly, and I can access the web pages on the web interface. And I have updated various scripts to use --run-as-root until I can get around to reconfiguring mailman-core to use a non-root user.
Anyway, since that time, the EC2 instance that the containers are running on becomes "unreachable", to use AWS's parlance. I can restore functionality by rebooting the instance. According to AWS, this is an application or operating system issue, not their instance. Which means auto-recovery doesn't get activated.
This always happens overnight, usually after midnight, though one time it was around 10:40. I don't see any crontab entries that correlate. And the after midnight issues don't always happen at the same time, so it seemed unlikely that it was a crontab job that was causing it
I finally looked at the host (rather than the logs of each container) and realized that the oom-killer was killing django-admin. As a temporary workaround, I've limited memory on the mailman-django-uwsgi container to 4 GiB (though, since I just did that this morning, I won't know until after midnight (presumably) if it worked or not), and set the RestartPolicy to always.
I'm thinking that this points to a memory leak or some kind?
I did have to add a bunch of packages to my Dockerfile when I rebuilt the container, since the last build was like 5 years ago, so I suppose it could be that django is just using more memory than it used to, though at the moment, I see that it is around 1 GiB:
FROM python:3.12-alpine3.22
RUN apk update && apk add --no-cache --virtual build-deps
cargo gcc libffi-dev musl-dev &&
apk add --no-cache gettext postgresql-dev sassc build-base pcre-dev python3-dev libc-dev linux-headers &&
pip install --upgrade pip setuptools wheel &&
pip --no-cache-dir install django-haystack==3.3.0 hyperkitty==1.3.12
postorius==1.3.13 psycopg2 uwsgi==2.0.25 whoosh &&
apk update && apk del --no-cache build-deps
WORKDIR /opt/mailman/mailman-suite/mailman-suite_project USER mail
EXPOSE 8000
CMD ["uwsgi", "--ini", "uwsgi.ini"]
Any insights, ideas or assistance would be appreciated.
Thanks!
Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering - Infrastructure Fred Hutch Cancer Center O 206.667.4856
phirayam@fredhutch.org<mailto:phirayam@fredhutch.org>