Mailman 3 Installation Assistance
I am making a serious attempt at installing Mailman 3 using docker. I have a CentOS 7 server setup via Linode. I am using the following documentation to setup a Mailman 3 server:
https://xiaoxing.us/2018/01/02/Deploy-Mailman-3-on-AWS-using-Docker/
I have Docker, Postfix, and Nginx installed and working fine. The installation of the Mailman 3 container went fine. However I am getting the following error when I go to test Posterius via shell:
curl http://172.19.199.3:8000/postorius/lists/ curl: (7) Failed connect to 172.19.199.3:8000; Connection refused
I assume this has something to do with uwsgi which I have no experience with.
Appreciate any assistance with getting Mailman 3 up and running. Thanks!
Brian Carpenter
On Sat, Mar 31, 2018, at 9:33 AM, brian@emwd.com wrote:
I am making a serious attempt at installing Mailman 3 using docker. I have a CentOS 7 server setup via Linode. I am using the following documentation to setup a Mailman 3 server:
https://xiaoxing.us/2018/01/02/Deploy-Mailman-3-on-AWS-using-Docker/
The official docs for container images sits here1. Although, I wen through the above blog very quickly and it seems to me that most of parts related to Mailman are pretty straightforward and similar to official docs.
I have Docker, Postfix, and Nginx installed and working fine. The installation of the Mailman 3 container went fine. However I am getting the following error when I go to test Posterius via shell:
curl http://172.19.199.3:8000/postorius/lists/ curl: (7) Failed connect to 172.19.199.3:8000; Connection refused
There are few things you can do to debug:
check if mailman-web container is even running $ docker ps
check if there are anything in /opt/mailman/web/logs/ .
-- Abhilash Raj maxking@asynchronous.in
Here is the output of docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 073ecf209106 maxking/mailman-web:0.1 "docker-entrypoint..." 27 hours ago Up 27 hours 8000/tcp, 8080/tcp mailman-web aef8f44a7136 maxking/mailman-core:0.1 "docker-entrypoint..." 27 hours ago Up 27 hours 8001/tcp, 8024/tcp mailman-core 398e3fc05e2c postgres:9.6-alpine "docker-entrypoint..." 27 hours ago Restarting (1) 12 hours ago dockermailman_database_1
There is no logs directory at all:
cd /opt/mailman/web/logs -bash: cd: /opt/mailman/web/logs: No such file or directory
On Sun, Apr 1, 2018, at 11:19 AM, brian@emwd.com wrote:
Here is the output of docker ps:
CONTAINER ID IMAGE COMMAND
CREATED STATUS PORTS
NAMES 073ecf209106 maxking/mailman-web:0.1 "docker-entrypoint..."
27 hours ago Up 27 hours 8000/tcp, 8080/tcp
mailman-web aef8f44a7136 maxking/mailman-core:0.1 "docker-entrypoint..."
27 hours ago Up 27 hours 8001/tcp, 8024/tcp
mailman-core 398e3fc05e2c postgres:9.6-alpine "docker-entrypoint..."
27 hours ago Restarting (1) 12 hours ago
dockermailman_database_1
Looks like your database isn't up, because of which the Django app migh not have started at all.
Can you also show the output of:
$ docker logs mailman-web
It should have something relevant if the container didn't startup at all.
There is no logs directory at all:
cd /opt/mailman/web/logs -bash: cd: /opt/mailman/web/logs: No such file or directory
Mailman-users mailing list mailman-users@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Abhilash Raj maxking@asynchronous.in
It is filled with the following message:
psql: could not translate host name "database" to address: Try again Postgres is unavailable - sleeping
So it looks like no database was setup. My understand is the docker installation does that:
The default compose file do these things:
Run a wsgi server using uwsgi for the Mailman's Django based web frontend listening on http://172.19.199.3:8000/ . It will run 2 worker processes with 4 threads each. Run a PostgreSQL server with a default database, username and password as mentioned in the docker-compose.yaml . Run mailman-core listening an LMTP server at http://172.19.199.2:8024/ for messages from MTA.
I have Mariadb setup on the server which I assume is fine to work with Mailman 3 since it is a MySQL clone. Is that correct?
I created a database called mailman. I am not sure what to change the for the following variables in the docker-compose.yaml file:
mailman-core: environment: - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb - DATABASE_TYPE=postgres - DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase
mailman-web: environment: - DATABASE_TYPE=postgres - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb
database: environment: POSTGRES_DB: ? POSTGRES_USER: ? POSTGRES_PASSWORD: ?
Ok I just discovered the docker-compose-mysql.yaml file. I assume I can just use that file:
docker-compose-mysql -d
Is that correct? Do I need to remove the previous docker installation that I did using docker-compose.yaml?
Sorry for being so needy. This is the 2nd time I am installing an application using Docker. The first one was Discourse which had really good installation instructions. I am struggling with the instructions for Mailman 3 via Docker.
On Sun, Apr 1, 2018, at 12:40 PM, brian@emwd.com wrote:
Ok I just discovered the docker-compose-mysql.yaml file. I assume I can just use that file:
docker-compose-mysql -d
Yes, you can try using this and update the variables in the compose file similar to postgres.
If you update the environment variables to point to your local installation, you should be able to use just that instead of the the docker-version. You'd have to manually change the docker-compose.yaml file to use the on-host database though.
Is that correct? Do I need to remove the previous docker installation that I did using docker-compose.yaml?
Sorry for being so needy. This is the 2nd time I am installing an application using Docker. The first one was Discourse which had really good installation instructions. I am struggling with the instructions for Mailman 3 via Docker.
I am happy to update the docs to be more friendly and useful if you can point out the specifics of what isn't clear. Please feel free to open an issue on Github if you feel something isn't clear :)
-- Abhilash Raj maxking@asynchronous.in
On Sun, Apr 1, 2018, at 12:40 PM, brian(a)emwd.com wrote:
Yes, you can try using this and update the variables in the compose file similar to postgres.
Which variables are you referring to exactly?
If you update the environment variables to point to your local installation, you should be able to use just that instead of the the docker-version. You'd have to manually change the docker-compose.yaml file to use the on-host database though.
So I need to create the mysql database and username/password before running the docker installation?
Please explain "You'd have to manually change the docker-compose.yaml file to use the on-host database though." Even though I run docker-compose-mysql -d I still need to update the docker-compose.yaml file? See that is confusing for me. Again it is probably due to my inexperience in using docker.
I am happy to update the docs to be more friendly and useful if you can point out the specifics of what isn't clear. Please feel free to open an issue on Github if you feel something isn't clear :)
Thanks.
Ok I am trying this again. I am not sure why my last questions were not answered. Please let me know if I am doing something wrong in asking them. Here they are again:
"Please explain "You'd have to manually change the docker-compose.yaml file to use the on-host database though." Even though I run docker-compose-mysql -d I still need to update the docker-compose.yaml file? See that is confusing for me. Again it is probably due to my inexperience in using docker."
I think that is a legitimate question. There are two files being referenced here. If docker-compose-mysql.yaml has the correct setup then why do I need to manually change the docker-compose.yaml? It seems both files perform the same function except one references a Postgre DB and the other a MySQL DB.
On 04/04/2018 03:31 PM, brian@emwd.com wrote:
Ok I am trying this again. I am not sure why my last questions were not answered. Please let me know if I am doing something wrong in asking them. Here they are again:
"Please explain "You'd have to manually change the docker-compose.yaml file to use the on-host database though." Even though I run docker-compose-mysql -d I still need to update the docker-compose.yaml file? See that is confusing for me. Again it is probably due to my inexperience in using docker."
I think that is a legitimate question. There are two files being referenced here. If docker-compose-mysql.yaml has the correct setup then why do I need to manually change the docker-compose.yaml? It seems both files perform the same function except one references a Postgre DB and the other a MySQL DB.
The docker-compose files run a database inside a docker container as well. The difference between the "normal" and mysql version is the database being used. If you wish to run Mailman using docker with a database on the local host (not in a container) you will need to update the docker configurations to reflect that. The stuff you would need to change is:
- remove the database container from the compose file
- update the credentials to be able to connect to the database running on the host DATABASE_URL and DATABASE_TYPE. Note that each one of them exists twice!
There is no easy way to get the private IP address of the docker host in the compose-file. So you'll have to patch that in somehow. Here is an issue with pointers on how to do that https://github.com/docker/compose/issues/4092
As for your other issue: You don't need to setup anything yourself if you want to use docker for everything. It autocreates everything for you. You should change the credentials though before running it...
You are having trouble finding information because you want to partially use docker but leave the database on the host. If you want to tinker with how Mailman is supposed to be deployed using docker we can't start to document every possible thing you might want to change.
If you have specific issues with how to run them after reading https://asynchronous.in/docker-mailman/ please mention mention what part needs fixing/explanation
Hi Simon,
Thank you for your reply.
I did not realize that docker created everything for you so I prefer that. So let me get back to my original problems. I ran into the following after doing the normal docker installation:
curl http://172.19.199.3:8000/postorius/lists/ curl: (7) Failed connect to 172.19.199.3:8000; Connection refused
I was told to run docker ps. Here is the output from that:
ONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 073ecf209106 maxking/mailman-web:0.1 "docker-entrypoint..." 3 days ago Up 3 days 8000/tcp, 8080/tcp mailman-web aef8f44a7136 maxking/mailman-core:0.1 "docker-entrypoint..." 3 days ago Up 3 days 8001/tcp, 8024/tcp mailman-core 398e3fc05e2c postgres:9.6-alpine "docker-entrypoint..." 3 days ago Restarting (1) 36 hours ago dockermailman_database_1
I was asked to look at the logs which I don't have any: cd /opt/mailman/web/logs -bash: cd: /opt/mailman/web/logs: No such file or directory
I was asked to do docker logs mailman-web. It is filled with the following entries: psql: could not translate host name "database" to address: Try again Postgres is unavailable - sleeping
It is after the above when I got confused. So now I understand that the Docker installation creates the database and runs it. That is important to know. So if you can help me get through the above 4 issues, it would be very appreciated.
Thanks! Brian
- I was told to run docker ps. Here is the output from that:
ONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 073ecf209106 maxking/mailman-web:0.1 "docker-entrypoint..." 3 days ago Up 3 days 8000/tcp, 8080/tcp mailman-web aef8f44a7136 maxking/mailman-core:0.1 "docker-entrypoint..." 3 days ago Up 3 days 8001/tcp, 8024/tcp mailman-core 398e3fc05e2c postgres:9.6-alpine "docker-entrypoint..." 3 days ago Restarting (1) 36 hours ago dockermailman_database_1 the database container is failing. Inspect it's logs (docker logs dockermailman_database_1) to see what's going on. I'd recommend removing all directories that contain the database stuff before trying again.
The rest of the issues is just because the database is not there...
On 04/04/2018 05:43 PM, brian@emwd.com wrote:
The query came up:
chown: /var/lib/postgresql/data: Permission denied So your current user isn't allowed to write to the directories for the volumes in the docker-compose file.
Either provide the executing user with the privileges or change the location to somewhere the user can write to.
On 04/04/2018 05:43 PM, brian(a)emwd.com wrote: So your current user isn't allowed to write to the directories for the volumes in the docker-compose file.
Either provide the executing user with the privileges or change the location to somewhere the user can write to.
I am running everything as root user.
I bet it has something to do with selinux being enabled. I am disabling it now and rebooting the server.
Ok. That seemed to work and make everything worse. I tried to run 'docker-compose up -d' in the docker-mailman folder and got the following error:
Starting dockermailman_database_1 ... error
ERROR: for dockermailman_database_1 Cannot start service database: b'error creating overlay mount to /var/lib/docker/overlay2/360258d718714ab8b08bb01075ad11b0ac83e1e4fcaea1de12342f7038256be3/merged: invalid argument'
Here is a 2nd unanswered question:
"So I need to create the mysql database and username/password before running the docker installation?"
It seems simple enough to answer. The current installation documents does not make that clear. I recommend that you update the docs to where the above question is answered, at least for those who are using MySQL.
Ok. I made some headway here. I got the container to launch. Still getting the original error message "connection refused". However this time the database is running. Here is the error I get when I run:
docker logs mailman-web:
Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 307, in execute settings.INSTALLED_APPS File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 56, in __getattr__ self._setup(name) File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup self._wrapped = Settings(settings_module) File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 110, in __init__ mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) File "/opt/mailman-web/settings.py", line 60, in <module> MAILMAN_HOST_IP_AUTO = socket.gethostbyname('mailman-core') socket.gaierror: [Errno -2] Name does not resolve
On Wed, Apr 4, 2018, at 12:53 PM, brian@emwd.com wrote:
Ok. I made some headway here. I got the container to launch. Still getting the original error message "connection refused". However this time the database is running. Here is the error I get when I run:
docker logs mailman-web:
Traceback (most recent call last): File "manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/local/lib/python2.7/site-packages/django/core/management/ __init__.py", line 363, in execute_from_command_line utility.execute() File "/usr/local/lib/python2.7/site-packages/django/core/management/ __init__.py", line 307, in execute settings.INSTALLED_APPS File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 56, in __getattr__ self._setup(name) File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup self._wrapped = Settings(settings_module) File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 110, in __init__ mod = importlib.import_module(self.SETTINGS_MODULE) File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) File "/opt/mailman-web/settings.py", line 60, in <module> MAILMAN_HOST_IP_AUTO = socket.gethostbyname('mailman-core') socket.gaierror: [Errno -2] Name does not resolve
Can you show me how does your docker-compose.yaml looks like? Please remove anything sensitive like passwords or secrets.
This is a known error and has been fixed, but I am guessing that it isn't released yet, because of which you can still see this error. Can you open an issue here1 and I will fix it as soon as I can?
thanks, Abhilash
Mailman-users mailing list mailman-users@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- Abhilash Raj maxking@asynchronous.in
version: '2'
services: mailman-core: image: maxking/mailman-core:0.1 container_name: mailman-core hostname: mailman-core volumes: - /opt/mailman/core:/opt/mailman/ stop_grace_period: 30s links: - database:database depends_on: - database environment: - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb - DATABASE_TYPE=postgres - DATABASE_CLASS=mailman.database.postgresql.PostgreSQLDatabase - HYPERKITTY_API_KEY=***** networks: mailman: ipv4_address: 172.19.199.2
mailman-web: image: maxking/mailman-web:0.1 container_name: mailman-web hostname: mailman-web depends_on: - database links: - mailman-core:mailman-core - database:database volumes: - /opt/mailman/web:/opt/mailman-web-data environment: - DATABASE_TYPE=postgres - DATABASE_URL=postgres://mailman:mailmanpass@database/mailmandb - HYPERKITTY_API_KEY=***** - SECRET_KEY=******* - SERVE_FROM_DOMAIN=***** - MAILMAN_ADMIN_USER=***** - MAILMAN_ADMIN_EMAIL=***** networks: mailman: ipv4_address: 172.19.199.3
database: environment: POSTGRES_DB: mailmandb POSTGRES_USER: mailman POSTGRES_PASSWORD: mailmanpass restart: always image: postgres:9.6-alpine volumes: - /opt/mailman/database:/var/lib/postgresql/data networks: mailman: driver: bridge ipam: driver: default config: - subnet: 172.19.199.0/24
I have posted the error on Github. Did you see anything wrong with my docker-compose.yaml file?
I also have another question. For the HYPERKITTY_API_KEY setting, is there an API key I need to generate and retrieve from somewhere or do I just make one up?
Thanks, Brian
On Thu, Apr 5, 2018, at 8:36 AM, brian@emwd.com wrote:
I have posted the error on Github. Did you see anything wrong with my docker-compose.yaml file?
Probably nothing wrong with your config, I need to do a release :) It has been fixed.
I also have another question. For the HYPERKITTY_API_KEY setting, is there an API key I need to generate and retrieve from somewhere or do I just make one up?
It is just something random that you should create yourself and add.
-- Abhilash Raj maxking@asynchronous.in
Is there a way to get this new release and how do I update the existing one with it?
Another question:
Should I update the following settings with my own custom ones?
POSTGRES_DB: mailmandb POSTGRES_USER: mailman POSTGRES_PASSWORD: mailmanpass
Or does the docker installation configure those automatically?
Should I update the following settings with my own custom ones?
POSTGRES_DB: mailmandb POSTGRES_USER: mailman POSTGRES_PASSWORD: mailmanpass
These are the credentials that are used to create the postgres database. If you update them here, you need to update the database urls in the other parts of the config.
Sorry to dig up an old thread but was there any resolution to this? I've being trying the same thing but when I try to start the container get the same error:
[root@mailman-server docker-mailman]# docker-compose up -d Starting dockermailman_database_1 ... error
ERROR: for dockermailman_database_1 Cannot start service database: b'error creating overlay mount to /var/lib/docker/overlay2/55dfd09677e07a7b6a0dca0be5371e6c30eadc0ffa6ef297c3f401f1403d6348/merged: invalid argument'
ERROR: for database Cannot start service database: b'error creating overlay mount to /var/lib/docker/overlay2/55dfd09677e07a7b6a0dca0be5371e6c30eadc0ffa6ef297c3f401f1403d6348/merged: invalid argument' ERROR: Encountered errors while bringing up the project.
selinux is turned off on the server I'm using which is Centos 7 same as the OP.
The config files I've modified as mentioned in the documentation are: docker-compose.yaml in docker-mailman mailman-extra.cfg in mailman/core settings_local.py in mailman/web
Thanks!
Thanks!
Found the answer to this, the version of docker and/or docker-compose that comes from the Centos "official" repo was the problem. Once I used the download.docker.com repo, everything worked ok.
Found the install steps here - https://github.com/NaturalHistoryMuseum/scratchpads2/wiki/Install-Docker-and...)
participants (4)
-
Abhilash Raj
-
brian@emwd.com
-
john.kissane@tyndall.ie
-
Simon Hanna