docker mailman-core mailman3 image not responding on port 8024

bob B

17 Aug 2021 17 Aug '21

2:17 p.m.

Ok, I took a step back and started fresh... So the issue appears to be that mailman-core docker image is not replying on port 8024.

I did a packet capture and it shows (172.23.01 is the gateway for the docker network on the host, 172.23.03 is the IP of mailman-core when I did the capture)

1 2021-08-16 14:15:14.865053 172.23.0.1 172.23.0.3 TCP 74 34066 → 8024 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=985607751 TSecr=0 WS=128 2 2021-08-16 14:15:14.865084 172.23.0.3 172.23.0.1 TCP 54 8024 → 34066 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

The above show the docker host sending/relaying mail to mailman-core, but mailman-core does not acknowledge on port 8024.

The images are up i.e

58923db6fb9f maxking/mailman-core:0.3 "docker-entrypoint.s…" 26 minutes ago Up 25 minutes 127.0.0.1:8001->8001/tcp, 127.0.0.1:8024->8024/tcp mailman-core

Docker yamel has the port setup (8024)

services: mailman-core: image: maxking/mailman-core:0.3 container_name: mailman-core hostname: mailman-core . . ports: - "127.0.0.1:8001:8001" # API - "127.0.0.1:8024:8024" # LMTP - incoming emails networks: mailman:

I can nc to the ports for docker web & 8001 on core and they are listening

But 8024 is not responding

#>docker-mailman-host]# nc -v 172.23.0.4 8000 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.23.0.4:8000. ^C #>docker-mailman-host]# nc -v 172.23.0.4 8080 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.23.0.4:8080. ^C #>docker-mailman-host]# nc -v 172.23.0.3 8001 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.23.0.3:8001. ^C

#>docker-mailman-host]# nc -v 172.23.0.3 8024 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused

Docker logs show postfix is up on mailman-core

Postgres is up - continuing Using Postfix configuration Found configuration file at /opt/mailman/mailman-extra.cfg HYPERKITTY_URL not set, using the default value of http://mailman-web:8000/hyperkitty

Any ideas is there anything else I can do to troubleshoot? Is the issue with the mailman-core image?

Show replies by date

Abhilash Raj

17 Aug 17 Aug

6 p.m.

Sorry for stepping in this very late.

The problems here are possibly due to the recent changes in the docker-mailman related to removal of static IPs in the container images. The changes have been reported working by some folks, but i know that they aren’t really using the docker-compose.yaml provided in the repo or have highly customized versions of those so the combination of both might be broken.

It is also a problem that the documentation isn’t versioned properly to signify the difference that has been made in the repo since the last released version of the container image :(

Either way, can you share the contents of /etc/mailman.cfg from inside the mailman-core container? That is most definitive source of truth for what actually configuration is being passed to Mailman Core to infer which actual host:port is the LMTP runner listening to and if the configuration being passed to it are what we expect.

Abhilash

...

On Aug 17, 2021, at 7:17 AM, bob B <bader@cchmc.org> wrote:

Ok, I took a step back and started fresh... So the issue appears to be that mailman-core docker image is not replying on port 8024.

I did a packet capture and it shows (172.23.01 is the gateway for the docker network on the host, 172.23.03 is the IP of mailman-core when I did the capture)

1 2021-08-16 14:15:14.865053 172.23.0.1 172.23.0.3 TCP 74 34066 → 8024 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=985607751 TSecr=0 WS=128 2 2021-08-16 14:15:14.865084 172.23.0.3 172.23.0.1 TCP 54 8024 → 34066 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

The above show the docker host sending/relaying mail to mailman-core, but mailman-core does not acknowledge on port 8024.

The images are up i.e

58923db6fb9f maxking/mailman-core:0.3 "docker-entrypoint.s…" 26 minutes ago Up 25 minutes 127.0.0.1:8001->8001/tcp, 127.0.0.1:8024->8024/tcp mailman-core

Docker yamel has the port setup (8024)

services: mailman-core: image: maxking/mailman-core:0.3 container_name: mailman-core hostname: mailman-core . . ports:

"127.0.0.1:8001:8001" # API

"127.0.0.1:8024:8024" # LMTP - incoming emails networks: mailman:

I can nc to the ports for docker web & 8001 on core and they are listening

But 8024 is not responding

#>docker-mailman-host]# nc -v 172.23.0.4 8000 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.23.0.4:8000. ^C #>docker-mailman-host]# nc -v 172.23.0.4 8080 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.23.0.4:8080. ^C #>docker-mailman-host]# nc -v 172.23.0.3 8001 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connected to 172.23.0.3:8001. ^C

#>docker-mailman-host]# nc -v 172.23.0.3 8024 Ncat: Version 7.50 ( https://nmap.org/ncat ) Ncat: Connection refused

Docker logs show postfix is up on mailman-core

sudo docker logs mailman-core List of databases Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+---------+----------+------------+------------+--------------------- mailmandb | mailman | UTF8 | en_US.utf8 | en_US.utf8 | postgres | mailman | UTF8 | en_US.utf8 | en_US.utf8 | template0 | mailman | UTF8 | en_US.utf8 | en_US.utf8 | =c/mailman + | | | | | mailman=CTc/mailman template1 | mailman | UTF8 | en_US.utf8 | en_US.utf8 | =c/mailman + | | | | | mailman=CTc/mailman (4 rows)

Postgres is up - continuing Using Postfix configuration Found configuration file at /opt/mailman/mailman-extra.cfg HYPERKITTY_URL not set, using the default value of http://mailman-web:8000/hyperkitty

Any ideas is there anything else I can do to troubleshoot? Is the issue with the mailman-core image?

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

-- thanks, Abhilash Raj (maxking)

bob B

6:37 p.m.

I restarted the containers so the new/current ips are

core "IPAddress": "172.25.0.3", web "IPAddress": "172.25.0.4",

Here is the content of that file

docker exec -it mailman-core /bin/bash bash-5.0# cat /etc/mailman.cfg # This file is autogenerated at container startup. [database] class: mailman.database.postgresql.PostgreSQLDatabase url: postgres://######:#########@database/mailmandb [runner.retry] sleep_time: 10s

[webservice] hostname: 172.25.0.3 port: 8001 admin_user: restadmin admin_pass: restpass configuration: /etc/gunicorn.cfg

[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /etc/mailman-hyperkitty.cfg

[mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: 172.25.0.3 lmtp_port: 8024 smtp_host: 172.19.199.1 smtp_port: 25 configuration: /etc/postfix-mailman.cfg

# mailman-extra.cfg

[mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver # mailman-core hostname or IP from the Postfix server #lmtp_host: localhost lmtp_host: 127.0.0.1 lmtp_port: 8024 # Postfix server's hostname or IP from mailman-core smtp_host: ########## smtp_port: 25 configuration: /etc/postfix-mailman.cfg [mailman] # This address is the "site owner" address. Certain messages which must be # delivered to a human, but which can't be delivered to a list owner (e.g. a # bounce from a list owner), will be sent to this address. It should point to # a human. site_owner: ########## bash-5.0#

Abhilash Raj

8:19 p.m.

...

On Aug 17, 2021, at 11:37 AM, bob B <bader@cchmc.org> wrote:

I restarted the containers so the new/current ips are

core "IPAddress": "172.25.0.3", web "IPAddress": "172.25.0.4",

Here is the content of that file

docker exec -it mailman-core /bin/bash bash-5.0# cat /etc/mailman.cfg # This file is autogenerated at container startup.

<snip>

...

[mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver lmtp_host: 172.25.0.3 lmtp_port: 8024 smtp_host: 172.19.199.1 smtp_port: 25 configuration: /etc/postfix-mailman.cfg

Looking at your [mta] configs, it seems like incoming configs are correct, but outgoing configs aren’t. You have the Mailman Core binding to 172.25.0.3, which is the IP address allocated to it from Docker and it should be accepting connections on 172.25.0.3:8024.

The outgoing config will use the hard coded IP configs in the 0.3.11 release of the container images, but you can fix that by setting SMTP_HOST: 172.25.0.1 in both mailman-core and mailman-web container configurations in docker-compose.yaml file under “environment”. I am assuming that Postfix is listening on all interfaces (0.0.0.0, which is usually the case that I have seen, but something still that perhaps should be verified).

...

# mailman-extra.cfg

[mta] incoming: mailman.mta.postfix.LMTP outgoing: mailman.mta.deliver.deliver # mailman-core hostname or IP from the Postfix server #lmtp_host: localhost lmtp_host: 127.0.0.1 lmtp_port: 8024 # Postfix server's hostname or IP from mailman-core smtp_host: ########## smtp_port: 25 configuration: /etc/postfix-mailman.cfg [mailman] # This address is the "site owner" address. Certain messages which must be # delivered to a human, but which can't be delivered to a list owner (e.g. a # bounce from a list owner), will be sent to this address. It should point to # a human. site_owner: ########## bash-5.0#

This part is probably the one causing trouble, probably coming from what you added to mailman-extra.cfg. This overrides the configs above (which is right for incoming emails) ones. Mailman is binding to 127.0.0.1 inside the container, and that loopback interface isn’t accessible from anywhere outside the container itself, so you won’t be able to talk to it. It needs to listen either on 0.0.0.0 (all interfaces) or (at least, one of the) public IPs assigned to the container.

The fix is that you just want to delete the entire [mta] section of mailman-extra.cfg and restart the container. Do also setup the SMTP_HOST environment to fix that outbound email issue.

-- thanks, Abhilash Raj (maxking)

bob B

18 Aug 18 Aug

8 p.m.

Thanks,

but if I put SMTP_HOST: 172.25.0.1 in the docker.yaml, and then down and up (docker-compose down, docker-compose up -d) the images then container grab a new IP range, (like 172.26.0.X). but my smtphost is 172.25.0.1, so that would break again, correct? What should i put for SMTP_HOST: XXXXX in doicker.yaml to keep this from happening?

And are you saying my /opt/mailman/core/mailman-extra.cfg should be

# cat /opt/mailman/core/mailman-extra.cfg # mailman-extra.cfg

[mailman] # This address is the "site owner" address. Certain messages which must be # delivered to a human, but which can't be delivered to a list owner (e.g. a # bounce from a list owner), will be sent to this address. It should point to # a human. site_owner: XXXX@XXXX.XXXXX

Abhilash Raj

19 Aug 19 Aug

3:09 a.m.

...

On Aug 18, 2021, at 1:00 PM, bob B <bader@cchmc.org> wrote:

Thanks,

but if I put SMTP_HOST: 172.25.0.1 in the docker.yaml, and then down and up (docker-compose down, docker-compose up -d) the images then container grab a new IP range, (like 172.26.0.X). but my smtphost is 172.25.0.1, so that would break again, correct?

That is correct.

...

What should i put for SMTP_HOST: XXXXX in doicker.yaml to keep this from happening?

You can grab a static subnet and assign it to your compose file. This is what old version of the docker-compose.yaml used to do, see this for example

https://github.com/maxking/docker-mailman/blob/v0.3.11/docker-compose.yaml#L...

You probably don’t need to assign static IPs to each container like in the compose file above, since all you need the gateway address to be static to reach SMTP_HOST.

...

And are you saying my /opt/mailman/core/mailman-extra.cfg should be

# cat /opt/mailman/core/mailman-extra.cfg # mailman-extra.cfg

[mailman] # This address is the "site owner" address. Certain messages which must be # delivered to a human, but which can't be delivered to a list owner (e.g. a # bounce from a list owner), will be sent to this address. It should point to # a human. site_owner: XXXX@XXXX.XXXXX

That is correct!

-- thanks, Abhilash Raj (maxking)

bob B

23 Aug 23 Aug

7:01 p.m.

Sorry can you give a clear example of how to hard code ips,

I get this

docker-compose up -d Removing dockermailman_database_1 Recreating 2eaf91c7d09d_2eaf91c7d09d_dockermailman_database_1 ... error

ERROR: for 2eaf91c7d09d_2eaf91c7d09d_dockermailman_database_1 b'user specified IP address is supported only when connecting to networks with user configured subnets'

My yaml file is based of your example

cat docker-compose.yaml version: '2'

services: mailman-core: image: maxking/mailman-core:0.3 container_name: mailman-core hostname: mailman-core volumes: - /opt/mailman/core:/opt/mailman/ stop_grace_period: 30s links: - database:database depends_on: - database environment: - SMTP_HOST=172.19.199.2 . . . ports: - "127.0.0.1:8001:8001" # API - "127.0.0.1:8024:8024" # LMTP - incoming emails networks: mailman: ipv4_address: 172.19.199.2

mailman-web: image: maxking/mailman-web:0.3 container_name: mailman-web hostname: mailman-web depends_on: - database links: - mailman-core:mailman-core - database:database volumes: - /opt/mailman/web:/opt/mailman-web-data environment: . . . ports: - "127.0.0.1:8000:8000" # HTTP - "127.0.0.1:8080:8080" # uwsgi networks: mailman: ipv4_address: 172.19.199.3

database: environment: . . image: postgres:9.6-alpine volumes: - /opt/mailman/database:/var/lib/postgresql/data networks: mailman: ipv4_address: 172.19.199.4

networks: mailman: driver: bridge ipam: driver: default config: - subnet: 172.19.199.0/24

bob B

24 Aug 24 Aug

7:32 p.m.

Anyone have any ideas on how to hard code IPs in docker for mailman3

matthew＠alberti.us

25 Aug 25 Aug

12:12 p.m.

Hi Bob, Here is my working config. I'm not certain; but you may need to destroy your containers and recreate them at this point. I'm not sure if you will be able to just change the YAML and have the existing non-static IP containers work correctly.

cat docker-compose.yaml version: '2'

services: mailman-core: image: maxking/mailman-core:0.3 container_name: mailman-core hostname: mailman-core restart: always volumes: - /opt/mailman/core:/opt/mailman/ links: - database:database depends_on: - database environment: - DATABASE_URL=mysql+pymysql://mailman:[redacted]@database/mailmandb?charset=u tf8mb4&use_unicode=1 - DATABASE_TYPE=mysql - DATABASE_CLASS=mailman.database.mysql.MySQLDatabase - HYPERKITTY_API_KEY=[redacted] - MTA=postfix networks: mailman: ipv4_address: 172.19.199.2

mailman-web: image: maxking/mailman-web:0.3 container_name: mailman-web hostname: mailman-web restart: always depends_on: - database links: - mailman-core:mailman-core - database:database volumes: - /opt/mailman/web:/opt/mailman-web-data environment: - DATABASE_URL=mysql://mailman: [redacted]@database/mailmandb - DATABASE_TYPE=mysql - HYPERKITTY_API_KEY=[redacted] - SECRET_KEY=[redacted] - DYLD_LIBRARY_PATH=/usr/local/mysql/lib/ - SERVE_FROM_DOMAIN=[redacted] - MAILMAN_ADMIN_USER=mailmanadmin - MAILMAN_ADMIN_EMAIL=[redacted] networks: mailman: ipv4_address: 172.19.199.3

database: container_name: mailman-database environment: MYSQL_DATABASE: mailmandb MYSQL_USER: mailman MYSQL_PASSWORD: [redacted] MYSQL_RANDOM_ROOT_PASSWORD: "yes" restart: always image: mariadb:10.5 volumes: - /opt/mailman/database:/var/lib/mysql networks: mailman: ipv4_address: 172.19.199.4

networks: mailman: name: docker-mailman driver: bridge ipam: driver: default config: - subnet: 172.19.199.0/24

-----Original Message----- From: bob B <bader@cchmc.org> Sent: Wednesday, August 25, 2021 1:32 AM To: mailman-users@mailman3.org Subject: [MM3-users] Re: docker mailman-core mailman3 image not responding on port 8024

Anyone have any ideas on how to hard code IPs in docker for mailman3

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

bob B

4:05 p.m.

thanks, so i stop all containers

docker-mailman]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [SERVER1 docker-mailman]#

I added this to my docker.yaml

networks: mailman: driver: bridge ipam: driver: default config: - subnet: 172.19.199.0/24 gateway: 172.19.199.1

I specified ips on my containers in docker.yaml i.e

. . . mailman: ipv4_address: 172.19.199.3

When I try to start I get the below i get the below error?

docker-compose up -d Removing dockermailman_database_1 Recreating 2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_dockermailman_dRecreating 2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_dockermailman_database_1 ... error

ERROR: for 2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_2eaf91c7d09d_dockermailman_database_1 b'user specified IP address is supported only when connecting to networks with user configured subnets'

ERROR: for database b'user specified IP address is supported only when connecting to networks with user configured subnets' ERROR: Encountered errors while bringing up the project.

Here is my docker version could that be the issue???

Client: Docker Engine - Community Version: 20.10.7 API version: 1.41 Go version: go1.13.15 Git commit: f0df350 Built: Wed Jun 2 11:58:10 2021 OS/Arch: linux/amd64 Context: default Experimental: true

Server: Docker Engine - Community Engine: Version: 20.10.7 API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: b0f5bc3 Built: Wed Jun 2 11:56:35 2021 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.4.6 GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d runc: Version: 1.0.0-rc95 GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 docker-init: Version: 0.19.0 GitCommit: de40ad0 docker-mailman]#

bob B

5:11 p.m.

so it looks like i needed to do docker-compose down
and ten docker-compose up-d.

now containers are starting and smtp is getting into the mailman-core now.

1186

Age (days ago)

1194

Last active (days ago)

List overview

Download

10 comments

3 participants

participants (3)

Abhilash Raj
bob B
matthew＠alberti.us

docker mailman-core mailman3 image not responding on port 8024

tags

participants (3)