Mark Sapiro writes:

On 11/21/22 16:33, Hirayama, Pat wrote:

...

mailmansuite.log is full of:

ERROR 2022-11-22 00:25:40,365 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured

Set

MAILMAN_ARCHIVER_FROM = '172.31.0.1'

in your Django settings, or to accept any IP set

MAILMAN_ARCHIVER_FROM = '*'

Hm. Pat wrote:

settings_local.py:

MAILMAN_ARCHIVER_FROM = ('172.18.8.14', '172.31.0.1', '172.31.0.2', '172.31.0.3', '172.31.0.4', '172.31.0.5', '127.0.0.1', '::1')

where '172.31.0.1' is the second element. So we need to figure out where HyperKitty is getting its setting for MAILMAN_ARCHIVER_FROM.

1. Are there multiple copies of settings_local.py lying around?
2. Does settings.py have "from settings_local import *" in it?
3. Are there multiple copies of settings.py lying around?
4. What else may have changed in the migration? Any software upgrades in mailman, hyperkitty, mailman-web (or wherever the settings*.py are coming from)?

Steve

First, my apologies for not taking note of your MAILMAN_ARCHIVER_FROM setting in your OP.

On 11/22/22 08:52, Hirayama, Pat wrote:

Steve wrote:

...

2. Does settings.py have "from settings_local import *" in it?

Yes. I assume that overrides the MAILMAN_ARCHIVER_FROM and other related settings defined at the top of the settings.py file? Or should I update settings.py based on what is in settings_local.py?

Assuming the from settings_local import * is at the end of settings.py, anything defined in settings_local.py will override the definition in settings.py. There's no need to change settings.py.

...

3. Are there multiple copies of settings.py lying around?

Well, there are these:

./usr/local/lib/python3.8/site-packages/rest_framework/settings.py

./usr/local/lib/python3.8/site-packages/postorius/doc/settings.py

./usr/local/lib/python3.8/site-packages/django_q/tests/settings.py

./usr/local/lib/python3.8/site-packages/django_extensions/settings.py

./usr/local/lib/python3.8/site-packages/mailmanclient/restobjects/settings.py

./usr/local/lib/python3.8/site-packages/example_project/settings.py

Only the last has MAILMAN_ARCHIVER_FROM defined in it, but I was assuming from the example_project subdirectory name that this was just a sample file, and besides, "from settings_local import *" is also set in it.

Yes, those are all samples.

The

ERROR 2022-11-22 00:25:40,365 7 hyperkitty.views.mailman Access to the 
archiving API endpoint was forbidden from IP 172.31.0.1, your 
MAILMAN_ARCHIVER_FROM setting may be misconfigured

message comes from https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/mailman... which only occurs if '172.31.0.1' is not in MAILMAN_ARCHIVER_FROM. Given that it is in MAILMAN_ARCHIVER_FROM as defined in settings_local.py, the only thing I can conclude is that the Django settings actually used is not the settings.py you think it is.

I don't know offhand even what wsgi service is used in the docker container or how it is configured, but it needs to set its working directory to the directory containing settings.py.

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 11/22/22 13:21, Hirayama, Pat wrote:

This is the end of settings.py: # # HyperKitty-specific #

# Only display mailing-lists from the same virtual host as the webserver FILTER_VHOST = False

POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost:8000'

try: from settings_local import * except ImportError: pass

I'm assuming that is close enough to the end?

Yes.

So, looking at the Django container, I see this process popping up periodically. I think that means that it is running from the directory specified by "--pythonpath"" (which is where settings.py and settings_local.py are located). I am further assuming that "--settings settings" is ok without a .py extension.

53972 mail 0:00 {django-admin} /usr/local/bin/python /usr/local/bin/django-admin runjobs quarter_hourly --pythonpath /opt/mailman/mailman-suite/mailman-suite_project/. --settings settings 53978 mail 0:00 {django-admin} /usr/local/bin/python /usr/local/bin/django-admin runjobs minutely --pythonpath /opt/mailman/mailman-suite/mailman-suite_project/. --settings settings

Yes, it is --settings settings without a .py extension because that's the name that Python imports.

However, those are from cron jobs and say nothing about the wsgi process that runs the web UI and the hyperkitty api.

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Hirayama, Pat phirayam@fredhutch.org wrote:

OK, I see uwsgi running in process table. PID USER TIME COMMAND 1 mail 0:13 uwsgi --ini uwsgi.ini 7 mail 1h11 uwsgi --ini uwsgi.ini

There is only one uwsgi.ini file, and it is in the same directory as settings.py, settings_local.py and wsgi.py

I had some similar issues last week when migrating from Mailman_Suite to Mailman_Web. Are there any settings in settings_local.py which you know are valid such as the database connection which you can identify as being in use and maybe modify to ensure you are pulling in the files from the right place? For example if you set DEFAULT_FROM_EMAIL in your settings_local.py to a specific email address, then restart the container, password resets via the web interface should come from that new address. Its an amateurish way of doing it possibly but I've been bitten by this before.

Thanks. Andrew.

-----Original Message----- From: Mark Sapiro mark@msapiro.net Sent: Tuesday, November 22, 2022 2:21 PM To: mailman-users@mailman3.org Subject: [MM3-users] Re: Hyperkitty no longer archiving

On 11/22/22 13:40, Hirayama, Pat wrote:

...

There is only one uwsgi.ini file, and it is in the same directory as settings.py,

settings_local.py and wsgi.py

What is the content of that file?

uwsgi.ini: [uwsgi] uwsgi-socket = :8000 chdir = /opt/mailman/mailman-suite/mailman-suite_project wsgi-file = /opt/mailman/mailman-suite/mailman-suite_project/wsgi.py harakiri = 60

master = true process = 4 threads = 4

uid = mail gid = mail

attach-daemon = ./manage.py qcluster

The referenced wsgi.py looks pretty vanilla:

"""

import os

# import sys # import site

# For some unknown reason, sometimes mod_wsgi fails to set the python paths to # the virtualenv, with the 'python-path' option. You can do it here too. # # # Remember original sys.path. # prev_sys_path = list(sys.path) # # Add here, for the settings module # site.addsitedir(os.path.abspath(os.path.dirname(__file__))) # # Add the virtualenv # venv = os.path.join(os.path.abspath(os.path.dirname(__file__)), # '..', 'lib', 'python2.6', 'site-packages') # site.addsitedir(venv) # # Reorder sys.path so new directories at the front. # new_sys_path = [] # for item in list(sys.path): # if item not in prev_sys_path: # new_sys_path.append(item) # sys.path.remove(item) # sys.path[:0] = new_sys_path

from django.core.wsgi import get_wsgi_application

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")

application = get_wsgi_application()

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

---

Mailman-users mailing list -- mailman-users@mailman3.org

Thanks! -p

Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam@fredhutch.org

Thank you, Mark!

-----Original Message----- From: Mark Sapiro mark@msapiro.net Sent: Tuesday, November 22, 2022 3:37 PM To: mailman-users@mailman3.org Subject: [MM3-users] Re: Hyperkitty no longer archiving

<snip> > > Does that directory also contain __init__.py (probably empty)? It should. >

That was it! No __init__.py in /opt/mailman/mailman-suite/mailman-suite_project. Creating an empty file there and restarting the containers caused hyperkitty to start archiving all of the queued messages.

Thank you for your time, and Happy Thanksgiving!

		-p

-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org

Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam@fredhutch.org