Mailman was installed via https://docs.mailman3.org/en/latest/install/virtualenv.html instead of using the Debian package.

Please let me know if you have any recommended trouble shooting tips.

Thanks,

Gordon

This is the entire error sent via email every minute or so:

Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman-web", line 5, in <module> from mailman_web.manage import main ModuleNotFoundError: No module named 'mailman_web'

When I try to login to mailman then I get thse following error:

Service Unavailable: /mailman3/lists/

Report at /mailman3/lists/ Service Unavailable: /mailman3/lists/

Request Method: GET Request URL: http://host2.mailhub4u.com/mailman3/lists/ Django Version: 3.0.14 Python Executable: /opt/mailman/venv/bin/uwsgi Python Version: 3.9.2 Python Path: ['.', '', '/etc/mailman3', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/opt/mailman/venv/lib/python3.9/site-packages'] Server time: Tue, 31 Oct 2023 21:08:56 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')

Request information: USER: gordon

GET: No GET data

POST: No POST data

FILES: No FILES data

COOKIES: csrftoken = 'BgEa4GddvSCScj8Bm9aSJz2V2vuvQin3boRLAM4ZqjL8Aq1I0l989xbv6nI2LbuS' PHPSESSID = 'a36fftlm8j0khrlciu3uut8n5a' testing = '1' sid = '8a3f6ce1fe7c07389eea7d68f5346563' sessiontest = '1' Horde = '48dflipuqm8oi32on7dadjffl8' horde_secret_key = '4jhq7cri14q2vdkvs3r6r2auqk' sessionid = 'gsrrjekrl6huw5myr37lop21cyltpmst'

META: CSRF_COOKIE = 'BgEa4GddvSCScj8Bm9aSJz2V2vuvQin3boRLAM4ZqjL8Aq1I0l989xbv6nI2LbuS' HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING = 'gzip, deflate, br' HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.5' HTTP_AUTHORIZATION = 'Basic cm9vdDokQ3JAYmJ5LVAzbGljYW4k' HTTP_CONNECTION = 'Keep-Alive' HTTP_COOKIE = 'csrftoken=BgEa4GddvSCScj8Bm9aSJz2V2vuvQin3boRLAM4ZqjL8Aq1I0l989xbv6nI2LbuS; PHPSESSID=a36fftlm8j0khrlciu3uut8n5a; testing=1; sid=8a3f6ce1fe7c07389eea7d68f5346563; sessiontest=1; Horde=48dflipuqm8oi32on7dadjffl8; horde_secret_key=4jhq7cri14q2vdkvs3r6r2auqk; sessionid=gsrrjekrl6huw5myr37lop21cyltpmst' HTTP_HOST = '127.0.0.1:8000' HTTP_REFERER = 'https://host2.mailhub4u.com/mailman3/lists/' HTTP_SEC_FETCH_DEST = 'document' HTTP_SEC_FETCH_MODE = 'navigate' HTTP_SEC_FETCH_SITE = 'same-origin' HTTP_SEC_FETCH_USER = '?1' HTTP_UPGRADE_INSECURE_REQUESTS = '1' HTTP_USER_AGENT = 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/119.0' HTTP_X_FORWARDED_FOR = '173.165.166.34' HTTP_X_FORWARDED_HOST = 'host2.mailhub4u.com' HTTP_X_FORWARDED_SERVER = 'host2.mailhub4u.com' PATH_INFO = '/mailman3/lists/' QUERY_STRING = '' REMOTE_ADDR = '127.0.0.1' REQUEST_METHOD = 'GET' REQUEST_URI = '/mailman3/lists/' SCRIPT_NAME = '' SERVER_NAME = 'slc.mailhub4u.com' SERVER_PORT = '8000' SERVER_PROTOCOL = 'HTTP/1.1' uwsgi.core = 0 uwsgi.node = b'slc.mailhub4u.com' uwsgi.version = b'2.0.20' wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'> wsgi.file_wrapper = '' wsgi.input = <uwsgi._Input object at 0x7f509a270810> wsgi.multiprocess = False wsgi.multithread = True wsgi.run_once = False wsgi.url_scheme = 'http' wsgi.version = '(1, 0)'

Settings: Using settings module settings ABSOLUTE_URL_OVERRIDES = {} ACCOUNT_AUTHENTICATION_METHOD = 'username_email' ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https' ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_EMAIL_VERIFICATION = 'mandatory' ACCOUNT_UNIQUE_EMAIL = True ADMINS = "(('Mailman Suite Admin', 'gordon@mailhub4u.com'),)" ALLOWED_HOSTS = ['*'] APPEND_SLASH = True AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')" AUTH_PASSWORD_VALIDATORS = '********************' AUTH_USER_MODEL = 'auth.User' BASE_DIR = PosixPath('/opt/mailman/web') CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}} CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_KEY_PREFIX = '********************' CACHE_MIDDLEWARE_SECONDS = 600 COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'} COMPRESS_CACHEABLE_PRECOMPILERS = '()' COMPRESS_CACHE_BACKEND = 'default' COMPRESS_CACHE_KEY_FUNCTION = '********************' COMPRESS_CLEAN_CSS_ARGUMENTS = '' COMPRESS_CLEAN_CSS_BINARY = 'cleancss' COMPRESS_CLOSURE_COMPILER_ARGUMENTS = '' COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar' COMPRESS_CSS_HASHING_METHOD = 'mtime' COMPRESS_DATA_URI_MAX_SIZE = 1024 COMPRESS_DEBUG_TOGGLE = None COMPRESS_ENABLED = True COMPRESS_FILTERS = {'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'], 'js': ['compressor.filters.jsmin.rJSMinFilter']} COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f509d83d0d0> COMPRESS_MINT_DELAY = 30 COMPRESS_MTIME_DELAY = 10 COMPRESS_OFFLINE = True COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/static/'} COMPRESS_OFFLINE_MANIFEST = 'manifest.json' COMPRESS_OFFLINE_TIMEOUT = 31536000 COMPRESS_OUTPUT_DIR = 'CACHE' COMPRESS_PARSER = 'compressor.parser.AutoSelectParser' COMPRESS_PRECOMPILERS = "(('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))" COMPRESS_REBUILD_TIMEOUT = 2592000 COMPRESS_ROOT = '/var/lib/mailman3/web/static' COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage' COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/static/'} COMPRESS_URL = '/static/' COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/' COMPRESS_VERBOSE = False COMPRESS_YUGLIFY_BINARY = 'yuglify' COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal' COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal' COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar' COMPRESS_YUI_CSS_ARGUMENTS = '' COMPRESS_YUI_JS_ARGUMENTS = '' CSRF_COOKIE_AGE = 31449600 CSRF_COOKIE_DOMAIN = None CSRF_COOKIE_HTTPONLY = False CSRF_COOKIE_NAME = 'csrftoken' CSRF_COOKIE_PATH = '/' CSRF_COOKIE_SAMESITE = 'Lax' CSRF_COOKIE_SECURE = False CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure' CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN' CSRF_TRUSTED_ORIGINS = [] CSRF_USE_SESSIONS = False DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4'}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}} DATABASE_ROUTERS = [] DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440 DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000 DATETIME_FORMAT = 'N j, Y, P' DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y'] DATE_FORMAT = 'N j, Y' DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y'] DEBUG = False DEBUG_PROPAGATE_EXCEPTIONS = False DECIMAL_SEPARATOR = '.' DEFAULT_CHARSET = 'utf-8' DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter' DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage' DEFAULT_FROM_EMAIL = 'postorius@host2.mailhub4u.com' DEFAULT_INDEX_TABLESPACE = '' DEFAULT_TABLESPACE = '' DISALLOWED_USER_AGENTS = [] EMAILNAME = 'host2.mailhub4u.com' EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST = 'localhost' EMAIL_HOST_PASSWORD = '********************' EMAIL_HOST_USER = '' EMAIL_PORT = 25 EMAIL_SSL_CERTFILE = None EMAIL_SSL_KEYFILE = '********************' EMAIL_SUBJECT_PREFIX = '[Django] ' EMAIL_TIMEOUT = None EMAIL_USE_LOCALTIME = False EMAIL_USE_SSL = False EMAIL_USE_TLS = False FILE_CHARSET = 'utf-8' FILE_UPLOAD_DIRECTORY_PERMISSIONS = None FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler'] FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440 FILE_UPLOAD_PERMISSIONS = 420 FILE_UPLOAD_TEMP_DIR = None FILTER_VHOST = False FIRST_DAY_OF_WEEK = 0 FIXTURE_DIRS = [] FORCE_SCRIPT_NAME = None FORMAT_MODULE_PATH = None FORM_RENDERER = 'django.forms.renderers.DjangoTemplates' HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': 'fulltext_index'}} HYPERKITTY_ENABLE_GRAVATAR = True IGNORABLE_404_URLS = [] INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')" INTERNAL_IPS = [] LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('hy', 'Armenian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kab', 'Kabyle'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('uz', 'Uzbek'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')] LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur'] LANGUAGE_CODE = 'en-us' LANGUAGE_COOKIE_AGE = None LANGUAGE_COOKIE_DOMAIN = None LANGUAGE_COOKIE_HTTPONLY = False LANGUAGE_COOKIE_NAME = 'django_language' LANGUAGE_COOKIE_PATH = '/' LANGUAGE_COOKIE_SAMESITE = None LANGUAGE_COOKIE_SECURE = False LOCALE_PATHS = [] LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.WatchedFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'ERROR', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'ERROR', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'DEBUG', 'propagate': True}, 'postorius': {'handlers': ['console', 'file'], 'level': 'INFO'}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}} LOGGING_CONFIG = 'logging.config.dictConfig' LOGIN_REDIRECT_URL = 'list_index' LOGIN_URL = 'account_login' LOGOUT_REDIRECT_URL = None LOGOUT_URL = 'account_logout' MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '209.95.44.128', '::1')" MAILMAN_ARCHIVER_KEY = '********************' MAILMAN_REST_API_PASS = '********************' MAILMAN_REST_API_URL = '********************' MAILMAN_REST_API_USER = '********************' MANAGERS = [] MEDIA_ROOT = '' MEDIA_URL = '' MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage' MESSAGE_TAGS = {40: 'danger'} MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')" MIGRATION_MODULES = {} MONTH_DAY_FORMAT = 'F j' NUMBER_GROUPING = 0 PASSWORD_HASHERS = '********************' PASSWORD_RESET_TIMEOUT_DAYS = '********************' POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/' PREPEND_WWW = False Q_CLUSTER = {'retry': 360, 'timeout': 300, 'save_limit': 100, 'orm': 'default'} ROOT_URLCONF = 'mailman_web.urls' SECRET_KEY = '********************' SECURE_BROWSER_XSS_FILTER = False SECURE_CONTENT_TYPE_NOSNIFF = True SECURE_HSTS_INCLUDE_SUBDOMAINS = False SECURE_HSTS_PRELOAD = False SECURE_HSTS_SECONDS = 0 SECURE_PROXY_SSL_HEADER = None SECURE_REDIRECT_EXEMPT = [] SECURE_REFERRER_POLICY = None SECURE_SSL_HOST = None SECURE_SSL_REDIRECT = False SERVER_EMAIL = 'root@host2.mailhub4u.com' SESSION_CACHE_ALIAS = 'default' SESSION_COOKIE_AGE = 1209600 SESSION_COOKIE_DOMAIN = None SESSION_COOKIE_HTTPONLY = True SESSION_COOKIE_NAME = 'sessionid' SESSION_COOKIE_PATH = '/' SESSION_COOKIE_SAMESITE = 'Lax' SESSION_COOKIE_SECURE = False SESSION_ENGINE = 'django.contrib.sessions.backends.db' SESSION_EXPIRE_AT_BROWSER_CLOSE = False SESSION_FILE_PATH = None SESSION_SAVE_EVERY_REQUEST = False SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer' SETTINGS_MODULE = 'settings' SHORT_DATETIME_FORMAT = 'm/d/Y P' SHORT_DATE_FORMAT = 'm/d/Y' SIGNING_BACKEND = 'django.core.signing.TimestampSigner' SILENCED_SYSTEM_CHECKS = [] SITE_ID = 1 SOCIALACCOUNT_PROVIDERS = {} STATICFILES_DIRS = '()' STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')" STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' STATIC_ROOT = '/var/lib/mailman3/web/static' STATIC_URL = '/static/' TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}] TEST_NON_SERIALIZED_APPS = [] TEST_RUNNER = 'django.test.runner.DiscoverRunner' THOUSAND_SEPARATOR = ',' TIME_FORMAT = 'P' TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] TIME_ZONE = 'UTC' USE_I18N = True USE_L10N = True USE_THOUSAND_SEPARATOR = False USE_TZ = True USE_X_FORWARDED_HOST = True USE_X_FORWARDED_PORT = False WSGI_APPLICATION = 'mailman_web.wsgi.application' X_FRAME_OPTIONS = 'DENY' YEAR_MONTH_FORMAT = 'F Y'

Gordon:

Yes, you will have to rebuild the venv every time you change major Python versions. This is a side effect of how Python and VENVs work.

Thomas

-----Original Message----- From: gordon@dickens.com <gordon@dickens.com> Sent: Thursday, November 2, 2023 10:51 AM To: mailman-users@mailman3.org Subject: [MM3-users] Re: ModuleNotFoundError: No module named 'mailman_web'

Hi Odhiambo and Mark,

Odhiambo, thanks very much for your post! You may have hit the nail on the head in that I am now running Python3.11 and I am sure that I got there with the "apt autoremove" command which I did use during the OS upgrade. Why doesn't mailman3 work with Python3.11? Does this mean that I will need to rebuild mailman3's venv everytime that python is upgraded?

Thanks again!

Gordon

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...

This message sent to teward@thomas-ward.net

gordon@dickens.com writes:

I am now running Python3.11 [using] the "apt autoremove" command which I did use during the OS upgrade.

This is not a problem, but you do have to upgrade your venv as well.

Why doesn't mailman3 work with Python3.11?

It works fine with Python 3.11. The problem is that the point of a venv is to freeze everything -- including the Python binary -- "in place". However, because (a) Python is extremely stable across minor releases and (b) it is important to keep up with bug fixes, especially security fixes, in Python, the venv usually symlinks to the /usr/bin/python3.x binary (which is usually a symlink to a 3.x.y executable). If /usr/bin/python3.9 disappears, that will, of course, leave your Python 3.9 venv with a bunch of dangling symlinks and nothing in the venv will work.

Does this mean that I will need to rebuild mailman3's venv everytime that python is upgraded?

No.

/path/to/new-python3.y -m venv --upgrade /path/to/venv

should upgrade the venv to use Python 3.y, without changing any of the installed package sources. I believe it also recompiles all the source.py to __pycache__/source.cpython-3y.pyc files for you, but that is merely a minor time saving for the next time you start Mailman since Python will automatically recompile when you change versions.

If you have upgraded the /usr/bin/python3 in place (apparently you did) you can simplify the above to

python3 -m venv --upgrade /path/to/venv

and venv will figure out the right symlink for you.

Caveat, I have never tried this, I just don't uninstall the old Python installations because as a developer I need to test Mailman against them. But that's what "python3 -m venv --help" tells me. :-)

If you don't trust my untested suggestions (and why should you? ;-), you can try

PYTHONPATH=/path/to/venv/site-packages python3 -m pip freeze

and if that looks sane (ie, a list of Mailman and Django related packages and their versions), you can redirect it into requirements.txt. Then if necessary you can blow away the original venv and start new with the upgraded Python 3 and exactly the same package versions you had before with

python3 -m pip install -r requirements.txt

You can also avoid the venv upgrade entirely by using

python3 -m venv --copies /path/to/venv

to build the venv in the first place, which as I understand it will copy the whole Python installation into the venv. But then you lose the benefit of Python bug fixes from your distro and of course the space saving of symlinks.

Steve

Hi Stephen and Odhiambo,

My venv is located at /opt/mailman3/venv and so I used the command, "python3 -m venv --upgrade /opt/mailman/venv" to upgrade the venv which appeared to run successfully, however, afterwards the mailman3 service always exits immediately after the command "systemctl start mailman3". The mailman3-web service appears to run correctly, however, when I browse to the mailman3 web interface then I get the following error:

Something went wrong Mailman REST API not available. Please start Mailman core.

I will note that the Django Administration web interface was not working until I executed "python3 -m venv --upgrade /opt/mailman/venv" after which the Django web interface started working properly. However, as I noted, the mailman3 service is still not working since it always exits immediately after being started.

I have run out of ideas. Does anyone have any suggestions?

Thanks,

Gordon

Hi Odhiambo,

My mailman3 logs are empty (/var/log/mailman3/mailman.log). Nothing has been written to the log files since the OS upgrade. I do see the following in /var/log/syslog for each time that I restart the service with "systemctl start mailman3":

2023-11-06T14:00:19.549985-05:00 slc systemd[1]: Starting mailman3.service - LSB: Mailman3 server...

But I cannot find any log entries that might indicate why the service is exiting.

Can you be more specific about how to create a new venv and move the "var" directory, etc.

Thanks,

Gordon

Hi Stephen,

Thanks for your help!....

"systemctl stautus mailman" generates the following output:

mailman3.service - LSB: Mailman3 server Loaded: loaded (/etc/init.d/mailman3; generated) Active: active (exited) since Mon 2023-11-06 14:00:19 EST; 2 days ago Docs: man:systemd-sysv-generator(8) Process: 715495 ExecStart=/etc/init.d/mailman3 start (code=exited, status=0/SUCCESS) CPU: 1ms

Nov 06 14:00:19 slc.mailhub4u.com systemd[1]: Starting mailman3.service - LSB: Mailman3 server... Nov 06 14:00:19 slc.mailhub4u.com systemd[1]: Started mailman3.service - LSB: Mailman3 server.

According to my logs, the upgrade from Debian 11 to 12 began at exactly 2023-10-31 13:50. "journalctl -S "2023-10-31 13:50" -u mailman -x" generates the following output:

Oct 31 14:18:49 slc.mailhub4u.com systemd[1]: Stopping mailman.service - LSB: Mailman Master Queue Runner... Subject: A stop job for unit mailman.service has begun execution Defined-By: systemd Support: https://www.debian.org/support

A stop job for unit mailman.service has begun execution.

The job identifier is 19729. Oct 31 14:18:49 slc.mailhub4u.com systemd[1]: mailman.service: Deactivated successfully. Subject: Unit succeeded Defined-By: systemd Support: https://www.debian.org/support

The unit mailman.service has successfully entered the 'dead' state. Oct 31 14:18:49 slc.mailhub4u.com systemd[1]: Stopped mailman.service - LSB: Mailman Master Queue Runner. Subject: A stop job for unit mailman.service has finished Defined-By: systemd Support: https://www.debian.org/support

A stop job for unit mailman.service has finished.

The job identifier is 19729 and the job result is done. -- Boot 17a6ba2bed024461b8fbe21ebc9c5000 -- Oct 31 14:19:43 slc.mailhub4u.com systemd[1]: Starting mailman.service - LSB: Mailman Master Queue Runner... Subject: A start job for unit mailman.service has begun execution Defined-By: systemd Support: https://www.debian.org/support

A start job for unit mailman.service has begun execution.

The job identifier is 139. Oct 31 14:19:43 slc.mailhub4u.com systemd[1]: Started mailman.service - LSB: Mailman Master Queue Runner. Subject: A start job for unit mailman.service has finished successfully Defined-By: systemd Support: https://www.debian.org/support

A start job for unit mailman.service has finished successfully.

The job identifier is 139.

Note that the above journalctl command does not generate any output after 2023-10-31 14:19:43 although I have been running the startup script often.

"ps -e -f | grep mailman" generates the following output:

www-data 1304 1 0 Oct31 ? 00:00:54 /opt/mailman/venv/bin/uwsgi --ini /etc/mailman3/uwsgi.ini www-data 2027 1304 0 Oct31 ? 00:01:22 /opt/mailman/venv/bin/uwsgi --ini /etc/mailman3/uwsgi.ini www-data 2487504 1304 0 17:30 ? 00:00:00 /opt/mailman/venv/bin/uwsgi --ini /etc/mailman3/uwsgi.ini root 2487577 2462310 0 17:31 pts/0 00:00:00 grep mailman

Basically, it does not appear that mailman nor the runners are active and rebooting does not help. "/opt/mailman/venv/bin/mailman start" generates errors as follows:

Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 6, in <module> from pkg_resources import load_entry_point File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3248, in <module> @_call_aside ^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3223, in _call_aside f(*args, **kwargs) File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3261, in _initialize_master_working_set working_set = WorkingSet._build_master() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 617, in _build_master ws.require(__requires__) File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 956, in require needed = self.resolve(parse_requirements(requirements)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 815, in resolve dist = self._resolve_dist( ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 856, in _resolve_dist raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'mailman==3.3.5' distribution was not found and is required by the application

Thanks again!

Gordon

Hi Mark and Stephen,

The command, "/opt/mailman/venv/bin/mailman -C /etc/mailman3/mailman.cfg" generates the following error message:

Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 6, in <module> from pkg_resources import load_entry_point File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3248, in <module> @_call_aside ^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3223, in _call_aside f(*args, **kwargs) File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3261, in _initialize_master_working_set working_set = WorkingSet._build_master() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 617, in _build_master ws.require(__requires__) File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 956, in require needed = self.resolve(parse_requirements(requirements)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 815, in resolve dist = self._resolve_dist( ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 856, in _resolve_dist raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'mailman==3.3.5' distribution was not found and is required by the application

Also, the command "/opt/mailman/venv/bin/mailman -C /etc/mailman3/mailman.cfg start" generates a similar error message.

The contents of my file, /opt/mailman/venv/bin/mailman, is different from your suggested contents as follows:

#!/opt/mailman/venv/bin/python3 # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman' __requires__ = 'mailman==3.3.5' import re import sys from pkg_resources import load_entry_point

if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit( load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')() )

So, I put Mark's suggested contents in /opt/mailman/venv/bin/mailman and then executed the command:

/opt/mailman/venv/bin/mailman -C /etc/mailman3/mailman.cfg

which resulted in the following error message:

Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 5, in <module> from mailman.bin.mailman import main ModuleNotFoundError: No module named 'mailman'

Any more ideas?

Thanks,

Gordon

Hi Mark,

The command "pip install --force-reinstall mailman" ran successfully. Then, the command "/opt/mailman/venv/bin/mailman -C /etc/mailman3/mailman.cfg" generated the following errors:

Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts', 'mailman')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1077, in main with self.make_context(prog_name, args, **extra) as ctx: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 943, in make_context self.parse_args(ctx, args) File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1644, in parse_args rest = super().parse_args(ctx, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1408, in parse_args value, args = param.handle_parse_result(ctx, opts, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 2400, in handle_parse_result value = self.process_value(ctx, value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 2362, in process_value value = self.callback(ctx, self, value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py", line 95, in initialize_config initialize(value) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py", line 229, in initialize initialize_2(propagate_logs=propagate_logs) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py", line 187, in initialize_2 config.db = getUtility(IDatabaseFactory, utility_name).create() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/database/factory.py", line 57, in create database.initialize() File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/database/base.py", line 117, in initialize self.engine = create_engine( ^^^^^^^^^^^^^^ File "<string>", line 2, in create_engine File "/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py", line 281, in warned return fn(*args, **kwargs) # type: ignore[no-any-return] ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 601, in create_engine dbapi = dbapi_meth(**dbapi_args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/dialects/mysql/pymysql.py", line 75, in import_dbapi return __import__("pymysql") ^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'pymysql'

What to do now?

Thanks,

Gordon

Hi Mark,

Thanks very much for your continued help and assistance!

The command "pip install pymysql" ran successfully after which the command "/opt/mailman/venv/bin/mailman -C /etc/mailman3/mailman.cfg" then generates the following errors:

Traceback (most recent call last): File "/opt/mailman/venv/bin/mailman", line 33, in <module> sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts', 'mailman')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1077, in main with self.make_context(prog_name, args, **extra) as ctx: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 943, in make_context self.parse_args(ctx, args) File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1644, in parse_args rest = super().parse_args(ctx, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 1408, in parse_args value, args = param.handle_parse_result(ctx, opts, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 2400, in handle_parse_result value = self.process_value(ctx, value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line 2362, in process_value value = self.callback(ctx, self, value) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py", line 95, in initialize_config initialize(value) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py", line 229, in initialize initialize_2(propagate_logs=propagate_logs) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py", line 198, in initialize_2 initialize_pipelines() File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/pipelines.py", line 73, in initialize add_components('handlers', IHandler, config.handlers) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/utilities/modules.py", line 218, in add_components for component_class in find_pluggable_components(subpackage, interface): File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/utilities/modules.py", line 177, in find_pluggable_components yield from find_components('mailman.' + subpackage, interface) File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/utilities/modules.py", line 150, in find_components module = import_module(module_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/opt/mailman/venv/lib/python3.11/site-packages/mailman/handlers/validate_authenticity.py", line 23, in <module> from authheaders import authenticate_message File "/opt/mailman/venv/lib/python3.11/site-packages/authheaders/__init__.py", line 25, in <module> from authheaders.dmarc_lookup import dns_query, receiver_record, receiver_record_walk, get_org_domain File "/opt/mailman/venv/lib/python3.11/site-packages/authheaders/dmarc_lookup.py", line 22, in <module> from pkg_resources import resource_filename # Part of setuptools ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3327, in <module> @_call_aside ^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3302, in _call_aside f(*args, **kwargs) File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3340, in _initialize_master_working_set working_set = WorkingSet._build_master() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 631, in _build_master ws.require(__requires__) File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 968, in require needed = self.resolve(parse_requirements(requirements)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 829, in resolve dist = self._resolve_dist( ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py", line 870, in _resolve_dist raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'flufl.lock>=5.1' distribution was not found and is required by mailman

So, I executed "pip install flufl.lock" which responded as follows:

Requirement already satisfied: flufl.lock in ./venv/lib/python3.11/site-packages (8.0.2) Requirement already satisfied: atpublic in ./venv/lib/python3.11/site-packages (from flufl.lock) (4.0) Requirement already satisfied: psutil in ./venv/lib/python3.11/site-packages (from flufl.lock) (5.9.6)

So, I don't understand why I am now getting " The 'flufl.lock>=5.1' distribution was not found and is required by mailman" since when I attempted to install flufl.lock then pip responded that flufl.lock version 8.0.2 was already installed.

Any ideas?

Thanks,

Gordon

gordon@dickens.com writes:

"systemctl stautus mailman" generates the following output:

That's what I expected, consistent with your previous report.

...

According to my logs, the upgrade from Debian 11 to 12 began at exactly 2023-10-31 13:50. "journalctl -S "2023-10-31 13:50" -u mailman -x" generates the following output:

I didn't want any of that. I just wanted the log from the most recent attempt to start Mailman.

Note that the above journalctl command does not generate any output after 2023-10-31 14:19:43 although I have been running the startup script often.

I'm a SYSV init person. Perhaps the journalctl command I suggested was insufficiently inclusive. But that seems very strange to me.

"ps -e -f | grep mailman" generates the following output:

Hm. This has nothing to do with your problem as far as I know, but why is a uwsgi command in your venv/bin? Debian has a package for that, are you not using it? Did you do anything specific to put uwsgi there?

Oh, wait ... there's a possibility that your best bet is to delete the old venv as inconsistent and create a new one. In that case be sure you know how to get uwsgi installed where your startup scripts expect it.

Basically, it does not appear that mailman nor the runners are active and rebooting does not help. "/opt/mailman/venv/bin/mailman start" generates errors as follows: [...]

...

pkg_resources.DistributionNotFound: The 'mailman==3.3.5' distribution was not found and is required by the application

3.3.5 is quite old now. Is that the version of Mailman you've installed and were expecting? It seems likely to me that somehow the Mailman package has been upgraded to a more recent version but the venv/bin/mailman command has not been upgraded. Is that command owned by the mailman user?

Steve