August 2024 - Mailman-users - lists.mailman3.org

Legit login attempts yield, "An error occurred while processing your request."
by Greg Newby 14 Aug '24

14 Aug '24

Hi friends. Following up on my message over the weekend about getting web (Django) logins for users who were ported from MM2 to MM3, I now have some people trying to create a new username. Something broke following a system update and reboot. MM3 is installed using the venv method, and the services (mailman and mailman-web) seem to be running ok. Mailing list delivery is happening as usual. I've now come upon a few related situations where a 500 error results, "Server error. An error occurred while processing your request." My primary question is: *How can I find out what's causing this*? I've looked in all the MM and web logfiles, by "tail -f" while generating the error, and nothing is logged there. Apache logs shows something like this: x.x.y.y- - [13/Aug/2024:21:46:41 -0700] "POST /accounts/password/reset/ HTTP/1.1" 500 1448 "https://lists.pglaf.org/accounts/password/reset/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0" I've come across this error while trying to do authentication-related activities that worked before the reboot: - "Forgot password" (happens whether or not the entered email existed in MM2 or already exists in MM3) - Sign up (ditto) - Email unsubscribes gave a verification email, but then didn't unsubscribe the address (might not be the same problem) Thanks for any breadcrumbs that might guide me to the solution. ~ Greg

2 1

Several Postorius errors over the weekend
by Andrew Hodgson 14 Aug '24

14 Aug '24

Hi. I've had several Postorius error messages which have resulted in an email. Is this someone up to mischief or have I missed a configuration somewhere? We had several of these over a short period and nothing since then which makes me think someone was running a script on the endpoint, but would be interested in confirmation from those that know better. Thanks. Andrew. Internal Server Error: /accounts/login/ TemplateDoesNotExist at /accounts/login/ 429.html Request Method: HEAD Request URL: http://lists.hodgsonfamily.org/accounts/login/?next=%2F%61%63%63%6F%75%6E%7… Django Version: 4.2.13 Python Executable: /opt/mailman/venv/bin/python3 Python Version: 3.12.3 Python Path: ['/opt/mailman', '/', '/opt/mailman/venv/bin', '/etc/mailman3', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/opt/mailman/venv/lib/python3.12/site-packages', '/usr/local/lib/python3.12/dist-packages', '/usr/lib/python3/dist-packages', '/etc/mailman3'] Server time: Fri, 09 Aug 2024 15:32:41 +0000 Installed Applications: ['hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount'] Installed Middleware: ('allauth.account.middleware.AccountMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Template loader postmortem Django tried loading these templates, in this order: Using engine django: This engine did not provide a list of tried templates. Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/django/views/generic/base.py", line 104, in view return self.dispatch(request, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/django/utils/decorators.py", line 46, in _wrapper return bound_method(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/allauth/decorators.py", line 10, in wrap resp = ratelimit.consume_or_429(request, action=action, **rl_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/allauth/core/ratelimit.py", line 140, in consume_or_429 return handler429(request) ^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/allauth/core/ratelimit.py", line 125, in _handler429 return render(request, "429." + app_settings.TEMPLATE_EXTENSION, status=429) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/django/shortcuts.py", line 24, in render content = loader.render_to_string(template_name, context, request, using=using) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/django/template/loader.py", line 61, in render_to_string template = get_template(template_name, using=using) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/mailman/venv/lib/python3.12/site-packages/django/template/loader.py", line 19, in get_template raise TemplateDoesNotExist(template_name, chain=chain) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Exception Type: TemplateDoesNotExist at /accounts/login/ Exception Value: 429.html Raised during: allauth.account.views.LoginView Request information: USER: AnonymousUser GET: next = '/accounts/login//**/AND/**/(SELECT/**/DIFFERENCE(NULL,NULL))/**/IS/**/NULL--/**/CiLH' POST: No POST data FILES: No FILES data COOKIES: csrftoken = '********************' META: CSRF_COOKIE = 'ArmbEBfvw1lQgbvFKczV9km53k5UFC6G' HTTP_ACCEPT = '*/*' HTTP_ACCEPT_ENCODING = 'gzip' HTTP_CONNECTION = 'close' HTTP_COOKIE = '********************' HTTP_HOST = 'lists.hodgsonfamily.org' HTTP_USER_AGENT = 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.15) Gecko/20110303 Ubuntu/10.04 (lucid) Firefox/3.6.15 FirePHP/0.5' HTTP_X_FORWARDED_FOR = '141.98.255.143' PATH_INFO = '/accounts/login/' QUERY_STRING = 'next=%2F%61%63%63%6F%75%6E%74%73%2F%6C%6F%67%69%6E%2F%2F%2A%2A%2F%41%4E%44%2F%2A%2A%2F%28%53%45%4C%45%43%54%2F%2A%2A%2F%44%49%46%46%45%52%45%4E%43%45%28%4E%55%4C%4C%2C%4E%55%4C%4C%29%29%2F%2A%2A%2F%49%53%2F%2A%2A%2F%4E%55%4C%4C%2D%2D%2F%2A%2A%2F%43%69%4C%48' RAW_URI = '/accounts/login/?next=%2F%61%63%63%6F%75%6E%74%73%2F%6C%6F%67%69%6E%2F%2F%2A%2A%2F%41%4E%44%2F%2A%2A%2F%28%53%45%4C%45%43%54%2F%2A%2A%2F%44%49%46%46%45%52%45%4E%43%45%28%4E%55%4C%4C%2C%4E%55%4C%4C%29%29%2F%2A%2A%2F%49%53%2F%2A%2A%2F%4E%55%4C%4C%2D%2D%2F%2A%2A%2F%43%69%4C%48' REMOTE_ADDR = '127.0.0.1' REMOTE_PORT = '35918' REQUEST_METHOD = 'HEAD' SCRIPT_NAME = '' SERVER_NAME = '127.0.0.1' SERVER_PORT = '8000' SERVER_PROTOCOL = 'HTTP/1.0' SERVER_SOFTWARE = 'gunicorn/22.0.0' gunicorn.socket = <socket.socket fd=4, family=2, type=1, proto=0, laddr=('127.0.0.1', 8000), raddr=('127.0.0.1', 35918)> wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7dddf0dddf60> wsgi.file_wrapper = <class 'gunicorn.http.wsgi.FileWrapper'> wsgi.input = <gunicorn.http.body.Body object at 0x7dddf0dde540> wsgi.input_terminated = True wsgi.multiprocess = False wsgi.multithread = False wsgi.run_once = False wsgi.url_scheme = 'http' wsgi.version = '(1, 0)' Settings: Using settings module settings ABSOLUTE_URL_OVERRIDES = {} ACCOUNT_AUTHENTICATION_METHOD = 'username_email' ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https' ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_EMAIL_VERIFICATION = 'mandatory' ACCOUNT_UNIQUE_EMAIL = True ADMINS = "(('Mailman Suite Admin', 'andrew(a)hodgson.io'),)" ALLOWED_HOSTS = ['localhost', 'lists.hodgsonfamily.org'] APPEND_SLASH = True AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')" AUTH_PASSWORD_VALIDATORS = '********************' AUTH_USER_MODEL = 'auth.User' BASE_DIR = PosixPath('/opt/mailman/web') CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}} CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_KEY_PREFIX = '********************' CACHE_MIDDLEWARE_SECONDS = 600 COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'} COMPRESS_CACHEABLE_PRECOMPILERS = '()' COMPRESS_CACHE_BACKEND = 'default' COMPRESS_CACHE_KEY_FUNCTION = '********************' COMPRESS_CLEAN_CSS_ARGUMENTS = '' COMPRESS_CLEAN_CSS_BINARY = 'cleancss' COMPRESS_CLOSURE_COMPILER_ARGUMENTS = '' COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar' COMPRESS_CSS_HASHING_METHOD = 'mtime' COMPRESS_DATA_URI_MAX_SIZE = 1024 COMPRESS_DEBUG_TOGGLE = None COMPRESS_ENABLED = True COMPRESS_FILTERS = {'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'], 'js': ['compressor.filters.jsmin.rJSMinFilter']} COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7dddf3f61440> COMPRESS_MINT_DELAY = 30 COMPRESS_MTIME_DELAY = 10 COMPRESS_OFFLINE = True COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/static/'} COMPRESS_OFFLINE_MANIFEST = 'manifest.json' COMPRESS_OFFLINE_MANIFEST_STORAGE = 'compressor.storage.OfflineManifestFileStorage' COMPRESS_OFFLINE_MANIFEST_STORAGE_ALIAS = 'compressor-offine' COMPRESS_OFFLINE_TIMEOUT = 31536000 COMPRESS_OUTPUT_DIR = 'CACHE' COMPRESS_PARSER = 'compressor.parser.AutoSelectParser' COMPRESS_PRECOMPILERS = "(('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))" COMPRESS_REBUILD_TIMEOUT = 2592000 COMPRESS_ROOT = '/opt/mailman/web/static' COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage' COMPRESS_STORAGE_ALIAS = 'compressor' COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/static/'} COMPRESS_URL = '/static/' COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/' COMPRESS_VERBOSE = False COMPRESS_YUGLIFY_BINARY = 'yuglify' COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal' COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal' COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar' COMPRESS_YUI_CSS_ARGUMENTS = '' COMPRESS_YUI_JS_ARGUMENTS = '' CSRF_COOKIE_AGE = 31449600 CSRF_COOKIE_DOMAIN = None CSRF_COOKIE_HTTPONLY = False CSRF_COOKIE_MASKED = False CSRF_COOKIE_NAME = 'csrftoken' CSRF_COOKIE_PATH = '/' CSRF_COOKIE_SAMESITE = 'Lax' CSRF_COOKIE_SECURE = False CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure' CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN' CSRF_TRUSTED_ORIGINS = ['https://lists.hodgsonfamily.org'] CSRF_USE_SESSIONS = False DATABASES = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'mailman', 'USER': 'mailman', 'PASSWORD': '********************', 'HOST': 'localhost', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'CONN_HEALTH_CHECKS': False, 'OPTIONS': {}, 'TIME_ZONE': None, 'PORT': '', 'TEST': {'CHARSET': None, 'COLLATION': None, 'MIGRATE': True, 'MIRROR': None, 'NAME': None}}} DATABASE_ROUTERS = [] DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440 DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000 DATA_UPLOAD_MAX_NUMBER_FILES = 100 DATETIME_FORMAT = 'N j, Y, P' DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M'] DATE_FORMAT = 'N j, Y' DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y'] DEBUG = False DEBUG_PROPAGATE_EXCEPTIONS = False DECIMAL_SEPARATOR = '.' DEFAULT_AUTO_FIELD = 'django.db.models.AutoField' DEFAULT_CHARSET = 'utf-8' DEFAULT_EXCEPTION_REPORTER = 'django.views.debug.ExceptionReporter' DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter' DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage' DEFAULT_FROM_EMAIL = 'postorius(a)lists.hodgsonfamily.org' DEFAULT_INDEX_TABLESPACE = '' DEFAULT_TABLESPACE = '' DISALLOWED_USER_AGENTS = [] EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST = 'localhost' EMAIL_HOST_PASSWORD = '********************' EMAIL_HOST_USER = '' EMAIL_PORT = 25 EMAIL_SSL_CERTFILE = None EMAIL_SSL_KEYFILE = '********************' EMAIL_SUBJECT_PREFIX = '[Django] ' EMAIL_TIMEOUT = None EMAIL_USE_LOCALTIME = False EMAIL_USE_SSL = False EMAIL_USE_TLS = False FILE_UPLOAD_DIRECTORY_PERMISSIONS = None FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler'] FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440 FILE_UPLOAD_PERMISSIONS = 420 FILE_UPLOAD_TEMP_DIR = None FILTER_VHOST = False FIRST_DAY_OF_WEEK = 0 FIXTURE_DIRS = [] FORCE_SCRIPT_NAME = None FORMAT_MODULE_PATH = None FORM_RENDERER = 'django.forms.renderers.DjangoTemplates' HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/opt/mailman/web/fulltext_index'}} HYPERKITTY_ENABLE_GRAVATAR = True IGNORABLE_404_URLS = [] INSTALLED_APPS = ['hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount'] INTERNAL_IPS = [] LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ar-dz', 'Algerian Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('ckb', 'Central Kurdish (Sorani)'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('hy', 'Armenian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('ig', 'Igbo'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kab', 'Kabyle'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('ky', 'Kyrgyz'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('ms', 'Malay'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('tg', 'Tajik'), ('th', 'Thai'), ('tk', 'Turkmen'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('uz', 'Uzbek'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')] LANGUAGES_BIDI = ['he', 'ar', 'ar-dz', 'ckb', 'fa', 'ur'] LANGUAGE_CODE = 'en-us' LANGUAGE_COOKIE_AGE = None LANGUAGE_COOKIE_DOMAIN = None LANGUAGE_COOKIE_HTTPONLY = False LANGUAGE_COOKIE_NAME = 'django_language' LANGUAGE_COOKIE_PATH = '/' LANGUAGE_COOKIE_SAMESITE = None LANGUAGE_COOKIE_SECURE = False LOCALE_PATHS = [] LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.TimedRotatingFileHandler', 'filename': '/opt/mailman/web/logs/mailmanweb.log', 'formatter': 'verbose', 'when': 'midnight', 'backupCount': 5}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'ERROR', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'ERROR', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'DEBUG', 'propagate': True}, 'postorius': {'handlers': ['console', 'file'], 'level': 'INFO'}, 'q': {'level': 'WARNING', 'propagate': False, 'handlers': ['console', 'file']}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}} LOGGING_CONFIG = 'logging.config.dictConfig' LOGIN_REDIRECT_URL = 'list_index' LOGIN_URL = 'account_login' LOGOUT_REDIRECT_URL = None LOGOUT_URL = 'account_logout' MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')" MAILMAN_ARCHIVER_KEY = '********************' MAILMAN_REST_API_PASS = '********************' MAILMAN_REST_API_URL = '********************' MAILMAN_REST_API_USER = '********************' MANAGERS = [] MEDIA_ROOT = '' MEDIA_URL = '/' MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage' MESSAGE_TAGS = {40: 'danger'} MIDDLEWARE = "('allauth.account.middleware.AccountMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')" MIGRATION_MODULES = {} MONTH_DAY_FORMAT = 'F j' NUMBER_GROUPING = 0 PASSWORD_HASHERS = '********************' PASSWORD_RESET_TIMEOUT = '********************' POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost:8000' PREPEND_WWW = False Q_CLUSTER = {'retry': 360, 'timeout': 300, 'save_limit': 100, 'orm': 'default', 'workers': 2} ROOT_URLCONF = 'mailman_web.urls' SECRET_KEY = '********************' SECRET_KEY_FALLBACKS = '********************' SECURE_CONTENT_TYPE_NOSNIFF = True SECURE_CROSS_ORIGIN_OPENER_POLICY = 'same-origin' SECURE_HSTS_INCLUDE_SUBDOMAINS = False SECURE_HSTS_PRELOAD = False SECURE_HSTS_SECONDS = 0 SECURE_PROXY_SSL_HEADER = None SECURE_REDIRECT_EXEMPT = [] SECURE_REFERRER_POLICY = 'same-origin' SECURE_SSL_HOST = None SECURE_SSL_REDIRECT = False SERVER_EMAIL = 'postorius(a)lists.hodgsonfamily.org' SESSION_CACHE_ALIAS = 'default' SESSION_COOKIE_AGE = 1209600 SESSION_COOKIE_DOMAIN = None SESSION_COOKIE_HTTPONLY = True SESSION_COOKIE_NAME = 'sessionid' SESSION_COOKIE_PATH = '/' SESSION_COOKIE_SAMESITE = 'Lax' SESSION_COOKIE_SECURE = False SESSION_ENGINE = 'django.contrib.sessions.backends.db' SESSION_EXPIRE_AT_BROWSER_CLOSE = False SESSION_FILE_PATH = None SESSION_SAVE_EVERY_REQUEST = False SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer' SETTINGS_MODULE = 'settings' SHORT_DATETIME_FORMAT = 'm/d/Y P' SHORT_DATE_FORMAT = 'm/d/Y' SIGNING_BACKEND = 'django.core.signing.TimestampSigner' SILENCED_SYSTEM_CHECKS = [] SITE_ID = 1 SOCIALACCOUNT_PROVIDERS = {'openid': {'SERVERS': [{'id': 'yahoo', 'name': 'Yahoo', 'openid_url': 'http://me.yahoo.com'}]}, 'google': {'SCOPE': ['profile', 'email'], 'AUTH_PARAMS': {'access_type': 'online'}}, 'facebook': {'METHOD': 'oauth2', 'SCOPE': ['email'], 'FIELDS': ['email', 'name', 'first_name', 'last_name', 'locale', 'timezone'], 'VERSION': 'v2.4'}} STATICFILES_DIRS = '()' STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')" STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' STATIC_ROOT = '/opt/mailman/web/static' STATIC_URL = '/static/' STORAGES = {'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'}, 'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage'}} TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}] TEST_NON_SERIALIZED_APPS = [] TEST_RUNNER = 'django.test.runner.DiscoverRunner' THOUSAND_SEPARATOR = ',' TIME_FORMAT = 'P' TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] TIME_ZONE = 'UTC' USE_DEPRECATED_PYTZ = False USE_I18N = True USE_L10N = True USE_THOUSAND_SEPARATOR = False USE_TZ = True USE_X_FORWARDED_HOST = False USE_X_FORWARDED_PORT = False WSGI_APPLICATION = 'mailman_web.wsgi.application' X_FRAME_OPTIONS = 'DENY' YEAR_MONTH_FORMAT = 'F Y'

2 1

Setting archive_policy "private" using REST changes web UI setting Archive policy, but does not hide archive until UI Save
by Henrik Rasmussen 14 Aug '24

14 Aug '24

I am working on disabling a list removing all members using REST and then setting archive_policy to "private" also using REST. This changes the UI Archive policy setting to Private archives as I would expect, but I just discovered that the (now previous) member, still logged in, and alså public (not logged in), still have access to the archive. It is not browser cache. At the moment I click Save in the UI settings/archiving, both logged in user and not logged in user (other browser) changed to "This mailing list is private. You must be subscribed to view the archives." on page reload. It seems that i need to click Save on every list having been changed, before the archive is hidden. Is that a bug? I am currently using Postorius Version 1.3.5 and HyperKitty version 1.3.4. /Henrik

2 1

RDS CA changes
by Hirayama, Pat 12 Aug '24

12 Aug '24

Hello, So, my Mailman3 installation (inherited – guy who set it up has left) is using PostgreSQL that is an AWS RDS instance. I have received an alert from AWS that the CA certificate used to create the database is expiring, and they want me to update the certificate before it expires: "Update Your Amazon RDS and Amazon Aurora SSL/TLS Certificates by August 22, 2024 " https://aws.amazon.com/blogs/aws/rotate-your-ssl-tls-certificates-now-amazo… Our DBA team is requesting that we update applications by installing the new CA within the application's certificate store. As near as I can tell from reading the Mailman 3 docs, and looking at my mailman.cfg and settings_local.py ... I don't know that Mailman 3 is using certificate verification for the PostgreSQL connection. At least, I can't see anything in those configuration files that suggest this. Does anyone know if this is the case – or which configuration file I should look at to verify? Will my Mailman 3 be ok if the DBAs go ahead and update the certificates on their end? Thank you! -p Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering - Infrastructure Fred Hutch Cancer Center O 206.667.4856 phirayam(a)fredhutch.org<mailto:phirayam@fredhutch.org>

2 2

Removing all moderatortors (and owners) from a list in one go
by Henrik Rasmussen 12 Aug '24

12 Aug '24

As there doesn't seem to be a "Disable list" REST call (not an "export list" REST call, I am trying to remove all members, moderators and owners, changing list description etc. I managed to remove all members from a list in one go using https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/…, but the same command, using /roster/moderator does not apply for moderators (now owners I guess). To me it seems that each moderator has to be removed one by one, either: * using REST https://docs.mailman3.org/projects/mailmanclient/en/latest/src/mailmanclien… * or using the client https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… * or using Python https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… Isn't there an easier way? And why not like https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/… ? /Henrik

2 1

User cannot reset password
by Greg Newby 11 Aug '24

11 Aug '24

I imported MM2.1 lists to Mailman3 (using import21 - details are in my previous messages to mailman-users), and have had a couple of reports of people for whom the password reset function via "Forgot password" on the list login screen doesn't work as expected. This is a venv installation using PostgreSQL as the database: $ mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.12.3 (main, Jul 31 2024, 17:43:48) [GCC 13.2.0] config file: /etc/mailman3/mailman.cfg db url: postgresql://mailman:redacated@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:redacted The user selects "Forgot password" and enters their email - which I've confirmed is subscribed and receiving messages. They get this email: You are receiving this email because you, or someone else, tried to access an account with email redactaed(a)gmail.com. However, we do not have any record of such an account in our database. This mail can be safely ignored if you did not initiate this action. If it was you, you can sign up for an account using the link below. I looked in the PostgreSQL address table and confirmed the entry is there, and also of course looked at the Members list in the web-based front end. Everything looks ok. I suspect that lots of people will have the same problem, but have not yet tried to login & reset their password. The earlier reports indicated that people's saved passwords did not work, but I haven't replicated that. Any advice on what's going on and how to address it? I've migrated over 200 lists with thousands of users, so would rather not need to deal with every user via the web-based front end. But at this point I don't even know what to try to fix things. With the previous reports I unsubscribed the user and had them resubscribe - that worked as expected but required back-and-forth discussion and, more importantly, user inconvenience. Thanks, Greg Newby

2 3

Python program to create lists and import settings from Mailman 2 to Mailman 3
by Greg Newby 10 Aug '24

10 Aug '24

Hi friends. A couple of weeks ago I asked for help with calling Mailman functions from within a Python program. This was to avoid using the web interface or shell commands to create a new list from a legacy Mailman 2.x list, and instead do it from within a Python program. Here is the program I ended up using, and I thought it would be worthwhile to get into the Mailman users list archives for future folks. Note that this is for the (recommended) venv installation of Mailman. Some things might be different for other installations. I'll paste the program and also attach it: #!/opt/mailman/venv/bin/python3 # Note: The line above should be the Python3 interpreter for your # installed version of Mailman3. It's the same as the first line in # your 'mailman' command, which might be named manage.py or something # else in your environment. # The venv version of Mailman used to develop this program is: # GNU Mailman 3.3.9 (Tom Sawyer) # Python 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0] # About: # The multiple steps of creating a Mailman 3 list based on a legacy # Mailman 2.1 list. # gbn July 28 2024. This little program is granted to the public domain. ######################################################################## # Adjust these variables for your list, domain, MM2 and MM3 locations: lists_to_import = [ # listname, domainname ('testlist','example.org'), ('testlist2','example.org') # Add more lists as needed ] ######################################################################## # Load prerequisite functionality: from mailman.config import config # Needed for transaction.commit from mailman.core.initialize import initialize # For initialize() from mailman.app.lifecycle import create_list # For create_list import pickle # Pickle file handling from contextlib import ExitStack # Pickle file handling from mailman.commands.cli_import import _Mailman, _Bouncer # "" from mailman.utilities.modules import hacked_sys_modules # "" import subprocess # Subprocess for import21 ######################################################################## # Step 0: Initialize the environment: # Without arguments, initialize() looks in the standard locations for your .cfg def do_init(): initialize() # Alternative: Load an explicit configuration file, such as: # initialize('/etc/mailman3/mailman.cfg') ######################################################################## # Step 1: Create the list # Note: This fails silently if the list already exists or something goes wrong def do_create(new_list): try: # The list might exist or otherwise not be creatable l = create_list(new_list) except: # I did not find useful return values from create_list to diagnose print('create_list('+new_list+') failed. Trying to proceed anyway') # This saves your new list: transaction = config.db transaction.commit() ######################################################################## # Step 2: Rewrite the pickle file from the legacy list. def do_pickle(old_pickle_file, new_pickle_file): # These definitions are so the old pickle file can be parsed: with ExitStack() as resources: resources.enter_context(hacked_sys_modules('Mailman', _Mailman)) resources.enter_context( hacked_sys_modules('Mailman.Bouncer', _Bouncer)) # Input the old pickle file from MM 2.1 with open(old_pickle_file, 'rb') as fp: data = pickle.load(fp) # ban_list was a problematic value from my MM2.1 lists: # print(data['ban_list']) data['ban_list'] = [] # Any other settings you want to change? Add them here. These # variables may be seen with MM2's 'config_list' command data['require_explicit_destination']=1 # yes data['member_moderation_action']=1 # reject data['generic_nonmember_action']=2 # reject data['max_num_recipients']=2 # block cc/bcc to list fp.seek(0) # probably not needed fp.close() # ditto # Output: Rewrite the file in the current MM3 format with open(new_pickle_file,'wb') as ofp: pickle.dump(data, ofp) ofp.close() ######################################################################## # Step 3: Import the legacy settings to the newly created list def do_import(new_list, new_pickle_file): # I did not see how to call import21 directly from Python, so am using # a subprocess which basically uses the CLI syntax. # # CLI syntax: mailman import21 [OPTIONS] LISTSPEC PICKLE_FILE def import_list(listname, pickle_file): command = [ 'mailman', 'import21', listname, pickle_file ] subprocess.run(command, check=True) # print(f"List {listname} imported successfully.") # Import the list via a subprocess: import_list(new_list, new_pickle_file) # This saves your changes: transaction = config.db transaction.commit() ######################################################################## # Main process: do_init() # Loop for all lists we want to create & import: for new_listname, new_domainname in lists_to_import: new_list = new_listname+'@'+new_domainname # Adjust for wherever your old MM2 lists are: old_pickle_file = '/var/lib/mailman/lists/'+new_listname+'/config.pck' print('Starting list '+new_list) # The new pickle_file doesn't need to be saved after this program exits new_pickle_file = '/opt/mailman/'+new_listname+'-new.pck' do_create(new_list) do_pickle(old_pickle_file, new_pickle_file) do_import(new_list, new_pickle_file) print('Done with list '+new_list) # Don't forget to import and then index your list for full-text # searching. We're not doing that here because it takes awhile # (depending on the size of the archive). Use a command from the bash # shell such as: # mailman-web hyperkitty_import --since=1971-01-01 -l listname@domainnamed /var/lib/mailman/archives/private/listname.mbox/listname.mbox # followed by: # mailman-web update_index_one_list listname@domainname ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Replaced multipart/alternative part with first alternative.

2 1

Display the last post time for a list or all lists.
by Henrik Rasmussen 09 Aug '24

09 Aug '24

Does Mailman 3 have something similar to https://www.msapiro.net/scripts/last_post.py displaying the last post time for a list or all lists?

3 4

Mailman/Django-allauth integration with Yahoo
by Дилян Палаузов 09 Aug '24

09 Aug '24

Hello, Has anybody managed to use the Django-Allauth Yahoo provider to authenticate towards Mailman3? I have integrated mailman3/django-allauth/gmail — it works. I want in the same way to integrate mailman3 with yahoo mail. As advised at https://docs.allauth.org/en/latest/socialaccount/providers/yahoo.html , in yahoo I register an application with redirect URL https://l.bapha.be/mm3/accounts/yahoo/login/callback/ , then I add credentials under mm3/admin/socialaccount/socialapp/: client ID and Private Key. I append in settings.py:INSTALLED_APPS = [… 'allauth.socialaccount.providers.yahoo' ]. After I do all of this, I eventually visit /mm3/accounts/login/?next=/mm3/postorius/lists/at.l.bapha.be/, click on Yahoo: “Sign In Via Yahoo . You are about to sign in using a third-party account from Yahoo.” → [Continue]. Now at https://login.yahoo.com I enter user name and password. Then I get a site, partially in English, partially localized. It is in my domain at address /mm3/accounts/confirm-email/ , and states: Messages: Confirmation of email address sent to ...(a)yahoo.com Menu: Entrance \r\n New registration Confirm your email address: We have sent an email to you for verification. Follow the link provided to finalize the signup process. If you do not see the verification email in your main inbox, check your spam folder. Please contact us if you do not receive the verification email within a few minutes. I have indeed received an email, which states: New signin from … Hi, Your Yahoo account … was just used to sign in on …. If you did this, you’re all set. If not, please use this link to stop sharing, and create a new password. Have a great day! Thanks,Yahoo So, the /account/confirm-email states I have received an email for verification, but the email sent says no verification is needed, I am all set up. Any idea what can I improve? In addition https://gitlab.com/mailman/mailman-suite/-/blob/master/mailman-suite_projec… suggests adding in settings.py: SOCIALACCOUNT_PROVIDERS = {'openid': {'SERVERS': [dict(id='yahoo', name='Yahoo', openid_url='http://me.yahoo.com')]}} which I have not done. That is, I do not use the “openid” provider, but the “yahoo” provider. Do I have to use the OpenID provider when using Yahoo? Greetings Дилян

2 3

migrate MM3 list from an old server to a new one
by noc＠iem.at 08 Aug '24

08 Aug '24

hi all, i have an old mailman3 instance (3.2.1), and would like to migrate it's lists and archives to a new instance (3.3.8) on a different host. for various reasons, i don't think i am able to upgrade the old instance in-place. unfortunately, i haven't found any documentation on how to proceed. there *is* documentation about migrating MM2 to MM3; and my understanding is that `mailman-web migrate` can be used to upgrade a given MM3 instance to a new version. but how can i export an entire MM3 list (including 20 years of archives) to some serialization format, and import it again in another instance? thanks for your help. gamsr4e IOhannes PS: (again) I'm resending this message, as the one i sent yesterday seems to have vanished (i *did* check the archvies). i have no clue why this happens, as i can post via the webinterface just fine (only when using my trusted MUA they just vanish).

4 10