- Mailman-users - lists.mailman3.org

Django Internal Server Error on held_messages
by Hasan Berkay Çağır 24 Mar '22

24 Mar '22

Hello, We've been running a Mailman3 instance with Debian 10's official package: Mailman Core Version: GNU Mailman 3.2.1 (La Villa Strangiato) Mailman Core API Version: 3.0 Mailman Core Python Version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] For some reason, and only on some lists, when we try to see held messages on Postorius, we get an error message "An error occurred while processing your request." and the following error email: """ Internal Server Error: /postorius/lists/list.example.com/held_messages KeyError at /postorius/lists/list.example.com/held_messages 'file' Request Method: GET Request URL: https://mailman.example.com/postorius/lists/list.example.com/held_messages Django Version: 1.11.29 Python Executable: /usr/bin/uwsgi-core Python Version: 3.7.3 Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Thu, 24 Mar 2022 08:39:10 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view 23. return view_func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/auth/decorators.py" in wrapper 58. return fn(*args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/list.py" in list_moderation 520. paginator_class=MailmanPaginator) File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in paginate 72. objects = paginator.page(page_num) File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in page 48. result = self.function(count=self.per_page, page=number) File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py" in get_held_page 134. return Page(self._connection, url, HeldMessage, count, page) File "/usr/lib/python3/dist-packages/mailmanclient/restbase/page.py" in __init__ 37. self._create_page() File "/usr/lib/python3/dist-packages/mailmanclient/restbase/page.py" in _create_page 62. response, content = self._connection.call(self._build_url()) File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None) During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python3/dist-packages/postorius/middleware.py" in __call__ 39. return self.get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 43. response = response_for_exception(request, exc) File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in response_for_exception 93. response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in handle_uncaught_exception 135. extra={'status_code': 500, 'request': request}, File "/usr/lib/python3.7/logging/__init__.py" in error 1412. self._log(ERROR, msg, args, **kwargs) File "/usr/lib/python3.7/logging/__init__.py" in _log 1519. self.handle(record) File "/usr/lib/python3.7/logging/__init__.py" in handle 1529. self.callHandlers(record) File "/usr/lib/python3.7/logging/__init__.py" in callHandlers 1591. hdlr.handle(record) File "/usr/lib/python3.7/logging/__init__.py" in handle 905. self.emit(record) File "/usr/lib/python3/dist-packages/django/utils/log.py" in emit 118. reporter = ExceptionReporter(request, is_email=True, *exc_info) File "/usr/lib/python3/dist-packages/django/views/debug.py" in __init__ 246. self.template_info = getattr(self.exc_value, 'template_debug', None) File "/usr/lib/python3.7/tempfile.py" in __getattr__ 614. file = self.__dict__['file'] Exception Type: KeyError at /postorius/lists/list.example.com/held_messages Exception Value: 'file' Request information: USER: root GET: No GET data POST: No POST data FILES: No FILES data COOKIES: rl_anonymous_id = '*' rl_user_id = '*' rl_group_id = '*' rl_trait = '*' rl_group_trait = '*' rl_page_init_referrer = '*' rl_page_init_referring_domain = '*' csrftoken = '*' sessionid = '*' META: CONTENT_LENGTH = '' CONTENT_TYPE = '' CSRF_COOKIE = '*' DOCUMENT_ROOT = '/etc/nginx/html' HTTPS = 'on' HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING = 'gzip, deflate, br' HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.5' HTTP_CACHE_CONTROL = 'max-age=0' HTTP_COOKIE = '*' HTTP_DNT = '1' HTTP_HOST = 'mailman.example.com' HTTP_REFERER = 'https://mailman.example.com/postorius/lists/list.example.com/' HTTP_SEC_FETCH_DEST = 'document' HTTP_SEC_FETCH_MODE = 'navigate' HTTP_SEC_FETCH_SITE = 'same-origin' HTTP_TE = 'trailers' HTTP_UPGRADE_INSECURE_REQUESTS = '1' HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:98.0) Gecko/20100101 Firefox/98.0' PATH_INFO = '/postorius/lists/list.example.com/held_messages' QUERY_STRING = '' REMOTE_ADDR = '::1' REMOTE_PORT = '58290' REQUEST_METHOD = 'GET' REQUEST_SCHEME = 'https' REQUEST_URI = '/postorius/lists/list.example.com/held_messages' SCRIPT_NAME = '' SERVER_NAME = 'mailman.example.com' SERVER_PORT = '443' SERVER_PROTOCOL = 'HTTP/2.0' uwsgi.core = 0 uwsgi.node = b'example.com' uwsgi.version = b'2.0.18-debian' wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'> wsgi.file_wrapper = '' wsgi.input = <uwsgi._Input object at 0x7f8f7a6bf870> wsgi.multiprocess = False wsgi.multithread = True wsgi.run_once = False wsgi.url_scheme = 'https' wsgi.version = '(1, 0)' Settings: Using settings module settings ABSOLUTE_URL_OVERRIDES = {} ACCOUNT_AUTHENTICATION_METHOD = 'username_email' ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https' ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_EMAIL_VERIFICATION = 'mandatory' ACCOUNT_UNIQUE_EMAIL = True ADMINS = "(('Mailman Suite Admin', 'admin(a)example.com'),)" ALLOWED_HOSTS = ['*'] APPEND_SLASH = True AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')" AUTH_PASSWORD_VALIDATORS = '********************' AUTH_USER_MODEL = 'auth.User' BASE_DIR = '/usr/share/mailman3-web' CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}} CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_KEY_PREFIX = '********************' CACHE_MIDDLEWARE_SECONDS = 600 COMPRESS_CACHEABLE_PRECOMPILERS = '()' COMPRESS_CACHE_BACKEND = 'default' COMPRESS_CACHE_KEY_FUNCTION = '********************' COMPRESS_CLEAN_CSS_ARGUMENTS = '' COMPRESS_CLEAN_CSS_BINARY = 'cleancss' COMPRESS_CLOSURE_COMPILER_ARGUMENTS = '' COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar' COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor' COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter'] COMPRESS_CSS_HASHING_METHOD = 'mtime' COMPRESS_DATA_URI_MAX_SIZE = 1024 COMPRESS_DEBUG_TOGGLE = None COMPRESS_ENABLED = True COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f8f7b50d598> COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor' COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter'] COMPRESS_MINT_DELAY = 30 COMPRESS_MTIME_DELAY = 10 COMPRESS_OFFLINE = True COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'} COMPRESS_OFFLINE_MANIFEST = 'manifest.json' COMPRESS_OFFLINE_TIMEOUT = 31536000 COMPRESS_OUTPUT_DIR = 'CACHE' COMPRESS_PARSER = 'compressor.parser.AutoSelectParser' COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))" COMPRESS_REBUILD_TIMEOUT = 2592000 COMPRESS_ROOT = '/var/lib/mailman3/web/static' COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage' COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'} COMPRESS_URL = '/mailman3/static/' COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/' COMPRESS_VERBOSE = False COMPRESS_YUGLIFY_BINARY = 'yuglify' COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal' COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal' COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar' COMPRESS_YUI_CSS_ARGUMENTS = '' COMPRESS_YUI_JS_ARGUMENTS = '' CSRF_COOKIE_AGE = 31449600 CSRF_COOKIE_DOMAIN = None CSRF_COOKIE_HTTPONLY = False CSRF_COOKIE_NAME = 'csrftoken' CSRF_COOKIE_PATH = '/' CSRF_COOKIE_SECURE = False CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure' CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN' CSRF_TRUSTED_ORIGINS = [] CSRF_USE_SESSIONS = False DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4'}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}} DATABASE_ROUTERS = [] DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440 DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000 DATETIME_FORMAT = 'N j, Y, P' DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y'] DATE_FORMAT = 'N j, Y' DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y'] DEBUG = False DEBUG_PROPAGATE_EXCEPTIONS = False DECIMAL_SEPARATOR = '.' DEFAULT_CHARSET = 'utf-8' DEFAULT_CONTENT_TYPE = 'text/html' DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter' DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage' DEFAULT_FROM_EMAIL = 'postorius(a)example.com' DEFAULT_INDEX_TABLESPACE = '' DEFAULT_TABLESPACE = '' DISALLOWED_USER_AGENTS = [] EMAILNAME = 'example.com' EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST = 'localhost' EMAIL_HOST_PASSWORD = '********************' EMAIL_HOST_USER = '' EMAIL_PORT = 25 EMAIL_SSL_CERTFILE = None EMAIL_SSL_KEYFILE = '********************' EMAIL_SUBJECT_PREFIX = '[Django] ' EMAIL_TIMEOUT = None EMAIL_USE_LOCALTIME = False EMAIL_USE_SSL = False EMAIL_USE_TLS = False FILE_CHARSET = 'utf-8' FILE_UPLOAD_DIRECTORY_PERMISSIONS = None FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler'] FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440 FILE_UPLOAD_PERMISSIONS = None FILE_UPLOAD_TEMP_DIR = None FILTER_VHOST = False FIRST_DAY_OF_WEEK = 0 FIXTURE_DIRS = [] FORCE_SCRIPT_NAME = None FORMAT_MODULE_PATH = None FORM_RENDERER = 'django.forms.renderers.DjangoTemplates' HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}} HOSTNAME = 'localhost.local' IGNORABLE_404_URLS = [] INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')" INTERNAL_IPS = [] LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')] LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur'] LANGUAGE_CODE = 'en-us' LANGUAGE_COOKIE_AGE = None LANGUAGE_COOKIE_DOMAIN = None LANGUAGE_COOKIE_NAME = 'django_language' LANGUAGE_COOKIE_PATH = '/' LOCALE_PATHS = [] LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}} LOGGING_CONFIG = 'logging.config.dictConfig' LOGIN_REDIRECT_URL = 'list_index' LOGIN_URL = 'account_login' LOGOUT_REDIRECT_URL = None LOGOUT_URL = 'account_logout' MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')" MAILMAN_ARCHIVER_KEY = '********************' MAILMAN_REST_API_PASS = '********************' MAILMAN_REST_API_URL = '********************' MAILMAN_REST_API_USER = '********************' MANAGERS = [] MEDIA_ROOT = '' MEDIA_URL = '' MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage' MESSAGE_TAGS = {40: 'danger'} MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')" MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware'] MIGRATION_MODULES = {} MONTH_DAY_FORMAT = 'F j' NUMBER_GROUPING = 0 PASSWORD_HASHERS = '********************' PASSWORD_RESET_TIMEOUT_DAYS = '********************' POSTORIUS_TEMPLATE_BASE_URL = 'https://mailman.example.com/' PREPEND_WWW = False Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5} ROOT_URLCONF = 'urls' SECRET_KEY = '********************' SECURE_BROWSER_XSS_FILTER = False SECURE_CONTENT_TYPE_NOSNIFF = False SECURE_HSTS_INCLUDE_SUBDOMAINS = False SECURE_HSTS_PRELOAD = False SECURE_HSTS_SECONDS = 0 SECURE_PROXY_SSL_HEADER = None SECURE_REDIRECT_EXEMPT = [] SECURE_SSL_HOST = None SECURE_SSL_REDIRECT = False SERVER_EMAIL = 'root(a)example.com' SESSION_CACHE_ALIAS = 'default' SESSION_COOKIE_AGE = 1209600 SESSION_COOKIE_DOMAIN = None SESSION_COOKIE_HTTPONLY = True SESSION_COOKIE_NAME = 'sessionid' SESSION_COOKIE_PATH = '/' SESSION_COOKIE_SECURE = False SESSION_ENGINE = 'django.contrib.sessions.backends.db' SESSION_EXPIRE_AT_BROWSER_CLOSE = False SESSION_FILE_PATH = None SESSION_SAVE_EVERY_REQUEST = False SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer' SETTINGS_MODULE = 'settings' SHORT_DATETIME_FORMAT = 'm/d/Y P' SHORT_DATE_FORMAT = 'm/d/Y' SIGNING_BACKEND = 'django.core.signing.TimestampSigner' SILENCED_SYSTEM_CHECKS = [] SITE_ID = 1 SOCIALACCOUNT_PROVIDERS = {} STATICFILES_DIRS = '()' STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')" STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' STATIC_ROOT = '/var/lib/mailman3/web/static' STATIC_URL = '/mailman3/static/' TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}] TEST_NON_SERIALIZED_APPS = [] TEST_RUNNER = 'django.test.runner.DiscoverRunner' THOUSAND_SEPARATOR = ',' TIME_FORMAT = 'P' TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] TIME_ZONE = 'UTC' USE_ETAGS = False USE_I18N = True USE_L10N = True USE_THOUSAND_SEPARATOR = False USE_TZ = True USE_X_FORWARDED_HOST = True USE_X_FORWARDED_PORT = False WSGI_APPLICATION = 'wsgi.application' X_FRAME_OPTIONS = 'SAMEORIGIN' YEAR_MONTH_FORMAT = 'F Y' """ I've tried to debug this issue as much as I can but couldn't figure out anything. I wonder if anyone has encountered something like this or might have a clue about what's the problem here? Thanks in advance, Berkay

3 2

postorius not displaying mailman logo
by sekamug＠yahoo.com 23 Mar '22

23 Mar '22

I successfully installed mailman3 but Postorius doesn't display images on the web interface. I have checked and the files exist at at the location /opt/mailman/web/static/postorius i have checked the logs and they appear as follows: [pid: 366783|app: 0|req: 43/104] () {34 vars in 687 bytes} [Tue Mar 22 09:36:30 2022] GET /static/postorius/js/script.js => generated 5280 bytes in 43 msecs (HTTP/1.1 404) 6 headers in 197 bytes (1 switches on core 0) [pid: 366783|app: 0|req: 44/105] () {34 vars in 715 bytes} [Tue Mar 22 09:36:30 2022] GET /static/postorius/img/favicon.ico => generated 5280 bytes in 43 msecs (HTTP/1.1 404) 6 headers in 197 bytes (1 switches on core 1) it seems the browser fails to access the directory where the pictures are stored. I have checked and mailman is the owner of that directory. Please advise.

4 7

anyway to change/set options for a bunch of users?
by bob B 23 Mar '22

23 Mar '22

We just migrated to mailman 3 and one of my admins asked is there a way to change their digest options, when adding a bunch of users via mas subscription? I was looking for an option to say all new users are set to digest, but I can't find it. Also, is there a way to select multiple users in list and change their digest options? I believe could do this in mailman2 but I am trying to figure out if this is possible for the GUI in mailman3?

2 4

Special Characters Issue when using Wordpress HTML messages
by ryan.anderson.4＠go.mnstate.edu 23 Mar '22

23 Mar '22

Hello, We are having trouble where we are using a WordPress server send messages our to a few of our lists as HTML embedded messages. We do not want the messages to be shown as Plaintext, so we have that setting turned off in WordPress and in all our lists' settings. The messages come through just fine, but several characters are being changed to "?". Some of these include ` and - . We recently upgraded from 2.1 to 3.3 and we love the new system, other than the fact that our news messages are looking bad because of all the "?" in them. The charset was set to US-ASCII, but I have changed it to UTF-8 and that has not seemed to do anything. Any ideas on how we can prevent these characters from being changed?

2 4

bounce score incremented resent on server startup
by Axel Rau 22 Mar '22

22 Mar '22

I just did an update of the database to PostgreSQL 13 and noticed: a bunch of 'bounce score incremented‘ notify mails. In mailman.log I find sqlalchemy crashing because DB server was not ready during startup. Axel --- PGP-Key: CDE74120 ☀ computing @ chaos claudius

2 1

ARC
by David Krantz 20 Mar '22

20 Mar '22

Hi all, Is the documentation correct that ARC should be as simple to configure in mailman 3 as to add the private DKIM key and restarting mailman? (https://mailman.readthedocs.io/en/latest/src/mailman/handlers/docs/arc_sign…) I've tried that without finding any effect in my so far limited tests. I have not found anything in logs although I might have missed looking in one. Are there any known limitations? I have previously a working DKIM signature on outgoing mail as well as correctly set up SPF in DNS and SPF-checking on received mail. The verification on incoming mail adds a few headers but I do not think that that should be an issue, neither would the DKIM signing change anything should mailman add new headers. I now run Mailman 3.2.3 (La Villa Strangiato) on Python 3.7.4. Cheers // David

6 11

Dealing with email bounces from web signups
by Andrew Hodgson 18 Mar '22

18 Mar '22

Hi. Back on the topic of bogus web signups again. I am dealing with several hundreds of these per month and we're talking a very low traffic system here. I have a cronjob to remove the users from the database. Right now a concern for me is handling the bounces for the confirmation emails that don't get delivered. These are coming from postorius(a)lists.hodgsonfamily.org<mailto:postorius@lists.hodgsonfamily.org> which I have redirected to a mailbox so as not to clutter up the mail queue. What I would ideally like is if we get a bounce from one of these confirmation mails we delete the user from the database but that is probably a future request. We don't get too many of these, around 20-30 a week. What are other people doing with these messages? Thanks. Andrew.

4 4

Replies
by Matthew R 18 Mar '22

18 Mar '22

Hi everyone, we're going to be using a Mailman3 list for the purpose of providing email support. So: * Anyone can post to the list * Agents will reply to the poster to receive support, and the reply must go back to the original poster (the client) and the mailing list for other agents to see * The poster's replies will be sent to the mailing list (so other agents can see) as well as to the agent who initially replied to the initial posting Is there any way to accomplish this? Right now, if the agent replies to the client's email correctly but the client doesn't hit "Reply All" the email is only sent to the agent that initially took the "case" and the other agents aren't aware that there was a response. Thank you, Matt Matthew R, AD, FSEN, FSO, FSCR Chief Director of Engineering & Chairman of the Board of Directors Library of Code sp-us matthew(a)staff.libraryofcode.org ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: application/pkcs7-signature Name: smime.p7s Replaced multipart/alternative part with first alternative.

2 1

How to promote non-members to members?
by Quanah Gibson-Mount 17 Mar '22

17 Mar '22

Hello, We have a constant issue with confirmation emails for subscribers getting lost and then the subscriber contacting us because their posts to the list never show up. Is there a way I can, as an administrator, do the confirmation on their behalf or in some other way promote them to being a member from non-member? Regards, Quanah

2 7

pending subscriptions lost?
by IEM Network Operation Center (IOhannes m zmölnig) 15 Mar '22

15 Mar '22

hi, after mark has helped me at the end of january to find users that were not attached to any mailing list (indicating people who pressed the wrong button when trying tosign up), we sent out a 2nd mass mail to our target audience to ask them to subscribe again. we are now getting feedback from people, who say that they are unable to sign up, with mailman3 telling them: > Subscription request already pending this is somewhat bad as: - at least some of the users (who reported the problem) tried to subscribe 2 months ago (2022-01-04) - if they did not confirm their email back then, i'm pretty sure that this email has been lost by now - my /etc/mailman3/mailman.cfg has "pending_request_life: 3d" - i (as administrator) do not see any "pending subscriptions" (the list is setup with a simple "Confirm" subscription policy - i (as administrator) do not find any mention of the user's email address anywhere either in postorious webinterface of the list, nor in the django - in the sql-dump (it's been a while since i worked with postgres on a higher level) i do see the user's email in the workflowstate, address and pendedkeyvalue tables. the installation is (still) mailman_3.2.1-1 on a Debian/buster system (using the Debian/buster packages for mailman3) i wonder: - why are the pending subscriptions not expired? (i guess they should have expired after 3 days) - why can't people re-subscribe while there's a subscription pending (e.g. if the confirmation mail got lost)? - how can i list all pending subscriptions (for a given mailinglist)? - how can i resolve the situtation (e.g. send out another reminder; manually expire the pendings; manually confirm the pendings,...) gfmdsd IOhannes

3 10

hyperkitty url not found
by Eggert Ehmke 15 Mar '22

15 Mar '22

I installed mailman3 in a virtualenv following this guide: https://docs.mailman3.org/en/latest/install/virtualenv.html[1] The mailinglists work very good. However, I did not manage to set up hyperkitty archiving. Archiving is enabled in mailman.cfg. I can call the Archive site from my configuration page, but it is empty. In my /opt/mailman/mm/var/logs/mailman.log file I find error messages like: HyperKitty failure on http://localhost/mailman/hyperkitty/api/mailman/archive: The requested URL was not found on this server. Indeed I cannot access that url using curl. Do I have to set up a separate apache vhost? The documentation is not clear about this, or I missed it. My external visible url is https://lists.[2]<tld>_/mailman3/mailman3/lists/_ -------- [1] https://docs.mailman3.org/en/latest/install/virtualenv.html [2] https://lists.ftv-spandau.de/mailman3/mailman3/lists/

2 5

Delete user
by Rainer Herbst 14 Mar '22

14 Mar '22

Hi *! Recently, I have updated my docker images to 0.4.2 and it worked great. In the Postorius GUI, I see a new topic "Users" and found it quite comfortable. But one problem remais open - how can I delete a user alltogether, i.e. including his subscription and account data from the database? The reason for this question are the GDPR regulations - we do subscribe new staff members to some lists automatically, and should unsubscribe them and delete the entries after they left our organization. Viele Grüße Rainer Herbst Leiter IT-Service Phone: +49 331 7499-257 e-mail: rainer.herbst(a)aip.de https://www.aip.de ----------------------------------------------------------------------------------------------- Leibniz-Institut für Astrophysik Potsdam (AIP) An der Sternwarte 16, 14482 Potsdam Vorstand: Prof. Dr. Matthias Steinmetz, Wolfram Rosenbach Stiftung bürgerlichen Rechts Stiftungsverzeichnis Brandenburg: 26 742-00/7026 ----------------------------------------------------------------------------------------------- ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: application/x-pkcs7-signature Name: smime.p7s

2 1

DKIM signatures with message footers or subject prefixes
by Stanisław Findeisen 14 Mar '22

14 Mar '22

Dear Experts What is the best way to go about reconciling GNU Mailman3-made message modifications (like message footers or subject prefixes) with sender-made DKIM signatures? I am aware of the "DMARC mitigations" tab in the list settings, not bad, but it looks that "replace From: with list address" doesn't help (of course), whereas wrapping the message in an outer message often looks ugly. Is there a better way?... BTW I would like to point out that broken DKIM signature can be a problem even with an easy DMARC policy; for example ProtonMail will display this flashy red warning: https://protonmail.com/support/knowledge-base/email-has-failed-its-domains-… in its webmail. (I guess nothing can be done about it except "DMARC Mitigate unconditionally"). This is GNU Mailman 3.1. Thanks!

2 5

list footer questions
by Mohsen Masoudfar 14 Mar '22

14 Mar '22

Hi, This seems to me like basic simple questions, I have tried FAQ and other resources, with not much success. I have two questions: 1. When I send an email to a list, in gmail and yahoo I see the following text with the links at the end of the email body: _______________________________________________ Testlist_dmark1 mailing list - listname(a)domain.tlb<mailto:testlist_dmark1@newlistserv.aaas.org> To unsubscribe send an email to listname-leave(a)domain.tlb In<mailto:listname-leave@domain.tlb%0b%0bIn> outlook this is added as an attachment in the form of a TEXT file, when you open it, you see the same text. Question: Is there a way to change this, that it looks in outlook the same way as it appears in gmail/yahoo? I know MS does it own stuff sometime and we have no impact on that, but I was thinking maybe we might change the footer in a way that outlooks sees that as part of email body. 2. How may I change the format and the text in footer? I was expecting it under list > setting, in postorius, but I could not find it. Best regards Mohsen Masoudfar Lead System Analyst, IT Operation American Association for the Advancement of Science 1200 New York Ave, NW, Washington, DC 20005 202-326-7087

5 11

Mailman3 reinstall fail, need some help
by Yvan ♞ 12 Mar '22

12 Mar '22

hello, trying to install mailman 3 on yunohost with 2 domains, have some trouble... server error, no more access Tried to remove, reinstall, and then error https://paste.yunohost.org/raw/xiqoyajegi Yvan

2 3

Why is this Postorius API public?
by Stanisław Findeisen 10 Mar '22

10 Mar '22

Hello I've just noticed that Postorius API is publicly accessible from the outside to an unauthenticated user, for example: https://HOST/postorius/api/templates/list/LISTNAME.VHOST/list:member:regula… Is this what is expected? Or a misconfiguration? How to fix this? This is GNU Mailman 3.1. Thanks!

2 1

New MM3 plugin: mailman_sieve
by Дилян Палаузов 10 Mar '22

10 Mar '22

Hello, I have published a new Mailman 3 plugin at https://mail.aegee.org/cgit/mailman_sieve/tree/ . It generates for each mailing list a Sieve script. When the Sieve script is evaluated during the SMTP dialog, it would reject some emails, which otherwise would be bounced by Mailman. Less bounces means less backscatters and better IP reputation. For the record, I do the evaluation of the Sieve scripts during the SMTP dialog with https://mail.aegee.org/cgit/aegee-milter . Since the latter is not documented yet, you cannot use it. The mailman_sieve plugin is not useful, unless you have software to evaluate the Sieve Scripts during the SMTP dialog. This is a very first version, which does what it is supposed to do. Some optimizations are possible, like not including the subscriber addresses in the Sieve scripts for announce-only mailing lists. I am not good at writing setuptools configuration. Neither do I understand the mailman system very good right now, creating possibly shorter scripts. But the plugin is there to keep by IP reputation good! See https://mail.aegee.org/cgit/mailman_sieve/tree/README.md . My reading of the mm3 source code is that there is no event triggered, when the mailing list configuration changes, like adding a new moderator, or switching the announce-only status. In the lack of such event triggers, the Sieve scripts are not updated. When the code improves, the plugin could be published in PyPi or in mailman/contrib . Greetings Дилян

1 0

GNU Mailman in Google Summer of Code 2022!
by Stephen J. Turnbull 10 Mar '22

10 Mar '22

Hi all, We are participating in Google Summer of Code 2022 (GSoC) under the Python Software Foundation (PSF). We did apply as a separate organization, but didn't get in. Many thanks to the PSF's GSoC organization for picking us up at the last minute! You can find our ideas page at https://wiki.list.org/DEV/Google%20Summer%20of%20Code%202022, and item on the PSF ideas page under https://python-gsoc.org/ideas.html (which isn't so interesting about Mailman, but you can check out who else is under the PSF banner there). Discussion and development of project ideas will take place on mailman-developers(a)python.org. If you have a developed idea not already listed that you're willing to discuss and advocate, and you're willing to grab the template (last item on our ideas page on list.org) and fill it out, please post it to mailman-developers(a)python.org. (This a deliberate hoop-jumping exercise to filter in "serious" ideas.) You can put more weight on it by expressing interest in mentoring the project, and even more by actually volunteering! ;-) If you have a half-baked idea, and/or you want to get input from other Mailman users, of course you should feel free to discuss it on mailman-users(a)mailman3.org. We *might* pick it up from there, even if it's just a drive-by post with little support from the list members. But no promises about that, most of our attention will go to mailman-developers. If you feel some idea should get more love, polish it a little and post that to mailman-developers (don't cross-post, please). Again, we'd love to have more mentors, including backup mentors. We currently only have two mentors. Although we're both core developers, and I'm on an academic schedule and can devote plenty of time to the student during the summer, not so for Abhilash. Volunteering at this stage is not a promise to do more than look at student proposals. You would only be asked to look at the proposals, and volunteer as mentor for that project if you have interest and some level of knowledge to contribute to it. Even if you're a site admin with little Python programming experience, vetting interns' brainstorms about list admin-visible features and holding their hands until the "pros from Dover" arrive is quite helpful in making the student feel at home in our community. Many students are shy about blogging and communicating about their projects to the community -- just talking to them about their project and suggesting that they blog what they told you is helpful. Coding skills and knowledge of some part of the Mailman 3 code base are most welcome, of course, but you can make your mentor role what you want it to be. Regards, Steve

1 0

address for email for owner
by bob B 09 Mar '22

09 Mar '22

I just migrated my lists to mailman3 in docker today... YEAHHH!!!!!!!! but now my admins have to setup accounts and I noticed when they get the email to confirm it comes from admnin(a)[XYZ.COM]. Where is this email set? how can I change it? it is set to an internal domain (not public) of ours, so it is getting flagged as spam in external email accounts I looked in my docker. yaml and the MAILMAN_ADMIN_EMAIL=mailman3(a)[123.com}. which is where I would expect the request/confirmation email to come from.

1 1

Problem with debian uipgrade
by Lars Schimmer 08 Mar '22

08 Mar '22

Hi! Today I tried a mailman3 upgrade of the debian packages and I got this error: django.db.migrations.exceptions.NodeNotFoundError: Migration hyperkitty.0007_allauth_20160808_1604 dependencies reference nonexistent parent node ('socialaccount', '0003_extra_data_default_dict') Anyone has a idea howto fix those? thank you. MfG, Lars Schimmer -- ------------------------------------------------------------- TU Graz, Institut für ComputerGraphik & WissensVisualisierung Tel: +43 316 873-5405 E-Mail: l.schimmer(a)cgv.tugraz.at Fax: +43 316 873-5402 PGP-Key-ID: 0x4A9B1723

2 4

User suspended after too much spam
by Eggert Ehmke 06 Mar '22

06 Mar '22

I have set up spamassassin as spamfilter, and it works reasonable so far. Some spam passes the filter and I try to use it for training. However, one of my users has a very restrictive spam filter and rejects all spam. This causes mailman to suspend that user. I have to enable the user each time. How can I change that behaviour? I do not want my users to be suspended. Cheers

2 1

Mailman3 not working following distribution upgrade from Debian 10 to 11.
by gordon＠dickens.com 06 Mar '22

06 Mar '22

I performed a distribution upgrade from Debian 10 Buster (which runs mailman 3.2.1) to Debian 11 Bullseye (which runs mailman 3.3.1) with the command "apt-get full-upgrade -y" and everything apparently worked properly except at the end of the upgrade when the following error dialog was generated by the mailman3-web and mailman3-full post-installation scripts: Setting up mailman3-web (0+20200530-2) ... Determining localhost credentials from /etc/mysql/debian.cnf: succeeded. dbconfig-common: writing config to /etc/dbconfig-common/mailman3-web.conf dbconfig-common: flushing administrative password sed: -e expression #2, char 77: unterminated `s' command dpkg: error processing package mailman3-web (--configure): installed mailman3-web package post-installation script subprocess returned error exit status 1 dpkg: dependency problems prevent configuration of mailman3-full: mailman3-full depends on mailman3-web; however: Package mailman3-web is not configured yet. dpkg: error processing package mailman3-full (--configure): dependency problems - leaving unconfigured Errors were encountered while processing: mailman3-web mailman3-full E: Sub-process /usr/bin/dpkg returned an error code (1) Now, every time that I execute "apt-get upgrade" then I continue to get the above error dialog. Also, the command "dpkg-reconfigure mailman3-web" generates this output: /usr/sbin/dpkg-reconfigure: mailman3-web is broken or not fully installed So, I am now stuck since apt-get is saying that mailman3-web is not configured yet, however, neither apt-get nor dpkg-reconfigure will successfully configure mailman3-web. Does anybody have any idea how to fix this? I would very much appreciate some suggestions. Thanks, Gordon

6 29

Remove login from list subscription page
by bob B 01 Mar '22

01 Mar '22

Is there any way to easily remove the login button for the List subscription page? 99% of our list do not use archiving, and log-in just confuses users so We would like for them just to be able to sign up to be on the list.

2 1

Mass un/subscribe optimization?
by Bryan Loniewski 28 Feb '22

28 Feb '22

Hi -- We've noticed slowness in mass un/subscribing users; looking through the issue(s) log we see both #419 [1] and #627 [2] open and related. From the cmd line we get better performance, but with the subscriber size on many of our Faculty/Staff/Students ad-hoc lists (think 1k-15K subscribed addresses) and with our number of lists (2600) and unique owners utilizing these actions it would be helpful to optimize the mass actions. We wanted to get some thoughts and guidance on what needs to be optimized/redesigned/fixed? Or is there already something in the works, but needs more time? We may be able to provide resources to encourage and/or assist with any development and/or testing... We run a split core/web setup with Apache,mod_wsgi and MySQL backend (AWS hosted). [1] - https://gitlab.com/mailman/mailman/-/issues/419 [2] - https://gitlab.com/mailman/mailman/-/issues/627

2 1

Migration of archives from Sympa to MM3
by Tom Fillmore 28 Feb '22

28 Feb '22

Hi, all - At the moment I have a site that uses Sympa. It has more than 150 lists. Moving existing lists to MM3 looks doable the archives are a different story. Has anyone ever migrated archives from Sympa to MM3? Sorry if this is answered somewhere else... Thanks T Fillmore southern California The earth is but one country and mankind its citizens... - Baha'u'llah

2 2

Migrating archives from mailman2 to mailman3 with attachments
by bob B 28 Feb '22

28 Feb '22

So I see the process for migrating an archive from mailman2 to mailman3 https://docs.mailman3.org/en/latest/migration.html.................. i.e python manage.py hyperkitty_import -l foo-list(a)example.com $var_prefix/archives/private/foo-list.mbox/foo-list.mbox but does the migration handle attachments?? attachments are in ..../archives/private/foo-list/attachments. I assume this is if you are upgrading in place? I am migrating to a new server and my plan is to follow these steps but copy the config.pck. and .mbox files to the new server and import them, but then it looks like I need to copy the attachment directory? if so how do I import the attachments to mailman3?

2 1

sending messages in response to posts from non-members
by David Bremner 27 Feb '22

27 Feb '22

I'd like to turn off the "bounce messages" sent by mailman to non-members. My reasoning is that most such posts are spammers, and often use forged return-paths / From's, so I end of sending steady stream of backscatter to amazon.com etc... Is this configurable (without postorius) somehow? d

2 2

"autorespond to list owner" setting when importing from mailman2 to mailman3
by bob B 25 Feb '22

25 Feb '22

In mailman2 a list was moderated, the admin would also get an email to approve the message, our lists mainly have admins and no moderators. The admins did the moderation. In mailman3 it looks this is now a setting "autorespond to list owner" was this a setting in mailman 2? Anyway, in importing email list from mailman 2 to 3, it appears that this setting is set to "No automatic response", is that right? So does this means if I import all my lists from mailman 2, then the admins will not get moderation notifications until I make that setting change on each list?

2 4

Python 3.9 support
by dancab＠caltech.edu 25 Feb '22

25 Feb '22

I just upgraded our Mailman3 system to use Python 3.9. About 6 hours after the deployment there was a PyMySQL error and delivery of messages stopped until I restarted the services. I'm pasting the full stack trace error from mailman.log below. It mentions PyMySQL. We have PyMySQL-1.0.2 installed. Thanks for your help. ---- Dan Caballero Systems Administrator Academic Computing Solutions IMSS - Caltech https://imss.caltech.edu Feb 23 23:27:10 2022 (300) Uncaught runner exception: (pymysql.err.OperationalError) (1053, 'Server shutdown in progress') [SQL: SELECT anon_1.member_id AS anon_1_member_id, anon_1.member__member_id AS anon_1_member__member_id, anon_1.member_role AS anon_1_member_role, anon_1.member_list_id AS anon_1_member_list_id, anon_1.member_moderation_action AS anon_1_member_moderation_action, anon_1.member_address_id AS anon_1_member_address_id, anon_1.member_preferences_id AS anon_1_member_preferences_id, anon_1.member_user_id AS anon_1_member_user_id, anon_1.member_bounce_score AS anon_1_member_bounce_score, anon_1.member_last_bounce_received AS anon_1_member_last_bounce_received, anon_1.member_last_warning_sent AS anon_1_member_last_warning_sent, anon_1.member_total_warnings_sent AS anon_1_member_total_warnings_sent FROM (SELECT `member`.id AS member_id, `member`._member_id AS member__member_id, `member`.`role` AS member_role, `member`.list_id AS member_list_id, `member`.moderation_action AS member_moderation_action, `member`.address_id AS member_address_id, `member`.preferences_id AS member_preferences_id, `member`.user_id AS member_user_id, `member`.bounce_score AS member_bounce_score, `member`.last_bounce_received AS member_last_bounce_received, `member`.last_warning_sent AS member_last_warning_sent, `member`.total_warnings_sent AS member_total_warnings_sent FROM `member`, address WHERE `member`.list_id = %(list_id_1)s AND `member`.`role` = %(role_1)s AND address.email = %(email_1)s AND `member`.address_id = address.id UNION SELECT `member`.id AS member_id, `member`._member_id AS member__member_id, `member`.`role` AS member_role, `member`.list_id AS member_list_id, `member`.moderation_action AS member_moderation_action, `member`.address_id AS member_address_id, `member`.preferences_id AS member_preferences_id, `member`.user_id AS member_user_id, `member`.bounce_score AS member_bounce_score, `member`.last_bounce_received AS member_last_bounce_received, `member`.last_warning_sent AS member_last_warning_sent, `member`.total_warnings_sent AS member_total_warnings_sent FROM `member`, address, user WHERE `member`.list_id = %(list_id_2)s AND `member`.`role` = %(role_2)s AND address.email = %(email_2)s AND `member`.user_id = user.id AND user._preferred_address_id = address.id) AS anon_1] [parameters: {'list_id_1': 'avery-disc.caltech.edu', 'role_1': 1, 'email_1': 'nandrade(a)caltech.edu', 'list_id_2': 'avery-disc.caltech.edu', 'role_2': 1, 'email_2': 'nandrade(a)caltech.edu'}] (Background on this error at: http://sqlalche.me/e/13/e3q8) Feb 23 23:27:10 2022 (300) Traceback (most recent call last): File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context self.dialect.do_execute( File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute cursor.execute(statement, parameters) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/cursors.py", line 148, in execute result = self._query(query) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/cursors.py", line 310, in _query conn.query(q) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 548, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 775, in _read_query_result result.read() File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 1156, in read first_packet = self.connection._read_packet() File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 725, in _read_packet packet.raise_for_error() File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.OperationalError: (1053, 'Server shutdown in progress') The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/mailmanve/lib/python3.9/site-packages/mailman/core/runner.py", line 176, in _one_iteration self._process_one_file(msg, msgdata) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/core/runner.py", line 269, in _process_one_file keepqueued = self._dispose(mlist, msg, msgdata) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/runners/outgoing.py", line 111, in _dispose self._func(mlist, msg, msgdata) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/mta/deliver.py", line 86, in deliver refused = agent.deliver(mlist, msg, msgdata) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/mta/base.py", line 164, in deliver member = mlist.members.get_member(recipient) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/model/roster.py", line 120, in get_member memberships = self._get_all_memberships(email) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/opt/mailmanve/lib/python3.9/site-packages/mailman/model/roster.py", line 116, in _get_all_memberships return members_a.union(members_u).all() File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 3373, in all return list(self) File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__ return self._execute_and_instances(context) File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1011, in execute return meth(self, multiparams, params) File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement ret = self._execute_context( File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context self._handle_dbapi_exception( File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception util.raise_( File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_ raise exception File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context self.dialect.do_execute( File "/opt/mailmanve/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute cursor.execute(statement, parameters) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/cursors.py", line 148, in execute result = self._query(query) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/cursors.py", line 310, in _query conn.query(q) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 548, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 775, in _read_query_result result.read() File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 1156, in read first_packet = self.connection._read_packet() File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/connections.py", line 725, in _read_packet packet.raise_for_error() File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/protocol.py", line 221, in raise_for_error err.raise_mysql_exception(self._data) File "/opt/mailmanve/lib/python3.9/site-packages/pymysql/err.py", line 143, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1053, 'Server shutdown in progress') [SQL: SELECT anon_1.member_id AS anon_1_member_id, anon_1.member__member_id AS anon_1_member__member_id, anon_1.member_role AS anon_1_member_role, anon_1.member_list_id AS anon_1_member_list_id, anon_1.member_moderation_action AS anon_1_member_moderation_action, anon_1.member_address_id AS anon_1_member_address_id, anon_1.member_preferences_id AS anon_1_member_preferences_id, anon_1.member_user_id AS anon_1_member_user_id, anon_1.member_bounce_score AS anon_1_member_bounce_score, anon_1.member_last_bounce_received AS anon_1_member_last_bounce_received, anon_1.member_last_warning_sent AS anon_1_member_last_warning_sent, anon_1.member_total_warnings_sent AS anon_1_member_total_warnings_sent FROM (SELECT `member`.id AS member_id, `member`._member_id AS member__member_id, `member`.`role` AS member_role, `member`.list_id AS member_list_id, `member`.moderation_action AS member_moderation_action, `member`.address_id AS member_address_id, `member`.preferences_id AS member_preferences_id, `member`.user_id AS member_user_id, `member`.bounce_score AS member_bounce_score, `member`.last_bounce_received AS member_last_bounce_received, `member`.last_warning_sent AS member_last_warning_sent, `member`.total_warnings_sent AS member_total_warnings_sent FROM `member`, address WHERE `member`.list_id = %(list_id_1)s AND `member`.`role` = %(role_1)s AND address.email = %(email_1)s AND `member`.address_id = address.id UNION SELECT `member`.id AS member_id, `member`._member_id AS member__member_id, `member`.`role` AS member_role, `member`.list_id AS member_list_id, `member`.moderation_action AS member_moderation_action, `member`.address_id AS member_address_id, `member`.preferences_id AS member_preferences_id, `member`.user_id AS member_user_id, `member`.bounce_score AS member_bounce_score, `member`.last_bounce_received AS member_last_bounce_received, `member`.last_warning_sent AS member_last_warning_sent, `member`.total_warnings_sent AS member_total_warnings_sent FROM `member`, address, user WHERE `member`.list_id = %(list_id_2)s AND `member`.`role` = %(role_2)s AND address.email = %(email_2)s AND `member`.user_id = user.id AND user._preferred_address_id = address.id) AS anon_1] [parameters: {'list_id_1': 'avery-disc.caltech.edu', 'role_1': 1, 'email_1': 'nandrade(a)caltech.edu', 'list_id_2': 'avery-disc.caltech.edu', 'role_2': 1, 'email_2': 'nandrade(a)caltech.edu'}] (Background on this error at: http://sqlalche.me/e/13/e3q8)

2 4

Hyperkitty failure?
by Stephen Daniel 24 Feb '22

24 Feb '22

Mailman3 people -- Hyperkitty is failing in my new setup. In the log I see messages that look like this: Feb 24 11:43:44 2022 (28621) HyperKitty failure on https://lists.example.com/archives/api/mailman/archive: <html> <head><title>401 Authorization Required</title> /opt/mailman/mm/mailman-hyperkitty.cfg looks like this: [general] base_url: https://lists.example.com/archives api_key: SOMESTRING settings_local.py includes this line: MAILMAN_ARCHIVER_KEY = 'SOMESTRING' and mailman.cfg contains [archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /opt/mailman/mm/mailman-hyperkitty.cfg Any suggestions on how to get hyperkitty working? Thanks! -- Steve

2 2

cleanup lost users completly - mailman shell
by Paul Paulchen 21 Feb '22

21 Feb '22

Hi community, I was wondering if there is a walk-through for the following issue - or is it even an issue? We got plenty lists with a lot of users come and go from time to time. Thanks to the thread, I understood how to pick on adress and delete them via the mailman shell; https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… >>> um = getUtility(IUserManager) >>> usr = um.get_user('user(a)example.com') >>> um.delete_user(usr) >>> commit() Thanks to this thread, I know how to find the lost souls in our mailman environment https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… >>> um = getUtility(IUserManager) >>> for usr in um.users: ... if len(list(usr.memberships.members)) > 0: ... continue ... adr = usr.addresses[0] ... print(f'Name: {adr.display_name}, Email: {adr.email}, Created: {usr.created_on}') So I can see them now. Is there a technique, to clean them up? To get rid of them out of the system? I am not a python programmer, but I guess there is a way to use, instead of a single user, a list of users in the delete command of the mailman shell? e.g. (sorry if this example leads to any pain for any coder/ programmer) user1(a)example.com user2(a)example.com user3(a)example.com >>> um = getUtility(IUserManager) >>> usr = um.get_user('user1(a)example.com','user2(a)example.com','user3(a)example.com') >>> um.delete_user(usr) >>> commit() Hope I made my intention clear. I want to get rid of the users who don't belong/ or are subscribe to a list anymore. Thanks in advance! Paul

4 3

aborted connection
by Sandor Dibuz 20 Feb '22

20 Feb '22

Am I right expecting that such warnings are harmless and probably come from timeouts and/or unterminated connections to the DB? Feb 19 14:15:18 localhost mariadbd[28859]: 2022-02-19 14:15:18 22154 [Warning] Aborted connection 22154 to db: 'mailman3' user: 'mailman3' host: 'localhost' (Got an error reading communication packets) Thank you. Sanyi

2 4

I did something stupid. Can i back out of it?
by William Oliver 20 Feb '22

20 Feb '22

So... I run a small mailinglist using mailman3 that has been workiing great for a couple of months. Last week, all my users from gmail got kicked off because gmail decided that the list messages were "unsolicited email". I didn't find out about it until the users hit their mailman3 bounce limit. I *thought* would be seeing all the bounce messages, but apparently not. I didn't see any config settings for "see all the bounce messages" so i thought to myself, "Self, well maybe that's because my mailinglist username doesn't have a mailbox in /home/listname". So, I decided to try that. I created /home/listname/Maildir/* and logged in as fplist to see if it would put mail there. In order to log in as listname, I gave the user listname a password. Then the world ended. The mailinglist works OK for *new* posts, but it is now impossible to *reply* to posts. So, if I post as a member from user(a)somewhere.org to listname(a)example.com, it works. If I hit reply, with the From: line rewritten as listname(a)example.com, it fails. The error I get is: The reported error was “Error performing TLS handshake: An unexpected TLS packet was received.”. My suspicion is that there are two passwords floating around for "listname" -- one in mailman3 and the one in /etc/shadow, but I don't know. In retrospect, this was not my finest moment. But is there an easy way to back out of this? Thanks! billo

2 8

certain msgs not archived
by Sandor Dibuz 18 Feb '22

18 Feb '22

Hi good folks, I'm new to the group, so bear with me please. A decade or more with Mm2.1 but recently upgraded to Mm3 so no experiences with this latter. Details: Debian 11.2 (stable), everything installed from deb packages, mariadb10.5, python3.9, mailman3.3.3, hyperkitty1.3.4, etc. everything stock Debian. A few low volume lists, one of them migrated from Mm2.1 but most started newly. Couple of glitches, resolved them most so far. but could not cope with this one yet so I must ask your advice. On one of my maillists certain messages can't seem to make it to the archive. They spread right among list members but stuck in /var/lib/mailman3/archives/hyperkitty/spool and hyperkitty can not handle and archive them due to finding an incorrect string. This is a mailman-web.log excerpt below believed to be relevant but please just tell me if you need further details. I didn't want to flood with longer quotation for now but I can share full Django error email if needed. 12:18:19 [Q] INFO Enqueued 8697 ERROR 2022-02-16 12:18:19,310 5898 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8C\\xB9\\x0D\\x0A...' for column `mailman3web`.`hyperkitty_email`.`conten t` at row 1") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py", line 40, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 81, in _decorator return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 127, in archive add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py", line 153, in add_to_list email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 743, in save self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 780, in save_base updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 873, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 910, in _do_insert return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) django.db.utils.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8C\\xB9\\x0D\\x0A...' for column `mailman3web`.`hyperkitty_email`.`content` a t row 1") ERROR 2022-02-16 12:18:19,310 5898 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xF0\\x9F\\x8C\\xB9\\x0D\\x0A...' for column `mailman3web`.`hyperkitty_email`.`conten t` at row 1") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py", line 40, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 81, in _decorator return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py", line 127, in archive add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py", line 153, in add_to_list email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 743, in save self.save_base(using=using, force_insert=force_insert, File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 780, in save_base updated = self._save_table( File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 873, in _save_table result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 910, in _do_insert return manager._insert([self], fields=fields, return_id=update_pk, File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 1186, in _insert return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers return executor(sql, params, many, context) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 71, in execute return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 209, in execute res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 315, in _query db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 226, in query _mysql.connection.query(self, query) Btw. I noticed this yesterday evening and there are 5 msgs stuck this way in spool so far. Thanks for you anticipated help. Sanyi

2 7

Thanks to this very helpful community!
by Stephen Daniel 18 Feb '22

18 Feb '22

All -- I want to express my heartfelt thanks to the people on this list who helped me stand up a modest mailman3 server. As of today I have 3 lists with a total of ~150 users running in production on a small VM in google's cloud. This implementation replaces a venerable mailman2 implementation running on servers that are to be decommissioned at some point in the near future. Thanks again! -- Stephen

1 0

FILTER_VHOST = True doesn't work fine with alias_domain
by Élodie BOSSIER 18 Feb '22

18 Feb '22

Hello, My mailman3 installation is now functional, but the config "FILTER_VHOST = True" doesn't work fine when I define "alias_domain" for the 2 domains. I need to define that because I need to get "postfix_vmap" file. As you can see, list from others domains are visible on this URL : https://ml.franceserv.fr/postorius/lists/ I would like https://ml.franceserv.fr/postorius/lists/ and https://ml.viticreation.fr/postorius/lists/ with their own lists. Where I should check please to solve this ? Thank you in advance.

2 5

Mailman3-web/Postorius behind Lighttpd
by Élodie BOSSIER 18 Feb '22

18 Feb '22

Hi, I'm using Mailman 3.3.3 (Tom Sawyer) on Debian Bullseye 11.2 from packages : #dpkg -l | grep mailman ii mailman3 3.3.3-1 all Mailing list management system ii mailman3-full 3.3.3-1 all Full Mailman3 mailing list management suite (metapackage) ii mailman3-web 0+20200530-2 all Django project integrating Mailman3 Postorius and HyperKitty ii python3-django-mailman3 1.3.5-2 all Django library to help interaction with Mailman3 (Python 3 version) ii python3-mailman-hyperkitty 1.1.0-10 all Mailman3 plugin to archive emails with HyperKitty ii python3-mailmanclient 3.3.2-1 all Python bindings for Mailman3 REST API (Python 3 version) I can access from localhost to http://127.0.0.1:8000/ which move me correctly to http://127.0.0.1:8000/postorius/lists/ and all seem to be correct. But 127.0.0.1:8000 listen only in localhost : #netstat -tanpu | grep 8000 tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 626175/python3 So I try to use my lighttpd as proxy as recommanded with this config : > scgi.protocol = "uwsgi" > scgi.server = ( > "/" => (( "host" => "127.0.0.1", "port" => 8000, "check-local" => > "disable" )), > ) but it's return me a "502 Bad Gateway". I see nothing on there logs : - /var/log/lighttpd/error.log - /var/log/mailman3/web/error.log - /var/log/mailman3/web/access.log But on the file : /var/log/mailman3/web/mailman-web.log I can see that : > QUERY_STRING > REQUEST_URI/postorius/lists/REDIRECT_STATUS200 > SCRIPT_NAME" 400 - I don't know what I'm doing wrong ... There is a way to solve it with lighttpd or bind the port 8000 to 0.0.0.0 instead 127.0.0.1 (which is not recommanded) ? Thank you in advance. Best regards, Elodie

2 2

High memory usage
by Prasanth Nair 18 Feb '22

18 Feb '22

Hi, We are having an issue with high memory usage on our Mailman3 server. Initially, our server had 16GB RAM and the memory usage went up from 5% to 92% within a week. So, we increased the memory capacity to 32GB RAM and we can see that the memory usage is going up in the same pattern. Any idea why it is happening? or any suggestions? Kind regards, Prasanth Nair

5 10

Template Ignored?
by Stephen Daniel 17 Feb '22

17 Feb '22

Mailman3 people -- I created a new list, and used postorius to create a template for " list:user:notice:welcome". I then used the mass subscription feature to add myself to the list, having checked "pre-confirmed", "pre-approved", and "pre-verified". I received a welcome message that appears to be the default welcome message. It certainly wasn't what I put into my template. Any suggestions on how I get mailman to use my template? Things I might be doing wrong? Thanks! -- Steve

2 5

changing the main template for subscriptions and more
by Jens Günther 15 Feb '22

15 Feb '22

Hey there, in Mailman2 there was a simple link in the moderation-emails, with which I was able to open directly the "held-messages". I know how to manually integrate such a link with the variables of mailman (https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/…). But: How can I change the template for all the lists (respectively maildomains/webdomains) so that I don't have to do this manually for all lists? I searched a bit for the corresponding template, but didn't find it. At the end it only says the following: "At your convenience, visit your dashboard to approve or deny the request." -> here I would like to add the following: "At your convenience, visit your dashboard to approve or deny the request: https://$domain/postorius/lists/$short_listname.$domain/held_messages" Or if someone sees another possibility to get us on the right track, it would be more than welcome, thank you! Kind regards, Jens.

4 7

Help with mailman user and mailman Web users
by Sherin 15 Feb '22

15 Feb '22

Hi, I have installed mailman3 server and configured MailmanWeb ( hyperkity + Postorious ) . The service and everything works fine, but I am little bit confuses with the user management in mailman3, 1) I created an owner user for a mailing list using the mailmanclient api tools, # from mailmanclient.client import Client 2) I can't login to the Mailmanweb with this user For further investigation shows the mailman have its own database and the web frontend have its own database too. So my question is how to configure the webfrontend ( hyperkit+ postorious ) to use the accounts logins created from the mailmanclient api from the mailman core database. Or is there a way to use only one user management system ? Any help will be appreciated. -- Sherin

2 1

converting bulk accept_these_nonmembers in migration from mailman 2 to 3
by Lucio Chiappetti 12 Feb '22

12 Feb '22

I have been instructed by Mark Sapiro to subscribe to this list to post questions related to mailman3. On Mon, 31 Jan 2022, Mark Sapiro wrote: > On 1/31/22 06:20, Lucio Chiappetti wrote: >> One of the lists of which I am a co-moderator has been moved from >> mailman 2 to 3. Actually the migration has been made operative yesterday (and so we got the first problems ...). Basic background. There are a number of lists within my institution (a nationwide research institution with many seats across our country), some are institutional (all staff subscribed ex officio) and managed by staff in one seat, some other are non-institutional (subscription is optional) and are managed by a few volunteers in other cities different from the seat where the mailman server runs. > In particular, the list settings for *_these_nonmembers are at Settings > -> Message Acceptance, but these settings in particular are deprecated > in Mailman 3 except for regexps. In Mailman 3 the preferred way to say > accept a nonmember is to create a nonmember for that user and set it's > Moderation action to Default Processing. If, a nonmember posts, the > nonmember record is automatically created, Let me explain our case, and how it was managed in mailman2. - originally our staff (of the order of 1000) had addresses like user(a)place.inaf.it or user(a)place.astro.it. Some had even two (say place1.inaf.it and place2.inaf.it), so they subscribed with ONE of their work addresses - later we got addresses like name.surname(a)inaf.it. New users subscribe with this address but MANY of the old ones continue to use the PREVIOUS address. - we implemented a spam filtering policy as follows: - staff (and former staff) can SUBSCRIBE with any address or CHANGE their address. Any subscribed address will be recognised. Mail is NOT held in moderation unless too bulky, or has undisclosed recipients - mail from non members with any address in the .it TLD or in the tng.iac.es domain (where some of our staff work) goes into moderation (this is achieved via regexp in hold_these_nonmembers) - anything else is silently discarded as spam (generic_nonmember_action = discard) - when some staff posts from a non-subscribed address (be it the home address, or one of the alternate addresses described above) it therefore goes into moderation because of hold_these_nonmembers - in mailman2 we could let them pass "manually" (if we recognised the person) or (in case they used an alternate address) tick to add them permanently to accept_these_nonmembers (in one tick) - at the end we got somehow fed up of this but we did NOT want to accept any address of the form ^.+@.+\.inaf.it (which could be forged) but found that we could simply put in accept_these_nonmembers two entries like @list1 @list2 where list1 and list2 are two institutional lists on the same server which contain all permanent and temporary staff or assimilated with their official address (these lists are automagically generated each week) (so at least any VALID EXISTING official address is accepted) What I found now (after the migration done by the administrators of the central seat) is that in mailman3: - accept_these_nonmembers is empty - there are 189 entries listed in Non-members (which look those formerly in accept_these_nonmembers) - hold_these_nonmembers was copied identical with the same regexp's as in mailman2 - I have inspected a few of them and found that they have Administration options Moderation set to List default - the effect of this apparently was that a post from some.guy(a)inaf.it (whose subscription address is guy(a)somewhere.inaf.it) was hold for moderation is this due to a loop with the regexp in hold_these_nonmembers ? the default action would be to let it pass Or should we set Administration options Moderation to Accept immediately ? - if so, how can we do it automatically for all 189 entries ? - or move back the 189 addresses to accept_these_nonmembers ? - or for the future have persons posting with alternate addrresses accepted for the future WITH ONE CLICk ? - AND LAST BUT NOT LEAST would the syntax @list1 in accept_these_nonmembers work in mailman3 ? already putting it back will save a lot of work. Many thanks -- Lucio Chiappetti - INAF/IASF - via Corti 12 - I-20133 Milano (Italy) For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html ------------------------------------------------------------------------ "All that is google does not glitter Nor all who use alpine/procmail are lost"

2 3

moderation notices for list-owner being redirected to mailman address
by Philippe B 10 Feb '22

10 Feb '22

We have seen a very strange behavior on 2 separate occasions/lists over the past month. A SENDER(a)SOME.DOMAIN emails OURLIST(a)OUR.DOMAIN, and that post is (correctly) held for moderation. The moderation notification is sent by OURLIST-owner(a)OUR.DOMAIN to OURLIST-owner(a)OUR.DOMAIN, and everything works fine. But in a few very rare cases, at some point during the convoluted delivery path to the mailbox of the actual human owner of OURLIST, the recipient is changed from OURLIST-owner(a)OUR.DOMAIN to mailman(a)OUR.DOMAIN. I am going to paste full headers below (with fake public IPs), but I think that these 2 particular lines highlight the problem: X-MailFrom: OURLIST-bounces+OURLIST-owner=OUR.DOMAIN(a)OUR.DOMAIN X-MailFrom: OURLIST-bounces+mailman=OUR.DOMAIN(a)OUR.DOMAIN because only the first one is present normally. We know from the headers that the recipient gets changed on the Mailman host, but we have no idea why or how, because all the aliases are set up correctly, and there is no connection between OURLIST's (non-)members/etc and mailman(a)OUR.DOMAIN. Do you have any troubleshooting suggestions? Thanks. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Note: ec2-12-34-56-78.us-west-2.compute.amazonaws.com, lists.OUR.DOMAIN, and MAILMAN-SERVER.OUR.DOMAIN are different ways of addressing essentially the same system. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Received: from SMTP-SERVER.OUR.DOMAIN (SMTP-SERVER.OUR.DOMAIN [6.7.8.9]) by MAILMAN-SERVER.OUR.DOMAIN (Postfix) with ESMTP id 2B6E51A288F for <mailman(a)lists.OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:52 -0800 (PST) Received: by SMTP-SERVER.OUR.DOMAIN (Postfix) id E825C207053B; Tue, 1 Feb 2022 07:10:51 -0800 (PST) Delivered-To: mailman(a)OUR.DOMAIN Received: from SMTP-SERVER.OUR.DOMAIN (localhost [127.0.0.1]) by SMTP-SERVER.OUR.DOMAIN (Postfix) with ESMTP id D468B2070A45 for <mailman(a)OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:51 -0800 (PST) X-Spam-Scanned: at OUR-ORGANIZATION on SMTP-SERVER.OUR.DOMAIN by amavisd-new Authentication-Results: SMTP-SERVER.OUR.DOMAIN (amavisd-new); dkim=pass (1024-bit key) header.d=OUR.DOMAIN header.b="********"; dkim=pass (1024-bit key) header.d=OUR.DOMAIN header.b="********" Received: from filter-send ([127.0.0.1]) by SMTP-SERVER.OUR.DOMAIN (SMTP-SERVER.OUR.DOMAIN [127.0.0.1]) (amavisd-new, port 12345) with LMTP id YgQELyXHaR1X for <mailman(a)OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:51 -0800 (PST) Received: from [172.17.0.2] (ec2-12-34-56-78.us-west-2.compute.amazonaws.com [12.34.56.78]) by SMTP-SERVER.OUR.DOMAIN (Postfix) with ESMTP id 9E8DE207053B for <mailman(a)OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:51 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 SMTP-SERVER.OUR.DOMAIN 9E8DE207053B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=OUR.DOMAIN; s=unixmail; t=**********; bh=*******************************************=; h=Subject:From:To:Date:List-Id:List-Help:List-Owner:List-Subscribe: List-Unsubscribe:From; b=***************************************************************** ****************************************************************** ****************************************= Received: from SMTP-SERVER.OUR.DOMAIN (SMTP-SERVER.OUR.DOMAIN [6.7.8.9]) by MAILMAN-SERVER.OUR.DOMAIN (Postfix) with ESMTP id B138C1A288F for <OURLIST-owner(a)lists.OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:49 -0800 (PST) Received: by SMTP-SERVER.OUR.DOMAIN (Postfix) id 7766E207053B; Tue, 1 Feb 2022 07:10:49 -0800 (PST) Delivered-To: OURLIST-owner(a)OUR.DOMAIN Received: from SMTP-SERVER.OUR.DOMAIN (localhost [127.0.0.1]) by SMTP-SERVER.OUR.DOMAIN (Postfix) with ESMTP id 676682070A73 for <OURLIST-owner(a)OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:49 -0800 (PST) X-Spam-Scanned: at OUR-ORGANIZATION on SMTP-SERVER.OUR.DOMAIN by amavisd-new Received: from filter-send ([127.0.0.1]) by SMTP-SERVER.OUR.DOMAIN (SMTP-SERVER.OUR.DOMAIN [127.0.0.1]) (amavisd-new, port 12345) with LMTP id j2xEFERoaVLS for <OURLIST-owner(a)OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:49 -0800 (PST) Received: from [172.17.0.2] (ec2-12-34-56-78.us-west-2.compute.amazonaws.com [12.34.56.78]) by SMTP-SERVER.OUR.DOMAIN (Postfix) with ESMTP id 2C264207053B for <OURLIST-owner(a)OUR.DOMAIN>; Tue, 1 Feb 2022 07:10:49 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 SMTP-SERVER.OUR.DOMAIN 2C264207053B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=OUR.DOMAIN; s=unixmail; t=**********; bh=*******************************************=; h=Subject:From:To:Date:List-Id:List-Help:List-Owner:List-Subscribe: List-Unsubscribe:From; b=***************************************************************** ****************************************************************** ****************************************= Subject: OURLIST(a)OUR.DOMAIN post from SENDER(a)SOME.DOMAIN requires approval From: OURLIST-owner(a)OUR.DOMAIN To: OURLIST-owner(a)OUR.DOMAIN MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6658935924254827098==" Message-ID: <164372824697.288.9427163605184740636@MAILMAN-SERVER> Date: Tue, 01 Feb 2022 07:10:46 -0800 Precedence: bulk X-Mailman-Version: 3.3.5 List-Id: <OURLIST.OUR.DOMAIN> List-Help: <mailto:OURLIST-request@OUR.DOMAIN?subject=help> List-Owner: <mailto:OURLIST-owner@OUR.DOMAIN> List-Subscribe: <mailto:OURLIST-join@OUR.DOMAIN> List-Unsubscribe: <mailto:OURLIST-leave@OUR.DOMAIN> X-MailFrom: OURLIST-bounces+OURLIST-owner=OUR.DOMAIN(a)OUR.DOMAIN X-MailFrom: OURLIST-bounces+mailman=OUR.DOMAIN(a)OUR.DOMAIN X-Mailman-Rule-Hits: implicit-dest X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-mailman.OUR.DOMAIN-0; nonmember-moderation; administrivia; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header

3 13

Customise Postorius templates?
by Duane Raymond 08 Feb '22

08 Feb '22

Hi, I'm looking to do some radical customisation of the Postorius (and Hyperkitty) templates over the next 6 ish months and was wondering if there is a 'best practice' way to do this that will survive updates to MM3. I've searched around and not found much and also tested copying venv/lib/python3.7/site-packages/postorius/templates/postorius/lists/summary.html to var/templates/lists/summary.html As suggested in some posts - but it didn't seem to pick up the customisation - only customisations in the venv path worked - and they get overwritten on update. Is there a 'best' way for this? I'd also like to eventually share them with the community and maybe add them to the repo if people want that....but for now I'll start with my own experiments. (and if there is documentation I have missed for this, let me know!)

5 10

Time Zone Issue in Postorius
by Andy Summers 08 Feb '22

08 Feb '22

Hello, I'm encountering an error that sounds similar to this previous thread: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… I've tried setting my TIME_ZONE to 'UTC' in my settings.py and settings_local.py files (and restarting Mailman), but when I check the datetime for a held message in Postorius, the datetime seems to be pushed +8 hours unexpectedly. My Postorius account time zone is set to "US/Pacific". As a workaround, I might be able to find a "wrong" time zone that would possibly reduce the unintentional additional hours difference and show the "correct" time zone in Postorius for the "Held Messages" page, but I don't think this is the right way to fix the issue. I think I've tried setting the TIME_ZONE setting to "America/Los_Angeles" in my settings.py file in place of 'UTC' in the past, but the result was still different than expected. Does anyone know if there are other places besides the mailman settings.py and settings_local.py files where something needs to be set so that I don't get an unintentional extra +8 hours of time difference when looking at messages on the "Held Messages" page of a mailing list? Thank you, Andrew

5 5

Error 500 on held_messages page
by Stephan Krinetzki 04 Feb '22

04 Feb '22

Hello all, we now have at least one mailing list (there may be more affected) that generates a 500 error when calling the held_messages page. After looking at the held messages via the mailman shell, we have identified three mails that have neither a subject nor a message body. How can we remove these three mails from the database? This is the mailman shell command we used: mailman shell -l listname(a)lists.example.com <removed> Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. Die Variable 'm' ist die listname(a)lists.example.com Mailingliste >>> msg_db = getUtility(IMessageStore) >>> req_db = IListRequests(m) >>> reqs = list(req_db.held_requests) >>> for req in reqs: ... print('{}: {}'.format(req.id, req.request_type)) ... 32598: 1 [...] 107656: 1 107737: 1 107758: 1 >>> from mailman.app.moderator import handle_message >>> handle_message(m, 107758, Action.discard) >>> reqs = list(req_db.held_requests) >>> for req in reqs: ... print('{}: {}'.format(req.id, req.request_type)) ... 32598: 1 [...] 107656: 1 107737: 1 >>> handle_message(m, 92927, Action.discard) Traceback (most recent call last): File "<console>", line 1, in <module> File "/opt/mailman/mailman-venv/lib64/python3.6/site-packages/mailman/app/moderator.py", line 112, in handle_message key, msgdata = requestdb.get_request(id) TypeError: 'NoneType' object is not iterable >>>

2 4

HELP: mailman send empty mails
by Axel Rau 04 Feb '22

04 Feb '22

In one mailing list all posts are empty, just one sentence: The content of this message was lost. It was probably cross-posted to multiple lists and previously handled on another list. What can I do? I'm lost Axel -- PGP-Key: CDE74120 ☀ computing @ chaos claudius

3 2

Docker mailman 3 and upgrading questions
by bob B 01 Feb '22

01 Feb '22

I am running mailman 3 in docker and trying to figure out -how to upgrade -can you back out. For example I installed docker with a git pull • git clone https://github.com/maxking/docker-mailman docker images shows REPOSITORY TAG IMAGE ID CREATED SIZE maxking/mailman-web 0.3 9ba9d3d7084d 4 months ago 487MB maxking/mailman-core 0.3 26c920c0094b 4 months ago 178MB **** database is hosted via external database I only installed it a few months ago, am I really on 0.3.0 , I assume I am on 3.0 from 2019 and that is what "git clone https://github.com/maxking/docker-mailman" gets???? looking at the release matrix "https://github.com/maxking/docker-mailman/releases?page=2" It looks like the latest releases are 0.4.1 for core & 0.4.2 for web, How do I upgrade, is it a "git pull" or "docker update" command? If something does not work, is there a way to back out? What version should I go to from 0.3? I am thinking it should be v0.3.12 cause that won't break anything? But how do I do that? The documentation at https://asynchronous.in/docker-mailman/. says

2 1

Is there an example of a complete installation for https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-install (ie. standard installation)?
by matt＠fes.org 01 Feb '22

01 Feb '22

What i'm looking for is the directory structure, files, configs of a working mailman3 installation (with passwords/secrets obscured, etc). - /etc/mailman3, /opt/mailman, /etc/nginx, /etc/postfix, etc... I've tried 3 or 4 installations with varying degrees of success. For me, more issues seem related to configuration, not the actual installation. So having an example installation to compare proper configuration (and perhaps file/directory placement) would really help to sanity check what i've installed. Since this installed copy would be created for each release, it would exist at some point. Is it possible this could be made available (with pws/secrets obscured...)? I've seen quite a few threads related to configuration issues...hopefully it would help with those as well.

2 3

National language messages in mm3 notifications
by Ivan Kuznetsov 01 Feb '22

01 Feb '22

Hello There is a problem with moderator/administrator notifications used national (non-acsii) characters in templates. Messages came with all non-ascii chars changed to '?' As I can see some parts of customizable text are originated from localized messages (mailman/messages/<country code>/LC_MESSAGES), other from site/domain/list templates. I made some localization/translation of LS_MESSAGES for Russian language via Weblate site and import .po file to test site setup. Web interface messages seems to be ok but email notifications are not. Content-type: and Content-Transfer-Encoding: headers are not suitable for national charsets. Mailman processes has enviroment variable LANG set to ru_RU.UTF-8. Preferred language for site administrator (listmaster) is Russian What am I missing in setup? Return-Path: <test-bounces(a)solvo.ru> [...] Received: from mail-g3-vlan10 (localhost [IPv6:::1]) by mail-g3 (Postfix) with ESMTP id 889784800512D for <kia(a)solvo.ru>; Tue, 18 Jan 2022 00:00:22 +0300 (MSK) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: ????????? ????????????? ???????? ? ?????? ???????? test(a)solvo.ru: 1. From: listmaster(a)solvo.ru To: =?utf-8?b?0JjQstCw0L0g0JrRg9C30L3QtdGG0L7Qsg==?= <kia(a)solvo.ru> Message-ID: <164245320388.3361727.10723274392705308145@mail-g3-vlan10> Date: Tue, 18 Jan 2022 00:00:03 +0300 Precedence: bulk X-Mailman-Version: 3.3.5 List-Id: =?utf-8?b?0KLQtdGB0YLQvtCy0YvQuSDRgdC/0LjRgdC+0Lo=?= <test.solvo.ru> List-Help: <mailto:test-request@solvo.ru?subject=help> List-Owner: <mailto:test-owner@solvo.ru> List-Subscribe: <mailto:test-join@solvo.ru> List-Unsubscribe: <mailto:test-leave@solvo.ru> [...] The test(a)solvo.ru list has 1 moderation requests waiting. ????????? ????????? ????????: ????????????: ikuznetsov(a)solvo.ru Please attend to this at your earliest convenience.

2 3