November 2022 - Mailman-users - lists.mailman3.org

Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
by Gordon Dickens Dec. 13, 2022

Dec. 13, 2022

My mailman3 installation has now been up and running successfully for several weeks without any issues and then the following error message was emailed to me by my server. What's up with this error message and what do I need to do? Thanks, Gordon -------- Forwarded Message -------- Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Date: Fri, 20 Nov 2020 01:25:41 -0000 From: [1]root(a)host2.mailhub4u.com To: [2]gordon(a)mailhub4u.com Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1") Request Method: POST Request URL: [3]http://localhost/mailman3/hyperkitty/api/mailman/archive?key=0123456 789abcdef0123456789ABCDEF Django Version: 1.11.29 Python Executable: /usr/bin/uwsgi-core Python Version: 3.7.3 Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Fri, 20 Nov 2020 01:25:40 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1")) was the direct cause of the following exception: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator 69. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view 58. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 115. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list 151. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 808. force_update=force_update, update_fields=update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 838. updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 924. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 963. using=using, raw=raw) File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 85. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1079. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1112. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 94. six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python3/dist-packages/django/utils/six.py" in reraise 685. raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1") Request information: USER: AnonymousUser GET: key = '0123456789abcdef0123456789ABCDEF' POST: mlist = '[4]members(a)georgiajets.org' FILES: message = <InMemoryUploadedFile: message.txt ()> COOKIES: No cookie data META: CONTENT_LENGTH = '7720' CONTENT_TYPE = 'multipart/form-data; boundary=8da553dce5f76815f62f12a1b8c0efac' CONTEXT_DOCUMENT_ROOT = '/var/www/html' CONTEXT_PREFIX = '' DOCUMENT_ROOT = '/var/www/html' GATEWAY_INTERFACE = 'CGI/1.1' HTTP_ACCEPT = '*/*' HTTP_ACCEPT_ENCODING = 'gzip, deflate' HTTP_CONNECTION = 'keep-alive' HTTP_HOST = 'localhost' HTTP_USER_AGENT = 'python-requests/2.21.0' PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' PATH_INFO = '/hyperkitty/api/mailman/archive' QUERY_STRING = 'key=0123456789abcdef0123456789ABCDEF' REMOTE_ADDR = '127.0.0.1' REMOTE_PORT = '53006' REQUEST_METHOD = 'POST' REQUEST_SCHEME = 'http' REQUEST_URI = '/mailman3/hyperkitty/api/mailman/archive?key=0123456789abcdef012345678 9ABCDEF' SCRIPT_FILENAME = 'proxy:uwsgi://localhost//hyperkitty/api/mailman/archive' SCRIPT_NAME = '/mailman3' SERVER_ADDR = '127.0.0.1' SERVER_ADMIN = 'webmaster@localhost' SERVER_NAME = 'localhost' SERVER_PORT = '80' SERVER_PROTOCOL = 'HTTP/1.1' SERVER_SIGNATURE = '<address>Apache/2.4.38 (Debian) Server at localhost Port 80</address>\n' SERVER_SOFTWARE = 'Apache/2.4.38 (Debian)' uwsgi.core = 0 uwsgi.node = b'slc.mailhub4u.com' uwsgi.version = b'2.0.18-debian' wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'> wsgi.file_wrapper = '' wsgi.input = <uwsgi._Input object at 0x7f6a278df600> wsgi.multiprocess = False wsgi.multithread = True wsgi.run_once = False wsgi.url_scheme = 'http' wsgi.version = '(1, 0)' Settings: Using settings module settings ABSOLUTE_URL_OVERRIDES = {} ACCOUNT_AUTHENTICATION_METHOD = 'username_email' ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https' ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_EMAIL_VERIFICATION = 'mandatory' ACCOUNT_UNIQUE_EMAIL = True ADMINS = "(('Mailman Suite Admin', '[5]gordon(a)mailhub4u.com'),)" ALLOWED_HOSTS = ['*'] APPEND_SLASH = True AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')" AUTH_PASSWORD_VALIDATORS = '********************' AUTH_USER_MODEL = 'auth.User' BASE_DIR = '/usr/share/mailman3-web' CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}} CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_KEY_PREFIX = '********************' CACHE_MIDDLEWARE_SECONDS = 600 COMPRESS_CACHEABLE_PRECOMPILERS = '()' COMPRESS_CACHE_BACKEND = 'default' COMPRESS_CACHE_KEY_FUNCTION = '********************' COMPRESS_CLEAN_CSS_ARGUMENTS = '' COMPRESS_CLEAN_CSS_BINARY = 'cleancss' COMPRESS_CLOSURE_COMPILER_ARGUMENTS = '' COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar' COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor' COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter'] COMPRESS_CSS_HASHING_METHOD = 'mtime' COMPRESS_DATA_URI_MAX_SIZE = 1024 COMPRESS_DEBUG_TOGGLE = None COMPRESS_ENABLED = True COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f6a2d586620> COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor' COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter'] COMPRESS_MINT_DELAY = 30 COMPRESS_MTIME_DELAY = 10 COMPRESS_OFFLINE = True COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'} COMPRESS_OFFLINE_MANIFEST = 'manifest.json' COMPRESS_OFFLINE_TIMEOUT = 31536000 COMPRESS_OUTPUT_DIR = 'CACHE' COMPRESS_PARSER = 'compressor.parser.AutoSelectParser' COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))" COMPRESS_REBUILD_TIMEOUT = 2592000 COMPRESS_ROOT = '/var/lib/mailman3/web/static' COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage' COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'} COMPRESS_URL = '/mailman3/static/' COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/' COMPRESS_VERBOSE = False COMPRESS_YUGLIFY_BINARY = 'yuglify' COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal' COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal' COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar' COMPRESS_YUI_CSS_ARGUMENTS = '' COMPRESS_YUI_JS_ARGUMENTS = '' CSRF_COOKIE_AGE = 31449600 CSRF_COOKIE_DOMAIN = None CSRF_COOKIE_HTTPONLY = False CSRF_COOKIE_NAME = 'csrftoken' CSRF_COOKIE_PATH = '/' CSRF_COOKIE_SECURE = False CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure' CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN' CSRF_TRUSTED_ORIGINS = [] CSRF_USE_SESSIONS = False DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}} DATABASE_ROUTERS = [] DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440 DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000 DATETIME_FORMAT = 'N j, Y, P' DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y'] DATE_FORMAT = 'N j, Y' DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y'] DEBUG = False DEBUG_PROPAGATE_EXCEPTIONS = False DECIMAL_SEPARATOR = '.' DEFAULT_CHARSET = 'utf-8' DEFAULT_CONTENT_TYPE = 'text/html' DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter' DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage' DEFAULT_FROM_EMAIL = '[6]postorius(a)host2.mailhub4u.com' DEFAULT_INDEX_TABLESPACE = '' DEFAULT_TABLESPACE = '' DISALLOWED_USER_AGENTS = [] EMAILNAME = 'host2.mailhub4u.com' EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST = 'localhost' EMAIL_HOST_PASSWORD = '********************' EMAIL_HOST_USER = '' EMAIL_PORT = 25 EMAIL_SSL_CERTFILE = None EMAIL_SSL_KEYFILE = '********************' EMAIL_SUBJECT_PREFIX = '[Django] ' EMAIL_TIMEOUT = None EMAIL_USE_LOCALTIME = False EMAIL_USE_SSL = False EMAIL_USE_TLS = False FILE_CHARSET = 'utf-8' FILE_UPLOAD_DIRECTORY_PERMISSIONS = None FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler'] FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440 FILE_UPLOAD_PERMISSIONS = None FILE_UPLOAD_TEMP_DIR = None FILTER_VHOST = False FIRST_DAY_OF_WEEK = 0 FIXTURE_DIRS = [] FORCE_SCRIPT_NAME = None FORMAT_MODULE_PATH = None FORM_RENDERER = 'django.forms.renderers.DjangoTemplates' HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}} HOSTNAME = 'localhost.local' IGNORABLE_404_URLS = [] INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')" INTERNAL_IPS = [] LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')] LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur'] LANGUAGE_CODE = 'en-us' LANGUAGE_COOKIE_AGE = None LANGUAGE_COOKIE_DOMAIN = None LANGUAGE_COOKIE_NAME = 'django_language' LANGUAGE_COOKIE_PATH = '/' LOCALE_PATHS = [] LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}} LOGGING_CONFIG = 'logging.config.dictConfig' LOGIN_REDIRECT_URL = 'list_index' LOGIN_URL = 'account_login' LOGOUT_REDIRECT_URL = None LOGOUT_URL = 'account_logout' MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')" MAILMAN_ARCHIVER_KEY = '********************' MAILMAN_REST_API_PASS = '********************' MAILMAN_REST_API_URL = '********************' MAILMAN_REST_API_USER = '********************' MANAGERS = [] MEDIA_ROOT = '' MEDIA_URL = '' MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage' MESSAGE_TAGS = {40: 'danger'} MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')" MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware'] MIGRATION_MODULES = {} MONTH_DAY_FORMAT = 'F j' NUMBER_GROUPING = 0 PASSWORD_HASHERS = '********************' PASSWORD_RESET_TIMEOUT_DAYS = '********************' POSTORIUS_TEMPLATE_BASE_URL = '[7]http://localhost/mailman3/' PREPEND_WWW = False Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5} ROOT_URLCONF = 'urls' SECRET_KEY = '********************' SECURE_BROWSER_XSS_FILTER = False SECURE_CONTENT_TYPE_NOSNIFF = False SECURE_HSTS_INCLUDE_SUBDOMAINS = False SECURE_HSTS_PRELOAD = False SECURE_HSTS_SECONDS = 0 SECURE_PROXY_SSL_HEADER = None SECURE_REDIRECT_EXEMPT = [] SECURE_SSL_HOST = None SECURE_SSL_REDIRECT = False SERVER_EMAIL = '[8]root(a)host2.mailhub4u.com' SESSION_CACHE_ALIAS = 'default' SESSION_COOKIE_AGE = 1209600 SESSION_COOKIE_DOMAIN = None SESSION_COOKIE_HTTPONLY = True SESSION_COOKIE_NAME = 'sessionid' SESSION_COOKIE_PATH = '/' SESSION_COOKIE_SECURE = False SESSION_ENGINE = 'django.contrib.sessions.backends.db' SESSION_EXPIRE_AT_BROWSER_CLOSE = False SESSION_FILE_PATH = None SESSION_SAVE_EVERY_REQUEST = False SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer' SETTINGS_MODULE = 'settings' SHORT_DATETIME_FORMAT = 'm/d/Y P' SHORT_DATE_FORMAT = 'm/d/Y' SIGNING_BACKEND = 'django.core.signing.TimestampSigner' SILENCED_SYSTEM_CHECKS = [] SITE_ID = 1 SOCIALACCOUNT_PROVIDERS = {} STATICFILES_DIRS = '()' STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')" STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' STATIC_ROOT = '/var/lib/mailman3/web/static' STATIC_URL = '/mailman3/static/' TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}] TEST_NON_SERIALIZED_APPS = [] TEST_RUNNER = 'django.test.runner.DiscoverRunner' THOUSAND_SEPARATOR = ',' TIME_FORMAT = 'P' TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] TIME_ZONE = 'UTC' USE_ETAGS = False USE_I18N = True USE_L10N = True USE_THOUSAND_SEPARATOR = False USE_TZ = True USE_X_FORWARDED_HOST = True USE_X_FORWARDED_PORT = False WSGI_APPLICATION = 'wsgi.application' X_FRAME_OPTIONS = 'SAMEORIGIN' YEAR_MONTH_FORMAT = 'F Y' References 1. mailto:root@host2.mailhub4u.com 2. mailto:gordon@mailhub4u.com 3. http://localhost/mailman3/hyperkitty/api/mailman/archive?key=0123456789abcd… 4. mailto:members@georgiajets.org 5. mailto:gordon@mailhub4u.com 6. mailto:postorius@host2.mailhub4u.com 7. http://localhost/mailman3/ 8. mailto:root@host2.mailhub4u.com

6 28

Permission denied for fulltext_index
by Jan Eden Dec. 1, 2022

Dec. 1, 2022

Hi, when trying to reattach a message thread in the Hyperkitty web interface, the following error occurs: > Internal Server Error: /archives/list/testlist(a)lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > > PermissionError at /archives/list/testlist(a)lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > [Errno 13] Permission denied: 'fulltext_index' The error occurs while haystack attempts to create a directory – > File "/opt/mailman/venv/lib/python3.10/site-packages/haystack/backends/whoosh_backend.py", line 412, in search > self.setup() > File "/opt/mailman/venv/lib/python3.10/site-packages/haystack/backends/whoosh_backend.py", line 133, in setup > os.makedirs(self.path) > File "/usr/lib/python3.10/os.py", line 225, in makedirs > mkdir(name, mode) > > Exception Type: PermissionError at /archives/list/testlist(a)lists.eden.one/thread/T6MCILAVQYV5QIOGHJB3JNJ26Z7E2Z35/reattach-suggest > Exception Value: [Errno 13] Permission denied: 'fulltext_index' > Raised during: hyperkitty.views.thread.reattach_suggest > Request information: > USER: mailman – but the existing directory /opt/mailman/fulltext_index (and its contents) are readable and writeable by mailman:mailman: > drwxrwxr-x 2 mailman mailman 4096 Nov 27 08:00 fulltext_index/ > > -rw-rw-r-- 1 mailman mailman 4473 Nov 27 08:00 _MAIN_65.toc > -rw-rw-r-- 1 mailman mailman 15456 Nov 27 08:00 MAIN_bc0cs2aohovftat6.seg > -rw-rw-r-- 1 mailman mailman 15454 Nov 27 06:00 MAIN_brbo8i8edmlz0za4.seg > -rw-rw-r-- 1 mailman mailman 15462 Nov 27 07:00 MAIN_ewnemc7lsnodd3y7.seg > -rw-rw-r-- 1 mailman mailman 157324 Nov 27 04:00 MAIN_qag60g1rrumshtc2.seg > -rw-rw-r-- 1 mailman mailman 15459 Nov 27 05:00 MAIN_w6e0ohurasusy61d.seg > -rwxrwxr-x 1 mailman mailman 0 Nov 24 16:00 MAIN_WRITELOCK* I fail to see where haystack tries (and fails) to create the directory. FWIW, deleting threads is not a problem. Thanks, Jan

2 10

Mailman web interface / uwsgi
by Jan Eden Nov. 28, 2022

Nov. 28, 2022

Hi, I followed the installation instructions at https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst…, which thankfully shielded me from configuring the Django applications for the web interface manually. All Django applications (postorius, hyperkitty, haystack etc) are loaded via mailman_web/settings/base.py, which is in turn referenced by the top-level settings file in /etc/mailman3. uwsgi is linked to mailman_web via /etc/mailman3/uwsgi.ini. So far, so good. I do not want to dive into the interaction between the various components and just want to switch from a http socket (http-socket = 0.0.0.0:8000) to a file socket (socket = /opt/mailman/mm/var/mailman.sock). After making this change in /etc/mailman3/uwsgi.ini, I applied it in the nginx proxy configuration proxy_pass http://127.0.0.1:8000; → proxy_pass http://unix:/opt/mailman/mm/var/mailman.sock; and received an error message (as expected): 2022/11/25 18:39:12 [error] 2978606#2978606: *3 upstream prematurely closed connection while reading response header from upstream, client: 37.201.155.65, server: lists.eden.one, request: "GET /mailman3/lists/ HTTP/2.0", upstream: "http://unix:/opt/mailman/mm/var/mailman.sock:/mailman3/lists/", host: "lists.eden.one" There are obviously a couple of places where the http-socket on port 8000 is referenced (e.g. mailman_web/settings/mailman.py, /opt/mailman/mm/hyperkitty.cfg). Is there an overview of the required changes when using a different (file) socket? Related question: My own (much simpler) Django applications use the following (e.g. in mysite_nginx.conf): upstream django { server unix:///private/tmp/mysite.sock; } location / { uwsgi_pass django; } instead of the "http://unix:/..." notation I tried for Mailman above. Is there any difference between the upstream/uwsgi_pass combination and the proxy_pass variant? Thanks, Jan

3 8

Default list styles
by Jan Eden Nov. 27, 2022

Nov. 27, 2022

Hi, the list creation web interface offers three list styles (announce, ordinary, ordinary w/ private archive). When selecting the announce style, I would expect a different default message acceptance setting for members (i.e. reject), because this is how an announce list should work. In practice, a list created with the announce style is configured just as an ordinary mailing list (as far as I can see in the web interface, and according to the handling of a test message sent by a regular member of an announce list). What am I missing here? - Jan

2 1

Mailman3 not able to deliver emails sent through list, but moderation notifications
by Daniel Böhmer Nov. 27, 2022

Nov. 27, 2022

Hello Mailman users, I've been using Mailman2 for 10+ years and have recently upgraded to Debian Buster with Mailman3. I've tried to setup Mailman3 similarly but it fails to deliver emails sent through a list. I do receive all moderation notifications at the same time. During startup /var/log/mailman3/mailman.log receives: > Cannot connect to SMTP server localhost on port 25 but I've made sure postfix is listening on *:25 for both IPv4 and IPv6. When sending a message to a list the same file receives: > Nov 25 11:38:14 2022 (70371) ACCEPT: <MESSAGEID@HOST> (message ID and host redacted) and nothing else happens: no more log messages, Postfix logs the email sent TO Mailman but doesn't log anything about emails sent BY Mailman. If Mailman fails to send emails via Postfix I'd expect it to log error messages but it doesn't seem to do so. I found this article stating the the SMTP error message might be misleading: https://blog.tfiu.de/mailman3-cannot-connect-to-smtp-server-localhost-on-po… How to further debug this? What could be wrong with my Mailman3 instance and how to fix it? Kind regards Daniel Böhmer

3 5

MTA authentication and smtpd_sender_login_maps
by Jan Eden Nov. 25, 2022

Nov. 25, 2022

Hi, I installed Mailman3 successfully and would like to change the default smtp settings to an authenticated connection (smtp_user, smtp_pass). But my Postfix configuration restricts sending messages via the smtpd_sender_login_maps directive (which points to a hashed file I created years ago) and correctly refuses connections for messages with the test mailing list's From header. I obviously do not want to add an entry to the map file for each mailing list created by someone on the server. Is there a way to programmatically create a sender login map with Mailman? Thanks, Jan

2 2

Hyperkitty no longer archiving
by Hirayama, Pat Nov. 23, 2022

Nov. 23, 2022

Greetings, A couple of months ago, I needed to move the Mailman3 server I inherited from one AWS account to another AWS account. This necessitated a change in the IP address of instance hosting the docker containers that comprise my mailman3 server. I thought I had updated the IP address wherever it needed to be, but I suppose I've missed something, because hyperkitty has not been archiving any messages. (Most of my lists don't archive, so it took a while before someone noticed). When I look at /opt/mailman/var/archives/hyperkitty/spool, I see messages dating back over the past couple of months waiting to be archived. So, they don't appear to be lost (and things are being mailed out normally). The thing is, the IP addresses of the docker containers haven't changed with respect to docker networking. docker networking ------------------------ docker host 172.31.0.1 mailman-core 172.31.0.3 mailman-nginx 172.31.0.5 mailman-postfix 172.31.0.4 mailman-django-uwsgi 172.31.0.2 settings_local.py: MAILMAN_ARCHIVER_FROM = ('172.18.8.14', '172.31.0.1', '172.31.0.2', '172.31.0.3', '172.31.0.4', '172.31.0.5', '127.0.0.1', '::1') During troubleshooting, I added 172.18.8.14 (new IP of the webserver), but that didn't seem to help. It seems like I added one of the docker addresses that was missing .... but again, that doesn't seem to have fixed the issue. mailmansuite.log is full of: ERROR 2022-11-22 00:25:40,365 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured ERROR 2022-11-22 00:25:40,399 7 hyperkitty.views.mailman Access to the archiving API endpoint was forbidden from IP 172.31.0.1, your MAILMAN_ARCHIVER_FROM setting may be misconfigured Any suggestions as to where I might look for a solution, or what setting I need to change? Thanks! -p Pat Hirayama Pronouns: he/him/his Systems Engineer IT | Systems Engineering Fred Hutchinson Cancer Center O 206.667.4856 phirayam(a)fredhutch.org<mailto:phirayam@fredhutch.org>

4 12

Upgrading a Ubuntu Mailman 3 Installation
by Allan Hansen Nov. 21, 2022

Nov. 21, 2022

Dear Mailman 3 wizards, Several years ago I tried upgrading to Mailman 3 (I did install Mailman 2), but quickly ran into a thicket too complex for me to get out of. Two others - way more experienced than me at system-level manipulations - tried as well but did not succeed. Brian Carpenter of EMWD installed the system seemingly effortlessly, and seemingly overnight. We had a good relationship, and I was a beta site for his new product, SAAS. Brian also did the system MM3 upgrades as they appeared and would check that all was well a couple of times a year. Unfortunately, and I respect that, the current management at EMWD "don't provide working hands service for foreign Mailman 3 installations." I do not dare touch it, based on my earlier attempts with the installation. With Brian's passing I'm thus faced with a system that has no capable system support (I manage the lists, not the core). I have 50+ lists with 10000+ subscription total, and a lot of people depend on this tool. (It's a non-profit organization). So, I'm interesting in hiring someone for part-time, occasional work (a few times a year). Update MM3 as new versions are made available. Diagnose and fix the system if anything goes awry. I will pay you, as I did Brian, out of my own pocket, for your time. Yours, Allan Hansen

2 1

Migrating to mailman-web and Ubuntu 22.04
by Andrew Hodgson Nov. 16, 2022

Nov. 16, 2022

Hi. For various reasons I want to migrate to a larger host running Ubuntu 22.04. I have set up the various packages ok and as part of the migration I want to use mailman-web instead of mailman-suite which is an older setup. I have instructions for Mailman-web for setting up Uwsgi but I would prefer to keep Gunicorn working as per the instructions here: DOC/Mailman 3 installation experience - Mailman Wiki (list.org)<https://wiki.list.org/DOC/Mailman%203%20installation%20experience> Has anyone done this before? Thanks. Andrew.

3 5

New Mailman Core Release
by Mark Sapiro Nov. 15, 2022

Nov. 15, 2022

We are pleased to announce the release of Mailman core 3.3.7. It is published to PyPI and you can find links to all the PyPI projects: https://docs.mailman3.org/en/latest/ The change log is at https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/NEWS.… - please note the `BREAKING CHANGE` documented there and due to our now using SQLAlchemy >-1.4. Finally, the minimum Python version required is 3.7 although, 3.9 is recommended if you are doing a new install for longest support. 3.10 is supported for Django apps, but Mailman Core is missing that due to one of the core dependencies (aiosmtpd) not supporting 3.10 yet. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 0