July 2022 - Mailman-users - lists.mailman3.org

Emergency moderation and clearing out a lot of held messages
by bob B 21 Dec '22

21 Dec '22

We had an issue where a program went rogue and sent out 30k email alerts to one of our lists, so now I have a few questions. When the issue happened I turned on emergency moderation and we have resolved the issue so.. 1. How do I easily delete 30k of held messages? It looks like the GUI only allows me to do 200 at a time? is there a way to flush all held messages easily? 2. When I enabled Emergency moderation, I expected it to stop notifying the admin on every held message, but it appears that is not the case? Also, does it reply to the sender of every held message when Emergency moderation is on? it seems like the steps are: turn on "Emergency moderation" turn OFF "Notify users of held messages" turn OFF "Admin immed notify"

6 28

Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
by Gordon Dickens 13 Dec '22

13 Dec '22

My mailman3 installation has now been up and running successfully for several weeks without any issues and then the following error message was emailed to me by my server. What's up with this error message and what do I need to do? Thanks, Gordon -------- Forwarded Message -------- Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Date: Fri, 20 Nov 2020 01:25:41 -0000 From: [1]root(a)host2.mailhub4u.com To: [2]gordon(a)mailhub4u.com Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1") Request Method: POST Request URL: [3]http://localhost/mailman3/hyperkitty/api/mailman/archive?key=0123456 789abcdef0123456789ABCDEF Django Version: 1.11.29 Python Executable: /usr/bin/uwsgi-core Python Version: 3.7.3 Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Fri, 20 Nov 2020 01:25:40 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Traceback: File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1")) was the direct cause of the following exception: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner 40. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator 69. return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view 58. return view_func(*args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive 115. add_to_list(mlist_fqdn, msg) File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list 151. email.save() File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save 808. force_update=force_update, update_fields=update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base 838. updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table 924. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert 963. using=using, raw=raw) File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method 85. return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert 1079. return query.get_compiler(using=using).execute_sql(return_id) File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql 1112. cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__ 94. six.reraise(dj_exc_type, dj_exc_value, traceback) File "/usr/lib/python3/dist-packages/django/utils/six.py" in reraise 685. raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute 64. return self.cursor.execute(sql, params) File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute 101. return self.cursor.execute(query, args) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 250. self.errorhandler(self, exc, value) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler 50. raise errorvalue File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute 247. res = self._query(query) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query 411. rowcount = self._do_query(q) File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query 374. db.query(q) File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query 292. _mysql.connection.query(self, query) Exception Type: OperationalError at /hyperkitty/api/mailman/archive Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1") Request information: USER: AnonymousUser GET: key = '0123456789abcdef0123456789ABCDEF' POST: mlist = '[4]members(a)georgiajets.org' FILES: message = <InMemoryUploadedFile: message.txt ()> COOKIES: No cookie data META: CONTENT_LENGTH = '7720' CONTENT_TYPE = 'multipart/form-data; boundary=8da553dce5f76815f62f12a1b8c0efac' CONTEXT_DOCUMENT_ROOT = '/var/www/html' CONTEXT_PREFIX = '' DOCUMENT_ROOT = '/var/www/html' GATEWAY_INTERFACE = 'CGI/1.1' HTTP_ACCEPT = '*/*' HTTP_ACCEPT_ENCODING = 'gzip, deflate' HTTP_CONNECTION = 'keep-alive' HTTP_HOST = 'localhost' HTTP_USER_AGENT = 'python-requests/2.21.0' PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' PATH_INFO = '/hyperkitty/api/mailman/archive' QUERY_STRING = 'key=0123456789abcdef0123456789ABCDEF' REMOTE_ADDR = '127.0.0.1' REMOTE_PORT = '53006' REQUEST_METHOD = 'POST' REQUEST_SCHEME = 'http' REQUEST_URI = '/mailman3/hyperkitty/api/mailman/archive?key=0123456789abcdef012345678 9ABCDEF' SCRIPT_FILENAME = 'proxy:uwsgi://localhost//hyperkitty/api/mailman/archive' SCRIPT_NAME = '/mailman3' SERVER_ADDR = '127.0.0.1' SERVER_ADMIN = 'webmaster@localhost' SERVER_NAME = 'localhost' SERVER_PORT = '80' SERVER_PROTOCOL = 'HTTP/1.1' SERVER_SIGNATURE = '<address>Apache/2.4.38 (Debian) Server at localhost Port 80</address>\n' SERVER_SOFTWARE = 'Apache/2.4.38 (Debian)' uwsgi.core = 0 uwsgi.node = b'slc.mailhub4u.com' uwsgi.version = b'2.0.18-debian' wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'> wsgi.file_wrapper = '' wsgi.input = <uwsgi._Input object at 0x7f6a278df600> wsgi.multiprocess = False wsgi.multithread = True wsgi.run_once = False wsgi.url_scheme = 'http' wsgi.version = '(1, 0)' Settings: Using settings module settings ABSOLUTE_URL_OVERRIDES = {} ACCOUNT_AUTHENTICATION_METHOD = 'username_email' ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https' ACCOUNT_EMAIL_REQUIRED = True ACCOUNT_EMAIL_VERIFICATION = 'mandatory' ACCOUNT_UNIQUE_EMAIL = True ADMINS = "(('Mailman Suite Admin', '[5]gordon(a)mailhub4u.com'),)" ALLOWED_HOSTS = ['*'] APPEND_SLASH = True AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')" AUTH_PASSWORD_VALIDATORS = '********************' AUTH_USER_MODEL = 'auth.User' BASE_DIR = '/usr/share/mailman3-web' CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}} CACHE_MIDDLEWARE_ALIAS = 'default' CACHE_MIDDLEWARE_KEY_PREFIX = '********************' CACHE_MIDDLEWARE_SECONDS = 600 COMPRESS_CACHEABLE_PRECOMPILERS = '()' COMPRESS_CACHE_BACKEND = 'default' COMPRESS_CACHE_KEY_FUNCTION = '********************' COMPRESS_CLEAN_CSS_ARGUMENTS = '' COMPRESS_CLEAN_CSS_BINARY = 'cleancss' COMPRESS_CLOSURE_COMPILER_ARGUMENTS = '' COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar' COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor' COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter'] COMPRESS_CSS_HASHING_METHOD = 'mtime' COMPRESS_DATA_URI_MAX_SIZE = 1024 COMPRESS_DEBUG_TOGGLE = None COMPRESS_ENABLED = True COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f6a2d586620> COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor' COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter'] COMPRESS_MINT_DELAY = 30 COMPRESS_MTIME_DELAY = 10 COMPRESS_OFFLINE = True COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'} COMPRESS_OFFLINE_MANIFEST = 'manifest.json' COMPRESS_OFFLINE_TIMEOUT = 31536000 COMPRESS_OUTPUT_DIR = 'CACHE' COMPRESS_PARSER = 'compressor.parser.AutoSelectParser' COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))" COMPRESS_REBUILD_TIMEOUT = 2592000 COMPRESS_ROOT = '/var/lib/mailman3/web/static' COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage' COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'} COMPRESS_URL = '/mailman3/static/' COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/' COMPRESS_VERBOSE = False COMPRESS_YUGLIFY_BINARY = 'yuglify' COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal' COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal' COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar' COMPRESS_YUI_CSS_ARGUMENTS = '' COMPRESS_YUI_JS_ARGUMENTS = '' CSRF_COOKIE_AGE = 31449600 CSRF_COOKIE_DOMAIN = None CSRF_COOKIE_HTTPONLY = False CSRF_COOKIE_NAME = 'csrftoken' CSRF_COOKIE_PATH = '/' CSRF_COOKIE_SECURE = False CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure' CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN' CSRF_TRUSTED_ORIGINS = [] CSRF_USE_SESSIONS = False DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}} DATABASE_ROUTERS = [] DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440 DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000 DATETIME_FORMAT = 'N j, Y, P' DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y'] DATE_FORMAT = 'N j, Y' DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y'] DEBUG = False DEBUG_PROPAGATE_EXCEPTIONS = False DECIMAL_SEPARATOR = '.' DEFAULT_CHARSET = 'utf-8' DEFAULT_CONTENT_TYPE = 'text/html' DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter' DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage' DEFAULT_FROM_EMAIL = '[6]postorius(a)host2.mailhub4u.com' DEFAULT_INDEX_TABLESPACE = '' DEFAULT_TABLESPACE = '' DISALLOWED_USER_AGENTS = [] EMAILNAME = 'host2.mailhub4u.com' EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend' EMAIL_HOST = 'localhost' EMAIL_HOST_PASSWORD = '********************' EMAIL_HOST_USER = '' EMAIL_PORT = 25 EMAIL_SSL_CERTFILE = None EMAIL_SSL_KEYFILE = '********************' EMAIL_SUBJECT_PREFIX = '[Django] ' EMAIL_TIMEOUT = None EMAIL_USE_LOCALTIME = False EMAIL_USE_SSL = False EMAIL_USE_TLS = False FILE_CHARSET = 'utf-8' FILE_UPLOAD_DIRECTORY_PERMISSIONS = None FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler'] FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440 FILE_UPLOAD_PERMISSIONS = None FILE_UPLOAD_TEMP_DIR = None FILTER_VHOST = False FIRST_DAY_OF_WEEK = 0 FIXTURE_DIRS = [] FORCE_SCRIPT_NAME = None FORMAT_MODULE_PATH = None FORM_RENDERER = 'django.forms.renderers.DjangoTemplates' HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}} HOSTNAME = 'localhost.local' IGNORABLE_404_URLS = [] INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')" INTERNAL_IPS = [] LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')] LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur'] LANGUAGE_CODE = 'en-us' LANGUAGE_COOKIE_AGE = None LANGUAGE_COOKIE_DOMAIN = None LANGUAGE_COOKIE_NAME = 'django_language' LANGUAGE_COOKIE_PATH = '/' LOCALE_PATHS = [] LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}} LOGGING_CONFIG = 'logging.config.dictConfig' LOGIN_REDIRECT_URL = 'list_index' LOGIN_URL = 'account_login' LOGOUT_REDIRECT_URL = None LOGOUT_URL = 'account_logout' MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')" MAILMAN_ARCHIVER_KEY = '********************' MAILMAN_REST_API_PASS = '********************' MAILMAN_REST_API_URL = '********************' MAILMAN_REST_API_USER = '********************' MANAGERS = [] MEDIA_ROOT = '' MEDIA_URL = '' MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage' MESSAGE_TAGS = {40: 'danger'} MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')" MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware'] MIGRATION_MODULES = {} MONTH_DAY_FORMAT = 'F j' NUMBER_GROUPING = 0 PASSWORD_HASHERS = '********************' PASSWORD_RESET_TIMEOUT_DAYS = '********************' POSTORIUS_TEMPLATE_BASE_URL = '[7]http://localhost/mailman3/' PREPEND_WWW = False Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5} ROOT_URLCONF = 'urls' SECRET_KEY = '********************' SECURE_BROWSER_XSS_FILTER = False SECURE_CONTENT_TYPE_NOSNIFF = False SECURE_HSTS_INCLUDE_SUBDOMAINS = False SECURE_HSTS_PRELOAD = False SECURE_HSTS_SECONDS = 0 SECURE_PROXY_SSL_HEADER = None SECURE_REDIRECT_EXEMPT = [] SECURE_SSL_HOST = None SECURE_SSL_REDIRECT = False SERVER_EMAIL = '[8]root(a)host2.mailhub4u.com' SESSION_CACHE_ALIAS = 'default' SESSION_COOKIE_AGE = 1209600 SESSION_COOKIE_DOMAIN = None SESSION_COOKIE_HTTPONLY = True SESSION_COOKIE_NAME = 'sessionid' SESSION_COOKIE_PATH = '/' SESSION_COOKIE_SECURE = False SESSION_ENGINE = 'django.contrib.sessions.backends.db' SESSION_EXPIRE_AT_BROWSER_CLOSE = False SESSION_FILE_PATH = None SESSION_SAVE_EVERY_REQUEST = False SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer' SETTINGS_MODULE = 'settings' SHORT_DATETIME_FORMAT = 'm/d/Y P' SHORT_DATE_FORMAT = 'm/d/Y' SIGNING_BACKEND = 'django.core.signing.TimestampSigner' SILENCED_SYSTEM_CHECKS = [] SITE_ID = 1 SOCIALACCOUNT_PROVIDERS = {} STATICFILES_DIRS = '()' STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')" STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage' STATIC_ROOT = '/var/lib/mailman3/web/static' STATIC_URL = '/mailman3/static/' TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}] TEST_NON_SERIALIZED_APPS = [] TEST_RUNNER = 'django.test.runner.DiscoverRunner' THOUSAND_SEPARATOR = ',' TIME_FORMAT = 'P' TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M'] TIME_ZONE = 'UTC' USE_ETAGS = False USE_I18N = True USE_L10N = True USE_THOUSAND_SEPARATOR = False USE_TZ = True USE_X_FORWARDED_HOST = True USE_X_FORWARDED_PORT = False WSGI_APPLICATION = 'wsgi.application' X_FRAME_OPTIONS = 'SAMEORIGIN' YEAR_MONTH_FORMAT = 'F Y' References 1. mailto:root@host2.mailhub4u.com 2. mailto:gordon@mailhub4u.com 3. http://localhost/mailman3/hyperkitty/api/mailman/archive?key=0123456789abcd… 4. mailto:members@georgiajets.org 5. mailto:gordon@mailhub4u.com 6. mailto:postorius@host2.mailhub4u.com 7. http://localhost/mailman3/ 8. mailto:root@host2.mailhub4u.com

6 28

All addresses in a bounced list posting seem to be considered as bouncing
by Andy Smith 01 Oct '22

01 Oct '22

Hi, I noticed this happened (bounce.log): Jul 09 12:46:51 2022 (71506) Member xxxxx(a)xxxxx.force9.co.uk already scored a bounce on list users.mailman.bitfolk.com today. Jul 09 12:46:51 2022 (71506) Bounce message for a non subscriber: Email users-leave(a)mailman.bitfolk.com is not a subcriber of users.mailman.bitfolk.com Jul 09 12:46:51 2022 (71506) Member andy(a)bitfolk.com on list users.mailman.bitfolk.com, bounce score = 1. Jul 09 12:46:51 2022 (71506) Bounce message for a non subscriber: Email users(a)mailman.bitfolk.com is not a subcriber of users.mailman.bitfolk.com What happened here is that a reply to one of my posts was rejected by subscriber xxxxx(a)xxxxx.force9.co.uk at smtp time (Exim mainlog): 2022-07-09 12:46:13 1oA9qc-007qOx-0T ** xxxxx(a)xxxxx.force9.co.uk R=dnslookup T=remote_smtp H=mx.avasin.plus.net [212.159.8.200] X=TLS1.2:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256 CV=yes DN="businessCategory=Private Organization,jurisdictionOfIncorporationCountryName=GB,serialNumber=03279013,C=GB,L=Sheffield,O=Plusnet plc,CN=mx.avasin.plus.net": SMTP error from remote mail server after end of data: 552 Spam Message Rejected The message contained my email address (andy(a)bitfolk.com) in quoted text, since it was a reply to a message of mine, and also contained addresses from the list's footer, which accounts for the log messages about users-leave(a)mailman.bitfolk.com and users(a)mailman.bitfolk.com. I'm concerned that Mailman seems to have scored my address as bouncing even though it isn't, merely because it appears inside a message that was rejected. Is that expected? Is there any way to avoid it? I see that Mailman has a bunch of heuristics to parse bounce messages. In this case though, there shouldn't have been a bounce message received to users-bounces@, as this was an SMTP-time rejection. Mailman seems to have parsed addresses out of the actual mail it was sending, rather than only scoring the subscriber address it was trying to send a list posting to. Cheers, Andy

3 11

invalid parameter response of the rest-api
by Jacob Sievert 22 Sep '22

22 Sep '22

Hello, Since a few days we get an error response from our automated membership script. We use the following code to control the membership for some lists: ``` def mailinglist_manage(list_address, members): """ make sure all 'members' are present on the list and no remove everyone else list_address : mail address of mailing list to manage addresses : mail address that are supposed to be on the list """ print("[*] update subscriptions for {}".format(list_address)) ml = mmc.get_list(list_address) # make sure we dont spam users ml.settings['send_welcome_message'] = False ml.settings.save() addresses = set(members.keys()) ml_members = ml.members ml_addresses = set((x.address.__str__() for x in ml_members)) for addr in ml_addresses - addresses: print("unsubscribe {} from {}".format(addr, list_address)) ml.unsubscribe(addr) for addr in addresses - ml_addresses: print("subscribe {} to {}".format(addr, list_address)) ml.subscribe(addr, members[addr], pre_verified=True, pre_confirmed=True, pre_approved=True) ``` Since Tuesday without any update performed by us we get the following error from the script when it's running: ``` Traceback (most recent call last): File "./mail_user_control.py", line 423, in <module> mailinglist_manage("XXX(a)EXAMPLE.COM", members) File "./mail_user_control.py", line 412, in mailinglist_manage pre_verified=True, pre_confirmed=True, pre_approved=True) File "/root/.local/lib/python3.6/site-packages/mailmanclient/restobjects/mailinglist.py", line 339, in subscribe response, content = self._connection.call('members', data) File "/root/.local/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call error_msg, response, None) urllib.error.HTTPError: HTTP Error 400: Invalid Parameter "subscriber": . ``` If you have an idea why that code now returns this. I would be thankful. regards Jacob Sievert

4 7

Re: mail stuck in "out" queue
by Dmitry Makovey 16 Sep '22

16 Sep '22

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

5 12

Message-Footer after migrating to Mailman3
by christopher.claus＠tgcamberg1848.de 13 Sep '22

13 Sep '22

Hello together, I migrated a number of lists from mailman2 to mailman3 using the import functionality. Everthing works fine and the new lists are in use. Currently I´m using the maxking-docker-images with postorius: Mailman Core Version GNU Mailman 3.3.3 (Tom Sawyer) Mailman Core API Version 3.1 Mailman Core Python Version 3.8.5 (default, Feb 19 2021, 09:27:16) [GCC 9.3.0] After this migration I started a refactoring with the list-styles specially with message-footers. Already the old lists in mailman2 were confused in some configuration details an I was clear to import this "misconfiguration" and clean it up afterwards. But this was much more difficult than expected: some lists are using a footer (from the mailman2-config) and there is no record in the template-section visible. Further I found no configuration item which points to this "old" footer. I found the file [code] /usr/lib/python3.8/site-packages/mailman/templates/de/list:member:generic:footer.txt [/code] which includes the old footer: [code] _______________________________________________ $display_name Mailingliste -- $listname Zur Abmeldung von dieser Mailingliste senden Sie eine Nachricht an ${short_listname}-leave@${domain} [/code] but no configuration item to disable this behaviour. I created a new template list:member:regular:footer for the list to override the footer. But the old footer is still used - in detail: the resulting message sent to all subcribers of the list contains only the footer and the content sent to the list is an attachement of that "footer-message". This is the basic reason why I want to change it. In the logfiles I found an error and a warning: [code] ERROR 2021-05-24 16:22:22,895 1648 django.security.DisallowedHost Invalid HTTP_HOST header: 'tgc_mailman_web:8000'. The domain name provided is not valid according to RFC 1034/1035. WARNING 2021-05-24 16:22:22,896 1648 django.request Bad Request: /postorius/api/templates/list/homepage.gruppe.tgcamberg1848.de/list:member:regular:footer [/code] Obviously I have a problem with the hostname. I set [code] POSTORIUS_TEMPLATE_BASE_URL=http://tgc_mailman_web:8000 [/code] where tgc_mailman_web is my container name. This URL ist stored in the DB-table "template" for the name=list:member:regular:footer. A call to my fqdn or to localhost [code] wget http://localhost:8000/postorius/api/templates/list/homepage.gruppe.tgcamber… [/code] will return the footer itself - this works fine. Therefore, should i set POSTORIUS_TEMPLATE_BASE_URL to http://localhost:8000? I think, resolving this issue will use the newly configured footer. But I started with the intention to remove alle footers due to the attachement situation described above. Does anybody have an idea how to change the configuration of my list? Are these problems probably related to the import? Currently we are using more then 25 lists with a lot of users. Manually recreating all lists is not possible an could only be a test for a single list. Best regards, chrclaus

6 18

FYI: Django security releases issued: 4.0.6 and 3.2.14
by Guillermo Hernandez (Oldno7) 29 Aug '22

29 Aug '22

https://www.djangoproject.com/weblog/2022/jul/04/security-releases/ Django security releases issued: 4.0.6 and 3.2.14 Posted by *Mariusz Felisiak* on Julio 4, 2022 In accordance with our security release policy <https://docs.djangoproject.com/en/dev/internals/security/>, the Django team is issuing Django 4.0.6 <https://docs.djangoproject.com/en/dev/releases/4.0.6/> and Django 3.2.14 <https://docs.djangoproject.com/en/dev/releases/3.2.14/>. These release addresses the security issue detailed below. We encourage all users of Django to upgrade as soon as possible. CVE-2022-34265: Potential SQL injection via Trunc(kind) and Extract(lookup_name) arguments Trunc() and Extract() database functions were subject to SQL injection if untrusted data was used as a kind/lookup_name value. Applications that constrain the lookup name and kind choice to a known safe list are unaffected. This security release mitigates the issue, but we have identified improvements to the Database API methods related to date extract and truncate that would be beneficial to add to Django 4.1 before it's final release. This will impact 3rd party database backends using Django 4.1 release candidate 1 or newer, until they are able to update to the API changes. We apologize for the inconvenience. Thanks Takuto Yoshikai (Aeye Security Lab) for the report. This issue has severity "high" according to the Django security policy. -- ___________________________________________ Mailman's content filtering has removed the following MIME parts from this message. Content-Type: image/png Name: firma-GHP-emails.png Replaced multipart/alternative part with first alternative.

5 18

Using REST with CLI using curl .... help needed
by Krešimir Mihalj 27 Aug '22

27 Aug '22

Hi ! I need simple add/delete member to list using rest ..... but all documentation is for python and not helpfull for me, .... and I need these in CLI using curl command .... To add member using CLI i tried this command: curl -d "{'list_id' : 'list.domain.com', 'subscriber' : 'user(a)email.com'}" -H 'Content-Type: application/json' --user restadmin:restpass http://localhost:8001/3.0/members and I got this response: Missing parameters: list_id, subscriber What is wrong in above command here ? ..... I need to add user: user(a)email.com to list: list(a)domain.com Also, if someone is willing to give example: delete user: user(a)email.com from list: list(a)domain.com I actually create PHP application .... but CLI example is easy to reproduce in PHP function AddUserToList($user,$lista) { $url = 'http://llocalhost:8001/3.0'; $request_url = $url . '/members'; $data = array("list_id" => $lista,"subscriber" => $user ); $postdata = json_encode($data); $ch = curl_init($request_url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_USERPWD, "restadmin:restpass"); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json')); $result = curl_exec($ch); curl_close($ch); print_r ($result); } AddUserToList('user(a)email.com','list.domain.com');

6 8

Mailman 3 questions (purge archives, list metrics, and auto discard held messages)
by bob B 02 Aug '22

02 Aug '22

Some quick mailman3 questions --Just wanted to check that there is still no way for mailman3 to automatically purge archived emails after a set amount of time? it looks like you have to export the database, purge the messages, then re-import, is this still the only/best way to purge old archived messages? --I noticed on the list info page you have metrics, which is straightforward except for "volume" My volume number is not increasing? What is that counting exactly? --Is there any way to auto discard held messages after x amount of time? I assume not but had a user ask if that was an option. Thanks for the assistance in advance.

3 4

Programmatic Creation of Users
by jciesla＠techsupportme.com 30 Jul '22

30 Jul '22

I am returning to Mailman after being away for a LONG time and I need to create 40 lists with ~350 subscribers spread across those lists. After a lot of testing I am now able to create users from the Core Shell and set (most of) the necessary values. Having jumped that hurdle I have hit a roadblock and I am hoping that the list can provide some insight and/or solutions. After a lot of research I now understand that what I am doing in the Core Shell is creating a User entity which has one or more email addresses as well as one or more roles and preferences that exist at the global level. This user entity can Subscribe to zero or more lists. If I am incorrect in the summation please let me know! Assuming that the previous information is correct that brings us to my current dilemma. Specifically, the User entity can Subscribe and interact with lists using the email commands, but they can't Login using the Postorius web interface. If I understand correctly the reason that they can't login is that the underlying Django system does not recognize their account because the account that was created in Core isn't accessible by Django. Instead the user needs to create a Django-specific account at which point the information from Core becomes available in the user's web UI. If that is in fact accurate then I want to take the next logical leap: is it possible to programmatically create an account in Django that is tied to the account that was created in Core? Any information is greatly appreciated! -John

2 1