Django Internal Server Error on held_messages
by Hasan Berkay Çağır
Hello,
We've been running a Mailman3 instance with Debian 10's official package:
Mailman Core Version: GNU Mailman 3.2.1 (La Villa Strangiato)
Mailman Core API Version: 3.0
Mailman Core Python Version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
For some reason, and only on some lists, when we try to see held messages on Postorius, we get an error message "An error occurred while processing your request." and the following error email:
"""
Internal Server Error: /postorius/lists/list.example.com/held_messages
KeyError at /postorius/lists/list.example.com/held_messages
'file'
Request Method: GET
Request URL: https://mailman.example.com/postorius/lists/list.example.com/held_messages
Django Version: 1.11.29
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.7.3
Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages']
Server time: Thu, 24 Mar 2022 08:39:10 +0000
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/postorius/auth/decorators.py" in wrapper
58. return fn(*args, **kwargs)
File "/usr/lib/python3/dist-packages/postorius/views/list.py" in list_moderation
520. paginator_class=MailmanPaginator)
File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in paginate
72. objects = paginator.page(page_num)
File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in page
48. result = self.function(count=self.per_page, page=number)
File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py" in get_held_page
134. return Page(self._connection, url, HeldMessage, count, page)
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/page.py" in __init__
37. self._create_page()
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/page.py" in _create_page
62. response, content = self._connection.call(self._build_url())
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call
99. raise HTTPError(url, response.status, content, response, None)
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/usr/lib/python3/dist-packages/postorius/middleware.py" in __call__
39. return self.get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
43. response = response_for_exception(request, exc)
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in response_for_exception
93. response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in handle_uncaught_exception
135. extra={'status_code': 500, 'request': request},
File "/usr/lib/python3.7/logging/__init__.py" in error
1412. self._log(ERROR, msg, args, **kwargs)
File "/usr/lib/python3.7/logging/__init__.py" in _log
1519. self.handle(record)
File "/usr/lib/python3.7/logging/__init__.py" in handle
1529. self.callHandlers(record)
File "/usr/lib/python3.7/logging/__init__.py" in callHandlers
1591. hdlr.handle(record)
File "/usr/lib/python3.7/logging/__init__.py" in handle
905. self.emit(record)
File "/usr/lib/python3/dist-packages/django/utils/log.py" in emit
118. reporter = ExceptionReporter(request, is_email=True, *exc_info)
File "/usr/lib/python3/dist-packages/django/views/debug.py" in __init__
246. self.template_info = getattr(self.exc_value, 'template_debug', None)
File "/usr/lib/python3.7/tempfile.py" in __getattr__
614. file = self.__dict__['file']
Exception Type: KeyError at /postorius/lists/list.example.com/held_messages
Exception Value: 'file'
Request information:
USER: root
GET: No GET data
POST: No POST data
FILES: No FILES data
COOKIES:
rl_anonymous_id = '*'
rl_user_id = '*'
rl_group_id = '*'
rl_trait = '*'
rl_group_trait = '*'
rl_page_init_referrer = '*'
rl_page_init_referring_domain = '*'
csrftoken = '*'
sessionid = '*'
META:
CONTENT_LENGTH = ''
CONTENT_TYPE = ''
CSRF_COOKIE = '*'
DOCUMENT_ROOT = '/etc/nginx/html'
HTTPS = 'on'
HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, br'
HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.5'
HTTP_CACHE_CONTROL = 'max-age=0'
HTTP_COOKIE = '*'
HTTP_DNT = '1'
HTTP_HOST = 'mailman.example.com'
HTTP_REFERER = 'https://mailman.example.com/postorius/lists/list.example.com/'
HTTP_SEC_FETCH_DEST = 'document'
HTTP_SEC_FETCH_MODE = 'navigate'
HTTP_SEC_FETCH_SITE = 'same-origin'
HTTP_TE = 'trailers'
HTTP_UPGRADE_INSECURE_REQUESTS = '1'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:98.0) Gecko/20100101 Firefox/98.0'
PATH_INFO = '/postorius/lists/list.example.com/held_messages'
QUERY_STRING = ''
REMOTE_ADDR = '::1'
REMOTE_PORT = '58290'
REQUEST_METHOD = 'GET'
REQUEST_SCHEME = 'https'
REQUEST_URI = '/postorius/lists/list.example.com/held_messages'
SCRIPT_NAME = ''
SERVER_NAME = 'mailman.example.com'
SERVER_PORT = '443'
SERVER_PROTOCOL = 'HTTP/2.0'
uwsgi.core = 0
uwsgi.node = b'example.com'
uwsgi.version = b'2.0.18-debian'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.file_wrapper = ''
wsgi.input = <uwsgi._Input object at 0x7f8f7a6bf870>
wsgi.multiprocess = False
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', 'admin(a)example.com'),)"
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/usr/share/mailman3-web'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor'
COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter']
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f8f7b50d598>
COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor'
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = True
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/var/lib/mailman3/web/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_URL = '/mailman3/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4'}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'postorius(a)example.com'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAILNAME = 'example.com'
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}}
HOSTNAME = 'localhost.local'
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')"
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')"
MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT_DAYS = '********************'
POSTORIUS_TEMPLATE_BASE_URL = 'https://mailman.example.com/'
PREPEND_WWW = False
Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5}
ROOT_URLCONF = 'urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = None
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'root(a)example.com'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 1
SOCIALACCOUNT_PROVIDERS = {}
STATICFILES_DIRS = '()'
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/lib/mailman3/web/static'
STATIC_URL = '/mailman3/static/'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_ETAGS = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = True
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'
"""
I've tried to debug this issue as much as I can but couldn't figure out anything. I wonder if anyone has encountered something like this or might have a clue about what's the problem here?
Thanks in advance,
Berkay
3 years, 2 months
Re: Moderate mails from owner
by Malcolm Austen
On Thu, 21 Mar 2019 16:58:45 -0000, Abhilash Raj <maxking(a)asynchronous.in>
wrote:
> On Thu, Mar 21, 2019, at 8:47 AM, Malcolm Austen wrote:
>> On Wed, 20 Mar 2019 18:42:10 -0000, Mark Sapiro <mark(a)msapiro.net>
>> wrote:
>>
>> > However, this is apparently not relevant because both the Member and
>> > Non-Member actions are Hold. Thus, the post should be held unless it
>> has
>> > an Approved: header with the list's moderator_password attribute if
>> > there is one, or it matches a Header filter rule with an accept
>> action.
>>
>> At the risk of getting this really wrong - I have only limited MM3
>> knowledge - I will ask what is, I fancy, a closely related question ...
>>
>> My only experience of MM3 is with Ancestry's 'RootsWeb' installation.
>> From
>> the emails I can see a header "X-Mailman-Version: 3.1.0" but I can't
>> find
>> anything to identify Postorius which is the only interface I have to the
>> system.
>>
>> From symptoms seen and described by others, I have deduced that the
>> Postorius admin-per-user moderation options of "Default processing" and
>> "List default" mean that a user may be set (by default on the rootsweb
>> system!) to use the system default moderation settings rather than the
>> list default settings.
>
> "Default Processing" means the default pipeline defined in
Thanks, I will do some (more!) reading.
> List > Settings > Alter Messages > 'Pipeline' ( in the bottom). This
> pipeline is
> system wide and is defined in source code, so it can't be customized by
> site administrator.
Unfortunately, that is not available to list-owners on the system
concerned. Ancestry must have removed it.
Malcolm.
--
Malcolm Austen <malcolm.austen(a)weald.org.uk>
6 years, 2 months
Re: Templates
by Mark Sapiro
On 11/17/20 1:23 AM, Odhiambo Washington wrote:
> On Tue, 17 Nov 2020 at 06:09, Mark Sapiro <mark(a)msapiro.net> wrote:
>
>>
>> How did you create this footer?. If you created it by creating a file in
>> Mailman's var/templates directiory, can Mailman read it?
>>
>>
> I went into Postorius and selected the list. It's the only one.
> Selected Templates -> New Template.
And previously
> requests.exceptions.HTTPError: 404 Client Error: Not Found for url:
> http://localhost:8000/mm3/mailman3/api/templates/list/kictanet-2.lists.kict…
...
> Message: 'Cannot retrieve template at
> http://localhost:8000/mm3/mailman3/api/templates/list/kictanet-2.lists.kict…
> (<no authorization>)'
This is Mailman core trying to get the template from Postorius. What
happens if you try to get that URL in a browser? What if you try your
domain instead of localhost:8000?
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
4 years, 7 months
Re: list footer web customization procedure
by Mark Sapiro
On 11/30/21 1:11 AM, Stephen J. Turnbull wrote:
>
> Does Django empty out the "spacetebasket" if it's all white space, or
> does spurious whitespace in that box lead to an (apparently) empty
> footer being added?
Postorius used to remove leading and trailing whitespace from the
template, but not since
https://gitlab.com/mailman/postorius/-/issues/301 was fixed.
> Anway, I recommend to the OP that they make sure there's no whitespace
> in the Data box.
A good recommendation. While a header or footer template with some
whitespace might not be noticeable when added to a text/plain message,
it will cause extra mime parts to be added to more complex messages.
Also, if the goal in setting an empty footer is to avoid breaking a DKIM
signature, even adding a bit of whitespace to a text/plain message is an
issue.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
3 years, 6 months
Mailman 3 Web Application Startup Time or Availability - 502 Bad Gateway and Postorius
by summersan＠nclack.k12.or.us
Hello,
I've been setting up a new Mailman 3 installation earlier this year, and I think things have been going okay so far.
I've noticed, however, that after a long amount of time passes (for example, a weekend), when I try to visit the local Mailman 3 web application (e.g., example.mailman.site/mailman3/lists/, where it says "Postorius" at the top left corner of the page and shows a list of mailing lists in the body of the page), I sometimes get a 502 Bad Gateway type of error (referenced from my notes). As far as I can tell, this looks like a normal web server (nginx?) error. If I try visiting the Mailman 3 server location again in a web browser shortly thereafter, I still might get a 502 Bad Gateway error. If I wait a while longer (maybe a minute or more?), the Mailman 3 web application appears to load as expected.
I've tried looking at the gunicorn documentation (https://docs.gunicorn.org/en/stable/index.html) and searching online (including in this Mailman3 mailing list) to see if there's something that can be done to fix this issue. The closest thing I can find for now is advice for something that, while possibly being technologically unrelated, includes the idea of "warmup requests", which makes me think of simply setting up some sort of script to periodically load a page on the Mailman 3 web application in order to keep it more available for visitors (https://stackoverflow.com/a/57056170)
Is there a setting or something in any of the Mailman 3 related programs that could help with keeping the Mailman 3 web application up and running more often without having to resort to making an ongoing, automated request to the Mailman 3 web application?
Thank you.
* Note: "Mailman 3" terminology is referenced from this page: https://lists.mailman3.org/archives/
4 years, 1 month
Re: Error -- postorius Un-handled exception
by Mark
> On Sat, Jan 27, 2024 at 8:17 AM Mark <imark(a)posteo.net> wrote:
>> 3. For the user "mailman" to access the databases I needed to change
>> the
>> owner to mailman (even though all privileges has been granted) ...
-------------------------------------------------------------------------------------------------------
On 2024-01-27 18:51, Odhiambo Washington wrote:
>
> The issue you're dealing with in that case is this change in
> PostgreSQL-15
> discussed in this blog:
> https://www.cybertec-postgresql.com/en/error-permission-denied-schema-publi…
>
> I have never needed to do `psql -d mailman -U mailman -W`.
> I used the steps documented above to enable mailman user to write to
> the
> database.
>
> postgres=# \c mailmanweb postgres
> You are now connected to database "mailmanweb" as user "postgres".
> mailmanweb=# GRANT ALL ON SCHEMA public TO mailman;
> GRANT
> mailmanweb=# \c mailman postgres
> You are now connected to database "mailman" as user "postgres".
> mailmanweb=# GRANT ALL ON SCHEMA public TO mailman;
> GRANT
>
> TBH, I am not sure which method is the best although I think this
> option is
> :)
>
-------------------------------------------------------------------------------------------------------
Thanks for the heads up on that blog Odhiambo.
Agreed. it is a better way of doing it.
--- SNIP ---
In PostgreSQL 14 and in prior versions, by default anybody can create a
table. The new table will simply end up in the PUBLIC schema. The
problem with this approach is that the public schema can quickly turn
into a trash can containing all kinds of used and unused tables – which
is neither desirable nor recommended. An even bigger problem is
security: almost any privilege escalation attack on a PostgreSQL
database requires the attacker to create malicious objects. ... The
public schema is the perfect vector for such a privilege escalation
attack. Therefore PostgreSQL has made a major leap forward and changed
this behavior. In version 15, only the database owner can create objects
in the public schema.
---
1 year, 4 months
Re: Template path
by Seth Seeger
> On Sep 17, 2020, at 12:54 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
>
>> I followed previous emails on this list which told me to set POSTORIUS_TEMPLATE_BASE_URL in /opt/mailman/web/settings_local.py, but that doesn’t seem to make any difference. (I set it to a different value and the error message didn’t change. I also tried setting it in /opt/mailman/core/settings_local.py to no effect. After each change, I restarted mailman-web and mailman-core containers.)
>
>
> How did you create the template? In Postorius or by manually creating a
> file somewhere? If by creating a file, POSTORIUS_TEMPLATE_BASE_URL is
> not relevant, but ensure that the file is findable and readable by Mailman.
I created the template in Postorius.
>> From within mailman-web and mailman-core containers, I can reach “mailman-web:8000”.
>>
>> In the webpage, I see: Mailman REST API not available. Please start Mailman core.
>
>
> The REST APT is normally exposed on port 8001, not 8000 (See `port` in
> `mailman conf -s webservice`.
Yes, it’s 8001:
[webservice] hostname: 172.19.199.2
[webservice] port: 8001
> What is your setting for MAILMAN_REST_API_URL? does the port match?
I’m not overriding it:
./postorius/mailman-web/settings.py:MAILMAN_REST_API_URL = os.environ.get('MAILMAN_REST_URL', 'http://mailman-core:8001')
>
>> In /opt/mailman/core/var/logs/mailman.log, I get:
>>
>> [17/Sep/2020:16:23:29 +0000] "GET /3.1/lists/xxx.example.com HTTP/1.1" 200 416 "-" "GNU Mailman REST client v3.3.1"
>
> It appears you are connecting to the REST API, but ...
> ...
>> urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
>
>
> Something appears to be trying to connect to REST on port 8000
Is this a red herring? Is something else wrong? Setting POSTORIUS_TEMPLATE_BASE_URL in settings_local.py doesn’t seem to make any difference.
Having set the POSTORIUS_TEMPLATE_BASE_URL, I deleted the template and recreated it. Now, when I approve a pending subscription it goes through, but the welcome email is blank.
Seth
4 years, 9 months
Re: Footer from templates not working - DB issue?
by Mark Sapiro
On 9/25/22 05:26, mailman--- via Mailman-users wrote:
> Dear All,
> I have a similar issue. Also on Debian 11. Also installed as Debian PKG. It's version GNU Mailman 3.3.3 (Tom Sawyer)
> I am using sqlite3. When I do the SQL-query I get an URI which is in multiple reasons wrong
> http://localhost/mailman3/postorius/api/templates/list/....../list:member:r…
The initial part of that URL is set from the Django setting
`POSTORIUS_TEMPLATE_BASE_URL`. If that is set to
`http://localhost/mailman3/`, it should probably be just
`http://localhost/`.
> OK - the hostname could be changed but the path is not existing at all.
> There is no file "list:member:regular:footer" on the complete filesystem.
It's not a file. It refers to an entry in the postorius_emailtemplate
table in the database which is retrieved via the Postorius API.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
2 years, 8 months
Re: Mailman, etc. upgrade woes and persistent bugs
by Stephen J. Turnbull
Brian Carpenter writes:
> I think treating all disabled options the same is short-sighted. They
> are all not the same.
>
> -- Some list members will disable their subscription for good reasons
> and will be really upset if a List Owner renables it through ignorance.
> So it is important to know a disable member is done by the member
> itself.
This is true in principle, but (1) I don't think that the user cares;
if they're looking at it, they know whether they want enabled or
disabled, how it got that way doesn't matter to them, and (2) I can't
see why an admin would disable delivery except on request from a user,
so I can't see why an admin would reenable except on request from a
user. The only case I can see would be a mass reenabling, but that's
not going to happen in the future (I hope).
> -- Some mailing lists will have multiple list owners managing them. So
> it is important for one List Owner to know when a subscription has been
> disabled by the actions of another List Owner.
I don't see why, see above.
> -- A List Owner may not know that some of his List Members are bouncing
> messages for various reasons, so reviewing their Membership roster, they
> see that they have some list members disabled due to bounces and can
> then address those particular problematic members.
This query is an important use case. But it's the only one, I think,
unless you really have List Owners arbitrarily disabling members'
subscriptions. And if you're thinking about reenabling from that
page, I think you need a lot more information. For example, if the
admin disabled, you need to know if that was a user request or
something else (what?). If bounce disabled, you want to know what the
bounce was ("no such mailbox"? probably not a good candidate for
reenabling), and when (5 minutes ago? ditto). I guess you could just
reenable and see what happens, but that could be risky (eg, sending
mail to non-existent users is frowned upon by some providers).
> @Abhilash, I highly recommend that you contact me off list about getting
> access to Affinity so you can see what I am talking about. I would love
> to show you what I have done for Member Management. I offered the same
> to Steve but he never was interested in taking a look.
It's not that I lack interest, it's that life got in the way.
Unfortunately, until I get enough time (man-weeks, I've never worked
on Postorius and very little in Django) to work on Postorius, or
somebody else starts to do it, a look at Affinity is low priority.
Also, it's pretty clear that a quick look isn't going to be very
helpful. The Mailman developers know what Mailman 2 looks like. I
think the benefits to a hands-on admin are pretty obvious vs the
current Postorius, as are Web 2.0 improvments like sorting on the
options. The more subtle improvements you've made are going to
require a guided tour and/or some study to identify and understand.
Aside: I have to assume that Postorius is aimed at the kind of
subscriber that most of us are, and that list administration was
something of an afterthought, and assumed to be mostly hands-off.
That's the only rationale I can come up with for the design where
list admins need to go to the individual pages to see user options --
it was easier to reuse the user option page and just give the admin
permission to access and change it, than to provide a (sortable) list
with user details.
Steve
4 years, 4 months
Re: Error -- postorius Un-handled exception
by Odhiambo Washington
On Sat, Jan 27, 2024 at 8:17 AM Mark <imark(a)posteo.net> wrote:
>
> -----------------------
> > >
> >
> "/opt/mailman/venv/lib/python3.11/site-packages/mailmanclient/restbase/connection.py",
> > > line 160, in call
> > > raise HTTPError(params.get('url'), response.status_code,
> > > urllib.error.HTTPError: HTTP Error 500: {"title": "500 Internal
> > Server
> > > Error"}
> > > ERROR 2024-01-24 05:41:46,333 51799 django.request Internal Server
> > > Error: /postorius/lists/testing.example.com/
>
> ---------------------------
>
> > Try running all the system initializations with "mailman info" and
> > "mailman conf". This often will elicit errors if the system is messed
> > up. More likely to get this error would be "mailman withlist $LIST"
> > (maybe you need an '-l' option to specify the list, I forget). If
> > that doesn't elicit the error, then check that Mailman's runners are
> > running with ps. Since it's Debian, I guess they should be running as
> > the 'list' user. Are Mailman's files and directories owned by 'list'
> > or sometimes 'www-data' for log files?
>
> Debian yes, but venv/pip installation so running as "mailman".
>
> >
> > The log is saying that when Postorius tried to look up that list, it
> > tried to call out to Mailman core via HTTP to the API to get the
> > information and that access failed with HTTP status 500. I think the
> > actual error is likely to be reported in mailman.log (or it might be
> > in error.log, "grep '05:41:46' *.log" in Mailman's log directory will
> > probably find it). I'm pretty sure there will be relevant log
> > entries, probably a matching traceback.
>
> ----------------------------
>
> Hi,
>
> Thank you Steve, Mark, Odhiambo for your responses.
>
> After working my way through your suggestions with no joy I discovered
> to my embarrassment that I hadn't created the "mailman-hyperkitty.cfg"
> file. Once created all was well.
>
> This was of course after I had begun the installation process again from
> scratch. My apologies for the waste of bandwidth and your time.
>
> Regarding my installation, (following the docs at
> https://docs.mailman3.org/en/latest/install/virtualenv.html) I found a
> couple of issues that others might also stumble over:
>
> 1. Installing Xapian needed clang. It didn't like gcc
> 2. Being a fresh Debian 12 OS, I needed to also install make,
> build-essential
> 3. For the user "mailman" to access the databases I needed to change the
> owner to mailman (even though all privileges has been granted) ...
postgres=# create database mailman;
> CREATE DATABASE
> postgres=# create database mailmanweb;
> CREATE DATABASE
> postgres=# create user mailman with encrypted password '<PASSWORD>';
> CREATE ROLE
> postgres=# grant all privileges on database mailman to mailman;
> GRANT
> postgres=# grant all privileges on database mailmanweb to mailman;
> GRANT
>
> psql -d mailman -U mailman -W
> $ psql -d mailman -U mailman -W
> Password:
>
> psql: error: connection to server on socket
> "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: Peer authentication
> failed for user "mailman"
>
> sudo -u postgres psql
> ALTER DATABASE mailman OWNER TO mailman;
> \q
>
> $ psql -h localhost -d mailman -U mailman -W
> Password:
>
> psql (15.5 (Debian 15.5-0+deb12u1))
> SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384,
> compression: off)
> Type "help" for help.
>
> ... and same for mailmanweb
>
> So far, so good.
>
> Thanks and apologies once again,
> Mark
>
The issue you're dealing with in that case is this change in PostgreSQL-15
discussed in this blog:
https://www.cybertec-postgresql.com/en/error-permission-denied-schema-publi…
I have never needed to do `psql -d mailman -U mailman -W`.
I used the steps documented above to enable mailman user to write to the
database.
postgres=# \c mailmanweb postgres
You are now connected to database "mailmanweb" as user "postgres".
mailmanweb=# GRANT ALL ON SCHEMA public TO mailman;
GRANT
mailmanweb=# \c mailman postgres
You are now connected to database "mailman" as user "postgres".
mailmanweb=# GRANT ALL ON SCHEMA public TO mailman;
GRANT
TBH, I am not sure which method is the best although I think this option is
:)
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
1 year, 4 months