Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
My mailman3 installation has now been up and running successfully for several weeks without any issues and then the following error message was emailed to me by my server. What's up with this error message and what do I need to do? Thanks, Gordon
-------- Forwarded Message --------
Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error:
/mailman3/hyperkitty/api/mailman/archive
Date: Fri, 20 Nov 2020 01:25:41 -0000
From: [1]root@host2.mailhub4u.com
To: [2]gordon@mailhub4u.com
Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
OperationalError at /hyperkitty/api/mailman/archive
(1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for
column mailman3web.hyperkitty_email.content at row 1")
Request Method: POST
Request URL:
[3]http://localhost/mailman3/hyperkitty/api/mailman/archive?key=0123456
789abcdef0123456789ABCDEF
Django Version: 1.11.29
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.7.3
Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7',
'/usr/lib/python3.7/lib-dynload',
'/usr/local/lib/python3.7/dist-packages',
'/usr/lib/python3/dist-packages']
Server time: Fri, 20 Nov 2020 01:25:40 +0000
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in
execute
64. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py"
in execute
101. return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
250. self.errorhandler(self, exc, value)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in
defaulterrorhandler
50. raise errorvalue
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
247. res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query
411. rowcount = self._do_query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query
374. db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query
292. _mysql.connection.query(self, query)
The above exception ((1366, "Incorrect string value:
'\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column
mailman3web.hyperkitty_email.content at row 1")) was the direct
cause of the following exception:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py"
in inner
41. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in
_get_response
187. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in
_get_response
185. response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "/usr/lib/python3/dist-packages/django/views/decorators/http.py"
in inner
40. return func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in
_decorator
69. return func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py"
in wrapped_view
58. return view_func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in
archive
115. add_to_list(mlist_fqdn, msg)
File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in
add_to_list
151. email.save()
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save
808. force_update=force_update, update_fields=update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in
save_base
838. updated = self._save_table(raw, cls, force_insert, force_update,
using, update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in
_save_table
924. result = self._do_insert(cls._base_manager, using, fields,
update_pk, raw)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in
_do_insert
963. using=using, raw=raw)
File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in
manager_method
85. return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py" in
_insert
1079. return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py"
in execute_sql
1112. cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in
execute
64. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__
94. six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python3/dist-packages/django/utils/six.py" in reraise
685. raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in
execute
64. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py"
in execute
101. return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
250. self.errorhandler(self, exc, value)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in
defaulterrorhandler
50. raise errorvalue
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
247. res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query
411. rowcount = self._do_query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query
374. db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query
292. _mysql.connection.query(self, query)
Exception Type: OperationalError at /hyperkitty/api/mailman/archive
Exception Value: (1366, "Incorrect string value:
'\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column
mailman3web.hyperkitty_email.content at row 1")
Request information:
USER: AnonymousUser
GET:
key = '0123456789abcdef0123456789ABCDEF'
POST:
mlist = '[4]members@georgiajets.org'
FILES:
message = <InMemoryUploadedFile: message.txt ()>
COOKIES: No cookie data
META:
CONTENT_LENGTH = '7720'
CONTENT_TYPE = 'multipart/form-data;
boundary=8da553dce5f76815f62f12a1b8c0efac'
CONTEXT_DOCUMENT_ROOT = '/var/www/html'
CONTEXT_PREFIX = ''
DOCUMENT_ROOT = '/var/www/html'
GATEWAY_INTERFACE = 'CGI/1.1'
HTTP_ACCEPT = '*/*'
HTTP_ACCEPT_ENCODING = 'gzip, deflate'
HTTP_CONNECTION = 'keep-alive'
HTTP_HOST = 'localhost'
HTTP_USER_AGENT = 'python-requests/2.21.0'
PATH = '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
PATH_INFO = '/hyperkitty/api/mailman/archive'
QUERY_STRING = 'key=0123456789abcdef0123456789ABCDEF'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '53006'
REQUEST_METHOD = 'POST'
REQUEST_SCHEME = 'http'
REQUEST_URI =
'/mailman3/hyperkitty/api/mailman/archive?key=0123456789abcdef012345678
9ABCDEF'
SCRIPT_FILENAME =
'proxy:uwsgi://localhost//hyperkitty/api/mailman/archive'
SCRIPT_NAME = '/mailman3'
SERVER_ADDR = '127.0.0.1'
SERVER_ADMIN = 'webmaster@localhost'
SERVER_NAME = 'localhost'
SERVER_PORT = '80'
SERVER_PROTOCOL = 'HTTP/1.1'
SERVER_SIGNATURE = '<address>Apache/2.4.38 (Debian) Server at localhost
Port 80</address>\n'
SERVER_SOFTWARE = 'Apache/2.4.38 (Debian)'
uwsgi.core = 0
uwsgi.node = b'slc.mailhub4u.com'
uwsgi.version = b'2.0.18-debian'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.file_wrapper = ''
wsgi.input = <uwsgi._Input object at 0x7f6a278df600>
wsgi.multiprocess = False
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version = '(1, 0)'
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', '[5]gordon@mailhub4u.com'),)"
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS =
"('django.contrib.auth.backends.ModelBackend',
'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/usr/share/mailman3-web'
CACHES = {'default': {'BACKEND':
'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor'
COMPRESS_CSS_FILTERS =
['compressor.filters.css_default.CssAbsoluteFilter']
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_JINJA2_GET_ENVIRONMENT = <function
CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f6a2d586620>
COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor'
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = True
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'),
('text/x-scss', 'sassc -t compressed {infile} {outfile}'),
('text/x-sass', 'sassc -t compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/var/lib/mailman3/web/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_URL = '/mailman3/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME':
'mailman3web', 'USER': 'mailman3web', 'PASSWORD':
'********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS':
{'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"},
'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0,
'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME':
None, 'MIRROR': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f',
'%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y
%H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S',
'%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y',
'%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B
%Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER =
'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = '[6]postorius@host2.mailhub4u.com'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAILNAME = 'host2.mailhub4u.com'
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS =
['django.core.files.uploadhandler.MemoryFileUploadHandler',
'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'ENGINE':
'haystack.backends.whoosh_backend.WhooshEngine', 'PATH':
'/var/lib/mailman3/web/fulltext_index'}}
HOSTNAME = 'localhost.local'
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3',
'django.contrib.admin', 'django.contrib.auth',
'django.contrib.contenttypes', 'django.contrib.sessions',
'django.contrib.sites', 'django.contrib.messages',
'django.contrib.staticfiles', 'rest_framework', 'django_gravatar',
'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth',
'allauth.account', 'allauth.socialaccount')"
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast',
'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be',
'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'),
('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'),
('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en',
'English'), ('en-au', 'Australian English'), ('en-gb', 'British
English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar',
'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx',
'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve',
'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa',
'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'),
('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he',
'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper
Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id',
'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'),
('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km',
'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'),
('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml',
'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my',
'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl',
'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa',
'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br',
'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk',
'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'),
('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'),
('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'),
('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur',
'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'),
('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters':
{'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}},
'handlers': {'mail_admins': {'level': 'ERROR', 'filters':
['require_debug_false'], 'class':
'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO',
'class': 'logging.handlers.RotatingFileHandler', 'filename':
'/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'},
'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}},
'loggers': {'django.request': {'handlers': ['mail_admins', 'file'],
'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'],
'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers':
['file'], 'level': 'INFO', 'propagate': True}, 'postorius':
{'handlers': ['file'], 'level': 'INFO', 'propagate': True}},
'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s
%(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s
%(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE =
'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')"
MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT_DAYS = '********************'
POSTORIUS_TEMPLATE_BASE_URL = '[7]http://localhost/mailman3/'
PREPEND_WWW = False
Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default',
'poll': 5}
ROOT_URLCONF = 'urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = None
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = '[8]root@host2.mailhub4u.com'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER =
'django.contrib.sessions.serializers.PickleSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 1
SOCIALACCOUNT_PROVIDERS = {}
STATICFILES_DIRS = '()'
STATICFILES_FINDERS =
"('django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE =
'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/lib/mailman3/web/static'
STATIC_URL = '/mailman3/static/'
TEMPLATES = [{'BACKEND':
'django.template.backends.django.DjangoTemplates', 'DIRS': [],
'APP_DIRS': True, 'OPTIONS': {'context_processors':
['django.template.context_processors.debug',
'django.template.context_processors.i18n',
'django.template.context_processors.media',
'django.template.context_processors.static',
'django.template.context_processors.tz',
'django.template.context_processors.csrf',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
'django_mailman3.context_processors.common',
'hyperkitty.context_processors.common',
'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_ETAGS = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = True
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'
References
- mailto:root@host2.mailhub4u.com
- mailto:gordon@mailhub4u.com
- http://localhost/mailman3/hyperkitty/api/mailman/archive?key=0123456789abcde...
- mailto:members@georgiajets.org
- mailto:gordon@mailhub4u.com
- mailto:postorius@host2.mailhub4u.com
- http://localhost/mailman3/
- mailto:root@host2.mailhub4u.com
On 11/20/20 8:24 AM, Gordon Dickens wrote:
My mailman3 installation has now been up and running successfully for several weeks without any issues and then the following error message was emailed to me by my server. What's up with this error message and what do I need to do? Thanks, Gordon
-------- Forwarded Message -------- Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Date: Fri, 20 Nov 2020 01:25:41 -0000 From: [1]root@host2.mailhub4u.com To: [2]gordon@mailhub4u.com Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\\xF0\\x9F\\xA4\\xA3\\xF0\\x9F...' for column
mailman3web.hyperkitty_email.contentat row 1")
'\xF0\x9F\xA4\xA3' is a utf-8 encoding for the emoji '🤣'. For MySQL and MariaDB, 4-byte encodings require the database allow utf8mb4. If your Django database settings use the mysql database engine as in
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
You need to add
'OPTIONS': {'charset': 'utf8mb4'}
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark!
I have added the following to the bottom of the OPTIONS section of /etc/mailiman3/mailman-web.py:
'charset': 'utf8mb4',
Hopefully that fixes the problem.
Thanks,
Gordon
Hi Gordon (and Mark)!
I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to:
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': 'xxxxxxxxxxxxxxxxxx', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } }
and restarted stuff, but I cannot tell that it has changed anything.
Do I need to do something with the database itself (which is somewhat outside my comfort zone...) to make this work?
Thanks very much for any advice you may have.
Cheers,
David
On 1/25/21 11:42 PM, David Partain via Mailman-users wrote:
Hi Gordon (and Mark)!
I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to:
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': 'xxxxxxxxxxxxxxxxxx', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } }
and restarted stuff, but I cannot tell that it has changed anything.
That should do it. I think the original message should have been saved as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory. If this is the case, it would have been retried and produced the same error with each new post, and after you made the above change, it would have been archived and the errors stopped.
Do I need to do something with the database itself (which is somewhat outside my comfort zone...) to make this work?
Are you saying you are still seeing these errors? If so, it may be a different error. What is the exact error message? We don't need all the Django stuff, just the part equivalent to
Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\xF0\x9F\xA4\xA3\xF0\x9F...' for column mailman3web.hyperkitty_email.content at row 1")
from the OP.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark,
Thank you for your kind answer. While I've been running a mailman site for many years, I'm just getting going with mailman3 (and migrating my 1400 lists...) and I feel lost occasionally :)
I wrote:
I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to:
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': 'xxxxxxxxxxxxxxxxxx', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } }
and restarted stuff, but I cannot tell that it has changed anything.
You responded:
That should do it. I think the original message should have been saved as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory.
That directory has all of the messages sent to that list:
# ls /var/lib/mailman3/archives/hyperkitty/spool/ 1611827925.8030825+925c996b9b6020ee18e9a689ce23384855159f77.pck 1611827926.1923075+e728f3c29c1fe6eeeca641314cdba7007cb6e132.pck 1611827925.8917305+d16b7004bdbbef98ce40fb8ee70e492715d1607d.pck 1611827926.2623458+1aac13b50d8e7f32eb14d13e6559656aa8f3bd10.pck 1611827925.9594557+7ad2a910b5ee9142c88f0bf26c60c6df960d4262.pck 1611827926.3488555+04a540651c5f829751518433e1c54168ab63a38b.pck 1611827926.0404096+a5e745f51a9f7d541aeb770a89604be17019f3c5.pck 1611827926.4206145+de861eec6404d0c8f9f8bdb180e29bff2de7c52b.pck 1611827926.1167526+0aa63639a4ec102bf63522827637e75e6a2b092a.pck
If this is the case, it would have been retried and produced the same error with each new post, and after you made the above change, it would have been archived and the errors stopped. Are you saying you are still seeing these errors? If so, it may be a different error. What is the exact error message? We don't need all the Django stuff, just the part equivalent to Internal Server Error: /mailman3/hyperkitty/api/mailman/archive OperationalError at /hyperkitty/api/mailman/archive (1366, "Incorrect string value: '\xF0\x9F\xA4\xA3\xF0\x9F...' for column mailman3web.hyperkitty_email.content at row 1")
Yes, I'm still seeing the errors. When I send mail to a list with a cute emoji, I see the following in /var/log/mailman3/web/mailman-web.log:
ERROR 2021-01-28 10:58:45,708 16923 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive (bunch of python errors omitted) DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column 'content' at row 1")
Also, a bunch of mail is sent to root, presumably one for every message that has previously failed, with the same error information, plus information about the POST method done. (If something there would be helpful, please let me know.)
Finally, I'm doing all of this on Ubuntu, with the packages available in the distro, which means:
mailman3 3.1.1-9
mailman3-doc 3.1.1-9
mailman3-full 3.1.1-9
mailman3-web 0+20170523-14
python-django-hyperkitty 1.1.4-4
python-django-mailman3 1.1.0-4
python-django-postorius 1.1.2-3
python-mailmanclient 3.1.1-5
python3-mailman-hyperkitty 1.1.0-4
Thanks very much for any help you can offer.
Cheers,
David
On 1/28/21 5:33 AM, David Partain via Mailman-users wrote:
I wrote:
I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to:
DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': 'xxxxxxxxxxxxxxxxxx', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } }
and restarted stuff, but I cannot tell that it has changed anything.
You responded:
That should do it. I think the original message should have been saved as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory.
That directory has all of the messages sent to that list:
# ls /var/lib/mailman3/archives/hyperkitty/spool/ 1611827925.8030825+925c996b9b6020ee18e9a689ce23384855159f77.pck 1611827926.1923075+e728f3c29c1fe6eeeca641314cdba7007cb6e132.pck 1611827925.8917305+d16b7004bdbbef98ce40fb8ee70e492715d1607d.pck 1611827926.2623458+1aac13b50d8e7f32eb14d13e6559656aa8f3bd10.pck 1611827925.9594557+7ad2a910b5ee9142c88f0bf26c60c6df960d4262.pck 1611827926.3488555+04a540651c5f829751518433e1c54168ab63a38b.pck 1611827926.0404096+a5e745f51a9f7d541aeb770a89604be17019f3c5.pck 1611827926.4206145+de861eec6404d0c8f9f8bdb180e29bff2de7c52b.pck 1611827926.1167526+0aa63639a4ec102bf63522827637e75e6a2b092a.pck
This is what's expected since the underlying issue isn't yet fixed.
Yes, I'm still seeing the errors. When I send mail to a list with a cute emoji, I see the following in /var/log/mailman3/web/mailman-web.log:
ERROR 2021-01-28 10:58:45,708 16923 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive (bunch of python errors omitted) DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column 'content' at row 1")
OK. The issue is with your database it doesn't like '\xF0\x9F\x98\x8A' which is a 4-byte utf-8 encoding for a smiley face emoji.
The issue is with MySQL and MariaDB which do not accept 4-byte utf-8 encodings by default. The settings you show above should be sufficient to allow 4-byte utf-8 encodings.
I'm not that familiar with how the Debian/Ubuntu package sets things up. Are you sure that /etc/mailman3/mailman-web.py is the settings file used by Django, and if so does it have at the end something like
try:
from settings_local import *
except ImportError:
pass
and if so is there something there that overrides the DATABASES setting?
If all that is OK, you may need to do something like
ALTER DATABASE mailman3web CHARACTER SET utf8mb4;
ALTER TABLE hyperkitty_email CHARACTER SET utf8mb4;
on the database itself.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 2021-01-28 at 12:03 -0800, Mark Sapiro wrote: On 1/28/21 5:33 AM, David Partain via Mailman-users wrote: I wrote: I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to: DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': 'xxxxxxxxxxxxxxxxxx', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': { 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4', }, } } and restarted stuff, but I cannot tell that it has changed anything. You responded: That should do it. I think the original message should have been saved as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory. That directory has all of the messages sent to that list: # ls /var/lib/mailman3/archives/hyperkitty/spool/ 1611827925.8030825+925c996b9b6020ee18e9a689ce23384855159f77.pck 1611827926.1923075+e728f3c29c1fe6eeeca641314cdba7007cb6e132.pck 1611827925.8917305+d16b7004bdbbef98ce40fb8ee70e492715d1607d.pck 1611827926.2623458+1aac13b50d8e7f32eb14d13e6559656aa8f3bd10.pck 1611827925.9594557+7ad2a910b5ee9142c88f0bf26c60c6df960d4262.pck 1611827926.3488555+04a540651c5f829751518433e1c54168ab63a38b.pck 1611827926.0404096+a5e745f51a9f7d541aeb770a89604be17019f3c5.pck 1611827926.4206145+de861eec6404d0c8f9f8bdb180e29bff2de7c52b.pck 1611827926.1167526+0aa63639a4ec102bf63522827637e75e6a2b092a.pck This is what's expected since the underlying issue isn't yet fixed. Yes, I'm still seeing the errors. When I send mail to a list with a cute emoji, I see the following in /var/log/mailman3/web/mailman-web.log: ERROR 2021-01-28 10:58:45,708 16923 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive (bunch of python errors omitted) DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column 'content' at row 1") OK. The issue is with your database it doesn't like '\xF0\x9F\x98\x8A' which is a 4-byte utf-8 encoding for a smiley face emoji. The issue is with MySQL and MariaDB which do not accept 4-byte utf-8 encodings by default. The settings you show above should be sufficient to allow 4-byte utf-8 encodings. I'm not that familiar with how the Debian/Ubuntu package sets things up. Are you sure that /etc/mailman3/mailman-web.py is the settings file used by Django, and if so does it have at the end something like ``` try: from settings_local import * except ImportError: pass ``` and if so is there something there that overrides the DATABASES setting? If all that is OK, you may need to do something like ``` ALTER DATABASE mailman3web CHARACTER SET utf8mb4; ALTER TABLE hyperkitty_email CHARACTER SET utf8mb4; ``` on the database itself. Hi Mark (and others), I'm now looking at this again after having to focus on other stuff. I'm befuddled, and I don't really know what to do now. I did the above. If I send mail to my list with those bleeping emojis, I see the same errors. In /var/log/mailman3/web/mailman-web.log I see DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x95\\xB5\\xF0\\x9F... new mail to root, /var/lib/mailman3/archives/hyperkitty/spool/ has a new file, etc. The mail gets delivered, of course. If I then look at the database itself, I see: mysql> use mailman3web; mysql> SELECT @@character_set_database, @@collation_database; +--------------------------+----------------------+ | @@character_set_database | @@collation_database | +--------------------------+----------------------+ | utf8mb4 | utf8mb4_general_ci | +--------------------------+----------------------+ 1 row in set (0.00 sec) mysql> SHOW TABLE STATUS where name like 'hyperkitty_email'; +------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+ | Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment | +------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+ | hyperkitty_email | InnoDB | 10 | Dynamic | 133 | 11949 | 1589248 | 0 | 196608 | 4194304 | 150 | 2021-01-29 17:23:13 | NULL | NULL | utf8mb4_general_ci | NULL | | | +------------------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+--------------------+----------+----------------+---------+ 1 row in set (0.00 sec) So, I go look elsewhere... In /etc/mailman3/mailman.cfg I see: class: mailman.database.mysql.MySQLDatabase url: mysql+pymysql://mailman3:NopeNotPastingThat@localhost/mailman3?charset=utf8&use_unicode=1 Should I change the latter to be something like: url: mysql+pymysql://mailman3:NopeNotPastingThat@localhost/mailman3?charset=utf8mb4&use_unicode=1 ? Cheers, David
On 2/25/21 6:32 AM, David Partain via Mailman-users wrote:
In /etc/mailman3/mailman.cfg I see:
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://mailman3:NopeNotPastingThat@localhost/mailman3?charset=utf8&use_unicode=1
Should I change the latter to be something like:
url: mysql+pymysql://mailman3:NopeNotPastingThat@localhost/mailman3?charset=utf8mb4&use_unicode=1
Yes, either make it charset=utf8mb4 or I think dropping the charset= option altogether will use the database default, however I'm not clear why this would be the issue as this only controls how Mailman core accesses the database, and your issue is with HyperKitty/Django and its access is controlled by the DATABASES setting in Django's settings which you say do specify 'charset': 'utf8mb4' in OPTIONS.
Also, it seems Mailman core is not even using the same database - i.e.
mailman3 whereas Django's is mailman3web.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 2021-02-25 at 13:24 -0800, Mark Sapiro wrote:
On 2/25/21 6:32 AM, David Partain via Mailman-users wrote:
In /etc/mailman3/mailman.cfg I see:
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://mailman3:NopeNotPastingThat@localhost/mailman3?charset=utf8&use_unicode=1
Should I change the latter to be something like:
url: mysql+pymysql://mailman3:NopeNotPastingThat@localhost/mailman3?charset=utf8mb4&use_unicode=1
Yes, either make it charset=utf8mb4 or I think dropping the charset=
option altogether will use the database default, however I'm not clear
why this would be the issue as this only controls how Mailman core
accesses the database, and your issue is with HyperKitty/Django and its
access is controlled by the DATABASES setting in Django's settings which
you say do specify 'charset': 'utf8mb4' in OPTIONS.
Also, it seems Mailman core is not even using the same database - i.e.
mailman3 whereas Django's is mailman3web.
Hi Mark,
Thanks for being patient with me. I'm still baffled but might have found the problem.
I've done the above, to no avail.
If I look at the queries being sent to mysql, I see stuff like this:
174 Connect mailman3web@localhost<mailto:mailman3web@localhost> on mailman3web using TCP/IP
174 Query SET sql_mode='STRICT_TRANS_TABLES'
174 Query COMMIT
174 Query SET AUTOCOMMIT = 0
174 Query SET AUTOCOMMIT = 1
174 Query SET AUTOCOMMIT = 0
I note that there is nothing about the charset, so something's clearly wonky.
As previously stated, /etc/mailman3/mailman-web.py has the right thing:
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
'charset': 'utf8mb4',},
but that latter is having no effect.
I then found /var/lib/ucf/cache/:etc:mailman3:mailman-web.py which doesn't have charset
(but does have init_command), but I _think_ that's only about the package management system.
So... I found that /usr/share/mailman3-web/settings_local.py is just a link to /etc/mailman3/mailman-web.py
(which obviously has the right thing), but what about the .pyc file?
# grep -i STRICT_TRANS_TABLES /usr/share/mailman3-web/settings_local.pyc
Binary file /usr/share/mailman3-web/settings_local.pyc matches
# grep -i charset /usr/share/mailman3-web/settings_local.pyc
#
It ain't in the .pyc file. Am I overly optimistic that that might the be problem?
If so, I'd be grateful if someone'd tell the newbie how to recreate that file without
trashing something.
Cheers,
David
On 3/1/21 11:53 AM, David Partain via Mailman-users wrote:
Thanks for being patient with me. I'm still baffled but might have found the problem.
I've done the above, to no avail.
If I look at the queries being sent to mysql, I see stuff like this:
174 Connect mailman3web@localhost<mailto:mailman3web@localhost> on mailman3web using TCP/IP
174 Query SET sql_mode='STRICT_TRANS_TABLES'
174 Query COMMIT
174 Query SET AUTOCOMMIT = 0
174 Query SET AUTOCOMMIT = 1
174 Query SET AUTOCOMMIT = 0
I note that there is nothing about the charset, so something's clearly wonky.
As previously stated, /etc/mailman3/mailman-web.py has the right thing:
'OPTIONS': {
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4',},
but that latter is having no effect.
I then found /var/lib/ucf/cache/:etc:mailman3:mailman-web.py which doesn't have charset
(but does have init_command), but I _think_ that's only about the package management system.
I think that's correct. I doubt that that's the problem, but if all else fails, you might just remove it. I don't think that would hurt as it's purpose is just todetect on upgrade of the package if you've changed the file from the distributed one and ask you what you want to do about it.
So... I found that /usr/share/mailman3-web/settings_local.py is just a link to /etc/mailman3/mailman-web.py
(which obviously has the right thing), but what about the .pyc file?
# grep -i STRICT_TRANS_TABLES /usr/share/mailman3-web/settings_local.pyc
Binary file /usr/share/mailman3-web/settings_local.pyc matches
# grep -i charset /usr/share/mailman3-web/settings_local.pyc
#
It ain't in the .pyc file. Am I overly optimistic that that might the be problem?
If so, I'd be grateful if someone'd tell the newbie how to recreate that file without
trashing something.
This is a bit strange: Here's how it works.
.pyc files are byte-compiled versions of the corresponding .py file.
When Python runs or imports from a .py file, it first looks for a corresponding .pyc file and if that is as new or newer than the .py, it uses the .pyc. If not, it uses the .py and byte-compiles it and if it has permission, writes an updated .pyc
Except recent Python3 (at least I think 3.5 and newer doesn't use a .pyc in the same directory as the .py Its uses a __pycache__ directory in the same directory as the .py and within that directory a file like in this case settings_local.cpython-37.pyc (or whatever the python version is)
In any case, I would remove the existing settings_local.pyc. That can't hurt, but I'm curious what
ls -lR /usr/share/mailman3-web/
shows.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/1/21 1:55 PM, Mark Sapiro wrote:
When Python runs or imports from a .py file, it first looks for a corresponding .pyc file and if that is as new or newer than the .py, it uses the .pyc. If not, it uses the .py and byte-compiles it and if it has permission, writes an updated .pyc
Except recent Python3 (at least I think 3.5 and newer doesn't use a .pyc in the same directory as the .py Its uses a __pycache__ directory in the same directory as the .py and within that directory a file like in this case settings_local.cpython-37.pyc (or whatever the python version is)
In any case, I would remove the existing settings_local.pyc. That can't hurt, but I'm curious what
ls -lR /usr/share/mailman3-web/
shows.
It just occurred to me what the issue is. It is almost certainly that there is a /usr/share/mailman3-web/settings_local.pyc that is as new or newer than the /usr/share/mailman3-web/settings_local.py symlink even though it is older than the target /etc/mailman3/mailman-web.py.
Just remove that .pyc, and that should fix it.
But I still wonder what python version this is since current Mailman requires python >= 3.6
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
It just occurred to me what the issue is. It is almost certainly that there is a /usr/share/mailman3-web/settings_local.pyc that is as new or newer than the /usr/share/mailman3-web/settings_local.py symlink even though it is older than the target /etc/mailman3/mailman-web.py.
Just remove that .pyc, and that should fix it.
But I still wonder what python version this is since current Mailman requires python >= 3.6
Hi Mark,
$ ls -lR /usr/share/mailman3-web/ /usr/share/mailman3-web/: total 52 -rwxr-xr-x 1 root root 242 May 23 2017 manage.py -rw-r--r-- 1 root root 14835 Apr 12 2018 settings.py -rw-r--r-- 1 root root 8157 Nov 25 20:02 settings.pyc lrwxrwxrwx 1 root root 28 Apr 12 2018 settings_local.py -> /etc/mailman3/mailman-web.py -rw-r--r-- 1 root root 5752 Mar 15 2018 settings_local.py.sample -rw-r----- 1 root root 2359 Dec 7 16:11 settings_local.pyc-210302 -rw-r--r-- 1 root root 1314 May 23 2017 urls.py -rw-r--r-- 1 root root 798 Dec 7 16:11 urls.pyc -rw-r--r-- 1 root root 1173 May 23 2017 wsgi.py
and
$ ls -l /etc/mailman3/mailman-web.py -rw-r----- 1 root www-data 5876 Mar 1 21:13 /etc/mailman3/mailman-web.py
Moving settings_local.pyc (as above) seems not to have had any effect at all.
I'm not on the latest mailman, unfortunately. I use the bundled packages on Ubuntu 18.04, which are
mailman3 3.1.1-9 mailman3-doc 3.1.1-9 mailman3-full 3.1.1-9 mailman3-web 0+20170523-14
From the mailman-web logs, it looks like it's using python2.
I did a wee experiment and copied /etc/mailman3/mailman-web.py to /usr/share/mailman3-web/settings_local.py (rather than the symlink) and life got totally strange. Suddenly I'm seeing this in /var/log/mailman3/web/mailman-web.log:
hyperkitty.lib.utils Failed to update the fulltext index: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
and I can no longer connect to the web site. All the postgres is commented out in /etc/mailman3/mailman-web.py (and thus settings_local.py ...)
Put the symlink back, and it works again. (But still without the emojis working, of course.)
Postgres is what's in settings.py, but I assume that should be overriden by what's in settings_local.py.
Baffling. I don't really know what to do next, but I'll keep looking.
Cheers,
David
On 3/2/21 12:10 AM, David Partain via Mailman-users wrote:
I'm not on the latest mailman, unfortunately. I use the bundled packages on Ubuntu 18.04, which are
mailman3 3.1.1-9 mailman3-doc 3.1.1-9 mailman3-full 3.1.1-9 mailman3-web 0+20170523-14
From the mailman-web logs, it looks like it's using python2.
That would explain a lot.
In particular, it would explain why the .pyc files aren't located per PEP 3147 <https://www.python.org/dev/peps/pep-3147/>.
But it also raise the question of how anything works at all. Python >= 3.6 is required.
HyperKitty and Postorius used to run under Python 2, but haven't for some time. I'm not sure what versions are in Debian/Ubuntu 0+20170523-14 package.
Put the symlink back, and it works again. (But still without the emojis working, of course.)
Postgres is what's in settings.py, but I assume that should be overriden by what's in settings_local.py.
Baffling. I don't really know what to do next, but I'll keep looking.
I suggest removing all the .pyc files and restarting the services. That will probably recreate .pyc files, but they should then be good.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi again,
On 3/2/21 12:10 AM, David Partain via Mailman-users wrote:
I'm not on the latest mailman, unfortunately. I use the bundled
packages on Ubuntu 18.04, which are
mailman3 3.1.1-9 mailman3-doc 3.1.1-9 mailman3-full 3.1.1-9 mailman3-web 0+20170523-14
From the mailman-web logs, it looks like it's using python2.
That would explain a lot.
I was wrong...
# ps -ef | grep python | grep mailman list 22170 1 0 08:30 ? 00:00:00 /usr/bin/python3 /usr/lib/mailman3/bin/master -C /etc/mailman3/mailman.cfg list 22176 22170 0 08:30 ? 00:00:05 /usr/bin/python3 /usr/lib/mailman3/bin/runner --runner=archive:0:1 -C /etc/mailman3/mailman.cfg list 22177 22170 0 08:30 ? 00:00:05 /usr/bin/python3 /usr/lib/mailman3/bin/runner --runner=bounces:0:1 -C /etc/mailman3/mailman.cfg (etc.)
However, if I look in mailman-web.log, I see lots of logging like:
DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x95\\xB5\\xF0\\x9F...' for column 'content' at row 1") ERROR 2021-03-02 08:58:29,407 24297 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive Traceback (most recent call last): File "/usr/lib/python2.7/dist- packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request)
Note that it's pointing to /usr/lib/python2.7/dist- packages/django/core/handlers/exception.py
This could explain all of this behavior... Clearly whoever installed this box included both.
So mailman core seems to be using python3 and mailman-web python2?
I assume this is because /usr/share/mailman3-web/manage.py, which has:
'#!/usr/bin/env python'
resolves to Python 2.7.17.
Should I could try changing that to '#!/usr/bin/env python3' and see what happens? :) I could also uninstall python2 but that becomes more of a nail-biter...
I suggest removing all the .pyc files and restarting the services. That will probably recreate .pyc files, but they should then be good.
Nothing was regenerated yesterday when I removed them, but that could be because of the python2 issue.
Cheers,
David
On Tue, 2021-03-02 at 19:56 +0000, David Partain via Mailman-users wrote:
I assume this is because /usr/share/mailman3-web/manage.py, which has:
'#!/usr/bin/env python'
resolves to Python 2.7.17.
Should I could try changing that to '#!/usr/bin/env python3' and see what happens? :) I could also uninstall python2 but that becomes more of a nail-biter...
I can report that the above doesn't help. Neither does just making /usr/bin/python link to /usr/bin/python3.6
I still get errors in mailman-web.log pointing to 2.7 as shown in my previous mail.
Cheers,
David
On Tue, 2021-03-02 at 20:07 +0000, David Partain via Mailman-users wrote:
On Tue, 2021-03-02 at 19:56 +0000, David Partain via Mailman-users wrote:
I assume this is because /usr/share/mailman3-web/manage.py, which has:
'#!/usr/bin/env python'
resolves to Python 2.7.17.
Should I could try changing that to '#!/usr/bin/env python3' and see what happens? :) I could also uninstall python2 but that becomes more of a nail-biter...
I can report that the above doesn't help. Neither does just making /usr/bin/python link to /usr/bin/python3.6
I still get errors in mailman-web.log pointing to 2.7 as shown in my previous mail.
If I force the issue by linking /usr/bin/python to /usr/bin/python3.6, I start seeing:
ModuleNotFoundError: No module named 'hyperkitty
in mailman-web.log.
I know next to nothing about python. Is
pip install hyperkitty
the right answer?
I have the following installed now:
# dpkg -l | grep hyperkitty python-django-hyperkitty 1.1.4-4 python3-mailman-hyperkitty 1.1.0-4
but this version of Ubuntu has no python3-django-hyperkitty or I'd just try installing that...
My, this is a weird journey.
Cheers,
David
"David" == David Partain via Mailman-users <mailman-users@mailman3.org> writes:
David> On Tue, 2021-03-02 at 20:07 +0000, David Partain via David> Mailman-users wrote:
On Tue, 2021-03-02 at 19:56 +0000, David Partain via Mailman-users wrote:
I assume this is because /usr/share/mailman3-web/manage.py, which has:
'#!/usr/bin/env python'
resolves to Python 2.7.17.
You may want to install python-is-python3 to make /usr/bin/python python3
PeterC
On 3/2/21 12:33 PM, David Partain via Mailman-users wrote:
On Tue, 2021-03-02 at 20:07 +0000, David Partain via Mailman-users wrote:
On Tue, 2021-03-02 at 19:56 +0000, David Partain via Mailman-users wrote:
I assume this is because /usr/share/mailman3-web/manage.py, which has:
'#!/usr/bin/env python'
resolves to Python 2.7.17.
Should I could try changing that to '#!/usr/bin/env python3' and see what happens? :) I could also uninstall python2 but that becomes more of a nail-biter...
NO! Do not do that.
I can report that the above doesn't help. Neither does just making /usr/bin/python link to /usr/bin/python3.6
I still get errors in mailman-web.log pointing to 2.7 as shown in my previous mail.
If I force the issue by linking /usr/bin/python to /usr/bin/python3.6, I start seeing:
ModuleNotFoundError: No module named 'hyperkitty
in mailman-web.log.
I know next to nothing about python. Is
pip install hyperkitty
the right answer?
No.
I have the following installed now:
# dpkg -l | grep hyperkitty python-django-hyperkitty 1.1.4-4 python3-mailman-hyperkitty 1.1.0-4
but this version of Ubuntu has no python3-django-hyperkitty or I'd just try installing that...
This is all because you are using the Ubuntu 18.04 package which is about 4 years out of date and is from the era where HyperKitty, Postorius and django_mailman3 still ran under Python 2.1.7.
Unless you want to remove the Ubuntu package and install everything with pip, you need to just stick with what you have that at least partly works and we'll try to determine how to fix the utf8mb4 issue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/2/21 1:43 PM, Mark Sapiro wrote:
Unless you want to remove the Ubuntu package and install everything with pip, you need to just stick with what you have that at least partly works and we'll try to determine how to fix the utf8mb4 issue.
In that vein, you need to remove any settings_local.pyc files and restart uWSGI or whatever provides the Django wsgi interface in your system. and see if that helps.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, 2021-03-02 at 13:54 -0800, Mark Sapiro wrote:
On 3/2/21 1:43 PM, Mark Sapiro wrote:
Unless you want to remove the Ubuntu package and install everything with pip, you need to just stick with what you have that at least partly works and we'll try to determine how to fix the utf8mb4 issue.
In that vein, you need to remove any settings_local.pyc files and restart uWSGI or whatever provides the Django wsgi interface in your system. and see if that helps.
Hi,
I removed /usr/share/mailman3-web/settings_local.pyc quite some time ago, have restarted both mailman3 and mailman3-web numerous times since then, and today I restarted uwsgi this morning.
No new settings_local.pyc has yet been created. Ideas..?
Cheers,
David
On 3/2/21 11:21 PM, David Partain via Mailman-users wrote:
I removed /usr/share/mailman3-web/settings_local.pyc quite some time ago, have restarted both mailman3 and mailman3-web numerous times since then, and today I restarted uwsgi this morning.
No new settings_local.pyc has yet been created. Ideas..?
No. Restarting Mailman core won't help with this. I don't know what's involved in restarting mailman3-web, but I think that restarting it and uwsgi should be enough and a new settings_local.pyc would be created the next time a django process is invoked (e.g., by visiting a Postorius or HyperKitty web page). If this isn't happening, I don't know why.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, 2021-03-02 at 13:43 -0800, Mark Sapiro wrote:
This is all because you are using the Ubuntu 18.04 package which is about 4 years out of date and is from the era where HyperKitty, Postorius and django_mailman3 still ran under Python 2.1.7.
Unless you want to remove the Ubuntu package and install everything with pip, you need to just stick with what you have that at least partly works and we'll try to determine how to fix the utf8mb4 issue.
Hi Mark,
You're a very patient soul. Thank you.
Yes, I'm resigned to exactly that. The archives when folks send emojis will break, but that's just life. Mail will get sent. Heck, I guess I could edit the file in /var/lib/mailman3/archives/hyperkitty/spool to get rid of the emoji in the worst case...
Once I've migrated my 1500 mailing lists (this weekend) to the server, I'll think about how to get a new server set up that I can shift to and I'll install that with pip, since that'll clearly give me something more supportable.
Is Ubuntu 20.04 a reasonable platform for that?
Cheers,
David
On 3/2/21 1:55 PM, David Partain via Mailman-users wrote:
On Tue, 2021-03-02 at 13:43 -0800, Mark Sapiro wrote:
Unless you want to remove the Ubuntu package and install everything with pip, you need to just stick with what you have that at least partly works and we'll try to determine how to fix the utf8mb4 issue.
Hi Mark,
You're a very patient soul. Thank you.
Yes, I'm resigned to exactly that. The archives when folks send emojis will break, but that's just life. Mail will get sent. Heck, I guess I could edit the file in /var/lib/mailman3/archives/hyperkitty/spool to get rid of the emoji in the worst case...
You would need to do that, because once one of the emoji messages gets
into /var/lib/mailman3/archives/hyperkitty/spool, I think no further
messages are archived because mailman_hyperkitty tries to process the
queued messages first and throws the exception.
But, it's still not clear to me whether you have removed the settings_local.pyc and restarted uWSGI or whatever interfaces to Django.
Once I've migrated my 1500 mailing lists (this weekend) to the server, I'll think about how to get a new server set up that I can shift to and I'll install that with pip, since that'll clearly give me something more supportable.
Is Ubuntu 20.04 a reasonable platform for that?
Yes it is. FWIW, this list runs bleeding edge Mailman 3 on an Ubuntu 14.04 server (yes we should upgrade), the only issue being that we needed to install Python 3.6 (the latest at the time) from source because the native Python 3 was 3.4
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Heck, I guess I could edit the file in /var/lib/mailman3/archives/hyperkitty/spool to get rid of the emoji in the worst case...
You would need to do that, because once one of the
emojimessages gets into /var/lib/mailman3/archives/hyperkitty/spool, I think no further messages are archived because mailman_hyperkitty tries to process the queued messages first and throws the exception.
Hi,
In this regard, I have three questions:
Am I correct in assuming that I can just use my favorite editor (vi) to remove the offending characters from the file?
If I cannot find the offending character, is it safe to just remove the file so that subsequent archiving can proceed?
Can I do something to make hyperkitty go through the spool?
Cheers,
David
On 3/2/21 11:33 PM, David Partain via Mailman-users wrote:
In this regard, I have three questions:
- Am I correct in assuming that I can just use my favorite editor (vi) to remove the offending characters from the file?
No. The file is a Python pickle and you can't edit it. You would need to
get the message and the metadata from the pickle (you could at least
view it with the mailman qfile command), edit the message and then
re-pickle the message and metadata.
- If I cannot find the offending character, is it safe to just remove the file so that subsequent archiving can proceed?
Yes.
- Can I do something to make hyperkitty go through the spool?
I don't know what you mean. Every time there is a new post to any list, Mailman core invokes the mailman_hyperkitty process to archive the message and the forst thing that process does is attempt to reprocess all the messages in the spool which fails because of the utf8mb4 issue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, 2021-03-03 at 08:29 -0800, Mark Sapiro wrote:
- Am I correct in assuming that I can just use my favorite editor (vi) to remove the offending characters from the file?
No. The file is a Python pickle and you can't edit it. You would need to get the message and the metadata from the pickle (you could at least view it with the
mailman qfilecommand), edit the message and then re-pickle the message and metadata.
And, in my impatience, I tried it before receiving your response and now I seem to have done silliness in my database :( New error in mailman-web.log:
AttributeError: 'NoneType' object has no attribute 'subject'
Sigh. Stupid me. If you have suggestions on fixing that...
Right now, my plan is simply to remove the first message in the queue (since I assume it'll be the "broken" one) and it'll just not be in the archive. Life's hard. Even if I unpack 'em to edit, I'll find mail in, for example, in base64, and I'm not going to fix the mail by decoding, editing, re-encoding, ... Nope.
- Can I do something to make hyperkitty go through the spool?
I don't know what you mean.
Sorry I wasn't clear. I simply meant, "Is there a way to start hyperkitty going through the spool manually?" It sounds like, though, that I don't need to do that if it's going to happen that often anyway.
Cheers,
David
On 3/4/21 2:09 AM, David Partain via Mailman-users wrote:
On Wed, 2021-03-03 at 08:29 -0800, Mark Sapiro wrote:
- Am I correct in assuming that I can just use my favorite editor (vi) to remove the offending characters from the file?
No. The file is a Python pickle and you can't edit it. You would need to get the message and the metadata from the pickle (you could at least view it with the
mailman qfilecommand), edit the message and then re-pickle the message and metadata.And, in my impatience, I tried it before receiving your response and now I seem to have done silliness in my database :( New error in mailman-web.log:
AttributeError: 'NoneType' object has no attribute 'subject'
Sigh. Stupid me. If you have suggestions on fixing that...
Is the file you edited still in the spool directory or is it gone? If it's still there, move it aside and see if that fixes the issue.
What is the full traceback accompanying that AttributeError message from the log?
Right now, my plan is simply to remove the first message in the queue (since I assume it'll be the "broken" one) and it'll just not be in the archive. Life's hard. Even if I unpack 'em to edit, I'll find mail in, for example, in base64, and I'm not going to fix the mail by decoding, editing, re-encoding, ... Nope.
Examine the messages in the spool with mailman qfile var/archive/hyperkitty/spool/filename. If any files throw exceptions on
that, move them aside. Also, if you can see any 4-byte utf-8 encodings
in the output, move that aside as well. Then see what happens after that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
The major things to be checked/changed appear to be:
- check database mailman3web default character set, make sure to have utf8mb4
- check /etc/mailman3/mailman-web.py 'OPTIONS': { # Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See # https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset':'utf8mb4',
- python is python3
- sudo systemctl restart mailman3
Finally, studying this page and applying some changes I could fix the issues on my server.
participants (6)
-
David Partain -
Gordon Dickens -
gordon＠dickens.com
-
Mark Sapiro -
peter＠chubb.wattle.id.au -
Wikinaut