Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
by Mail Person
Hello everyone. I am trying to sort out an error happening when messages are sent to the mailman server. The emails are being sent to the list but I get this error each time:
Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
OperationalError at /hyperkitty/api/mailman/archive
(1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1")
Request Method: POST
Request URL: http://localhost/mailman3/hyperkitty/api/mailman/archive?key=7OF5IogiQItSUK…
Django Version: 2.2.12
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.8.10
Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']
Server time: Thu, 17 Aug 2023 17:13:09 +0000
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount',
'django_mailman3.lib.auth.fedora')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute
84. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute
71. return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
209. res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query
315. db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query
226. _mysql.connection.query(self, query)
The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1")) was the direct cause of the following exception:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
34. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
115. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
113. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner
40. return func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator
71. return func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view
54. return view_func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive
117. add_to_list(mlist_fqdn, msg)
File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list
153. email.save()
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save
740. self.save_base(using=using, force_insert=force_insert,
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base
777. updated = self._save_table(
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table
870. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert
907. return manager._insert([self], fields=fields, return_id=update_pk,
File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method
82. return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert
1186. return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql
1375. cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute
67. return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute_with_wrappers
76. return executor(sql, params, many, context)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute
84. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__
89. raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in _execute
84. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute
71. return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
209. res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query
315. db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query
226. _mysql.connection.query(self, query)
Exception Type: OperationalError at /hyperkitty/api/mailman/archive
Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x90\\xBA\\x0A\\x0A...' for column 'content' at row 1")
Request information:
USER: AnonymousUser
3 years
Re: urls.py location on venv install
by Torsten
Am 2026-08-10 01:41, schrieb Mark Sapiro:
> On 8/9/26 09:08, Torsten wrote:
>>
>> In fact, my settings.py does contain these import lines:
>>
>> import os
>
> This should not be needed unless something further down references os.
>
I have removed this line (s. other mail where Odhiambo suggested to take
the config from the documentation and adjust it)
>> from mailman_web.settings.base import *
>> from mailman_web.settings.mailman import *
>>
>> BASE_DIR = ('/var/lib/mailman3/web')
>>
>> SECRET_KEY = '<MYSECRETKEY>'
>> DEBUG = False
>>
>> #: Default list of admins who receive the emails from error logging.
>> ADMINS = ( ('Mailman Suite Admin', 'torsten(a)bbadmin.de'),)
>>
>> ALLOWED_HOSTS = ['localhost', '127.0.0.1',]
>
> You may also want your public facing web domain here.
>
They are there, I have just omitted them here.
>> INSTALLED_APPS = [
>> 'django_mailman3',
>> 'postorius',
>> 'hyperkitty',
>> # Uncomment the next line to enable the admin:
>> 'django.contrib.admin',
>
>
> I'm hoping there's more to this file, but in any case unless you're
> adding or deleting something from the definition in
> mailman-web/settings/base.py, this is redundant and if you are adding,
> it's better to do
> ```
> INSTALLED_APPS.append('app.to.add')
> ```
> or to remove something, for example
> ```
> del INSTALLED_APPS[INSTALLED_APPS.index(django.contrib.admin')]
> ```
>
Yes, there's more in this config and also more in INSTALLED_APPS.
>> And I have the mailman_web/urls.py in the venv where mailman was
>> installed.
>
> If you are using gunicorn as your wsgi server, I think there is an
> issue in the documentation.
> <https://docs.mailman3.org/en/latest/install/virtualenv.html#setting-up-guni…>
> says etc/mailman3/gunicorn.conf should contain `chdir =
> "/opt/mailman/mm"`. I think this is wrong. It should be `chdir =
> "/etc/mailman3"`.
I am using uwsgi.
Here's the complete settings.py, I have in /etc/mailman3 now:
# Mailman Web configuration file.
# /etc/mailman3/settings.py
# Get the default settings.
from mailman_web.settings.base import *
from mailman_web.settings.mailman import *
# Settings below supplement or override the defaults.
#: Default list of admins who receive the emails from error logging.
ADMINS = (
('Mailman Suite Admin', 'torsten(a)bbadmin.de'),
)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/var/lib/mailman3/web/mailman3web.db',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
}
# 'collectstatic' command will copy all the static files here.
# Alias this location from your webserver to `/static`
STATIC_ROOT = '/var/lib/mailman3/web/static'
# enable the 'compress' command.
COMPRESS_ENABLED = True
# Make sure that this directory is created or Django will fail on start.
LOGGING['handlers']['file']['filename'] =
'/var/log/mailman3/web/mailmanweb.log'
#: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts
ALLOWED_HOSTS = [
"localhost", # Archiving API from Mailman, keep it.
"127.0.0.1",
"mail.<MYDOMAIN1>",
"<MYDOMAIN1>",
"<MYDOMAIN2>",
# "lists.your-domain.org",
# Add here all production domains you have.
]
#: See
https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins
#: For Django <4.0 these are of the form 'lists.example.com' or
#: '.example.com' to include subdomains and for Django >=4.0 they
include
#: the scheme as in 'https://lists.example.com' or
'https://*.example.com'.
CSRF_TRUSTED_ORIGINS = [
# "lists.your-domain.org",
"https://*.<MYDOMAIN1>",
"https://*.<MYDOMAIN2>",
# Add here all production domains you have.
]
#: Current Django Site being served. This is used to customize the web
host
#: being used to serve the current website. For more details about
Django
#: site, see: https://docs.djangoproject.com/en/dev/ref/contrib/sites/
SITE_ID = 1
# Set this to a new secret value.
SECRET_KEY = '<MYSECRETKEY>'
# Set this to match the api_key setting in
# /opt/mailman/mm/mailman-hyperkitty.cfg (quoted here, not there).
MAILMAN_ARCHIVER_KEY = '<ANOTHERSECRETKEY>'
# The sender of emails from Django such as address confirmation
requests.
# Set this to a valid email address.
EMAILNAME = '<MYDOMAIN1>'
DEFAULT_FROM_EMAIL = 'postorius@{}'.format(EMAILNAME)
# The sender of error messages from Django. Set this to a valid email
# address.
SERVER_EMAIL = 'root@{}'.format(EMAILNAME)
3 weeks
Today no question, but my toolbox: set "global" user preferences; list or delete held messages
by Wikinaut
After some (sorry for bothering you) questions here, I was able to set up a mailman3 system which I regard as perfect for my use. Perhaps, you had similar problems with your installation, especially when coming from mailman2 and, like me, from majordomo......
I collected a lot of information which was presented in this mailling list archive here and also answers of my questions (thanks Mark and others)
News: https://github.com/Wikinaut/mailman3-tools has all of these tools.
README says:
Scripts for maintenance or setting-up of modern mailman3 instances
set preferences for all users
list held messages
delete held messages older n days
My setting is strictly as described in the documentation:
mailman3 + postorius/hyperkitty are installed via pip in an (venv) /opt/mailman/
My user scripts and a huge mailing list generator (not published here) are in a directory apart from that: user maillistgen home directory /home/maillistgen
user maillistgen has some sudo rights set uo in /etc/sudoers
a wrapper script is /home/maillistgen/run_mailman.sh which sets the correct path for Python, mailman3 and the (venv).
I hope, you like my tiny part of contribution and attribution back to Mark.
1 year
Today no question, but my toolbox: set "global" user preferences; list or delete held messages
by Wikinaut
After some (sorry for bothering you) questions here, I was able to set up a mailman3 system which I regard as perfect for my use. Perhaps, you had similar problems with your installation, especially when coming from mailman2 and, like me, from majordomo......
I collected a lot of information which was presented in this mailling list archive here and also answers of my questions (thanks Mark and others)
News: https://github.com/Wikinaut/mailman3-tools has all of these tools.
README says:
Scripts for maintenance or setting-up of modern mailman3 instances
set preferences for all users
list held messages
delete held messages older n days
My setting is strictly as described in the documentation:
mailman3 + postorius/hyperkitty are installed via pip in an (venv) /opt/mailman/
My user scripts and a huge mailing list generator (not published here) are in a directory apart from that: user maillistgen home directory /home/maillistgen
user maillistgen has some sudo rights set uo in /etc/sudoers
a wrapper script is /home/maillistgen/run_mailman.sh which sets the correct path for Python, mailman3 and the (venv).
I hope, you like my tiny part of contribution and attribution back to Mark.
1 year
Re: hyperkitty threads by month question
by Bryan Fields
Sorry for the delay, I was on some travel last week.
On 7/19/24 7:30 PM, Mark Sapiro wrote:
> That URL gives me as server error. What's in the Django (mailman-web)
> logs for that?
I did notice hitting a 500 error on occasion, sometimes it works.
When this happens, I get an email error below.
> Are you running all the Django periodic jobs and is qcluster running?
Ok, the docs said that qcluster service wasn't needed under uWSGI
https://docs.mailman3.org/en/latest/install/virtualenv.html#automatically-s…
qcluster does appear to be running:
218916 ? Sl 0:00 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/mailman-web qcluster
218929 ? Sl 0:00 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/mailman-web qcluster
3940851 ? Ss 0:00 /bin/sh -c /opt/mailman/venv/bin/mailman-web qcluster
3940854 ? S 0:01 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/mailman-web qcluster
3940856 ? S 117:32 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/mailman-web qcluster
3940860 ? S 0:28 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/mailman-web qcluster
3940861 ? Sl 3048:11 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/mailman-web qcluster
=== Email Error ===
Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /archives/list/nanog(a)nanog-mm3.keekles.org/
Internal Server Error: /archives/list/nanog(a)nanog-mm3.keekles.org/
NoReverseMatch at /archives/list/nanog(a)nanog-mm3.keekles.org/
Reverse for 'hk_archives_with_month' with keyword arguments '{'year': 999, 'mlist_fqdn': 'nanog(a)nanog-mm3.keekles.org', 'month': 12}' not found. 2 pattern(s) tried: ['archives/list/(?P<mlist_fqdn>[^/@]+@[^/@]+)/(?P<year>\\d{4})/(?P<month>\\d\\d?)/$', 'hyperkitty/list/(?P<mlist_fqdn>[^/@]+@[^/@]+)/(?P<year>\\d{4})/(?P<month>\\d\\d?)/$']
Request Method: GET
Request URL: http://nanog-mm3.keekles.org/archives/list/nanog@nanog-mm3.keekles.org/
Django Version: 4.2.13
Python Executable: /opt/mailman/venv/bin/uwsgi
Python Version: 3.10.12
Python Path: ['.', '', '/etc/mailman3', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/opt/mailman/venv/lib/python3.10/site-packages']
Server time: Thu, 25 Jul 2024 18:16:17 +0000
Installed Applications:
['hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.humanize',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount']
Installed Middleware:
('allauth.account.middleware.AccountMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback (most recent call last):
File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/lib/view_helpers.py", line 137, in inner
return func(request, *args, **kwargs)
File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/views/mlist.py", line 239, in overview
return render(request, "hyperkitty/overview.html", context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/shortcuts.py", line 24, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 175, in render
return self._render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 167, in _render
return self.nodelist.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 157, in render
return compiled_parent._render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 167, in _render
return self.nodelist.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 63, in render
result = block.nodelist.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/templatetags/cache.py", line 53, in render
value = self.nodelist.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 208, in render
return template.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 177, in render
return self._render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 167, in _render
return self.nodelist.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in render
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 1005, in <listcomp>
return SafeString("".join([node.render_annotated(context) for node in self]))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/defaulttags.py", line 238, in render
nodelist.append(node.render_annotated(context))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/defaulttags.py", line 238, in render
nodelist.append(node.render_annotated(context))
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/base.py", line 966, in render_annotated
return self.render(context)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/template/defaulttags.py", line 471, in render
url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/urls/base.py", line 88, in reverse
return resolver._reverse_with_prefix(view, prefix, *args, **kwargs)
File "/opt/mailman/venv/lib/python3.10/site-packages/django/urls/resolvers.py", line 828, in _reverse_with_prefix
raise NoReverseMatch(msg)
Exception Type: NoReverseMatch at /archives/list/nanog(a)nanog-mm3.keekles.org/
Exception Value: Reverse for 'hk_archives_with_month' with keyword arguments '{'year': 999, 'mlist_fqdn': 'nanog(a)nanog-mm3.keekles.org', 'month': 12}' not found. 2 pattern(s) tried: ['archives/list/(?P<mlist_fqdn>[^/@]+@[^/@]+)/(?P<year>\\d{4})/(?P<month>\\d\\d?)/$', 'hyperkitty/list/(?P<mlist_fqdn>[^/@]+@[^/@]+)/(?P<year>\\d{4})/(?P<month>\\d\\d?)/$']
Raised during: hyperkitty.views.mlist.overview
Request information:
USER: nanogadmin
GET: No GET data
POST: No POST data
FILES: No FILES data
COOKIES:
csrftoken = '********************'
sessionid = '********************'
META:
CSRF_COOKIE = '6F7ioB7blmJwLwKQLH0DPuAoWTW8mQGI'
HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, br'
HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.5'
HTTP_CONNECTION = 'Keep-Alive'
HTTP_COOKIE = '********************'
HTTP_HOST = 'nanog-mm3.keekles.org'
HTTP_SEC_FETCH_DEST = 'document'
HTTP_SEC_FETCH_MODE = 'navigate'
HTTP_SEC_FETCH_SITE = 'none'
HTTP_SEC_FETCH_USER = '?1'
HTTP_UPGRADE_INSECURE_REQUESTS = '1'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:109.0) Gecko/20100101 Firefox/115.0'
HTTP_X_FORWARDED_FOR = '47.206.239.202'
HTTP_X_FORWARDED_HOST = 'nanog-mm3.keekles.org'
HTTP_X_FORWARDED_SERVER = 'nanog-mm3.keekles.org'
PATH_INFO = '/archives/list/nanog(a)nanog-mm3.keekles.org/'
QUERY_STRING = ''
REMOTE_ADDR = '127.0.0.1'
REQUEST_METHOD = 'GET'
REQUEST_URI = '/archives/list/nanog(a)nanog-mm3.keekles.org/'
SCRIPT_NAME = ''
SERVER_NAME = 'nanog-mm3'
SERVER_PORT = '8000'
SERVER_PROTOCOL = 'HTTP/1.1'
uwsgi.core = 1
uwsgi.node = b'nanog-mm3'
uwsgi.version = b'2.0.26'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.file_wrapper = <built-in function uwsgi_sendfile>
wsgi.input = <uwsgi._Input object at 0x7ff7bcd9f850>
wsgi.multiprocess = True
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version = '(1, 0)'
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', 'root@localhost'), ('bryan fields', 'bryan(a)bryanfields.net'))"
ALLOWED_HOSTS = ['localhost', '127.0.0.1', 'nanog-mm3.keekles.org']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = PosixPath('/opt/mailman/web')
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'}
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_FILTERS = {'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'], 'js': ['compressor.filters.jsmin.rJSMinFilter']}
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7ff7bdb09750>
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = True
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_MANIFEST_STORAGE = 'compressor.storage.OfflineManifestFileStorage'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/opt/mailman/web/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_URL = '/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_MASKED = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = ['https://nanog-mm3.keekles.org']
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'mailmanweb', 'USER': 'mailman', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '5432', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'CONN_HEALTH_CHECKS': False, 'OPTIONS': {}, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'MIGRATE': True, 'MIRROR': None, 'NAME': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATA_UPLOAD_MAX_NUMBER_FILES = 100
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_EXCEPTION_REPORTER = 'django.views.debug.ExceptionReporter'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'admin(a)nanog-mm3.keekles.org'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = 420
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'PATH': '/opt/mailman/web/xapian_index', 'ENGINE': 'xapian_backend.XapianEngine'}}
HYPERKITTY_ENABLE_GRAVATAR = True
IGNORABLE_404_URLS = []
INSTALLED_APPS = ['hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount']
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ar-dz', 'Algerian Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('ckb', 'Central Kurdish (Sorani)'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('hy', 'Armenian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('ig', 'Igbo'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kab', 'Kabyle'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('ky', 'Kyrgyz'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('ms', 'Malay'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('tg', 'Tajik'), ('th', 'Thai'), ('tk', 'Turkmen'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('uz', 'Uzbek'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'ar-dz', 'ckb', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_HTTPONLY = False
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LANGUAGE_COOKIE_SAMESITE = None
LANGUAGE_COOKIE_SECURE = False
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.WatchedFileHandler', 'filename': '/opt/mailman/web/logs/mailmanweb.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'ERROR', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'ERROR', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'DEBUG', 'propagate': True}, 'postorius': {'handlers': ['console', 'file'], 'level': 'INFO'}, 'q': {'level': 'WARNING', 'propagate': False, 'handlers': ['console', 'file']}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = '/'
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('allauth.account.middleware.AccountMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')"
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT = '********************'
POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost:8000'
PREPEND_WWW = False
Q_CLUSTER = {'retry': 360, 'timeout': 300, 'save_limit': 100, 'orm': 'default', 'workers': 2}
ROOT_URLCONF = 'mailman_web.urls'
SECRET_KEY = '********************'
SECRET_KEY_FALLBACKS = '********************'
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_CROSS_ORIGIN_OPENER_POLICY = 'same-origin'
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = None
SECURE_REDIRECT_EXEMPT = []
SECURE_REFERRER_POLICY = 'same-origin'
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'admin(a)nanog-mm3.keekles.org'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SAMESITE = 'Lax'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 1
SOCIALACCOUNT_PROVIDERS = {'openid': {'SERVERS': [{'id': 'yahoo', 'name': 'Yahoo', 'openid_url': 'http://me.yahoo.com'}]}, 'google': {'SCOPE': ['profile', 'email'], 'AUTH_PARAMS': {'access_type': 'online'}}, 'facebook': {'METHOD': 'oauth2', 'SCOPE': ['email'], 'FIELDS': ['email', 'name', 'first_name', 'last_name', 'locale', 'timezone'], 'VERSION': 'v2.4'}}
STATICFILES_DIRS = '()'
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/opt/mailman/web/static'
STATIC_URL = '/static/'
STORAGES = {'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'}, 'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage'}}
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_DEPRECATED_PYTZ = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = False
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'mailman_web.wsgi.application'
X_FRAME_OPTIONS = 'DENY'
YEAR_MONTH_FORMAT = 'F Y'
--
Bryan Fields
727-409-1194 - Voice
http://bryanfields.net
2 years, 1 month
Re: Hyperkitty fulltext search performances
by Mark Dadgar
On Oct 4, 2020, at 8:13 AM, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 10/4/20 8:07 AM, Brian Carpenter wrote:
>>
>> On 10/4/20 8:59 AM, Loïc Dachary wrote:
>>> After a little digging it turns out the mailman3-web Debian GNU/Linux
>>> package I'm using is configured to use Woosh as a haystack backend.
>>> And it is not fit for volumes greater than a few hundred mega bytes. I
>>> should switch to something else from the list of supported
>>> backendshttps://django-haystack.readthedocs.io/en/master/backend_support.html
>>>
>> For our servers running Postorius/Hyperkitty, we use Xapian. I believe
>> the mailman-users (MM3) list also uses Xapian. It is a lot quicker. For
>> our Affinity/Empathy servers, we use Elasticsearch which we are very
>> happy with. Elasticsearch does have a higher memory requirement I
>> believe than Xapian.
>
>
> Brian is correct about this list. We started with Whoosh, then went to
> Elasticsearch and ultimately to Xapian. Each step gave better
> performance and a smaller footprint than the one before.
+1
I have ~700K posts in my archive and Xapian is FAST.
- Mark
-----
mark(a)pdc-racing.net | 408-348-2878
5 years, 10 months
Re: Exception with processing subscription confirmation email
by Eric Broens
Thanks Mark,
We are indeed running mailman core 3.3.2 and using mysql.
The data appears indeed to be truncated judging from the missing closing
curly bracket (see below).
Would it be OK to already extend the data column to VARCHAR(510) before
upgrading to mailman core 3.3.3, or will this cause problems with upgrading
to a newer version?
+---------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
------------------------------+
| data
|
+---------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
------------------------------+
| {"pre_approved": false, "pre_confirmed": false, "pre_verified": false,
"invitation": false, "address_key": "******************@************",
"subscriber_key": 1, "user_key": "a44e8b9a34c64775b712f383d438ce1d",
"token_owner_key": 1, "send_welcome_message" |
+---------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
------------------------------+
Best Regards,
Eric
-----Original Message-----
From: Mark Sapiro [mailto:mark@msapiro.net]
Sent: donderdag 29 april 2021 18:43
To: mailman-users(a)mailman3.org
Subject: [MM3-users] Re: Exception with processing subscription confirmation
email
On 4/29/21 7:56 AM, Eric Broens wrote:
> Hi,
>
> A new user tried to subscribe to a mailinglist, received the confirmation
> mail, and replied to it.
>
> The subscriber is however not subscribed and remains listed in the
"pending
> confirmation" list.
>
> The following exception appears in the error.log.
>
> Does anyone knows how to resolve this? In the past we didn't experience
any
> issues and no changes but security updates have been installed.
>
> Apr 29 16:15:55 2021 (1581) deque: do_confirm_verify
>
> Traceback (most recent call last):
>
> File
> "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/workflow.py",
> line 69, in __next__
>
> return step()
>
> File
>
"/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/subscriptions.p
> y", line 356, in _step_send_confirmation
>
> raise StopIteration
>
> StopIteration
Tracebacks like the above are normal and probably shouldn't be logged at
all.
> Apr 29 16:16:28 2021 (1490) Uncaught runner exception: Expecting ':'
> delimiter: line 1 column 256 (char 255)
>
> Apr 29 16:16:28 2021 (1490) Traceback (most recent call last):
>
> File
> "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py",
> line 173, in _one_iteration
>
> self._process_one_file(msg, msgdata)
>
> File
> "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/core/runner.py",
> line 266, in _process_one_file
>
> keepqueued = self._dispose(mlist, msg, msgdata)
>
> File
>
"/opt/mailman/venv/lib64/python3.6/site-packages/mailman/runners/command.py"
> , line 208, in _dispose
>
> mlist, msg, msgdata, parts, results)
>
> File
>
"/opt/mailman/venv/lib64/python3.6/site-packages/mailman/commands/eml_confir
> m.py", line 54, in process
>
> mlist).confirm(token)
>
> File
>
"/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/subscriptions.p
> y", line 578, in confirm
>
> workflow.restore()
>
> File
> "/opt/mailman/venv/lib64/python3.6/site-packages/mailman/app/workflow.py",
> line 146, in restore
>
> data = json.loads(state.data)
>
> File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
>
> return _default_decoder.decode(s)
>
> File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
>
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>
> File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
>
> obj, end = self.scan_once(s, idx)
>
> json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 256
> (char 255)
>
> Apr 29 16:16:28 2021 (1490) SHUNTING:
> 1619705788.5856323+57a1a759275c59b732cfa8e2985f76cfcc27dda5
This one is the issue. There was some problem with the user's
confirmation reply.
View the shunted message with the command
```
mailman qfile
path/to/var/queue/shunt/1619705788.5856323+57a1a759275c59b732cfa8e2985f76cfc
c27dda5.pck
```
although that probably won't show anything wrong as it appears that the
issue is with the saved workflow. You can get the token from the shunted
message and then look at the database with something like
```
SELECT data FROM workflowstate WHERE token = '<the token>'
```
but I suspect the issue is because the database is MySQL or MariaDB,
your Mailman core is older than 3.3.3 and your database server mode is
such that data too long for column results in truncation, and the data
in this case is truncated to 255 bytes. In Mailman 3.3.3, this column
was changed from VARCHAR(255) to VARCHAR(510) for MySQL.
As far as subscribing the user is concerned, you can in Postorius view
the subscriptions waiting user confirmation and delete this one and then
`mass subscribe` the user.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org
To unsubscribe send an email to mailman-users-leave(a)mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
5 years, 4 months
Re: Can't deliver mails in queue
by Mark Sapiro
On 5/20/21 4:01 AM, Nathanael Schneider wrote:
>
> It would have been nice though to have a more refined error to figure
> this out. But its solved now and hopefully this will help someone else
> as well.
I'm glad you resolved it, and I'm sorry I didn't consider the template
issue myself. You actually had two separate template issues. If you
created templates in Postorius, they were inaccessible because of the
POSTORIUS_TEMPLATE_BASE_URL issue, but your initial problem with
imported lists is different. It probably resulted from running the
`mailman import21` command as root which you should never do. You should
always run `mailman` commands as the Mailman user. This is mentioned at
https://docs.mailman3.org/en/latest/migration.html#id1
The `mailman import21` command can create list specific templates in
Mailman's var/templates/lists/LIST_ID/LC/ directory and if run as root
these will be user:group root:root and not world readable and therefore
can't be read by Mailman. Also, deleting and recreating the list with
the same LIST_ID doesn't remove those templates, so the list still
doesn't work.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
5 years, 3 months
Re: modify subscription page by mailinglist?
by Stephen J. Turnbull
Lars Schimmer writes:
> And I think I could fiddle out how it works. But in general, e.g. a
> custom page per mailinglist for the "list information page" is a bit
> more work than expected and I will not follow this idea.
> (e.g. adding a logo per list, ...)
Just adding a list-specific logo is not hard. Choose a naming
convention (probably something like f"logo-{list_id}.png"), put the
logo PNG in one of the static directories and add
<IMG SRC="logo-{{list_id}}.png" ALT="Logo for {{list_id}} list." />
in the right place in each template (maybe even "base.html", but more
likely an intermediate template specifically to allow putting such
customizations in one place for list-specific pages). Adding the
ability for a list admin to easily upload a file and set the logo to
it would require surgery to Postorius's Python code, though.
I will admit that though I think this is a cool idea, I am not
interested in doing it myself. But if your interest is at the "I'd
rather wait a few months than do myself", GSoC is coming up again, and
you could lobby for it when the students start applying. ;-)
Steve
4 years, 7 months
Re: Please help - Held message is crashing mailman.
by Mark Sapiro
On 4/5/19 10:27 PM, Alex King via Mailman-users wrote:
> Hi,
>
> I've used mailman a lot in the past, and decided to install mailman3 in
> a recent install. I'm unimpressed so far, it seems buggy.
>
> Running on Ubuntu 18.04.2 LTS from packages, mailman3 3.1.1-9,
> mailman3-web 0+20170523-14, python-django-postorius 1.1.2-3.
>
> After a few messages, I now can't go to the "Held Messages" in
> postorius, I get a 500 error.
>
> I tried to interact on the command line, but could not work out how.
> Reading
> https://mailman.readthedocs.io/en/latest/src/mailman/docs/install.html
> and
> https://mailman.readthedocs.io/en/latest/src/mailman/commands/docs/shell.ht…,
> I tried:
>
> # mailman shell
> Welcome to the GNU Mailman shell
>
>>>> command = cli('mailman.commands.cli_withlist.shell')
> Traceback (most recent call last):
> File "/usr/lib/python3.6/code.py", line 91, in runcode
> exec(code, self.locals)
> File "<console>", line 1, in <module>
> NameError: name 'cli' is not defined
>
> I did a lot of reading of the manual and googled, but I have no idea
> where the cli object is supposed to come from.....? The documentation
> is confusing.
The line
command = cli('mailman.commands.cli_withlist.shell')
in
<https://mailman.readthedocs.io/en/latest/src/mailman/commands/docs/shell.ht…>
is dependent upon some behind the scenes setup to make the doctests
work. Granted this is confusing, but basically all that does is set
things up for the doctests to run the 'mailman shell' command. It is not
intended to be run from within 'mailman shell'.
> Anyway, after some trial and error, i found:
>
> # mailman shell
> Welcome to the GNU Mailman shell
>>>> list_manager = getUtility(IListManager)
>>>> m=list_manager.get("committee(a)[redacted].org")
>>>> from mailman.interfaces.requests import IListRequests
>>>> requests = IListRequests(m)
>>>> [x for x in requests.held_requests][0].id
> 8
>>>> requests.get_request(8)
> Traceback (most recent call last):
> File "/usr/lib/python3.6/code.py", line 91, in runcode
> exec(code, self.locals)
> File "<console>", line 1, in <module>
> File "/usr/lib/python3/dist-packages/mailman/database/transaction.py",
> line 85, in wrapper
> return function(args[0], config.db.store, *args[1:], **kws)
> File "/usr/lib/python3/dist-packages/mailman/model/requests.py", line
> 120, in get_request
> result.data_hash, expunge=False)
> File "/usr/lib/python3/dist-packages/mailman/database/transaction.py",
> line 85, in wrapper
> return function(args[0], config.db.store, *args[1:], **kws)
> File "/usr/lib/python3/dist-packages/mailman/model/pending.py", line
> 138, in confirm
> value = json.loads(keyvalue.value)
> File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
> return _default_decoder.decode(s)
> File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
> File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
> obj, end = self.scan_once(s, idx)
> json.decoder.JSONDecodeError: Unterminated string starting at: line 1
> column 1 (char 0)
>
> It seems something was written out to disk that can't be read in for
> some reason (assuming a json object on disk?)
This is <https://gitlab.com/mailman/mailman/issues/385> which should be
fixed by <https://gitlab.com/mailman/mailman/merge_requests/333>. The
fix in in Mailman core version 3.2.0
> How do I debug this further? How do I find the json being decoded? Any
> help would be appreciated. (I found mailman2 just worked, I was happy
> with that.)
The issue is in storing a pickled representation of the json encoding of
'rule_misses'. This gets very long if you have lots of header filters,
and with at least the mysql backend gets truncated in the database which
causes the Unterminated string error.
I'm not sure what your options are for upgrading. The current Debian
packages are all >= 3.2.0 [1], but Ubuntu doesn't get 3.2.0 until cosmic
(18.10) [2].
[1] https://packages.debian.org/search?keywords=mailman3
[2] https://packages.ubuntu.com/search?keywords=mailman3
> (My colleague posted a bug about this I believe but I don't have the
> link to that.)
I haven't seen it.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
7 years, 4 months