Re: Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
by David Partain
Hi Mark,
Thank you for your kind answer. While I've been running a mailman site for many years, I'm just getting going with mailman3 (and migrating my 1400 lists...) and I feel lost occasionally :)
I wrote:
> I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to:
>
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'mailman3web',
> 'USER': 'mailman3web',
> 'PASSWORD': 'xxxxxxxxxxxxxxxxxx',
> 'HOST': 'localhost',
> 'PORT': '',
> 'OPTIONS': {
> 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
> 'charset': 'utf8mb4',
> },
> }
> }
>
> and restarted stuff, but I cannot tell that it has changed anything.
You responded:
> That should do it. I think the original message should have been saved
> as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory.
That directory has all of the messages sent to that list:
# ls /var/lib/mailman3/archives/hyperkitty/spool/
1611827925.8030825+925c996b9b6020ee18e9a689ce23384855159f77.pck 1611827926.1923075+e728f3c29c1fe6eeeca641314cdba7007cb6e132.pck
1611827925.8917305+d16b7004bdbbef98ce40fb8ee70e492715d1607d.pck 1611827926.2623458+1aac13b50d8e7f32eb14d13e6559656aa8f3bd10.pck
1611827925.9594557+7ad2a910b5ee9142c88f0bf26c60c6df960d4262.pck 1611827926.3488555+04a540651c5f829751518433e1c54168ab63a38b.pck
1611827926.0404096+a5e745f51a9f7d541aeb770a89604be17019f3c5.pck 1611827926.4206145+de861eec6404d0c8f9f8bdb180e29bff2de7c52b.pck
1611827926.1167526+0aa63639a4ec102bf63522827637e75e6a2b092a.pck
> If this is the case, it would have been retried and produced the same
> error with each new post, and after you made the above change, it would
> have been archived and the errors stopped.
> Are you saying you are still seeing these errors? If so, it may be a
> different error. What is the exact error message? We don't need all the
> Django stuff, just the part equivalent to
> Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
> OperationalError at /hyperkitty/api/mailman/archive
> (1366, "Incorrect string value: '\xF0\x9F\xA4\xA3\xF0\x9F...' for
> column mailman3web.hyperkitty_email.content at row 1")
Yes, I'm still seeing the errors. When I send mail to a list with a cute emoji, I see the following in /var/log/mailman3/web/mailman-web.log:
ERROR 2021-01-28 10:58:45,708 16923 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
(bunch of python errors omitted)
DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column 'content' at row 1")
Also, a bunch of mail is sent to root, presumably one for every message that has previously failed, with the same error information, plus information about the POST method done. (If something there would be helpful, please let me know.)
Finally, I'm doing all of this on Ubuntu, with the packages available in the distro, which means:
mailman3 3.1.1-9
mailman3-doc 3.1.1-9
mailman3-full 3.1.1-9
mailman3-web 0+20170523-14
python-django-hyperkitty 1.1.4-4
python-django-mailman3 1.1.0-4
python-django-postorius 1.1.2-3
python-mailmanclient 3.1.1-5
python3-mailman-hyperkitty 1.1.0-4
Thanks very much for any help you can offer.
Cheers,
David
5 years, 8 months
Re: Reg Archive Inactive Status
by Nirmal J
Yes I have it. I am also sharing that also settings_local.py file
# This file is imported by the Mailman Suite. It is used to override
# the default settings from /usr/share/mailman3-web/settings.py.
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'vWzgIJ1Hvjgy4yXJz7l71vpxX6+pJcBLMdv8mWNNAd0ajVr1'
ADMINS = (
('Mailman Suite Admin', 'nirmal(a)iitm.ac.in'),
)
# Hosts/domain names that are valid for this site; required if DEBUG is False
# See https://docs.djangoproject.com/en/1.8/ref/settings/#allowed-hosts
# Set to '*' per default in the Deian package to allow all hostnames. Mailman3
# is meant to run behind a webserver reverse proxy anyway.
ALLOWED_HOSTS = [
#"localhost", # Archiving API from Mailman, keep it.
# "lists.your-domain.org",
# Add here all production URLs you may have.
'*'
]
# Mailman API credentials
MAILMAN_REST_API_URL = 'http://localhost:8001'
MAILMAN_REST_API_USER = 'restadmin'
MAILMAN_REST_API_PASS = '/EqNGU+Mj8udFpw4ueIkhWXzAy3kJIy17wwzChrXb3P8Vh4e'
MAILMAN_ARCHIVER_KEY = 'n/ZZnR5bRgNdUEyNK4p731C1+gWJLw0/'
MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1', '10.24.5.52')
# Application definition
INSTALLED_APPS = (
'hyperkitty',
'postorius',
'django_mailman3',
# Uncomment the next line to enable the admin:
'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
# 'django.contrib.admindocs',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor', 'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount',
'django_mailman3.lib.auth.fedora',
#'allauth.socialaccount.providers.openid',
#'allauth.socialaccount.providers.github',
#'allauth.socialaccount.providers.gitlab',
#'allauth.socialaccount.providers.google',
#'allauth.socialaccount.providers.facebook',
#'allauth.socialaccount.providers.twitter',
#'allauth.socialaccount.providers.stackexchange',
)
# Database
# https://docs.djangoproject.com/en/1.8/ref/settings/#databases
DATABASES = {
'default': {
# Use 'sqlite3', 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'ENGINE': 'django.db.backends.sqlite3',
#'ENGINE': 'django.db.backends.postgresql_psycopg2',
#'ENGINE': 'django.db.backends.mysql',
# DB name or path to database file if using sqlite3.
'NAME': '/var/lib/mailman3/web/mailman3web.db',
# The following settings are not used with sqlite3:
'USER': '',
'PASSWORD': '',
# HOST: empty for localhost through domain sockets or '127.0.0.1' for
# localhost through TCP.
'HOST': '',
# PORT: set to empty string for default.
'PORT': '',
# OPTIONS: Extra parameters to use when connecting to the database.
'OPTIONS': {
# Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See
# https://docs.djangoproject.com/en/1.11/ref/ # databases/#setting-sql-mode
#'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
},
}
}
# If you're behind a proxy, use the X-Forwarded-Host header
# See https://docs.djangoproject.com/en/1.8/ref/settings/#use-x-forwarded-host
USE_X_FORWARDED_HOST = True
# And if your proxy does your SSL encoding for you, set SECURE_PROXY_SSL_HEADER
# https://docs.djangoproject.com/en/1.8/ref/settings/#secure-proxy-ssl-header
# SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_SCHEME', 'https')
# Other security settings
# SECURE_SSL_REDIRECT = True
# If you set SECURE_SSL_REDIRECT to True, make sure the SECURE_REDIRECT_EXEMPT
# contains at least this line:
# SECURE_REDIRECT_EXEMPT = [
# "archives/api/mailman/.*", # Request from Mailman.
# ]
# SESSION_COOKIE_SECURE = True
# SECURE_CONTENT_TYPE_NOSNIFF = True
# SECURE_BROWSER_XSS_FILTER = True
# CSRF_COOKIE_SECURE = True
# CSRF_COOKIE_HTTPONLY = True
# X_FRAME_OPTIONS = 'DENY'
# Internationalization
# https://docs.djangoproject.com/en/1.8/topics/i18n/ LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Set default domain for email addresses.
EMAILNAME = 'list1.iitm.ac.in'
# If you enable internal authentication, this is the address that the emails
# will appear to be coming from. Make sure you set a valid domain name,
# otherwise the emails may get rejected.
# https://docs.djangoproject.com/en/1.8/ref/settings/#default-from-email
# DEFAULT_FROM_EMAIL = "mailing-lists(a)you-domain.org"
DEFAULT_FROM_EMAIL = 'postorius@{}'.format(EMAILNAME)
# If you enable email reporting for error messages, this is where those emails
# will appear to be coming from. Make sure you set a valid domain name,
# otherwise the emails may get rejected.
# https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SERVER_EMAIL
# SERVER_EMAIL = 'root(a)your-domain.org'
SERVER_EMAIL = 'root@{}'.format(EMAILNAME)
# Django Allauth
ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https"
#
# Social auth
#
SOCIALACCOUNT_PROVIDERS = {
#'openid': {
# 'SERVERS': [
# dict(id='yahoo',
# name='Yahoo',
# openid_url='http://me.yahoo.com'),
# ],
#},
#'google': {
# 'SCOPE': ['profile', 'email'],
# 'AUTH_PARAMS': {'access_type': 'online'},
#}, #'facebook': {
# 'METHOD': 'oauth2',
# 'SCOPE': ['email'],
# 'FIELDS': [
# 'email',
# 'name',
# 'first_name',
# 'last_name',
# 'locale',
# 'timezone',
# ],
# 'VERSION': 'v2.4',
#},
}
# On a production setup, setting COMPRESS_OFFLINE to True will bring a
# significant performance improvement, as CSS files will not need to be
# recompiled on each requests. It means running an additional "compress"
# management command after each code upgrade.
# http://django-compressor.readthedocs.io/en/latest/usage/#offline-compression
COMPRESS_OFFLINE = True
POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/'
SITE_ID = 2
import ldap
from django_auth_ldap.config import LDAPSearch # LDAP server URI
AUTH_LDAP_SERVER_URI = "ldap://127.0.0.1"
# Bind DN & password (service account from IITM LDAP admin)
AUTH_LDAP_BIND_DN = "cn=ldapreader,ou=Services,dc=iitm,dc=ac,dc=in"
AUTH_LDAP_BIND_PASSWORD = "642126"
# Where to search for users
AUTH_LDAP_USER_SEARCH = LDAPSearch(
"ou=People,dc=iitm,dc=ac,dc=in",
ldap.SCOPE_SUBTREE,
"(uid=%(user)s)"
)
# Map LDAP attributes to Django user fields
AUTH_LDAP_USER_ATTR_MAP = {
"first_name": "cn",
"last_name": "sn",
# "email": "mail" # optional if not present
}
# Authentication backends (LDAP first, fallback to Django DB)
AUTHENTICATION_BACKENDS = [
"django_auth_ldap.backend.LDAPBackend",
"django.contrib.auth.backends.ModelBackend",
]
import logging
logger = logging.getLogger('django_auth_ldap')
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler())
-----Original Message-----
From: Nirmal <nirmal(a)iitm.ac.in>
To: Stephen <steve(a)turnbull.jp>
Cc: mailman-users <mailman-users(a)mailman3.org>
Date: Friday, 26 September 2025 3:27 PM IST
Subject: Re: [MM3-users] Re: Reg Archive Inactive Status
Yes I have it. I am also sharing that also settings_local.py
-----Original Message-----
From: Stephen <steve(a)turnbull.jp>
To: Nirmal <nirmal(a)iitm.ac.in>
Cc: mailman-users <mailman-users(a)mailman3.org>
Date: Friday, 26 September 2025 3:09 PM IST
Subject: [MM3-users] Re: Reg Archive Inactive Status
Nirmal J via Mailman-users writes:
> Hereby I am sharing the full settings.py file.
Are you sure this is the settings.py being used by your installation?
It appears to be the distribution file, unchanged.
> MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')# Application definition
If this is from the installed settings.py, it isn't going to work,
since you specified a routable domain (I think, although
"list1.iitm.ac.in" is not in the public DNS), which will have a
routable address, not localhost.
What IP does your DNS report for "list1.iitm.ac.in"?
> HOSTNAME = 'localhost.local'
That isn't what you told Mailman via hyperkitty.cfg. I'm not sure it
matters, but I think you should make sure all the config files agree
on the name of the host.
> # If you enable email reporting for error messages, this is where those emails
> # will appear to be coming from. Make sure you set a valid domain name,
> # otherwise the emails may get rejected.
> # https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SERVER_EMAIL
> # SERVER_EMAIL = 'root(a)your-domain.org'
> SERVER_EMAIL = 'root@{}'.format(HOSTNAME)
This can be problematic, at least if you enable error messages by email.
> try:
> from settings_local import *
> except ImportError:
> pass
Do you have a "settings_local.py" file?
--
GNU Mailman consultant (installation, migration, customization)
Sirius Open Source https://www.siriusopensource.com/
Software systems consulting in Europe, North America, and Japan
1 year
Daily Cronjob fails
by Stefan Tatschner
Hey hey,
I applied a recent Update from Debian Buster, und this night a received an email about a failing cronjob.
The traceback can be reproduced using:
# sudo -u www-data flock -n /var/run/mailman3-web/cron.daily /usr/share/mailman3-web/manage.py runjobs daily
ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty)
START TRACEBACK:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
psycopg2.IntegrityError: null value in column "description" violates not-null constraint
DETAIL: Failing row contains (2, hb-events(a)hackbrettl.de, hb-events, null, [hb-events] , 1, 2015-10-22 09:54:53.684306+00, hb-events.hackbrettl.de).
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/django_extensions/management/commands/runjobs.py", line 36, in runjobs
job().execute()
File "/usr/lib/python3/dist-packages/hyperkitty/jobs/sync_mailman.py", line 35, in execute
sync_with_mailman()
File "/usr/lib/python3/dist-packages/hyperkitty/lib/mailman.py", line 131, in sync_with_mailman
mlist.update_from_mailman()
File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py", line 192, in update_from_mailman
self.save()
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 808, in save
force_update=force_update, update_fields=update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 838, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 905, in _save_table
forced_update)
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 955, in _do_update
return filtered._update(values) > 0
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 667, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
raise original_exception
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python3/dist-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: null value in column "description" violates not-null constraint
DETAIL: Failing row contains (2, hb-events(a)hackbrettl.de, hb-events, null, [hb-events] , 1, 2015-10-22 09:54:53.684306+00, hb-events.hackbrettl.de).
END TRACEBACK
Appearently there is something wrong with the database. Is there anything what I can do about this?
Running the migrations doesn't help:
# sudo -u www-data flock -n /var/run/mailman3-web/cron.daily /usr/share/mailman3-web/manage.py migrate
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount
Running migrations:
No migrations to apply.
Thanks!
Stefan
7 years, 7 months
Setting for archiving multi email domains?
by Stephen J. Turnbull
Duane Raymond writes:
> I have a Debian-postfix-postgres-nginx-Hyperkitty-Postorius setup
> with a primary domain (e.g. primarydomain.com) and several
> 'secondary' domains used for email lists (e.g. emaillistA.com and
> emaillistB.com) When sending emails to a list using the
> primarydomain.com address, the email is received, sent to
> subscribers and archived properly. But when it is sent to one of
> the secondary domains like tes(a)emaillistA.com - the email doesn't
> seem to be sent and isn't archived - despite it being set to be.
What do you mean by "set"? That the list has archiving enabled?
What is in the Postfix logs? Where does Postfix try to deliver the
second class of email (list-specific domains)? Is there a "bounce"
(more precisely, a Delivery Status Notice telling you that the mail
was not delivered, and with luck, why Postfix had a problem)? What is
in the Mailman logs?
I assume from your description (one server, multiple domains) that
these list-specific domains are "virtual" domains that are handled by
the single server. It seems likely that something is going wrong with
the routing in Postfix.
> What I've tried so far (after searching these archives and looking
> at the documentation) is to:
> 1) Adding emaillistA.com and emaillistB.com to the
> MAILMAN_ARCHIVER_FROM setting in the settings_local.py file (and
> then restart mailman)
> 2) Adding emaillistA.com and emaillistB.com to the mydestination
> setting in the postfix main.cf file (and the restarting
> postfix).
(2) above will allow receiving mail for those domains, but is unlikely
to be sufficient for routing to lists. You also need to inform
Mailman about the virtual domains, because the internal name of the
mailing list includes the domain.
Postfix has extensive support for virtual domains. This is somewhat
tricky. Are you familiar with this support, as documented in
http://www.postfix.org/VIRTUAL_README.html, and the general docs? For
Mailman's needs from Postfix, see
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h….
If the above aren't sufficient hints, we need more information about
both the Postfix configuration and the Mailman configuration.
In a later message
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…,
you report at the end of the traceback (line folded for readability):
> AttributeError: 'NoneType' object has no attribute 'preferences'
> Mar 04 14:08:24 2021 (5044) SHUNTING:
> 1614866904.2741017+f1f320a5d0006e885586be5b33543e364e3829b4
This seems to indicate that either your list has a member that doesn't
exist in Mailman's subscriber database or (maybe) that there are no
subscribers, because it's trying to get profile information about a
nonexistent list member (represented by the Python 'None' object).
Did this occur only once? How many messages are in the "shunt" queue?
5 years, 7 months
re-sending confirmation mails (was Re: Re: pending subscriptions lost?)
by IEM Network Operation Center (IOhannes m zmölnig)
On 3/14/22 18:36, Mark Sapiro wrote:
> On 3/14/22 09:18, IEM Network Operation Center (IOhannes m zmölnig) wrote:
>>
>> this is with "Postorius Version 1.2.4" (as installed by the mailman3
>> Debian package).
>
> The Debian packages are OK for people who want to install things via a
> package manager, but they are quite out of date.
i know.
the problem is, that right now i have a running (Debian-based, thus
outdated) mailman3 installation, and I still haven't found a proper
strategy to upgrade and keep staying up-to-date.
so for the time being i'm stuck with what i have.
>
> Something like the following in `mailman shell` can do it.
thanks a lot.
afaiu, this snippet allows me to purge outdated pending subscriptions.
modifying it slightly, also allows me to just list the pending
subscriptions along with their tokens and print them out.
is it still possible to just use these tokens to confirm the pending
subscription (even though it is long overdue)?
if so, i could just send out a reminder to those 50 or so people who
have not confirmed yet.
while investigating this a bit, i also noticed that it seems that with
mailman3 the only option to confirm a pending subscription is via email
("leaving the subject intact") - at least that is the only option that
is listed in the confirmation mail.
now i would like to massage this into what i think is a bit more
user-friendly.
first of all: most email confirmation systems allow me to either "reply
by email" or "click this link".
is there a way to do this for mailman3? that is: is there a (hidden) URL
that would accept the token in order to confirm a subscription?
also: the confirmation mail says that the subject must be kept intact.
however, i noticed that the actual "From" address of the confirmation
mail already contains the token.
would it be enough if people just sent an email to this special
email-address (that contains the token), regardless of the subject?
i'm thinking of sending a rather personalized email that has a more
attractive subject than "confirm flkdsajfsalkjflsajfsoisjdl".
finally: what else is required to approve the confirmation? will the
sender's email-address be taken into account? (e.g. if they subscribed
"foo+mylist(a)example.com" but send the confirmation-reply from
"foo(a)example.com", will that still work?
sorry for so many questions.
i *did* search the documentation, but simply haven't found anything that
would help me.
i've probably looked in the wrong places, but mailman3 is rather huge so
it's easy to miss the imprtant parts.
gmdsa
IOhannes
4 years, 7 months
Re: Hyperkitty "All Threads" results in error
by Mark Dadgar
Anyone have thoughts on this? Anyone? Buehler? Anyone?
- Mark
-----
mark(a)pdc-racing.net | 408-348-2878
> On Oct 4, 2020, at 10:34 AM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>
> So I have been wrestling with this problem for a while now. Most times, but not every time, when someone clicks on the All Threads link in hyperkitty, a 500 error is generated.
>
> I googled it, obviously, and at first I thought it was related to this issue, but that has not proven to be the case:
>
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…>
>
> I’ve attached a “backtrace” below.
>
> The archives were imported from a previous mm3 install a couple of months ago. The problem occurs even with a list state full of immediately recent posts.
>
> Anyone have thoughts on how I can fix this?
>
> - Mark
> -----
> mark(a)pdc-racing.net | 408-348-2878
>
> —
>
> Internal Server Error: /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
>
> VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
> Failed lookup for key [sender] in None
>
> Request Method: GET
> Request URL: https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest <https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest>
> Django Version: 2.2.14
> Python Executable: /opt/mailman/mm/venv/bin/python
> Python Version: 3.8.2
> Python Path: ['/opt/mailman/mm', '/opt/mailman/mm/', '/opt/mailman/mm/bin', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/opt/mailman/mm/venv/lib/python3.8/site-packages']
> Server time: Sun, 4 Oct 2020 10:27:10 -0700
> Installed Applications:
> ('hyperkitty',
> 'postorius',
> 'django_mailman3',
> 'django.contrib.admin',
> 'django.contrib.auth',
> 'django.contrib.contenttypes',
> 'django.contrib.sessions',
> 'django.contrib.sites',
> 'django.contrib.messages',
> 'django.contrib.staticfiles',
> 'rest_framework',
> 'django_gravatar',
> 'compressor',
> 'haystack',
> 'django_extensions',
> 'django_q',
> 'allauth',
> 'allauth.account',
> 'allauth.socialaccount')
> Installed Middleware:
> ('django.contrib.sessions.middleware.SessionMiddleware',
> 'django.middleware.common.CommonMiddleware',
> 'django.middleware.csrf.CsrfViewMiddleware',
> 'django.middleware.locale.LocaleMiddleware',
> 'django.contrib.auth.middleware.AuthenticationMiddleware',
> 'django.contrib.messages.middleware.MessageMiddleware',
> 'django.middleware.clickjacking.XFrameOptionsMiddleware',
> 'django.middleware.security.SecurityMiddleware',
> 'django_mailman3.middleware.TimezoneMiddleware',
> 'postorius.middleware.PostoriusMiddleware')
>
>
> Traceback:
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _resolve_lookup
> 829. current = current[bit]
>
> During handling of the above exception ('NoneType' object is not subscriptable), another exception occurred:
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _resolve_lookup
> 837. current = getattr(current, bit)
>
> During handling of the above exception ('NoneType' object has no attribute 'sender'), another exception occurred:
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _resolve_lookup
> 843. current = current[int(bit)]
>
> During handling of the above exception (invalid literal for int() with base 10: 'sender'), another exception occurred:
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/exception.py" in inner
> 34. response = get_response(request)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py" in _get_response
> 115. response = self.process_exception_by_middleware(e, request)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py" in _get_response
> 113. response = wrapped_callback(request, *callback_args, **callback_kwargs)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/lib/view_helpers.py" in inner
> 134. return func(request, *args, **kwargs)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py" in archives
> 114. return _thread_list(request, mlist, threads, extra_context=extra_context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py" in _thread_list
> 143. return render(request, template_name, context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/shortcuts.py" in render
> 36. content = loader.render_to_string(template_name, context, request, using=using)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader.py" in render_to_string
> 62. return template.render(context, request)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py" in render
> 61. return self.template.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 171. return self._render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _render
> 163. return self.nodelist.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 937. bit = node.render_annotated(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render_annotated
> 904. return self.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py" in render
> 150. return compiled_parent._render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _render
> 163. return self.nodelist.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 937. bit = node.render_annotated(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render_annotated
> 904. return self.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py" in render
> 62. result = block.nodelist.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 937. bit = node.render_annotated(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render_annotated
> 904. return self.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py" in render
> 209. nodelist.append(node.render_annotated(context))
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render_annotated
> 904. return self.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py" in render
> 188. return template.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 173. return self._render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _render
> 163. return self.nodelist.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 937. bit = node.render_annotated(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render_annotated
> 904. return self.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py" in render
> 513. return self.nodelist.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render
> 937. bit = node.render_annotated(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in render_annotated
> 904. return self.render(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py" in render
> 132. context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py" in <dictcomp>
> 132. context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in resolve
> 692. arg_vals.append(arg.resolve(context))
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in resolve
> 796. value = self._resolve_lookup(context)
>
> File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py" in _resolve_lookup
> 848. raise VariableDoesNotExist("Failed lookup for key "
>
> Exception Type: VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
> Exception Value: Failed lookup for key [sender] in None
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
6 years
Re: Twitter OAuth2
by Odhiambo Washington
On Thu, Dec 29, 2022 at 3:44 AM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/28/22 11:37, Odhiambo Washington wrote:
> > On Wed, Dec 28, 2022 at 8:23 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> >
> >> On 12/27/22 21:46, Odhiambo Washington wrote:
> >>> On Tue, Dec 27, 2022 at 8:08 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> >>>>
> >>>> I see from another post that you've decided against using Twitter, but
> >>>> to answer the question, Twitter does not require adding anything to
> >>>> SOCIALACCOUNT_PROVIDERS.
> >>>>
> >>>
> >>> I tried it and ended up with:
> >>>
> >>> ERROR 2022-12-28 08:40:18,895 97440 django.request Internal Server
> Error:
> >>> /accounts/twitter/login/
> >>> Traceback (most recent call last):
> >> ...
> >>> allauth.socialaccount.models.SocialApp.DoesNotExist: SocialApp matching
> >>> query does not exist.
> >>
> >>
> >> Do you have
> >>
> >> 'allauth.socialaccount.providers.twitter',
> >>
> >> in INSTALLED_APPS?
> >>
> >
> > Here is what I have in settings_local.py:
> >
> > INSTALLED_APPS = (
> > 'hyperkitty',
> > 'postorius',
> > 'django_mailman3',
> > # Uncomment the next line to enable the admin:
> > 'django.contrib.admin',
> > # Uncomment the next line to enable admin documentation:
> > 'django.contrib.admindocs',
> > 'django.contrib.auth',
> > 'django.contrib.contenttypes',
> > 'django.contrib.sessions',
> > 'django.contrib.sites',
> > 'django.contrib.messages',
> > 'django.contrib.staticfiles',
> > 'rest_framework',
> > 'django_gravatar',
> > 'compressor',
> > 'haystack',
> > 'django_extensions',
> > 'django_q',
> > 'allauth',
> > 'allauth.account',
> > 'allauth.socialaccount',
> > #'django_mailman3.lib.auth.fedora',
> > #'allauth.socialaccount.providers.openid',
> > 'allauth.socialaccount.providers.github',
> > #'allauth.socialaccount.providers.gitlab',
> > 'allauth.socialaccount.providers.google',
> > 'allauth.socialaccount.providers.facebook',
> > 'allauth.socialaccount.providers.twitter',
> > #'allauth.socialaccount.providers.stackexchange',
> > )
>
>
> Did you add an app at https://site.name/admin/socialaccount/socialapp/
> with twitter as a provider? See
>
> https://django-allauth.readthedocs.io/en/latest/providers.html#twitter-oaut…
Exactly, and I have entered all the credentials from the "OAuth 2.0 Client
ID and Client Secret"
of the twitter developer portal.
https://webmail.kictanet.or.ke/~wash/twitter-oauth.png
I have checked my settings_local.py several times trying to find out why
the entry is ignored.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
3 years, 9 months
Re: Digests not working correctly
by Joel Lord
The May 4th digest that went out was _also_ size-triggered, so this may
have nothing to do with periodic digests at all, and possibly my
periodic digests aren't working. I'm not on any of my own lists in
digest mode, I'm slowly extracting diagnostic information out of people
who are. Also, since this is a ~2 month cycle, it's really difficult to
get data points to work with. I'll need to remember to go in and look
when this settles down again (new cycle of activity started last night)
to see if there's anything left pending.
(venv) root@host2:/home/lists/mailman/venv/bin# pip freeze | grep -i hyper
HyperKitty==1.3.7
On 6/4/2023 10:05 PM, Mark Sapiro wrote:
> On 6/4/23 18:35, Joel Lord wrote:
>>
>> The periodic digests do seem to be coming out. I also now have
>> confirmation that the one message in this morning's digest that was
>> from May 4th was also included in the last digest back on May 4th, so
>> it seems that the one message was left behind in the digest queue when
>> the periodic digest was sent.
>
> I don't see how that can happen. The process that sends a digest renames
> the var/lists/<list-id>/digest.mmdf mailbox file in which the messages
> are accumulated to var/lists/<list-id>/digest.<volume>.<issue>.mmdf,
> where <volume> and <issue> are the volume and issue numbers of that
> digest, and then queues a message in the `digest` queue to tell the
> digest runner to create the digest from the messages in that mbox and
> send it. Thus, it leaves no var/lists/<list-id>/digest.mmdf mailbox file
> behind and that is created anew when the next post arrives. Further, if
> there is a non-empty digest.mmdf file, its messages should be sent no
> later than the next 11 PM `cron digests`.
>
>
>> There was one earlier message to the list back on May 4th, before the
>> one that got duplicated, but I can't tell if that triggered a
>> size-based digest to be sent: the logs aren't clear enough on that
>> detail for me to tell >
>
> OK
>
>
>> Just to inform things:
>>
>> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep mailman
>> django-mailman3==1.3.9
>> mailman==3.3.8
>> mailman-hyperkitty==1.2.1
>> mailman-web==0.0.6
>> mailmanclient==3.3.5
>> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep hyper
>> mailman-hyperkitty==1.2.1
>
> Actually, it's HyperKitty, not hyperkitty, but I assume HyperKitty is up
> to date as are the others.
>
>> (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep post
>> postorius==1.3.8
>>
>>
>
--
Joel Lord
3 years, 4 months
using SSH/TLS with external MTA
by Roland Giesler
I have managed thus far to get things working on my new install, but I
need to use a secure logon to send mail from an external MTA. I have
set up:
/etc/mailman/mailman.cfg:
smtp_host: box2.gtahardware.co.za
smtp_port: 465
smtp_user: roland(a)giesler.za.net
smtp_pass: <hidden>
smtp_secure_mode: smtps
smtp_verify_cert: no
smtp_verify_hostname: no
I'll get a cert installed later, for now just want to get it going.
This error occurs when I try to create a new user in postorius.
ERROR 2024-07-24 15:29:45,572 2150 django.request Internal Server Error:
/accounts/login/
Traceback (most recent call last):
File
"/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line
47, in inner
response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py",
line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/views/generic/base.py",
line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/django/utils/decorators.py",
line 43, in _wrapper
return bound_method(*args, **kwargs)
File
"/usr/lib/python3/dist-packages/django/views/decorators/debug.py", line
89, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/allauth/account/views.py", line
146, in dispatch
return super(LoginView, self).dispatch(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/allauth/account/views.py", line
74, in dispatch
response = super(RedirectAuthenticatedUserMixin, self).dispatch(
File "/usr/lib/python3/dist-packages/django/views/generic/base.py",
line 98, in dispatch
return handler(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/allauth/account/views.py", line
102, in post
response = self.form_valid(form)
File "/usr/lib/python3/dist-packages/allauth/account/views.py", line
159, in form_valid
return form.login(self.request, redirect_url=success_url)
File "/usr/lib/python3/dist-packages/allauth/account/forms.py", line
196, in login
ret = perform_login(
File "/usr/lib/python3/dist-packages/allauth/account/utils.py", line
175, in perform_login
send_email_confirmation(request, user, signup=signup, email=email)
File "/usr/lib/python3/dist-packages/allauth/account/utils.py", line
346, in send_email_confirmation
email_address.send_confirmation(request, signup=signup)
File "/usr/lib/python3/dist-packages/allauth/account/models.py", line
62, in send_confirmation
confirmation.send(request, signup=signup)
File "/usr/lib/python3/dist-packages/allauth/account/models.py", line
169, in send
get_adapter(request).send_confirmation_mail(request, self, signup)
File "/usr/lib/python3/dist-packages/allauth/account/adapter.py",
line 464, in send_confirmation_mail
self.send_mail(email_template,
emailconfirmation.email_address.email, ctx)
File "/usr/lib/python3/dist-packages/allauth/account/adapter.py",
line 136, in send_mail
msg.send()
File "/usr/lib/python3/dist-packages/django/core/mail/message.py",
line 284, in send
return self.get_connection(fail_silently).send_messages([self])
File
"/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line
109, in send_messages
sent = self._send(message)
File
"/usr/lib/python3/dist-packages/django/core/mail/backends/smtp.py", line
125, in _send
self.connection.sendmail(from_email, recipients,
message.as_bytes(linesep='\r\n'))
File "/usr/lib/python3.10/smtplib.py", line 901, in sendmail
raise SMTPRecipientsRefused(senderrs)
smtplib.SMTPRecipientsRefused: {'roland(a)giesler.za.net': (454, b'4.7.1
<roland(a)giesler.za.net>: Relay access denied')}
However, the mailserver is in daily use and accept logon all the time.
I believe I'm missing something that should be installed to make SMTPS
(SSL) work, but what?
2 years, 2 months
Re: Mass operations on large list - Internal Server Error
by Liam Reilly
Stephen J. Turnbull wrote:
> Reilly, Liam writes:
> > Here is an example error from the log for unsubscribe all
> > operation.
> > Which log is this? I would guess it's the mailmanweb.log.
> > [2025-12-15 15:41:02 +0000] [1022464] [ERROR] Error handling request /mailman3/lists/uclucu-members.ucl.ac.uk/unsubscribe_all
> > Traceback (most recent call last):
> > File "/data/mailman/venv/lib64/python3.12/site-packages/gunicorn/workers/sync.py", line 134, in handle
> > self.handle_request(listener, req, client, addr)
> > This is the front-end gunicorn WSGI server, handling a request for
> Postorius.
> > File "/data/mailman/venv/lib64/python3.12/site-packages/mailmanclient/restobjects/mailinglist.py", line 519, in unsubscribe
> > response, json = self._connection.call(path, data, method='DELETE')
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > This is mailman client translating to the REST API.
> > File "/usr/lib64/python3.12/socket.py", line 720, in readinto
> > return self._sock.recv_into(b)
> > ^^^^^^^^^^^^^^^^^^^^^^^
> > This is the Python socket library actually talking to the REST API of
> Mailman core. An unspecified error occurs in the C code. It is not
> handled.
> > File "/data/mailman/venv/lib64/python3.12/site-packages/gunicorn/workers/base.py", line 204, in handle_abort
> > sys.exit(1)
> > SystemExit: 1
> > This is the "Something's wrong, hope you have a clue 'cause I don't"
> last line of defense error handler.
> My guess is that it's a timeout on a socket.
> I think you should also see log messages and a traceback in the
> mailman.log.
> The timeout for the mailmanweb gunicorn is configured in the gunicorn
> conf file, usually in /etc/mailman3. This is relatively short (I
> think my installation is default at 120 seconds).
> The timeout for the REST API gunicorn is in
> site-packages/mailman/config/gunicorn.cfg. This is long (360 seconds)
> so if my conjecture about timeout is correct it's the mailmanweb
> gunicorn that's timing out. If you need to change this one, you may
> be able to simply put "timeout: 360" in the [webservice] section of
> your mailman.cfg file, but if that doesn't work you can copy
> gunicorn.cfg to /etc/mailman3/rest-wsgi.cfg, change the timeout in it,
> and add "configuration: /etc/mailman3/rest-wsgi.cfg" to the
> [webservice] section of mailman.cfg.
> --
> GNU Mailman consultant (installation, migration, customization)
> Sirius Open Source https://www.siriusopensource.com/
> Software systems consulting in Europe, North America, and Japan
As always, thank you for your prompt replies Stephen.
The entries are from error.log and are all I could find when grepping for the list name. I'll check the other logs for anything else and also try increasing the timeout config as you've suggested to see if it helps.
10 months