Re: moving lists to another server
by Odhiambo Washington
On Tue, Nov 7, 2023 at 2:07 PM Stephen J. Turnbull <
turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote:
> bryan.kartzman(a)yu.edu writes:
>
> > I have both my mailman lists and django config in postgres
> > databases.
>
> These are Mailman 3 lists? What Django config? Do you mean Postorius
> and HyperKitty? Do you intend to use Postorius and HyperKitty on the
> new server?
>
> > If I want to move my lists to another server that is running
> > mailman 3, aside from copying the databases to the other server,
>
> Do you mean the other server is already actively distributing Mailman
> list traffic, managing users and archives? Or is it fresh instance?
>
Hello Stephen et al,
Since this topic is relevant to something I am faced with at the moment, I
believe it's okay if I piggyback on the thread.
I have a server that is already actively distributing Mailman list traffic,
and managing users and archives.
I have offered to take over another community ML (also using MM3) from
another server and host the list on my server.
I am scratching my head bald trying to figure out how to bring in this
other MM3 site and list into my existing server, especially with regard to
the DB backend.
The list - let's call it users(a)domainX.name was using MySQL backend and
had a separate DB for Mailman core and Mailman web.
I believe combining these into one isn't a problem. But getting to merge
this DB into my current one is what I am not sure about.
Any pointers on how to bring in this new site into my current setup??
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
2 years, 8 months
Can login at /admin, but cannot login at the account/login page.
by pabsta＠gmail.com
Hi All,
thanks for your contibution to opensource softwares. I have done the installation walkthrough on the wiki page (thanks to Brian Carpenter) and I am almost there. I can run the three services and they provide the relevant web interfaces (admin, postorius, hyperkitty). In comparison to the walkthrough, I did two changes; First, I used different port numbers , as it would otherwise conflict with other services on the server. Second, I disabled the social accounts authentification mechanisms. I am still running on sqllite, as I am still testing things (one task at a time).
Using my "superuser" credentials, I can log to the admin section in a fraction of a second. The server response time is less than one second. Once logged in, I can navigate through all the site as an authenticated user, including in the postorius pages related to list management. The services work normally.
However, if logout and attempt to login from the postorius website, I get a « 502 bad gateway » error after roughly 30 seconds. The logs in Systemctl for gunicorn register a "DIGEST-MD5 common mech free" error message and the nginx error logs just show a connection drop:
2021/08/12 18:11:09 [error] 26026#26026: *31 upstream prematurely closed connection while reading response header from upstream, client: [ip redacted], server:
[domain redacted], request: "POST /accounts/login/ HTTP/2.0", upstream: "http://127.0.0.1:8006/accounts/login/", host: "[domain redacted]", referrer: "https://[domain redacted]/accounts/login/?next=/mailman3/lists/"
Two additional clues may help understanding what is going on:
1- If I attempt to login with improper credentials, the services respond normally, and I get the expected red box with "invalid credentials" message on the postorius page. It is only when proper credentials are given that the connection fails.
2- If I attempt to register a new email adress, the services also fail to respond (although with no DIGEST-MD5 message registered).
It thus seems to be a post "POST" problem, but what puzzles me is that it is not an issue for the admin page.
Below the relevant sections of my settings.py page. The skipped ([...]) sections are as in the walkthrough files. In particular, the AUTH_PASSWORD_VALIDATORS, the EMAIL_BACKEND are as is and the "USE_X_FORWARDED_HOST" is still commented.
##Settings.py
[...]
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '[redacted]'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False
ADMINS = (
('ME', 'my@email'),
)
SITE_ID = 1
# Hosts/domain names that are valid for this site; required if DEBUG is False
# See https://docs.djangoproject.com/en/1.8/ref/settings/#allowed-hosts
ALLOWED_HOSTS = [
"localhost", # Archiving API from Mailman, keep it.
"[domain]",
# Add here all production URLs you may have.
]
# Mailman API credentials
MAILMAN_REST_API_URL = 'http://localhost:8005'
MAILMAN_REST_API_USER = '[redacted]'
MAILMAN_REST_API_PASS = '[redacted]'
MAILMAN_ARCHIVER_KEY = '[redacted]'
MAILMAN_ARCHIVER_FROM = ('127.0.0.1', '::1')
# Application definition
INSTALLED_APPS = (
'hyperkitty',
'postorius',
'django_mailman3',
# Uncomment the next line to enable the admin:
'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
# 'django.contrib.admindocs',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount',
#'django_mailman3.lib.auth.fedora',
#'allauth.socialaccount.providers.openid',
#'allauth.socialaccount.providers.github',
#'allauth.socialaccount.providers.gitlab',
#'allauth.socialaccount.providers.google',
#'allauth.socialaccount.providers.facebook',
#'allauth.socialaccount.providers.twitter',
#'allauth.socialaccount.providers.stackexchange',
)
[...]
# Database
# https://docs.djangoproject.com/en/1.8/ref/settings/#databases
DATABASES = {
'default': {
# Use 'sqlite3', 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'ENGINE': 'django.db.backends.sqlite3',
# DB name or path to database file if using sqlite3.
'NAME': os.path.join(BASE_DIR, 'mailmansuite.db'),
# The following settings are not used with sqlite3:
'USER': 'XXX',
'PASSWORD': 'XXX',
# HOST: empty for localhost through domain sockets or '127.0.0.1' for
# localhost through TCP.
'HOST': '',
# PORT: set to empty string for default.
'PORT': '',
# OPTIONS: for mysql engine only, do not use with other engines.
# 'OPTIONS': {'charset': 'utf8mb4'} # Enable utf8 4-byte encodings.
}
}
[...]
# Password validation
# https://docs.djangoproject.com/en/1.9/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME':
'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME':
'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME':
'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME':
'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Internationalization
# https://docs.djangoproject.com/en/1.8/topics/i18n/
LANGUAGE_CODE = 'fr'
TIME_ZONE = 'America/New_York'
[...]
# If you enable internal authentication, this is the address that the emails
# will appear to be coming from. Make sure you set a valid domain name,
# otherwise the emails may get rejected.
# https://docs.djangoproject.com/en/1.8/ref/settings/#default-from-email
# DEFAULT_FROM_EMAIL = "mailing-lists(a)you-domain.org"
DEFAULT_FROM_EMAIL = 'username@domain'
# If you enable email reporting for error messages, this is where those emails
# will appear to be coming from. Make sure you set a valid domain name,
# otherwise the emails may get rejected.
# https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-SERVER_EMAIL
# SERVER_EMAIL = 'root(a)your-domain.org'
SERVER_EMAIL = 'username@domain'
# Change this when you have a real email backend
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
[...]
#
# Social auth
#
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend',
'allauth.account.auth_backends.AuthenticationBackend',
)
# Django Allauth
ACCOUNT_AUTHENTICATION_METHOD = "username_email"
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = "mandatory"
# You probably want https in production, but this is a dev setup file
ACCOUNT_DEFAULT_HTTP_PROTOCOL = "https"
ACCOUNT_UNIQUE_EMAIL = True
#
# Asynchronous tasks
#
Q_CLUSTER = {
'timeout': 300,
'save_limit': 100,
'retry':360,
'orm': 'default',
}
[...]
# Only display mailing-lists from the same virtual host as the webserver
FILTER_VHOST = True
POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost:8006'
## (End of settings.py)
I'm happy to provide additional information if it can help.
Best,
Pier-André
4 years, 11 months
Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius
by Simon Handfield
Thanks for your advice, Stephen.
As I've removed the file from the shunt queue I don't believe that file is causing the 500 error any more.
I've confirmed the error is not cached in the browser.
If the issue does not reside in the shunt queue (no other files indicate parse errors when checking via command line) are there any other places I should look for data that is addressed by that page?
Cheers,
Simon.
________________________________
From: Stephen J. Turnbull <turnbull.stephen.fw(a)u.tsukuba.ac.jp>
Sent: Monday, 31 August 2020 23:51
To: Simon Handfield <simon.handfield(a)sl.nsw.gov.au>
Cc: mailman-users(a)mailman3.org <mailman-users(a)mailman3.org>
Subject: [MM3-users] Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius
Simon Handfield via Mailman-users writes:
> I have been able to identify a file in the shunt queue that threw a
> matching error when I performed mailman qfile on it.
> I have run a loop over all remaining files in the shunt queue and
> all the remaining files can be accessed by mailman qfile without
> error.
Good. But there are still a bunch of problems with those files since
they got shunted.
> I moved this file out of the queue in the hope that I would then be
> able to load the held_messages page successfully. Unfortunately I
> am still seeing a 500 error on loading the page.
If this is a Mailman problem, I don't understand it. The shunt queue
is not the moderation queue. The shunt queue is for messages that
Mailman cannot handle without throwing an exception. Once a message
is sent to the shunt queue, Mailman ignores it, so it shouldn't cause
problems for the moderation page.
Have you tried restarting Mailman since moving the qfile?
If that doesn't work, and if it's the same error message and
traceback, the easy suspect is browser caching (although we should be
disabling that, I hope). Try a manual refresh (usually Ctrl-R or in
the menu). If that doesn't work, try a private browsing window or
another browser if you don't want to clear the cache or restart the
browser.
> I temporarily reenabled debug logging to confirm that the same
> error was being thrown.
>
> Assuming that file was the cause are there other steps I need to
> perform in addition to removing the file from the shunt queue
> directory?
The only one I can think of is the check for browser-side caching.
Unfortunately, Mark's out of town (like, on top of a mountain with no
bars on his phone), and I am not going to be able to do much with this
until Thursday at the earliest. If you do get a new traceback, post
it here (somebody else may have an idea).
If you like you may send me the qfile at the address in From. I'll see
if I can reproduce the bug, and I'm pretty sure I can tell you what's
in the file so at least you'll know whether it's important to somebody
who isn't a scammer/spammer.
Steve
_
[State Library of NSW]<https://www.sl.nsw.gov.au/>
Simon Handfield
Desktop and Infrastructure Leader
simon.handfield(a)sl.nsw.gov.au<mailto:simon.handfield@sl.nsw.gov.au>
+61 2 9273 1535
Macquarie Street, Sydney NSW 2000, Australia
www.sl.nsw.gov.au<https://www.sl.nsw.gov.au/>
This message is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Views expressed in this message are those of the individual sender, and are not necessarily the views of their organisation.
5 years, 11 months
Notes about successfuly upgrade from python 3.7 to 3.9
by Guillermo Hernandez (Oldno7)
I've successfuly upgrade servers with mailman3 lists from FreeBSD 12.3
with python 3.7 to FreeBSD 13.1 with python 3.9
After doing the O.S. and all the python related apps upgraded I did the
following steps:
(note that my mailman3 installation is system-wide, pip drived and
apache24/uwsgi served. The user that owns mailman3 is "mailman3" and all
the stuff lies in "/usr/local/mailman3")
-> su
-> cd /usr/local/mailman3
-> pip install --upgrade django-mailman3 mailman mailman-hyperkitty
mailmanclient postorius django hyperkitty uwsgi mod-wsgi
-> mod_wsgi-express module-config (it returns the path for the apache24
httpd.conf adjustments. I edited it)
-> su -m mailman3 -c "python3 manage.py collectstatic"
-> su -m mailman3 -c "python3 manage.py migrate"
-> su -m mailman3 -c "python3 manage.py compress"
-> cd /usr/local/lib/python3.9/site-packages/
-> cd postorius
-> django-admin compilemessages
-> cd ../hyperkitty
-> django-admin compilemessages
-> cd ../django_mailman3
-> django-admin compilemessages
I had to edit again
/usr/local/lib/python3.9/site-packages/mistune/scanner.py and add this
two lines (this is a little bit annoying)
from mistune.util import escape_html
from mistune.util import escape_url
And that was all. Restarted the servers and all is going smoothly
All of this steps have been documented in this list by Mark & Stephen
whom I'm very grateful.
P.S.:
As I did some upgrades ago, I didn't have to modify settings.py in the
Hyperkitty area to add
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
But if you come from an old mailman/Hyperkitty version, you'll have to
include in it.
And the it's same if you are using UWSGI, you have to include in your
apache config:
WSGIPassAuthorization On
--
___________________________________________
Mailman's content filtering has removed the
following MIME parts from this message.
Content-Type: image/png
Name: firma-GHP-emails.png
Replaced multipart/alternative part with first alternative.
4 years
Re: Internal Server Error: /mailman3/accounts/fedora/login/
by Stephen J. Turnbull
Bernie,
Thanks for the heads-up!
Some sites may prefer a /robots.txt with
User-agent: *
Disallow: /mailman3/
Disallow: /archives/
Disallow: /accounts/
Disallow: /user-profile/
and if these paths are enabled
Disallow: /postorius/
Disallow: /hyperkitty/
Disallow: /admin/
Note that the latter disallows indexing of /admin/docs/ if that URL is
enabled. Presumably that's not a problem, since the general public
should be going to djangoproject.com for documentation anyway.
Bernie Hoeneisen writes:
> Not sure though, how they harvested the URL that is triggering the
> error. Maybe that URL is exposed somewhere, e.g. in Postorius or
> Hyperkitty.
I would imagine that somebody naive (eg, no robots.txt at all) has the
top page with a sign-in or sign-up button exposed, which will lead to
a page with social login buttons, including the fedora/login button.
--
GNU Mailman consultant (installation, migration, customization)
Sirius Open Source https://www.siriusopensource.com/
Software systems consulting in Europe, North America, and Japan
8 months, 2 weeks
Django Internal Server Error on held_messages
by Hasan Berkay Çağır
Hello,
We've been running a Mailman3 instance with Debian 10's official package:
Mailman Core Version: GNU Mailman 3.2.1 (La Villa Strangiato)
Mailman Core API Version: 3.0
Mailman Core Python Version: 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
For some reason, and only on some lists, when we try to see held messages on Postorius, we get an error message "An error occurred while processing your request." and the following error email:
"""
Internal Server Error: /postorius/lists/list.example.com/held_messages
KeyError at /postorius/lists/list.example.com/held_messages
'file'
Request Method: GET
Request URL: https://mailman.example.com/postorius/lists/list.example.com/held_messages
Django Version: 1.11.29
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.7.3
Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages']
Server time: Thu, 24 Mar 2022 08:39:10 +0000
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/postorius/auth/decorators.py" in wrapper
58. return fn(*args, **kwargs)
File "/usr/lib/python3/dist-packages/postorius/views/list.py" in list_moderation
520. paginator_class=MailmanPaginator)
File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in paginate
72. objects = paginator.page(page_num)
File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in page
48. result = self.function(count=self.per_page, page=number)
File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py" in get_held_page
134. return Page(self._connection, url, HeldMessage, count, page)
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/page.py" in __init__
37. self._create_page()
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/page.py" in _create_page
62. response, content = self._connection.call(self._build_url())
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call
99. raise HTTPError(url, response.status, content, response, None)
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/usr/lib/python3/dist-packages/postorius/middleware.py" in __call__
39. return self.get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
43. response = response_for_exception(request, exc)
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in response_for_exception
93. response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in handle_uncaught_exception
135. extra={'status_code': 500, 'request': request},
File "/usr/lib/python3.7/logging/__init__.py" in error
1412. self._log(ERROR, msg, args, **kwargs)
File "/usr/lib/python3.7/logging/__init__.py" in _log
1519. self.handle(record)
File "/usr/lib/python3.7/logging/__init__.py" in handle
1529. self.callHandlers(record)
File "/usr/lib/python3.7/logging/__init__.py" in callHandlers
1591. hdlr.handle(record)
File "/usr/lib/python3.7/logging/__init__.py" in handle
905. self.emit(record)
File "/usr/lib/python3/dist-packages/django/utils/log.py" in emit
118. reporter = ExceptionReporter(request, is_email=True, *exc_info)
File "/usr/lib/python3/dist-packages/django/views/debug.py" in __init__
246. self.template_info = getattr(self.exc_value, 'template_debug', None)
File "/usr/lib/python3.7/tempfile.py" in __getattr__
614. file = self.__dict__['file']
Exception Type: KeyError at /postorius/lists/list.example.com/held_messages
Exception Value: 'file'
Request information:
USER: root
GET: No GET data
POST: No POST data
FILES: No FILES data
COOKIES:
rl_anonymous_id = '*'
rl_user_id = '*'
rl_group_id = '*'
rl_trait = '*'
rl_group_trait = '*'
rl_page_init_referrer = '*'
rl_page_init_referring_domain = '*'
csrftoken = '*'
sessionid = '*'
META:
CONTENT_LENGTH = ''
CONTENT_TYPE = ''
CSRF_COOKIE = '*'
DOCUMENT_ROOT = '/etc/nginx/html'
HTTPS = 'on'
HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, br'
HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.5'
HTTP_CACHE_CONTROL = 'max-age=0'
HTTP_COOKIE = '*'
HTTP_DNT = '1'
HTTP_HOST = 'mailman.example.com'
HTTP_REFERER = 'https://mailman.example.com/postorius/lists/list.example.com/'
HTTP_SEC_FETCH_DEST = 'document'
HTTP_SEC_FETCH_MODE = 'navigate'
HTTP_SEC_FETCH_SITE = 'same-origin'
HTTP_TE = 'trailers'
HTTP_UPGRADE_INSECURE_REQUESTS = '1'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:98.0) Gecko/20100101 Firefox/98.0'
PATH_INFO = '/postorius/lists/list.example.com/held_messages'
QUERY_STRING = ''
REMOTE_ADDR = '::1'
REMOTE_PORT = '58290'
REQUEST_METHOD = 'GET'
REQUEST_SCHEME = 'https'
REQUEST_URI = '/postorius/lists/list.example.com/held_messages'
SCRIPT_NAME = ''
SERVER_NAME = 'mailman.example.com'
SERVER_PORT = '443'
SERVER_PROTOCOL = 'HTTP/2.0'
uwsgi.core = 0
uwsgi.node = b'example.com'
uwsgi.version = b'2.0.18-debian'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.file_wrapper = ''
wsgi.input = <uwsgi._Input object at 0x7f8f7a6bf870>
wsgi.multiprocess = False
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', 'admin(a)example.com'),)"
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/usr/share/mailman3-web'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor'
COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter']
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f8f7b50d598>
COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor'
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = True
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/var/lib/mailman3/web/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_URL = '/mailman3/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'", 'charset': 'utf8mb4'}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'postorius(a)example.com'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAILNAME = 'example.com'
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}}
HOSTNAME = 'localhost.local'
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')"
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')"
MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT_DAYS = '********************'
POSTORIUS_TEMPLATE_BASE_URL = 'https://mailman.example.com/'
PREPEND_WWW = False
Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5}
ROOT_URLCONF = 'urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = None
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'root(a)example.com'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 1
SOCIALACCOUNT_PROVIDERS = {}
STATICFILES_DIRS = '()'
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/lib/mailman3/web/static'
STATIC_URL = '/mailman3/static/'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_ETAGS = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = True
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'
"""
I've tried to debug this issue as much as I can but couldn't figure out anything. I wonder if anyone has encountered something like this or might have a clue about what's the problem here?
Thanks in advance,
Berkay
4 years, 4 months
Re: Error while mass operation, no progress display while mass operation is running
by Mark Sapiro
On 2/6/25 04:53, Markus Grandpré wrote:
>
> At the University of Konstanz we have mailing list administrators who
> frequently add and remove up to 10000 (and sometimes more) list members.
> I have therefore tested the “Mass subscription” and “Mass removal”
> operations and found that both the “Remove ALL members” operation and
> the “Mass subscription” operation report an internal server error on the
> website, but continue to run in the background until all new members
> have been successfully added or deleted.
The internal server error is probably due to the wsgi server timing out
waiting for a response. See
https://docs.gunicorn.org/en/stable/settings.html#timeout for Gunicorn
or https://uwsgi-docs.readthedocs.io/en/latest/Options.html#harakiri for
uWSGI
> Unfortunately, I have not found
> any error message in the log files.
If you are using uWSGI, a harakiri timeout should be logged in uWSGI's log.
> I also tested that mass operations
> “Remove listed users” and “Remove Selected” ran successfully and without
> any errors.
You may want to increase the appropriate timeout and see if that helps.
> Regardless of whether a mass operation process runs without errors or
> continues to run despite an error, I find it hard that Postorius does
> not display the progress of such a mass operation in order to monitor
> the process, as mass operations with more than 1000 members are very
> time-consuming. Perhaps this could be changed in a future version of
> Postorius?
This is a reasonable request. The best place for such a request is
https://gitlab.com/mailman/postorius/-/issues/new
Requests and suggestions on the mailing list, no matter how reasonable,
get lost over time.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
1 year, 6 months
Re: Moderator e-mails only going out once a day
by Stephen J. Turnbull
jkoral(a)luc.edu writes:
> -Mailman was installed using the virtual environment.
> -Mailman info:
> -I use systemd to stop and start the services.
OK, this is all about as close to the environment most of the Mailman
devs run as you can get. On the one hand, it's easiest for us to
understand, but it also makes it more mysterious when something
doesn't work ... "works for me", you know.
> REST credentials: restadmin:restpass
You should change this. It just makes it a little more annoying for
someone unauthorized who gets access to port 8001, but it's very low
cost to change.
> -Virgin is running:
> Note: Not sure how to check the queue, but when I run mailman3
> notify, the moderator e-mails get sent.
Good. The queues are implemented with Python pickle files in the
"queue" subtree, probably /opt/mailman/var/queue in your case, or
maybe /opt/mailman/mm/var/queue if you use Mark's preferred setup.
You're looking for .pck files in subdirectories of that directory
which are older than about 5 minutes.
I don't have a suggestion why the "notify immediately" option isn't
working. Both the 'mailman notify' command and the 'notify
immediately' option work the same way, by composing the notification
and dropping it in the virgin queue.
This should tell you the same thing that Postorius does, but you could
try "su - mailman" followed by activating the virtual environment,
then type
mailman withlist -l list.with.problem
and then
m.admin_immed_notify
expecting the response True according to your setting. If it comes up
False, there's your problem but we need to figure out why that's
different from what Postorius is telling you. You should be able to
change it as well by typing
m.admin_immed_notify = True
commit()
and exiting.
Steve
1 year, 4 months
Re: Template - "You can use these variables in the templates" - The variable for "leave" is missing
by Mark Sapiro
On 11/4/23 06:28, Stephen J. Turnbull wrote:
> Wikinaut writes:
>
> > fxied in merge request
> > https://gitlab.com/mailman/postorius/-/merge_requests/852
>
> I wish you wouldn't. It's just bloat in the code and in the
> docstring. Better to document that the administrative addresses can
> be constructed using ${short_listname}. I believe Mailman still
> supports both "list-leave" and "list-unsubscribe", as well.
Actually, https://gitlab.com/mailman/postorius/-/merge_requests/852 only
added documentation of the existing $description replacement which is
appropriate, but it duplicates part of
https://gitlab.com/mailman/postorius/-/merge_requests/851 which does
that and adds a bit more documentation.
Also see https://gitlab.com/mailman/mailman/-/issues/1108 and
https://gitlab.com/mailman/mailman/-/merge_requests/1153
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
2 years, 9 months
Re: Held messages: Long time waiating for response of the Mailman API
by Stephen J. Turnbull
Krinetzki, Stephan writes:
> Hi everyone,
>
> any news here? We are running with Pyton 3.6 - maybe a Python
> Upgrade can speed Mailman up?
It would help if you provide the original description.
I see you're running Apache + uswgi. uwsgi has been pretty finicky,
and most of the developers use gunicorn as the WSGI host. I suggest
you ask on the uswgi channels as well. (I don't think that's a
*better* place to ask, just a relevant place.)
You wrote:
> If we try to approve them via Postorius, the message is delivered
> to the list immediately, but the success message in Postorius takes
> a long time - on average the process hangs for three minutes until
> the message is no longer displayed.
You jump to the conclusion that the REST API is slow, but there's zero
evidence to localize to the REST API. If your browser isn't running
on the same host as the webserver it could even be some kind of
network timeout (reverse DNS?) completely unrelated to Mailman
(unlikely, but can't be ruled out with the information provided).
1. What is in the various logs? Mailman core, Django, your backend
database, the webserver, the WSGI hosts all may be writing to
logs, especially if there's some kind of time out or other error
condition.
2. Are all the relevant components running on the same host? Are
there other web services running under Apache and/or WSGI?
3. What are the relevant configurations (Apache, uswgi, gunicorn,
Mailman, Postorius, Django, database, DNS A/AAAA/CNAME)? This
would be anything like port numbers and host names or IP
addresses, user names and passwords (we don't need to know these,
but you should provide obfuscated versions so we can check syntax,
and of course you should check for typos).
Steve
4 years, 6 months