Re: Hyperkitty performance problem
by monochromec
Following the root cause analysis which Tobias started all those weeks ago we (the admin team behind the installation) are still struggling with the following phenomenon: messages on average take more than 24 hours to be processed, more precisely, the average lifetime of a pickled message object in `/var/lib/mailman3/queue/pipeline` clocks in at around 26 hours).
Couple of stats of the installation: standard installation from Bookworm OS repos, Hyperkitty as archiver and Postorius as web frontend as explained above, running Python 3.11.2 from the standard systemd service as packaged with the Bookworm deb file. All backends (Core + Hyperkitty) are supported by Postgres version 15+248. The MTA is a standard Postfix installation, again from OS packages.
The underlying VM has 7 cores with just under 24 GB of main memory. This production instance is handling less than 130 mailing list (MLs) with an average of less than 10 postings per day per ML. CPU core utilisation hovers around 50% with the lion share allocated to the four pipeline runners as part of the MM configuration.
OS resource utilisation is well below bounds (approx. 8 GB of main memory allocated to running processes), plenty of available sockets space (I noticed some transient `Connection lost during _handle_client()` warnings in the logs so I checked that the SMTP runner can connect to Postfix for delivering the messages after processing by checked the socket allocation of the running processes).
Cursory review of the corresponding Core classes (runner + pipeline implementation in addition to `posting_pipeline`) didn't reveal any further pointers. What I did notice though that increasing the logging levels of the components (namely `smtp`, `pipeline` and `lmtp` to `debug`) in `/etc/mailman3/mailman.cfg` didn't add any useful information to the logs as configured after restarting the Core.
As outlined above, Hyperkitty doesn't seem to do a check based on ML and message ID before archiving a message in the database. But this only add a REST roundtrip and Postgres communication through Hyperkitty's underlying Django framework to the overall system load and the driving UWSGI instance is well within CPU cycle bounds.
Any pointers are appreciated - more than happy to provide more info if required.
1 year, 10 months
Re: Converting discussion list to read-only list (settings)
by Mark Sapiro
On 10/14/19 5:40 PM, Paul Arenson wrote:
>
>> Assuming MUA means a mail application. And if that mail application has no reply list function then the setting doesn't work.
MUA means Mail User Agent which is another name for mail application, YES.
> Uh-oh: I AM LOST AGAIN. (will all caps give me a larger font?)
>
>> It is the presence of the
>> List-Post: header in the message that allows the MUA to offer/process
>> the "reply-list" function.
>
> Ok, I asked how to make it ANNOUNCE ONLY and you said to use "allow_list_posts is False" which is the same as "Include the list post header".
Not exactly. You asked what settings were different between discussion
style and announce only style. What I answered was that the only setting
that differed as set by the style was what Postorius calls "Include the
list post header".
What I didn't say explicitly was that this is not sufficient to prevent
posting to the list.
> So does this mean that for a subset of users I cannot make it ANNOUNCE ONLY? And is it some other setting (send to moderator) that prevents it from being a DISCUSSION list?
What exactly do you want? You found some info on configuring an announce
only list and I referred you to a FAQ on that subject. Both of these do
not directly address Mailman 3, but they do give useful information.
I am going to stop guessing at what you want to accomplish, so tell me
what you want to happen when someone sends a post to the list either
from HyperKitty or by email. What should happen to the message? What if
anything should be sent to the user? Should this behavior be the same
for all users or just a subset. If a subset, how do you identify that
subset and what should happen when others attempt to post?
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
7 years
Re: What's the difference between (purpose of) reject and ignore
by denis.bitouze＠univ-littoral.fr
Hello,
Andy Smith wrote:
> On Sun, Jun 26, 2022 at 12:37:05PM -0000, denis.bitouze(a)univ-littoral.fr wrote:
>
>> AFAICS, it is not documented since, at Postorius → Settings → Message
>> Acceptance → Default action to take when a (non-)member posts to the
>> list, I only read: “Discard: this simply discards the message, with
>> no notice sent to the post's author”. So no mention that this would
>> set it to (silently) discard all future mails from them.
>
> Well that is because that isn't what that part does. As it says,
> that part is the "Default action to take when a member/non-member
> posts to the list". So that's where you set what you want to happen
> when ANY non-member posts. This is what will happen any time any
> non-member posts.
> By contrast, when you set an action on a particular held message, it
> is asking you want you want to do with posts from THAT INDIVIDUAL
> ADDRESS for the future.
> So hopefully you can see that these things are different. One is a
> whole-list property that applies to posts from classes of posters
> (members, non-members). The other is per-poster property as a
> reaction to as held message from them.
OK, I see, thanks.
>> Just to be sure: for this feature, is it necessary to click on the
>> subject and to choose in the drop-down list “Discard (no
>> notification)”, or is it enough, on the .../held_messages page, to
>> select the messages and click on “Discard”?
>
> Doing it from the held messages page will be a one-time thing; it is
> only by clicking on the individual message that you can choose an
> action for future messages from that address.
OK. Too bad this cannot be done from the list of messages on the
.../held_messages page: it would be very helpful when you are
overwhelmed by spams...
4 years, 4 months
Re: moving lists to another server
by Mark Sapiro
On 11/20/23 00:58, Odhiambo Washington wrote:
>
> What is the command to export the archive as a mbox from the DB?
> I can't figure out which option it is by looking at 'mailman-web -h'.
Normally you do this on the old server via the HyperKitty web UI which
has download buttons which do things like
https://example.com/archives/list/list@example.com/export/list@example.com.…
or for a partial archive
https://example.com/archives/list/list@example.com/export/list@example.com-…
However, in your case I don't think you have the old installation
working any more so the above won't help.
> BTW, how will ' mailman addmembers' treat a file containing:
> ```
> Firstname Lastname <email@address>
> Onlyname <email2@address>
> ```
Correctly. It will import email@address with display name = 'Firstname
Lastname' and email2@address with display name = 'Onlyname'.
> I have opted for the simple way to do this:
> 1. Create the site in Django Admin and create/configure the list in
> Postorius
> 2. Add the members from the file dump, with delivery set to regular.
> 3. Send the members an email informing them that they need to set their
> delivery preferences.
>
> I am stuck on the issue of exporting the archives to mbox file as already
> stated above.
Perhaps you can export the mbox via the web UI on the old server or
maybe temporarily create a second mailman-web on your server with the
old mailman-web database and use that web UI. Otherwise, if you have a
working mailman-web that can access this database, you could try
something like
```
$ mailman-web shell
>>> from hyperkitty.lib.compat import get_list_by_name
>>> mlist = get_list_by_name('list_name_only', 'list_domain')
>>> with open('output.mbox', 'wb') as fp:
... for email in mlist.emails.order_by("archived_date").all():
... fp.write(email.as_bytes() + b'\n')
...
>>>
```
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
2 years, 11 months
How to add new Domain?
by c.benner＠apd-it.eu
Hello, i'm really new to mailman3 and I've been using mm2.1 couple of years.
It's really hardto understand documentation. But I'll do one in german language next time. I've got it installed ob Debian12 and it's working now.
It takes a little more time to understand all settings and also the difference between postorius and django and what is when used for.
But what I do not understand 'til now is how to add a new domain.
My running config has one domain (e.g.) testdomain1.org and everything is working fine.
The Domain has a virtualhost which listens on *:443 (with LE cert)
Postfix works like a charm.
The interface of the machine is bound to one IP Address public but for example here written as: 192.168.200.15
Id like to add a new Domain.
My first step would be to add another IP Address to the interface: (e.g.) 192.168.200.16
Then I'll change the virtual host settings in apache conf from first domain which is actually listeing to *:443 to 192.168.200.15:443
And add a second site with 192.168.200.16:443
So the "physical" part is done
Is that right 'til now?
After that I'll go to the admin page xxxx/mailman3/admin which brings django admin page up.
On that page I'll add "websites" and add testdomain2.net
Last step is adding "mail domain" site: testdomain2.net mail domain: testdomain2.net
After that I can add lists to that domain and mails to any list @testdomain2,net should be routed right?
Maybe I'm to old (56yrs) but I've read a lot of documentation and everything was clear and understandable. It was never a question of linux or windows OS but on who'S writing the doc. I can't find any doc about the web part? What about Users having no acces to the shell but the Web-Frontend? Some pictures and a "How to" and "what if" would be great!
But it's alos possible that I'm not finding it and it's somewhere out there.
Thanks in advance.
1 year, 9 months
Re: Bad Signature with Gmail
by Massimo Zappalà
Hi Mark, thanks for the reply.
my /etc/opendkim.conf
AutoRestart Yes
AutoRestartRate 10/1h
UMask 002
Syslog yes
SyslogSuccess Yes
LogWhy Yes
Canonicalization relaxed/simple
ExternalIgnoreList refile:/etc/opendkim/TrustedHosts
InternalHosts refile:/etc/opendkim/TrustedHosts
KeyTable refile:/etc/opendkim/KeyTable
SigningTable refile:/etc/opendkim/SigningTable
Mode sv
PidFile /var/run/opendkim/opendkim.pid
SignatureAlgorithm rsa-sha256
UserID opendkim:opendkim
Socket inet:12301@localhost
my /etc/opendkim/SigningTable
*(a)catania.linux.it mailselector._domainkey.catania.linux.it
full email django error
Internal Server Error: /hyperkitty/api/mailman/archive
OperationalError at /hyperkitty/api/mailman/archive
(1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1")
Request Method: POST
Request URL: https://lists.catania.linux.it/hyperkitty/api/mailman/archive?key=mykey
Django Version: 1.11.29
Python Executable: /usr/bin/uwsgi-core
Python Version: 3.7.3
Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages']
Server time: Sab, 19 Set 2020 20:05:41 +0200
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute
101. return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
250. self.errorhandler(self, exc, value)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler
50. raise errorvalue
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
247. res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query
411. rowcount = self._do_query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query
374. db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query
292. _mysql.connection.query(self, query)
The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1")) was the direct cause of the following exception:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/views/decorators/http.py" in inner
40. return func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in _decorator
69. return func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/mailman.py" in archive
115. add_to_list(mlist_fqdn, msg)
File "/usr/lib/python3/dist-packages/hyperkitty/lib/incoming.py" in add_to_list
151. email.save()
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save
808. force_update=force_update, update_fields=update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in save_base
838. updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _save_table
924. result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/lib/python3/dist-packages/django/db/models/base.py" in _do_insert
963. using=using, raw=raw)
File "/usr/lib/python3/dist-packages/django/db/models/manager.py" in manager_method
85. return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/lib/python3/dist-packages/django/db/models/query.py" in _insert
1079. return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py" in execute_sql
1112. cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/utils.py" in __exit__
94. six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python3/dist-packages/django/utils/six.py" in reraise
685. raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py" in execute
101. return self.cursor.execute(query, args)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
250. self.errorhandler(self, exc, value)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in defaulterrorhandler
50. raise errorvalue
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in execute
247. res = self._query(query)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _query
411. rowcount = self._do_query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py" in _do_query
374. db.query(q)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py" in query
292. _mysql.connection.query(self, query)
Exception Type: OperationalError at /hyperkitty/api/mailman/archive
Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\x98\\xB1\\xF0\\x9F...' for column `mailman3web`.`hyperkitty_email`.`content` at row 1")
Request information:
USER: AnonymousUser
GET:
key = 'mykey'
POST:
mlist = 'lug(a)catania.linux.it'
FILES:
message = <InMemoryUploadedFile: message.txt ()>
COOKIES: No cookie data
META:
CONTENT_LENGTH = '2485'
CONTENT_TYPE = 'multipart/form-data; boundary=3fc2cf34ecf58287e2b148ab43b4b111'
DOCUMENT_ROOT = '/usr/share/nginx/html'
HTTPS = 'on'
HTTP_ACCEPT = '*/*'
HTTP_ACCEPT_ENCODING = 'gzip, deflate'
HTTP_CONNECTION = 'keep-alive'
HTTP_CONTENT_LENGTH = '2485'
HTTP_CONTENT_TYPE = 'multipart/form-data; boundary=3fc2cf34ecf58287e2b148ab43b4b111'
HTTP_HOST = 'lists.catania.linux.it'
HTTP_USER_AGENT = 'python-requests/2.21.0'
PATH_INFO = '/hyperkitty/api/mailman/archive'
QUERY_STRING = 'key=mykey'
REMOTE_ADDR = '95.216.191.209'
REMOTE_PORT = '35022'
REQUEST_METHOD = 'POST'
REQUEST_SCHEME = 'https'
REQUEST_URI = '/hyperkitty/api/mailman/archive?key=mykey'
SCRIPT_NAME = ''
SERVER_NAME = 'lists.catania.linux.it'
SERVER_PORT = '443'
SERVER_PROTOCOL = 'HTTP/1.1'
uwsgi.core = 0
uwsgi.node = b'glugct.catania.linux.it'
uwsgi.version = b'2.0.18-debian'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.file_wrapper = ''
wsgi.input = <uwsgi._Input object at 0x7f9cb9209438>
wsgi.multiprocess = False
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', 'admin(a)catania.linux.it'),)"
ALLOWED_HOSTS = ['*']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/usr/share/mailman3-web'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_COMPRESSOR = 'compressor.css.CssCompressor'
COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter']
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f9cb829cd90>
COMPRESS_JS_COMPRESSOR = 'compressor.js.JsCompressor'
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = True
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/var/lib/mailman3/web/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/mailman3/static/'}
COMPRESS_URL = '/mailman3/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3web', 'USER': 'mailman3web', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': '', 'OPTIONS': {'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"}, 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_CONTENT_TYPE = 'text/html'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'postorius(a)lists.catania.linux.it'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAILNAME = 'lists.catania.linux.it'
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 25
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = None
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/lib/mailman3/web/fulltext_index'}}
HOSTNAME = 'localhost.local'
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')"
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'it-IT'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'INFO', 'class': 'logging.handlers.RotatingFileHandler', 'filename': '/var/log/mailman3/web/mailman-web.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'INFO', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}, 'postorius': {'handlers': ['file'], 'level': 'INFO', 'propagate': True}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1', '::ffff:127.0.0.1', '95.216.191.209')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')"
MIDDLEWARE_CLASSES = ['django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT_DAYS = '********************'
POSTORIUS_TEMPLATE_BASE_URL = 'http://localhost/mailman3/'
PREPEND_WWW = False
Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default', 'poll': 5}
ROOT_URLCONF = 'urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = False
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = None
SECURE_REDIRECT_EXEMPT = []
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'root(a)lists.catania.linux.it'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 1
SOCIALACCOUNT_PROVIDERS = {}
STATICFILES_DIRS = '()'
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/lib/mailman3/web/static'
STATIC_URL = '/mailman3/static/'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'Europe/Rome'
USE_ETAGS = False
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = True
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'wsgi.application'
X_FRAME_OPTIONS = 'SAMEORIGIN'
YEAR_MONTH_FORMAT = 'F Y'
Max
6 years, 1 month
Re: One-click unsubscribe?
by Jered Floyd
Reviving this thread I started in February because delivery rates continue to decline, and I get this threat from Google any time I check their Dashboard:
"One-click Unsubscribe: Needs work — Ensure one-click unsubscribe is correctly implemented for your subscription messages
One-click unsubscribe allows recipients to easily unsubscribe instead of marking your messages as spam"
Having recently implemented "Feedback-ID" (Google's proprietary-but-required Feedback Loop mechanism) at my MTA, all this has done is confirm that 0.05-0.10% of subscribers report messages as spam and would probably rather be unsubscribed but cannot be bothered to figure this out. Having the headers that make Google/Yahoo/Apple/etc ask "Would you like to unsubscribe from this?" when they hit the "Junk" button will most likely resolve the problem.
I'm pretty sure, as below, that we need to implement RFC 8058 headers in order to maintain deliverability to GMail. The only tracker issue I can find related to this is https://gitlab.com/mailman/mailman/-/issues/1099 -- is that the best place to discuss implementation?
--Jered
----- On Feb 5, 2025, at 3:36 AM, Stephen J. Turnbull turnbull(a)sk.tsukuba.ac.jp wrote:
[..]
> If we're talking about a new feature, I think the most straightforward
> approach is to use RFC 8058[1] "one click" URLs. I believe it is
> already implemented by Gmail and Yahoo, I don't know about Outlook or
> the open source MUAs. This would require a substantial (but
> straightforward[2]) addition of code, and of course personalized
> delivery (I don't know if anyone balks at that any more, though).
[..]
> Footnotes:
> [1] https://datatracker.ietf.org/doc/html/rfc8058
> [2] That is, we'd need to add a new location in Postorius and code to
> accept POST access to it, encrypt/decrypt functionality to handle an
> opaque token (to prevent automated denial of service by folks without
> access to the person's email) or a table of one-time tokens with
> expiration (same purpose, and expiring so the database doesn't grow
> infinitely). What we wouldn't need to do is to change any exsting
> code.
11 months, 1 week
Re: moderation notices for list-owner being redirected to mailman address
by Mark Sapiro
On 2/7/22 16:11, Philippe B wrote:
> I think that your last sentence helped us figure out what's going on.
>
> When Mailman generates a moderation message "To: LIST-owner" it actually delivers it to the address(es) of the LIST's moderator(s), which makes sense.
No, it delivers the message to the union of the sets of owner addresses and moderator addresses.
> In all the cases where we've seen that behavior, it turns out that the list involved didn't have any moderator, and we have site_owner set to "mailman"...
You don't need to have moderators. The owners get the message too.
> However, the lists had an owner, and we're not seeing those moderation messages being sent to their owner (instead). Is that something that needs to be configured explicitly somewhere?
The only things that might affect this are if Settings -> Automatic Responses -> Autorespond to list owner is Respond and discard message or if the owners delivery_status is other than DeliveryStatus.enabled.
Note that finding an owner's delivery status is tricky. The owner can find it in Postorius by going to https://www.example.com/mailman3/accounts/subscriptions/ and clicking on the entry for the list with role owner. This delivery status is not the same as the delivery status for the same address with a member role if there is one.
For an admin you could do
```
$ bin/mailman shell -l list.example.com
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
The variable 'm' is the list.example.com mailing list
>>> owner = m.owners.get_member('user(a)example.com')
>>> owner.delivery_status
<DeliveryStatus.enabled: 1>
If you needed to set it here, you could do
>>> owner.preferences.delivery_status = DeliveryStatus.enabled
>>> commit()
>>>
```
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
4 years, 9 months
signup / registration error - permissions and cert chains
by David Newman
I'd like for regular (non-admin) list subscribers to be able to manage
their subscription preferences and view list archives.
However, when new users attempt to create logins from the Postorius
"Sign Up" page, the server returns the error pasted below, and displays
a "server error" page to users.
If I'm reading the error correctly, this is related to an inability to
verify the cert chain. The /etc/mailman3/settings.py file points to the
same cert and key files used by Nginx, Postfix, and Dovecot. Since TLS
works OK with these other services, it appears the cert and key are
valid. (Viewing the cert in a web browser also supports this.)
The keyfile has 0400 permissions. I tried changing this to 0640 and
making the keyfile owned by root:mailman. Even after restarting mailman3
and mailmanweb the same error occurs. (Permissions in the cert and key
directories and above are not blocking access.)
The etc/mailman3/settings.py file includes these settings:
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = 'dnewman(a)networktest.com'
EMAIL_HOST_PASSWORD = 'wouldnt-you-like-to-know'
EMAIL_USE_TLS = 'True'
EMAIL_SSL_CERTFILE = '/etc/ssl/certs/myhost.crt'
EMAIL_SSL_KEYFILE = '/etc/ssl/private/myhost.key'
But this might only be for email, not Postorius/Django. I checked the
Mailman Web docs and settings reference but didn't see anything relevant
on either page.
https://docs.mailman3.org/en/latest/config-web.html
https://docs.mailman3.org/projects/mailman-web/en/latest/settings.html
What additional configuration is needed to allow regular users to create
and manage their own accounts?
Thanks.
dn
ERROR 2021-12-30 16:03:53,787 436935 django.request Internal Server
Error: /accounts/signup/
Traceback (most recent call last):
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/core/handlers/exception.py",
line 34, in inner
response = get_response(request)
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/core/handlers/base.py",
line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/core/handlers/base.py",
line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/views/generic/base.py",
line 71, in view
return self.dispatch(request, *args, **kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/utils/decorators.py",
line 43, in _wrapper
return bound_method(*args, **kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/views/decorators/debug.py",
line 76, in sensitive_post_parameters_wrapper
return view(request, *args, **kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/views.py", line
230, in dispatch
return super(SignupView, self).dispatch(request, *args, **kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/views.py", line
74, in dispatch
response = super(RedirectAuthenticatedUserMixin, self).dispatch(
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/views.py", line
204, in dispatch
return super(CloseableSignupMixin, self).dispatch(request, *args,
**kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/views/generic/base.py",
line 97, in dispatch
return handler(request, *args, **kwargs)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/views.py", line
102, in post
response = self.form_valid(form)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/views.py", line
248, in form_valid
return complete_signup(
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/utils.py", line
209, in complete_signup
return perform_login(
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/utils.py", line
175, in perform_login
send_email_confirmation(request, user, signup=signup, email=email)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/utils.py", line
346, in send_email_confirmation
email_address.send_confirmation(request, signup=signup)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/models.py",
line 62, in send_confirmation
confirmation.send(request, signup=signup)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/models.py",
line 169, in send
get_adapter(request).send_confirmation_mail(request, self, signup)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/adapter.py",
line 464, in send_confirmation_mail
self.send_mail(email_template,
emailconfirmation.email_address.email, ctx)
File
"/opt/mailman/venv/lib/python3.9/site-packages/allauth/account/adapter.py",
line 136, in send_mail
msg.send()
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/core/mail/message.py",
line 284, in send
return self.get_connection(fail_silently).send_messages([self])
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/core/mail/backends/smtp.py",
line 102, in send_messages
new_conn_created = self.open()
File
"/opt/mailman/venv/lib/python3.9/site-packages/django/core/mail/backends/smtp.py",
line 67, in open
self.connection.starttls(keyfile=self.ssl_keyfile,
certfile=self.ssl_certfile)
File "/usr/lib/python3.9/smtplib.py", line 772, in starttls
context = ssl._create_stdlib_context(certfile=certfile,
File "/usr/lib/python3.9/ssl.py", line 787, in _create_unverified_context
context.load_cert_chain(certfile, keyfile)
PermissionError: [Errno 13] Permission denied
4 years, 10 months
Re: Mailman3 migration and/or archiving problem
by Odhiambo Washington
On Fri, Sep 6, 2024 at 11:43 AM <hnpilot(a)phnet.fi> wrote:
> I have installed a mailman3 server following this documentation:
> https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst…
>
> After a few hiccups I got mailman and postorius working and migrated one
> list according to this documentation:
> https://docs.mailman3.org/en/latest/migration.html
>
> It seemed to work and also hyperkitty_import was run without errors (or at
> least none shown up). Running update_index command said that there were 0
> threads copied but I kjnow that the archive isn't empty. But now when I
> tried to open the list info page from lists view it dies with 500 error.
>
> After diving into the different log files I found an error message that it
> cannot find /etc/mailman3/mailman-hyperkitty.cfg. Great. Only that the cfg
> file exists and seems to my (unexpert) eyes correct.
>
> After that problem I tried to create a new list. Creation seemed to go all
> right but also with it, trying to open the list info page it shows 500
> error.
>
> When googling for a solution I landed on a message stating that there
> might be double entries for the same list in listarchiver table but when I
> checked ot turns out that the table is empty!
>
> I am a bit confused how to figure out the base cause of the problem and
> would very much appreciate any pointers.
>
The documentation you followed is quite okay and reasonably detailed.
Just wipe out your virtualenv and start over again. Practice it till you
get it working.
I suspect you missed a step - or did not understand well. You just need to
seek clarification on that.
Looking at the mailman-web logfile mentioned in that documentation might
also help you.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
2 years, 2 months