Re: Mailman issues after re-boot
by Odhiambo Washington
On Sat, Sep 16, 2023 at 5:49 PM Paul Rubin <paulrubin37(a)gmail.com> wrote:
> *<< I sent this to the mailman3 issues on gitlab and Mark replied but asked
> me to move to this list, so I copied everything and inserted responses. I
> hope this makes sense and someone can help me.>>*
>
> After a lengthy power failure I had to reboot a machine that has been
> running for several years. the WEB-UI does not seem to be able to see
> mailman-core. I have looked for log entries that might explain what the
> issue is and I am stumped to even find logs that will update when I get an
> error on the browser. Mailman is definitely running, postgres is running
> and mailman web is running. if I go to the lists page I see lists, if I try
> to access a specific list I get Server error: An error occurred while
> processing your request.
>
> I looked in /var/logs/mailman /opt/mailman/web/mailmanweb
>
> All I can find is:
>
> django.urls.exceptions.NoReverseMatch: Reverse for
> 'list_unsubscription_requests' not found.
> 'list_unsubscription_requests' is not a valid view function or pattern
> name.16688 ERROR 2023-09-12 13:56:58,430 1760 django.request Internal
> Server Error: /mailman3/lists/goodstein.goodstein.org/ File
> "/opt/mailman/venv/lib64/python3.8/site-packages/haystack/admin.py",
> line 8, in <module> from django.utils.translation import
> ungettextImportError: cannot import name 'ungettext' from
> 'django.utils.translation'
>
> (/opt/mailman/venv/lib64/python3.8/site-packages/django/utils/translation/__init__.py)
>
> Also I have to run mailman from "mailman start" because the systemctl
> returns this:
>
> ● mailman3.service - GNU Mailing List Manager Loaded: loaded
> (/etc/systemd/system/mailman3.service; enabled; vendor preset:
> disabled) Active: failed (Result: exit-code) since Tue 2023-09-12
> 10:06:31 EDT; 13s ago Process: 31159
> ExecStart=/opt/mailman/venv/bin/mailman start (code=exited,
> status=1/FAILURE) CPU: 806msSep 12 10:06:31 mail
> mailman[31159]: Error in atexit._run_exitfuncs:Sep 12 10:06:31 mail
> mailman[31159]: Traceback (most recent call last):Sep 12 10:06:31 mail
> mailman[31159]: File "/usr/lib64/python3.8/logging/__init__.py",
> line 2126, in shutdownSep 12 10:06:31 mail mailman[31159]:
> h.flush()Sep 12 10:06:31 mail mailman[31159]: File
> "/opt/mailman/venv/lib64/python3.8/site-packages/mailman/core/logging.py",
> line 67, in flushSep 12 10:06:31 mail mailman[31159]: if
> self._stream:Sep 12 10:06:31 mail mailman[31159]: AttributeError:
> 'ReopenableFileHandler' object has no attribute '_stream'Sep 12
> 10:06:31 mail systemd[1]: mailman3.service: Control process exited,
> code=exited, status=1/FAILURESep 12 10:06:31 mail systemd[1]:
> mailman3.service: Failed with result 'exit-code'.Sep 12 10:06:31 mail
> systemd[1]: Failed to start GNU Mailing List Manager.
>
> Please tell me where to start looking or adjusting.
>
>
> Reply from Mark Sapiro,* Edited to include responses:*
>
> Mark Sapiro <https://gitlab.com/msapiro>@msapiro
> <https://gitlab.com/msapiro>· 1 day ago
> <https://gitlab.com/mailman/mailman-web/-/issues/23#note_1557601916>
> Owner
> Add reaction
> More actions
>
> The mailman-users(a)mailman3.org list is probably a more appropriate place
> for issues like this.
>
> django.urls.exceptions.NoReverseMatch: Reverse for
> 'list_unsubscription_requests' not found. 'list_unsubscription_requests' is
> not a valid view function or pattern name. 16688 ERROR 2023-09-12
> 13:56:58,430 1760 django.request Internal Server Error: /mailman3/lists/
> goodstein.goodstein.org/
>
> /opt/mailman/venv/lib64/python3.8/site-packages/postorius/urls.py should
> contain
>
> re_path( r'^unsubscription_requests$',
> list_views.list_unsubscription_requests,
> name='list_unsubscription_requests', ),
>
> Y
>
> *Yes this exists*
>
> and /opt/mailman/venv/lib64/python3.8/site-packages/postorius/views/list.py
> should define list_unsubscription_requests.
>
> *Yes there is a method by this name defined*
>
> File "/opt/mailman/venv/lib64/python3.8/site-packages/haystack/admin.py",
> line 8, in from django.utils.translation import ungettext ImportError:
> cannot import name 'ungettext' from 'django.utils.translation'
> (/opt/mailman/venv/lib64/python3.8/site-packages/django/utils/translation/
> *init*.py)
>
> See
>
> https://docs.djangoproject.com/en/4.2/internals/deprecation/#deprecation-re…
> -
> ungettext() is removed from django>=4.0
>
> Sep 12 10:06:31 mail mailman[31159]: AttributeError:
> 'ReopenableFileHandler' object has no attribute '_stream'
>
> I don't understand this one offhand, but what is the content of
> /etc/systemd/system/mailman3.service, in particular the settings for User,
> Group and ExecStart?
>
> *The service looks right:*
>
> 1 [Unit]
> 2 Description=GNU Mailing List Manager
> 3 After=syslog.target network.target postgresql.service
> 4
> 5 [Service]
> 6 Type=forking
> 7 PIDFile=/opt/mailman/mm/var/master.pid
> 8 User=mailman
> 9 Group=mailman
> 10 ExecStart=/opt/mailman/venv/bin/mailman start
> 11 ExecReload=/opt/mailman/venv/bin/mailman restart
> 12 ExecStop=/opt/mailman/venv/bin/mailman stop
> 13
> 14 [Install]
> 15 WantedBy=multi-user.target
>
> *It seems I may have two versions of the mailman script:*
>
> mail:/opt/mailman # which mailman
> /usr/bin/mailman
> mail:/opt/mailman # ls -al /usr/bin/mailman
> -rwxr-xr-x 1 root root 382 May 10 2021 /usr/bin/mailman
> mail:/opt/mailman # ls -al /opt/mailman/venv/bin/mailman
> -rwxr-xr-x 1 mailman mailman 231 Sep 12 10:02 /opt/mailman/venv/bin/mailman
>
> Somehow, your installation is corrupted. I don't know how
>
> *I need to know what to remove, how to correctly update, etc., That said
> the two scripts appear to be very similar:*
>
> mail:/opt/mailman # head /usr/bin/mailman
>
> #!/usr/bin/python3
> # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.4','console_scripts','mailman'
> __requires__ = 'mailman==3.3.4'
> import re
> import sys
> from pkg_resources import load_entry_point
>
> if __name__ == '__main__':
> sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
> sys.exit(
>
>
> mail:/opt/mailman # head /opt/mailman/venv/bin/mailman
> #!/opt/mailman/venv/bin/python3
> # -*- coding: utf-8 -*-
> import re
> import sys
> from mailman.bin.mailman import main
> if __name__ == '__main__':
> sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
> sys.exit(main())
> mail:/opt/mailman #
>
> *Given the two years since the last reboot I also tried following *
> https://docs.mailman3.org/en/latest/upgrade-guide.html *to make sure I was
> running the latest*
>
> pip install -U mailman postorius django-mailman3 hyperkitty mailman-web
>
Why don't you try and do a fresh virtualenv install and move your configs
into it?
1. cd /opt
2. mv mailman _mailman.BAK
3. Create a backup of the database.
Then follow this:
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst…,
but do not create the database as it already exists.
You also do not need to run any database migrations, so you can actually
skip those
You just need to ensure that `mailman info` says that its configuration
files are in /etc/mailman3/
Create the necessary directories as per the above installation instructions.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
1 year, 9 months
Re: Database deadlock when trying to delete a member.
by Abhilash Raj
This is helpful.
The two pending queries looks like the job of bounce runner. Since it runs every 2m to
check if any Member’s bounce score is above the threshold and if any warning needs
to be sent or delivery needs to be disabled.
It looks to me that the database query has been running for 1min 58sec, which is longer
than I’d expect. Are you seeing that query *always* there when you dump the list of
queries?
I am not sure what the empty COMMIT queries are really.
Abhilash
> On Mar 18, 2021, at 5:31 PM, s.coggins(a)cqu.edu.au wrote:
>
> Just to add to this. Looking at the active queries in PostgreSQL there appear to be a bunch of commits that haven't finished. I don't know how it all fits together but it looks like the commits aren't finishing and transactions are staying open, so when I hit an open transaction all write operations for the DB hang until the transaction has finished.
>
> postgres=# SELECT pid, age(clock_timestamp(), query_start), usename, query
> FROM pg_stat_activity
> WHERE query != '<IDLE>' AND query NOT ILIKE '%pg_stat_activity%'
> ORDER BY query_start desc;
> pid | age | usename |
> query
>
> --------+-----------------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> -------------------------------------------------------------------------------------------------------
> 15767 | 00:01:58.30353 | mailman | SELECT member.id AS member_id, member._member_id AS member__member_id, member.role AS member_role, member.list_id AS member_list_id, member.moderation_action AS member_moderation_action, mem
> ber.address_id AS member_address_id, member.preferences_id AS member_preferences_id, member.user_id AS member_user_id, member.bounce_score AS member_bounce_score, member.last_bounce_received AS member_last_bounce_received, member.la
> st_warning_sent AS member_last_warning_sent, member.total_warnings_sent AS member_total_warnings_sent +
> | | | FROM member JOIN mailinglist ON member.list_id = mailinglist.list_id JOIN preferences ON preferences.id = member.preferences_id
>
> +
> | | | WHERE mailinglist.process_bounces = true AND member.total_warnings_sent >= mailinglist.bounce_you_are_disabled_warnings AND preferences.delivery_status = 3
> 18033 | 00:05:28.255636 | mailman | DELETE FROM member WHERE member.id = 169331
> 15762 | 00:11:42.842237 | mailman | COMMIT
> 15740 | 00:11:59.12324 | mailman | COMMIT
> 15760 | 00:12:01.212976 | mailman | COMMIT
> 15757 | 00:12:01.542144 | mailman | COMMIT
> 15755 | 00:12:01.708328 | mailman | COMMIT
> 15754 | 00:12:01.804737 | mailman | COMMIT
> 15751 | 00:12:01.921644 | mailman | COMMIT
> 15745 | 00:12:02.72934 | mailman | COMMIT
> 15743 | 00:12:02.795885 | mailman | COMMIT
> 15742 | 00:12:02.948795 | mailman | COMMIT
> 15738 | 00:12:03.3471 | mailman | COMMIT
> 15664 | 00:12:08.480084 | mailman | COMMIT
> (21 rows)
>
>
> I've checked my production system and I can see the same COMMIT queries running. Some for 2 days (since the last restart).
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
--
thanks,
Abhilash Raj (maxking)
4 years, 3 months
Re: docker mailman-core mailman3 image not responding on port 8024
by Abhilash Raj
> On Aug 17, 2021, at 11:37 AM, bob B <bader(a)cchmc.org> wrote:
>
> I restarted the containers so the new/current ips are
>
>
> core "IPAddress": "172.25.0.3",
> web "IPAddress": "172.25.0.4",
>
>
>
> Here is the content of that file
>
> **************************
>
> docker exec -it mailman-core /bin/bash
> bash-5.0# cat /etc/mailman.cfg
> # This file is autogenerated at container startup.
<snip>
> [mta]
> incoming: mailman.mta.postfix.LMTP
> outgoing: mailman.mta.deliver.deliver
> lmtp_host: 172.25.0.3
> lmtp_port: 8024
> smtp_host: 172.19.199.1
> smtp_port: 25
> configuration: /etc/postfix-mailman.cfg
Looking at your [mta] configs, it seems like incoming configs are correct, but outgoing configs aren’t. You have the Mailman Core binding to 172.25.0.3, which is the IP address allocated to it from Docker and it should be accepting connections on 172.25.0.3:8024.
The outgoing config will use the hard coded IP configs in the 0.3.11 release of the container images, but you can fix that by setting SMTP_HOST: 172.25.0.1 in both mailman-core and mailman-web container configurations in docker-compose.yaml file under “environment”. I am assuming that Postfix is listening on all interfaces (0.0.0.0, which is usually the case that I have seen, but something still that perhaps should be verified).
> # mailman-extra.cfg
>
> [mta]
> incoming: mailman.mta.postfix.LMTP
> outgoing: mailman.mta.deliver.deliver
> # mailman-core hostname or IP from the Postfix server
> #lmtp_host: localhost
> lmtp_host: 127.0.0.1
> lmtp_port: 8024
> # Postfix server's hostname or IP from mailman-core
> smtp_host: ##########
> smtp_port: 25
> configuration: /etc/postfix-mailman.cfg
> [mailman]
> # This address is the "site owner" address. Certain messages which must be
> # delivered to a human, but which can't be delivered to a list owner (e.g. a
> # bounce from a list owner), will be sent to this address. It should point to
> # a human.
> site_owner: ##########
> bash-5.0#
This part is probably the one causing trouble, probably coming from what you added to mailman-extra.cfg. This overrides the configs above (which is right for incoming emails) ones. Mailman is binding to 127.0.0.1 inside the container, and that loopback interface isn’t accessible from anywhere outside the container itself, so you won’t be able to talk to it. It needs to listen either on 0.0.0.0 (all interfaces) or (at least, one of the) public IPs assigned to the container.
The fix is that you just want to delete the entire [mta] section of mailman-extra.cfg and restart the container. Do also setup the SMTP_HOST environment to fix that outbound email issue.
--
thanks,
Abhilash Raj (maxking)
3 years, 10 months
Re: Mailman 3 database restore unexpected behavior
by Tibor Molnar
Hi Odhiambo,
I have looked at the tables, and member and user tables have same fields but I'm not sure if that is the source of issue. In my experience is whenever I restore the databases the bounce message hits all users.
Followed through the mailman logs also and all I can see is everyone is getting unsubscribed from all lists. However all members are still stay part of every list.
Before restored the databases I have dropped them all the time.
On 28/03/2023 13:36, Odhiambo Washington wrote:
Have you looked at the tables and actually seen the duplicates?
SELECT COUNT...
Again, restoring a database usually should drop the table before restoring.
On Tue, Mar 28, 2023, 14:00 Tibor Molnar <tibor.molnar(a)waltoninstitute.ie<mailto:tibor.molnar@waltoninstitute.ie>> wrote:
Mailman3 built from source using virtual environment.
Version:
GNU Mailman 3.3.8 (Tom Sawyer)
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]
config file: /etc/mailman3/mailman.cfg
Issue:
I have restored the database number of times recently and experienced that every time after restoration (an hour after) all members getting an email that they are unsubscribed due to bounces, however non of the users are actually unsubscribed from any lists. It looks like there are duplicates in database and all the duplicates are getting unsubscribed?
Bounces message:
name(a)email.com<mailto:name@email.com> unsubscribed from [listname] mailing list due to bounces
Only restoring two files: mailman3 and mailman3web
Did anyone come acroos this behavior ?
It is not right that after restoring a database all 1000 user get an email that they are unsubscribed, but they are actually not.
I may do something wrong during the restoring process.
Please advise me.
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org<mailto:mailman-users@mailman3.org>
To unsubscribe send an email to mailman-users-leave(a)mailman3.org<mailto:mailman-users-leave@mailman3.org>
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
This message sent to odhiambo(a)gmail.com<mailto:odhiambo@gmail.com>
--
[Walton icon]
Tibor Molnar [website icon] <https://waltoninstitute.ie/> [linkedin icon] <https://www.linkedin.com/in/tibor-molnar-564293106> [twitter icon] <https://twitter.com/username> [facebook icon] <https://www.facebook.com/username>
System Administrator
tibor.molnar(a)waltoninstitute.ie<mailto:name.name@waltoninstitute.ie> | +353 (0)51 30 6145
Walton Institute, South East Technological University, West Campus, Carriganore, Waterford, Ireland, X91 P20H
[walton logo]<https://waltoninstitute.ie/> [setu logo] <https://www.setu.ie/> [DORA logo] <https://sfdora.org/read/>
________________________________
This email is subject to the following disclaimer(s) available at SETU Disclaimer.<https://www.wit.ie/about_wit/documents_and_policies/email-disclaimer>
Tá an ríomhphost seo faoi réir an tséanta/na séanta seo leanas atá le fáil ag SETU Séanad<https://www.wit.ie/about_wit/documents_and_policies/email-disclaimer>
[SETU]<https://www.setu.ie>
2 years, 3 months
Re: [Django] ERROR (EXTERNAL IP): Service Unavailable
by dancab＠caltech.edu
Thanks Mark. I can't see the held messages view at all for this list.
I don't think there's any sensitive info so I'm posting the entire message I get to the server admin email.
[Django] ERROR (EXTERNAL IP): Service Unavailable: /postorius/lists/cpa-announcement.caltech.edu/held_messages
root(a)mailman3.caltech.edu
Tue 2/23/2021 10:30 AM
Service Unavailable: /postorius/lists/cpa-announcement.caltech.edu/held_messages
Report at /postorius/lists/cpa-announcement.caltech.edu/held_messages
Service Unavailable: /postorius/lists/cpa-announcement.caltech.edu/held_messages
Request Method: GET
Request URL: https://mailman3.caltech.edu/postorius/lists/cpa-announcement.caltech.edu/h…
Django Version: 3.0.10
Python Executable: /opt/mailmanve/bin/python3
Python Version: 3.6.8
Python Path: ['/var/www/mailman-www', '/opt/mailmanve/bin', '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/opt/mailmanve/lib64/python3.6/site-packages', '/opt/mailmanve/lib/python3.6/site-packages']
Server time: Tue, 23 Feb 2021 18:30:19 +0000
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Request information:
USER: dancab
GET: No GET data
POST: No POST data
FILES: No FILES data
COOKIES:
_ga = 'GA1.2.1144367399.1613604556'
csrftoken = 'CtA6fFlrwV2fkDonIRHKdBX3V1lRQJA7UjRxYE2nCGEi6s65mFkkLDe3EEKkxgyo'
sessionid = 'm1dqvcmvcs12dxc8pui8u2wrz5cncam4'
META:
CSRF_COOKIE = 'CtA6fFlrwV2fkDonIRHKdBX3V1lRQJA7UjRxYE2nCGEi6s65mFkkLDe3EEKkxgyo'
HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, br'
HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.9,es-US;q=0.8,es;q=0.7'
HTTP_CONNECTION = 'Keep-Alive'
HTTP_COOKIE = '_ga=GA1.2.1144367399.1613604556; csrftoken=CtA6fFlrwV2fkDonIRHKdBX3V1lRQJA7UjRxYE2nCGEi6s65mFkkLDe3EEKkxgyo; sessionid=m1dqvcmvcs12dxc8pui8u2wrz5cncam4'
HTTP_HOST = 'mailman3.caltech.edu'
HTTP_PRAGMA = 'akamai-x-cache-on, akamai-x-cache-remote-on, akamai-x-check-cacheable, akamai-x-get-cache-key, akamai-x-get-extracted-values, akamai-x-get-ssl-client-session-id, akamai-x-get-true-cache-key, akamai-x-serial-no, akamai-x-get-request-id,akamai-x-get-nonces,akamai-x-get-client-ip,akamai-x-feo-trace'
HTTP_REFERER = 'https://mailman3.caltech.edu/postorius/lists/cpa-announcement.caltech.edu/'
HTTP_SEC_FETCH_DEST = 'document'
HTTP_SEC_FETCH_MODE = 'navigate'
HTTP_SEC_FETCH_SITE = 'same-origin'
HTTP_SEC_FETCH_USER = '?1'
HTTP_UPGRADE_INSECURE_REQUESTS = '1'
HTTP_USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36'
HTTP_X_FORWARDED_FOR = '131.215.248.132'
HTTP_X_FORWARDED_HOST = 'mailman3.caltech.edu'
HTTP_X_FORWARDED_PORT = '443'
HTTP_X_FORWARDED_PROTO = 'https'
HTTP_X_FORWARDED_SERVER = 'mailman3.caltech.edu'
PATH_INFO = '/postorius/lists/cpa-announcement.caltech.edu/held_messages'
QUERY_STRING = ''
RAW_URI = '/postorius/lists/cpa-announcement.caltech.edu/held_messages'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '49702'
REQUEST_METHOD = 'GET'
SCRIPT_NAME = ''
SERVER_NAME = '127.0.0.1'
SERVER_PORT = '8000'
SERVER_PROTOCOL = 'HTTP/1.1'
SERVER_SOFTWARE = 'gunicorn/20.0.4'
gunicorn.socket = <socket.socket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8000), raddr=('127.0.0.1', 49702)>
wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f31f8c22f60>
wsgi.file_wrapper = ''
wsgi.input = <gunicorn.http.body.Body object at 0x7f31f8c22ef0>
wsgi.input_terminated = True
wsgi.multiprocess = False
wsgi.multithread = False
wsgi.run_once = False
wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', 'dancab(a)its.caltech.edu'),)"
ALLOWED_HOSTS = ['api.mailman3.caltech.edu', 'mailman3.caltech.edu', 'www.mailman3.caltech.edu', 'www.lists.caltech.edu', 'lists.caltech.edu', 'localhost']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django_auth_ldap.backend.LDAPBackend', 'django.contrib.auth.backends.ModelBackend')"
AUTH_LDAP_ALWAYS_UPDATE_USER = True
AUTH_LDAP_BIND_DN = ''
AUTH_LDAP_BIND_PASSWORD = '********************'
AUTH_LDAP_SERVER_URI = 'ldap://ldapslave5.caltech.edu'
AUTH_LDAP_START_TLS = True
AUTH_LDAP_USER_ATTR_MAP = {'username': 'uid', 'first_name': 'givenName', 'last_name': 'sn', 'email': 'mail'}
AUTH_LDAP_USER_SEARCH = <LDAPSearch: ou=Accounts,ou=ITS,o=Caltech,c=US>
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = '/var/www/mailman-www'
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'}
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_FILTERS = {'js': ['compressor.filters.jsmin.JSMinFilter'], 'css': ['compressor.filters.css_default.CssAbsoluteFilter']}
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f31ffd0d268>
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = False
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/less', 'lessc {infile} {outfile}'), ('text/x-scss', 'sassc -t compressed {infile} {outfile}'), ('text/x-sass', 'sassc -t compressed {infile} {outfile}'))"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/var/www/mailman-www/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_URL = '/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SECURE = False
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', 'NAME': 'mailman3_prod', 'USER': 'mailman', 'PASSWORD': '********************', 'HOST': 'aus-sql1.caltech.edu', 'PORT': '3306', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'OPTIONS': {}, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'NAME': None, 'MIRROR': None}}}
DATABASE_ROUTERS = []
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%Y-%m-%d', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%Y', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M', '%m/%d/%y']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = 'postmaster(a)mailman3.caltech.edu'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_TABLESPACE = ''
DISALLOWED_USER_AGENTS = []
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_HOST_PASSWORD = '********************'
EMAIL_HOST_USER = ''
EMAIL_PORT = 587
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '********************'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = False
FILE_CHARSET = 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = 420
FILE_UPLOAD_TEMP_DIR = None
FILTER_VHOST = False
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
HAYSTACK_CONNECTIONS = {'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/var/www/mailman-www/fulltext_index'}}
IGNORABLE_404_URLS = []
INSTALLED_APPS = "('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount')"
INTERNAL_IPS = []
LANGUAGES = [('af', 'Afrikaans'), ('ar', 'Arabic'), ('ast', 'Asturian'), ('az', 'Azerbaijani'), ('bg', 'Bulgarian'), ('be', 'Belarusian'), ('bn', 'Bengali'), ('br', 'Breton'), ('bs', 'Bosnian'), ('ca', 'Catalan'), ('cs', 'Czech'), ('cy', 'Welsh'), ('da', 'Danish'), ('de', 'German'), ('dsb', 'Lower Sorbian'), ('el', 'Greek'), ('en', 'English'), ('en-au', 'Australian English'), ('en-gb', 'British English'), ('eo', 'Esperanto'), ('es', 'Spanish'), ('es-ar', 'Argentinian Spanish'), ('es-co', 'Colombian Spanish'), ('es-mx', 'Mexican Spanish'), ('es-ni', 'Nicaraguan Spanish'), ('es-ve', 'Venezuelan Spanish'), ('et', 'Estonian'), ('eu', 'Basque'), ('fa', 'Persian'), ('fi', 'Finnish'), ('fr', 'French'), ('fy', 'Frisian'), ('ga', 'Irish'), ('gd', 'Scottish Gaelic'), ('gl', 'Galician'), ('he', 'Hebrew'), ('hi', 'Hindi'), ('hr', 'Croatian'), ('hsb', 'Upper Sorbian'), ('hu', 'Hungarian'), ('hy', 'Armenian'), ('ia', 'Interlingua'), ('id', 'Indonesian'), ('io', 'Ido'), ('is', 'Icelandic'), ('it', 'Italian'), ('ja', 'Japanese'), ('ka', 'Georgian'), ('kab', 'Kabyle'), ('kk', 'Kazakh'), ('km', 'Khmer'), ('kn', 'Kannada'), ('ko', 'Korean'), ('lb', 'Luxembourgish'), ('lt', 'Lithuanian'), ('lv', 'Latvian'), ('mk', 'Macedonian'), ('ml', 'Malayalam'), ('mn', 'Mongolian'), ('mr', 'Marathi'), ('my', 'Burmese'), ('nb', 'Norwegian Bokmål'), ('ne', 'Nepali'), ('nl', 'Dutch'), ('nn', 'Norwegian Nynorsk'), ('os', 'Ossetic'), ('pa', 'Punjabi'), ('pl', 'Polish'), ('pt', 'Portuguese'), ('pt-br', 'Brazilian Portuguese'), ('ro', 'Romanian'), ('ru', 'Russian'), ('sk', 'Slovak'), ('sl', 'Slovenian'), ('sq', 'Albanian'), ('sr', 'Serbian'), ('sr-latn', 'Serbian Latin'), ('sv', 'Swedish'), ('sw', 'Swahili'), ('ta', 'Tamil'), ('te', 'Telugu'), ('th', 'Thai'), ('tr', 'Turkish'), ('tt', 'Tatar'), ('udm', 'Udmurt'), ('uk', 'Ukrainian'), ('ur', 'Urdu'), ('uz', 'Uzbek'), ('vi', 'Vietnamese'), ('zh-hans', 'Simplified Chinese'), ('zh-hant', 'Traditional Chinese')]
LANGUAGES_BIDI = ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE = 'en-us'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_HTTPONLY = False
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LANGUAGE_COOKIE_SAMESITE = None
LANGUAGE_COOKIE_SECURE = False
LOCALE_PATHS = []
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}}, 'handlers': {'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'DEBUG', 'class': 'logging.handlers.WatchedFileHandler', 'filename': '/var/www/mailman-www/logs/mailmansuite.log', 'formatter': 'verbose'}, 'console': {'class': 'logging.StreamHandler', 'formatter': 'simple'}}, 'loggers': {'django.request': {'handlers': ['mail_admins', 'file'], 'level': 'ERROR', 'propagate': True}, 'django': {'handlers': ['file'], 'level': 'ERROR', 'propagate': True}, 'hyperkitty': {'handlers': ['file'], 'level': 'DEBUG', 'propagate': True}, 'postorius': {'handlers': ['console', 'file'], 'level': 'DEBUG'}}, 'formatters': {'verbose': {'format': '%(levelname)s %(asctime)s %(process)d %(name)s %(message)s'}, 'simple': {'format': '%(levelname)s %(message)s'}}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = 'list_index'
LOGIN_URL = 'account_login'
LOGOUT_REDIRECT_URL = None
LOGOUT_URL = 'account_logout'
MAILMAN_ARCHIVER_FROM = "('127.0.0.1', '::1')"
MAILMAN_ARCHIVER_KEY = '********************'
MAILMAN_REST_API_PASS = '********************'
MAILMAN_REST_API_URL = '********************'
MAILMAN_REST_API_USER = '********************'
MANAGERS = []
MEDIA_ROOT = ''
MEDIA_URL = ''
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {40: 'danger'}
MIDDLEWARE = "('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')"
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
PASSWORD_HASHERS = '********************'
PASSWORD_RESET_TIMEOUT_DAYS = '********************'
POSTORIUS_TEMPLATE_BASE_URL = 'http://mailman3.caltech.edu'
PREPEND_WWW = False
Q_CLUSTER = {'timeout': 300, 'save_limit': 100, 'orm': 'default'}
ROOT_URLCONF = 'urls'
SECRET_KEY = '********************'
SECURE_BROWSER_XSS_FILTER = False
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_HSTS_INCLUDE_SUBDOMAINS = False
SECURE_HSTS_PRELOAD = False
SECURE_HSTS_SECONDS = 0
SECURE_PROXY_SSL_HEADER = "('HTTP_X_FORWARDED_SCHEME', 'https')"
SECURE_REDIRECT_EXEMPT = []
SECURE_REFERRER_POLICY = None
SECURE_SSL_HOST = None
SECURE_SSL_REDIRECT = False
SERVER_EMAIL = 'root(a)mailman3.caltech.edu'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SAMESITE = 'Lax'
SESSION_COOKIE_SECURE = False
SESSION_ENGINE = 'django.contrib.sessions.backends.db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer'
SETTINGS_MODULE = 'settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
SITE_ID = 1
STATICFILES_DIRS = []
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATIC_ROOT = '/var/www/mailman-www/static/'
STATIC_URL = '/static/'
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.i18n', 'django.template.context_processors.media', 'django.template.context_processors.static', 'django.template.context_processors.tz', 'django.template.context_processors.csrf', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django_mailman3.context_processors.common', 'hyperkitty.context_processors.common', 'postorius.context_processors.postorius']}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = True
USE_X_FORWARDED_PORT = False
WSGI_APPLICATION = 'wsgi.application'
X_FRAME_OPTIONS = 'DENY'
YEAR_MONTH_FORMAT = 'F Y'
4 years, 4 months
Can login at /admin, but cannot login at the account/login page.
by Stephen J. Turnbull
I'm sorry, I'm in a "second vaccination" fog at the moment (mending
but concentration lasts only about 10 minutes), so I'll answer what I
know offhand since Abhilash is busy and Mark is probably offline for
vacation by now. Perhaps later somebody can give a fuller answer.
pabsta(a)gmail.com writes:
> In comparison to the walkthrough, I did two changes; First, I used
> different port numbers , as it would otherwise conflict with other
> services on the server.
Thank you for the very precise description. It's very helpful.
> Second, I disabled the social accounts authentification mechanisms.
This should not be a problem since you disabled only the providers (I
think the socialauth base module is required even if you don't use it.)
> I am still running on sqllite, as I am still testing things (one
> task at a time).
This should be no problem.
> Using my "superuser" credentials, I can log to the admin section in
> a fraction of a second.
If you mean the django admin page https://localhost:800something/admin,
that doesn't involve any part of Mailman at all.
> However, if logout and attempt to login from the postorius website,
> I get a « 502 bad gateway » error after roughly 30 seconds. The
> logs in Systemctl for gunicorn register a "DIGEST-MD5 common mech
> free" error message
I would guess this is also just a connection drop because it doesn't
receive the expected response within timeout.
> and the nginx error logs just show a connection drop:
>
> 2021/08/12 18:11:09 [error] 26026#26026: *31 upstream prematurely closed connbection while reading response header from upstream, client: [ip redacted], server:
> [domain redacted], request: "POST /accounts/login/ HTTP/2.0", upstream: "http://127.0.0.1:8006/accounts/login/", host: "[domain redacted]", referrer: "https://[domain redacted]/accounts/login/?next=/mailman3/lists/"
a >
> 1- If I attempt to login with improper credentials, the services
> respond normally, and I get the expected red box with "invalid
> credentials" message on the postorius page. It is only when proper
> credentials are given that the connection fails.
The only thing I can think of is that having sucessfully logged in,
Postorius can't find Mailman core to populate your user page. That
would explain why an invalid login (which is entirely within Postorius
and Django) proceeds as expected, but a valid login would fail so
spectacularly.
> # Mailman API credentials
> MAILMAN_REST_API_URL = 'http://localhost:8005'
Is Mailman core listening on localhost:8005? Is there possibly a
conflict with something else on that port?
Steve
3 years, 10 months
Re: Newbie question 2: Rewriting issue for bounced emails
by William Oliver
On Sat, 2021-12-25 at 10:29 -0800, Mark Sapiro wrote:
>
>
> This is the issue.
>
> See
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h…
>
> for information on configuring Postfix with mailing lists in a
> virtual
> domain.
>
>
Thanks, at least that changed the error :-)
Clearly that was the problem, but it uncovered a new one -- the mailman
vmap doesn't seem to be created. In the installation docs, it says
about /opt/mailman/mm/data/postfix_vmap:
"If this is done and the configured MTA is Postfix, Mailman will
create an additional /path-to-mailman/var/data/postfix_vmap file with
mappings from the example.com addresses to the corresponding addresses
in the alias_domain and will use the alias_domain in the other files. "
But, alas, I get a "temporary lookup failure" which isn't all that
helpful. However, if I wander over to /var/logs/syslog, I see:
Dec 25 15:18:23 libertyfp postfix/smtpd[20248]: error: open database
/opt/mailman/mm/var/data/postfix_vmap.db: No such file or directory
Dec 25 15:18:23 libertyfp postfix/smtpd[20248]: connect from
localhost[::1]
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: error: open
database /opt/mailman/mm/var/data/postfix_vmap.db: No such file or
directory
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: warning:
hash:/opt/mailman/mm/var/data/postfix_vmap is unavailable. open
database /opt/mailman/mm/var/data/postfix_vmap.db: No such file or
directory
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: warning:
virtual_alias_domains: hash:/opt/mailman/mm/var/data/postfix_vmap:
table lookup problem
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: warning:
virtual_alias_domains lookup failure
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: warning:
hash:/opt/mailman/mm/var/data/postfix_vmap is unavailable. open
database /opt/mailman/mm/var/data/postfix_vmap.db: No such file or
directory
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: warning:
virtual_alias_domains: hash:/opt/mailman/mm/var/data/postfix_vmap:
table lookup problem
Dec 25 15:18:23 libertyfp postfix/trivial-rewrite[20252]: warning:
virtual_alias_domains lookup failure
Dec 25 15:18:23 libertyfp postfix/smtpd[20248]: NOQUEUE: reject: RCPT
from localhost[::1]: 451 4.3.0 <billo(a)billoblog.com>: Temporary lookup
failure; from=<mailman(a)libertyfp.org> to=<billo(a)billoblog.com>
proto=ESMTP helo=<mail.libertyfp.org>
Dec 25 15:18:23 libertyfp postfix/smtpd[20254]: error: open database
/opt/mailman/mm/var/data/postfix_vmap.db: No such file or directory
and on and on. This looks like a clue to me -- that mailman3 did not
automagically make postfix_vmap.db.
Is there a "make-the-freaking-virtual- map-database" command in
mailman3, or is this postfix thing? Should I just touch the thing and
create an empty file? I'm a little scared to... The error message
makes me think that it should be populated with something...
Sigh. That's enough for today. I'm off to eat Christmas turkey.
Thanks for all your help this morning.
billo
3 years, 6 months
Sending a message to a subscription applicant when a subscription request is made
by Allan Hansen
Dear Mailman 3 wizards,
Our subscription process requires that the applicant answers a few questions and sends them to the list moderator. Not having a template that gets sent on all subscribe requests, I'm using the 'list:admin:action:subscribe' template. This gets sent to the list moderator, who then sends it forward to the applicant. This is rather tedious. I would like the template to be sent directly to the applicant.
There is a template, 'list:user:action:subscribe' that almost does this, but it gets sent only if the email address of the applicant is unverified. A good number of our applicants are already on our lists, so they would not get this template if we set the Subscription Policy to 'Confirm, then moderate'.
Our Subscription Policy is currently: 'Moderate' for all lists.
I figure that three modifications can be used to fix this:
a. That we change the lines in the Python program in our installation that send 'list:admin:action:subscribe' to the list admins to instead send the template to the applying subscriber.
b. That we change the lines in the Python program in our installation that send 'list:user:action:subscribe' to the subscriber with an unverified address to always do this, even when the incoming address has already been verified.
c. That the current 'list:user:action:subscribe' be changed in published Mailman 3 source to 'list:user:action:confirm' and a new template ''list:user:action:subscribe' be set up to be sent when any subscribe request is made. This is more work, of course, so not likely a good thing to hack locally. It could be useful to other users.
Any suggestions on how to best and most quickly do the above would be appreciated, starting with (a.) or (b.).
Clearly, (a.) and (b.), if we do one of those, are hacks that will need to be applied upon Mailman updates. That's fine by me, as I'm in rush to get this done. I'd welcome pointers as to which file to change and how. Of these two options, which would be the least work?
(c.) could be entered as a longer-term enhancement request. I can submit that request if you like.
Yours,
Allan Hansen
P.S.: Thank you for the enhancement that lets moderators see the member lists. It has been a big help for us.
GNU Mailman 3.3.8 (Tom Sawyer)
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]
config file: /opt/mailman/mm/mailman.cfg
db url: postgresql://mailman:VWy26K4pUfU4jw@localhost/mailman
devmode: DISABLED
REST root url: http://localhost:8001/3.1/
REST credentials: restadmin:restpass
2 years, 3 months
Re: Mailman backend maintenance task
by Abhilash Raj
On Sun, Aug 22, 2021, at 10:54 AM, eugenio.jordan(a)esa.int wrote:
> Dear all.
>
> My apologies if the answer to this question is obvious, but I am pretty
> newbie for 3 version, and to be very honest I haven't had enough time
> yet to have a deep look into the documentation.
>
> Our customer is currently using PostGRESQL as backend, and we would
> like to perform some maintenance tasks, namely running vacuum full, or
> at least trying to rebuild hyperkitty_email primary key related index.
> We have been asked on the real impact of putting in place such
> initiative. Though the latter is related to archiving, I haven't found
> a way to stop just Hyperkitty or Django related processes other than
> stopping Mailman's core, hence preventing mails addressed to
> distribution lists from being delivered, could you please confirm if I
> am correct?
You can indeed stop Hyperkitty separate from stopping Core. Depending on how your client installed Mailman, there are two separate scripts/processes/commands to start and stop Mailman Core and Web components.
You should be able to safely stop Hyperkitty, run maintenance and re-start it. Core will continue to function during this duration, sending out emails and such. The archiving will obviously not happen, but it will be queued and Core we retry to archive that, which should succeed whenever HK comes back up.
>
> Regarding the former, as far as I have read, the "mappings" lists ->
> addresses are stored just in the database, so if we run some kind of
> procedure or task like vacuum which will lock exclusively tables, or
> want anyway to have the database stopped for a cold backup or whatever,
> Mailman willl not work, that is, again the mails addressed to the
> distribution lists will not be delivered. Will you please confirm this
> point, too?
>
Both Core and Web components have separate databases. Although, technically speaking, the database name is passed in through configs and you can pass in the same database name to both and since they use separate table names, there is a good chance it would work.
Considering that your client has configured separate databases for both, you can run maintenance for the Web database without affecting any operation of Core (other than archiving). Core stores the mappings and data related to MailingLists. While Hyperkitty also has a partial copy of that data in its database that it needs to maintain the archives, that data can be synchronized with a command and is also done routinely using a cron job.
A database backup would be a good idea either way.
Hope this helps!
--
thanks,
Abhilash Raj (maxking)
3 years, 10 months
Re: Mailman backend maintenance task
by Abhilash Raj
> On Aug 24, 2021, at 11:22 AM, Stephen J. Turnbull <stephenjturnbull(a)gmail.com> wrote:
>
> Eugenio Jordan (external) writes:
>
>> The idea we were considering was more or less stopping all of the
>> components, then starting up only PostGRESQL, and then performing each
>> one of the mentioned tasks.
>
> In that case, your Mailman installation will basically be down (no web
> presence, no deliveries) for the duration of the database maintenance.
>
>> You mention that regardless the connectivity that
>> Mailman's core processes might have with the database, the emails will
>> be received by and queued until the connection is back again and they
>> can be delivered to the distribution lists' members. Could you please
>> confirm whether this task would be actually delegated to Postfix (or
>> any MTA integrated with Mailman)?
>
> If Mailman core is running, it will accept the mail and store it in
> its own queue. If not, the MTA will store it in its deferred queue
> until Mailman comes back up.
>
> The big problem with delegating this to the MTA is that after two days
> the MTA probably won't retry for whole days, but if you do an MTA
> queue flush, it will probably stop other mail delivery for quite a
> while, since it will be occupied with processing the Mailman backlog.
>
> The way Mailman works is that there is a master process, which manages
> a suite of runners. Each runner is responsible for a particular
> stage. The LMTP runner does nothing except accept messages from the
> MTA, and store each one in a file for the next runner in the chain.
Although, looking at the code[1], it seems like LMTP runner will actually
try to verify that the incoming email in destined to a valid email address
before it will accept the email.
[1]: https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/runners/lmtp.p…
So, if the database isn’t reachable, I think LMTP runner will start rejecting
emails from MTA. This is perhaps handling the situation where the transport
maps are out-of-date and Mailman receives an email from MTA for a list
that has been deleted.
While I haven’t tested this situation, I don’t think it would work. So
far I am seeing that holding emails off at MTA level and letting it retry after
the database maintenance has been complete might be the only thing
that works if the entire Postgresql instance has to be brought down.
If MTA will retry continuously for 2 days, would that not be enough time
to complete the migration and bring the database back up?
--
thanks,
Abhilash Raj (maxking)
3 years, 10 months