September 2024 - Mailman-users - lists.mailman3.org

Cache mailman3
by pau.baiges＠csuc.cat 22 Dec '24

22 Dec '24

Is there a way to delete the mailman3 cache? It's because it gives us this error and we think it's because of the cache. Internal Server Error: /mailman3/accounts/login/ TemplateDoesNotExist at /accounts/login/ django_saml2_auth/error.html Request Method: GET Request URL: http://llistes.csuc.cat/mailman3/accounts/login/?next=/mailman3/postorius/l… Django Version: 2.2.12 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.10 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: dv., 15 Març 2024 10:33:00 +0100 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware') Template loader postmortem Django tried loading these templates, in this order: Using engine django: This engine did not provide a list of tried templates. Traceback: File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/utils.py" in wrapper 179. result = function(request) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/views.py" in signin 271. saml_client = get_saml_client(get_assertion_url(request), acs) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/saml.py" in get_saml_client 169. metadata = get_metadata(user_id) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/saml.py" in get_metadata 131. raise SAMLAuthError("Invalid metadata URL.", extra={ During handling of the above exception (Invalid metadata URL.), another exception occurred: File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/utils.py" in wrapper 181. result = handle_exception(exc, request) File "/usr/local/lib/python3.8/dist-packages/django_saml2_auth/utils.py" in handle_exception 162. return render(request, File "/usr/lib/python3/dist-packages/django/shortcuts.py" in render 36. content = loader.render_to_string(template_name, context, request, using=using) File "/usr/lib/python3/dist-packages/django/template/loader.py" in render_to_string 61. template = get_template(template_name, using=using) File "/usr/lib/python3/dist-packages/django/template/loader.py" in get_template 19. raise TemplateDoesNotExist(template_name, chain=chain) Exception Type: TemplateDoesNotExist at /accounts/login/ Exception Value: django_saml2_auth/error.html Request information: USER: AnonymousUser

3 2

Re: mail stuck in "out" queue
by Dmitry Makovey 18 Dec '24

18 Dec '24

So our mailman got stuck again, but I did follow your advise. Here's what I've found: On 3/29/19 2:58 PM, Mark Sapiro wrote: > On 3/29/19 2:28 PM, Dmitry Makovey wrote: >> I've had encountered this situation now twice: mail is being stuck in >> "out" queue and nothing in the logs indicates the issue. Is there a way >> to ask mailman-core politely to attempt delivery (perhaps with increased >> debugging in the logs) *without* restarting the core (which does solve >> *this* problem but creates other problems, like availability of the >> mailman as a whole) > > > Is the "out" runner running? yes: # docker top mailman-core | grep -F out: 100 21681 13215 0 Mar28 ? 01:14:16 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg > ps -fwwA |grep runner=out > > If not, you should be able to start it as the Mailman user with a > command like > > /path/to/python3 /path/to/mailman/bin/runner -C /path/to/mailman.cfg > --runner=out:0:1 > > See 'ps -fwwA|grep runner' for examples > > If the out runner is running, is there a file named *.bak along with all > the *.pck files in queue/out/ there is one, yes: # ls -l /srv/mail/mailman/core/var/queue/out/*.bak -rw-rw---- 1 100 65533 21512 Apr 14 19:45 /srv/mail/mailman/core/var/queue/out/1555296331.8386078+2c905825b82708e5a682b872b0aff320c9dd0b2e.bak what does it indicate and what should my steps be to resolve this (other than restarting queues)? > And if it's not, is there anything in Mailman's logs to indicate why it > died? > > Also, 'mailman restart' should only take a matter of seconds, but if the > out runner is running but not processing its queue, I think sending it a > SIGINT should restart it. SIGINT killed the process allright, but it didn't auto-respawn. I ended up spawning it with: /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg but soon realized it doesn't go to background nicely so I've nohup'ed it in the end. but it did solve my problem. -- Sr System and DevOps Engineer SoM IRT

6 17

How to Increase outgoing mail volume for Mailman 3
by Brian Carpenter 09 Dec '24

09 Dec '24

Hello everyone, We had a new client come on board that brought a list that had the largest out-going mail volume that I have ever dealt with. So I did what any sane server administrator would do that is running Mailman 3: I ran to Mark Sapiro for help! Here is what the Sage of all things Mailman counseled me to do: 1. Disabled VERP delivery. VERP can help improve mail delivery by delivering 1 message per SMTP transaction (I think I got that right). Using VERP is counter-productive as it can seriously delay outgoing mail delivery for a high-volume list. So you can use something like the following in the MTA section of your mailman.cfg: [mta] verp_confirmations: no verp_personalized_deliveries: no verp_delivery_interval: 0 2. I configured Mailman 3 to run 4 outgoing runners. The default is 1. This action was the game changer. It allows Mailman to handle 4 incoming messages at a time instead of 1. My client's incoming list volume was high enough where it was overwhelming the single outgoing runner. Here is what I added to the mailman.cfg file: [runner.out] instances: 4 Doing this step immediately solved the backlog in var/queue/out. 3. If you start up with a new server for a list that is high volume then keep an eye out for Proofpoint and Vade Security as they will block your new IP address quicker than Mark Sapiro can say "This is a bug. See <some Gitlab.com url>" Mark, thank you! -- Brian Carpenter Harmonylists.com Emwd.com

5 10

Aborted connection message in log in every 5 minutes
by zoltan＠circle-interactive.co.uk 03 Dec '24

03 Dec '24

Hi Everyone, we have mailman 3.3.8 venv test installation which in general works fine however our mysql error log is filled up with messages below because they are so frequent. 2023-02-24 0:31:30 199567 [Warning] Aborted connection 199567 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:36:30 201063 [Warning] Aborted connection 201063 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:41:31 202559 [Warning] Aborted connection 202559 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) 2023-02-24 0:46:31 204056 [Warning] Aborted connection 204056 to db: 'mailman' user: 'mailman' host: 'localhost' (Got an error reading communication packets) As suggested here https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… I tried to correlate with error logs and /opt/mailman/web/logs/mailmanweb.log shows some hyperkitty.lib.utils Failed to update the fulltext index: (2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (111)" /opt/mailman/web/logs/uwsgi-error.log Message: OperationalError(2013, "Lost connection to MySQL server at 'handshake: reading initial communication packet', sy stem error: 104") but they have not such frequency so unsure if they are related. Thanks for the suggestions

6 16

Moving existing lists to a new domain
by Jeremy Stanley 21 Nov '24

21 Nov '24

I've moved lists and archives between domains in Mailman 2.1 and Pipermail, but not yet with Mailman 3 and Hyperkitty. I have a working multi-domain deployment, and one of the organizations using it is working on changing their domain name. Forwarding incoming messages at the MTA and setting up redirects on the Web server are simple enough, but internally within MM3/Django reside the bits I'm less certain about. Since all the lists for the given domain will be moving to the same unused new domain on the existing server, and can move at the same time, I suppose this could be done through (offline if necessary) backend database update queries. Is there anything outside the DB which would need updating or regenerating for this too? Is this how others have tackled similar tasks in the past, or is it recommended to export/import lists one at a time out of one domain and into another? Any tips are greatly appreciated! -- Jeremy Stanley

2 6

Migrating from old V2 server to new V3 server. Questions.
by jkoral＠luc.edu 14 Nov '24

14 Nov '24

We have figured out how to migrate from one server to another, but after removing the list from the old server, the e-mails are still trying to go there. Is there something that needs to be done in mailman so that the e-mail list will hit the new server instead? Does it just take time? Thank you, Joe Koral

2 26

Backup and restore a single list
by sherin＠raidm.com 06 Nov '24

06 Nov '24

Hello, I am new to mailman3 . I am looking for the procedure to backup and restore a single mailing list from mailman3. If any one know the procedure or the documentation , please point me

3 7

Timeout error on anonymous subscribe
by vogt＠spamcop.net 25 Oct '24

25 Oct '24

Hi, I am running a mailman 3.3.9 server (pip installation). It's working fine so far. However, I have problems setting a template. I set a '[list:user:action:subscribe]' template for a list with some simple text like "test1234". When someone wants to subscribe on the list page without creating an account, it takes five seconds until "HTTPSConnectionPool(host='lists.example.org', port=443): Read timed out. (read timeout=5)" appears on the page. The mail address has been added pending confirmation but no mail gets send. mailman.log shows just exceptions about the timeout itself: Sep 09 15:01:45 2024 (2070549) deque: do_confirm_verify Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.11/site-packages/urllib3/connectionpool.py", line 536, in _make_request response = conn.getresponse() ^^^^^^^^^^^^^^^^^^ ... requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='lists.dkrz.de', port=443): Read timed out. (read timeout=5) [09/Sep/2024:15:01:45 +0200] "POST /3.1/members HTTP/1.1" 400 132 "-" "GNU Mailman REST client v3.3.5" The access log suggests that getting the template took 5s: ==> /opt/mailman/mm/var/logs/access.log <== 127.0.0.1 - - [09/Sep/2024:13:01:45 +0000] "GET /mailman3/api/templates/list/community.lists.example.org/list:user:action:subscribe HTTP/1.1" 200 8 "-" "python-requests/2.32.3" Running curl on that url returns immediately $ curl -v 'http://localhost:8000/mailman3/api/templates/list/community.lists.example.o…' I have captured the network traffic on localhost interface lo and I can see that the request is sent through the tcp connection and acknowledged. Then 5 seconds later the server (port 8000) sends a push data with the response and closes the connection with FIN. The data exchanged on the connection looks like this: GET /mailman3/api/templates/list/community.lists.example.org/list:user:action:subscribe HTTP/1.1 Host: lists.example.org User-Agent: python-requests/2.32.3 Accept-Encoding: gzip, deflate Accept: */* X-Forwarded-Proto: https X-Forwarded-For: 2001:638:1234:1224::111 X-Forwarded-Host: lists.example.org X-Forwarded-Server: lists.example.org Connection: Keep-Alive HTTP/1.1 200 OK Server: gunicorn Date: Mon, 09 Sep 2024 13:01:45 GMT Connection: close Content-Type: text/plain; charset=utf-8 X-Content-Type-Options: nosniff Referrer-Policy: same-origin Cross-Origin-Opener-Policy: same-origin X-Frame-Options: DENY Vary: Accept-Language, Cookie Content-Language: en Content-Length: 8 test1234 The timestamp 13:01:45 is the time of the timeout. The request started five seconds before that. It looks to me as if the connection/keepalive handling is not working properly or the server isn't pushing the response immediately, but it gets buffered until the 5s timeout. Occassionally, I have capture a requests, where the client side would send a FIN to the server and only then the server would send the http response with the final packets. Does anyone have an idea how I can fix this? Thanks, Gerald

3 9

Logging subscribes/unsubscribes
by Brian Carpenter 22 Oct '24

22 Oct '24

Does Mailman 3 log subscribes and unsubscribes for each list the way Mailman 2 did? Brian

5 7

makemigrations --merge error after Debian upgrade
by Chupin Maxime 03 Oct '24

03 Oct '24

Hi, I upgraded my debian version, and since, it seems that I have a problem. I tried `mailman-web migrate` (and `mailman-web collectstatic` etc.) and I get the error message: ``` CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0010_auto_20220618_2315, 0014_schedule_cluster in django_q; 0014_auto_20220618_2315, 0018_alter_emailtemplate_language in postorius). To fix them run 'python manage.py makemigrations --merge' ``` So I tried `mailman-web makemigrations --merge` that ended with ``` PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/postorius/migrations/0019_merge_20240930_1159.py' ``` There is no such a file in the migration directory. Any idea ? Thanks in advance, All the best Maxime Chupin

4 19