Search results for query "sapiro"
- 6186 messages
[MM3-users] Re: Member Issue Discovered
by Abhilash Raj
I am a bit late to this thread, but I am going to try to respond to some of the points raised in the thread.
On 10/19/20 7:42 PM, Brian Carpenter wrote:
> On 10/19/20 10:23 PM, Mark Sapiro wrote:
>> And the import21 created the user and address records for this user.
> Does the same thing for a new subscriber as well. So there is no pathway
> to change a real name that is associated with an email address. None.
> Zilch.
Yes, this is a limitation of the REST API, the display_name attribute can't be updated. Hence, Postorius can't provide a way for anyone to update the name. I have opened an issue for this[1] and I think it should be allowed.
[1]: https://gitlab.com/mailman/mailman/-/issues/786
> I am admin for a list. Users got subscribed with bulk import during
> transition from MM2 to MM3.
> So in a sense I am responsible for adding these to a db. BUT there is
> no way for me to remove them. Only mailman admin with access to the db
> can do it manually.
The fact that a user's memberships & User as an entity that owns email address is a design choice that Mailman developers made after some experience of using and administering Mailing lists running on Mailman 2. It was annoying to have to manage several different passwords and accounts to manage membership on same mailing list server.
The policy decision that Mailman would allow a user/address to exist even if there are no subscriptions on it isn't un-resonable in certain environments and maybe it is in some. I don't think anyone here denies that either of those behavior could be an expectation from someone using Mailman. But Mailman is an open source project with decent amount of extensibility via HTTP API & Python plugin system.
If the default policy doesn't match the expectations of some users, I don't think it is difficult to cleanup users/address with 0 subscriptions in a nightly cron script. It is trivial with some basic scripting and we are happy to help out with such a mechanism. I wouldn't be opposed to directly adding a command in Core that lets you clean up all sorts of stuff.
On Wed, Oct 21, 2020, at 10:20 AM, Apollinaris Schöll wrote:
> There might be a handful of these.
>
> But the bigger problem is a user can’t be removed entirely.
>
> User can delete account and unsubscribe from all lists. But the email
> and user name is still kept in the db. Just tested with Affinity
> List admin can’t do it either.
Front-ends like Affinity should be able to implement such a policy by checking subscriptions of an address on every un-subscription call and delete the user & address if it comes out to be 0. The API is documented here[2], but if there are gaps in there like I mentioned above in my email, please open issues so we know about them.
[2]: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/…
> I think the correct action is to remove all records if a email is
> unsubscribed from all lists and that user has not registered for an
> account.
> If a user has registered for an account all information including
> display name must be removed from that particular list subscription.
> It should not matter if a user unsubscribes or a list admin
> unsubscribes. There shouldn’t be any data kept.
Also, like mentioned above, as a list admin, you can request the Site Admin to get the address fully scrubbed or if this is an expectation from all list admins, the Site Admin has the ability to automate such a task. The fact the Mailman doesn't do it by default is because the definition of "correct" action is different.
User should have the ability to delete themselves from the system and if it doesn't wipe everything yet from database, it should be considered a bug in Postorius and be fixed. Postorius track users separately than Core, so the it should make sure to delete in both places and there are appropriate APIs in core to delete Users and Addresses.
Now, the ability in Postorius to edit the display_name. The side effect of the model where User is a Site level thing, makes it impossible to customize user's display_name on a per-list basis, for both list admin and the user themselves. That is a trade-off and it is possible that there were use-cases where that was desirable in Mailman 2, but with the superior subscription management functionality in Postorius, I feel the decision is justified. I respect that some of y'all think that it probably isn't if it doesn't suit you particular use cases.
From the thread, I gather than folks mostly agree that a list admin shouldn't be allowed to edit a user's display name. Site Admins however should be able to edit that, which brings me to next point.
Like Mark brought up, there is definitely a big missing feature of User management in Postorius. This future management page would be for Site Admins (Superusers) to manage Users and have the ability to change pretty much everything about users including their addresses, memberships and preferences and also be able to wipe off the users from all the databases if they want to. If there are more features than that which folks think should be a part of this, please feel free to add to it. As it is hopefully implied from the description, it is a big task and needs substantial chunk of time, part of which this hasn't been picked up yet.
>
>
>
>
>
>
> > On Oct 21, 2020, at 1:43 AM, Jörg Schulz <info(a)joergschulz.de> wrote:
> >
> > So, the issue seems to be, like in the following scenario
> >
> > We imagine a list server for the domains fruiteaters and meateaters.
> > We have two listadmins, anna (fruit) and carl (meat). They don't share anything.
> > We have a user who subscribes to fruiteaters and meateaters using his address hunger(a)eatall.com
> >
> > He wants to show up in the fruiteaters list as fullname wormy wonder, but in the meateaters list as fritz the cat.
> > While he can do so using two different email addresses, he cannot do so using
> >
> > wormy wonder <hunger(a)eatall.com>
> > fritz the cat <hunger(a)eatall.com>
> >
> > as from: addresses because all requests will be matched to the one email address.
> >
> > Worse, if Carl would be able to change the name to from wormy wonder to fritz the cat, Anna would see that name change and like to change it back because the user she had known, user has been identified as wormy wonder.
> >
> > So, if these user properties are stored somewhere not related to the lists, we don't seem to have a chance of display name changes by list admins as long as the data model isn't changed. This doesn't seem to be easy.
> >
> > So, only some kind of super-Admin should currently be able to change these display names, and these changes would be visible for all lists, and one email address can only have one display name for all lists.
> >
> > Not optimal for some rare conditions - that's the way it seems to be???
> > _______________________________________________
> > Mailman-users mailing list -- mailman-users(a)mailman3.org
> > To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
--
thanks,
Abhilash Raj (maxking)
5 years, 4 months
[MM3-users] Re: using SSH/TLS with external MTA
by Odhiambo Washington
On Sun, Jul 28, 2024 at 4:42 PM Roland Giesler via Mailman-users <
mailman-users(a)mailman3.org> wrote:
>
> On 2024/07/28 15:26, Roland Giesler via Mailman-users wrote:
> > On 2024/07/28 00:39, Mark Sapiro wrote:
> >> On 7/27/24 15:16, Roland Giesler via Mailman-users wrote:
> >>>
> >>> The server runs power-mailinabox, which uses Postfix indeed. All my
> >>> mail, and quite a few others' mail is sent from the using SMTP-Auth
> >>> on port 465. Actually, this message is sent via that server too.
> >>> So it's clear to me that MM3 is not authenticating for some reason.
> >>
> >> What do you see in Mailman's smtp.log and in the mail.log on the mail
> >> server?
> >
> > The smtp log is in syslog...
>
> I also see this in /var/log/mailman3/web/mailman-web.log
>
> *** Starting uWSGI 2.0.20-debian (64bit) on [Sun Jul 28 15:27:48 2024] ***
> compiled with version: 11.2.0 on 21 March 2022 11:00:44
> os: Linux-5.15.108-1-pve #1 SMP PVE 5.15.108-2 (2023-07-20T10:06Z)
> nodename: mailman
> machine: x86_64
> clock source: unix
> pcre jit disabled
> detected number of CPU cores: 1
> current working directory: /
> detected binary path: /usr/bin/uwsgi-core
> setgid() to 33
> setuid() to 33
> chdir() to /usr/share/mailman3-web
> your processes number limit is 513922
> your memory page size is 4096 bytes
> detected max file descriptor number: 1024
> lock engine: pthread robust mutexes
> thunder lock: disabled (you can enable it with --thunder-lock)
> uwsgi socket 0 bound to UNIX address /run/mailman3-web/uwsgi.sock fd 4
> Python version: 3.10.12 (main, Mar 22 2024, 16:50:05) [GCC 11.4.0]
> Python main interpreter initialized at 0x561ce813d1d0
> python threads support enabled
> your server socket listen backlog is limited to 100 connections
> your mercy for graceful operations on workers is 60 seconds
> mapped 166752 bytes (162 KB) for 2 cores
> *** Operational MODE: threaded ***
> /usr/lib/python3/dist-packages/django_q/conf.py:139: UserWarning: Retry
> and timeout are misconfigured. Set retry larger than timeout,
> failure to do so will cause the tasks to be retriggered before
> completion.
> See
> https://django-q.readthedocs.io/en/latest/configure.html#retry for
> details.
> warn(
> WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter
> 0x561ce813d1d0 pid: 113 (default app)
> *** uWSGI is running in multiple interpreter mode ***
> spawned uWSGI master process (pid: 113)
> spawned uWSGI worker 1 (pid: 339, cores: 2)
> [uwsgi-daemons] spawning "python3 manage.py qcluster" (uid: 33 gid: 33)
> /usr/lib/python3/dist-packages/django_q/conf.py:139: UserWarning: Retry
> and timeout are misconfigured. Set retry larger than timeout,
> failure to do so will cause the tasks to be retriggered before
> completion.
> See
> https://django-q.readthedocs.io/en/latest/configure.html#retry for
> details.
> warn(
> System check identified some issues:
>
> WARNINGS:
> django_mailman3.MailDomain: (models.W042) Auto-created primary key used
> when not defining a primary key type, by default
> 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> DjangoMailman3Config.default_auto_field attribute to point to a subclass
> of AutoField, e.g. 'django.db.models.BigAutoField'.
> django_mailman3.Profile: (models.W042) Auto-created primary key used
> when not defining a primary key type, by default
> 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> DjangoMailman3Config.default_auto_field attribute to point to a subclass
> of AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Attachment: (models.W042) Auto-created primary key used when
> not defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Email: (models.W042) Auto-created primary key used when not
> defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Favorite: (models.W042) Auto-created primary key used when
> not defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.LastView: (models.W042) Auto-created primary key used when
> not defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.MailingList: (models.W042) Auto-created primary key used when
> not defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Profile: (models.W042) Auto-created primary key used when not
> defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Tag: (models.W042) Auto-created primary key used when not
> defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Tagging: (models.W042) Auto-created primary key used when not
> defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Thread: (models.W042) Auto-created primary key used when not
> defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.ThreadCategory: (models.W042) Auto-created primary key used
> when not defining a primary key type, by default
> 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> hyperkitty.Vote: (models.W042) Auto-created primary key used when not
> defining a primary key type, by default 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> HyperKittyConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> postorius.EmailTemplate: (models.W042) Auto-created primary key used
> when not defining a primary key type, by default
> 'django.db.models.AutoField'.
> HINT: Configure the DEFAULT_AUTO_FIELD setting or the
> PostoriusConfig.default_auto_field attribute to point to a subclass of
> AutoField, e.g. 'django.db.models.BigAutoField'.
> 15:27:55 [Q] INFO Q Cluster low-vermont-harry-fillet starting.
> 15:27:55 [Q] INFO Process-1 guarding cluster low-vermont-harry-fillet
> 15:27:55 [Q] INFO Q Cluster low-vermont-harry-fillet running.
> 15:27:55 [Q] INFO Process-1:3 pushing tasks at 386
> 15:27:55 [Q] INFO Process-1:2 monitoring at 385
> 15:27:55 [Q] INFO Process-1:1 ready for work at 384
>
# Asynchronous tasks
# https://django-q.readthedocs.io/en/latest/configure.html
Q_CLUSTER = {
'name': 'mailman3-web',
'workers': 8,
'retry': 360,
'timeout': 300,
'save_limit': 100,
'orm': 'default',
}
# Maintain type of autogenerated keys going forward
#
https://docs.djangoproject.com/en/3.2/releases/3.2/#customizing-type-of-aut…
DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
1 year, 7 months
[MM3-users] Re: error changed after restart
by Abhilash Raj
On Sun, Feb 7, 2021, at 1:07 AM, Guillermo Hernandez (Oldno7) via Mailman-users wrote:
> On 6/2/21 21:19, Abhilash Raj wrote:
> >
> >
> > On Sat, Feb 6, 2021 at 19:44, Guillermo Hernandez (Oldno7) via
> > Mailman-users <mailman-users(a)mailman3.org> wrote:
> >> On 6/2/21 18:08, Abhilash Raj wrote:
> >>
> >> On Sat, Feb 6, 2021, at 3:04 AM, Guillermo Hernandez (Oldno7) via
> >> Mailman-users wrote:
> >>
> >> I restarted de server and the error changed. Now the log
> >> shows "KeyError: 'subscription_mode'":
> >>
> >> Did you also restart Mailman Core after the upgrade?
> >>
> >> Yes, indeed: I stopped mailman core and all the processes related.
> >> Did the upgrades. Started all again. Find the errors in the web user
> >> interface. Stopped all again. Looked for errors in the log. Restarted
> >> the complete server. Found the second error that this second mail is
> >> about to. It happens when, in the main page that shows all the lists
> >> you click to see one of them. It seems to me that it has been
> >> database structure changes in django that the upgrade is not aware...
> >> but it's a very long shot from my side.
> > `subscription_mode` was added in Mailman Core 3.3.2 and it is actually
> > a derived atrribute and not stored in Database. Mailman's API should be
> > returning this attribute for each Member, but for some reason it seems
> > to me like it isn't doing that even though do have Mailman 3.3.3 running
> > like you said.
> > If you have Curl installed, can you send me the output of:
> > $ curl -u <user>:<pass> http://localhost:8001/3.1/members?count=5&page=1
>
> This is the output you asked for (it's the same you can see when you try
> to interact with one list):
>
> /usr/local/mailman3 # curl -u XXXXXX:XXXXXX
> http://localhost:8001/3.1/members?count=5&page=1
> /usr/local/mailman3 # <html>
> <head>
> <title>Internal Server Error</title>
> </head>
> <body>
> <h1><p>Internal Server Error</p></h1>
>
> </body>
> </html>
>
> No log entry has been produced...
This is weird, if you have working Core, then there should be *some* json returned
from the above command. Do you have the Gunicorn running? What is the
output of `ps -ef | grep mailman`?
Are you able to run `mailman members` command to list the members of a list?
Also, how did you actually install Mailman?
Abhilash.
> TIA.
>
>
>
>
> > It should ideally return an output that looks something like shown
> > here[1]. You
> > can put the username/password of Core's API server in the above command.
> > [1]:
> > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/…
> >
> > Abhilash
> >> I'm using sqlite as django database and mysql for mailman.
> >>
> >> ERROR 2021-02-06 11:47:49,015 26798 django.request Internal
> >> Server Error:
> >> /mailman3/mailman3/lists/name_and_domain.of.the.list
> >> Traceback (most recent call last): File
> >> "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py",
> >> line 119, in __getattr__ return self._get(name) File
> >> "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py",
> >> line 86, in _get raise KeyError(key) KeyError:
> >> 'subscription_mode' During handling of the above exception,
> >> another exception occurred: Traceback (most recent call
> >> last): File
> >> "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py",
> >> line 34, in inner response = get_response(request) File
> >> "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py",
> >> line 115, in _get_response response =
> >> self.process_exception_by_middleware(e, request) File
> >> "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py",
> >> line 113, in _get_response response =
> >> wrapped_callback(request, *callback_args, **callback_kwargs)
> >> File
> >> "/usr/local/lib/python3.7/site-packages/django/views/generic/base.py",
> >> line 71, in view return self.dispatch(request, *args,
> >> **kwargs) File
> >> "/usr/local/lib/python3.7/site-packages/postorius/views/generic.py",
> >> line 74, in dispatch return super(MailingListView,
> >> self).dispatch(request, *args, **kwargs) File
> >> "/usr/local/lib/python3.7/site-packages/django/views/generic/base.py",
> >> line 97, in dispatch return handler(request, *args,
> >> **kwargs) File
> >> "/usr/local/lib/python3.7/site-packages/postorius/views/list.py",
> >> line 295, in get member.subscription_mode == File
> >> "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py",
> >> line 124, in __getattr__ self.__class__.__name__, name))
> >> AttributeError: 'Member' object has no attribute
> >> 'subscription_mode' *********************** some info about
> >> the installed versions via pip list: pip list | grep django
> >> django-allauth 0.44.0
> >> django-appconf 1.0.4
> >> django-compressor 2.4
> >> django-extensions 3.1.0
> >> django-gravatar2 1.4.4
> >> django-haystack 3.0
> >> django-mailman3 1.3.5
> >> django-picklefield 3.0.1
> >> django-q 1.3.4
> >> djangorestframework 3.12.2 pip list | grep mailman
> >> django-mailman3 1.3.5
> >> mailman 3.3.3
> >> mailman-hyperkitty 1.1.0
> >> mailmanclient 3.3.2 pip list | grep postorius
> >> postorius 1.3.4 On 6/2/21 11:12,
> >> Guillermo Hernandez (Oldno7) via Mailman-users wrote:
> >>
> >> I've just upgrade mailman 3 installation following Mr.
> >> Sapiro advice: did a pip install --upgrade
> >> django-mailman3 hyperkitty mailman mailmanclient
> >> mailman-hyperkitty postorius I did a "python3 manage.py
> >> migrate" after, too. And all seemed to run well. All the
> >> lists showed in postorius via web, but when I try to
> >> accesss into one of them the browser shows an error. In
> >> the log you can see: *-*-*-*-*-*-* Traceback (most recent
> >> call last): File
> >> "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py",
> >> line 119, in __getattr__ return self._get(name)
> >> File
> >> "/usr/local/lib/python3.7/site-packages/mailmanclient/restbase/base.py",
> >> line 86, in _get raise KeyError(key) KeyError:
> >> 'get_requests_count' .. (And after all the traceback
> >> lines) AttributeError: 'MailingList' object has no
> >> attribute 'get_requests_count' *-*-*-*-*-*-*-* The lists
> >> seem to be distributing messeages well.. but I cannot
> >> acces via web administration (django/postorius) Can
> >> anyone point me in the right direction to solve this,
> >> please? _______________________________________________
> >> Mailman-users mailing list -- mailman-users(a)mailman3.org
> >> <mailto:mailman-users@mailman3.org> To unsubscribe send
> >> an email to mailman-users-leave(a)mailman3.org
> >> <mailto:mailman-users-leave@mailman3.org>
> >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> >> <https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3org/>
> >>
> >>
> >> _______________________________________________ Mailman-users
> >> mailing list -- mailman-users(a)mailman3.org
> >> <mailto:mailman-users@mailman3.org> To unsubscribe send an
> >> email to mailman-users-leave(a)mailman3.org
> >> <mailto:mailman-users-leave@mailman3.org>
> >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> >> <https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3org/>
> >>
> >>
> >> _______________________________________________ Mailman-users mailing
> >> list -- mailman-users(a)mailman3.org
> >> <mailto:mailman-users@mailman3.org> To unsubscribe send an email to
> >> mailman-users-leave(a)mailman3.org
> >> <mailto:mailman-users-leave@mailman3.org>
> >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> >> <https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3org/>
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
--
thanks,
Abhilash Raj (maxking)
5 years, 1 month
[MM3-users] Re: Hyperkitty "All Threads" results in error
by Mark Dadgar
Aaaand I spoke too soon. I am now seeing the same error when I click the All Threads button in hyperkitty. Not every time, because that would be too easy, but often.
I really only have one high-volume list with archives, so I can’t tell if it’s list-specific.
I ran the mailman-post-update script after the last update, which I believe executes all the commands suggested below.
Here’s the crash trace.
Any thoughts?
- Mark
Internal Server Error: /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
Failed lookup for key [sender] in None
Request Method: GET
Request URL: https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest <https://mail.pdc-racing.net/archives/list/trackjunkies@pdc-racing.net/latest>
Django Version: 3.0.12
Python Executable: /opt/mailman/mm/venv/bin/python
Python Version: 3.8.5
Python Path: ['/opt/mailman/mm', '/opt/mailman/mm/', '/opt/mailman/mm/bin', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/opt/mailman/mm/venv/lib/python3.8/site-packages']
Server time: Mon, 1 Mar 2021 23:25:02 -0800
Installed Applications:
('hyperkitty',
'postorius',
'django_mailman3',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'django_gravatar',
'compressor',
'haystack',
'django_extensions',
'django_q',
'allauth',
'allauth.account',
'allauth.socialaccount')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'django_mailman3.middleware.TimezoneMiddleware',
'postorius.middleware.PostoriusMiddleware')
Traceback (most recent call last):
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 828, in _resolve_lookup
current = current[bit]
During handling of the above exception ('NoneType' object is not subscriptable), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 836, in _resolve_lookup
current = getattr(current, bit)
During handling of the above exception ('NoneType' object has no attribute 'sender'), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 842, in _resolve_lookup
current = current[int(bit)]
During handling of the above exception (invalid literal for int() with base 10: 'sender'), another exception occurred:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/lib/view_helpers.py", line 134, in inner
return func(request, *args, **kwargs)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py", line 115, in archives
return _thread_list(request, mlist, threads, extra_context=extra_context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/views/mlist.py", line 144, in _thread_list
return render(request, template_name, context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/shortcuts.py", line 19, in render
content = loader.render_to_string(template_name, context, request, using=using)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader.py", line 62, in render_to_string
return template.render(context, request)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 61, in render
return self.template.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 171, in render
return self._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 150, in render
return compiled_parent._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 62, in render
result = block.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py", line 209, in render
nodelist.append(node.render_annotated(context))
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/loader_tags.py", line 188, in render
return template.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 173, in render
return self._render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 163, in _render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/defaulttags.py", line 513, in render
return self.nodelist.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 936, in render
bit = node.render_annotated(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 903, in render_annotated
return self.render(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py", line 132, in render
context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/templatetags/i18n.py", line 132, in <dictcomp>
context.update({var: val.resolve(context) for var, val in self.extra_context.items()})
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 691, in resolve
arg_vals.append(arg.resolve(context))
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 795, in resolve
value = self._resolve_lookup(context)
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/base.py", line 847, in _resolve_lookup
raise VariableDoesNotExist("Failed lookup for key "
Exception Type: VariableDoesNotExist at /archives/list/trackjunkies(a)pdc-racing.net <mailto:archives/list/trackjunkies@pdc-racing.net>/latest
Exception Value: Failed lookup for key [sender] in None
Request information:
USER: mark
GET: No GET data
POST: No POST data
FILES: No FILES data
On Feb 17, 2021, at 5:34 PM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>
> On Oct 14, 2020, at 4:37 PM, Mark Sapiro <mark(a)msapiro.net> wrote:
>> On 10/14/20 11:12 AM, Mark Dadgar wrote:
>>>> On Oct 4, 2020, at 10:34 AM, Mark Dadgar <mark(a)pdc-racing.net> wrote:
>>>>
>>>> So I have been wrestling with this problem for a while now. Most times, but not every time, when someone clicks on the All Threads link in hyperkitty, a 500 error is generated.
>>
>>
>> Does this happen on all lists, only some lists or only one list?
>>
>>
>>>> I googled it, obviously, and at first I thought it was related to this issue, but that has not proven to be the case:
>>>>
>>>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
>>
>> What did you google? This query gets a lot of stuff with some hints, but
>> nothing definitive.
>>
>> django render raises VariableDoesNotExist("Failed lookup for key "
>>
>> One thing I noted at
>> <https://askbot.org/en/question/3095/django-error-variabledoesnotexist-faile…>
>> was
>>
>> models out of date - run python manage.py syncdb and python manage.py
>> migrate
>>
>> You could try that. Note that your django admin command might be
>> something other than `python manage.py`. Also note syncdb has been
>> removed from Django >=1.9. Currently, the above becomes `makemigrations`
>> followed by `migrate`.
>>
>> I also suggest running the django admin commands
>>
>> `collectstatic --clear --noinput`
>> and
>> `compress`
>
>
> My re-install seems to have fixed this issue as well. BONUS!
>
> - Mark
> -----
> mark(a)pdc-racing.net | 408-348-2878
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
- Mark
-----
mark(a)pdc-racing.net | 408-348-2878
5 years
[MM3-users] Re: psycopg2 error?
by Odhiambo Washington
On Fri, Oct 27, 2023 at 6:23 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 10/27/23 01:09, Odhiambo Washington wrote:
> >
> > So my database is PostgreSQL-15:
>
> Try the following in your venv.
> ```
> pip uninstall psycopg2-binary
> pip cache remove psycopg2-binary
> pip install psycopg2-binary
> ```
>
```
root@debian12:/home/wash# su - mailman
(venv) mailman@debian12:~$ pip uninstall psycopg2-binary
Found existing installation: psycopg2-binary 2.8.6
Uninstalling psycopg2-binary-2.8.6:
Would remove:
/opt/mailman/venv/lib/python3.11/site-packages/psycopg2/*
/opt/mailman/venv/lib/python3.11/site-packages/psycopg2_binary-2.8.6.dist-info/*
Proceed (Y/n)? y
Successfully uninstalled psycopg2-binary-2.8.6
(venv) mailman@debian12:~$ pip cache remove psycopg2-binary
WARNING: No matching packages for pattern "psycopg2-binary"
Files removed: 0
(venv) mailman@debian12:~$ pip install psycopg2-binary
Collecting psycopg2-binary
Using cached
psycopg2_binary-2.9.9-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
(3.0 MB)
Installing collected packages: psycopg2-binary
Successfully installed psycopg2-binary-2.9.9
(venv) mailman@debian12:~$ mailman info
Traceback (most recent call last):
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 1969, in _exec_single_context
self.dialect.do_execute(
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py",
line 922, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.InsufficientPrivilege: permission denied for schema public
LINE 2: CREATE TABLE ban (
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman", line 33, in <module>
sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts',
'mailman')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1077, in main
with self.make_context(prog_name, args, **extra) as ctx:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
943, in make_context
self.parse_args(ctx, args)
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1644, in parse_args
rest = super().parse_args(ctx, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1408, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
2400, in handle_parse_result
value = self.process_value(ctx, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
2362, in process_value
value = self.callback(ctx, self, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py",
line 95, in initialize_config
initialize(value)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py",
line 229, in initialize
initialize_2(propagate_logs=propagate_logs)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py",
line 187, in initialize_2
config.db = getUtility(IDatabaseFactory, utility_name).create()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/database/factory.py",
line 58, in create
SchemaManager(database).setup_database()
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/database/factory.py",
line 97, in setup_database
Model.metadata.create_all(self._database.engine)
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/schema.py",
line 5828, in create_all
bind._run_ddl_visitor(
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 3243, in _run_ddl_visitor
conn._run_ddl_visitor(visitorcallable, element, **kwargs)
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 2447, in _run_ddl_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/visitors.py",
line 671, in traverse_single
return meth(obj, **kw)
^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/ddl.py",
line 919, in visit_metadata
self.traverse_single(
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/visitors.py",
line 671, in traverse_single
return meth(obj, **kw)
^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/ddl.py",
line 957, in visit_table
)._invoke_with(self.connection)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/ddl.py",
line 315, in _invoke_with
return bind.execute(self)
^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 1416, in execute
return meth(
^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/sql/ddl.py",
line 181, in _execute_on_connection
return connection._execute_ddl(
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 1528, in _execute_ddl
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 1848, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 1988, in _exec_single_context
self._handle_dbapi_exception(
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 2343, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
line 1969, in _exec_single_context
self.dialect.do_execute(
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py",
line 922, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InsufficientPrivilege)
permission denied for schema public
LINE 2: CREATE TABLE ban (
^
[SQL:
CREATE TABLE ban (
id SERIAL NOT NULL,
email VARCHAR,
list_id VARCHAR,
PRIMARY KEY (id)
)
]
(Background on this error at: https://sqlalche.me/e/20/f405)
```
I then followed the directions for GRANTing access the public schema:
```
postgres=# \c mailman postgres
You are now connected to database "mailman" as user "postgres".
mailman=# GRANT ALL ON SCHEMA public TO mailman;
GRANT
mailman=# \c mailmanweb postgres
You are now connected to database "mailmanweb" as user "postgres".
mailmanweb=# GRANT ALL ON SCHEMA public TO mailman;
GRANT
mailmanweb=#
```
...and tested IF mailman user could create a table in the mailman db:
```
(venv) mailman@debian12:~$ psql -h localhost -U mailman -d mailman -W
Password:
psql (15.3 (Debian 15.3-0+deb12u1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384,
compression: off)
Type "help" for help.
mailman=> create table xyz(a varchar(100));
CREATE TABLE
mailman=> DROP TABLE xyz;
DROP TABLE
mailman=> \q
```
But then again `mailman info` groks!
```
(venv) mailman@debian12:~$ mailman info
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman", line 33, in <module>
sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts',
'mailman')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1077, in main
with self.make_context(prog_name, args, **extra) as ctx:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
943, in make_context
self.parse_args(ctx, args)
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1644, in parse_args
rest = super().parse_args(ctx, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1408, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
2400, in handle_parse_result
value = self.process_value(ctx, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
2362, in process_value
value = self.callback(ctx, self, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py",
line 95, in initialize_config
initialize(value)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py",
line 229, in initialize
initialize_2(propagate_logs=propagate_logs)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py",
line 198, in initialize_2
initialize_pipelines()
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/pipelines.py",
line 73, in initialize
add_components('handlers', IHandler, config.handlers)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/utilities/modules.py",
line 218, in add_components
for component_class in find_pluggable_components(subpackage, interface):
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/utilities/modules.py",
line 177, in find_pluggable_components
yield from find_components('mailman.' + subpackage, interface)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/utilities/modules.py",
line 150, in find_components
module = import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in
import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
File "<frozen importlib._bootstrap>", line 1149, in
_find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in
_call_with_frames_removed
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/handlers/arc_sign.py",
line 22, in <module>
from authheaders import sign_message
File
"/opt/mailman/venv/lib/python3.11/site-packages/authheaders/__init__.py",
line 25, in <module>
from authheaders.dmarc_lookup import dns_query, receiver_record,
receiver_record_walk, get_org_domain
File
"/opt/mailman/venv/lib/python3.11/site-packages/authheaders/dmarc_lookup.py",
line 22, in <module>
from pkg_resources import resource_filename # Part of setuptools
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 3248, in <module>
@_call_aside
^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 3223, in _call_aside
f(*args, **kwargs)
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 3261, in _initialize_master_working_set
working_set = WorkingSet._build_master()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 617, in _build_master
ws.require(__requires__)
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 956, in require
needed = self.resolve(parse_requirements(requirements))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 815, in resolve
dist = self._resolve_dist(
^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/pkg_resources/__init__.py",
line 856, in _resolve_dist
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'flufl.lock>=5.1' distribution was
not found and is required by mailman
```
Looking at what I have:
(venv) mailman@debian12:~$ pip freeze | grep flufl.lock
flufl.lock==8.0.2
(venv) mailman@debian12:~$
Now I am lost as to what the issue could be.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
2 years, 4 months
[MM3-users] Re: Newbie question 2: Rewriting issue for bounced emails
by William Oliver
On Sat, 2021-12-25 at 13:34 -0800, Mark Sapiro wrote:
> On 12/25/21 12:42 PM, William Oliver wrote:
> >
> > Is there a "make-the-freaking-virtual- map-database" command in
> > mailman3, or is this postfix thing? Should I just touch the thing
> > and
> > create an empty file? I'm a little scared to... The error message
> > makes me think that it should be populated with something...
>
>
> Did you add an Alias Domain (different from the Mail Host domain) to
> the
> entry for your domain? This is what triggers the creation of
> postfix_vmap.
>
Well, I hope youi had/have/will have a great holiday. I just finished
our dinner and I am now amusing myself by pretending to enjoy the
Hallmark movies my wife is watching and debugging my mailman3
installation when she's not looking. It was either "The twelve cats of
Christmas" or "Die Hard." I put my foot down and said we were watching
Die Hard. So of course we are watching a movie about 12 kittens in a
fire truck.
In any case, I added a line for "virtual_alias_domains" in my main.cf
for postfix. But it doesn't do anything, apparently, except generate
another error line (see the syslog output below). So, just for a lark,
I created an empty /opt/mailman/mm/var/data/postfix_vmap file and ran
postmap on it to create a postfix_vmap.db file.
Now, it no longer complains about there not being a postfix_vmap.db
file. Instead it complains about it being empty. Here's what my
mailmanweb.log has to say:
smtplib.SMTPRecipientsRefused: {'redacted(a)libertyfp.org': (550, b'5.1.1
<redacted(a)libertyfp.org>: Recipient address rejected: User unknown in
virtual alias table')}
I redacted the name because this is supposed to be the admin account --
and it fails on login.
So, here's syslog:
Dec 25 19:08:49 libertyfp postfix/smtpd[44664]: connect from
localhost[::1]
Dec 25 19:08:49 libertyfp postfix/trivial-rewrite[44670]: warning: do
not list domain libertyfp.org in BOTH virtual_alias_domains and
virtual_mailbox_domains
Dec 25 19:08:49 libertyfp postfix/trivial-rewrite[44670]: warning: do
not list domain libertyfp.org in BOTH virtual_alias_domains and
virtual_mailbox_domains
Dec 25 19:08:49 libertyfp postfix/smtpd[44664]: NOQUEUE: reject: RCPT
from localhost[::1]: 550 5.1.1 <redacted(a)libertyfp.org>: Recipient
address rejected: User unknown in virtual alias table;
from=<redacted2(a)libertyfp.org> to=<redacted(a)libertyfp.org> proto=ESMTP
helo=<mail.libertyfp.org>
Dec 25 19:08:49 libertyfp postfix/smtpd[44672]: connect from
localhost[::1]
Dec 25 19:08:49 libertyfp postfix/trivial-rewrite[44670]: warning: do
not list domain libertyfp.org in BOTH virtual_alias_domains and
virtual_mailbox_domains
Dec 25 19:08:49 libertyfp postfix/smtpd[44672]: NOQUEUE: reject: RCPT
from localhost[::1]: 550 5.1.1 <redacted(a)libertyfp.org>: Recipient
address rejected: User unknown in virtual alias table;
from=<root(a)localhost.local> to=<redacted(a)libertyfp.org> proto=ESMTP
helo=<mail.libertyfp.org>
Dec 25 19:08:49 libertyfp postfix/smtpd[44672]: disconnect from
localhost[::1] ehlo=1 auth=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=5/6
where "redacted2" is the name in settings.py for
EMAIL_HOST_USER and DEFAULT_FROM_EMAIL
and "redacted" is the address for "Mailman Suite Admin"
I don't know where "root(a)localhost.local" comes from.
OK, it didn't work when I just had virtual_mailbox_domains defined, and
it didn't work when I had both defined, so I commented out
virtual_mailbox_domains and just left virtual_alias_domains.
That didn't help, either, though it got rid of the warnings in syslog:
Dec 25 19:43:33 libertyfp postfix/smtpd[55192]: connect from
localhost[::1]
Dec 25 19:43:33 libertyfp postfix/smtpd[55192]: NOQUEUE: reject: RCPT
from localhost[::1]: 550 5.1.1 <redacted(a)libertyfp.org>: Recipient
address rejected: User unknown in virtual alias table;
from=<redacted2(a)libertyfp.org> to=<redacted(a)libertyfp.org> proto=ESMTP
helo=<mail.libertyfp.org>
Dec 25 19:43:34 libertyfp postfix/smtpd[55199]: connect from
localhost[::1]
Dec 25 19:43:34 libertyfp postfix/smtpd[55199]: NOQUEUE: reject: RCPT
from localhost[::1]: 550 5.1.1 <redacted(a)libertyfp.org>: Recipient
address rejected: User unknown in virtual alias table;
from=<root(a)localhost.local> to=<reacted(a)libertyfp.org> proto=ESMTP
helo=<mail.libertyfp.org>
Dec 25 19:43:34 libertyfp postfix/smtpd[55199]: disconnect from
localhost[::1] ehlo=1 auth=1 mail=1 rcpt=0/1 rset=1 quit=1 commands=5/6
Here's my postfix configuration:
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
always_add_missing_headers = yes
command_directory = /usr/sbin
compatibility_level = 2
daemon_directory = /usr/lib/postfix/sbin
data_directory = /var/lib/postfix
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd
$daemon_directory/$process_name $process_id & sleep 5
default_destination_concurrency_limit = 15
default_destination_recipient_limit = 30
header_checks = regexp:/etc/postfix/header_checks
home_mailbox = Maildir/
inet_interfaces = all
inet_protocols = all
local_recipient_maps = proxy:unix:passwd.byname $alias_maps
hash:/opt/mailman/mm/var/data/postfix_lmtp
mail_owner = postfix
mailq_path = /usr/bin/mailq
mydomain = libertyfp.org
myhostname = mail.libertyfp.org
mynetworks = 127.0.0.0/8 10.0.0.0/24
mynetworks_style = host
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases
owner_request_special = no
relay_domains = hash:/opt/mailman/mm/var/data/postfix_domains
sendmail_path = /usr/sbin/postfix
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_banner = $myhostname ESMTP
smtpd_recipient_restrictions = permit_mynetworks,
permit_auth_destination, permit_sasl_authenticated, reject
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain = $myhostname
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_tls_cert_file = /etc/letsencrypt/live/libertyfp.org/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/libertyfp.org/privkey.pem
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes
transport_maps = hash:/opt/mailman/mm/var/data/postfix_lmtp
unknown_local_recipient_reject_code = 550
virtual_alias_domains = libertyfp.org
virtual_alias_maps = hash:/opt/mailman/mm/var/data/postfix_vmap
virtual_gid_maps = static:20000
virtual_mailbox_base = /home/vmail
virtual_mailbox_maps = hash:/etc/postfix/virtual-mailbox
virtual_uid_maps = static:20000
Here's my settings.py:
# Mailman Web configuration file.
# /etc/mailman3/settings.py
from mailman_web.settings.base import *
from mailman_web.settings.mailman import *
#: Default list of admins who receive the emails from error logging.
ADMINS = (
('Mailman Suite Admin', 'redacted2(a)libertyfp.org'),
)
# Postgresql database setup.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'mailman',
'USER': 'mailman',
# TODO: Replace this with the password.
'PASSWORD': 'passwd',
'HOST': 'localhost',
'PORT': '5432',
}
}
# 'collectstatic' command will copy all the static files here.
# Alias this location from your webserver to `/static`
STATIC_ROOT = '/opt/mailman/web/static'
# Make sure that this directory is created or Django will fail on
start.
LOGGING['handlers']['file']['filename'] =
'/opt/mailman/web/logs/mailmanweb.log'
#: See
https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts
ALLOWED_HOSTS = [
"localhost", # Archiving API from Mailman, keep it.
"www.libertyfp.org",
"libertyfp.org",
"mail.libertyfp.org",
"list.libertyfp.org",
"127.0.0.1",
"2.56.57.28"
# "lists.your-domain.org",
# Add here all production domains you have.
]
#: Current Django Site being served. This is used to customize the web
host
#: being used to serve the current website. For more details about
Django
#: site, see: https://docs.djangoproject.com/en/dev/ref/contrib/sites/
SITE_ID = 1
# Set this to a new secret value.
SECRET_KEY = 'passwd'
# Set this to match the api_key setting in
# /opt/mailman/mm/mailman-hyperkitty.cfg (quoted here, not there).
MAILMAN_ARCHIVER_KEY = 'passwd'
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
'PATH': os.path.join(BASE_DIR, "fulltext_index"),
},
}
# To be added to Django's settings.py
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'localhost'
EMAIL_PORT = 25
EMAIL_HOST_USER = 'redacted(a)libertyfp.org'
EMAIL_HOST_PASSWORD = 'passwd'
DEFAULT_FROM_EMAIL = 'redacted2(a)libertyfp.org'
SITE_ID = 2
Here's my mailman.cfg:
/etc/mailman3/mailman.cfg
[paths.here]
var_dir: /opt/mailman/mm/var
[mailman]
layout: here
# This address is the "site owner" address. Certain messages which must
be
# delivered to a human, but which can't be delivered to a list owner
(e.g. a
# bounce from a list owner), will be sent to this address. It should
point to
# a human.
site_owner: redacted(a)libertyfp.org
[database]
class: mailman.database.postgresql.PostgreSQLDatabase
url: postgres://redacted2:passwd@localhost/mailman
[archiver.prototype]
enable: yes
# For the HyperKitty archiver.
[archiver.hyperkitty]
class: mailman_hyperkitty.Archiver
enable: yes
configuration: /opt/mailman/mm/mailman-hyperkitty.cfg
# And, create the /opt/mailman/mm/mailman-hyperkitty.cfg file
containing
# these settings uncommented
#[general]
#base_url: http://127.0.0.1:8000/archives/
#api_key: Secret_Hyperkitty_API_Key
[shell]
history_file: $var_dir/history.py
[shell]
history_file: $var_dir/history.py
[mta]
verp_confirmations: yes
verp_personalized_deliveries: yes
verp_delivery_interval: 1
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: mail.libertyfp.org
lmtp_port: 8024
smtp_host: mail.libertyfp.org
smtp_port: 25
configuration: python:mailman.config.postfix
recipient_delimiter = +
unknown_local_recipient_reject_code = 550
owner_request_special = no
Here's uwsgi.ini
# /etc/mailman3/uwsgi.ini
#
[uwsgi]
# Port on which uwsgi will be listening.
#http-socket = 127.0.0.1:8000
http-socket = 0.0.0.0:8000
# If running uwsgi from the virtual environment ...
virtualenv = /opt/mailman/venv/
module=mailman_web.wsgi:application
# Add to python import path.
pythonpath = /etc/mailman3/
# The default settings module.
env = DJANGO_SETTINGS_MODULE=settings
# Setup default number of processes and threads per process.
master = true
processes = 2
threads = 2
# Setup the django_q related worker processes.
attach-daemon = /opt/mailman/venv/bin/mailman-web qcluster
# Setup the request log.
req-logger = file:/opt/mailman/web/logs/uwsgi.log
# Log qcluster commands separately.
logger = qcluster file:/opt/mailman/web/logs/uwsgi-qcluster.log
log-route = qcluster uwsgi-daemons
# Last log and it logs the rest of the stuff.
logger = file:/opt/mailman/web/logs/uwsgi-error.log
And, last but not least here's mailman-hyperkitty.cfg:
[general]
base_url: http://127.0.0.1:8000/archives/
api_key: passwd
4 years, 2 months
[MM3-users] Re: Migration from old server to new server failed
by Helio Loureiro
Hi,
Any suggestions here?
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
On Mon, 19 Feb 2024 at 14:44, Helio Loureiro <helio(a)loureiro.eng.br> wrote:
> Hi,
>
> I repeated the steps today.
>
> First I waited to end the errors raised because the mailman3-web wasn't
> running.
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
> line 793, in urlopen
> response = self._make_request(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
> line 496, in _make_request
> conn.request(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
> line 400, in request
> self.endheaders()
> File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
> self._send_output(message_body, encode_chunked=encode_chunked)
> File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
> self.send(msg)
> File "/usr/lib/python3.10/http/client.py", line 976, in send
> self.connect()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
> line 238, in connect
> self.sock = self._new_conn()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
> line 213, in _new_conn
> raise NewConnectionError(
> urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection
> object at 0x7f0fdc2f72b0>: Failed to establish a new connection: [Errno
> 111] Connection refused
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/requests/adapters.py",
> line 486, in send
> resp = conn.urlopen(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
> line 847, in urlopen
> retries = retries.increment(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/util/retry.py",
> line 515, in increment
> raise MaxRetryError(_pool, url, reason) from reason # type:
> ignore[arg-type]
> urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1',
> port=8000): Max retries exceeded with url: /archives/api/mailman/archive
> (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at
> 0x7f0fdc2f72b0>: Failed to establish a new connection: [Errno 111]
> Connection refused'))
>
> Which seems to keep forever. So eventually I gave up and moved forward.
> And I changed to use the uwsgi socket instead of the direct port.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|SIGINT]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying account.0003_alter_emailaddress_create_unique_verified_email...
> OK
> Applying account.0004_alter_emailaddress_drop_unique_email... OK
> Applying account.0005_emailaddress_idx_upper_email... OK
> Applying admin.0003_logentry_add_action_flag_choices... OK
> Applying auth.0009_alter_user_last_name_max_length... OK
> Applying auth.0010_alter_group_name_max_length... OK
> Applying auth.0011_update_proxy_permissions... OK
> Applying auth.0012_alter_user_first_name_max_length... OK
> Applying django_mailman3.0003_sessions... OK
> Applying django_q.0010_auto_20200610_0856... OK
> Applying django_q.0011_auto_20200628_1055... OK
> Applying django_q.0012_auto_20200702_1608... OK
> Applying django_q.0013_task_attempt_count... OK
> Applying django_q.0014_schedule_cluster... OK
> Applying hyperkitty.0016_auto_20180309_0056... OK
> Applying hyperkitty.0017_file_attachments... OK
> Applying hyperkitty.0018_threadcategory_color... OK
> Applying hyperkitty.0019_auto_20190127_null_description... OK
> Applying hyperkitty.0020_auto_20190907_1927... OK
> Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
> last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table
> 'hyperkitty_mailinglist_moderators' already exists")
>
> Then dropping table hyperkitty_mailinglist_moderators.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1)> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
> last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table 'hyperkitty_mailinglist_owners'
> already exists")
>
> After dropping table hyperkitty_mailinglist_owners.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
> last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table
> 'hyperkitty_mailinglist_moderators' already exists")
>
> After dropping table hyperkitty_mailinglist_moderators.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying hyperkitty.0021_add_owners_mods... OK
> Applying hyperkitty.0022_mailinglist_archive_rendering_mode... OK
> Applying hyperkitty.0023_alter_mailinglist_name... OK
> Applying postorius.0004_create_email_template...Traceback (most recent
> call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table 'postorius_emailtemplate' already
> exists")
>
> After dropping table postorius_emailtemplate.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying postorius.0004_create_email_template... OK
> Applying postorius.0005_auto_20180707_1107... OK
> Applying postorius.0006_auto_20180711_1359... OK
> Applying postorius.0007_auto_20180712_0536... OK
> Applying postorius.0008_auto_20190310_0717... OK
> Applying postorius.0009_auto_20190508_1604... OK
> Applying postorius.0010_auto_20190821_0621... OK
> Applying postorius.0011_auto_20191109_1219... OK
> Applying postorius.0012_auto_20200420_2136... OK
> Applying postorius.0013_auto_20201116_0058... OK
> Applying postorius.0014_auto_20210329_2248... OK
> Applying postorius.0015_auto_20210619_0509... OK
> Applying postorius.0016_auto_20210810_2157... OK
> Applying postorius.0017_alter_emailtemplate_language... OK
> Applying postorius.0018_alter_emailtemplate_language... OK
> Applying socialaccount.0004_app_provider_id_settings... OK
> Applying socialaccount.0005_socialtoken_nullable_app... OK
> Applying socialaccount.0006_alter_socialaccount_extra_data... OK
>
> At this point the errors remains about not reaching web part. So I
> started mailman3-web.
>
> Then I see these errors on var/log/mailman.log
>
> Feb 19 15:35:14 2024 (543890) Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_hyperkitty/__init__.py",
> line 158, in _archive_message
> url = self._send_message(mlist, msg)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_hyperkitty/__init__.py",
> line 228, in _send_message
> raise ValueError(result.text)
> ValueError: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL was not found on this server.</p>
> <hr>
> <address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
> </body></html>
>
> Feb 19 15:35:14 2024 (543890) HyperKitty failure on
> http://127.0.0.1/archives/api/mailman/archive: <!DOCTYPE HTML PUBLIC
> "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL was not found on this server.</p>
> <hr>
> <address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
> </body></html>
> (404)
> Feb 19 15:35:14 2024 (543890) Could not archive the message with id <
> ff1707bb-d105-4f37-b3f8-9766b3563127(a)SESAMR604.domain.com>
> Feb 19 15:35:14 2024 (543890) archiving failed, re-queuing (mailing-list
> mylist.new-mailman.domain.com, message <
> ff1707bb-d105-4f37-b3f8-9766b3563127(a)machine.domain.com>)
> Feb 19 15:35:14 2024 (543890) Exception in the HyperKitty archiver:
> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL was not found on this server.</p>
> <hr>
> <address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
> </body></html>
>
> And I can't get access to the hyperkitty at all.
>
> On var/logs/uwsgi-error.log:
>
> --- Logging error ---
> Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 356, in pusher
> task = SignedPackage.loads(task[1])
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/signing.py",
> line 25, in loads
> return signing.loads(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 35, in loads
> base64d = force_bytes(TimestampSigner(key, salt=salt).unsign(s,
> max_age=max_age))
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 70, in unsign
> result = super(TimestampSigner, self).unsign(value)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 55, in unsign
> raise BadSignature('Signature "%s" does not match' % sig)
> django.core.signing.BadSignature: Signature "48do9gGvueBxakX_a4uNmSwD7cs"
> does not match
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/lib/python3.10/logging/__init__.py", line 1100, in emit
> msg = self.format(record)
> File "/usr/lib/python3.10/logging/__init__.py", line 943, in format
> return fmt.format(record)
> File "/usr/lib/python3.10/logging/__init__.py", line 678, in format
> record.message = record.getMessage()
> File "/usr/lib/python3.10/logging/__init__.py", line 368, in getMessage
> msg = msg % self.args
> TypeError: not all arguments converted during string formatting
> Call stack:
> File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
> sys.exit(main())
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
> line 90, in main
> execute_from_command_line(sys.argv)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
> line 446, in execute_from_command_line
> utility.execute()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
> line 440, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 402, in run_from_argv
> self.execute(*args, **cmd_options)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 448, in execute
> output = self.handle(*args, **options)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/management/commands/qcluster.py",
> line 22, in handle
> q.start()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 78, in start
> self.sentinel.start()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
> self._popen = self._Popen(self)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in
> _Popen
> return _default_context.get_context().Process._Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 281, in
> _Popen
> return Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in
> __init__
> self._launch(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in
> _launch
> code = process_obj._bootstrap(parent_sentinel=child_r)
> File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in
> _bootstrap
> self.run()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
> self._target(*self._args, **self._kwargs)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 168, in __init__
> self.start()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 172, in start
> self.spawn_cluster()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 248, in spawn_cluster
> self.pusher = self.spawn_pusher()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 201, in spawn_pusher
> return self.spawn_process(pusher, self.task_queue, self.event_out,
> self.broker)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 197, in spawn_process
> p.start()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
> self._popen = self._Popen(self)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in
> _Popen
> return _default_context.get_context().Process._Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 281, in
> _Popen
> return Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in
> __init__
> self._launch(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in
> _launch
> code = process_obj._bootstrap(parent_sentinel=child_r)
> File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in
> _bootstrap
> self.run()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
> self._target(*self._args, **self._kwargs)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 358, in pusher
> logger.error(e, traceback.format_exc())
> Message: BadSignature('Signature "48do9gGvueBxakX_a4uNmSwD7cs" does not
> match')
> Arguments: ('Traceback (most recent call last):\n File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 356, in pusher\n task = SignedPackage.loads(task[1])\n File
> "/local/mailman/venv/lib/python3.10/sit
> e-packages/django_q/signing.py", line 25, in loads\n return
> signing.loads(\n File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 35, in loads\n base64d = force_bytes(TimestampSigner(key, salt=sa
> lt).unsign(s, max_age=max_age))\n File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 70, in unsign\n result = super(TimestampSigner,
> self).unsign(value)\n File "/local/mailman/venv/lib/python3.10
> /site-packages/django_q/core_signing.py", line 55, in unsign\n raise
> BadSignature(\'Signature "%s" does not match\' %
> sig)\ndjango.core.signing.BadSignature: Signature
> "48do9gGvueBxakX_a4uNmSwD7cs" does not match\n',)
>
> I'm using uwsgi via apache:
>
> (venv) mailman@new-mailman3 ~ (v3.1.1)> more
> /etc/apache2/conf-enabled/mailman3.conf
> Alias /mailman3/favicon.ico /local/mailman/web/static/favicon.ico
> Alias /mailman/favicon.ico /local/mailman/web/static/favicon.ico
> Alias /favicon.ico /local/mailman/web/static/favicon.ico
> Alias /mailman3/static /local/mailman/web/static
> Alias /mailman/static /local/mailman/web/static
> Alias /static /local/mailman/web/static
>
> <Directory "/local/mailman/web/static">
> Require all granted
> </Directory>
>
> <IfModule mod_proxy_uwsgi.c>
> ProxyPass /mailman3/favicon.ico !
> ProxyPass /mailman/favicon.ico !
> ProxyPass /favicon.ico !
> ProxyPass /mailman3/static !
> ProxyPass /mailman/static !
> ProxyPass /static !
> ProxyPass /mailman3
> unix:/local/mailman/var/uwsgi.sock|uwsgi://localhost/
> ProxyPass /mailman
> unix:/local/mailman/var/uwsgi.sock|uwsgi://localhost/
> #ProxyPass /mailman3 http://localhost:8000/ timeout=180
> #ProxyPass /mailman http://localhost:8000/ timeout=180
> #ProxyPass / http://localhost:8000/ timeout=180
> </IfModule>
>
> Best Regards,
> Helio Loureiro
> https://helio.loureiro.eng.br
> https://github.com/helioloureiro
> https://mastodon.social/@helioloureiro
>
>
> On Fri, 16 Feb 2024 at 22:39, Mark Sapiro <mark(a)msapiro.net> wrote:
>
>> On 2/16/24 07:12, Helio Loureiro wrote:
>>
>> > - start mailman3 and mailman-3web
>> > - run mailman3-web migrate
>>
>> You should run mailman3-web migrate when mailman-3web is not running.
>>
>> > django.db.utils.OperationalError: (1050, "Table
>> > 'hyperkitty_mailinglist_moderators' already exists")
>> >
>> > I tried to drop the table, but then it complained about another one. I
>> > dropped all the complained tables and as result hypperkitty never
>> > started. So I went back to see why the first error happened.
>>
>> The errors occurred because you had recently added tables in your
>> database so they couldn't be added by the migrations. Dropping the
>> tables was the correct thing to do.
>>
>> Did you ultimately successfully run all the migrations?
>>
>> When you say `hypperkitty never started`, what exactly does that mean?
>> What happens when you try to access the archives? Can you access
>> postorius? Are any errors logged in mailman3-web's log?
>>
>> --
>> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>> San Francisco Bay Area, California better use your sense - B. Dylan
>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent to helio(a)loureiro.eng.br
>>
>
2 years
[MM3-users] RFC-2369 List-Unsubscribe header missing
by paul.sparks＠us.idemia.com
Thanks to Mark Sapiro and others who have answered questions on this list. It has made my job easier. I'm hoping someone can help with another issue I'm seeing.
I'm setting up a new mailman 3 server and doing some test posts to a list. This server is intended only as an announcement list with limited members allowed to post. The resulting emails coming from the list do not include the List-Unsubscribe header although other "List-XXX" headers are included. I'm trying to figure out why the header is missing. It seems clear from the code that the header should be included.
Here is partial text of the email source as received from the list. I've tried viewing in Outlook.com, Thunderbird, and even captured the email text at postfix prior to sending to look for the missing header. My domain has been replaced with "example.com" throughout. This is from the postfix capture
________
...
To: "test1(a)example.com"
<test1(a)example.com>
Thread-Topic: Post 8 test
Thread-Index: AQHb8oBBzPBOYgrDxUOIiAxhvUzthA==
Date: Fri, 11 Jul 2025 16:24:38 +0000
Accept-Language: en-US
Content-Language: en-US
msip_labels:
authentication-results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=example.com;
MIME-Version: 1.0
Message-ID-Hash: FWHZCRE4EA6IFP4BZTPIF23DKAKVRMDT
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop;
banned-address; emergency; member-moderation; nonmember-moderation;
administrivia; implicit-dest; max-recipients; max-size; news-moderation;
no-subject; digests; suspicious-header
X-Content-Filtered-By: Mailman/MimeDel 3.3.10
From: "Somewhere1, SW" <test1(a)example.com>
X-Mailman-Version: 3.3.10
Precedence: list
Reply-To: no-reply(a)example.com
Subject: [Test1] Post 8 test
List-Id: "Somewhere1, SW" <test1.example.com>
List-Help:
<mailto:test1-request@example.com?subject=help>
List-Owner: <mailto:test1-owner@example.com>
List-Post: NO
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Message-Id: <20250711163343.017C31DBB79(a)example.com>
Body text
...
__________
Details on my test setup:
# Mailman version.
__version__ = '3.3.10'
OS: Amazon Linux 2023 (RHEL 9 compatible)
MTA: postfix-3.7.2
I am using the following code with mailman shell to configure the list. include_rfc2369_headers is left at the default value of True (verified via mailman shell)
____________
# Description value must be passed as a parameter
def configure_list(ml, description):
#ml.accept_these_nonmembers = allowed_to_post
ml.admin_immed_notify = False
ml.advertised = False
ml.allow_list_posts = False
ml.anonymous_list = True
ml.archive_policy = ArchivePolicy.never
ml.bounce_info_stale_after = datetime.timedelta(days=90)
ml.bounce_notify_owner_on_disable = False
ml.bounce_notify_owner_on_removal = False
ml.bounce_score_threshold = 3
ml.bounce_you_are_disabled_warnings = 0
ml.default_member_action = Action.discard
ml.default_nonmember_action = Action.discard
ml.description = description
ml.digests_enabled = False
ml.filter_content = True
ml.first_strip_reply_to = True
ml.forward_auto_discards = False
ml.max_days_to_hold = 1
ml.max_message_size = 0
ml.nntp_prefix_subject_too = False
ml.pass_extensions = []
ml.pass_types = ['text/plain','multipart/alternative']
ml.reply_goes_to_list = ReplyToMunging.explicit_header
ml.reply_to_address = "no-reply(a)example.com"
ml.require_explicit_destination = False
ml.respond_to_post_requests = False
# Allow select members to post to list
for mbr in ml.members.members:
if str(mbr.address) in allowed_to_post:
mbr.moderation_action = Action.defer
print('end',mbr.address,mbr.moderation_action)
_______________
/etc/mailman3/mailman.cfg file:
# /etc/mailman3/mailman.cfg
# These settings override the defaults in the following 2 config files
# /opt/mailman/lib/python3.9/site-packages/mailman/config/schema.cfg
# /opt/mailman/lib/python3.9/site-packages/mailman/config/mailman.cfg
[paths.custom]
var_dir: /data/lib/mailman
pid_file: /data/lib/mailman/pid/master.pid
lock_file: /data/lib/mailman/pid/master.lock
[mailman]
layout: custom
# This address is the "site owner" address. Certain messages which must be
# delivered to a human, but which can't be delivered to a list owner (e.g. a
# bounce from a list owner), will be sent to this address. It should point to
# a human.
site_owner: ml-owner(a)example.com
[database]
class: mailman.database.postgresql.PostgreSQLDatabase
url: postgresql:///mailman
# No archiving enabled
[archiver.prototype]
enable: no
[archiver.hyperkitty]
enable: no
[shell]
history_file: $var_dir/history.py
[mta]
verp_confirmations: yes
verp_personalized_deliveries: yes
verp_delivery_interval: 1
[passwords]
# When Mailman generates them, this is the default length of passwords.
password_length: 15
[logging.debug]
path: debug.log
level: debug
______________
With debug enabled I see the rfc-2369 pipeline being run when processing the post to the list.
Jul 11 17:51:32 2025 (444088) [IncomingRunner] starting oneloop
Jul 11 17:51:32 2025 (444088) [IncomingRunner] processing filebase: 1752256291.9136803+bfd305a28151ce750dde5114b413b351fa4d1284
Jul 11 17:51:32 2025 (444088) [IncomingRunner] processing onefile
Jul 11 17:51:32 2025 (444090) [OutgoingRunner] starting oneloop
Jul 11 17:51:32 2025 (444090) [OutgoingRunner] ending oneloop: 0
Jul 11 17:51:32 2025 (444097) [DigestRunner] starting oneloop
Jul 11 17:51:32 2025 (444097) [DigestRunner] ending oneloop: 0
Jul 11 17:51:32 2025 (444088) [IncomingRunner] finishing filebase: 1752256291.9136803+bfd305a28151ce750dde5114b413b351fa4d1284
Jul 11 17:51:32 2025 (444088) [IncomingRunner] doing periodic
Jul 11 17:51:32 2025 (444088) [IncomingRunner] committing transaction
Jul 11 17:51:32 2025 (444088) [IncomingRunner] checking short circuit
Jul 11 17:51:32 2025 (444088) [IncomingRunner] ending oneloop: 1
Jul 11 17:51:32 2025 (444088) [IncomingRunner] starting oneloop
Jul 11 17:51:32 2025 (444088) [IncomingRunner] ending oneloop: 0
Jul 11 17:51:32 2025 (444087) [CommandRunner] starting oneloop
Jul 11 17:51:32 2025 (444087) [CommandRunner] ending oneloop: 0
Jul 11 17:51:32 2025 (444091) [PipelineRunner] starting oneloop
Jul 11 17:51:32 2025 (444091) [PipelineRunner] processing filebase: 1752256292.373131+a372784a3ff7a76a662ed1c62c224a53a8008b8b
Jul 11 17:51:32 2025 (444091) [PipelineRunner] processing onefile
Jul 11 17:51:32 2025 (444096) [VirginRunner] starting oneloop
Jul 11 17:51:32 2025 (444096) [VirginRunner] ending oneloop: 0
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: validate-authenticity
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: mime-delete
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: tagger
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: member-recipients
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: avoid-duplicates
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: cleanse
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: cleanse-dkim
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: cook-headers
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: subject-prefix
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: rfc-2369
Jul 11 17:51:32 2025 (444091) RFC 2369 process
Jul 11 17:51:32 2025 (444091) RFC 2369 process continue 1
Jul 11 17:51:32 2025 (444091) RFC 2369 process headers extend
Jul 11 17:51:32 2025 (444091) RFC 2369 process header for loop
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: to-archive
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: to-digest
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: to-usenet
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: after-delivery
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: acknowledge
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: dmarc
Jul 11 17:51:32 2025 (444091) <20250711175131.B8CEC1DA51B(a)example.com> pipeline default-posting-pipeline processing: to-outgoing
Jul 11 17:51:32 2025 (444091) [PipelineRunner] finishing filebase: 1752256292.373131+a372784a3ff7a76a662ed1c62c224a53a8008b8b
Jul 11 17:51:32 2025 (444091) [PipelineRunner] doing periodic
Jul 11 17:51:32 2025 (444091) [PipelineRunner] committing transaction
Jul 11 17:51:32 2025 (444091) [PipelineRunner] checking short circuit
Jul 11 17:51:32 2025 (444091) [PipelineRunner] ending oneloop: 1
Jul 11 17:51:32 2025 (444091) [PipelineRunner] starting oneloop
Jul 11 17:51:32 2025 (444091) [PipelineRunner] ending oneloop: 0
Jul 11 17:51:32 2025 (444085) [ArchiveRunner] starting oneloop
Jul 11 17:51:32 2025 (444085) [ArchiveRunner] ending oneloop: 0
Jul 11 17:51:33 2025 (444090) [OutgoingRunner] starting oneloop
Jul 11 17:51:33 2025 (444090) [OutgoingRunner] processing filebase: 1752256292.8885148+5e5c4a68eacfead13561df616fe7ca19e1d1d182
Jul 11 17:51:33 2025 (444090) [OutgoingRunner] processing onefile
Jul 11 17:51:33 2025 (444097) [DigestRunner] starting oneloop
Jul 11 17:51:33 2025 (444097) [DigestRunner] ending oneloop: 0
Jul 11 17:51:33 2025 (444090) [outgoing] <function deliver at 0x7f21ed7b1310>: <175225629286.444091.14942604217243398224(a)ip-10-0-0-168.us-gov-west-1.compute.internal>
Jul 11 17:51:33 2025 (444088) [IncomingRunner] starting oneloop
_____________
I tried inserting some log.debug calls into lib/python3.9/site-packages/mailman/handlers/rfc_2369.py to see where headers were being removed and added.
I modified: log = logging.getLogger('mailman.debug')
and inserted log.debug throughout the process function and one in the top level.
___________
...
@public
@implementer(IHandler)
class RFC2369:
"""Add the RFC 2369 List-* headers."""
name = 'rfc-2369'
description = _('Add the RFC 2369 List-* headers.')
log.debug('RFC 2369 processing')
def process(self, mlist, msg, msgdata):
"""See `IHandler`."""
process(mlist, msg, msgdata)
__________
When running with debug enabled, I see this top level message many times at the top of the debug.log file during mailman startup, but not after that. I don't see any of my other debug messages.
_____________
Jul 11 16:03:36 2025 (409226) RFC 2369 processing
Jul 11 16:03:40 2025 (409244) RFC 2369 processing
Jul 11 16:03:41 2025 (409250) RFC 2369 processing
Jul 11 16:03:49 2025 (409271) RFC 2369 processing
Jul 11 16:03:50 2025 (409271) [DigestRunner] starting oneloop
Jul 11 16:03:50 2025 (409271) [DigestRunner] ending oneloop: 0
Jul 11 16:03:50 2025 (409261) RFC 2369 processing
Jul 11 16:03:50 2025 (409261) [BounceRunner] starting oneloop
Jul 11 16:03:50 2025 (409261) [BounceRunner] ending oneloop: 0
Jul 11 16:03:50 2025 (409264) RFC 2369 processing
Jul 11 16:03:50 2025 (409267) RFC 2369 processing
Jul 11 16:03:51 2025 (409262) RFC 2369 processing
Jul 11 16:03:51 2025 (409262) [CommandRunner] starting oneloop
Jul 11 16:03:51 2025 (409262) [CommandRunner] ending oneloop: 0
Jul 11 16:03:51 2025 (409266) RFC 2369 processing
Jul 11 16:03:51 2025 (409266) [PipelineRunner] starting oneloop
Jul 11 16:03:51 2025 (409266) [PipelineRunner] ending oneloop: 0
...
__________
Any suggestions on where else I can look? Any thoughts on why my debug calls are not showing in the logs?
8 months
[MM3-users] Re: Hyperkitty CPU usage
by Alain Kohli
I'm running a custom image which is based on an older version of the one
here: https://github.com/maxking/docker-mailman. I attached it below.
But I separated postorius and hyperkitty, so hyperkitty is running in
its own container. I'm deploying the image with a plain 'docker run'
behind nginx. I made fulltext_index persistent now, but it didn't get
populated with anything yet. I don't really have an error traceback
because there is never an error thrown. The only thing with some content
is uwsgi-error.log, which you can find below. I'm also still getting the
"A string literal cannot contain NUL (0x00) characters." messages. I
also noticed that it takes incredibly long for the webinterface to load
(several minutes) even though there doesn't seem to be any process
consuming notable resources apart from the minutely job.
Funnily enough, I have the exact same image deployed on a second server
as well for testing. On that one everything works fine. The only
difference is that on the problematic one I have a lot more mailing
lists/archives and that I imported them from mailman2. Could something
have gone wrong during the import? I used the regular hyperkitty_import
command.
uwsgi-error.log:
*** Starting uWSGI 2.0.18 (64bit) on [Sat Apr 27 22:50:17 2019] ***
compiled with version: 6.4.0 on 27 April 2019 22:48:42
os: Linux-4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19)
nodename: hyperkitty.docker
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /home/hyperkitty
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
setgid() to 82
setuid() to 82
chdir() to /home/hyperkitty
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8081 fd 8
uwsgi socket 1 bound to TCP address 0.0.0.0:8080 fd 9
Python version: 3.6.8 (default, Jan 30 2019, 23:54:38) [GCC 6.4.0]
Python main interpreter initialized at 0x55dfaa41c980
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
[uwsgi-cron] command "./manage.py runjobs minutely" registered as
cron task
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" registered
as cron task
[uwsgi-cron] command "./manage.py runjobs hourly" registered as cron
task
[uwsgi-cron] command "./manage.py runjobs daily" registered as cron task
[uwsgi-cron] command "./manage.py runjobs monthly" registered as
cron task
[uwsgi-cron] command "./manage.py runjobs weekly" registered as cron
task
[uwsgi-cron] command "./manage.py runjobs yearly" registered as cron
task
mapped 208576 bytes (203 KB) for 4 cores
*** Operational MODE: threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter
0x55dfaa41c980 pid: 1 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 40, cores: 4)
Sat Apr 27 22:50:18 2019 - [uwsgi-cron] running "./manage.py runjobs
minutely" (pid 45)
[uwsgi-daemons] spawning "./manage.py qcluster" (uid: 82 gid: 82)
22:50:21 [Q] INFO Q Cluster-47 starting.
22:50:21 [Q] INFO Process-1:1 ready for work at 59
22:50:21 [Q] INFO Process-1:2 ready for work at 60
22:50:21 [Q] INFO Process-1:3 ready for work at 61
22:50:21 [Q] INFO Process-1:4 ready for work at 62
22:50:21 [Q] INFO Process-1:5 monitoring at 63
22:50:21 [Q] INFO Process-1 guarding cluster at 58
22:50:21 [Q] INFO Process-1:6 pushing tasks at 64
22:50:21 [Q] INFO Q Cluster-47 running.
22:59:31 [Q] INFO Enqueued 3403
22:59:31 [Q] INFO Process-1:1 processing [update_from_mailman]
22:59:33 [Q] INFO Processed [update_from_mailman]
Sat Apr 27 23:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 73)
Sat Apr 27 23:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
hourly" (pid 74)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 73 exited after 64 second(s)
23:01:28 [Q] INFO Enqueued 3404
23:01:29 [Q] INFO Process-1:2 processing
[rebuild_mailinglist_cache_recent]
[uwsgi-cron] command "./manage.py runjobs hourly" running with pid
74 exited after 91 second(s)
Sat Apr 27 23:01:36 2019 - uwsgi_response_write_body_do(): Broken
pipe [core/writer.c line 341] during GET / (212.203.58.154)
OSError: write error
23:01:36 [Q] INFO Processed [rebuild_mailinglist_cache_recent]
Sat Apr 27 23:15:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 88)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 88 exited after 4 second(s)
23:28:24 [Q] INFO Enqueued 3405
23:28:24 [Q] INFO Process-1:3 processing [update_from_mailman]
23:28:25 [Q] INFO Processed [update_from_mailman]
Sat Apr 27 23:30:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 96)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 96 exited after 4 second(s)
23:44:40 [Q] INFO Enqueued 3406
23:44:40 [Q] INFO Process-1:4 processing [update_from_mailman]
23:44:41 [Q] INFO Processed [update_from_mailman]
Sat Apr 27 23:45:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 104)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 104 exited after 4 second(s)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
quarter_hourly" (pid 113)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
hourly" (pid 114)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
daily" (pid 115)
Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
weekly" (pid 116)
[uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
with pid 113 exited after 55 second(s)
[uwsgi-cron] command "./manage.py runjobs weekly" running with pid
116 exited after 55 second(s)
00:01:36 [Q] INFO Enqueued 3407
00:01:36 [Q] INFO Process-1:1 processing
[rebuild_mailinglist_cache_recent]
[uwsgi-cron] command "./manage.py runjobs hourly" running with pid
114 exited after 99 second(s)
00:01:50 [Q] INFO Processed [rebuild_mailinglist_cache_recent]
00:04:52 [Q] INFO Enqueued 3408
00:04:52 [Q] INFO Process-1:2 processing [update_from_mailman]
00:04:54 [Q] INFO Processed [update_from_mailman]
Dockerfile:
FROM python:3.6-alpine3.7 # Add startup script to container COPY
assets/docker-entrypoint.sh /usr/local/bin/ # Install packages and
dependencies for hyperkitty and add user for executing apps. # It's
important that the user has the UID/GID 82 so nginx can access the
files. RUN set -ex \&& apk add --no-cache --virtual .build-deps gcc
libc-dev linux-headers git \postgresql-dev \&& apk add --no-cache
--virtual .mailman-rundeps bash sassc mailcap \postgresql-client
curl \&& pip install -U django==2.2 \&& pip install
git+https://gitlab.com/eestec/mailmanclient
\git+https://gitlab.com/mailman/hyperkitty@c9fa4d4bfc295438d3e01cd93090064d004cf44d
\git+https://gitlab.com/eestec/django-mailman3 \whoosh \uwsgi
\psycopg2 \dj-database-url \typing \&& apk del .build-deps \&&
addgroup -S -g 82 hyperkitty \&& adduser -S -u 82 -G hyperkitty
hyperkitty \&& chmod u+x /usr/local/bin/docker-entrypoint.sh# Add
needed files for uwsgi server + settings for django COPY
assets/__init__.py /home/hyperkittyCOPY assets/manage.py
/home/hyperkittyCOPY assets/urls.py /home/hyperkittyCOPY
assets/wsgi.py /home/hyperkittyCOPY assets/uwsgi.ini
/home/hyperkittyCOPY assets/settings.py /home/hyperkitty# Change
ownership for uwsgi+django files and set execution rights for
management script RUN chown -R hyperkitty /home/hyperkitty && chmod
u+x /home/hyperkitty/manage.py# Make sure we are in the correct
working dir WORKDIR /home/hyperkittyEXPOSE 8080 8081# Use stop
signal for uwsgi server STOPSIGNAL SIGINTENTRYPOINT
["docker-entrypoint.sh"]CMD ["uwsgi", "--ini",
"/home/hyperkitty/uwsgi.ini"]
On 4/27/19 7:58 PM, Abhilash Raj wrote:
> On Sat, Apr 27, 2019, at 9:40 AM, Alain Kohli wrote:
>> I have run "python manage.py rebuild_index" before, doesn't that do
>> clear_index as well? Apart from that, I run hyperkitty in a docker
>> container and didn't know fulltext_index should be persistent, so that
>> got deleted after every version update for sure.
> Which images are you using and how are you deploying them?
>
> You should persist fulltext_index, yes, and possibly logs if you need
> them for debugging later.
>
> Can you paste the entire error traceback?
>
>>
>> On 4/26/19 10:18 PM, Mark Sapiro wrote:
>>> On 4/26/19 11:14 AM, Alain Kohli wrote:
>>>> I see loads of "A string literal cannot contain NUL (0x00) characters."
>>>> messages, but I haven't found missing messages in the archives yet. Not
>>>> sure how that could be related, though. Apart from that I don't see
>>>> anything unusual. The other jobs (quarter_hourly, hourly, etc.) seem to
>>>> run and finish normally.
>>> Did you upgrade from a Python 2.7 version of HyperKitty to a Python 3
>>> version? The Haystack/Whoosh search engine databases are not compatible
>>> between the two and "A string literal cannot contain NUL (0x00)
>>> characters." is the symptom.
>>>
>>> You need to run 'python manage.py clear_index' or just remove all the
>>> files from the directory defined as 'PATH' under HAYSTACK_CONNECTIONS in
>>> your settings file (normally 'fulltext_index' in the same directory that
>>> contains your settings.py.
>>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
6 years, 10 months
[MM3-users] Re: Held messages not delivered after approval
by Krinetzki, Stephan
Hi all,
after the Weekend, this is the status of the queues:
/opt/mailman/var/queue/archive:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 08:47 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/bad:
total 3016
drwxrwx--- 2 mailman mailman 4096 Aug 2 00:02 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
-rw-rw---- 1 mailman mailman 221723 Aug 1 17:26 1754061973.4885209+46b1ae3716439bf3ef98090296dfce0320fc3017.psv
-rw-rw---- 1 mailman mailman 32912 Aug 2 00:00 1754085602.191851+3576cf33232db110fa7761233f67245564553652.psv
-rw-rw---- 1 mailman mailman 416 Aug 2 00:00 1754085604.0204346+ad485da0c45cb0ad17a5dc42613c3eb3f313c20e.psv
-rw-rw---- 1 mailman mailman 1407649 Aug 2 00:00 1754085623.275817+f23139c8127c454b4fe65453af3db18e558b0e87.psv
-rw-rw---- 1 mailman mailman 1407634 Aug 2 00:02 1754085729.3529432+1643f907bac39a22a7d71e50b031c4f8a574082c.psv
/opt/mailman/var/queue/bounces:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 05:22 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/command:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 08:14 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/digest:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 08:21 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/in:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 08:47 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/nntp:
total 0
drwxrwx--- 2 mailman mailman 6 Jun 27 2024 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/out:
total 1772
drwxrwx--- 2 mailman mailman 4096 Aug 4 08:49 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
-rw-rw---- 1 mailman mailman 1407649 Aug 2 00:00 1754085626.995262+ebf03275f7441b1bc7bbaf063cb6238bec30ff9f.pck.tmp
-rw-rw---- 1 mailman mailman 50244 Aug 3 00:00 1754172045.4400518+1da9c6fd82ee0dbc0e893eeca713d498a7273150.pck.tmp
-rw-rw---- 1 mailman mailman 31122 Aug 4 08:49 1754290173.7476344+7237c062741059807b66024091571ce3399d8eda.pck
-rw-rw---- 1 mailman mailman 18091 Aug 4 08:49 1754290173.8104873+cb4c78dffcd7147a5deca51b2cfd92829c06af9d.pck
-rw-rw---- 1 mailman mailman 18243 Aug 4 08:49 1754290173.8333225+d77876f47de73141dd5f6f9ac8d33d937bf6a727.pck
-rw-rw---- 1 mailman mailman 17585 Aug 4 08:49 1754290173.8802657+cb8103f6db5193a75706a7c1bc0d4f90126e587a.pck
-rw-rw---- 1 mailman mailman 217073 Aug 4 08:49 1754290173.9709926+90b2bfd3adf5faa7d0ec336f07dbe2267e666c75.pck
-rw-rw---- 1 mailman mailman 33494 Aug 4 08:49 1754290174.019171+553fc3546ae6c5f8e1b0f27ffc60e154629b9f1d.pck
/opt/mailman/var/queue/pipeline:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 08:47 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/retry:
total 0
drwxrwx--- 2 mailman mailman 6 Feb 18 08:57 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
/opt/mailman/var/queue/shunt:
total 9012
drwxrwx--- 2 mailman mailman 8192 Aug 4 00:00 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
-rw-rw---- 1 mailman mailman 490 Aug 1 10:26 1754036797.3635633+4a7750d6b8765f9f982dbfcbf9e972d8055bb4c5.pck
-rw-rw---- 1 mailman mailman 445 Aug 1 10:26 1754036797.618993+a6e6aefca6a76ed55c5bb4fae448f59c42a6f246.pck
-rw-rw---- 1 mailman mailman 443 Aug 1 10:26 1754036797.647361+67d36f287c76ea09b0524dcf03e080d68604d063.pck
-rw-rw---- 1 mailman mailman 443 Aug 1 10:26 1754036797.67375+0a78febb8e89d073607be70ccf90196b3e7fac17.pck
-rw-rw---- 1 mailman mailman 14233 Aug 1 10:50 1754038238.585611+3814bb1ce4232c97991a9cbcaed482966788e7c6.pck
-rw-rw---- 1 mailman mailman 14206 Aug 1 10:50 1754038251.6686325+2cacc5d5709cf5c9d8ce571947c8395eb0da37c9.pck
-rw-rw---- 1 mailman mailman 14466 Aug 1 10:51 1754038263.7415857+7f8260ca4bdf6f109be00206e07d03e717bfa6e1.pck
-rw-rw---- 1 mailman mailman 14297 Aug 1 10:51 1754038273.8673453+9ccb00ea1dc05fcbfb2bc0ef071e1f3083b0e73f.pck
-rw-rw---- 1 mailman mailman 443 Aug 1 13:30 1754047820.8963482+a122a01a47aa5c6dd8240d8ea7bc35fb3960a46d.pck
-rw-rw---- 1 mailman mailman 10870 Aug 1 15:21 1754054475.6638494+f8156ebc84effc7680b64ec03edc7d86b8f6eb65.pck
-rw-rw---- 1 mailman mailman 14782 Aug 1 16:02 1754056965.1972625+8273637db3056c2325a0903b7171725fb6f4d5e8.pck
-rw-rw---- 1 mailman mailman 221723 Aug 1 17:26 1754061973.4941757+4dc6368d88536bc195afdbce9432375166817413.pck
-rw-rw---- 1 mailman mailman 32912 Aug 2 00:00 1754085603.42874+d0b308c26d31e682d256e6cda3786797cad7b062.pck.tmp
-rw-rw---- 1 mailman mailman 17773 Aug 2 00:00 1754085627.0487497+cf822cd6608fcca545f5350124eae405df598f59.pck
-rw-rw---- 1 mailman mailman 478 Aug 2 00:00 1754085627.2411385+cbc654e510b8d618a2999274476b000542feff0e.pck
-rw-rw---- 1 mailman mailman 733213 Aug 2 00:00 1754085627.264005+5a16138d0e33252f2606ceb69577bb58d88bb2e5.pck.tmp
-rw-rw---- 1 mailman mailman 86108 Aug 2 00:00 1754085646.590158+624db84af49b742971e10004557f4289a2c5bccf.pck
-rw-rw---- 1 mailman mailman 446 Aug 2 00:00 1754085646.7170782+9ee0dedf32447f0e860dbe589fed9a272ceeadf8.pck.tmp
-rw-rw---- 1 mailman mailman 18750 Aug 2 00:01 1754085665.8180223+2d11df5f3b4b96ba710fc1bd24551cf67598be79.pck
-rw-rw---- 1 mailman mailman 21926 Aug 2 00:01 1754085665.823551+63198e22da918bc7193d93ec2e3cc01f8aaf5e44.pck
-rw-rw---- 1 mailman mailman 60639 Aug 2 00:01 1754085665.901899+4e47db603c553fa357350865de35092a4ebef5fe.pck
-rw-rw---- 1 mailman mailman 85888 Aug 2 00:01 1754085665.9268048+8ec4679a850410c2e574d4e63f0a4f3ab602af3b.pck
-rw-rw---- 1 mailman mailman 449 Aug 2 00:01 1754085666.0164568+fee1f138066b27b2752812f59e8154f47a534df7.pck
-rw-rw---- 1 mailman mailman 1407870 Aug 2 00:01 1754085696.4017332+94ab6c71d0e984764abd160f14775bb7f73b1222.pck
-rw-rw---- 1 mailman mailman 475 Aug 2 00:01 1754085696.6109+ced325862a46537b64b26257214a9190c3dde0f2.pck
-rw-rw---- 1 mailman mailman 311196 Aug 2 00:01 1754085715.0974422+7b9989730c03cff548e08961591887e17cc2324c.pck
-rw-rw---- 1 mailman mailman 437 Aug 2 00:01 1754085715.1049805+27e5b8943991e847faf240e5e29f8d0c08af4503.pck
-rw-rw---- 1 mailman mailman 29278 Aug 2 00:01 1754085715.1054037+b2fa0b025564ef9ba8d6917b8390cbe8fadf7b94.pck
-rw-rw---- 1 mailman mailman 20878 Aug 2 00:01 1754085715.1229646+805071d12ddfe493768a1e83f05db70aa92fb3bb.pck
-rw-rw---- 1 mailman mailman 7376 Aug 2 00:01 1754085715.2501519+16602d78716c922920e770b6d9e17609bdbc4adb.pck
-rw-rw---- 1 mailman mailman 1407649 Aug 2 00:02 1754085734.1192129+c43923f712a65cc4a111ad09a8f610df855a8692.pck.tmp
-rw-rw---- 1 mailman mailman 458 Aug 2 00:02 1754085734.1719563+31c0d3d15642b9c7c9f8fbfb94cc0ad8bfccd912.pck
-rw-rw---- 1 mailman mailman 12960 Aug 2 00:02 1754085734.21608+0265fcd448c9fb539c801f3a19ddb18c4956aa0e.pck
-rw-rw---- 1 mailman mailman 17254 Aug 2 15:26 1754141215.9892564+3b3dc4f6023516ad41b8a0430887f10701fccb3c.pck
-rw-rw---- 1 mailman mailman 38992 Aug 3 00:00 1754172001.3344436+8a1c1a9b3703f0421263c2b24fc27c9a9bb9116d.pck
-rw-rw---- 1 mailman mailman 1332842 Aug 3 00:00 1754172001.3540711+13acca94280dbcbb1a1cb8e730fbf87971b31aa9.pck
-rw-rw---- 1 mailman mailman 752482 Aug 3 00:00 1754172001.3596764+d2d772c50ff9812632bd6b054dfb9973758ed5c4.pck
-rw-rw---- 1 mailman mailman 18267 Aug 3 00:00 1754172026.4244561+b2e097258a1123092a199858a50f7d4cb4d5ca65.pck
-rw-rw---- 1 mailman mailman 17241 Aug 3 00:00 1754172026.4761071+9818446a413647ef57fc5f60d8fa51da43d85da8.pck
-rw-rw---- 1 mailman mailman 1407668 Aug 3 00:00 1754172026.4913034+c3055e1f383c0b3226169006e1cb79322e09846d.pck
-rw-rw---- 1 mailman mailman 50244 Aug 3 00:00 1754172045.543335+521b1a4d393691e85c86fd3a313efc6d90008b30.pck
-rw-rw---- 1 mailman mailman 32173 Aug 3 00:00 1754172045.5646617+ea2a4f2a232dfa9750e58c5b8cf9150c933e8ae7.pck
-rw-rw---- 1 mailman mailman 37411 Aug 3 00:01 1754172102.5104077+c9cabab5571fb2db62cbc688b6cc0e05f2fa2bfb.pck
-rw-rw---- 1 mailman mailman 25419 Aug 3 00:01 1754172102.5282032+20946f26d1adf7c481e504de4d589cdb1cc21edb.pck
-rw-rw---- 1 mailman mailman 276542 Aug 3 00:02 1754172120.9135556+ddbc3a3e9ce4afc78b95d03d73b0c8737f3f6606.pck
-rw-rw---- 1 mailman mailman 18018 Aug 3 00:02 1754172140.6559258+621dad8dc6a84e2ddc5413cedd7f8860b267c27c.pck
-rw-rw---- 1 mailman mailman 12887 Aug 4 00:00 1754258401.069818+189eeede43be7771f2932f2fb0954f1b09cb9fb9.pck
-rw-rw---- 1 mailman mailman 311196 Aug 4 00:00 1754258421.5529852+ca4e152d308b15023f5203309cc583929177cda2.pck
-rw-rw---- 1 mailman mailman 66982 Aug 4 00:00 1754258441.022148+91e14500fe3abe2bdcf69e56f5f200ca851e3a7e.pck
-rw-rw---- 1 mailman mailman 84863 Aug 4 00:00 1754258441.0448039+48b144d7abc1fe88b2fc6aef25398993c6bdc2e5.pck
-rw-rw---- 1 mailman mailman 20951 Aug 4 00:00 1754258459.2980123+485ba78453a56be2211e9a02277087ad5cf12b22.pck
/opt/mailman/var/queue/virgin:
total 0
drwxrwx--- 2 mailman mailman 6 Aug 4 08:47 .
drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
So let's start with bad queue:
mailman qfile /opt/mailman/var/queue/bad/1754061973.4885209+46b1ae3716439bf3ef98090296dfce0320fc3017.psv
Traceback (most recent call last):
File "/opt/mailman/mailman-venv/bin/mailman", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/mailman/bin/mailman.py", line 69, in invoke
return super().invoke(ctx)
^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/mailman-venv/lib64/python3.11/site-packages/mailman/commands/cli_qfile.py", line 63, in qfile
m.append(pickle.load(fp))
^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x95 in position 25: invalid start byte
Seems to be an decoding error
mailman qfile /opt/mailman/var/queue/bad/1754061973.4885209+46b1ae3716439bf3ef98090296dfce0320fc3017.psv
First Object is the E-Mail with a lot of HTML (Outlook seems to be the client, so...)
The Second:
{ '_parsemsg': False,
'approved': True,
'envsender': 'noreply(a)lists.example.com,
'lang': 'de',
'listid': 'kennziffern.lists.example.com,
'member_moderation_action': 'hold',
'moderation_reasons': ['The message comes from a moderated member'],
'moderation_sender': '<Sender Address>,
'moderator_approved': True,
'original_sender': <Sender Address>',
'original_size': 28614,
'original_subject': '=?iso-8859-1?Q?=C4nderungen_im_Orgaverzeichnis?=',
'received_time': datetime.datetime(2025, 7, 31, 11, 16, 46, 475199),
'recipients': { <some recipients>},
'rule_hits': ['member-moderation'],
'rule_misses': [ 'dmarc-mitigation',
'no-senders',
'approved',
'loop',
'banned-address',
'header-match-config-1',
'emergency'],
'stripped_subject': 'Änderungen im Orgaverzeichnis',
'to_list': True,
'type': 'data',
'verp': False,
'version': 3,
'whichq': 'out'}
I don't see here a problem. But the timestamp seems to be related to the restart of mailman. Can I skip this in the logrotate?
mailman qfile /opt/mailman/var/queue/bad/1754085604.0204346+ad485da0c45cb0ad17a5dc42613c3eb3f313c20e.psv
It's a digest:
[----- start pickle -----]
<----- start object 1 ----->
<----- start object 2 ----->
{ '_parsemsg': False,
'digest_number': 7,
'digest_path': '/opt/mailman/var/lists/doc-infos.lists.example.com/digest.134.7.mmdf',
'listid': 'doc-infos.lists.example.com',
'version': 3,
'volume': 134}
[----- end pickle -----]
Btw: The crontab is the following:
#####0 */2 * * * apache /opt/mailman/mailman-venv/bin/django-admin runjobs minutely --pythonpath /opt/mailman/mailman-suite/mailman-suite_project --settings settings
#*/30 * * * * mailman /opt/mailman/mailman-venv/bin/django-admin runjobs minutely --pythonpath /etc/mailman3/ --settings settings
@hourly mailman /opt/mailman/mailman-venv/bin/django-admin runjobs hourly --pythonpath /etc/mailman3/ --settings settings
#####@daily apache /opt/mailman/mailman-venv/bin/django-admin runjobs daily --pythonpath /etc/mailman3/ --settings settings
@monthly mailman /opt/mailman/mailman-venv/bin/django-admin runjobs monthly --pythonpath /etc/mailman3/ --settings settings
@yearly mailman /opt/mailman/mailman-venv/bin/django-admin runjobs yearly --pythonpath /etc/mailman3/ --settings settings
@daily mailman cd /opt/mailman; source /opt/mailman/mailman-venv/bin/activate; /opt/mailman/mailman-venv/bin/mailman digests --send > /dev/null 2>&1
mailman qfile /opt/mailman/var/queue/bad/1754085623.275817+f23139c8127c454b4fe65453af3db18e558b0e87.psv
That's the Mail which should be send to ~43000 Members.
Header says:
Received: from <ext. Server> (<IP>)
by lists.example.com (Postfix) with ESMTPS id F16D88016D1C
for <mm(a)lists.example.com>; Thu, 31 Jul 2025 09:58:38 +0200 (CEST)
The date of the file is
-rw-rw---- 1 mailman mailman 1407649 Aug 2 00:00 1754085623.275817+f23139c8127c454b4fe65453af3db18e558b0e87.psv
So i checked the mailman.log:
[2025-08-01 00:00:02 +0200] [324558] [INFO] Handling signal: term
[2025-08-01 00:00:02 +0200] [324568] [INFO] Worker exiting (pid: 324568)
[2025-08-01 00:00:02 +0200] [324571] [INFO] Worker exiting (pid: 324571)
[2025-08-01 00:00:02 +0200] [324572] [INFO] Worker exiting (pid: 324572)
[2025-08-01 00:00:02 +0200] [324574] [INFO] Worker exiting (pid: 324574)
[2025-08-01 00:00:02 +0200] [324558] [ERROR] Worker (pid:324571) was sent SIGTERM!
[2025-08-01 00:00:02 +0200] [324558] [ERROR] Worker (pid:324572) was sent SIGTERM!
[2025-08-01 00:00:02 +0200] [324558] [ERROR] Worker (pid:324568) was sent SIGTERM!
[2025-08-01 00:00:02 +0200] [324558] [ERROR] Worker (pid:324574) was sent SIGTERM!
[2025-08-01 00:00:02 +0200] [324558] [INFO] Shutting down: Master
Aug 01 00:00:11 2025 (567061) Task runner evicted 0 expired pendings
[2025-08-01 00:00:12 +0200] [567059] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:00:12 +0200] [567059] [INFO] Listening at: http://127.0.0.1:8001 (567059)
[2025-08-01 00:00:12 +0200] [567059] [INFO] Using worker: sync
[2025-08-01 00:00:12 +0200] [567069] [INFO] Booting worker with pid: 567069
[2025-08-01 00:00:12 +0200] [567070] [INFO] Booting worker with pid: 567070
[2025-08-01 00:00:12 +0200] [567071] [INFO] Booting worker with pid: 567071
[2025-08-01 00:00:12 +0200] [567073] [INFO] Booting worker with pid: 567073
Aug 01 00:00:13 2025 (567061) Task runner deleted 0 orphaned workflows
<Ommited GET Request>
Aug 01 00:00:21 2025 (567061) Task runner deleted 0 orphaned requests
[2025-08-01 00:00:23 +0200] [567059] [INFO] Handling signal: term
[2025-08-01 00:00:23 +0200] [567073] [INFO] Worker exiting (pid: 567073)
[2025-08-01 00:00:23 +0200] [567069] [INFO] Worker exiting (pid: 567069)
[2025-08-01 00:00:23 +0200] [567070] [INFO] Worker exiting (pid: 567070)
[2025-08-01 00:00:23 +0200] [567071] [INFO] Worker exiting (pid: 567071)
[2025-08-01 00:00:23 +0200] [567059] [ERROR] Worker (pid:567073) was sent SIGTERM!
[2025-08-01 00:00:23 +0200] [567059] [ERROR] Worker (pid:567070) was sent SIGTERM!
[2025-08-01 00:00:23 +0200] [567059] [ERROR] Worker (pid:567071) was sent SIGTERM!
[2025-08-01 00:00:23 +0200] [567059] [ERROR] Worker (pid:567069) was sent SIGTERM!
[2025-08-01 00:00:23 +0200] [567059] [INFO] Shutting down: Master
[2025-08-01 00:00:35 +0200] [567206] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:00:35 +0200] [567206] [INFO] Listening at: http://127.0.0.1:8001 (567206)
[2025-08-01 00:00:35 +0200] [567206] [INFO] Using worker: sync
[2025-08-01 00:00:35 +0200] [567246] [INFO] Booting worker with pid: 567246
[2025-08-01 00:00:35 +0200] [567250] [INFO] Booting worker with pid: 567250
[2025-08-01 00:00:35 +0200] [567252] [INFO] Booting worker with pid: 567252
[2025-08-01 00:00:35 +0200] [567253] [INFO] Booting worker with pid: 567253
<Ommited GET Requests>
Aug 01 00:00:36 2025 (567208) Task runner evicted 0 expired pendings
<Ommited GET Requests>
Aug 01 00:00:38 2025 (567208) Task runner deleted 0 orphaned workflows
[2025-08-01 00:00:42 +0200] [567206] [INFO] Handling signal: term
[2025-08-01 00:00:42 +0200] [567246] [INFO] Worker exiting (pid: 567246)
[2025-08-01 00:00:42 +0200] [567253] [INFO] Worker exiting (pid: 567253)
[2025-08-01 00:00:42 +0200] [567250] [INFO] Worker exiting (pid: 567250)
[2025-08-01 00:00:42 +0200] [567252] [INFO] Worker exiting (pid: 567252)
[2025-08-01 00:00:42 +0200] [567206] [ERROR] Worker (pid:567250) was sent SIGTERM!
[2025-08-01 00:00:42 +0200] [567206] [ERROR] Worker (pid:567252) was sent SIGTERM!
[2025-08-01 00:00:42 +0200] [567206] [ERROR] Worker (pid:567246) was sent SIGTERM!
[2025-08-01 00:00:42 +0200] [567206] [ERROR] Worker (pid:567253) was sent SIGTERM!
[2025-08-01 00:00:42 +0200] [567206] [INFO] Shutting down: Master
Aug 01 00:00:54 2025 (567280) Task runner evicted 2 expired pendings
Aug 01 00:00:56 2025 (567280) Task runner deleted 0 orphaned workflows
[2025-08-01 00:00:58 +0200] [567278] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:00:58 +0200] [567278] [INFO] Listening at: http://127.0.0.1:8001 (567278)
[2025-08-01 00:00:58 +0200] [567278] [INFO] Using worker: sync
[2025-08-01 00:00:58 +0200] [567327] [INFO] Booting worker with pid: 567327
[2025-08-01 00:00:58 +0200] [567328] [INFO] Booting worker with pid: 567328
[2025-08-01 00:00:58 +0200] [567329] [INFO] Booting worker with pid: 567329
[2025-08-01 00:00:58 +0200] [567330] [INFO] Booting worker with pid: 567330
<Ommited GET Requests>
[2025-08-01 00:01:00 +0200] [567278] [INFO] Handling signal: term
[2025-08-01 00:01:00 +0200] [567327] [INFO] Worker exiting (pid: 567327)
[2025-08-01 00:01:00 +0200] [567328] [INFO] Worker exiting (pid: 567328)
[2025-08-01 00:01:00 +0200] [567329] [INFO] Worker exiting (pid: 567329)
[2025-08-01 00:01:00 +0200] [567330] [INFO] Worker exiting (pid: 567330)
[2025-08-01 00:01:01 +0200] [567278] [ERROR] Worker (pid:567328) was sent SIGTERM!
[2025-08-01 00:01:01 +0200] [567278] [ERROR] Worker (pid:567329) was sent SIGTERM!
[2025-08-01 00:01:01 +0200] [567278] [ERROR] Worker (pid:567330) was sent SIGTERM!
[2025-08-01 00:01:01 +0200] [567278] [ERROR] Worker (pid:567327) was sent SIGTERM!
[2025-08-01 00:01:01 +0200] [567278] [INFO] Shutting down: Master
Aug 01 00:01:12 2025 (567381) Task runner evicted 2 expired pendings
[2025-08-01 00:01:13 +0200] [567379] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:01:13 +0200] [567379] [INFO] Listening at: http://127.0.0.1:8001 (567379)
[2025-08-01 00:01:13 +0200] [567379] [INFO] Using worker: sync
[2025-08-01 00:01:13 +0200] [567397] [INFO] Booting worker with pid: 567397
[2025-08-01 00:01:13 +0200] [567398] [INFO] Booting worker with pid: 567398
[2025-08-01 00:01:13 +0200] [567399] [INFO] Booting worker with pid: 567399
[2025-08-01 00:01:13 +0200] [567400] [INFO] Booting worker with pid: 567400
<Ommited GET Request>
Aug 01 00:01:13 2025 (567381) Task runner deleted 0 orphaned workflows
<Ommited GET Requests>
Aug 01 00:01:19 2025 (567381) Task runner deleted 0 orphaned requests
<Ommited GET Requesta>
[2025-08-01 00:01:34 +0200] [567379] [INFO] Handling signal: term
[2025-08-01 00:01:34 +0200] [567397] [INFO] Worker exiting (pid: 567397)
[2025-08-01 00:01:34 +0200] [567399] [INFO] Worker exiting (pid: 567399)
[2025-08-01 00:01:34 +0200] [567398] [INFO] Worker exiting (pid: 567398)
[2025-08-01 00:01:34 +0200] [567400] [INFO] Worker exiting (pid: 567400)
[2025-08-01 00:01:34 +0200] [567379] [ERROR] Worker (pid:567399) was sent SIGTERM!
[2025-08-01 00:01:34 +0200] [567379] [ERROR] Worker (pid:567398) was sent SIGTERM!
[2025-08-01 00:01:34 +0200] [567379] [ERROR] Worker (pid:567397) was sent SIGTERM!
[2025-08-01 00:01:34 +0200] [567379] [ERROR] Worker (pid:567400) was sent SIGTERM!
[2025-08-01 00:01:34 +0200] [567379] [INFO] Shutting down: Master
[2025-08-01 00:01:46 +0200] [567516] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:01:46 +0200] [567516] [INFO] Listening at: http://127.0.0.1:8001 (567516)
[2025-08-01 00:01:46 +0200] [567516] [INFO] Using worker: sync
[2025-08-01 00:01:46 +0200] [567525] [INFO] Booting worker with pid: 567525
[2025-08-01 00:01:46 +0200] [567526] [INFO] Booting worker with pid: 567526
[2025-08-01 00:01:46 +0200] [567527] [INFO] Booting worker with pid: 567527
[2025-08-01 00:01:46 +0200] [567528] [INFO] Booting worker with pid: 567528
Aug 01 00:01:47 2025 (567518) Task runner evicted 2 expired pendings
Aug 01 00:01:48 2025 (567518) Task runner deleted 0 orphaned workflows
<Ommited GET Request>
[2025-08-01 00:01:52 +0200] [567516] [INFO] Handling signal: term
[2025-08-01 00:01:52 +0200] [567526] [INFO] Worker exiting (pid: 567526)
[2025-08-01 00:01:52 +0200] [567525] [INFO] Worker exiting (pid: 567525)
[2025-08-01 00:01:52 +0200] [567527] [INFO] Worker exiting (pid: 567527)
[2025-08-01 00:01:52 +0200] [567528] [INFO] Worker exiting (pid: 567528)
[2025-08-01 00:01:52 +0200] [567516] [ERROR] Worker (pid:567526) was sent SIGTERM!
[2025-08-01 00:01:52 +0200] [567516] [ERROR] Worker (pid:567525) was sent SIGTERM!
[2025-08-01 00:01:52 +0200] [567516] [ERROR] Worker (pid:567527) was sent SIGTERM!
[2025-08-01 00:01:52 +0200] [567516] [ERROR] Worker (pid:567528) was sent SIGTERM!
[2025-08-01 00:01:52 +0200] [567516] [INFO] Shutting down: Master
Aug 01 00:02:06 2025 (567648) Task runner evicted 2 expired pendings
[2025-08-01 00:02:06 +0200] [567646] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:02:06 +0200] [567646] [INFO] Listening at: http://127.0.0.1:8001 (567646)
[2025-08-01 00:02:06 +0200] [567646] [INFO] Using worker: sync
[2025-08-01 00:02:06 +0200] [567688] [INFO] Booting worker with pid: 567688
[2025-08-01 00:02:06 +0200] [567689] [INFO] Booting worker with pid: 567689
[2025-08-01 00:02:06 +0200] [567690] [INFO] Booting worker with pid: 567690
[2025-08-01 00:02:06 +0200] [567691] [INFO] Booting worker with pid: 567691
Aug 01 00:02:07 2025 (567648) Task runner deleted 0 orphaned workflows
[2025-08-01 00:02:11 +0200] [567646] [INFO] Handling signal: term
[2025-08-01 00:02:11 +0200] [567689] [INFO] Worker exiting (pid: 567689)
[2025-08-01 00:02:11 +0200] [567688] [INFO] Worker exiting (pid: 567688)
[2025-08-01 00:02:11 +0200] [567690] [INFO] Worker exiting (pid: 567690)
[2025-08-01 00:02:11 +0200] [567691] [INFO] Worker exiting (pid: 567691)
[2025-08-01 00:02:11 +0200] [567646] [ERROR] Worker (pid:567688) was sent SIGTERM!
[2025-08-01 00:02:11 +0200] [567646] [ERROR] Worker (pid:567689) was sent SIGTERM!
[2025-08-01 00:02:11 +0200] [567646] [ERROR] Worker (pid:567690) was sent SIGTERM!
[2025-08-01 00:02:11 +0200] [567646] [ERROR] Worker (pid:567691) was sent SIGTERM!
[2025-08-01 00:02:11 +0200] [567646] [INFO] Shutting down: Master
[2025-08-01 00:02:24 +0200] [567717] [INFO] Starting gunicorn 22.0.0
[2025-08-01 00:02:24 +0200] [567717] [INFO] Listening at: http://127.0.0.1:8001 (567717)
[2025-08-01 00:02:24 +0200] [567717] [INFO] Using worker: sync
[2025-08-01 00:02:24 +0200] [567786] [INFO] Booting worker with pid: 567786
[2025-08-01 00:02:24 +0200] [567789] [INFO] Booting worker with pid: 567789
[2025-08-01 00:02:24 +0200] [567792] [INFO] Booting worker with pid: 567792
[2025-08-01 00:02:24 +0200] [567794] [INFO] Booting worker with pid: 567794
<Ommited GET Requests>
Aug 01 00:02:25 2025 (567719) Task runner evicted 2 expired pendings
Aug 01 00:02:26 2025 (567719) Task runner deleted 0 orphaned workflows
<Ommited GET Requests>
Aug 01 00:02:33 2025 (567719) Task runner deleted 0 orphaned requests
[01/Aug/2025:00:02:35 +0200] "GET /3.1/lists/ifip-tc6(a)lists.rwth-aachen.de HTTP/1.1" 200 423 "-" "GNU Mailman REST client v3.3.5"
[01/Aug/2025:00:02:35 +0200] "GET /3.1/lists/smartlist(a)lists.rwth-aachen.de HTTP/1.1" 200 438 "-" "GNU Mailman REST client v3.3.5"
Aug 01 00:02:42 2025 (567719) Task runner deleted 2 orphaned messages
Aug 01 00:02:42 2025 (567719) Task runner deleted 0 orphaned message files
Aug 01 00:02:42 2025 (567719) Task runner evicted 2 expired bounce events
Aug 01 00:02:42 2025 (567719) Task runner evicted expired cache entries
Well...i will stop the restart after the log rotate today.
>>
>> IIRC all of the shunted messages that Stephen looked at with qfiles
> were those special digest messages (ie, message component empty,
>> pointer to lists/$LIST/something.mmdf in the msg_data component). So
>> something is going wrong in the to-digest handler.
>
>
>And for every one of those shunted messages there should be an exception with traceback logged in mailman.log. Those tracebacks should be helpful.
If there were any. Maybe the "debug" level should be "info". But for which logs?
Maybe the restart at night after the lograte maybe the issue.
--
Stephan Krinetzki
IT Center
Gruppe: Anwendungsbetrieb und Cloud
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24866
Fax: +49 241 80-22134
krinetzki(a)itc.rwth-aachen.de
www.itc.rwth-aachen.de
Social Media Kanäle des IT Centers:
https://blog.rwth-aachen.de/itc/
https://www.facebook.com/itcenterrwth
https://www.linkedin.com/company/itcenterrwth
https://twitter.com/ITCenterRWTH
https://www.youtube.com/channel/UCKKDJJukeRwO0LP-ac8x8rQ
-----Original Message-----
From: Mark Sapiro <mark(a)msapiro.net>
Sent: Saturday, August 2, 2025 6:11 PM
To: Stephen J. Turnbull <steve(a)turnbull.jp>
Cc: mailman-users(a)mailman3.org
Subject: [MM3-users] Re: Held messages not delivered after approval
On 8/2/25 01:51, Stephen J. Turnbull wrote:
>
> IIRC all of the shunted messages that Stephen looked at with qfiles
> were those special digest messages (ie, message component empty,
> pointer to lists/$LIST/something.mmdf in the msg_data component). So
> something is going wrong in the to-digest handler.
And for every one of those shunted messages there should be an exception with traceback logged in mailman.log. Those tracebacks should be helpful.
> The number of .tmp files lying around bothers me. AFA grep CS, the
> only place that can happen is in switchboard.py:136 in .enqueue:
>
> with open(tmpfile, 'wb') as fp:
> fp.write(msgsave)
> pickle.dump(data, fp, protocol)
> fp.flush()
> os.fsync(fp.fileno())
> os.rename(tmpfile, filename)
>
> where `msgsave` is already a pickled object. So either pickle.dump is
> choking on something in data (the metadata, which I believe is all
> primitive Python data types), or something (OOM kill?) is happening at
> the OS level. A crash in pickle.dump should leave an exception log
> and backtrace in the logs.
>
> AFAIK, Mailman does not clean up .tmp files at startup, right?
That is correct. The *.pck.tmp file is created by the above code and immediately after writing is renamed to *.pck. It is done this way to prevent another process picking up a partially written *.pck.
If a *.pck.tmp file is somehow left behind, it is never looked at or deleted by any Mailman code.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
This message sent to krinetzki(a)itc.rwth-aachen.de
7 months, 1 week