Search results for query "sapiro"
- 6292 messages
[MM3-users] Re: Held messages not delivered after approval
by Odhiambo Washington
On Thu, Jul 31, 2025 at 8:21 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 7/31/25 04:29, Stephan Krinetzki wrote:
> >
> > /opt/mailman/var/queue/out:
> > total 2868
> > drwxrwx--- 2 mailman mailman 4096 Jul 31 13:26 .
> > drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
> > -rw-rw---- 1 mailman mailman 221708 Jul 30 13:16
> 1753874180.0845337+0cc7849043859a79dc3678a0d8b63c1c66df0c66.pck.tmp
> > -rw-rw---- 1 mailman mailman 733425 Jul 31 00:00
> 1753912841.8847518+da55f8789ae41b75d19a02b595e3fd6d45983ade.pck.tmp
>
> These indicate a problem of some sort. .pck.tmp files should only exist
> for a fraction of a second. See
>
> https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/core/switchboa…
>
> What does `mailman qfile report for these? Are they the missing messages?
>
> > -rw-rw---- 1 mailman mailman 17758 Jul 31 13:26
> 1753961185.0481033+6adf966467266567275f1146f5054c95e4365c13.pck
> > -rw-rw---- 1 mailman mailman 31122 Jul 31 13:26
> 1753961185.1064982+b1f502e2af56b9b11680135c6de5fcc5285d967e.bak
> > -rw-rw---- 1 mailman mailman 12945 Jul 31 13:26
> 1753961185.1562705+a88534cfa9b03012c248ac00cbb54f4fc08c8dcc.pck
> > -rw-rw---- 1 mailman mailman 1407634 Jul 31 13:26
> 1753961185.2598634+96e501d7aa8e72fbb9f9a5c953d48fc2c7db2bc7.pck
> > -rw-rw---- 1 mailman mailman 17570 Jul 31 13:26
> 1753961185.3654013+d693065f2b9a2338f887d3ac9c0669c957131100.pck
> > -rw-rw---- 1 mailman mailman 60624 Jul 31 13:26
> 1753961185.4132354+8998ee9eb9a8a99faf1fcf045d604e669f5d0c9e.pck
> > -rw-rw---- 1 mailman mailman 7361 Jul 31 13:26
> 1753961185.413462+50dcef9bb178f54ecb2d4707ccfe02773f92336c.pck
> > -rw-rw---- 1 mailman mailman 84848 Jul 31 13:26
> 1753961185.4464717+17836b8875aa6643f7e5fb0744e72940ce5603c6.pck
> > -rw-rw---- 1 mailman mailman 18003 Jul 31 13:26
> 1753961185.4929028+17e9c78a2721cbc344124c6e04a129217ea6c794.pck
> > -rw-rw---- 1 mailman mailman 66982 Jul 31 13:26
> 1753961185.5228019+4ef59e61c6a79dd4cdee6d1110fb0582fcfafb6c.pck
> > -rw-rw---- 1 mailman mailman 221723 Jul 31 13:26
> 1753961185.5756643+6de90ae99b3d528ce5234b78a287e19f7b6686fc.pck
>
>
> Also, this is a lot of outgoing messages to be queued. Are you sending
> periodic digests at this time? If you normally see a lot of messages in
> the out queue, you might consider setting it's `instances` to 2 or 4. E.g.,
> ```
> [runner.out]
> instances: 4
> ```
> in mailman.cfg
>
> > /opt/mailman/var/queue/shunt:
> > total 3304
> > drwxrwx--- 2 mailman mailman 4096 Jul 31 11:44 .
> > drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
> > -rw-rw---- 1 mailman mailman 451 Jul 31 00:00
> 1753912822.2651796+28eceef7e18eb70393377b88dc7117af8f9362a0.pck
> > -rw-rw---- 1 mailman mailman 490 Jul 31 00:00
> 1753912838.4197352+ea531cf0262c1faa58b1679b907fee92bc16822c.pck
> > -rw-rw---- 1 mailman mailman 1407870 Jul 31 00:00
> 1753912841.9177196+ccea15bdefce3a54301281c8eddf86e8230244a6.pck
> > -rw-rw---- 1 mailman mailman 86108 Jul 31 00:00
> 1753912841.9197443+7dcef4febc71e44c6d9309a24a08b08753e1ff42.pck
> > -rw-rw---- 1 mailman mailman 1407668 Jul 31 00:00
> 1753912841.9849963+a3f1869b750060c97262ece38737480d91652828.pck
> > -rw-rw---- 1 mailman mailman 38992 Jul 31 00:01
> 1753912860.5167956+940584c4f361cbd8c29e390b2f60590558effe40.pck
> > -rw-rw---- 1 mailman mailman 440 Jul 31 00:01
> 1753912860.6972685+635c065bac8dff5f9d562275d707001d773b84c1.pck
> > -rw-rw---- 1 mailman mailman 445 Jul 31 00:01
> 1753912868.7903054+befa066f254d7a3529a8555a6c942a554715d837.pck
> > -rw-rw---- 1 mailman mailman 33494 Jul 31 00:01
> 1753912878.7562895+82b724fd93260ab9a2bb49709d3a42a2f32f2c80.pck
> > -rw-rw---- 1 mailman mailman 217073 Jul 31 00:01
> 1753912878.9337828+de73a65d9c6febfa80275853921f4b53fd1d9e2a.pck
> > -rw-rw---- 1 mailman mailman 85888 Jul 31 00:02
> 1753912950.303359+8a87bce0be63ac1df8493c6b1ad6ae154fcedba7.pck
> > -rw-rw---- 1 mailman mailman 50244 Jul 31 00:02
> 1753912950.4970112+d44d493912bb3024547b8a5112f86f035dcb352f.pck
> > -rw-rw---- 1 mailman mailman 12887 Jul 31 00:02
> 1753912970.038427+31bcbd7fb2ebdf81f6de24b7283b50bcda6ded21.pck.tmp
> > -rw-rw---- 1 mailman mailman 443 Jul 31 11:44
> 1753955094.900898+67bc76525412da66a7c76363f65f583989716305.pck
> >
> > /opt/mailman/var/queue/virgin:
> > total 32
> > drwxrwx--- 2 mailman mailman 81 Jul 31 13:17 .
> > drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
> > -rw-rw---- 1 mailman mailman 32013 Jan 11 2025
> 1736550035.6163204+472f81ece5e45a2651a4499bef418f611b43c619.pck.tmp
> >
> > Nothing special there (shunt should be checked, but not in correlation
> with my mail).
>
>
> Actually those .pck.tmp files indicate some issue.
>
> > I have one list which archives the mail. In the archive there is the
> mail, but it doesn't get deliverd
>
> Which says the pipeline to_archive handler is adding the message to the
> archive queue and the archive runner is handling it, so presumably the
> pipeline to_outgoing handler is adding the message to the out queue, so
> the issue is in the outging runner.
>
In one response, he mentioned that "Other mailinglists on the same host are
doing fine. Even a 1:1 copy of the list works - only the original list does
not work".
Does it mean that "the issue is in the outgoing runner" only for a single
list?
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
1 year, 1 month
[MM3-users] Re: Apache+mod_wsgi issue
by Odhiambo Washington
On Sat, Dec 24, 2022 at 9:40 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/24/22 00:00, Odhiambo Washington wrote:
> >
> > These are my files:
> > 1. urls.py
> > <CUT >
> > from django.conf.urls import include
> > from django.urls import re_path
> > from django.contrib import admin
> > from django.urls import reverse_lazy
> > from django.views.generic import RedirectView
> >
> > urlpatterns = [
> > re_path(r'^$', RedirectView.as_view(
> > url=reverse_lazy('list_index'),
> > permanent=True)),
> > re_path(r'^postorius/', include('postorius.urls')),
> > re_path(r'^hyperkitty/', include('hyperkitty.urls')),
> > re_path(r'', include('django_mailman3.urls')),
> > re_path(r'^accounts/', include('allauth.urls')),
> > # Django admin
> > re_path(r'^admin/', admin.site.urls),
> > re_path(r'^mailman3/', include('postorius.urls')),
> > re_path(r'^archives/', include('hyperkitty.urls')),
> > ]
> > </CUT>
>
>
> You don't need all of those. ^postorius/ and ^mailman3/ are two
> different ways to access Postorius, and ^hyperkitty/ and ^archives/ are
> two different ways to access HypereKitty. You only need one of each, but
> having both is OK.
>
That urls.py was picked from
https://wiki.list.org/DOC/Mailman%203%20installation%20experience.
Maybe that page needs some edits/revision now?
> 2. apache.conf
> > <CUT>
> > ...
> > Alias /static "/opt/mailman/mm/static"
> > <Directory "/opt/mailman/mm/static">
> > Require all granted
> > </Directory>
> > # WSGIScriptAlias /mailman3 /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /accounts /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /admin /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /archives /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /user-profile /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /hyperkitty /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /postorius /opt/mailman/mm/wsgi.py
> > WSGIScriptAlias / /opt/mailman/mm/wsgi.py
>
>
> This will send every http request to mod_wsgi except for requests to
> /static. This is OK if there are no other web sites on this server and
> the only http requests are for Django related things.
>
This is just a virtual host. There are other websites on the server.
...
> > I have tested with creating all those commented out aliases and that
> ended
> > up still giving me /mailman3/mailman3/* which I did not
> > like, although everything appeared to work.
> > I then ended up with only "WSGIScriptAlias / /opt/mailman/mm/wsgi.py"
> > and now when I type https://mm3-lists.kictanet.or.ke I end
> > up at https://mm3-lists.kictanet.or.ke/mailman3/lists/. Of course
> something
> > has appended "/mailman3"
> > So this has resulted in all URLs having /mailman3/ appended, except for
> the
> > /archives link.
>
>
> How about https://mm3-lists.kictanet.or.ke/user_profile/ ?
>
I have uncommented the line "WSGIScriptAlias /user-profile
/opt/mailman/mm/wsgi.py",
I don't see any definition for it in urls.py as well. Perhaps that's the
issue?
When I uncomment /user-profile, I end up with an ugly URL:
https://mm3-lists.kictanet.or.ke/user-profile/user-profile/
So instead of aliasing the /user-profile, I have decided to add another
definition to urls.py:
re_path(r'^user-profile/', include('allauth.urls')),
Which I think gives me a cleaner URL:
https://mm3-lists.kictanet.or.ke/user-profile/login/?next=/user-profile/
I also had to alias /accounts, which then brings me to:
https://mm3-lists.kictanet.or.ke/accounts/mailman3/lists/ - which I believe
is fine because one gets the
option to either login or signup.
And I think that "/account" and "/user-profile" are synonymous.
> Now my question is about what could be adding the /mailman3. Is it
> possible
> > that I installed something out of the ordinary?
>
>
> It's a mod_wsgi thing. I haven't used mod_wsgi for Mailman for some time
> (I now use gunicorn on all the sites I manage), but when I did I think I
> had
> WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py
>
> and all the URLs were like https://example.com/mm3/mailman3/...,
> https://example.com/mm3/archives/..., etc.
>
I already had gunicorn working too, but I just need to also figure out this
mod_wsgi.
I will stop bothering about the /mailman3 being appended, as long as it
serves the required purpose.
Thank you for all the help.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
3 years, 8 months
[MM3-users] Re: hello， Discussion on how to subscribe to the mailing list
by Stephen J. Turnbull
Mark Sapiro writes:
> On 1/16/23 18:55, 朱超 wrote:
> > Hi, When I subscribe the email list in Subscription Policy is
> > confirm, Some problems happen in here：
> > 1.Some mail mua reject send the confirm the message, because the
> > length of recipient is too long. for exmaple:
> > requirements-confirm+9605556e0b35a5da280c1de0d34a3e6f770f8ea5@domain,
> > and the mail mua reject to modify about this.
>
> The default English template says in part:
>
> Before you can start using GNU Mailman at this site, you must first
> confirm that this is your email address. You can do this by replying to
> this message.
>
> Or you should include the following line -- and only the following
> line -- in a message to $request_email:
>
> confirm $token
>
> I.e. It gives an alternative method if `reply` doesn't work. You can
> provide a custom version of this template with different wording if you
> think you can improve it.
The problem is the reply address, though. Changing the template to
suggest only the "confirm $token" in body method doesn't help people
who hit R expecting that should work, especially since many "user
fiendly" (misspelling intentional) MUAs don't display the address.
Avoiding that requires code changes, I guess?
> > 2.It is inconvenient for users to use. Many people will
> > mistakenly think that after clicking the subscribe button, it
> > will be successful, and they will not process the email.
> step, you can set Subscription Policy to Open, but then anyone can
> maliciously subscribe third parties to the list.
*Please* do not do this if your server is exposed to the Internet.
There are bots that search for such servers, and sell lists of them to
people who want to DOS mailboxes. You may find yourself banned across
the Internet.
The only real alternative to confirmation is approval by moderators.
I believe Mailman 3 offers that alternative as Mailman 2 did, can't
check at the moment.
> > Regarding the discussion on the subscription method of the
> > mailing list, should we consider optimizing the subscription
> > method of the mailing list, for example:
> > 1. use the method of sending a verification code to subscribe by
> > email，When you receive the verification code sent by the
> > mailing list and fill it in again, you can subscribe
> > successfully。
>
> How is this different from the present method.
Just wording. I think they are thinking of the "invite" option that
was provided in Mailman 2 mass subscribe (again, I think Mailman 3
retains it but I never ever used it so I don't recall for sure). But
it doesn't make sense for a user-initiated interaction.
> If you want, you can make
> a custom template and include something like:
>
> Alternatively, you can confirm by going to the URL
>
> https://www.example.com/mailman3/lists/$list_id/confirm/?token=$token
If I understand the OP's PoV correctly, I think I would change the
list information page to say "Send a personal invitation to join the
list to [email address here]. If you aren't a subscriber, you can
invite yourself." And change the word "confirm" in the confirmation
template to "subscribe".
> We don't include that in the default template because Mailman core
> doesn't know if you even have a web UI or how to access it if you
> do.
We should fix that, at least if the web interface is Postorius
(wishlist, assigned to me):
https://gitlab.com/mailman/mailman/-/issues/1055
@mark
https://gitlab.com/mailman/mailman/-/issues/901
Maybe we can close this? I'm not sure why it was left open since
there's no action proposed for Mailman improvement.
> > 2. use the email to send a connection to confirm the
> > subscription，People click the link to be redirected to a
> > confirmation page to subscribe to the email。
>
> See above.
Indeed, AFAICS almost everything the OP is suggesting amounts to an
identical procedure as far as the Mailman server is concerned, it's
just described to the subscriber differently. The one exception is
the suggestion to eliminate the confirmation step, and while that is
supported, it's quite likely to cause problems for third parties and
eventually for the Mailman site unless you have an alternative ground
source of truth such as an employee database.
Steve
3 years, 7 months
[MM3-users] Re: Postorius claims 2 messages held when there are none - out of sync
by Ken Alker
--On Monday, June 26, 2023 8:24 AM -0700 Mark Sapiro <mark(a)msapiro.net>
wrote:
> On 6/26/23 12:01 AM, Ken Alker wrote:
>> When I view a list via Postorius the "Held Messages" menu item has an
>> inverse-red-"2" next to it, as if there are two held messages. There
>> are, in fact, no messages in "/opt/mailman/mm/var/queue/*" (assuming the
>> held messages should be in "pipeline" subdirectory). When I click on
>> "Held Messages" I get "Something went wrong" followed by "HTTP Error
>> 500: {"title": "500 Internal Server Error"}". How do I fix this?
>
> The held messages are not in any queue. They are stored in Mailman's
> var/messages/ hierarchy. They can be examined with `mailman qfile`
Is the var/messages hierarchy also where all the archived emails are stored
for future viewing via the web interface?
> Also, there is a long thread at
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/threa
> d/UWCUQYU7Q5X5LWNII57ZZ4DDXE4DEFX7/ and maybe others that might be
> relevant.
I have never used the mailman shell before, but based on the referenced
thread I learned to do this (note that the thread also taught me how to
rewrite it to narrow down to just the list with issues so I didn't delete
held messages in other lists, if my case):
for verification:
venv) mailman@speedy:~/web/logs$ mailman shell -l
sbarc-list(a)lists.netlojix.com
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list
>>> requests = IListRequests(m)
>>> first = True
>>> for request in requests.held_requests:
... key, data = requests.get_request(request.id)
... if first:
... first = False
... print(m.list_id)
... print (f"""\
... Sender: {data['_mod_sender']}
... Subject: {data['_mod_subject']}
... Date: {data['_mod_hold_date']}
... Reason: {data['_mod_reason']}
... """)
...
and then to delete the two messages:
(venv) mailman@speedy:~/web/logs$ mailman shell -l
sbarc-list(a)lists.netlojix.com
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list
>>> requests = IListRequests(m)
>>> first = True
>>> for request in requests.held_requests:
... key, data = requests.get_request(request.id)
... if first:
... first = False
... print(m.list_id)
... print(f'request id: {request.id}, deleting')
... requests.delete_request(request.id)
...
sbarc-list.lists.netlojix.com
request id: 27, deleting
request id: 28, deleting
>>>
The result of this, however, was that the web interface still claimed I had
two held messages. I wasn't sure if there was some cron-type "sweeper"
process I had to wait to auto-run or if I had to manually do something to
create an update. I then realized that the sample script had a ' >>>
commit ()" ' at the end, which I'd missed. So, I re-entered the script
(but it didn't spit out that it deleted anything this time) and got an
error when I did the commit. So, I figured the messages HAD been delete
and the commit wasn't really necessary. When I went back to the web
interface, the number of held messages was at 0 and I no longer get the
original error when clicking on "Held messages", so my issue is solved.
But this does leave me with some educational questions:
1) Did I affect something somehow with my "commit", or was there a job that
cleaned things up (if so, what job)?
2) Is the "commit" even necessary (I saw it in someone else's script in the
thread)?
3) Is there a way to save a script and then execute it via shell rather
than typing it in every time (it appears from the examples that people are
typing them in every time, which is super tedious; I didn't paste for fear
of doing something damaging, but maybe that would work)?
4) Are the request.id's of the messages dynamic/changing, or are they fixed
for life? I thought I'd delete based on exact request.id without looping
but I was afraid maybe they were changing over time and that this wouldn't
be a good idea. Might be nice to know for the future.
Thanks for the guidance!
Ken
3 years, 2 months
[MM3-users] Re: Attachement processings questions
by Lars Schimmer
On 2020-02-03 20:24, Mark Sapiro wrote:
> On 2/3/20 4:27 AM, Lars Schimmer wrote:
>>
>> In a mailinglist it seems some mails are not forwarded to a member, but
>> is listed in archive in hyperkitty.
>> That mail did include a PDF attachement, but HyperKitty does not list
>> any attachement.
>
>
> There are two issues here. Why wasn't the message delivered to list
> members?, and why is there no attachment in Hyperkitty?
>
> For the first issue, you need to examine mailman.log for the time of the
> post looking for errors and tracebacks. Also, the MTA logs may have
> useful info if the messages were sent but not delivered.
Thank you so far, I maye found the issue:
Uncaught runner exception: 'ascii' codec can't encode character '\xe9'
in position 109: ordinal not in range(128)
Uncaught runner exception: 'ascii' codec can't encode character '\xa0'
in position 37: ordinal not in range(128)
Seems to be the problemm, as it happens the same moment as the
problematic email arrives in exim4:
2020-01-31 03:02:41 1ixLdp-0003Mo-5z => user(a)abc.de R=mailman3_router
T=mailman3_transport H=localhost [127.0.0.1] C="250 Ok"
2020-01-31 03:02:41 1ixLdp-0003Mo-5z Completed
and nothing more. No sign of mailman3 sending out any mail via this list.
Other emails getting to that list will be send out via user-bounces(a)abc.de.
Those mails are in the mailinglist archives, not on hold or anything.
Just the attachement missing.
So, please, howto fix the ascii encoding error in mailman3?
Full trace:
Jan 31 03:02:43 2020 (32591) Uncaught runner exception: 'ascii' codec
can't encode character '\xe9' in position 109: ordinal not in range(128)
Jan 31 03:02:43 2020 (32591) Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line
173, in _one_iteration
self._process_one_file(msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line
266, in _process_one_file
keepqueued = self._dispose(mlist, msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/runners/pipeline.py",
line 37, in _dispose
process(mlist, msg, msgdata, pipeline)
File "/usr/lib/python3/dist-packages/mailman/core/pipelines.py", line
50, in process
handler.process(mlist, msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/handlers/to_digest.py",
line 48, in process
mbox.add(msg)
File "/usr/lib/python3.7/mailbox.py", line 603, in add
self._toc[self._next_key] = self._append_message(message)
File "/usr/lib/python3.7/mailbox.py", line 758, in _append_message
offsets = self._install_message(message)
File "/usr/lib/python3.7/mailbox.py", line 830, in _install_message
self._dump_message(message, self._file, self._mangle_from_)
File "/usr/lib/python3.7/mailbox.py", line 215, in _dump_message
gen.flatten(message)
File "/usr/lib/python3.7/email/generator.py", line 116, in flatten
self._write(msg)
File "/usr/lib/python3.7/email/generator.py", line 181, in _write
self._dispatch(msg)
File "/usr/lib/python3.7/email/generator.py", line 214, in _dispatch
meth(msg)
File "/usr/lib/python3.7/email/generator.py", line 432, in _handle_text
super(BytesGenerator,self)._handle_text(msg)
File "/usr/lib/python3.7/email/generator.py", line 249, in _handle_text
self._write_lines(payload)
File "/usr/lib/python3.7/email/generator.py", line 155, in _write_lines
self.write(line)
File "/usr/lib/python3.7/email/generator.py", line 406, in write
self._fp.write(s.encode('ascii', 'surrogateescape'))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in
position 109: ordinal not in range(128)
> For the second, there are many content filtering settings not (yet)
> exposed in Postorius. These include filter_extensions, filter_types,
> pass_extensions and pass_types which control which MIME parts are
> removed. For a newly created Mailman 3 list, these four settings are
> empty, but importing a Mailman 2.1 list will set them according to the
> corresponding MM 2.1 list's settings.
>
>
>> All I found was "convert HTML to plaintext", but does this setting
>> remove attachements?
>
>
> It would only affect HTML parts, not pdfs.
>
>
>> Is there any generel documentation about attachement processing in MM3?
>
>
> <https://mailman.readthedocs.io/en/latest/src/mailman/handlers/docs/filterin…>.
>
>
>
>> I would like this mailinglist to accept mails with attachements, send it
>> to the members and archvie the attachments in HyperKitty, to.
>
>
> You can set Settings -> Alter Messages -> Filter content to No in
> Postorius to not do any content filtering at all. This will override the
> Collapse alternatives and Convert html to plaintext settings and pass
> the message without removing any MIME parts.
>
> But, I think you must have other issues, either with this mail in
> particular or in general as evidenced by the mail not being received by
> list members.
Ok, first fix the ascii encode error for me. Any idea how?
thank you.
MfG,
Lars Schimmer
--
-------------------------------------------------------------
TU Graz, Institut für ComputerGraphik & WissensVisualisierung
Tel: +43 316 873-5405 E-Mail: l.schimmer(a)cgv.tugraz.at
Fax: +43 316 873-5402 PGP-Key-ID: 0x4A9B1723
6 years, 7 months
[MM3-users] Re: integrating mm3 with postfix / lmtp
by Thor Atle Rustad
I will try to get a grip on uid namespaces then...
2017-11-13 10:49 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
> On Nov 13 2017, at 1:34 am, Thor Atle Rustad <thor.rustad(a)gmail.com>
> wrote:
>
>> There is a way around it!
>>
>> I have had two issues with the the maxking docker image. One is that the
>> regexp is not working properly. I reported that, and it has been fixed in
>> newer code. My other problem is that the docker image creates a user,
>> mailman, that receives uid 103. Well, uid 103 on my system is already taken
>> by systemd-bus-proxy (grep 103 /etc/passwd returns
>> "systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/
>> false").
>>
>
> Containers usually run with uid namespace, so it doesn't really matter
> what uid is used outside of container.
>
> Unless, you mount /etc/shadow from host in to the container, which isn't
> really needed for the images.
>
>
>> My solution includes downloading the corrected postfix.py, and replacing
>> the Dockerfile. I put the postfix.py in <docker-mailman>/core/assets/.
>>
>> My Dockerfile:
>> FROM maxking/mailman-core
>>
>> RUN grep mailman /etc/passwd && grep mailman /etc/group \
>> && deluser mailman \
>> && addgroup -S -g900 mailman \
>> && adduser -S -u900 mailman mailman \
>> && grep mailman /etc/passwd && grep mailman /etc/group
>> COPY assets/postfix.py /usr/local/lib/python3.6/site-
>> packages/mailman/mta/postfix.py
>>
>> I then run docker build (with -t parameter, you must look up that
>> yourself). I use a different name for my images, so I end up with (note,
>> there are two tags per image):
>> root@mailer:/home/mailman/docker/docker-mailman_mods/core# docker images
>> REPOSITORY TAG IMAGE ID CREATED
>> SIZE
>> local/mailman_core_900 20171110_2 9649e84767e1 2 days
>> ago 176MB
>> local/mailman_core_900 latest 9649e84767e1 2 days
>> ago 176MB
>> local/mailman_web_900 20171110_2 07a9b3d7ddd6 2 days
>> ago 247MB
>> local/mailman_web_900 latest 07a9b3d7ddd6 2 days
>> ago 247MB
>>
>> I do the same with the web image, as I need to change the user there, too.
>>
>> Then, in docker-compose.yaml, I change the line(s) referring to the image
>> name(s):
>>
>> services:
>> mailman-core:
>> image: local/mailman_core_900
>> container_name: mailman-core
>> hostname: mailman-core
>>
>>
>> mailman-web:
>> image: local/mailman_web_900
>> container_name: mailman-web
>> hostname: mailman-web
>>
>> I don't know if this is a good solution, but at least it fixes some
>> serious issues with the 3.1's postfix integration that wouldn't otherwise
>> be fixed until the 3.2 release. The bottom line is that it works for me,
>> but it adds an additional layer of complication.
>>
>> 2017-11-03 19:40 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
>>
>> On Fri, Nov 3, 2017, at 08:29 AM, Fabian A. Santiago wrote:
>> > October 26, 2017 11:07 PM, "Mark Sapiro" <mark(a)msapiro.net> wrote:
>> >
>> > > On October 26, 2017 7:30:35 PM PDT, "Fabian A. Santiago" <
>> fsantiago(a)garbage-juice.com> wrote:
>> > >
>> > >> That was it. Perfect. I manually modified my regexp map and it works
>> > >> now. Excellent and Thank you. You're the man. Does mm3 ever refresh
>> > >> those maps or only as I add new domains / lists?
>> > >
>> > > Only when you make changes to domains or lists.
>> > >
>> > > --
>> > > Mark Sapiro <mark(a)msapiro.net>
>> > > Sent from my Not_an_iThing with standards compliant, open source
>> software.
>> >
>> > Mark,
>> >
>> > I've noticed that even simply restarting the mm3 components those alias
>> > maps get rewritten and the problem returns until I can manually edit it.
>>
>> Yeah, that is true. Transport maps are re-generated everytime the
>> container restarts.
>>
>> I don't think think there is any way around this right now :(
>>
>>
>> --
>> Abhilash Raj
>> maxking(a)asynchronous.in
>> _______________________________________________
>> Mailman-users mailing list
>> mailman-users(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
>>
>>
8 years, 10 months
[MM3-users] Re: Errors while importing mm2 list archives
by Odhiambo Washington
On Thu, Dec 8, 2022 at 10:41 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/8/22 07:23, Odhiambo Washington wrote:
> > MM3-3.3.7, Hyperkitty-1.3.6
> >
> > While importing, I encountered several of this error:
> >
> > <CUT>
> > MySQLdb.OperationalError: (1366, "Incorrect string value:
> > '\\xF0\\x9F\\x98\\x8A, ...' for column 'content' at row 1")
> > The above exception was the direct cause of the following exception:
> ...
> > File
> >
> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/mysql/base.py",
>
>
> That's a 4-byte utf-8 encoding for an emoji. MySQL and MariaDB have to be
> told to accept 4-byte encodings.
>
> In the DATABASES['default'] definition, you need to set `'OPTIONS':
> {'charset': 'utf8mb4'}`
>
I added that:
[database]
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://mailman_user:XXXXXXXXX@localhost
/mailmansuite?charset=utf8mb4&use_unicode=1
> Also, although it doesn't affect this, see
>
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/datab…
I read this too. Did not pick much from there.
> You might also need to modify the tables, particularly hyperkitty_email
> with something like
> ```
> ALTER TABLE hyperkitty_email
> DEFAULT CHARACTER SET utf8mb4,
> MODIFY subject varchar(512)
> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
> MODIFY content
> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
>
So I removed my list, dropped the database, and proceeded as follows:
create database mailmansuite CHARACTER SET utf8mb4 COLLATE
utf8mb4_0900_ai_ci;
ALTER TABLE hyperkitty_email DEFAULT CHARACTER SET utf8mb4;
ALTER TABLE hyperkitty_email MODIFY subject varchar(512) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY content longtext CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY sender_name varchar(255) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY in_reply_to varchar(255) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
I did all those (although looks overzealous) just to be on the safe side. I
have seen people putting emojis in their names.
After that, I did the mailman-web migrate, created the list, imported the
config from MM2 and then proceeded to import the archives.
However, the errors still are there, and several messages were not imported
- I have several "failed to import" warnings.
<CUT>
django.db.utils.OperationalError: (1366, "Incorrect string value:
'\\xF0\\x9F\\x98\\x8A, ...' for column 'content' at row 1")
Message 253d9d44-cc3e-fed1-144b-34da88c52b39(a)emailplus.org failed to
import, skipping *<================== several of these "failed to import"
messages*
Computing thread structure
Synchronizing properties with Mailman
905 emails left to refresh, checked 0
765 emails left to refresh, checked 0
Warming up cache
Traceback (most recent call last):
File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module>
sys.exit(execute_from_command_line())
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_import.py",
line 411, in handle
call_command("hyperkitty_warm_up_cache", list_address)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 198, in call_command
return command.execute(*args, **defaults)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_warm_up_cache.py",
line 53, in handle
self.warm_up_mlist(mlist, options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_warm_up_cache.py",
line 61, in warm_up_mlist
for thread in mlist.recent_threads:
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 162, in recent_threads
return self.cached_values["recent_threads"]()
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/common.py",
line 58, in __call__
return self.get_or_set(*args, **kwargs)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 317, in get_or_set
return [Thread.objects.get(pk=pk) for pk in thread_ids]
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 317, in <listcomp>
return [Thread.objects.get(pk=pk) for pk in thread_ids]
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py",
line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py",
line 650, in get
raise self.model.DoesNotExist(
hyperkitty.models.thread.Thread.DoesNotExist: Thread matching query does
not exist.
(venv) [mailman@gw ~/mm]$
It therefore means I have several broken threads in the archives.
I am still willing to try other solutions if they are suggested.
And what do I make of the final error -
*hyperkitty.models.thread.Thread.DoesNotExist:
Thread matching query does not exist*. ??
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
3 years, 9 months
[MM3-users] Re: Docker Support Sucks
by Brian Carpenter
On 7/21/20 8:55 PM, Mark Sapiro wrote:
> On 7/21/20 5:40 PM, Brian Carpenter wrote:
>
> Abhilash (@maxking) is one person and he has a $DayJob. We're all doing
> the best we can.
Well, I still need support.
> This is not a Docker issue per se. It is a consequence of upgrading
> Mailman core to 3.3.1+ and processing many old, stale, previously
> unprocessed bounces. See
> <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…>
I am not worried about the bounce issues at this point. I am just trying
to get a membership list exported for a client.
>
>> So, I want to totally move away from the use of Docker (mainly because I have found it buggy and the support is simply lacking) and install Mailman 3 for this client from source. However I need to get an export of their list members. But of course, I can't. Rest API times out and I don't know how to apply the instructions I was given to fix it and I don't have days to way for one. So please help.
>>
>> I increased the gunicorn workers to 4, hoping that would allow me to do the CSV export. No. So was given these instructions:
>>
>> ###
>> The issues seems to be due to timeout not due to high load. You want to increase the timeout for gunicorn and you can do that by specifying a gunicorn.ini config file:
>>
>> [webservice]
>> configuration: /path/to/gunicorn.cfg
>>
>> and then in the gunicorn.cfg you can add
>>
>> [gunicorn]
>> timeout = 90
>> graceful_timeout = 30
>> ###
>>
>> I applied the above changes to /opt/mailman/core/mailman-extra.cfg and created the gunicorn.cfg file. Restarted the docker container (mailman-core) and no access to Postorius due to no REST api.
>>
>> After restarting docker-mailman, I can't access Postorius due to the following error:
>>
>> ERROR 2020-07-20 20:16:27,351 15 postorius Mailman REST API not available
> Is Mailman core running? If not, whats in core's mailman.log as to why not?
It looks like but no error message in mailman.log. However I noted that
the gunicorn process did not start.
This is with the configuration line enabled in mailman-extra.cfg:
# docker-compose restart mailman-core
Restarting mailman-core ... done
[root@nicu-net docker-mailman]# tail -f
/opt/mailman/core/var/logs/mailman.log
Jul 22 01:25:17 2020 (22) in runner caught SIGTERM. Stopping.
Jul 22 01:25:17 2020 (25) out runner caught SIGTERM. Stopping.
Jul 22 01:25:17 2020 (25) out runner exiting.
Jul 22 01:25:17 2020 (21) command runner exiting.
Jul 22 01:25:17 2020 (22) in runner exiting.
Jul 22 01:25:17 2020 (24) nntp runner caught SIGTERM. Stopping.
Jul 22 01:25:17 2020 (24) nntp runner exiting.
[2020-07-22 01:25:19 +0000] [27] [INFO] Shutting down: Master
Jul 22 01:25:21 2020 (1) Master stopped
Jul 22 01:25:26 2020 (1) Master started
Jul 22 01:25:44 2020 (27) rest runner started.
Jul 22 01:25:44 2020 (26) pipeline runner started.
Jul 22 01:25:44 2020 (20) bounces runner started.
Jul 22 01:25:45 2020 (22) in runner started.
Jul 22 01:25:45 2020 (28) retry runner started.
Jul 22 01:25:46 2020 (29) virgin runner started.
Jul 22 01:25:46 2020 (25) out runner started.
Jul 22 01:25:46 2020 (23) lmtp runner started.
Jul 22 01:25:46 2020 (24) nntp runner started.
Jul 22 01:25:47 2020 (30) digest runner started.
Jul 22 01:25:47 2020 (19) archive runner started.
Jul 22 01:25:47 2020 (21) command runner started.
This is with the configuration line disabled in mailman-extra.cfg:
# docker-compose restart mailman-core
Restarting mailman-core ... done
[root@nicu-net docker-mailman]# tail -f
/opt/mailman/core/var/logs/mailman.log
Jul 22 01:27:49 2020 (1) Master stopped
Jul 22 01:27:53 2020 (1) Master started
Jul 22 01:28:10 2020 (30) digest runner started.
Jul 22 01:28:10 2020 (29) virgin runner started.
Jul 22 01:28:10 2020 (24) nntp runner started.
Jul 22 01:28:10 2020 (25) out runner started.
Jul 22 01:28:10 2020 (22) in runner started.
Jul 22 01:28:11 2020 (28) retry runner started.
Jul 22 01:28:11 2020 (27) rest runner started.
[2020-07-22 01:28:11 +0000] [27] [INFO] Starting gunicorn 19.9.0
[2020-07-22 01:28:11 +0000] [27] [INFO] Listening at:
http://172.19.199.2:8001 (27)
[2020-07-22 01:28:11 +0000] [27] [INFO] Using worker: sync
[2020-07-22 01:28:11 +0000] [32] [INFO] Booting worker with pid: 32
[2020-07-22 01:28:11 +0000] [33] [INFO] Booting worker with pid: 33
[2020-07-22 01:28:11 +0000] [34] [INFO] Booting worker with pid: 34
[2020-07-22 01:28:11 +0000] [35] [INFO] Booting worker with pid: 35
Jul 22 01:28:11 2020 (19) archive runner started.
Jul 22 01:28:11 2020 (21) command runner started.
Jul 22 01:28:12 2020 (23) lmtp runner started.
Jul 22 01:28:12 2020 (20) bounces runner started.
Jul 22 01:28:12 2020 (26) pipeline runner started.
--
Please let me know if you need further assistance.
Thank you for your business. We appreciate our clients.
Brian Carpenter
EMWD.com
--
EMWD's Knowledgebase:
https://clientarea.emwd.com/index.php/knowledgebase
EMWD's Community Forums
http://discourse.emwd.com/
6 years, 1 month
[MM3-users] Re: django-allauth failures, but only for some users (independent of browser?)
by Jered Floyd
FYI, The django-allauth author has proposed fixing both of these with an even simpler change, which is now upstream and queued for a future release.
--Jered
--- a/allauth/socialaccount/adapter.py
+++ b/allauth/socialaccount/adapter.py
@@ -228,7 +228,10 @@ class DefaultSocialAccountAdapter(object):
provider_to_apps = {}
# First, populate it with the DB backed apps.
- db_apps = SocialApp.objects.on_site(request)
+ if request:
+ db_apps = SocialApp.objects.on_site(request)
+ else:
+ db_apps = SocialApp.objects.all()
if provider:
db_apps = db_apps.filter(Q(provider=provider) | Q(provider_id=provider))
if client_id:
----- On Dec 7, 2023, at 3:14 PM, Jered Floyd jered(a)convivian.com wrote:
> This was a related issue. See
> https://github.com/pennersr/django-allauth/pull/3554
>
> --Jered
>
>
> --- a/allauth/socialaccount/models.py
> +++ b/allauth/socialaccount/models.py
> @@ -144,7 +144,7 @@ class SocialAccount(models.Model):
> return provider
>
> def get_provider_account(self):
> - return self.get_provider().wrap_account(self)
> + return self.get_provider(context.request).wrap_account(self)
>
>
> class SocialToken(models.Model):
>
>
>
> ----- On Dec 6, 2023, at 7:57 PM, Mark Sapiro mark(a)msapiro.net wrote:
>
>> On 12/6/23 14:49, Jered Floyd wrote:
>>>
>>> The difference in behavior was using social login buttons for an existing
>>> account vs. creating a new account.
>>>
>>> This is another damn django-allauth regression, starting in 0.57.0. It seem to
>>> be this commit that is to blame:
>>> https://github.com/pennersr/django-allauth/commit/be779dfee5a328a3a42edc2c9…
>>>
>>> In allauth/socialaccount/models.py:lookup(self):285 there is an attempt to look
>>> up the social account as one that already exists. If that fails, this commit
>>> calls self.account.get_provider() without passing the request object, which
>>> leads to the eventual error.
>>>
>>> I believe this is a django-allauth bug, and have submitted a pull request to fix
>>> it here:
>>> https://github.com/pennersr/django-allauth/pull/3548
>>
>> Thanks for all your work on this. It is much appreciated. However, your
>> patch doesn't fix my issue with the URL
>> https://example.com/accounts/social/connections/ which still produces
>> the following:
>> ```
>> ERROR 2023-12-07 00:05:55,032 169452 django.request Internal Server
>> Error: /accounts/social/connections/
>> Traceback (most recent call last):
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/exception.py",
>> line 55, in inner
>> response = get_response(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py",
>> line 220, in _get_response
>> response = response.render()
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/response.py",
>> line 114, in render
>> self.content = self.rendered_content
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/response.py",
>> line 92, in rendered_content
>> return template.render(context, self._request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py",
>> line 61, in render
>> return self.template.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 175, in render
>> return self._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 157, in render
>> return compiled_parent._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 157, in render
>> return compiled_parent._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 157, in render
>> return compiled_parent._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 63, in render
>> result = block.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 63, in render
>> result = block.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 321, in render
>> return nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 238, in render
>> nodelist.append(node.render_annotated(context))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 539, in render
>> values = {key: val.resolve(context) for key, val in
>> self.extra_context.items()}
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 539, in <dictcomp>
>> values = {key: val.resolve(context) for key, val in
>> self.extra_context.items()}
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 715, in resolve
>> obj = self.var.resolve(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 847, in resolve
>> value = self._resolve_lookup(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 914, in _resolve_lookup
>> current = current()
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py",
>> line 147, in get_provider_account
>> return self.get_provider().wrap_account(self)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py",
>> line 141, in get_provider
>> provider = self._provider = adapter.get_provider(
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py",
>> line 204, in get_provider
>> app = self.get_app(request, provider=provider)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py",
>> line 284, in get_app
>> apps = self.list_apps(request, provider=provider, client_id=client_id)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py",
>> line 231, in list_apps
>> db_apps = SocialApp.objects.on_site(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py",
>> line 27, in on_site
>> site = get_current_site(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/contrib/sites/shortcuts.py",
>> line 16, in get_current_site
>> return Site.objects.get_current(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/contrib/sites/models.py",
>> line 63, in get_current
>> raise ImproperlyConfigured(
>> django.core.exceptions.ImproperlyConfigured: You're using the Django
>> "sites framework" without having set the SITE_ID setting. Create a site
>> in your database and set the SITE_ID setting or pass a request to
>> Site.objects.get_current() to fix this error.
>> ```
>>
>> --
>> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>> San Francisco Bay Area, California better use your sense - B. Dylan
>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent to jered(a)convivian.com
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to jered(a)convivian.com
2 years, 9 months
[MM3-users] Re: Huge performance issue: Mailman doesn't (seem to) receive any mails.
by Henrik Rasmussen
Mark Sapiro wrote:
> This seems to say that the Postfix container can connect to
> 172.19.199.2:8024 and receive the greeting from Mailman's lmtp runner.
> So, why can't Postfix send to 172.19.199.2:8024?
That is why I figured that the problem could be Mailman bloking for more mails, somhow.
> The only log entry you
> postes just says "Sep 18 10:07:47 postfix postfix/qmgr[98]: warning:
> mail for [172.19.199.2]:8024 is using up 7844 of 7844 active queue entries".
> What other log entries relate to attempts to send to 172.19.199.2:8024?
> You may have to go back to when this first started to see them.
Docker deleted it's log when restaring, as the container is deleted and removed, but the starte of the current log says:
```
Sep 18 13:28:57 postfix rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="20" x-info="http://www.rsyslog.com"] start
Sep 18 13:29:00 postfix postfix/postfix-script[94]: starting the Postfix mail system
Sep 18 13:29:00 postfix postfix/master[96]: daemon started -- version 2.10.1, configuration /etc/postfix
Sep 18 13:29:02 postfix postfix/smtpd[102]: connect from incoming.gateway.example.com[192.168.111.222]
Sep 18 13:29:04 postfix postfix/smtpd[102]: EFF017E9D65AE: client=incoming.gateway.example.com[192.168.111.222]
Sep 18 13:29:04 postfix postfix/cleanup[106]: EFF017E9D65AE: message-id=<2E40950170CA0A4A8D0F0C0CF1C0630601B8B9F412(a)P1KITMBX03WC02.unicph.domain>
Sep 18 13:29:04 postfix opendkim[19]: EFF017E9D65AE: incoming.gateway.example.com [192.168.111.222] not internal
Sep 18 13:29:04 postfix opendkim[19]: EFF017E9D65AE: not authenticated
Sep 18 13:29:04 postfix postfix/smtpd[102]: disconnect from incoming.gateway.example.com[192.168.111.222]
Sep 18 13:29:18 postfix postfix/qmgr[98]: 0E53F7E9D7D8D: from=<>, size=212010, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: C51DE7E9920B8: from=<>, size=212004, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 2923A7E99B41F: from=<>, size=212040, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: D0A1E7E981E26: from=<quarantine(a)messaging.microsoft.com>, size=14542, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 95FA77E9815A5: from=<>, size=211977, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 208777E993635: from=<sender(a)example.com>, size=9826184, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 757C27E9BD45C: from=<>, size=211997, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 738C87E9B94BE: from=<>, size=212141, nrcpt=1 (queue active)
[SLIP]
Sep 18 13:32:24 postfix postfix/anvil[103]: statistics: max cache size 1 at Sep 18 13:29:02
Sep 18 13:43:48 postfix postfix/smtpd[992]: connect from gateway[172.19.199.1]
Sep 18 13:43:48 postfix postfix/smtpd[992]: 9E0D37E9D65AF: client=gateway[172.19.199.1]
Sep 18 13:43:48 postfix postfix/cleanup[996]: 9E0D37E9D65AF: message-id=<3e08121bcdf5b0ebeb779a26ab2fe6ef(a)mailing.einnews.com>
Sep 18 13:43:48 postfix opendkim[19]: 9E0D37E9D65AF: DKIM-Signature field added (s=default, d=sendergateway.example.com)
Sep 18 13:43:48 postfix postfix/smtpd[992]: disconnect from gateway[172.19.199.1]
Sep 18 13:43:48 postfix postfix/qmgr[98]: 9E0D37E9D65AF: from=<sender(a)example.com>, size=59393, nrcpt=1 (queue active)
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: mail for [172.19.199.2]:8024 is using up 7863 of 7863 active queue entries
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: you may need to reduce lmtp connect and helo timeouts
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: so that Postfix quickly skips unavailable hosts
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: you may need to increase the main.cf minimal_backoff_time and maximal_backoff_time
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: so that Postfix wastes less time on undeliverable mail
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: you may need to increase the master.cf lmtp process limit
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: please avoid flushing the whole queue when you have
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: lots of deferred mail, that is bad for performance
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: to turn off these warnings specify: qmgr_clog_warn_time = 0
```
We see the queueing in Postfix as Mailman is not receiving and that this is why Postfix is using up all it's active queue entries.
However, suddenly this morning, everything came alive all by itself. The question left is: What held the flow. If we don't know what held it, it may happen again, so I would really like some help to understand the problem.
```
Sep 20 07:05:23 postfix opendkim[19]: 23AED7E98F748: incoming.gateway.example.com [192.168.111.222] not internal
Sep 20 07:05:23 postfix opendkim[19]: 23AED7E98F748: not authenticated
Sep 20 07:05:23 postfix postfix/smtpd[20713]: disconnect from incoming.gateway.example.com[192.168.111.222]
Sep 20 07:05:23 postfix postfix/qmgr[16738]: 23AED7E98F748: from=<someone(a)foo.bar>, size=4379, nrcpt=1 (queue active)
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: mail for [172.19.199.2]:8024 is using up 5363 of 5364 active queue entries
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: you may need to increase the main.cf lmtp_destination_concurrency_limit from 20
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: please avoid flushing the whole queue when you have
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: lots of deferred mail, that is bad for performance
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: to turn off these warnings specify: qmgr_clog_warn_time = 0
Sep 20 07:05:23 postfix postfix/lmtp[20393]: B64F97E9D0BF4: to=<LISTNAME-bounces(a)lists.example.com>, relay=172.19.199.2[172.19.199.2]:8024, conn_use=4, delay=663077, delays=659443/3624/1/8.1, dsn=2.0.0, status=sent (250 Ok)
Sep 20 07:05:23 postfix postfix/qmgr[16738]: B64F97E9D0BF4: removed
```
/ Henrik Rasmussen
6 years, 11 months