Search results for query "sapiro"
- 6045 messages
[MM3-users] Re: Attachement processings questions
by Lars Schimmer
On 2020-02-03 20:24, Mark Sapiro wrote:
> On 2/3/20 4:27 AM, Lars Schimmer wrote:
>>
>> In a mailinglist it seems some mails are not forwarded to a member, but
>> is listed in archive in hyperkitty.
>> That mail did include a PDF attachement, but HyperKitty does not list
>> any attachement.
>
>
> There are two issues here. Why wasn't the message delivered to list
> members?, and why is there no attachment in Hyperkitty?
>
> For the first issue, you need to examine mailman.log for the time of the
> post looking for errors and tracebacks. Also, the MTA logs may have
> useful info if the messages were sent but not delivered.
Thank you so far, I maye found the issue:
Uncaught runner exception: 'ascii' codec can't encode character '\xe9'
in position 109: ordinal not in range(128)
Uncaught runner exception: 'ascii' codec can't encode character '\xa0'
in position 37: ordinal not in range(128)
Seems to be the problemm, as it happens the same moment as the
problematic email arrives in exim4:
2020-01-31 03:02:41 1ixLdp-0003Mo-5z => user(a)abc.de R=mailman3_router
T=mailman3_transport H=localhost [127.0.0.1] C="250 Ok"
2020-01-31 03:02:41 1ixLdp-0003Mo-5z Completed
and nothing more. No sign of mailman3 sending out any mail via this list.
Other emails getting to that list will be send out via user-bounces(a)abc.de.
Those mails are in the mailinglist archives, not on hold or anything.
Just the attachement missing.
So, please, howto fix the ascii encoding error in mailman3?
Full trace:
Jan 31 03:02:43 2020 (32591) Uncaught runner exception: 'ascii' codec
can't encode character '\xe9' in position 109: ordinal not in range(128)
Jan 31 03:02:43 2020 (32591) Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line
173, in _one_iteration
self._process_one_file(msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line
266, in _process_one_file
keepqueued = self._dispose(mlist, msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/runners/pipeline.py",
line 37, in _dispose
process(mlist, msg, msgdata, pipeline)
File "/usr/lib/python3/dist-packages/mailman/core/pipelines.py", line
50, in process
handler.process(mlist, msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/handlers/to_digest.py",
line 48, in process
mbox.add(msg)
File "/usr/lib/python3.7/mailbox.py", line 603, in add
self._toc[self._next_key] = self._append_message(message)
File "/usr/lib/python3.7/mailbox.py", line 758, in _append_message
offsets = self._install_message(message)
File "/usr/lib/python3.7/mailbox.py", line 830, in _install_message
self._dump_message(message, self._file, self._mangle_from_)
File "/usr/lib/python3.7/mailbox.py", line 215, in _dump_message
gen.flatten(message)
File "/usr/lib/python3.7/email/generator.py", line 116, in flatten
self._write(msg)
File "/usr/lib/python3.7/email/generator.py", line 181, in _write
self._dispatch(msg)
File "/usr/lib/python3.7/email/generator.py", line 214, in _dispatch
meth(msg)
File "/usr/lib/python3.7/email/generator.py", line 432, in _handle_text
super(BytesGenerator,self)._handle_text(msg)
File "/usr/lib/python3.7/email/generator.py", line 249, in _handle_text
self._write_lines(payload)
File "/usr/lib/python3.7/email/generator.py", line 155, in _write_lines
self.write(line)
File "/usr/lib/python3.7/email/generator.py", line 406, in write
self._fp.write(s.encode('ascii', 'surrogateescape'))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in
position 109: ordinal not in range(128)
> For the second, there are many content filtering settings not (yet)
> exposed in Postorius. These include filter_extensions, filter_types,
> pass_extensions and pass_types which control which MIME parts are
> removed. For a newly created Mailman 3 list, these four settings are
> empty, but importing a Mailman 2.1 list will set them according to the
> corresponding MM 2.1 list's settings.
>
>
>> All I found was "convert HTML to plaintext", but does this setting
>> remove attachements?
>
>
> It would only affect HTML parts, not pdfs.
>
>
>> Is there any generel documentation about attachement processing in MM3?
>
>
> <https://mailman.readthedocs.io/en/latest/src/mailman/handlers/docs/filterin…>.
>
>
>
>> I would like this mailinglist to accept mails with attachements, send it
>> to the members and archvie the attachments in HyperKitty, to.
>
>
> You can set Settings -> Alter Messages -> Filter content to No in
> Postorius to not do any content filtering at all. This will override the
> Collapse alternatives and Convert html to plaintext settings and pass
> the message without removing any MIME parts.
>
> But, I think you must have other issues, either with this mail in
> particular or in general as evidenced by the mail not being received by
> list members.
Ok, first fix the ascii encode error for me. Any idea how?
thank you.
MfG,
Lars Schimmer
--
-------------------------------------------------------------
TU Graz, Institut für ComputerGraphik & WissensVisualisierung
Tel: +43 316 873-5405 E-Mail: l.schimmer(a)cgv.tugraz.at
Fax: +43 316 873-5402 PGP-Key-ID: 0x4A9B1723
5 years, 11 months
[MM3-users] Re: integrating mm3 with postfix / lmtp
by Thor Atle Rustad
I will try to get a grip on uid namespaces then...
2017-11-13 10:49 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
> On Nov 13 2017, at 1:34 am, Thor Atle Rustad <thor.rustad(a)gmail.com>
> wrote:
>
>> There is a way around it!
>>
>> I have had two issues with the the maxking docker image. One is that the
>> regexp is not working properly. I reported that, and it has been fixed in
>> newer code. My other problem is that the docker image creates a user,
>> mailman, that receives uid 103. Well, uid 103 on my system is already taken
>> by systemd-bus-proxy (grep 103 /etc/passwd returns
>> "systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/
>> false").
>>
>
> Containers usually run with uid namespace, so it doesn't really matter
> what uid is used outside of container.
>
> Unless, you mount /etc/shadow from host in to the container, which isn't
> really needed for the images.
>
>
>> My solution includes downloading the corrected postfix.py, and replacing
>> the Dockerfile. I put the postfix.py in <docker-mailman>/core/assets/.
>>
>> My Dockerfile:
>> FROM maxking/mailman-core
>>
>> RUN grep mailman /etc/passwd && grep mailman /etc/group \
>> && deluser mailman \
>> && addgroup -S -g900 mailman \
>> && adduser -S -u900 mailman mailman \
>> && grep mailman /etc/passwd && grep mailman /etc/group
>> COPY assets/postfix.py /usr/local/lib/python3.6/site-
>> packages/mailman/mta/postfix.py
>>
>> I then run docker build (with -t parameter, you must look up that
>> yourself). I use a different name for my images, so I end up with (note,
>> there are two tags per image):
>> root@mailer:/home/mailman/docker/docker-mailman_mods/core# docker images
>> REPOSITORY TAG IMAGE ID CREATED
>> SIZE
>> local/mailman_core_900 20171110_2 9649e84767e1 2 days
>> ago 176MB
>> local/mailman_core_900 latest 9649e84767e1 2 days
>> ago 176MB
>> local/mailman_web_900 20171110_2 07a9b3d7ddd6 2 days
>> ago 247MB
>> local/mailman_web_900 latest 07a9b3d7ddd6 2 days
>> ago 247MB
>>
>> I do the same with the web image, as I need to change the user there, too.
>>
>> Then, in docker-compose.yaml, I change the line(s) referring to the image
>> name(s):
>>
>> services:
>> mailman-core:
>> image: local/mailman_core_900
>> container_name: mailman-core
>> hostname: mailman-core
>>
>>
>> mailman-web:
>> image: local/mailman_web_900
>> container_name: mailman-web
>> hostname: mailman-web
>>
>> I don't know if this is a good solution, but at least it fixes some
>> serious issues with the 3.1's postfix integration that wouldn't otherwise
>> be fixed until the 3.2 release. The bottom line is that it works for me,
>> but it adds an additional layer of complication.
>>
>> 2017-11-03 19:40 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
>>
>> On Fri, Nov 3, 2017, at 08:29 AM, Fabian A. Santiago wrote:
>> > October 26, 2017 11:07 PM, "Mark Sapiro" <mark(a)msapiro.net> wrote:
>> >
>> > > On October 26, 2017 7:30:35 PM PDT, "Fabian A. Santiago" <
>> fsantiago(a)garbage-juice.com> wrote:
>> > >
>> > >> That was it. Perfect. I manually modified my regexp map and it works
>> > >> now. Excellent and Thank you. You're the man. Does mm3 ever refresh
>> > >> those maps or only as I add new domains / lists?
>> > >
>> > > Only when you make changes to domains or lists.
>> > >
>> > > --
>> > > Mark Sapiro <mark(a)msapiro.net>
>> > > Sent from my Not_an_iThing with standards compliant, open source
>> software.
>> >
>> > Mark,
>> >
>> > I've noticed that even simply restarting the mm3 components those alias
>> > maps get rewritten and the problem returns until I can manually edit it.
>>
>> Yeah, that is true. Transport maps are re-generated everytime the
>> container restarts.
>>
>> I don't think think there is any way around this right now :(
>>
>>
>> --
>> Abhilash Raj
>> maxking(a)asynchronous.in
>> _______________________________________________
>> Mailman-users mailing list
>> mailman-users(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
>>
>>
8 years, 2 months
[MM3-users] Re: Errors while importing mm2 list archives
by Odhiambo Washington
On Thu, Dec 8, 2022 at 10:41 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/8/22 07:23, Odhiambo Washington wrote:
> > MM3-3.3.7, Hyperkitty-1.3.6
> >
> > While importing, I encountered several of this error:
> >
> > <CUT>
> > MySQLdb.OperationalError: (1366, "Incorrect string value:
> > '\\xF0\\x9F\\x98\\x8A, ...' for column 'content' at row 1")
> > The above exception was the direct cause of the following exception:
> ...
> > File
> >
> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/mysql/base.py",
>
>
> That's a 4-byte utf-8 encoding for an emoji. MySQL and MariaDB have to be
> told to accept 4-byte encodings.
>
> In the DATABASES['default'] definition, you need to set `'OPTIONS':
> {'charset': 'utf8mb4'}`
>
I added that:
[database]
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://mailman_user:XXXXXXXXX@localhost
/mailmansuite?charset=utf8mb4&use_unicode=1
> Also, although it doesn't affect this, see
>
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/datab…
I read this too. Did not pick much from there.
> You might also need to modify the tables, particularly hyperkitty_email
> with something like
> ```
> ALTER TABLE hyperkitty_email
> DEFAULT CHARACTER SET utf8mb4,
> MODIFY subject varchar(512)
> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
> MODIFY content
> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
>
So I removed my list, dropped the database, and proceeded as follows:
create database mailmansuite CHARACTER SET utf8mb4 COLLATE
utf8mb4_0900_ai_ci;
ALTER TABLE hyperkitty_email DEFAULT CHARACTER SET utf8mb4;
ALTER TABLE hyperkitty_email MODIFY subject varchar(512) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY content longtext CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY sender_name varchar(255) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY in_reply_to varchar(255) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
I did all those (although looks overzealous) just to be on the safe side. I
have seen people putting emojis in their names.
After that, I did the mailman-web migrate, created the list, imported the
config from MM2 and then proceeded to import the archives.
However, the errors still are there, and several messages were not imported
- I have several "failed to import" warnings.
<CUT>
django.db.utils.OperationalError: (1366, "Incorrect string value:
'\\xF0\\x9F\\x98\\x8A, ...' for column 'content' at row 1")
Message 253d9d44-cc3e-fed1-144b-34da88c52b39(a)emailplus.org failed to
import, skipping *<================== several of these "failed to import"
messages*
Computing thread structure
Synchronizing properties with Mailman
905 emails left to refresh, checked 0
765 emails left to refresh, checked 0
Warming up cache
Traceback (most recent call last):
File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module>
sys.exit(execute_from_command_line())
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_import.py",
line 411, in handle
call_command("hyperkitty_warm_up_cache", list_address)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 198, in call_command
return command.execute(*args, **defaults)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_warm_up_cache.py",
line 53, in handle
self.warm_up_mlist(mlist, options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_warm_up_cache.py",
line 61, in warm_up_mlist
for thread in mlist.recent_threads:
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 162, in recent_threads
return self.cached_values["recent_threads"]()
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/common.py",
line 58, in __call__
return self.get_or_set(*args, **kwargs)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 317, in get_or_set
return [Thread.objects.get(pk=pk) for pk in thread_ids]
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 317, in <listcomp>
return [Thread.objects.get(pk=pk) for pk in thread_ids]
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py",
line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py",
line 650, in get
raise self.model.DoesNotExist(
hyperkitty.models.thread.Thread.DoesNotExist: Thread matching query does
not exist.
(venv) [mailman@gw ~/mm]$
It therefore means I have several broken threads in the archives.
I am still willing to try other solutions if they are suggested.
And what do I make of the final error -
*hyperkitty.models.thread.Thread.DoesNotExist:
Thread matching query does not exist*. ??
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
3 years, 1 month
[MM3-users] Re: Docker Support Sucks
by Brian Carpenter
On 7/21/20 8:55 PM, Mark Sapiro wrote:
> On 7/21/20 5:40 PM, Brian Carpenter wrote:
>
> Abhilash (@maxking) is one person and he has a $DayJob. We're all doing
> the best we can.
Well, I still need support.
> This is not a Docker issue per se. It is a consequence of upgrading
> Mailman core to 3.3.1+ and processing many old, stale, previously
> unprocessed bounces. See
> <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…>
I am not worried about the bounce issues at this point. I am just trying
to get a membership list exported for a client.
>
>> So, I want to totally move away from the use of Docker (mainly because I have found it buggy and the support is simply lacking) and install Mailman 3 for this client from source. However I need to get an export of their list members. But of course, I can't. Rest API times out and I don't know how to apply the instructions I was given to fix it and I don't have days to way for one. So please help.
>>
>> I increased the gunicorn workers to 4, hoping that would allow me to do the CSV export. No. So was given these instructions:
>>
>> ###
>> The issues seems to be due to timeout not due to high load. You want to increase the timeout for gunicorn and you can do that by specifying a gunicorn.ini config file:
>>
>> [webservice]
>> configuration: /path/to/gunicorn.cfg
>>
>> and then in the gunicorn.cfg you can add
>>
>> [gunicorn]
>> timeout = 90
>> graceful_timeout = 30
>> ###
>>
>> I applied the above changes to /opt/mailman/core/mailman-extra.cfg and created the gunicorn.cfg file. Restarted the docker container (mailman-core) and no access to Postorius due to no REST api.
>>
>> After restarting docker-mailman, I can't access Postorius due to the following error:
>>
>> ERROR 2020-07-20 20:16:27,351 15 postorius Mailman REST API not available
> Is Mailman core running? If not, whats in core's mailman.log as to why not?
It looks like but no error message in mailman.log. However I noted that
the gunicorn process did not start.
This is with the configuration line enabled in mailman-extra.cfg:
# docker-compose restart mailman-core
Restarting mailman-core ... done
[root@nicu-net docker-mailman]# tail -f
/opt/mailman/core/var/logs/mailman.log
Jul 22 01:25:17 2020 (22) in runner caught SIGTERM. Stopping.
Jul 22 01:25:17 2020 (25) out runner caught SIGTERM. Stopping.
Jul 22 01:25:17 2020 (25) out runner exiting.
Jul 22 01:25:17 2020 (21) command runner exiting.
Jul 22 01:25:17 2020 (22) in runner exiting.
Jul 22 01:25:17 2020 (24) nntp runner caught SIGTERM. Stopping.
Jul 22 01:25:17 2020 (24) nntp runner exiting.
[2020-07-22 01:25:19 +0000] [27] [INFO] Shutting down: Master
Jul 22 01:25:21 2020 (1) Master stopped
Jul 22 01:25:26 2020 (1) Master started
Jul 22 01:25:44 2020 (27) rest runner started.
Jul 22 01:25:44 2020 (26) pipeline runner started.
Jul 22 01:25:44 2020 (20) bounces runner started.
Jul 22 01:25:45 2020 (22) in runner started.
Jul 22 01:25:45 2020 (28) retry runner started.
Jul 22 01:25:46 2020 (29) virgin runner started.
Jul 22 01:25:46 2020 (25) out runner started.
Jul 22 01:25:46 2020 (23) lmtp runner started.
Jul 22 01:25:46 2020 (24) nntp runner started.
Jul 22 01:25:47 2020 (30) digest runner started.
Jul 22 01:25:47 2020 (19) archive runner started.
Jul 22 01:25:47 2020 (21) command runner started.
This is with the configuration line disabled in mailman-extra.cfg:
# docker-compose restart mailman-core
Restarting mailman-core ... done
[root@nicu-net docker-mailman]# tail -f
/opt/mailman/core/var/logs/mailman.log
Jul 22 01:27:49 2020 (1) Master stopped
Jul 22 01:27:53 2020 (1) Master started
Jul 22 01:28:10 2020 (30) digest runner started.
Jul 22 01:28:10 2020 (29) virgin runner started.
Jul 22 01:28:10 2020 (24) nntp runner started.
Jul 22 01:28:10 2020 (25) out runner started.
Jul 22 01:28:10 2020 (22) in runner started.
Jul 22 01:28:11 2020 (28) retry runner started.
Jul 22 01:28:11 2020 (27) rest runner started.
[2020-07-22 01:28:11 +0000] [27] [INFO] Starting gunicorn 19.9.0
[2020-07-22 01:28:11 +0000] [27] [INFO] Listening at:
http://172.19.199.2:8001 (27)
[2020-07-22 01:28:11 +0000] [27] [INFO] Using worker: sync
[2020-07-22 01:28:11 +0000] [32] [INFO] Booting worker with pid: 32
[2020-07-22 01:28:11 +0000] [33] [INFO] Booting worker with pid: 33
[2020-07-22 01:28:11 +0000] [34] [INFO] Booting worker with pid: 34
[2020-07-22 01:28:11 +0000] [35] [INFO] Booting worker with pid: 35
Jul 22 01:28:11 2020 (19) archive runner started.
Jul 22 01:28:11 2020 (21) command runner started.
Jul 22 01:28:12 2020 (23) lmtp runner started.
Jul 22 01:28:12 2020 (20) bounces runner started.
Jul 22 01:28:12 2020 (26) pipeline runner started.
--
Please let me know if you need further assistance.
Thank you for your business. We appreciate our clients.
Brian Carpenter
EMWD.com
--
EMWD's Knowledgebase:
https://clientarea.emwd.com/index.php/knowledgebase
EMWD's Community Forums
http://discourse.emwd.com/
5 years, 5 months
[MM3-users] Re: django-allauth failures, but only for some users (independent of browser?)
by Jered Floyd
FYI, The django-allauth author has proposed fixing both of these with an even simpler change, which is now upstream and queued for a future release.
--Jered
--- a/allauth/socialaccount/adapter.py
+++ b/allauth/socialaccount/adapter.py
@@ -228,7 +228,10 @@ class DefaultSocialAccountAdapter(object):
provider_to_apps = {}
# First, populate it with the DB backed apps.
- db_apps = SocialApp.objects.on_site(request)
+ if request:
+ db_apps = SocialApp.objects.on_site(request)
+ else:
+ db_apps = SocialApp.objects.all()
if provider:
db_apps = db_apps.filter(Q(provider=provider) | Q(provider_id=provider))
if client_id:
----- On Dec 7, 2023, at 3:14 PM, Jered Floyd jered(a)convivian.com wrote:
> This was a related issue. See
> https://github.com/pennersr/django-allauth/pull/3554
>
> --Jered
>
>
> --- a/allauth/socialaccount/models.py
> +++ b/allauth/socialaccount/models.py
> @@ -144,7 +144,7 @@ class SocialAccount(models.Model):
> return provider
>
> def get_provider_account(self):
> - return self.get_provider().wrap_account(self)
> + return self.get_provider(context.request).wrap_account(self)
>
>
> class SocialToken(models.Model):
>
>
>
> ----- On Dec 6, 2023, at 7:57 PM, Mark Sapiro mark(a)msapiro.net wrote:
>
>> On 12/6/23 14:49, Jered Floyd wrote:
>>>
>>> The difference in behavior was using social login buttons for an existing
>>> account vs. creating a new account.
>>>
>>> This is another damn django-allauth regression, starting in 0.57.0. It seem to
>>> be this commit that is to blame:
>>> https://github.com/pennersr/django-allauth/commit/be779dfee5a328a3a42edc2c9…
>>>
>>> In allauth/socialaccount/models.py:lookup(self):285 there is an attempt to look
>>> up the social account as one that already exists. If that fails, this commit
>>> calls self.account.get_provider() without passing the request object, which
>>> leads to the eventual error.
>>>
>>> I believe this is a django-allauth bug, and have submitted a pull request to fix
>>> it here:
>>> https://github.com/pennersr/django-allauth/pull/3548
>>
>> Thanks for all your work on this. It is much appreciated. However, your
>> patch doesn't fix my issue with the URL
>> https://example.com/accounts/social/connections/ which still produces
>> the following:
>> ```
>> ERROR 2023-12-07 00:05:55,032 169452 django.request Internal Server
>> Error: /accounts/social/connections/
>> Traceback (most recent call last):
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/exception.py",
>> line 55, in inner
>> response = get_response(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/handlers/base.py",
>> line 220, in _get_response
>> response = response.render()
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/response.py",
>> line 114, in render
>> self.content = self.rendered_content
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/response.py",
>> line 92, in rendered_content
>> return template.render(context, self._request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/backends/django.py",
>> line 61, in render
>> return self.template.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 175, in render
>> return self._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 157, in render
>> return compiled_parent._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 157, in render
>> return compiled_parent._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 157, in render
>> return compiled_parent._render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 167, in _render
>> return self.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 63, in render
>> result = block.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/loader_tags.py",
>> line 63, in render
>> result = block.nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 321, in render
>> return nodelist.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in render
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 1005, in <listcomp>
>> return SafeString("".join([node.render_annotated(context) for node
>> in self]))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 238, in render
>> nodelist.append(node.render_annotated(context))
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 966, in render_annotated
>> return self.render(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 539, in render
>> values = {key: val.resolve(context) for key, val in
>> self.extra_context.items()}
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/defaulttags.py",
>> line 539, in <dictcomp>
>> values = {key: val.resolve(context) for key, val in
>> self.extra_context.items()}
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 715, in resolve
>> obj = self.var.resolve(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 847, in resolve
>> value = self._resolve_lookup(context)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/template/base.py",
>> line 914, in _resolve_lookup
>> current = current()
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py",
>> line 147, in get_provider_account
>> return self.get_provider().wrap_account(self)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py",
>> line 141, in get_provider
>> provider = self._provider = adapter.get_provider(
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py",
>> line 204, in get_provider
>> app = self.get_app(request, provider=provider)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py",
>> line 284, in get_app
>> apps = self.list_apps(request, provider=provider, client_id=client_id)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/adapter.py",
>> line 231, in list_apps
>> db_apps = SocialApp.objects.on_site(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/allauth/socialaccount/models.py",
>> line 27, in on_site
>> site = get_current_site(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/contrib/sites/shortcuts.py",
>> line 16, in get_current_site
>> return Site.objects.get_current(request)
>> File
>> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/contrib/sites/models.py",
>> line 63, in get_current
>> raise ImproperlyConfigured(
>> django.core.exceptions.ImproperlyConfigured: You're using the Django
>> "sites framework" without having set the SITE_ID setting. Create a site
>> in your database and set the SITE_ID setting or pass a request to
>> Site.objects.get_current() to fix this error.
>> ```
>>
>> --
>> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>> San Francisco Bay Area, California better use your sense - B. Dylan
>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent to jered(a)convivian.com
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to jered(a)convivian.com
2 years, 1 month
[MM3-users] Re: Huge performance issue: Mailman doesn't (seem to) receive any mails.
by Henrik Rasmussen
Mark Sapiro wrote:
> This seems to say that the Postfix container can connect to
> 172.19.199.2:8024 and receive the greeting from Mailman's lmtp runner.
> So, why can't Postfix send to 172.19.199.2:8024?
That is why I figured that the problem could be Mailman bloking for more mails, somhow.
> The only log entry you
> postes just says "Sep 18 10:07:47 postfix postfix/qmgr[98]: warning:
> mail for [172.19.199.2]:8024 is using up 7844 of 7844 active queue entries".
> What other log entries relate to attempts to send to 172.19.199.2:8024?
> You may have to go back to when this first started to see them.
Docker deleted it's log when restaring, as the container is deleted and removed, but the starte of the current log says:
```
Sep 18 13:28:57 postfix rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="20" x-info="http://www.rsyslog.com"] start
Sep 18 13:29:00 postfix postfix/postfix-script[94]: starting the Postfix mail system
Sep 18 13:29:00 postfix postfix/master[96]: daemon started -- version 2.10.1, configuration /etc/postfix
Sep 18 13:29:02 postfix postfix/smtpd[102]: connect from incoming.gateway.example.com[192.168.111.222]
Sep 18 13:29:04 postfix postfix/smtpd[102]: EFF017E9D65AE: client=incoming.gateway.example.com[192.168.111.222]
Sep 18 13:29:04 postfix postfix/cleanup[106]: EFF017E9D65AE: message-id=<2E40950170CA0A4A8D0F0C0CF1C0630601B8B9F412(a)P1KITMBX03WC02.unicph.domain>
Sep 18 13:29:04 postfix opendkim[19]: EFF017E9D65AE: incoming.gateway.example.com [192.168.111.222] not internal
Sep 18 13:29:04 postfix opendkim[19]: EFF017E9D65AE: not authenticated
Sep 18 13:29:04 postfix postfix/smtpd[102]: disconnect from incoming.gateway.example.com[192.168.111.222]
Sep 18 13:29:18 postfix postfix/qmgr[98]: 0E53F7E9D7D8D: from=<>, size=212010, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: C51DE7E9920B8: from=<>, size=212004, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 2923A7E99B41F: from=<>, size=212040, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: D0A1E7E981E26: from=<quarantine(a)messaging.microsoft.com>, size=14542, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 95FA77E9815A5: from=<>, size=211977, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 208777E993635: from=<sender(a)example.com>, size=9826184, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 757C27E9BD45C: from=<>, size=211997, nrcpt=1 (queue active)
Sep 18 13:29:18 postfix postfix/qmgr[98]: 738C87E9B94BE: from=<>, size=212141, nrcpt=1 (queue active)
[SLIP]
Sep 18 13:32:24 postfix postfix/anvil[103]: statistics: max cache size 1 at Sep 18 13:29:02
Sep 18 13:43:48 postfix postfix/smtpd[992]: connect from gateway[172.19.199.1]
Sep 18 13:43:48 postfix postfix/smtpd[992]: 9E0D37E9D65AF: client=gateway[172.19.199.1]
Sep 18 13:43:48 postfix postfix/cleanup[996]: 9E0D37E9D65AF: message-id=<3e08121bcdf5b0ebeb779a26ab2fe6ef(a)mailing.einnews.com>
Sep 18 13:43:48 postfix opendkim[19]: 9E0D37E9D65AF: DKIM-Signature field added (s=default, d=sendergateway.example.com)
Sep 18 13:43:48 postfix postfix/smtpd[992]: disconnect from gateway[172.19.199.1]
Sep 18 13:43:48 postfix postfix/qmgr[98]: 9E0D37E9D65AF: from=<sender(a)example.com>, size=59393, nrcpt=1 (queue active)
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: mail for [172.19.199.2]:8024 is using up 7863 of 7863 active queue entries
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: you may need to reduce lmtp connect and helo timeouts
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: so that Postfix quickly skips unavailable hosts
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: you may need to increase the main.cf minimal_backoff_time and maximal_backoff_time
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: so that Postfix wastes less time on undeliverable mail
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: you may need to increase the master.cf lmtp process limit
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: please avoid flushing the whole queue when you have
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: lots of deferred mail, that is bad for performance
Sep 18 13:43:48 postfix postfix/qmgr[98]: warning: to turn off these warnings specify: qmgr_clog_warn_time = 0
```
We see the queueing in Postfix as Mailman is not receiving and that this is why Postfix is using up all it's active queue entries.
However, suddenly this morning, everything came alive all by itself. The question left is: What held the flow. If we don't know what held it, it may happen again, so I would really like some help to understand the problem.
```
Sep 20 07:05:23 postfix opendkim[19]: 23AED7E98F748: incoming.gateway.example.com [192.168.111.222] not internal
Sep 20 07:05:23 postfix opendkim[19]: 23AED7E98F748: not authenticated
Sep 20 07:05:23 postfix postfix/smtpd[20713]: disconnect from incoming.gateway.example.com[192.168.111.222]
Sep 20 07:05:23 postfix postfix/qmgr[16738]: 23AED7E98F748: from=<someone(a)foo.bar>, size=4379, nrcpt=1 (queue active)
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: mail for [172.19.199.2]:8024 is using up 5363 of 5364 active queue entries
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: you may need to increase the main.cf lmtp_destination_concurrency_limit from 20
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: please avoid flushing the whole queue when you have
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: lots of deferred mail, that is bad for performance
Sep 20 07:05:23 postfix postfix/qmgr[16738]: warning: to turn off these warnings specify: qmgr_clog_warn_time = 0
Sep 20 07:05:23 postfix postfix/lmtp[20393]: B64F97E9D0BF4: to=<LISTNAME-bounces(a)lists.example.com>, relay=172.19.199.2[172.19.199.2]:8024, conn_use=4, delay=663077, delays=659443/3624/1/8.1, dsn=2.0.0, status=sent (250 Ok)
Sep 20 07:05:23 postfix postfix/qmgr[16738]: B64F97E9D0BF4: removed
```
/ Henrik Rasmussen
6 years, 3 months
[MM3-users] Re: [Mailman-Users] Mailman 3 on Ubuntu 18.04
by Odhiambo Washington
On Sun, 9 Dec 2018 at 22:37, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/9/18 10:14 AM, Odhiambo Washington wrote:
> >
> > In my quest to run Mailman3, I obtained a VM running Ubuntu 18.04 and
> > started on getting to run Mailman3.
> > I found this link -> https://github.com/iomarmochtar/mailman3_ei
> > It has instructions which looked fairly simple to get Mailman3 installed.
> > However, I did encounter some hitches since the script is not meant for
> > Ubuntu.
>
>
> First, a more appropriate list for Mailman 3 is
> mailman-users(a)mailman3.org
> <https://lists.mailman3.org/mailman3/lists/mailman-users@mailman3.org/>.
>
I am shifting to that list, after this.
>
> Also, we find it very difficult to address issues arising from third
> party packages and how-tos. See <https://wiki.list.org/x/12812344>.
>
> And, there is a Mailman 3 package for Debian/Ubuntu
>
> apt install mailman3-full
>
In all my searching, I never found this in any documentation. I must have
been searching in the wild, or my FreeBSD-mindedness got me stuck to
containers and VENVs.
> > With a few runs and observing the logs and modifying the script at every
> > step it encountered and error, I finally managed to install Mailman3. It
> > seems this is what you guys call a virtual environment in the Linux
> world?
> > :-)
>
>
> I think the virtual environment in this case is a Python thing, not a
> Linux thing.
>
I agree with you on that.
> <snip>
>
>
> Much of what I snipped is specific to
> <https://github.com/iomarmochtar/mailman3_ei>, the details of which I'm
> not interested in learning.
>
That is fine. I have put it aside for now.
>
>
> >>From this point now is where I need help - serious help in smoothening
> > things up and getting to understand these venv stuff!
> >
> > So I did run 'service mailman3 start' but this seems to be waiting for
> too
> > long to drop me back into the CLI.
> > I also run 'systemctl start supervisord' and that seems to work. What I
> am
> > not sure is whether 'systemctl stop supervisord' actually does what it is
> > expected to do because after I execute it, I still see processes running
> > that I think are related.
> >
> > So far, I have been able to access the webUI on my VM using
> > https://N.N.N.N:9090
> > I created a domain.
> > But when I create a test list, I get an error - and I do not have a clue
> > which logfile would have the error details - nginx or mailman... (I am
> 100%
> > new to nginx).
>
>
> The nginx logs generally aren't too useful. Both Mailman core and Django
> write logs. Core's log is probably in var/log/mailman.log where is
> defined in mailman.cfg as var_dir. Django's log is defined in Django's
> settings in the LOGGING setting.
>
Which file contains Django's settings?
> > So now, I am stuck at:
> > 1. Creating the mailing list
> > 2. Getting to know whether the archiver is installed and running
> > 3. Knowing is there are cron jobs to be running (Mailman2 type of
> thinking!)
>
>
> See
> <http://docs.mailman3.org/en/latest/config-core.html#configuring-cron-jobs
> >
> and
> <
> http://docs.mailman3.org/en/latest/config-web.html#scheduled-tasks-required
> >
>
Great to see that. I'll read and see if I make heads or tails of the
writeup.
> > Ultimately I need to figure out:
> > 1. How the mailman queue runners are started and stopped
>
> mailman start
>
Why not systemctl mailman start?? Just wondering, as that is what I am
seeing is common in Ubuntu.
Does this make the components start after a reboot?
> > 2. Migrate s few lists from Mailman2 to this Mailman3
>
> mailman import21
> <django management> hyperkitty_import
>
Please show me where that is documented. I have to pull my lists from an
old machine to this new one.
> > 3. Mailman3 should use MySQL storage and Exim4 as the MTA (I have
> > configured these bits in mailman.cfg)
>
>
> You also need to configure MySQL in Django. See the DATABASES setting in
> the Django settings.
>
So, when you talk about Django settings, you are referring to mailman.cfg
file??? :-)
I have spent all my weekend till now, trying to figure all this out. Life
seems a little easier with mm2.x ....
Thank you.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
7 years, 1 month
[MM3-users] Re: Message is received but is not sent out. Another, similar message, is processed fine.
by Allan Hansen
Thank you, Mark.
I see this in mailman.log. I had looked there for her email address, but it does not appear, so I moved on to the smtp.log.
In
Jun 26 17:18:11 2023 (1243315) ACCEPT: ^M
<CH3PR12MB82593801A1DF2E329E977046BE27A(a)CH3PR12MB8259.namprd12.prod.outlook.com>
Jun 26 17:18:11 2023 (1243319) Uncaught runner exception: unknown encoding: iso-8859-8-i
Jun 26 17:18:11 2023 (1243319) Traceback (most recent call last):
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/core/runner.py", line 179, in _one_iteration
self._process_one_file(msg, msgdata)
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/core/runner.py", line 272, in _process_one_file
keepqueued = self._dispose(mlist, msg, msgdata)
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/runners/pipeline.py", line 37, in _dispose
process(mlist, msg, msgdata, pipeline)
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/core/pipelines.py", line 53, in process
handler.process(mlist, msg, msgdata)
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/handlers/mime_delete.py", line 382, in process
process(mlist, msg, msgdata)
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/handlers/mime_delete.py", line 151, in process
reset_payload(msg, firstalt)
File "/opt/mailman/mm/venv/lib/python3.9/site-packages/mailman/handlers/mime_delete.py", line 202, in reset_payload
msg.set_payload(subpart.get_payload(decode=True).decode(
LookupError: unknown encoding: iso-8859-8-i
Jun 26 17:18:11 2023 (1243319) SHUNTING: 1687825091.8457294+478565324778f2eb6f0fecaeccbd29596edaef53
I also see this in queue/shunt:
mailman@list:~/var/queue/shunt$ ls -ltr
total 88
-rw-rw---- 1 mailman mailman 7168 Feb 17 23:03 1676703781.903885+fdcba88c680f50a0799c672cd0abe82e4bfe19b3.pck
-rw-rw---- 1 mailman mailman 19821 Apr 2 07:08 1680444501.0831451+e110b8c5c20ba3f87c1cea1c5f38e74a6cf8f1c5.pck
-rw-rw---- 1 mailman mailman 19489 Jun 18 11:52 1687114364.41209+1684717da96ff76d1ce773a0cf39c5f8b63e9078.pck
-rw-rw---- 1 mailman mailman 20274 Jun 18 19:01 1687140067.64057+7f77c1ebca490831f58ef9cfde06109081a4e7d6.pck
-rw-rw---- 1 mailman mailman 20085 Jun 26 17:18 1687825091.8457294+478565324778f2eb6f0fecaeccbd29596edaef53.pck
The three last entries coincide with the captured messages.
Finally, in the captured message source (the last which was Cc'd to me) I do see this:
Content-Type: text/plain; charset="iso-8859-8-i"
Which matches the complaint in the mailman log.
In the message that did get accepted, I see this:
Content-Type: text/plain; charset="utf-8"
Another difference:
The posted message shows
Content-Transfer-Encoding: base64
The shunted message shows
Content-Transfer-Encoding: quoted-printable
It appears that the two messages are, indeed, different, despite the similar looks when rendered.
My question thus changes to:
a. Should she be doing something special when sending to the lists?
b. If not, is my setup in need of change?
Yours,
Allan
On 6/26/23, 20:44, "Mark Sapiro" <mark(a)msapiro.net <mailto:mark@msapiro.net>> wrote:
On 6/26/23 8:09 PM, Allan Hansen wrote:
> Hi all,
>
> One of my subscribers is trying to send a message to a list, but it is not being posted.
> The smtp.log shows that her first message was posted, but a second message (with 2 retries) was not:
>
> mailman@list:~/logs$ grep herName smtp.log
> Jun 18 11:22:05 2023 (1243316) ('127.0.0.1', 56686) >> b'MAIL FROM:<herName(a)hotmail.com <mailto:herName@hotmail.com>> SIZE=40109'
> Jun 18 11:22:05 2023 (1243316) ('127.0.0.1', 56686) sender: herName(a)hotmail.com <mailto:herName@hotmail.com>
> Jun 18 11:25:01 2023 (1243318) <CH3PR12MB82593510B90D8F05177C606BBE5EA(a)CH3PR12MB8259.namprd12.prod.outlook.com <mailto:CH3PR12MB82593510B90D8F05177C606BBE5EA@CH3PR12MB8259.namprd12.prod.outlook.com>> post to theList(a)domain.tlc <mailto:theList@domain.tlc> from herName(a)hotmail.com <mailto:herName@hotmail.com>, 40061 bytes
> Jun 18 11:52:42 2023 (1243316) ('127.0.0.1', 48448) >> b'MAIL FROM:<herName(a)hotmail.com <mailto:herName@hotmail.com>> SIZE=18343'
> Jun 18 11:52:42 2023 (1243316) ('127.0.0.1', 48448) sender: herName(a)hotmail.com <mailto:herName@hotmail.com>
> Jun 18 19:01:04 2023 (1243316) ('127.0.0.1', 59550) >> b'MAIL FROM:<herName(a)hotmail.com <mailto:herName@hotmail.com>> SIZE=19123'
> Jun 18 19:01:04 2023 (1243316) ('127.0.0.1', 59550) sender: herName(a)hotmail.com <mailto:herName@hotmail.com>
> Jun 26 17:18:11 2023 (1243316) ('127.0.0.1', 44732) >> b'MAIL FROM:<herName(a)hotmail.com <mailto:herName@hotmail.com>> SIZE=18930'
> Jun 26 17:18:11 2023 (1243316) ('127.0.0.1', 44732) sender: herName(a)hotmail.com <mailto:herName@hotmail.com>
>
> At the second retry she copied me and I saw nothing untowards in her message.
> She's not getting any error return messages and her subscription is OK.
> Both messages were multipart/alternative and both contained Hebrew text.
>
> I told her I'd look at this, but I'm at a loss.
smtp.log is not very useful for diagnosing things like this. It only
says a message was received, not why it didn't result in a post from the
list as in the first case. What's in mailman.log with those timestamps?
--
Mark Sapiro <mark(a)msapiro.net <mailto:mark@msapiro.net>> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users(a)mailman3.org <mailto:mailman-users@mailman3.org>
To unsubscribe send an email to mailman-users-leave(a)mailman3.org <mailto:mailman-users-leave@mailman3.org>
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ <https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/>
Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org <mailto:mailman-users@mailman3.org>/message/F4K7HYUYSWEKF6LT35UCT3KBJX6GHY4I/
This message sent to hansen(a)rc.org <mailto:hansen@rc.org>
2 years, 6 months
[MM3-users] Re: old usernames appearing in mailman.log
by Ken Alker
--On Monday, June 26, 2023 5:03 PM -0700 Mark Sapiro <mark(a)msapiro.net>
wrote:
> On 6/25/23 10:57 PM, Ken Alker wrote:
>>
>> I have studied the output, looked at the code lines referenced in the
>> Traceback, and I can't figure out the problem (I feel like the sad guy
>> on the side of the street with his hood up staring into the engine bay
>> with stream pouring out and not knowing what to do next). I did some
>> analyses on the log file and determined that there are 71 messages it is
>> having problems processing and there are 71 messages "stuck" in
>> /opt/mailman/mm/var/archives/hyperkitty/spool/ (and that it has tried to
>> process each message about 20 times).
>
>
> Each time there is a new post, it retries all the messages in the spool/
> directory.
>
>
>> Since upgrading to V3.3.8 I've
>> not seen new messages get stuck like this (although it's only been
>> running smoothly for 24 hours now and there hasn't been much activity).
>> But I'm suspecting there was something awry between the conversion from
>> V2 to V3 in February and me installing V3.3.8 that caused this (based on
>> message time stamps). It is HIGHLY likely these messages are in the
>> archiving spool as a result of me unshunting the shunt queue and that
>> they are copies of the messages that got delivered to the subscribers
>> when I did that... but are now hung up in the archiving process.
>
>
> Are new messages being archived OK. If so, that is a real mystery. It
> says mailman-hyperkitty can talk to hyperkitty and archive new messages,
> but it can't talk to hyperkitty when attempting to process a queued
> message.
Ahhh! I had convinced myself that new messages were being archived, but
now that you ask, they ARE NOT. I may have said they were at some point,
and I apologize if I did. Now I feel less confused.
It appears that archiving stopped when I did the migration from "Debian
package layout" to "source layout" and upgraded to V3.3.8. I'll bet I
munged a config file or two. I was struggling with what to keep from
Debian layout and what to bring over from the source install instructions
and I haven't gone back through to clean them up. I had previously figured
it could not be a configuration issue since some messages were making it
into the archive and some were not, but now that I see NONE are making it,
this makes the problem less daunting.
>> Following is an excerpt from mailman.log. Note that the "first
>> section" (from DOCTYPE at top to 404 at bottom) is repeated four times
>> (maybe not identical; but similar) (so I did not include it to save
>> room) before the Traceback appears.
>>
>> Jun 25 00:13:04 2023 (386086) No cached copy of the public suffix list
>> found
>> Jun 25 00:13:04 2023 (386086) ACCEPT:
>> <7500f0e2-1ae1-309d-60d7-6e592c7abf7a(a)west.net>
>> Jun 25 00:13:04 2023 (386090) HyperKitty failure on
>> https://lists.netlojix.com/hyperkitty/api/mailman/urls:
>
> I just went to that URL and got a 404. This is a serious problem.
> https://lists.netlojix.com/hyperkitty/api/ also returns a 404. It should
> return some documentation of the API.
>
> https://lists.netlojix.com/archives/api/ does work and returns the
> expected information.
>
> Your urls.py file probably contains in its urlpatterns list
> ```
> re_path(r'^mailman3/', include('postorius.urls')),
> re_path(r'^archives/', include('hyperkitty.urls')),
> ```
> The simplest fix is to add
> ```
> re_path(r'^postorius/', include('postorius.urls')),
> re_path(r'^hyperkitty/', include('hyperkitty.urls')),
> ```
> to that list.
I see many ursl.py files all over the system, but none that appear to be
where I'd expect them to be for processing by any executables. Is this a
file I should have copied over and edited, or created from scratch and
somehow missed? I don't see anything about it in the instructions at
<https://docs.mailman3.org/en/latest/install/virtualenv.html> (although I
do see a post from last July from someone stating it got missed in the
instructions back then).
> Another way to fix this is in your mailman-hyperkitty.cfg, you have
> ```
> base_url: https://lists.netlojix.com/hyperkitty
> ```
> Change that to
> ```
> base_url: https://lists.netlojix.com/archives
> ```
Done, and restarted both services, but I'm not seeing a difference when I
go to <https://lists.netlojix.com/hyperkitty/api/> (I was getting a "Page
not found", and still get that).
> but it wouldn't hurt to add the `^postorius/` and `^hyperkitty/` patterns
> to urls.py anyway.
2 years, 6 months
[MM3-users] Re: Postorius no connection to REST API
by Richard Rosner
Mark Sapiro wrote:
> > You are in a better position to answer that than am I.
> What does sudo netstat -lntp show?
A lot. But since most of that isn't relevant here:
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:465 0.0.0.0:* LISTEN 20241/master
tcp 0 0 127.0.0.1:8024 0.0.0.0:* LISTEN 14076/python3
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 20241/master
tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 14080/python3
tcp 0 0 0.0.0.0:587 0.0.0.0:* LISTEN 20241/master
tcp6 0 0 :::80 :::* LISTEN 13882/apache2
tcp6 0 0 :::465 :::* LISTEN 20241/master
tcp6 0 0 :::25 :::* LISTEN 20241/master
tcp6 0 0 :::443 :::* LISTEN 13882/apache2
tcp6 0 0 :::587 :::* LISTEN 20241/master
> What does ps -fwwa|grep rest show?
root 15055 14843 0 12:58 pts/1 00:00:00 grep rest
So whatever it's supposed to find, it's not there
> > mailman3-web.service must also run as list.
I changed that. It didn't like it.
systemctl status mailman3-web.service
● mailman3-web.service - Mailman3-web uWSGI service
Loaded: loaded (/lib/systemd/system/mailman3-web.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2021-08-11 13:05:39 CEST; 32s ago
Docs: file:///usr/share/doc/mailman3-web/README.rst
Process: 15570 ExecStart=/usr/bin/uwsgi --plugin python3 --ini /etc/mailman3/uwsgi.ini (code=exited, status=1/FAILURE)
Main PID: 15570 (code=exited, status=1/FAILURE)
Status: "initializing uWSGI"
Aug 11 13:05:39 mail systemd[1]: Starting Mailman3-web uWSGI service...
Aug 11 13:05:39 mail systemd[1]: mailman3-web.service: Main process exited, code=exited, status=1/FAILURE
Aug 11 13:05:39 mail systemd[1]: mailman3-web.service: Failed with result 'exit-code'.
Aug 11 13:05:39 mail systemd[1]: Failed to start Mailman3-web uWSGI service.
Aug 11 13:05:39 mail systemd[1]: mailman3-web.service: Service RestartSec=100ms expired, scheduling restart.
Aug 11 13:05:39 mail systemd[1]: mailman3-web.service: Scheduled restart job, restart counter is at 5.
Aug 11 13:05:39 mail systemd[1]: Stopped Mailman3-web uWSGI service.
Aug 11 13:05:39 mail systemd[1]: mailman3-web.service: Start request repeated too quickly.
Aug 11 13:05:39 mail systemd[1]: mailman3-web.service: Failed with result 'exit-code'.
Aug 11 13:05:39 mail systemd[1]: Failed to start Mailman3-web uWSGI service.
> > So, what do you have in your apache config for proxying to uwsgi and
> what's your uwsgi configuration.
lists-ssl.conf:
<VirtualHost *:443>
ServerAdmin admin(a)domain.de
ServerName lists.domain.de
Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico
Alias /mailman3/static /var/lib/mailman3/web/static
<Directory "/var/lib/mailman3/web/static">
Require all granted
</Directory>
<IfModule mod_proxy_uwsgi.c>
ProxyPass /mailman3/favicon.ico !
ProxyPass /mailman3/static !
ProxyPass "/mailman3" "unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost:8001/"
</IfModule>
# SSL Engine Switch:
# Enable/Disable SSL for this virtual host.
SSLEngine on
SSLCertificateFile /etc/ssl/certs/lists.domain.de.cert.pem
SSLCertificateKeyFile /etc/ssl/private/lists.domain.de.private.pem
SSLCertificateChainFile /etc/ssl/certs/dfnca.pem
SSLCACertificateFile /etc/ssl/certs/rwthcert.pem
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
Protocols h2 http/1.1
SSLCipherSuite ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256
SSLHonorCipherOrder off
#RewriteEngine on
#RewriteRule ^/$ https://lists.domain.de/listinfo
Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
Header always set X-Frame-Options: "SAMEORIGIN"
Header always set X-Xss-Protection "1; mode=block"
Header always set X-Content-Type-Options "nosniff"
Header always set Content-Security-Policy "default-src 'self' *.domain.de; script-src 'self' *.domain.de; connect-src 'self' *.domain.de; img-src 'self' *.domain.de; style-src 'self' 'unsafe-inline' *.domain.de; object-src 'self' *.domain.de; frame-src 'self' *.domain.de;"
Header always set Referrer-Policy "no-referrer-when-downgrade"
</VirtualHost>
<VirtualHost *:80>
RewriteEngine On
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [R=301,L]
</VirtualHost>
I guess with uwsgi config you mean the /etc/mailman3/uwsgi.ini file?
[uwsgi]
# Port on which uwsgi will be listening.
uwsgi-socket = /run/mailman3-web/uwsgi.sock
#Enable threading for python
enable-threads = true
# Move to the directory wher the django files are.
chdir = /usr/share/mailman3-web
# Use the wsgi file provided with the django project.
wsgi-file = wsgi.py
# Setup default number of processes and threads per process.
master = true
process = 2
threads = 2
# Drop privielges and don't run as root.
uid = www-data
gid = www-data
plugins = python3
# Setup the django_q related worker processes.
attach-daemon = python3 manage.py qcluster
# Setup hyperkitty's cron jobs.
#unique-cron = -1 -1 -1 -1 -1 ./manage.py runjobs minutely
#unique-cron = -15 -1 -1 -1 -1 ./manage.py runjobs quarter_hourly
#unique-cron = 0 -1 -1 -1 -1 ./manage.py runjobs hourly
#unique-cron = 0 0 -1 -1 -1 ./manage.py runjobs daily
#unique-cron = 0 0 1 -1 -1 ./manage.py runjobs monthly
#unique-cron = 0 0 -1 -1 0 ./manage.py runjobs weekly
#unique-cron = 0 0 1 1 -1 ./manage.py runjobs yearly
# Setup the request log.
#req-logger = file:/var/log/mailman3/web/mailman-web.log
# Log cron seperately.
#logger = cron file:/var/log/mailman3/web/mailman-web-cron.log
#log-route = cron uwsgi-cron
# Log qcluster commands seperately.
#logger = qcluster file:/var/log/mailman3/web/mailman-web-qcluster.log
#log-route = qcluster uwsgi-daemons
# Last log and it logs the rest of the stuff.
#logger = file:/var/log/mailman3/web/mailman-web-error.log
logto = /var/log/mailman3/web/mailman-web.log
4 years, 5 months