Re: SMTPAuthenticationError at /accounts/login/
by Mohsen Masoudfar
Hi Stephen,
Thank you for the hint. I was focused too much on the documentation and was looking for the answer in the wrong place. Problem fixed.
My problem started with "SMTP authentication not supported" and after fixing that, which was a Postfix parameter, I got this issue and was also looking in Postfix or Django configuration for the answer.
This problem is solved and I wished there was a chance to update the documentation, so that these sort of information could be added in the documentation for other people who try to use mailman3.
Thanks
Mohsen
-----Original Message-----
From: Stephen J. Turnbull <turnbull.stephen.fw(a)u.tsukuba.ac.jp>
Sent: Thursday, April 9, 2020 4:06 AM
To: Mohsen Masoudfar <mmasoudf(a)aaas.org>
Cc: mailman-users(a)mailman3.org
Subject: [MM3-users] SMTPAuthenticationError at /accounts/login/
[EXTERNAL EMAIL]
mmasoudf(a)aaas.org writes:
> I have setup the stack according to:
> Mailman Suite Documentation. Release 3.3 PDF file.
> I configured
> EMAIL_HOST_USER and
> EMAIL_HOST_PASSWORD
> in Django settings.py. I could not find the place that I might > configure it for Postfix
I think it helps to start thinking about it from the other end. That is, Postfix is the mail server for all users on a multiuser system.
Therefore in "normal" configurations it will use the authentication mechanisms provided by the host operating system.[1] You need to have a user for Mailman in any case, typically called "mailman" or "list".[2]
Once you've configured a user for Mailman on the Postfix side, you use that user and authentication data for Mailman core, Postorius, and HyperKitty to send mail via Postfix.
> I am running the site on AWS and to be able to use AWS mail-servers > I have to define following parameter for AWS:
> smtp_sasl_password_maps which points to a hash for AWS 'ACCESS KEY > ID' and 'SECRET ACCESS KEY'
I don't think that has anything to do with the Mailman-to-Postfix side of things; that's for Postfix to send to AWS's mail exchanger (aka MX).
Steve
Footnotes:
[1] It may also be possible to create Postfix-only "virtual" users who have no existence as far as the OS is concerned, but I don't know anything about that.
[2] I haven't thought carefully about whether using Mailman's user in Django would be a security risk, but as long as it only has access to Mailman I don't see how it can do more harm than Mailman itself can, and I don't see why it would be more subject to exploitation than Mailman itself is. Anybody with security chops who wants to weigh in, please do.
5 years, 6 months
Re: how to administrate mailman3 in CLI ?
by testeur
Sorry. I didn't see in the good place.
when i see parameters here :
https://URL/mailman3/postorius/accounts/per-subscription-preferences/
the options or the specific language chosen (not for the ui) for the
list is finally changing when i use the code used before...
i should use shell with mailman user. ok i note it.
I ve seen this link about python script to manage mailman3 by shell :
https://fog.ccsf.edu/~msapiro/scripts/ (thx you)
Regards,
On 23/09/2023 17:33, Mark Sapiro wrote:
> On 9/23/23 07:40, testeur via Mailman-users wrote:
>>
>> |# mailman shell -l testlist(a)domain.tld |
>
> It looks like you are running `mailman shell` as `root`. You should
> always run mailman commands including starting mailman as the Mailman
> user, never as root. Running thing as root can create things that
> later can't be read by the mailman user.
>
>> ||>>> from mailman.interfaces.action import Action >>> for mbr in
>> m.members.members: mbr.moderation_action = Action.defer |||commit()|
>>
>> |||>>> from mailman.interfaces.action import Action |||||||>>>
>> english = getUtility(ILanguageManager).get('en')| >>> for mbr in
>> m.members.members: || mbr.preferences.acknowledge_posts = False
>> mbr.preferences.delivery_mode = DeliveryMode.regular
>> mbr.preferences.delivery_status = DeliveryStatus.enabled
>> mbr.preferences.hide_address = False
>> mbr.preferences.preferred_language = english
>> mbr.preferences.receive_list_copy = False
>> mbr.preferences.receive_own_postings = True|commit()||
>
>
> I'm not sure what the above actually says. The interaction should look
> like
> ```
> >>> from mailman.interfaces.action import Action
> >>> english = getUtility(ILanguageManager).get('en')
> >>> for mbr in m.members.members:
> ... mbr.preferences.acknowledge_posts = False
> ... mbr.preferences.delivery_mode = DeliveryMode.regular
> ... mbr.preferences.delivery_status = DeliveryStatus.enabled
> ... mbr.preferences.hide_address = False
> ... mbr.preferences.preferred_language = english
> ... mbr.preferences.receive_list_copy = False
> ... mbr.preferences.receive_own_postings = True
> ...
> >>> commit()
> ```
> Actually, importing Action is not necessary as it is among the things
> defined by default in `mailman shell` in interactive mode.
>
>> Before the shell command, the web interface was with an other
>> language (i tested with italian one) and other options preferences,
>> but after the shell command, after a web interface refresh, it
>> doesn't change.
>
>
> The above shell interaction sets the preferred_language for all list
> members to english. This has nothing to do with the language for the
> web UI. This is set by Django. See
> <https://docs.djangoproject.com/en/4.2/topics/i18n/translation/#how-django-d…>.
>
>
2 years
Lessons learned and a couple little gotcha for newbies
by William Oliver
I just finished, with great help from Mark Sapiro on the list, a new
installation of mailman3.
I used the instructions at
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/insta…
and
https://docs.mailman3.org/en/latest/install/virtualenv.html
There were a couple of gotchas that I didn't see well documented.
1) I got the error
django.template.library.InvalidTemplateLibrary: Invalid template
library specified. ImportError raised when trying to load
'hyperkitty.templatetags.decorate': cannot import name 'escape_html'
from 'mistune.scanner' (/opt/mailman/venv/lib/python3.9/site-
packages/mistune/scanner.py)
when I ran 'mailman-web migrate'
The fix was trivial: I just added the lines:
from mistune.util import escape_html
from mistune.util import escape_url
to
/opt/mailman/venv/lib/python3.9/site-packages/mistune/scanner.py
It's not hard to search for those definitions, but if someone doesn't
know a smattering of python, it might not be obvious.
2) Similarly, migrate (using postgres) also gave the error:
File "/opt/mailman/venv/lib/python3.9/site-
packages/django/db/backends/postgresql/utils.py", line 6, in
utc_tzinfo_factory
raise AssertionError("database connection isn't set to UTC")
I was able to find a discusson of this one at
https://stackoverflow.com/questions/68024060/assertionerror-database-connec…
which had me downgrade to an older version of psycopg2-binary:
pip install psycopg2-binary==2.8.6
There might be a different fix, but this worked.
3) When I ran the test "uwsgi --ini /etc/mailman3/uwsgi.ini" I got the
error:
ModuleNotFoundError: No module named 'settings'
This was also trivially fixed by adding the path:
export PYTHONPATH=$PYTHONPATH:/etc/mailman3
in /opt/mailman/.bashrc
4) Sometimes when I rebooted the machine, when I ran 'systemctl start
mailman3' I'd get the error that mailman wouldn't start because the
lock was stll there. I never could figure out where the lock was, so I
worked around it by replacing
ExecStart=/opt/mailman/venv/bin/mailman start
with
ExecStart=/opt/mailman/venv/bin/mailman start --force
in /etc/systemd/system/mailman3.service
I'm sure there's something bad about doing that, but it works for me,
so far.
5) I had a problem with mail to outside people not being delivered
while mail to the local host was delivered. It turned out that this was
because in my mailman.cfg I had "smtp_host:my.listserver.tld when it
should have been localhost. The error was "relay access denied." To
fix this, I had to add my.listserver.tld to "mynetworks" in
/etc/postfix/main.cf, ie change:
mynetworks = 127.0.0.0/8 10.0.0.0/24
to
mynetworks = 127.0.0.0/8 10.0.0.0/24 111.222.333.444 <--- my ip address
However, Mark Sapiro (a God) told me this was because I had changed a
line in the [mta] section of mailman.cfg. I had changed
smtp_host: localhost
to
smtp_host: mydomain.tld
Here's the thing. The reason I did this was because the instructions at
https://docs.mailman3.org/en/latest/config-core.html
give this as their model:
[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: mail.example.com
lmtp_port: 8024
smtp_host: mail.example.com <---- not localhost
smtp_port: 25
The instructions are good, but there are a few places where one size
does not fit all, I guess.
6) A second error in delivery to outside addresses resulted in an
"Access denied" error. A search on the internet revealed that this had
to do with (for me) smtpd_recipient_restrictions in postfix. See, for
instance:
https://know.mailsbestfriend.com/postfix_error_relay_access_denied_noqueue_…
Changing this fixed things.
I'm not sure, but I think this shows up mostly with installations that
include dovecot.
7) If you are using virtual domains in postfix, you need to use the
"unusual postfix configuration" in
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h…
virtual_alias_maps =
hash:/path-to-mailman/var/data/postfix_vmap
8) You need to manually set the local virtual host for mailman3.
Somehow, I got the impression from the installation instructions that
this was done automagically to the default x.mydomain.tld if your
domain is mydomain.tld. It doesn't.
9) Follow the instructions verbatim, and follow them in the order
provided. I installed postfix and nginx out of order, and it did OK,
but once you get into the python virtual environment, it is easy to
miss something if you don't do it in order.
10) Jot down lots of notes. This is one of the more complicated
installations I've done, mostly because it involved configuring a
zillion packages to talk to each other: Postfix, nginx, postorius,
hyperkitty, django, and others (I'd never heard of "whoosh" or "sass"
for instance). All of these can be unforgiving with configuration
errors.
The default installation instructions assumed a number of packages I'd
never used. I'm an apache/dovecot/mysql kind of person, and I had to
switch to nginx/postgres to use the instructions as the written
defaults. I'm sure I could have used apache and mysql, but it would
mean trying to figure out what didn't work without following the
installations verbatim. I'd never even heard of postorius, hyperkitty,
or django.
So, it was easy to get confused a bit. Taking notes of changes I made
was important in being able to back out of disastrous choices. Plus,
once I'd installed it on one machine, I sat down and installed it on
another machine, and it was very useful to have written down how I
fixed an issue when it came up again -- and I didn't remember exactly
what I had done.
11) Make multiple sequential backups of configuration files. I went
down a couple of paths I shouldn't have gone down. For instance, with
the vmap issue, I got it in my head that it was a postfix configuration
issue, not a mailman issue. So I kept changing things in postfix.
Once I was told that it was in *mailman* I had to set the virtual host,
I needed to back out of about 8 tweaks I'd made to main.cf in
/etc/postfix. If I hadn't been making main.cf.backup.01,
main.cf.backup.02, etc. I would have had to spend another few hours
trying to remember what inappropriate changes I had made.
This is particularly true for me because I am not a great typist, and
when I try to retype something to back out, there's a reasonable chance
I'll mistype something and it still won't work. With these backups, I
don't have both manually back out of things *and* proof my typing again
and again. It's a lot easier to delete backups once things are working
than to try to remember what you did late at night after a couple of
bourbons.
12) Choose one set of installation instructions and follow it. Don't
try to mix instructions from different tutorials unless you really know
what you are doing. I didn't. For instance, there's a different set
of mailman3 installation instructions at:
https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10
It does things a little differently. It's useful to look at that when
you are debugging why something's not working, but you can get into
some trouble trying to do both installations at the same time -- which
I found myself doing when something wasn't working. It didn't help.
billo
3 years, 9 months
Re: Re-send subscription response and file permission error in queue
by Mark Sapiro
On 9/8/20 10:56 PM, accounts-mailman(a)holbrook.no wrote:
> A user sent subscription request to the `..-join@` address of a list. Email domain wasn't set up properly, so response got rejected. Now, if the user tries to send another request, no reponse comes back. In the log I find a python stack trace saying subscription already pending.
>
> ```
> Sep 08 10:32:44 2020 (19918) Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, in _one_iteration
> self._process_one_file(msg, msgdata)
> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, in _process_one_file
> keepqueued = self._dispose(mlist, msg, msgdata)
> File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 196, in _dispose
> mlist, msg, msgdata, parts, results)
> File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py", line 104, in process
> ISubscriptionManager(mlist).register(subscriber)
> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 520, in register
> list(workflow)
> File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, in __next__
> return step()
> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 227, in _step_sanity_checks
> raise SubscriptionPendingError(self.mlist, self.address.email)
> mailman.interfaces.subscriptions.SubscriptionPendingError
> ```
I'll look at this when I get some time. I'm still catching up after
being off line for 12 days.
> Furthermore I am getting a permission denied error in the attempt of shunting the message, which I don't quite understand because the folder has user `list` group `list` and `700` mode:
>
> ```
> Sep 08 10:25:29 2020 (19918) SHUNTING: 1599553529.9889646+88be5afa0bf3111a83e68cc97ce2000723ad93fd
> Sep 08 10:29:49 2020 (19918) Uncaught runner exception: [Errno 13] Permission denied: '/var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck'
> Sep 08 10:29:49 2020 (19918) Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 156, in _one_iteration
> msg, msgdata = self.switchboard.dequeue(filebase)
> File "/usr/lib/python3/dist-packages/mailman/core/switchboard.py", line 150, in dequeue
> with open(filename, 'rb') as fp:
> PermissionError: [Errno 13] Permission denied: '/var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck'
> ```
>
> How can I remove the pending subscription, or re-send the response email?
In Postorius, pull down Subscription Requests and select Pending
Confirmation and delete the request.
> And what could be the cause of the permission error?
Mailman can't read the
/var/lib/mailman3/queue/command/1599553789.306307+6fee74af69263553b5ffd0e73cc095ab651bd855.pck
file. This is strange since it wrote it, but check the modes and
user/group of all the directories in the path and the file itself.
Could it be a SELinux or apparmor issue?
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
5 years, 1 month
Re: multiple sites on a mailman3 server question
by David Newman
> On Dec 29, 2021, at 08:49, William Oliver <billo(a)billoblog.com> wrote:
>
> On Tue, 2021-12-28 at 18:23 -0800, David Newman wrote:
>>>
>>> [snip]
>>> I'm using nginx
>> I'd asked about this a few weeks ago for a MM3 host running Ngnix
>> with a
>> few virtual domains, one of which runs Roundcubemail as its default.
>>
>> Mark Sapiro suggested the following, and it seems to be working OK.
>> You'll need these lines in the Nginx config file for each virtual
>> host
>> you define.
>>
>> # begin mailman3 stuff
>>
>> location /static/ {
>> alias /opt/mailman/web/static/;
>> }
>>
> [snip]
>
>
>
> No joy. This just makes it kind of work as the mailman interface when
> it gets rewritten.
>
> My problem is that the url is getting referred to
> domainname/mailman3/lists even when I don't want it to.
That sounds like an nginx configuration problem, not a Mailman3 issue.
One assumption in the MM3 web setup docs is that MM3/Django/Postorius/Hyperkitty are the only web services running. If they’re not, and there are other services running, you’ll need to tell Nginx where they are.
The setup I provided is an excerpt and covers only how to point to MM3 in such a scenario. It does not cover how to point to something else as root at, say, www.domain.tld.
This is off-topic for a Mailman3 list, but maybe check your default files in the sites-available directory, and also anything they point to, paying particular attention to where / points and which root directory you’re using. Same thing with any other config files in that directory.
dn
> Adding these
> changes just (partially) loads the mailman3 interface when it gets
> rewritten.
>
> What I'm trying to figure out is where uwsgi yanks control from nginx
> and does the rewriting. Pulling that proxypass stuff out will stop
> mailman3 from coming up, and adding it allows mailman3 to show its
> interface, but the address gets referred regardless.
>
> Somehow I need to tell uwsgi to ignore www.domain.com *or* somehow tell
> nginx not to push it to port 8000. But I don't see where nginx is
> getting told to go to 8000 in the www.domain.com section.
>
> billo
>
>
>
>
>
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
3 years, 9 months
ValueError: ...:EVP_DigestInit_ex:disabled for fips
by Ralf Wiegand
(venv-3.6) [root@maildev mailman-suite_project]# python manage.py migrate
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, openid, postorius, sessions,
sites, socialaccount
Running migrations:
Applying contenttypes.0001_initial...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/core/management/__init__.py",
line 381, in execute_from_command_line
utility.execute()
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/core/management/__init__.py",
line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/core/management/base.py",
line 323, in run_from_argv
self.execute(*args, **cmd_options)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/core/management/base.py",
line 364, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/core/management/base.py",
line 83, in wrapped
res = handle_func(*args, **kwargs)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/core/management/commands/migrate.py",
line 234, in handle
fake_initial=fake_initial,
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/migrations/executor.py",
line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan,
fake=fake, fake_initial=fake_initial)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/migrations/executor.py",
line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake,
fake_initial=fake_initial)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/migrations/executor.py",
line 245, in apply_migration
state = migration.apply(state, schema_editor)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/migrations/migration.py",
line 124, in apply
operation.database_forwards(self.app_label, schema_editor,
old_state, project_state)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/migrations/operations/models.py",
line 530, in database_forwards
getattr(new_model._meta, self.option_name, set()),
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/base/schema.py",
line 367, in alter_unique_together
self.execute(self._create_unique_sql(model, columns))
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/base/schema.py",
line 126, in execute
sql = str(sql)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/ddl_references.py",
line 194, in __str__
return self.template % self.parts
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/ddl_references.py",
line 103, in __str__
return self.create_index_name(self.table, self.columns, self.suffix)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/base/schema.py",
line 1049, in create_unique_name
return self.quote_name(self._create_index_name(*args, **kwargs))
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/base/schema.py",
line 884, in _create_index_name
hash_suffix_part = '%s%s' % (names_digest(table_name,
*column_names, length=8), suffix)
File
"/opt/mailman/venv-3.6/lib64/python3.6/site-packages/Django-2.2.12-py3.6.egg/django/db/backends/utils.py",
line 221, in names_digest
h = hashlib.md5()
*ValueError: error:060800A3:digital envelope
routines:EVP_DigestInit_ex:disabled for fips*
Hello group, I am running into this error. any idea what is going
wrong here?
Help is greatly appreciated.
Ralf
5 years, 4 months
Re: Mailman REST API not available. Please start Mailman Core.
by Shashikanth Komandoor
Thank you Mark for your response.
I understood that gunicorn is a front end that is used like web server like
nginx.
But I don't think I am using it instead I am using the httpd to redirect
requests to postorius.
And I am starting the mailman-suite with the command "python manage.py
runserver 0.0.0.0:8000 --insecure &". Hope this does not use gunicorn
(which of course I don't know what it is until now).
Please correct me if any point I am mentioning is wrong.
So, in this case, do you think configuring gunicorn as below would work:
*calling gunicorn.cfg in mailman.cfg as below *
*[webservice]*
*configuration: /var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg*
*and content of gunicorn.cfg as below*
*(venv3) [root@lsmgr mailman]# cat
/var/lib/mailman/mailman/src/mailman/config/gunicorn.cfg[gunicorn]# All the
options that can be put here are available at#
http://docs.gunicorn.org/en/stable/settings.html#settings
<http://docs.gunicorn.org/en/stable/settings.html#settings># No of seconds
to wait before killing the worker processes.*
*workers = 4graceful_timeout = 90*
Also what is the difference between timeout and graceful_timeout ?
Please help me if I can increase the management better. Any guide is also
helpful for me.
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
On Thu, Jul 23, 2020 at 4:10 AM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 7/22/20 9:21 AM, Shashikanth Komandoor wrote:
> > Hi Mark,
> >
> > Never mind please I am sending to you only instead of group as I am
> > sending you the production content along with the mail.
>
>
> See my other reply.
>
>
> > I have configured the workers count as 4 and timeout as 900 in
> > mailman.cfg under [webservice]. This is FYI.
>
> You can't configure Mailman's REST gunicorn workers directly in the
> [webservice] section of mailman.cfg. The only thing you can configure
> there is:
>
> configuration: /path/to/gunicorn.cfg
>
> That file, which is separate from your web server gunicorn.cfg, is where
> you put things like
>
> [gunicorn]
> timeout = 900
>
> It also will not work to put a gunicorn section in mailman.cfg. It must
> be in a separate file pointed to by webservice.configuration.
>
> Increasing the number of workers won't help this issue.
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
>
--
Thanks & Regards,
Shashi Kanth.K
9052671936
5 years, 2 months
Re: Fwd: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
by David Partain
Hi Mark,
Thank you for your kind answer. While I've been running a mailman site for many years, I'm just getting going with mailman3 (and migrating my 1400 lists...) and I feel lost occasionally :)
I wrote:
> I have the same issue, and have changed the configuration in /etc/mailman3/mailman-web.py to:
>
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.mysql',
> 'NAME': 'mailman3web',
> 'USER': 'mailman3web',
> 'PASSWORD': 'xxxxxxxxxxxxxxxxxx',
> 'HOST': 'localhost',
> 'PORT': '',
> 'OPTIONS': {
> 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
> 'charset': 'utf8mb4',
> },
> }
> }
>
> and restarted stuff, but I cannot tell that it has changed anything.
You responded:
> That should do it. I think the original message should have been saved
> as a queue entry in Mailman's var/archives/hyperkitty/spool/ directory.
That directory has all of the messages sent to that list:
# ls /var/lib/mailman3/archives/hyperkitty/spool/
1611827925.8030825+925c996b9b6020ee18e9a689ce23384855159f77.pck 1611827926.1923075+e728f3c29c1fe6eeeca641314cdba7007cb6e132.pck
1611827925.8917305+d16b7004bdbbef98ce40fb8ee70e492715d1607d.pck 1611827926.2623458+1aac13b50d8e7f32eb14d13e6559656aa8f3bd10.pck
1611827925.9594557+7ad2a910b5ee9142c88f0bf26c60c6df960d4262.pck 1611827926.3488555+04a540651c5f829751518433e1c54168ab63a38b.pck
1611827926.0404096+a5e745f51a9f7d541aeb770a89604be17019f3c5.pck 1611827926.4206145+de861eec6404d0c8f9f8bdb180e29bff2de7c52b.pck
1611827926.1167526+0aa63639a4ec102bf63522827637e75e6a2b092a.pck
> If this is the case, it would have been retried and produced the same
> error with each new post, and after you made the above change, it would
> have been archived and the errors stopped.
> Are you saying you are still seeing these errors? If so, it may be a
> different error. What is the exact error message? We don't need all the
> Django stuff, just the part equivalent to
> Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
> OperationalError at /hyperkitty/api/mailman/archive
> (1366, "Incorrect string value: '\xF0\x9F\xA4\xA3\xF0\x9F...' for
> column mailman3web.hyperkitty_email.content at row 1")
Yes, I'm still seeing the errors. When I send mail to a list with a cute emoji, I see the following in /var/log/mailman3/web/mailman-web.log:
ERROR 2021-01-28 10:58:45,708 16923 django.request Internal Server Error: /mailman3/hyperkitty/api/mailman/archive
(bunch of python errors omitted)
DataError: (1366, u"Incorrect string value: '\\xF0\\x9F\\x98\\x8A\\x0A\\x0A...' for column 'content' at row 1")
Also, a bunch of mail is sent to root, presumably one for every message that has previously failed, with the same error information, plus information about the POST method done. (If something there would be helpful, please let me know.)
Finally, I'm doing all of this on Ubuntu, with the packages available in the distro, which means:
mailman3 3.1.1-9
mailman3-doc 3.1.1-9
mailman3-full 3.1.1-9
mailman3-web 0+20170523-14
python-django-hyperkitty 1.1.4-4
python-django-mailman3 1.1.0-4
python-django-postorius 1.1.2-3
python-mailmanclient 3.1.1-5
python3-mailman-hyperkitty 1.1.0-4
Thanks very much for any help you can offer.
Cheers,
David
4 years, 8 months
Re: Welcome Message
by brian＠emwd.com
So I create a template with just one line of text and I save it. When I go subscribe to the list, Postorius throws this error message:
"A server error occurred. Please contact the administrator."
The core mailman.log records a whole bunch of exemptions:
Apr 16 14:36:05 2019 (28) 172.19.199.3 - - "GET /3.1/lists/announce.testlist.emwd.com/roster/owner HTTP/1.1" 200 612
Apr 16 14:36:05 2019 (28) 172.19.199.3 - - "GET /3.1/lists/announce.testlist.emwd.com/roster/moderator HTTP/1.1" 200 90
Apr 16 14:36:10 2019 (28) deque:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 384, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/mailman/app/workflow.py", line 69, in __next__
return step()
File "/usr/lib/python3.6/site-packages/mailman/app/subscriptions.py", line 326, in _step_do_subscription
self.member = self.mlist.subscribe(self.subscriber)
File "/usr/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.6/site-packages/mailman/model/mailinglist.py", line 494, in subscribe
notify(SubscriptionEvent(self, member))
File "/usr/lib/python3.6/site-packages/zope/event/__init__.py", line 32, in notify
subscriber(event)
File "/usr/lib/python3.6/site-packages/mailman/app/membership.py", line 169, in handle_SubscriptionEvent
send_welcome_message(mlist, member, member.preferred_language)
File "/usr/lib/python3.6/site-packages/mailman/app/notifications.py", line 52, in send_welcome_message
'list:user:notice:welcome', mlist, language=language.code))
File "/usr/lib/python3.6/site-packages/mailman/model/template.py", line 188, in get
name, lookup_context, **substitutions)
File "/usr/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.6/site-packages/mailman/model/template.py", line 109, in get
contents = protocols.get(actual_uri, **auth)
File "/usr/lib/python3.6/site-packages/mailman/utilities/protocols.py", line 38, in get
response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
Apr 16 14:36:10 2019 (28) REST request handler error:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 384, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.6/http/client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3.6/site-packages/urllib3/util/retry.py", line 367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/wsgiref/handlers.py", line 137, in run
self.result = application(self.environ, self.start_response)
File "/usr/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper
rtn = function(*args, **kws)
File "/usr/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 218, in __call__
return super().__call__(environ, start_response)
File "/usr/lib/python3.6/site-packages/falcon/api.py", line 244, in __call__
responder(req, resp, **params)
File "/usr/lib/python3.6/site-packages/mailman/rest/members.py", line 266, in on_post
pre_approved=pre_approved)
File "/usr/lib/python3.6/site-packages/mailman/app/subscriptions.py", line 520, in register
list(workflow)
File "/usr/lib/python3.6/site-packages/mailman/app/workflow.py", line 69, in __next__
return step()
File "/usr/lib/python3.6/site-packages/mailman/app/subscriptions.py", line 326, in _step_do_subscription
self.member = self.mlist.subscribe(self.subscriber)
File "/usr/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.6/site-packages/mailman/model/mailinglist.py", line 494, in subscribe
notify(SubscriptionEvent(self, member))
File "/usr/lib/python3.6/site-packages/zope/event/__init__.py", line 32, in notify
subscriber(event)
File "/usr/lib/python3.6/site-packages/mailman/app/membership.py", line 169, in handle_SubscriptionEvent
send_welcome_message(mlist, member, member.preferred_language)
File "/usr/lib/python3.6/site-packages/mailman/app/notifications.py", line 52, in send_welcome_message
'list:user:notice:welcome', mlist, language=language.code))
File "/usr/lib/python3.6/site-packages/mailman/model/template.py", line 188, in get
name, lookup_context, **substitutions)
File "/usr/lib/python3.6/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.6/site-packages/mailman/model/template.py", line 109, in get
contents = protocols.get(actual_uri, **auth)
File "/usr/lib/python3.6/site-packages/mailman/utilities/protocols.py", line 38, in get
response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='mailman-web', port=8000): Read timed out. (read timeout=5)
Removing the template allows normal subscriptions and the generic welcome message is sent out.
Brian
6 years, 5 months
Daily Cronjob fails
by Stefan Tatschner
Hey hey,
I applied a recent Update from Debian Buster, und this night a received an email about a failing cronjob.
The traceback can be reproduced using:
# sudo -u www-data flock -n /var/run/mailman3-web/cron.daily /usr/share/mailman3-web/manage.py runjobs daily
ERROR OCCURED IN DAILY JOB: sync_mailman (APP: hyperkitty)
START TRACEBACK:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
psycopg2.IntegrityError: null value in column "description" violates not-null constraint
DETAIL: Failing row contains (2, hb-events(a)hackbrettl.de, hb-events, null, [hb-events] , 1, 2015-10-22 09:54:53.684306+00, hb-events.hackbrettl.de).
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/django_extensions/management/commands/runjobs.py", line 36, in runjobs
job().execute()
File "/usr/lib/python3/dist-packages/hyperkitty/jobs/sync_mailman.py", line 35, in execute
sync_with_mailman()
File "/usr/lib/python3/dist-packages/hyperkitty/lib/mailman.py", line 131, in sync_with_mailman
mlist.update_from_mailman()
File "/usr/lib/python3/dist-packages/hyperkitty/models/mailinglist.py", line 192, in update_from_mailman
self.save()
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 808, in save
force_update=force_update, update_fields=update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 838, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 905, in _save_table
forced_update)
File "/usr/lib/python3/dist-packages/django/db/models/base.py", line 955, in _do_update
return filtered._update(values) > 0
File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 667, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
raise original_exception
File "/usr/lib/python3/dist-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3/dist-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python3/dist-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: null value in column "description" violates not-null constraint
DETAIL: Failing row contains (2, hb-events(a)hackbrettl.de, hb-events, null, [hb-events] , 1, 2015-10-22 09:54:53.684306+00, hb-events.hackbrettl.de).
END TRACEBACK
Appearently there is something wrong with the database. Is there anything what I can do about this?
Running the migrations doesn't help:
# sudo -u www-data flock -n /var/run/mailman3-web/cron.daily /usr/share/mailman3-web/manage.py migrate
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes, django_mailman3, django_q, hyperkitty, postorius, sessions, sites, socialaccount
Running migrations:
No migrations to apply.
Thanks!
Stefan
6 years, 7 months