Re: Migration from old server to new server failed
by Helio Loureiro
Hi,
I repeated the steps today.
First I waited to end the errors raised because the mailman3-web wasn't
running.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
line 793, in urlopen
response = self._make_request(
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
line 496, in _make_request
conn.request(
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
line 400, in request
self.endheaders()
File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 976, in send
self.connect()
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
line 238, in connect
self.sock = self._new_conn()
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection
object at 0x7f0fdc2f72b0>: Failed to establish a new connection: [Errno
111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/requests/adapters.py",
line 486, in send
resp = conn.urlopen(
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
line 847, in urlopen
retries = retries.increment(
File
"/local/mailman/venv/lib/python3.10/site-packages/urllib3/util/retry.py",
line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type:
ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1',
port=8000): Max retries exceeded with url: /archives/api/mailman/archive
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at
0x7f0fdc2f72b0>: Failed to establish a new connection: [Errno 111]
Connection refused'))
Which seems to keep forever. So eventually I gave up and moved forward.
And I changed to use the uwsgi socket instead of the direct port.
(venv) mailman@new-mailman3 ~ (v3.1.1) [0|SIGINT]> mailman-web migrate
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you
don't care about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't
care about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying account.0003_alter_emailaddress_create_unique_verified_email...
OK
Applying account.0004_alter_emailaddress_drop_unique_email... OK
Applying account.0005_emailaddress_idx_upper_email... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying django_mailman3.0003_sessions... OK
Applying django_q.0010_auto_20200610_0856... OK
Applying django_q.0011_auto_20200628_1055... OK
Applying django_q.0012_auto_20200702_1608... OK
Applying django_q.0013_task_attempt_count... OK
Applying django_q.0014_schedule_cluster... OK
Applying hyperkitty.0016_auto_20180309_0056... OK
Applying hyperkitty.0017_file_attachments... OK
Applying hyperkitty.0018_threadcategory_color... OK
Applying hyperkitty.0019_auto_20190127_null_description... OK
Applying hyperkitty.0020_auto_20190907_1927... OK
Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 261, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1050, "Table 'hyperkitty_mailinglist_moderators'
already exists")
Then dropping table hyperkitty_mailinglist_moderators.
(venv) mailman@new-mailman3 ~ (v3.1.1)> mailman-web migrate
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you
don't care about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't
care about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 261, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1050, "Table 'hyperkitty_mailinglist_owners'
already exists")
After dropping table hyperkitty_mailinglist_owners.
(venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you
don't care about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't
care about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 261, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1050, "Table 'hyperkitty_mailinglist_moderators'
already exists")
After dropping table hyperkitty_mailinglist_moderators.
(venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you
don't care about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't
care about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying hyperkitty.0021_add_owners_mods... OK
Applying hyperkitty.0022_mailinglist_archive_rendering_mode... OK
Applying hyperkitty.0023_alter_mailinglist_name... OK
Applying postorius.0004_create_email_template...Traceback (most recent
call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 261, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1050, "Table 'postorius_emailtemplate' already
exists")
After dropping table postorius_emailtemplate.
(venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you
don't care about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't
care about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying postorius.0004_create_email_template... OK
Applying postorius.0005_auto_20180707_1107... OK
Applying postorius.0006_auto_20180711_1359... OK
Applying postorius.0007_auto_20180712_0536... OK
Applying postorius.0008_auto_20190310_0717... OK
Applying postorius.0009_auto_20190508_1604... OK
Applying postorius.0010_auto_20190821_0621... OK
Applying postorius.0011_auto_20191109_1219... OK
Applying postorius.0012_auto_20200420_2136... OK
Applying postorius.0013_auto_20201116_0058... OK
Applying postorius.0014_auto_20210329_2248... OK
Applying postorius.0015_auto_20210619_0509... OK
Applying postorius.0016_auto_20210810_2157... OK
Applying postorius.0017_alter_emailtemplate_language... OK
Applying postorius.0018_alter_emailtemplate_language... OK
Applying socialaccount.0004_app_provider_id_settings... OK
Applying socialaccount.0005_socialtoken_nullable_app... OK
Applying socialaccount.0006_alter_socialaccount_extra_data... OK
At this point the errors remains about not reaching web part. So I started
mailman3-web.
Then I see these errors on var/log/mailman.log
Feb 19 15:35:14 2024 (543890) Traceback (most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/mailman_hyperkitty/__init__.py",
line 158, in _archive_message
url = self._send_message(mlist, msg)
File
"/local/mailman/venv/lib/python3.10/site-packages/mailman_hyperkitty/__init__.py",
line 228, in _send_message
raise ValueError(result.text)
ValueError: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
</body></html>
Feb 19 15:35:14 2024 (543890) HyperKitty failure on
http://127.0.0.1/archives/api/mailman/archive: <!DOCTYPE HTML PUBLIC
"-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
</body></html>
(404)
Feb 19 15:35:14 2024 (543890) Could not archive the message with id <
ff1707bb-d105-4f37-b3f8-9766b3563127(a)SESAMR604.domain.com>
Feb 19 15:35:14 2024 (543890) archiving failed, re-queuing (mailing-list
mylist.new-mailman.domain.com, message <
ff1707bb-d105-4f37-b3f8-9766b3563127(a)machine.domain.com>)
Feb 19 15:35:14 2024 (543890) Exception in the HyperKitty archiver:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
</body></html>
And I can't get access to the hyperkitty at all.
On var/logs/uwsgi-error.log:
--- Logging error ---
Traceback (most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 356, in pusher
task = SignedPackage.loads(task[1])
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/signing.py",
line 25, in loads
return signing.loads(
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
line 35, in loads
base64d = force_bytes(TimestampSigner(key, salt=salt).unsign(s,
max_age=max_age))
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
line 70, in unsign
result = super(TimestampSigner, self).unsign(value)
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
line 55, in unsign
raise BadSignature('Signature "%s" does not match' % sig)
django.core.signing.BadSignature: Signature "48do9gGvueBxakX_a4uNmSwD7cs"
does not match
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/logging/__init__.py", line 1100, in emit
msg = self.format(record)
File "/usr/lib/python3.10/logging/__init__.py", line 943, in format
return fmt.format(record)
File "/usr/lib/python3.10/logging/__init__.py", line 678, in format
record.message = record.getMessage()
File "/usr/lib/python3.10/logging/__init__.py", line 368, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
sys.exit(main())
File
"/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
line 90, in main
execute_from_command_line(sys.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/management/commands/qcluster.py",
line 22, in handle
q.start()
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 78, in start
self.sentinel.start()
File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/usr/lib/python3.10/multiprocessing/context.py", line 281, in _Popen
return Popen(process_obj)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in
__init__
self._launch(process_obj)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in
_launch
code = process_obj._bootstrap(parent_sentinel=child_r)
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in
_bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 168, in __init__
self.start()
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 172, in start
self.spawn_cluster()
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 248, in spawn_cluster
self.pusher = self.spawn_pusher()
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 201, in spawn_pusher
return self.spawn_process(pusher, self.task_queue, self.event_out,
self.broker)
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 197, in spawn_process
p.start()
File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/usr/lib/python3.10/multiprocessing/context.py", line 281, in _Popen
return Popen(process_obj)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in
__init__
self._launch(process_obj)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in
_launch
code = process_obj._bootstrap(parent_sentinel=child_r)
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in
_bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 358, in pusher
logger.error(e, traceback.format_exc())
Message: BadSignature('Signature "48do9gGvueBxakX_a4uNmSwD7cs" does not
match')
Arguments: ('Traceback (most recent call last):\n File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
line 356, in pusher\n task = SignedPackage.loads(task[1])\n File
"/local/mailman/venv/lib/python3.10/sit
e-packages/django_q/signing.py", line 25, in loads\n return
signing.loads(\n File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
line 35, in loads\n base64d = force_bytes(TimestampSigner(key, salt=sa
lt).unsign(s, max_age=max_age))\n File
"/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
line 70, in unsign\n result = super(TimestampSigner,
self).unsign(value)\n File "/local/mailman/venv/lib/python3.10
/site-packages/django_q/core_signing.py", line 55, in unsign\n raise
BadSignature(\'Signature "%s" does not match\' %
sig)\ndjango.core.signing.BadSignature: Signature
"48do9gGvueBxakX_a4uNmSwD7cs" does not match\n',)
I'm using uwsgi via apache:
(venv) mailman@new-mailman3 ~ (v3.1.1)> more
/etc/apache2/conf-enabled/mailman3.conf
Alias /mailman3/favicon.ico /local/mailman/web/static/favicon.ico
Alias /mailman/favicon.ico /local/mailman/web/static/favicon.ico
Alias /favicon.ico /local/mailman/web/static/favicon.ico
Alias /mailman3/static /local/mailman/web/static
Alias /mailman/static /local/mailman/web/static
Alias /static /local/mailman/web/static
<Directory "/local/mailman/web/static">
Require all granted
</Directory>
<IfModule mod_proxy_uwsgi.c>
ProxyPass /mailman3/favicon.ico !
ProxyPass /mailman/favicon.ico !
ProxyPass /favicon.ico !
ProxyPass /mailman3/static !
ProxyPass /mailman/static !
ProxyPass /static !
ProxyPass /mailman3
unix:/local/mailman/var/uwsgi.sock|uwsgi://localhost/
ProxyPass /mailman
unix:/local/mailman/var/uwsgi.sock|uwsgi://localhost/
#ProxyPass /mailman3 http://localhost:8000/ timeout=180
#ProxyPass /mailman http://localhost:8000/ timeout=180
#ProxyPass / http://localhost:8000/ timeout=180
</IfModule>
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
On Fri, 16 Feb 2024 at 22:39, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 2/16/24 07:12, Helio Loureiro wrote:
>
> > - start mailman3 and mailman-3web
> > - run mailman3-web migrate
>
> You should run mailman3-web migrate when mailman-3web is not running.
>
> > django.db.utils.OperationalError: (1050, "Table
> > 'hyperkitty_mailinglist_moderators' already exists")
> >
> > I tried to drop the table, but then it complained about another one. I
> > dropped all the complained tables and as result hypperkitty never
> > started. So I went back to see why the first error happened.
>
> The errors occurred because you had recently added tables in your
> database so they couldn't be added by the migrations. Dropping the
> tables was the correct thing to do.
>
> Did you ultimately successfully run all the migrations?
>
> When you say `hypperkitty never started`, what exactly does that mean?
> What happens when you try to access the archives? Can you access
> postorius? Are any errors logged in mailman3-web's log?
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to helio(a)loureiro.eng.br
>
2 years, 8 months
Re: Starting from scratch
by Mark Sapiro
On 04/09/2018 10:01 AM, Benjamin Ragheb wrote:
>
> The two alternatives I am weighing right now are:
>
> 1. Stay the course, and see if I can fix this by learning more about Python, Pip, Django and tinkering with the configuration.
>
> 2. Ride the bleeding edge, and install postorius and hyperkitty from git. (I'll probably stick with Mailman Core from Pip, since that seemed to work just fine.)
>
> Right now I'm leaning toward #2, because it means I'll be in a better position to patch things if I need to, and it seems like Mailman 3 is still rough enough that I will need to. Also, it looks like the master branches of both postorius and hyperkitty are now Python 3 compatible, which could save me from managing two different versions of Python.
>
> Which is the least crazy option?
If you are comfortable with 2), I think that's the better option. After
PyCon in May when we release 3.2 and the rest of the suite, you should
be able to install everything with pip and have it work. I'm not sure
what the issues are now with a pip install.
There is a 3rd option. See
<https://packages.debian.org/stretch-backports/mailman3-full>
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
8 years, 7 months
Re: [Django] ERROR (EXTERNAL IP): Internal Server Error: /hyperkitty/list/direttivo@catania.linux.it/top-threads
by Mark Sapiro
On 2/28/21 10:17 AM, blackout69 wrote:
> Hello everybody,I received this error message and from that moment the
> maiilan3 has stopped working.I restarted the following services mailman3
> and mailman3-web and everything started working fine.Can you tell me
> what is the problem my django server has encountered?
It looks to me like a race condition of some kind or possibly an
incompletely deleted thread. Have you deleted any threads?
I'm curious as to how you happen to be going to
<https://lists.catania.linux.it/hyperkitty/list/direttivo@catania.linux.it/t…>
as opposed to
<https://lists.catania.linux.it/hyperkitty/list/direttivo@catania.linux.it/#…>.
I know that ULS and similar ones are in HyperKitty's urls.py, but I
don't see them exposed on any pages.
Anyway, when you say "from that moment the maiilan3 has stopped
working." (sic) exactly what stopped working. Did core stop delivering
messages? Did Postorius stop working. If so, this all may be related to
some underlying database issue or something like that that was fixed by
the restarts. I don't see how this HyperKitty error could affect
Postorius or Mailman core.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
5 years, 8 months
Re: Message-Footer after migrating to Mailman3
by christopher.claus＠tgcamberg1848.de
Mark Sapiro wrote:
> > ERROR 2021-05-24 16:22:22,895 1648 django.security.DisallowedHost Invalid HTTP_HOST header: 'tgc_mailman_web:8000'. The domain name provided is not valid according to RFC 1034/1035.
> > WARNING 2021-05-24 16:22:22,896 1648 django.request Bad Request: /postorius/api/templates/list/homepage.gruppe.tgcamberg1848.de/list:member:regular:footer
> > It appears you created the new template with Postorius, but there is an
> issue for Postorius templates. Possibly your Django setting for
> POSTORIUS_TEMPLATE_BASE_URL needs to be adjusted.
> > Obviously I have a problem with the hostname. I set
> > [code]
> > POSTORIUS_TEMPLATE_BASE_URL=http://tgc_mailman_web:8000
> > [/code]
> > where tgc_mailman_web is my container name. This URL ist stored in the DB-table "template" for the name=list:member:regular:footer. A call to my fqdn or to localhost
> > [code]
> > wget http://localhost:8000/postorius/api/templates/list/homepage.gruppe.tgcamberg...
> > [/code]
> > will return the footer itself - this works fine. Therefore, should i set POSTORIUS_TEMPLATE_BASE_URL to http://localhost:8000?
> > If that works, yes.
Hi Mark,
thanks for your reply and excuse my delay. I was very busy in the last weeks but made some tests. Newly created templates with POSTORIUS_TEMPLATE_BASE_URL=http://localhost:8000 did not work. I changed it to my fqdn - at the moment I am able to access the template via this URl (e.G. http://lists.example.de:8000/postorius/api/templates/list/homepage.lists.ex…)
But if I try to send an email I got the following error:
Jul 09 16:00:37 2021 (27) Uncaught runner exception: HTTPConnectionPool(host='lists.example.de', port=8000): Max retries exceeded with url: /postorius/api/templates/list/homepage.lists.example.de/list:admin:action:post (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe80eaaec40>: Failed to establish a new connection: [Errno 111] Connection refused'))
Jul 09 16:00:37 2021 (27) Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.8/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 187, in connect
conn = self._new_conn()
File "/usr/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe80eaaec40>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
retries = retries.increment(
File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='lists.example.de', port=8000): Max retries exceeded with url: /postorius/api/templates/list/lists.example.de/list:admin:action:post (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe80eaaec40>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/mailman/core/runner.py", line 173, in _one_iteration
self._process_one_file(msg, msgdata)
File "/usr/lib/python3.8/site-packages/mailman/core/runner.py", line 266, in _process_one_file
keepqueued = self._dispose(mlist, msg, msgdata)
File "/usr/lib/python3.8/site-packages/mailman/runners/incoming.py", line 79, in _dispose
process(mlist, msg, msgdata, start_chain)
File "/usr/lib/python3.8/site-packages/mailman/core/chains.py", line 79, in process
link.function(mlist, msg, msgdata)
File "/usr/lib/python3.8/site-packages/mailman/chains/hold.py", line 232, in _process
template = getUtility(ITemplateLoader).get(
File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 188, in get
contents = getUtility(ITemplateManager).get(
File "/usr/lib/python3.8/site-packages/mailman/database/transaction.py", line 85, in wrapper
return function(args[0], config.db.store, *args[1:], **kws)
File "/usr/lib/python3.8/site-packages/mailman/model/template.py", line 110, in get
contents = protocols.get(actual_uri, **auth)
File "/usr/lib/python3.8/site-packages/mailman/utilities/protocols.py", line 38, in get
response = requests.get(url, timeout=REQUEST_TIMEOUT, **kws)
File "/usr/lib/python3.8/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='lists.example.de', port=8000): Max retries exceeded with url: /postorius/api/templates/list/lists.example.de/list:admin:action:post (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe80eaaec40>: Failed to establish a new connection: [Errno 111] Connection refused'))
I set POSTORIUS_TEMPLATE_BASE_URL=lists.example.de:8000 and I have no idea, why the connection is refused. Do you see any configuration error d or have another idea?
Best regards,
chrclaus
5 years, 4 months
Re: Bounce counter visibility for List Admins
by Mark Sapiro
On 5/19/21 4:10 AM, smorytko(a)gmail.com wrote:
> As you may know, we use MM3 but the Affinity/Empathy flavor and we only have List Administrator using the Postorious/Affinity. Unlike many here, we have no access to any server-based files.
>
> Can someone tell me if Postorious gives list administrators visibility into the bounce counters?
User's bounce scores are not available in REST so neither Affinity nor
Postorius has access to them.
> While List Admins get to set various bounce counters we have no visibility of those counters until, all of a sudden, an entire source of subscribers hits the bounce threshold and gets bounced off the list. It would be really nice to be able to see the bounce counters and possibly head off trouble before users/subscriber mail hosts. It can serve as an effective early warning system that something is wrong.
>
> Assuming Postorious cannot, is there another way List Admins can inspect bounce counters? Email command?
In the next release of Mailman core (3.3.5), lists will have a
`bounce_notify_owner_on_bounce_increment` setting which if True, will
send to the list admins a notice including the triggering DSN when a
user's bounce score is incremented. This setting will also be exposed in
Postorius 1.3.5.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
5 years, 5 months
Re: seems solved (was: Re: error changed after restart)
by Guillermo Hernandez (Oldno7)
On 8/2/21 11:26, Guillermo Hernandez (Oldno7) via Mailman-users wrote:
> On 7/2/21 19:33, Guillermo Hernandez (Oldno7) via Mailman-users wrote:
>> On 7/2/21 19:04, Abhilash Raj wrote:
>
> Well... I did reinstall all mailman relationed apps with a
>
> pip install --upgrade --force-reinstall --no-deps --no-cache-dir
> django-mailman3 mailman mailman-hyperkitty mailmanclient
>
> Restarted thee apache24 server...
>
> Aaaaaand closed de admin session and reopened it... (just for a "what
> if" sensation).
>
> I can not tell what have been the solution here, but now I can access
> the administrative pages and attend de pending subscriptions and
> discard retained messages.
>
> Thanks Abhilash and Mark for your support.
Some things did not work well enough after that (the subscriptions
pending approval, between others), and then I did an
pip install --upgrade --force-reinstall --no-deps --no-cache-dir
django-mailman3 mailman mailman-hyperkitty mailmanclient postorius django
(adding postorius and django to my previous reinstall)
And a
su -m mailman3 -c "python3 manage.py collectstatic"
in the mailman dir (mailman3 is the user who exectues all mailman
related things)
and again a
su -m mailman3 -c "python3 manage.py migrate"
And it seems there were things pendig to update...
Restarted the server and now it seems all is OK.
Thanks all and sorry for the noise.
5 years, 9 months
Re: User management page [was: Changing subscription address]
by Abhilash Raj
> On Mar 22, 2021, at 7:56 AM, Stephen J. Turnbull <turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote:
>
> Abhilash Raj writes:
>
>> Typo :( Page in Postorius is what I meant to say.
>
> :D No problemo, mon, I just did it too ->
>
>> Is that necessarily required? Every user in Django has a corresponding
>> user in Core, but the converse isn’t true and I am not sure if that is even
>> required.
>
> I meant "accessed from Postorius or HyperKitty." :-)
>
>> You will be able to see:
>> - Does the user have a Django account?
>> - All Subscriptions for the User
>> - All addresses of the User
>>
>> You will be able to perform following actions:
>> - Add or remove a user’s address
>> - Verify or un-verify an address
>> - Set any address as primary for the user
>> - Update display name of the user
>> - Update address on any of the subscriptions
>
> I don't think we *need* a Django account for what I'm thinking of. I
> do think that creating the account on User access from Postorius might
> make the implementation logic simpler.
>
>> What I am on fence about:
>> - Provide a full blown preferences menu to the admin for that user. This
>> will also make the UI extremely complex to see and implement.
>
> Wait to decide on that until I have a mockup or maybe a full MR.
>
>> But, when you have an account in Django with a different email
>> (email2) and a different Mailman user (user2) and you add a second
>> email (email1) to your account and are able to prove ownership by
>> verifying the address, Django will ask Core to add that second address
>> (email1) to your Mailman user (user2) and absorb “user1”. This
>> process is already implemented.
>
> Good.
>
>> However, if two Django accounts are created for “user1” and “user2”
>> then situation becomes more tricky and currently we don’t have a
>> way to resolve that short of delete one of the accounts and add the
>> addresses to other account.
>
> Delete "that" and add its attributes to "this" is what I had in mind.
We could automate the absorb other account and copy all settings
and subscriptions over in a user based workflow. I don’t see a lot of
unknowns in implementing that.
You have to manually do it today, and deleting account will delete
subscriptions so when you add them to your other account you have
to resubscribe. So, the path could really use some changes. I think
I have a ticket open in P to allow deleting P account but not delete
the Mailman User and subscriptions, which would make this manual
process much better.
--
thanks,
Abhilash Raj (maxking)
5 years, 7 months
Re: Mailman3 on virtual domains
by Mark Sapiro
On 10/7/23 09:25, Lars Bjørndal wrote:
> [Mark]
>
> [...]
>
>> See https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h…
>
> Ok, so I have to create an alias_domain in Mailman. Is it possible to do that from the web interace?
It has been in postorius since 1.2.0.
> I find an Edit button for each domain, and then I may change name and description. But I can't find an option to add an alias_domain. As I'm blind, and using a text only browser, some information may have been missing. Or can I do it from the command line?
If your Postorius is >=1.2.0, there should be 3 fields on the Edit
domain page - Description, Alias Domain and Web Host. The whole page
looks something like:
Edit domain example.com
Description
Alias Domain
Normally empty. Used only for unusual Postfix configs.
Web Host
The domain from which you want the web UI to be served from. This can be
same or different from the Mail Host. You can edit the list of available
web hosts here.
You can also set an alias_domain in `mailman shell`. E.g.,
```
>>> domain = getUtility(IDomainManager).get('example.com')
>>> domain.alias_domain = 'x.example.com'
>>> commit()
```
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
3 years, 1 month
Re: Core, Hyperkitty and Postorius releases
by Odhiambo Washington
On Fri, May 14, 2021 at 11:13 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 5/14/21 12:50 PM, Odhiambo Washington wrote:
> > On Fri, May 14, 2021 at 7:04 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> >
> >> On 5/14/21 5:02 AM, Odhiambo Washington wrote:
> >>> On Fri, May 14, 2021 at 2:52 PM Jeff Steele <jeff(a)steele.com> wrote:
> >>>
> >>>>
> >>>>
> >>>> On May 14, 2021, at 7:49 AM, Odhiambo Washington <odhiambo(a)gmail.com>
> >>>> wrote:
> >>>>
> >>>> /bin/sh: sass: Permission denied*
> ...
> >> Try `which sass`
> >>
> >
> > root@gw:/usr/home/wash # su mailman3
> > (venv) [mailman3@gw /usr/home/wash]$ which sass
> > (venv) [mailman3@gw /usr/home/wash]$
> >
> > There is no sass.
>
>
> There has to be. You can't get a permissions error on sass if it doesn't
> exist (well, maybe if you don't have permission to search a
> superordinate directory, but that's unlikely).
>
> Try `find /usr -name sass`
>
/usr/local/include/sass
That is the only thing found that is in the base system.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v "^$|^.*#" :-)
5 years, 5 months
Re: hyperkitty won't list more than 5 posts and shows empty grey Activity Summary box
by Abhilash Raj
On Wed, 9 Aug 2017 15:45:48 -0700
Mark Sapiro <mark(a)msapiro.net> wrote:
> On 08/09/2017 02:35 PM, Abhilash Raj wrote:
> > Hi,
> >
> > On Wed, 9 Aug 2017 13:42:20 -0700
> > Mark Sapiro <mark(a)msapiro.net> wrote:
> [...]
> >
> > I wasn't aware that Core needed cron jobs too, is that true for 3.1 release as
> > well?
>
>
> It's only that one and whether it's needed or not depends on list
> configuration. The list has attributes digest_send_periodic and
> digest_size_threshold. When a post is received it is added to the list's
> digest mbox, and if that causes the size of the mbox to reach
> digest_size_threshold K bytes, a digest is sent. Additionally, if
> digest_send_periodic is true, a digest is sent periodically whether or
> not it has reached digest_size_threshold.
>
> However, the sending of periodic digests depends on cron running
> something periodically to actually trigger the sending.
Can this be done using the REST API? If not, maybe we should do that so that
Postorius can take of the periodic tasks?
I am thinking mostly about the container images because the mailman-core
container runs nothing else other than Core. I'd have to another cron for
this which isn't a lot of overhead but since mailman-web container already has
periodic tasks, it would be great if I could trigger it from there.
>
> Note: I've just looked at the code and there seems to be an issue with
> using 'mailman digests --send' to send the periodic digests. It appears
> that if a list's digest mbox has any messages, 'mailman digests --send'
> will send a digest even if digest_send_periodic is false. I've just
> reported <https://gitlab.com/mailman/mailman/issues/384> for this issue.
Also, it seems that Postorius doesn't have any way to set
`digest_send_periodic` and only has a setting to set the max_threshold for the
digest. I opened another issue
(https://gitlab.com/mailman/postorius/issues/211) to add this feature.
--
thanks,
Abhilash Raj
9 years, 3 months