[MM3-users]Re: mailman 2 vs. mailman 3: what happened to multipart/digest?
by Simon Hanna
On 05/12/2016 09:26 PM, Barry Warsaw wrote:
> On May 12, 2016, at 12:18 PM, Mark Sapiro wrote:
>
>> It is the intent that Postorius will eventually expose all these
>> settings. We're not there yet. At present you could possibly do this by
>> interacting with mailmanclient, although I'm not certain the settings are
>> exposed there.
>
> I'm not sure if they're exposed in mailmanclient, but they are exposed in the
> core's REST API, on the member resource. It's the `delivery_mode` attribute
> and it takes as legal values one of 'regular', 'plaintext_digests',
> 'mime_digests', and 'summary_digests' (IOW, the enum item names for the
> DeliveryMode enum).
They are exposed in mailmanclient and are used in postorius. They can be
found in the users personal subscription settings
https://lists.mailman3.org/mailman3/accounts/mailmansettings/
>
> Cheers,
> -Barry
> _______________________________________________
> Mailman-users mailing list
> mailman-users(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
10 years
Re: Migration from old server to new server failed
by Helio Loureiro
Hi,
Any suggestions here?
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
On Mon, 19 Feb 2024 at 14:44, Helio Loureiro <helio(a)loureiro.eng.br> wrote:
> Hi,
>
> I repeated the steps today.
>
> First I waited to end the errors raised because the mailman3-web wasn't
> running.
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
> line 793, in urlopen
> response = self._make_request(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
> line 496, in _make_request
> conn.request(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
> line 400, in request
> self.endheaders()
> File "/usr/lib/python3.10/http/client.py", line 1278, in endheaders
> self._send_output(message_body, encode_chunked=encode_chunked)
> File "/usr/lib/python3.10/http/client.py", line 1038, in _send_output
> self.send(msg)
> File "/usr/lib/python3.10/http/client.py", line 976, in send
> self.connect()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
> line 238, in connect
> self.sock = self._new_conn()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connection.py",
> line 213, in _new_conn
> raise NewConnectionError(
> urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection
> object at 0x7f0fdc2f72b0>: Failed to establish a new connection: [Errno
> 111] Connection refused
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/requests/adapters.py",
> line 486, in send
> resp = conn.urlopen(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/connectionpool.py",
> line 847, in urlopen
> retries = retries.increment(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/urllib3/util/retry.py",
> line 515, in increment
> raise MaxRetryError(_pool, url, reason) from reason # type:
> ignore[arg-type]
> urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1',
> port=8000): Max retries exceeded with url: /archives/api/mailman/archive
> (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at
> 0x7f0fdc2f72b0>: Failed to establish a new connection: [Errno 111]
> Connection refused'))
>
> Which seems to keep forever. So eventually I gave up and moved forward.
> And I changed to use the uwsgi socket instead of the direct port.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|SIGINT]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying account.0003_alter_emailaddress_create_unique_verified_email...
> OK
> Applying account.0004_alter_emailaddress_drop_unique_email... OK
> Applying account.0005_emailaddress_idx_upper_email... OK
> Applying admin.0003_logentry_add_action_flag_choices... OK
> Applying auth.0009_alter_user_last_name_max_length... OK
> Applying auth.0010_alter_group_name_max_length... OK
> Applying auth.0011_update_proxy_permissions... OK
> Applying auth.0012_alter_user_first_name_max_length... OK
> Applying django_mailman3.0003_sessions... OK
> Applying django_q.0010_auto_20200610_0856... OK
> Applying django_q.0011_auto_20200628_1055... OK
> Applying django_q.0012_auto_20200702_1608... OK
> Applying django_q.0013_task_attempt_count... OK
> Applying django_q.0014_schedule_cluster... OK
> Applying hyperkitty.0016_auto_20180309_0056... OK
> Applying hyperkitty.0017_file_attachments... OK
> Applying hyperkitty.0018_threadcategory_color... OK
> Applying hyperkitty.0019_auto_20190127_null_description... OK
> Applying hyperkitty.0020_auto_20190907_1927... OK
> Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
> last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table
> 'hyperkitty_mailinglist_moderators' already exists")
>
> Then dropping table hyperkitty_mailinglist_moderators.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1)> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
> last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table 'hyperkitty_mailinglist_owners'
> already exists")
>
> After dropping table hyperkitty_mailinglist_owners.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
> last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table
> 'hyperkitty_mailinglist_moderators' already exists")
>
> After dropping table hyperkitty_mailinglist_moderators.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying hyperkitty.0021_add_owners_mods... OK
> Applying hyperkitty.0022_mailinglist_archive_rendering_mode... OK
> Applying hyperkitty.0023_alter_mailinglist_name... OK
> Applying postorius.0004_create_email_template...Traceback (most recent
> call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
> line 87, in _execute
> return self.cursor.execute(sql)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
> line 75, in execute
> return self.cursor.execute(query, args)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 179, in execute
> res = self._query(mogrified_query)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
> 330, in _query
> db.query(q)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
> line 261, in query
> _mysql.connection.query(self, query)
> MySQLdb.OperationalError: (1050, "Table 'postorius_emailtemplate' already
> exists")
>
> After dropping table postorius_emailtemplate.
>
> (venv) mailman@new-mailman3 ~ (v3.1.1) [0|1]> mailman-web migrate
> System check identified some issues:
>
> WARNINGS:
> account.EmailAddress: (models.W036) MariaDB does not support unique
> constraints with conditions.
> HINT: A constraint won't be created. Silence this warning if you
> don't care about it.
> account.EmailAddress: (models.W043) MariaDB does not support indexes on
> expressions.
> HINT: An index won't be created. Silence this warning if you don't
> care about it.
> Operations to perform:
> Apply all migrations: account, admin, auth, contenttypes,
> django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> socialaccount
> Running migrations:
> Applying postorius.0004_create_email_template... OK
> Applying postorius.0005_auto_20180707_1107... OK
> Applying postorius.0006_auto_20180711_1359... OK
> Applying postorius.0007_auto_20180712_0536... OK
> Applying postorius.0008_auto_20190310_0717... OK
> Applying postorius.0009_auto_20190508_1604... OK
> Applying postorius.0010_auto_20190821_0621... OK
> Applying postorius.0011_auto_20191109_1219... OK
> Applying postorius.0012_auto_20200420_2136... OK
> Applying postorius.0013_auto_20201116_0058... OK
> Applying postorius.0014_auto_20210329_2248... OK
> Applying postorius.0015_auto_20210619_0509... OK
> Applying postorius.0016_auto_20210810_2157... OK
> Applying postorius.0017_alter_emailtemplate_language... OK
> Applying postorius.0018_alter_emailtemplate_language... OK
> Applying socialaccount.0004_app_provider_id_settings... OK
> Applying socialaccount.0005_socialtoken_nullable_app... OK
> Applying socialaccount.0006_alter_socialaccount_extra_data... OK
>
> At this point the errors remains about not reaching web part. So I
> started mailman3-web.
>
> Then I see these errors on var/log/mailman.log
>
> Feb 19 15:35:14 2024 (543890) Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_hyperkitty/__init__.py",
> line 158, in _archive_message
> url = self._send_message(mlist, msg)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_hyperkitty/__init__.py",
> line 228, in _send_message
> raise ValueError(result.text)
> ValueError: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL was not found on this server.</p>
> <hr>
> <address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
> </body></html>
>
> Feb 19 15:35:14 2024 (543890) HyperKitty failure on
> http://127.0.0.1/archives/api/mailman/archive: <!DOCTYPE HTML PUBLIC
> "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL was not found on this server.</p>
> <hr>
> <address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
> </body></html>
> (404)
> Feb 19 15:35:14 2024 (543890) Could not archive the message with id <
> ff1707bb-d105-4f37-b3f8-9766b3563127(a)SESAMR604.domain.com>
> Feb 19 15:35:14 2024 (543890) archiving failed, re-queuing (mailing-list
> mylist.new-mailman.domain.com, message <
> ff1707bb-d105-4f37-b3f8-9766b3563127(a)machine.domain.com>)
> Feb 19 15:35:14 2024 (543890) Exception in the HyperKitty archiver:
> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
> <html><head>
> <title>404 Not Found</title>
> </head><body>
> <h1>Not Found</h1>
> <p>The requested URL was not found on this server.</p>
> <hr>
> <address>Apache/2.4.52 (Ubuntu) Server at 127.0.0.1 Port 80</address>
> </body></html>
>
> And I can't get access to the hyperkitty at all.
>
> On var/logs/uwsgi-error.log:
>
> --- Logging error ---
> Traceback (most recent call last):
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 356, in pusher
> task = SignedPackage.loads(task[1])
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/signing.py",
> line 25, in loads
> return signing.loads(
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 35, in loads
> base64d = force_bytes(TimestampSigner(key, salt=salt).unsign(s,
> max_age=max_age))
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 70, in unsign
> result = super(TimestampSigner, self).unsign(value)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 55, in unsign
> raise BadSignature('Signature "%s" does not match' % sig)
> django.core.signing.BadSignature: Signature "48do9gGvueBxakX_a4uNmSwD7cs"
> does not match
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "/usr/lib/python3.10/logging/__init__.py", line 1100, in emit
> msg = self.format(record)
> File "/usr/lib/python3.10/logging/__init__.py", line 943, in format
> return fmt.format(record)
> File "/usr/lib/python3.10/logging/__init__.py", line 678, in format
> record.message = record.getMessage()
> File "/usr/lib/python3.10/logging/__init__.py", line 368, in getMessage
> msg = msg % self.args
> TypeError: not all arguments converted during string formatting
> Call stack:
> File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
> sys.exit(main())
> File
> "/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
> line 90, in main
> execute_from_command_line(sys.argv)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
> line 446, in execute_from_command_line
> utility.execute()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
> line 440, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 402, in run_from_argv
> self.execute(*args, **cmd_options)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
> line 448, in execute
> output = self.handle(*args, **options)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/management/commands/qcluster.py",
> line 22, in handle
> q.start()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 78, in start
> self.sentinel.start()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
> self._popen = self._Popen(self)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in
> _Popen
> return _default_context.get_context().Process._Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 281, in
> _Popen
> return Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in
> __init__
> self._launch(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in
> _launch
> code = process_obj._bootstrap(parent_sentinel=child_r)
> File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in
> _bootstrap
> self.run()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
> self._target(*self._args, **self._kwargs)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 168, in __init__
> self.start()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 172, in start
> self.spawn_cluster()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 248, in spawn_cluster
> self.pusher = self.spawn_pusher()
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 201, in spawn_pusher
> return self.spawn_process(pusher, self.task_queue, self.event_out,
> self.broker)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 197, in spawn_process
> p.start()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
> self._popen = self._Popen(self)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in
> _Popen
> return _default_context.get_context().Process._Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/context.py", line 281, in
> _Popen
> return Popen(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in
> __init__
> self._launch(process_obj)
> File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in
> _launch
> code = process_obj._bootstrap(parent_sentinel=child_r)
> File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in
> _bootstrap
> self.run()
> File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
> self._target(*self._args, **self._kwargs)
> File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 358, in pusher
> logger.error(e, traceback.format_exc())
> Message: BadSignature('Signature "48do9gGvueBxakX_a4uNmSwD7cs" does not
> match')
> Arguments: ('Traceback (most recent call last):\n File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/cluster.py",
> line 356, in pusher\n task = SignedPackage.loads(task[1])\n File
> "/local/mailman/venv/lib/python3.10/sit
> e-packages/django_q/signing.py", line 25, in loads\n return
> signing.loads(\n File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 35, in loads\n base64d = force_bytes(TimestampSigner(key, salt=sa
> lt).unsign(s, max_age=max_age))\n File
> "/local/mailman/venv/lib/python3.10/site-packages/django_q/core_signing.py",
> line 70, in unsign\n result = super(TimestampSigner,
> self).unsign(value)\n File "/local/mailman/venv/lib/python3.10
> /site-packages/django_q/core_signing.py", line 55, in unsign\n raise
> BadSignature(\'Signature "%s" does not match\' %
> sig)\ndjango.core.signing.BadSignature: Signature
> "48do9gGvueBxakX_a4uNmSwD7cs" does not match\n',)
>
> I'm using uwsgi via apache:
>
> (venv) mailman@new-mailman3 ~ (v3.1.1)> more
> /etc/apache2/conf-enabled/mailman3.conf
> Alias /mailman3/favicon.ico /local/mailman/web/static/favicon.ico
> Alias /mailman/favicon.ico /local/mailman/web/static/favicon.ico
> Alias /favicon.ico /local/mailman/web/static/favicon.ico
> Alias /mailman3/static /local/mailman/web/static
> Alias /mailman/static /local/mailman/web/static
> Alias /static /local/mailman/web/static
>
> <Directory "/local/mailman/web/static">
> Require all granted
> </Directory>
>
> <IfModule mod_proxy_uwsgi.c>
> ProxyPass /mailman3/favicon.ico !
> ProxyPass /mailman/favicon.ico !
> ProxyPass /favicon.ico !
> ProxyPass /mailman3/static !
> ProxyPass /mailman/static !
> ProxyPass /static !
> ProxyPass /mailman3
> unix:/local/mailman/var/uwsgi.sock|uwsgi://localhost/
> ProxyPass /mailman
> unix:/local/mailman/var/uwsgi.sock|uwsgi://localhost/
> #ProxyPass /mailman3 http://localhost:8000/ timeout=180
> #ProxyPass /mailman http://localhost:8000/ timeout=180
> #ProxyPass / http://localhost:8000/ timeout=180
> </IfModule>
>
> Best Regards,
> Helio Loureiro
> https://helio.loureiro.eng.br
> https://github.com/helioloureiro
> https://mastodon.social/@helioloureiro
>
>
> On Fri, 16 Feb 2024 at 22:39, Mark Sapiro <mark(a)msapiro.net> wrote:
>
>> On 2/16/24 07:12, Helio Loureiro wrote:
>>
>> > - start mailman3 and mailman-3web
>> > - run mailman3-web migrate
>>
>> You should run mailman3-web migrate when mailman-3web is not running.
>>
>> > django.db.utils.OperationalError: (1050, "Table
>> > 'hyperkitty_mailinglist_moderators' already exists")
>> >
>> > I tried to drop the table, but then it complained about another one. I
>> > dropped all the complained tables and as result hypperkitty never
>> > started. So I went back to see why the first error happened.
>>
>> The errors occurred because you had recently added tables in your
>> database so they couldn't be added by the migrations. Dropping the
>> tables was the correct thing to do.
>>
>> Did you ultimately successfully run all the migrations?
>>
>> When you say `hypperkitty never started`, what exactly does that mean?
>> What happens when you try to access the archives? Can you access
>> postorius? Are any errors logged in mailman3-web's log?
>>
>> --
>> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>> San Francisco Bay Area, California better use your sense - B. Dylan
>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>> Archived at:
>> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>>
>> This message sent to helio(a)loureiro.eng.br
>>
>
2 years, 2 months
Re: Mailman not able to process "big" requests
by Abhilash Raj
On Mon, Apr 23, 2018, at 10:41 PM, Jonas Felber wrote:
> Hi,
>
> We migrated from mailman2 to mailman3 a week ago. Since then we are
> unable to send messages the our students through the mailing list.
> Here are some symptoms:
>
> 1. I go to mass subscribe and subscribe 400 emails
> 2. I click subscribe, the browser (in my case safari) tries to load the
> page for a few seconds
> 3. The request times out
> 4. Once the service becomes responsible again, I see that only part of
> my emails got subscribed (~50 in this case).
>
> For me this really looks like the emails are sent on the "web-thread"
> that gets killed once the request times out.
> Looking at this
> https://gitlab.com/mailman/postorius/blob/master/src/postorius/views/list.p…
> <https://gitlab.com/mailman/postorius/blob/master/src/postorius/views/list.p…>
> seems to confirm my suspicions.
Yes, you are correct. Mass Subscription current just subscribes each address one-by-one and thus times-out.
I have created an issue for this https://gitlab.com/mailman/postorius/issues/264.
>
> Some quick overview of our infrastructure: The TCP endpoints are:
> browser -> haproxy in kubernetes-entry-router -> nginx in kubernetes-
> ingress -> nginx in mailman -> mailman
I would very much appreciate if you'd share you configuration and/or a blog post on your setup. I have been trying to set Mailman use Kubernetes for a time, but haven't had time to do it.
> We mitigated the issue by giving it more CPU power and increasing the
> timeouts in one of the nginx endpoints.
> We also have a problem with sending out emails to "large" (>50) groups,
> as they seem to not getting sent out at all. Do you have any idea what
> the problem here could be?
Which MTA are you using? What do you see in mailman's logs and MTA's logs?
I haven't seen this kind of problem before with Mailman 3. This very list is hosted using Mailman 3 and I believe has more than 50 subscribers.
>
> We would be really glad if you can help us getting our mailman up and
> running, as I currently have to send emails out by "hand".
> I will happily provide you with more details about our configuration,
> just let me know what you need.
>
> Kind regards,
>
> Jonas
>
> Jonas Felber <jonas.felber(a)vis.ethz.ch <mailto:jonas.felber@vis.ethz.ch>>
> Aktuar & CTF Präsident
>
> VIS - Verein der Informatik Studierenden an der ETH Zürich
> CAB E 31, Universitätstr. 6, ETH Zentrum, CH-8092 Zürich
> https://www.vis.ethz.ch <https://www.vis.ethz.ch/>
> _______________________________________________
> Mailman-users mailing list
> mailman-users(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Email had 1 attachment:
> + signature.asc
> 1k (application/pgp-signature)
--
Abhilash Raj
maxking(a)asynchronous.in
8 years
Re: Where should Postfix, Django, Hyperkitty, Postorius, Dart-Sass, etc. be installed if MM3 is in a python virtual environment?
by Andrew Hodgson
Ron / BCLUG <admin(a)bclug.ca> wrote:
>Keith Christian wrote on 2024-05-24 16:41:
>> pip install wheel mailman psycopg2-binary
>> Should, or must, any of Django, Hyperkitty, Postorius, Dart-Sass, or
>> any other supporting programs be installed into this virtual
>> environment ?
>As I understand it, mailman-hyperkitty get installed into the venv, but django and postorius are pulled in as a dependency of mailman-web (I
>believe?)
Yes that is correct. I've actually a slightly different approach and when creating the env choose to bring in system packages as well:
python3 -m venv --system-site-packages --symlinks venv
The packages I install on a fresh system are:
sudo apt install python3-dev python3-wheel python3-psycopg2 sassc python3-rcssmin python3-venv python3-certbot-nginx postgresql exim4-daemon-light
Then in venv I just run
pip install mailman mailman-hyperkitty mailman-web uwsgi
My goal here is that I want as minimum compilation tools on the system as possible. I installed the Ubuntu version of the psycopg2 package as it's the same version of the Postgresql software also maintained by Ubuntu. I seem to remember rcsmin needing compilation tools so installed the Ubuntu maintained package. I typically don't install Links on my systems but remember it was a requirement at some point so will probably include it on a new Ubuntu 24.04 host I am planning. The Certbott installation does itself pull in a few Python libraries that Mailman uses so when installing Mailman in the venv it doesn't have to pull as many packages. I did try and see what packages I could include from the system in this way but when we start adding more packages Apt starts to get crazy with dependencies, so I made the goal of just pulling in packages which required compilation.
Originally I used Gunicorn as the wsgi backend for the Django stuff from Nginx. This is because Gunicorn gets installed as a dependency when you pull in Mailman. When I did it this way I didn't need to install Python3-Dev, I only needed to install the Python3-Venv package. However my old setup using Gunicorn didn't use Mailman-Web, and when I migrated to using Mailman-Web as part of a Ubuntu 22.04 setup I had trouble getting Gunicorn to pick up the Django backend:
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
I discussed this here: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
Apparently the installation doc is now updated with a Gunicorn example so when I go Ubuntu 24.04 I will try that method.
I have configured a system in this way in the past using Xapian and pulled in the Ubuntu Xapian packages. My new system uses Woosh as I migrated to Mailman-Web and that installed Woosh as a dependency, and I had forgotten to back up the Xapian indexes.
Andrew.
1 year, 11 months
Re: Show number of subscribers
by Mark Sapiro
On 6/28/19 3:19 AM, andrew.bernard(a)gmail.com wrote:
> Would there be a way to show the number of subscribers to a list in the web UI?
>
> The number of participants per message is great, but this would be good too. My users seem very interested in this number. I guess they want to know how big the audience is that is reading their posts.
I don't know how difficult it would be to add this to HyperKitty, and
I'm not sure how useful it would be in general. HyperKitty is an
archiver. It deals with archiving posts. It knows who has sent a post,
but doesn't know or care if that person is, was or will be a member of
the list.
Postorius knows who the list members are and this information should be
available to list members or the world depending on the list's
member_roster_visibility setting, but see
<https://gitlab.com/mailman/postorius/issues/369>.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
6 years, 10 months
m3 database contents
by Stephen J. Turnbull
Alexander Inzinger-Zrock via Mailman-users writes:
> except for [1] i was not able to find anything about what kind of data
> will be stored in the database(s) of a mailman3-system.
I don't think there's a formal schema in the documentation. It's an
object database backed by a relational database managed by
SQLAlchemy. To get the SQL schema you can get it from your RDBMS, I
suppose.
Regarding the semantics of the data, it's in
mailman/src/mailman/models.
About sizes, I have a brand-new, nearly empty dedicated Mailman
PostgreSQL installation that takes up 69MB for the whole cluster.
(That's for 129 users, split across 6 lists of about 25 subscribers
each, data for all three components: Postorius, HyperKitty, and core.
HyperKitty usage is neglible, only 10 messages total so far.)
A Mailman 3 installation with a 20 year history (recently upgraded
from Mailman 2), somewhere between 500 and 1000 lists, and on the
order of 100,000 users takes up 103MB as a text dump[1]. That's going
to compress by a double-digit factor when loaded back into Postgres, I
bet. This is just core and Postorius data.[2] In other words, the
mailing list cost of Mailman is going to be insignificant compared to
the overhead of a PostgreSQL installation unless you're really huge.
There's a third system I don't have access to any more (another recent
upgrade), and IIRC the (dedicated to Mailman) database cluster took up
about 128MB on disk, with ~20k lists and ~50k users. I don't even
want to think about mail throughput! Some of that database space was
archives, but it's hard to say how much. 10s of megabytes at least,
probably (the predecessor Mailman 2 installation was about 15 years
old). The primary use case for lists was periodic automatically
generated reports on system health and usage statistics, and
notifications, which weren't archived. But there were hundreds of
discussion and announce lists for customers and internal users, many
of which are archived.
The Mailman and Postorius tables in the database don't grow
significantly with time. The deciding factors are the number of users
and the number of lists.
So the bottom line is, Mailman database size and growth is a rounding
error on any modern system.
Archives are another matter. But again, the database cost of the
metadata is insignificant compared to message storage and (likely to
be even bigger) full-text indexing. I'd estimate that the 500-list
site adds about 5TB of message data a year and about the same in
indexing.[2] But the rate of growth for any given site is going to be
extremely case-dependent, and the part that's due to the database
metadata is going to be a tiny fraction, unless your users are
extremely economical with their messages.
Steve
[1] The PostgreSQL cluster supports several other applications, it
takes up gigabytes, almost none of it Mailman.
[2] It doesn't use HyperKitty for archiving so I don't know how much
it would add in database metadata.
1 year, 3 months
Re: cc'd e-mail addresses rewritten with @invalid.domain
by Mark Sapiro
On 9/23/20 12:48 PM, pwilliam(a)fredhutch.org wrote:
> Hi Mailman Users,
>
> I've done some more testing with this issue and have determined that this seems to be an issue with how commas within the display name in cc'd messages are being processed with messages sent from the Outlook client.
>
> Here are some test scenarios. Note I set the mailing list I was testing with to hold messages in moderation. Once the message was held I copied the information from the headers directly from Postorius held message information into the space below:
>
> Test Scenario 1:
> Display name in Outlook: Williamss, Peterr C
> Outcome: Does not work as intended. The CC address is formatted as Williamss(a)invalid.domain in messages received by mailing list members, oddly the "Peterr C" part of the display name is not present anywhere in the CC addresses received by mailing list members.
> Header information from held message in Postorius
> CC: "os-x-lion(a)lists.test.org" <os-x-lion(a)lists.test.org>,
> "Williamss,
> Peterr C" <peterbw(a)test.com>
"Williamss, Peterr C" <peterbw(a)test.com>
is correct for how this address spec should be represented. If in fact,
it is split into two lines
"Williamss,
Peterr C" <peterbw(a)test.com>
as it appears above, it looks to me as if the actual header should be
> CC: "os-x-lion(a)lists.test.org" <os-x-lion(a)lists.test.org>, "Williamss, Peterr C" <peterbw(a)test.com>
but is being folded on the whitespace between Williamss and Peterr.
This is <https://gitlab.com/mailman/mailman/-/issues/725>, fixed at
<https://gitlab.com/mailman/mailman/-/merge_requests/652>. The fix will
be in the next (3.3.2) release of Mailman core.
This is really an Outlook issue. See
<https://gitlab.com/mailman/mailman/-/issues/725#note_351314868> for why
folding within a quoted string is allowed, but in cases like this SHOULD
not be done.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
5 years, 7 months
Re: Starting new thread in Postorius -> disappears
by Thor Atle Rustad
>
>
> > When I send a message from the Postorius interface, it is neither sent
> out
> > or stored. I briefly see this message: "Your reply has been sent and is
> > being processed.", and then nothing happens.
>
> When you say nothing happens, you just mean that you don't see anything in
> the
> UI or you don't see anything in the logs?
>
In the UI, I get a green line with text saying the message has been sent.
However, the new message is not added to the the thread in any way. In the
logs there is nothing that pertains to that particular action
>
> Logs for mailman-web are in /opt/mailman/web/logs/mailmanweb.log (on the
> host).
>
The log mailmanweb.log is empty. uwsgi.log is not empty. Interestingly, in
the mailman-web container, there is a mailmanweb.log in the root of the
server which gets updated like this:
INFO 2017-11-21 10:53:33,765 36 hyperkitty.views.mailman Archived message
<20171121115328.2dc8970f@thor-E6530> to https://mailer.domain.com/
hyperkitty/list/aaa(a)klubb.domain.com/message/2XYZL6G35IJYOW5WG6IGXI424Q44DG
FK/
but nothing is written when I try to send message from Postorius.
>
> >
> > I have installed maxking's docker-mailman and copied my old setup with
> > basic stuff like passwords and such. The only new thing is that I'm now
> > running in userspace by putting in /etc/subuid and subguid:
> >
> > mailman:10000:65536
>
> This ideally shouldn't affect anything, I haven't tried running this way
> though.
>
> >
> >
> > and in /etc/docker/daemon.json:
> >
> > {
> > "userns-remap": "mailman"
> > }
> >
> > So I get the following ownership in /opt/mailman:
> >
> >
> > drwxrwsr-x 3 10100 10000 4096 nov. 21 09:00 core
> > drwx------ 19 10070 10000 4096 nov. 21 09:27 database
> > drwxr-sr-x 5 10100 10101 4096 nov. 21 09:24 web
> >
> > (I have created a user named mailman with uid 10100 so, in reality, it
> says
> > "mailman" there instead of 10100 but I guess that is irrelevant here)
>
> If would be interesting to see what processes are running inside each
> containers
> and who their owners are, in case this is at all relevant to the issue,
> which I
> am not so sure about.
>
> Web container:
bash-4.3# ps aux
PID USER TIME COMMAND
1 mailman 0:00 uwsgi --ini /opt/mailman-web/uwsgi.ini
36 mailman 0:00 uwsgi --ini /opt/mailman-web/uwsgi.ini
37 mailman 0:00 /bin/sh -c ./manage.py qcluster
41 mailman 0:00 python ./manage.py qcluster
54 mailman 0:00 python ./manage.py qcluster
55 mailman 0:00 python ./manage.py qcluster
56 mailman 0:00 python ./manage.py qcluster
57 mailman 0:00 python ./manage.py qcluster
bash-4.3# ls -l
total 96
-rw-rw-r-- 1 mailman root 0 Mar 29 2017 __init__.py
-rw-rw-r-- 1 mailman root 42308 Nov 21 10:53 mailmanweb.log
-rwxrwxr-x 1 mailman root 242 Mar 29 2017 manage.py
-rw-rw-r-- 1 mailman root 12171 Oct 1 22:26 settings.py
-rw-r--r-- 1 root root 9643 Nov 21 09:21 settings.pyc
-rw-rw-r-- 1 mailman root 1777 May 23 21:31 settings_local.py
-rw-r--r-- 1 root root 1196 Nov 21 09:21 settings_local.pyc
-rw-rw-r-- 1 mailman root 1314 Mar 29 2017 urls.py
-rw-r--r-- 1 root root 791 Nov 21 09:21 urls.pyc
-rw-rw-r-- 1 mailman root 1425 Aug 6 17:04 uwsgi.ini
-rwxrwxr-x 1 mailman root 1173 Mar 29 2017 wsgi.py
core container:
bash-4.3# ps aux
PID USER TIME COMMAND
1 mailman 0:00 {master} /usr/local/bin/python /usr/local/bin/master
26 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=archive:0:1 -C /etc/mailman.cfg
27 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=bounces:0:1 -C /etc/mailman.cfg
28 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=command:0:1 -C /etc/mailman.cfg
29 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=in:0:1 -C /etc/mailman.cfg
30 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=lmtp:0:1 -C /etc/mailman.cfg
31 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=nntp:0:1 -C /etc/mailman.cfg
32 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=out:0:1 -C /etc/mailman.cfg
33 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=pipeline:0:1 -C /etc/mailman.cfg
34 mailman 0:01 /usr/local/bin/python /usr/local/bin/runner
--runner=rest:0:1 -C /etc/mailman.cfg
35 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=retry:0:1 -C /etc/mailman.cfg
36 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=virgin:0:1 -C /etc/mailman.cfg
37 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner
--runner=digest:0:1 -C /etc/mailman.cfg
2017-11-21 11:18 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
> Hi Thor,
>
> On Tue, 2017-11-21 at 10:53 +0100, Thor Atle Rustad wrote:
> > I have had this working before, so I guess it's a simple setup error.
> >
> > When I send a message from the Postorius interface, it is neither sent
> out
> > or stored. I briefly see this message: "Your reply has been sent and is
> > being processed.", and then nothing happens.
>
> When you say nothing happens, you just mean that you don't see anything in
> the
> UI or you don't see anything in the logs?
>
> Logs for mailman-web are in /opt/mailman/web/logs/mailmanweb.log (on the
> host).
>
>
> >
> > I have installed maxking's docker-mailman and copied my old setup with
> > basic stuff like passwords and such. The only new thing is that I'm now
> > running in userspace by putting in /etc/subuid and subguid:
> >
> > mailman:10000:65536
>
> This ideally shouldn't affect anything, I haven't tried running this way
> though.
>
> >
> >
> > and in /etc/docker/daemon.json:
> >
> > {
> > "userns-remap": "mailman"
> > }
> >
> > So I get the following ownership in /opt/mailman:
> >
> >
> > drwxrwsr-x 3 10100 10000 4096 nov. 21 09:00 core
> > drwx------ 19 10070 10000 4096 nov. 21 09:27 database
> > drwxr-sr-x 5 10100 10101 4096 nov. 21 09:24 web
> >
> > (I have created a user named mailman with uid 10100 so, in reality, it
> says
> > "mailman" there instead of 10100 but I guess that is irrelevant here)
>
> If would be interesting to see what processes are running inside each
> containers
> and who their owners are, in case this is at all relevant to the issue,
> which I
> am not so sure about.
>
> > _______________________________________________
> > Mailman-users mailing list
> > mailman-users(a)mailman3.org
> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
> --
> thanks,
> Abhilash Raj
>
8 years, 5 months
Re: Another interesting error...
by Darren Smith
Well, using the mailman shell we were able to find the request that was
causing the problem - it was apparently a spam message that had some
chinese characters in it.
We used the shell to delete the held request...
And postorius is still giving me an error 500.
If I'm not mistaken the held messages are kept in .pck files - do you know
if when postorius asks for held messages through the API, does mailman go
to the database or to the .pck? I would actually suspect the latter, or
removing the request should have fixed the problem.
I'm hoping this makes sense...
On Mon, Apr 23, 2018 at 3:39 PM, Darren Smith <silas.crutherton(a)gmail.com>
wrote:
> Very good, I'll give that a try and let you know!
>
>
> On Mon, Apr 23, 2018 at 3:37 PM, Simon Hanna <simon(a)hannaweb.eu> wrote:
>
>> On 04/23/2018 11:29 PM, Darren Smith wrote:
>> > "It's in Mailman's database in the 'message' table."
>> >
>> > I can see the message table - is there a way to see where it is marked
>> as
>> > "held"? All I'm seeing in that table is id, message_id, message_id_hash,
>> > and path.
>> >
>> > I'm wanting to be able to query for held messages for a given mailing
>> list,
>> > as there are currently over 98,000 entries in the message table.
>> This should be possible using the mailman shell
>> http://mailman.readthedocs.io/en/latest/src/mailman/model/do
>> cs/requests.html?highlight=held
>>
>> I haven't used the shell extensively and didn't read the whole docs, but
>> it looks like it's the right place to look. If you can't do that using
>> the shell, I'd say it's worth a bug report :-)
>> _______________________________________________
>> Mailman-users mailing list
>> mailman-users(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
>
>
8 years
Re: Mm3 - new adventure
by Mark Sapiro
On 03/14/2017 09:14 AM, SP2L wrote:
>
> Lately started adventurous journey with Mailman3
> which is way, way different already from the beginning.
>
> Used Mailman bundler to install GNU Mailman, Postorius and HyperKitty
> testing instance in virtual environment, mainly relaying on belows recipe:
>
> https://gitlab.com/mailman/mailman-bundler/
mailman-bundler is out of date and deprecated. You might find
<https://wiki.list.org/x/17891998> of interest.
> Present status.
>
> Above trio and MySQLServer are "talking" to each other
> but because there isproper database on MySQLServer
> I am facing strange and unexpected issues, Hi!
>
> Executing last two commands:
>
> (venv)$ ./bin/mailman start
> (venv)$ ./bin/mailman-web-django-admin runserver 0.0.0.0:8000 &
>
> I see Postorius and HyperKitty up & runningbut...
>
> Belowvery condensed messages from the console:
>
>
> /HyperKitty/
>
> DEBUG = False
>
> [12/Mar/2017 18:59:43] "GET /archives/ HTTP/1.1" 200 8002
> WARNING Not Found:
> /static/hyperkitty/libs/jquery/smoothness/jquery-ui-1.10.3.custom.min.css
...
Have you run the script that bundler installs as bin/mailman-post-update?
> /mailman3/
>
> DEBUG = False
>
> Server Error (500)
>
>
> DEBUG = True
>
> HTTP Error 500: A server error occurred. Please contact the administrator.
>
> Exception Location:
> /var/lib/mailman3/mailman-bundler/eggs/mailmanclient-1.0.1-py2.7.egg/mailmanclient/_client.py
> in call, line 109
>
> Then VERY, verylong list...
Without knowing what the exception was and at least the start of the
traceback, it's hard to say what this might be.
> By means of PhpMyAdmin I created manually mailman3 database
> and few tables (looking at 0001_initial.py) , but it is not working O.K.
> This handmade table is not doing it's job.
You shouldn't need to do this. If you have the proper settings and
credentials in the DATABASES setting in mailman-web/settings_local.py
and the [database] section in deployment/mailman.cfg, the tables should
be created for you.
--
Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
9 years, 2 months