Search results for query "sapiro"
- 6318 messages
[MM3-users] Re: psycopg2 error?
by Odhiambo Washington
On Fri, Oct 27, 2023 at 4:38 AM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 10/26/23 01:03, Odhiambo Washington wrote:
> >
> > <CUT>
> > root@debian12:/home/wash# systemctl status mailman3.service
> > × mailman3.service - GNU Mailing List Manager
> > Loaded: loaded (/etc/systemd/system/mailman3.service; enabled;
> preset:
> > enabled)
> > Active: failed (Result: exit-code) since Thu 2023-10-26 09:36:41
> EAT;
> > 8s ago
> > Process: 1893842 ExecStart=/opt/mailman/venv/bin/mailman start
> > (code=exited, status=1/FAILURE)
> > CPU: 789ms
> >
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]: dbapi =
> > dbapi_meth(**dbapi_args)
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]:
> > ^^^^^^^^^^^^^^^^^^^^^^^^
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]: File
> >
> "/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py",
> > line 690, in import_dbapi
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]: import psycopg2
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]: File
> > "/opt/mailman/venv/lib/python3.11/site-packages/psycopg2/__init__.py",
> line
> > 51, in <module>
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]: from
> > psycopg2._psycopg import ( # noqa
> > Oct 26 09:36:41 debian12.wash.lan mailman[1893842]: SystemError:
> > initialization of _psycopg raised unreported exception
> > Oct 26 09:36:41 debian12.wash.lan systemd[1]: mailman3.service: Control
> > process exited, code=exited, status=1/FAILURE
> > Oct 26 09:36:41 debian12.wash.lan systemd[1]: mailman3.service: Failed
> with
> > result 'exit-code'.
> > Oct 26 09:36:41 debian12.wash.lan systemd[1]: Failed to start
> > mailman3.service - GNU Mailing List Manager.
> > </CUT>
>
> Is your database PostgreSQL or some other? If it's not PostgreSQL, see
> the thread at
>
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/…
> - particularly
>
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> Also see
>
> https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-mail…
> where it now says in part "This will install latest release of Mailman
> Core, and Python bindings for Postgresql database. If you are using a
> different database manager, do not install psycopg2-binary."
>
So my database is PostgreSQL-15:
```
postgres@debian12:~$ psql
psql (15.3 (Debian 15.3-0+deb12u1))
Type "help" for help.
postgres=#
```
Below is the output of `mailman info` which for some reason I forgot to
attach in the initial post. Maybe it contains something useful?
```
(venv) mailman@debian12:~$ mailman info
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman", line 33, in <module>
sys.exit(load_entry_point('mailman==3.3.9', 'console_scripts',
'mailman')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1077, in main
with self.make_context(prog_name, args, **extra) as ctx:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
943, in make_context
self.parse_args(ctx, args)
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1644, in parse_args
rest = super().parse_args(ctx, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
1408, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
2400, in handle_parse_result
value = self.process_value(ctx, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/mailman/venv/lib/python3.11/site-packages/click/core.py", line
2362, in process_value
value = self.callback(ctx, self, value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/bin/mailman.py",
line 95, in initialize_config
initialize(value)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py",
line 229, in initialize
initialize_2(propagate_logs=propagate_logs)
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/core/initialize.py",
line 187, in initialize_2
config.db = getUtility(IDatabaseFactory, utility_name).create()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/database/factory.py",
line 57, in create
database.initialize()
File
"/opt/mailman/venv/lib/python3.11/site-packages/mailman/database/base.py",
line 117, in initialize
self.engine = create_engine(
^^^^^^^^^^^^^^
File "<string>", line 2, in create_engine
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/util/deprecations.py",
line 281, in warned
return fn(*args, **kwargs) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/engine/create.py",
line 601, in create_engine
dbapi = dbapi_meth(**dbapi_args)
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/mailman/venv/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py",
line 690, in import_dbapi
import psycopg2
File
"/opt/mailman/venv/lib/python3.11/site-packages/psycopg2/__init__.py", line
51, in <module>
from psycopg2._psycopg import ( # noqa
SystemError: initialization of _psycopg raised unreported exception
```
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
3 years
[MM3-users] Re: moving lists to another server
by Odhiambo Washington
On Mon, Nov 20, 2023 at 8:53 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 11/20/23 00:58, Odhiambo Washington wrote:
> >
> > What is the command to export the archive as a mbox from the DB?
> > I can't figure out which option it is by looking at 'mailman-web -h'.
>
> Normally you do this on the old server via the HyperKitty web UI which
> has download buttons which do things like
>
>
> https://example.com/archives/list/list@example.com/export/list@example.com.…
>
> or for a partial archive
>
>
> https://example.com/archives/list/list@example.com/export/list@example.com-…
>
> However, in your case I don't think you have the old installation working
> any more so the above won't help.
>
I had a dump of the old installation actually - both databases.
The above has actually helped and I obtained the archive.
> > BTW, how will ' mailman addmembers' treat a file containing:
> > ```
> > Firstname Lastname <email@address>
> > Onlyname <email2@address>
> > ```
>
> Correctly. It will import email@address with display name = 'Firstname
> Lastname' and email2@address with display name = 'Onlyname'.
>
Super!
> > I have opted for the simple way to do this:
> > 1. Create the site in Django Admin and create/configure the list in
> > Postorius
> > 2. Add the members from the file dump, with delivery set to regular.
> > 3. Send the members an email informing them that they need to set their
> > delivery preferences.
> >
> > I am stuck on the issue of exporting the archives to mbox file as already
> > stated above.
>
> Perhaps you can export the mbox via the web UI on the old server or
> maybe temporarily create a second mailman-web on your server with the
> old mailman-web database and use that web UI.
That's exactly how I did it and got the archives.
> Otherwise, if you have a working mailman-web that can access this
> database, you could try
> something like
> ```
> $ mailman-web shell
>
> >>> from hyperkitty.lib.compat import get_list_by_name
> >>> mlist = get_list_by_name('list_name_only', 'list_domain')
> >>> with open('output.mbox', 'wb') as fp:
> ... for email in mlist.emails.order_by("archived_date").all():
> ... fp.write(email.as_bytes() + b'\n')
> ...
> >>>
> ```
>
I tested the above out of curiosity. It failed. There are 9 lists in the
setup, but I am only interested in 1.
```
(venv) mailman@debian12:~$ mailman-web shell
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.17.2 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from hyperkitty.lib.compat import get_list_by_name
In [2]: mlist = get_list_by_name('list_name_only', 'list_domain')
---------------------------------------------------------------------------
Http404 Traceback (most recent call last)
Cell In[2], line 1
----> 1 mlist = get_list_by_name('list_name_only', 'list_domain')
File ~/venv/lib/python3.11/site-packages/hyperkitty/lib/compat.py:37, in
get_list_by_name(list_name, domain)
35 matching =
list(MailingList.objects.filter(name__startswith=list_name+"@"))
36 if len(matching) == 0: # no candidate found
---> 37 raise Http404("No archived mailinglist by that name")
38 if len(matching) == 1: # only one candidate
39 return matching[0]
Http404: No archived mailinglist by that name
```
Was I supposed to substitute something for the real listname of the list I
wanted?
Another thing - totally unrelated - that surprised me:
While using the commented-out block below, I could not click and get
anything beyond http://http://list.name/mailman3 as anything to do with
/postorius/xxxxx gave "Page not found" error.
However, with the second config block that uses the UNIX socket, I am able
to access everything
```
#<IfModule mod_proxy.c>
# ProxyPreserveHost On
# ProxyPass "/mailman3" "http://127.0.0.1:8010/mailman3"
# ProxyPass "/archives" "http://127.0.0.1:8010/archives"
# ProxyPass "/accounts" "http://127.0.0.1:8010/accounts"
# ProxyPass "/admin" "http://127.0.0.1:8010/admin"
# ProxyPass "/user-profile" "http://127.0.0.1:8010/user-profile"
# </IfModule>
ProxyPassMatch ^/static/ !
ProxyPassMatch ^/favicon.ico !
ProxyPass / unix:/opt/mailman/mm/var/uwsgi.sock|uwsgi://localhost/
ProxyPassReverse / unix:/opt/mailman/mm/var/uwsgi.sock|uwsgi://localhost/
```
Why is it that the UNIX socket option worked flawlessly while the other
option failed???
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
2 years, 11 months
[MM3-users] Re: Migrating mailman3 to latest ubuntu lts
by Helio Loureiro
Hi,
No luck :(
(venv) mailman@new-server ~ (v3.3.9)> *pip freeze | egrep
"mailman-web|django-mailman3|django-allauth"*
django-allauth==0.59.0
django-mailman3==1.3.11
mailman-web==0.0.8
(venv) mailman@new-server ~ (v3.3.9)> *pip install -U
django-allauth==0.58.0*
Collecting django-allauth==0.58.0
Downloading django-allauth-0.58.0.tar.gz (861 kB)
---------------------------------------- 861.7/861.7 KB 9.4 MB/s eta
0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests-oauthlib>=0.3.0 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (1.3.1)
Requirement already satisfied: Django>=3.2 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (4.1.13)
Requirement already satisfied: pyjwt[crypto]>=1.7 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.8.0)
Requirement already satisfied: requests>=2.0.0 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.31.0)
Requirement already satisfied: python3-openid>=3.0.8 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (3.2.0)
Requirement already satisfied: asgiref<4,>=3.5.2 in
./venv/lib/python3.10/site-packages (from
Django>=3.2->django-allauth==0.58.0) (3.7.2)
Requirement already satisfied: sqlparse>=0.2.2 in
./venv/lib/python3.10/site-packages (from
Django>=3.2->django-allauth==0.58.0) (0.4.4)
Requirement already satisfied: cryptography>=3.4.0 in
./venv/lib/python3.10/site-packages (from
pyjwt[crypto]>=1.7->django-allauth==0.58.0) (41.0.7)
Requirement already satisfied: defusedxml in
./venv/lib/python3.10/site-packages (from
python3-openid>=3.0.8->django-allauth==0.58.0) (0.7.1)
Requirement already satisfied: urllib3<3,>=1.21.1 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (2.1.0)
Requirement already satisfied: charset-normalizer<4,>=2 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (3.6)
Requirement already satisfied: certifi>=2017.4.17 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (2023.11.17)
Requirement already satisfied: oauthlib>=3.0.0 in
./venv/lib/python3.10/site-packages (from
requests-oauthlib>=0.3.0->django-allauth==0.58.0) (3.2.2)
Requirement already satisfied: typing-extensions>=4 in
./venv/lib/python3.10/site-packages (from
asgiref<4,>=3.5.2->Django>=3.2->django-allauth==0.58.0) (4.9.0)
Requirement already satisfied: cffi>=1.12 in
./venv/lib/python3.10/site-packages (from
cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (1.16.0)
Requirement already satisfied: pycparser in
./venv/lib/python3.10/site-packages (from
cffi>=1.12->cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0)
(2.21)
Building wheels for collected packages: django-allauth
Building wheel for django-allauth (pyproject.toml) ... done
Created wheel for django-allauth:
filename=django_allauth-0.58.0-py3-none-any.whl size=1157319
sha256=a430c552101d1ad47bc00b16d1c1d6df728afacdd13823927b4cbfb02c35dbfc
Stored in directory:
/local/mailman/.cache-ubuntu-22.04/pip/wheels/55/0a/79/e199827a18f310906c2a90b0e92b89c41daf21d2a502db6710
Successfully built django-allauth
Installing collected packages: django-allauth
Attempting uninstall: django-allauth
Found existing installation: django-allauth 0.59.0
Uninstalling django-allauth-0.59.0:
Successfully uninstalled django-allauth-0.59.0
Successfully installed django-allauth-0.58.0
(venv) mailman@new-server ~ (v3.3.9)> *mailman-web migrate*
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you don't care
about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't care
about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying account.0004_alter_emailaddress_drop_unique_email...Traceback
(most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 89, in _execute
return self.cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 257, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during
query')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
sys.exit(main())
File
"/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
line 90, in main
execute_from_command_line(sys.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 96, in wrapped
res = handle_func(*args, **kwargs)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py",
line 349, in handle
post_migrate_state = executor.migrate(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 135, in migrate
state = self._migrate_all_forwards(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 167, in _migrate_all_forwards
state = self.apply_migration(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 252, in apply_migration
state = migration.apply(state, schema_editor)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py",
line 130, in apply
operation.database_forwards(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py",
line 235, in database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 788, in alter_field
self._alter_field(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 858, in _alter_field
self.execute(self._delete_unique_sql(model, constraint_name))
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 199, in execute
cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 67, in execute
return self._execute_with_wrappers(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 84, in _execute
with self.db.wrap_database_errors:
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line
91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 89, in _execute
return self.cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 257, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server
during query')
(venv) mailman@new-server ~ (v3.3.9) [0|1]> *more /etc/mailman3/settings.py*
# Mailman Web configuration file.
# /etc/mailman3/settings.py
# Get the default settings.
from mailman_web.settings.base import *
from mailman_web.settings.mailman import *
# Settings below supplement or override the defaults.
#: Default list of admins who receive the emails from error logging.
ADMINS = (
('Mailman Suite Admin', 'root@localhost'),
)
# Postgresql database setup.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mailman3web',
'USER': 'mailman3web',
# TODO: Replace this with the password.
'PASSWORD': '***********',
'HOST': 'localhost',
# PORT: set to empty string for default.
'PORT': '3306',
# OPTIONS: Extra parameters to use when connecting to the database.
#'OPTIONS': {
# Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See
# https://docs.djangoproject.com/en/1.11/ref/
# databases/#setting-sql-mode
# 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
# 'charset': 'utf8mb4',
#},
}
}
# 'collectstatic' command will copy all the static files here.
# Alias this location from your webserver to `/static`
STATIC_ROOT = '/local/mailman/web/static'
# enable the 'compress' command.
COMPRESS_ENABLED = True
# Make sure that this directory is created or Django will fail on start.
LOGGING['handlers']['file']['filename'] =
'/local/mailman/web/logs/mailmanweb.log'
#: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts
ALLOWED_HOSTS = [
"localhost", # Archiving API from Mailman, keep it.
"127.0.0.1",
# "lists.your-domain.org",
# Add here all production domains you have.
"*"
]
#: See
https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins
(venv) mailman@new-server ~ (v3.3.9)> *mysql -umailman3web -p -h localhost
mailman3web*
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 32
Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input
statement.
MariaDB [mailman3web]> show tables;
+-------------------------------+
| Tables_in_mailman3web |
+-------------------------------+
| account_emailaddress |
| account_emailconfirmation |
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| django_admin_log |
| django_content_type |
| django_mailman3_maildomain |
| django_mailman3_profile |
| django_migrations |
| django_q_ormq |
| django_q_schedule |
| django_q_task |
| django_session |
| django_site |
| hyperkitty_attachment |
| hyperkitty_email |
| hyperkitty_favorite |
| hyperkitty_lastview |
| hyperkitty_mailinglist |
| hyperkitty_profile |
| hyperkitty_sender |
| hyperkitty_tag |
| hyperkitty_tagging |
| hyperkitty_thread |
| hyperkitty_threadcategory |
| hyperkitty_vote |
| socialaccount_socialaccount |
| socialaccount_socialapp |
| socialaccount_socialapp_sites |
| socialaccount_socialtoken |
+-------------------------------+
34 rows in set (0.000 sec)
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
On Mon, 18 Dec 2023 at 17:11, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/18/23 6:24 AM, Helio Loureiro wrote:
> > Hi,
> >
> > Indeed it was the configuration. It was placed into
> > /etc/mailman3/mailman-web.py. After a I changed to
> > /etc/mailman3/settings.py a few things advanced a little bit more.
> >
> > I had to figure out how to fix mysqlclient installation since there
> isn't a
> > mention about it and the simple "pip install mysqclient" was breaking
> with
> > pkg-config issues. But it did work at the end.
> >
> > Now I can see further messages on mailman3-web than before.
> >
> > (venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate
> > System check identified some issues:
> >
> > WARNINGS:
> > account.EmailAddress: (models.W036) MariaDB does not support unique
> > constraints with conditions.
> > HINT: A constraint won't be created. Silence this warning if you don't
> care
> > about it.
> > account.EmailAddress: (models.W043) MariaDB does not support indexes on
> > expressions.
> > HINT: An index won't be created. Silence this warning if you don't care
> > about it.
> > Operations to perform:
> > Apply all migrations: account, admin, auth, contenttypes,
> > django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> > socialaccount
> > Running migrations:
> > Applying account.0004_alter_emailaddress_drop_unique_email...Traceback
> > (most recent call last):
>
>
> I'm not sure why there would be an issue with this migration, but there
> is a possible compatibility issue depending on how you installed things.
>
> django-mailman3<=1.3.11 is not compatible with django-allauth>=0.58.
>
> In your venv, try
> ```
> pip install django-allauth\<0.58
> ```
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to helio(a)loureiro.eng.br
>
2 years, 10 months
[MM3-users] Re: Held messages not delivered after approval
by Odhiambo Washington
On Thu, Jul 31, 2025 at 8:21 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 7/31/25 04:29, Stephan Krinetzki wrote:
> >
> > /opt/mailman/var/queue/out:
> > total 2868
> > drwxrwx--- 2 mailman mailman 4096 Jul 31 13:26 .
> > drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
> > -rw-rw---- 1 mailman mailman 221708 Jul 30 13:16
> 1753874180.0845337+0cc7849043859a79dc3678a0d8b63c1c66df0c66.pck.tmp
> > -rw-rw---- 1 mailman mailman 733425 Jul 31 00:00
> 1753912841.8847518+da55f8789ae41b75d19a02b595e3fd6d45983ade.pck.tmp
>
> These indicate a problem of some sort. .pck.tmp files should only exist
> for a fraction of a second. See
>
> https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/core/switchboa…
>
> What does `mailman qfile report for these? Are they the missing messages?
>
> > -rw-rw---- 1 mailman mailman 17758 Jul 31 13:26
> 1753961185.0481033+6adf966467266567275f1146f5054c95e4365c13.pck
> > -rw-rw---- 1 mailman mailman 31122 Jul 31 13:26
> 1753961185.1064982+b1f502e2af56b9b11680135c6de5fcc5285d967e.bak
> > -rw-rw---- 1 mailman mailman 12945 Jul 31 13:26
> 1753961185.1562705+a88534cfa9b03012c248ac00cbb54f4fc08c8dcc.pck
> > -rw-rw---- 1 mailman mailman 1407634 Jul 31 13:26
> 1753961185.2598634+96e501d7aa8e72fbb9f9a5c953d48fc2c7db2bc7.pck
> > -rw-rw---- 1 mailman mailman 17570 Jul 31 13:26
> 1753961185.3654013+d693065f2b9a2338f887d3ac9c0669c957131100.pck
> > -rw-rw---- 1 mailman mailman 60624 Jul 31 13:26
> 1753961185.4132354+8998ee9eb9a8a99faf1fcf045d604e669f5d0c9e.pck
> > -rw-rw---- 1 mailman mailman 7361 Jul 31 13:26
> 1753961185.413462+50dcef9bb178f54ecb2d4707ccfe02773f92336c.pck
> > -rw-rw---- 1 mailman mailman 84848 Jul 31 13:26
> 1753961185.4464717+17836b8875aa6643f7e5fb0744e72940ce5603c6.pck
> > -rw-rw---- 1 mailman mailman 18003 Jul 31 13:26
> 1753961185.4929028+17e9c78a2721cbc344124c6e04a129217ea6c794.pck
> > -rw-rw---- 1 mailman mailman 66982 Jul 31 13:26
> 1753961185.5228019+4ef59e61c6a79dd4cdee6d1110fb0582fcfafb6c.pck
> > -rw-rw---- 1 mailman mailman 221723 Jul 31 13:26
> 1753961185.5756643+6de90ae99b3d528ce5234b78a287e19f7b6686fc.pck
>
>
> Also, this is a lot of outgoing messages to be queued. Are you sending
> periodic digests at this time? If you normally see a lot of messages in
> the out queue, you might consider setting it's `instances` to 2 or 4. E.g.,
> ```
> [runner.out]
> instances: 4
> ```
> in mailman.cfg
>
> > /opt/mailman/var/queue/shunt:
> > total 3304
> > drwxrwx--- 2 mailman mailman 4096 Jul 31 11:44 .
> > drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
> > -rw-rw---- 1 mailman mailman 451 Jul 31 00:00
> 1753912822.2651796+28eceef7e18eb70393377b88dc7117af8f9362a0.pck
> > -rw-rw---- 1 mailman mailman 490 Jul 31 00:00
> 1753912838.4197352+ea531cf0262c1faa58b1679b907fee92bc16822c.pck
> > -rw-rw---- 1 mailman mailman 1407870 Jul 31 00:00
> 1753912841.9177196+ccea15bdefce3a54301281c8eddf86e8230244a6.pck
> > -rw-rw---- 1 mailman mailman 86108 Jul 31 00:00
> 1753912841.9197443+7dcef4febc71e44c6d9309a24a08b08753e1ff42.pck
> > -rw-rw---- 1 mailman mailman 1407668 Jul 31 00:00
> 1753912841.9849963+a3f1869b750060c97262ece38737480d91652828.pck
> > -rw-rw---- 1 mailman mailman 38992 Jul 31 00:01
> 1753912860.5167956+940584c4f361cbd8c29e390b2f60590558effe40.pck
> > -rw-rw---- 1 mailman mailman 440 Jul 31 00:01
> 1753912860.6972685+635c065bac8dff5f9d562275d707001d773b84c1.pck
> > -rw-rw---- 1 mailman mailman 445 Jul 31 00:01
> 1753912868.7903054+befa066f254d7a3529a8555a6c942a554715d837.pck
> > -rw-rw---- 1 mailman mailman 33494 Jul 31 00:01
> 1753912878.7562895+82b724fd93260ab9a2bb49709d3a42a2f32f2c80.pck
> > -rw-rw---- 1 mailman mailman 217073 Jul 31 00:01
> 1753912878.9337828+de73a65d9c6febfa80275853921f4b53fd1d9e2a.pck
> > -rw-rw---- 1 mailman mailman 85888 Jul 31 00:02
> 1753912950.303359+8a87bce0be63ac1df8493c6b1ad6ae154fcedba7.pck
> > -rw-rw---- 1 mailman mailman 50244 Jul 31 00:02
> 1753912950.4970112+d44d493912bb3024547b8a5112f86f035dcb352f.pck
> > -rw-rw---- 1 mailman mailman 12887 Jul 31 00:02
> 1753912970.038427+31bcbd7fb2ebdf81f6de24b7283b50bcda6ded21.pck.tmp
> > -rw-rw---- 1 mailman mailman 443 Jul 31 11:44
> 1753955094.900898+67bc76525412da66a7c76363f65f583989716305.pck
> >
> > /opt/mailman/var/queue/virgin:
> > total 32
> > drwxrwx--- 2 mailman mailman 81 Jul 31 13:17 .
> > drwxr-xr-x 14 mailman mailman 165 Jun 27 2024 ..
> > -rw-rw---- 1 mailman mailman 32013 Jan 11 2025
> 1736550035.6163204+472f81ece5e45a2651a4499bef418f611b43c619.pck.tmp
> >
> > Nothing special there (shunt should be checked, but not in correlation
> with my mail).
>
>
> Actually those .pck.tmp files indicate some issue.
>
> > I have one list which archives the mail. In the archive there is the
> mail, but it doesn't get deliverd
>
> Which says the pipeline to_archive handler is adding the message to the
> archive queue and the archive runner is handling it, so presumably the
> pipeline to_outgoing handler is adding the message to the out queue, so
> the issue is in the outging runner.
>
In one response, he mentioned that "Other mailinglists on the same host are
doing fine. Even a 1:1 copy of the list works - only the original list does
not work".
Does it mean that "the issue is in the outgoing runner" only for a single
list?
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
In an Internet failure case, the #1 suspect is a constant: DNS.
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
[How to ask smart questions:
http://www.catb.org/~esr/faqs/smart-questions.html]
1 year, 3 months
[MM3-users] Re: Apache+mod_wsgi issue
by Odhiambo Washington
On Sat, Dec 24, 2022 at 9:40 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/24/22 00:00, Odhiambo Washington wrote:
> >
> > These are my files:
> > 1. urls.py
> > <CUT >
> > from django.conf.urls import include
> > from django.urls import re_path
> > from django.contrib import admin
> > from django.urls import reverse_lazy
> > from django.views.generic import RedirectView
> >
> > urlpatterns = [
> > re_path(r'^$', RedirectView.as_view(
> > url=reverse_lazy('list_index'),
> > permanent=True)),
> > re_path(r'^postorius/', include('postorius.urls')),
> > re_path(r'^hyperkitty/', include('hyperkitty.urls')),
> > re_path(r'', include('django_mailman3.urls')),
> > re_path(r'^accounts/', include('allauth.urls')),
> > # Django admin
> > re_path(r'^admin/', admin.site.urls),
> > re_path(r'^mailman3/', include('postorius.urls')),
> > re_path(r'^archives/', include('hyperkitty.urls')),
> > ]
> > </CUT>
>
>
> You don't need all of those. ^postorius/ and ^mailman3/ are two
> different ways to access Postorius, and ^hyperkitty/ and ^archives/ are
> two different ways to access HypereKitty. You only need one of each, but
> having both is OK.
>
That urls.py was picked from
https://wiki.list.org/DOC/Mailman%203%20installation%20experience.
Maybe that page needs some edits/revision now?
> 2. apache.conf
> > <CUT>
> > ...
> > Alias /static "/opt/mailman/mm/static"
> > <Directory "/opt/mailman/mm/static">
> > Require all granted
> > </Directory>
> > # WSGIScriptAlias /mailman3 /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /accounts /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /admin /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /archives /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /user-profile /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /hyperkitty /opt/mailman/mm/wsgi.py
> > # WSGIScriptAlias /postorius /opt/mailman/mm/wsgi.py
> > WSGIScriptAlias / /opt/mailman/mm/wsgi.py
>
>
> This will send every http request to mod_wsgi except for requests to
> /static. This is OK if there are no other web sites on this server and
> the only http requests are for Django related things.
>
This is just a virtual host. There are other websites on the server.
...
> > I have tested with creating all those commented out aliases and that
> ended
> > up still giving me /mailman3/mailman3/* which I did not
> > like, although everything appeared to work.
> > I then ended up with only "WSGIScriptAlias / /opt/mailman/mm/wsgi.py"
> > and now when I type https://mm3-lists.kictanet.or.ke I end
> > up at https://mm3-lists.kictanet.or.ke/mailman3/lists/. Of course
> something
> > has appended "/mailman3"
> > So this has resulted in all URLs having /mailman3/ appended, except for
> the
> > /archives link.
>
>
> How about https://mm3-lists.kictanet.or.ke/user_profile/ ?
>
I have uncommented the line "WSGIScriptAlias /user-profile
/opt/mailman/mm/wsgi.py",
I don't see any definition for it in urls.py as well. Perhaps that's the
issue?
When I uncomment /user-profile, I end up with an ugly URL:
https://mm3-lists.kictanet.or.ke/user-profile/user-profile/
So instead of aliasing the /user-profile, I have decided to add another
definition to urls.py:
re_path(r'^user-profile/', include('allauth.urls')),
Which I think gives me a cleaner URL:
https://mm3-lists.kictanet.or.ke/user-profile/login/?next=/user-profile/
I also had to alias /accounts, which then brings me to:
https://mm3-lists.kictanet.or.ke/accounts/mailman3/lists/ - which I believe
is fine because one gets the
option to either login or signup.
And I think that "/account" and "/user-profile" are synonymous.
> Now my question is about what could be adding the /mailman3. Is it
> possible
> > that I installed something out of the ordinary?
>
>
> It's a mod_wsgi thing. I haven't used mod_wsgi for Mailman for some time
> (I now use gunicorn on all the sites I manage), but when I did I think I
> had
> WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py
>
> and all the URLs were like https://example.com/mm3/mailman3/...,
> https://example.com/mm3/archives/..., etc.
>
I already had gunicorn working too, but I just need to also figure out this
mod_wsgi.
I will stop bothering about the /mailman3 being appended, as long as it
serves the required purpose.
Thank you for all the help.
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
3 years, 10 months
[MM3-users] Re: hello， Discussion on how to subscribe to the mailing list
by Stephen J. Turnbull
Mark Sapiro writes:
> On 1/16/23 18:55, 朱超 wrote:
> > Hi, When I subscribe the email list in Subscription Policy is
> > confirm, Some problems happen in here：
> > 1.Some mail mua reject send the confirm the message, because the
> > length of recipient is too long. for exmaple:
> > requirements-confirm+9605556e0b35a5da280c1de0d34a3e6f770f8ea5@domain,
> > and the mail mua reject to modify about this.
>
> The default English template says in part:
>
> Before you can start using GNU Mailman at this site, you must first
> confirm that this is your email address. You can do this by replying to
> this message.
>
> Or you should include the following line -- and only the following
> line -- in a message to $request_email:
>
> confirm $token
>
> I.e. It gives an alternative method if `reply` doesn't work. You can
> provide a custom version of this template with different wording if you
> think you can improve it.
The problem is the reply address, though. Changing the template to
suggest only the "confirm $token" in body method doesn't help people
who hit R expecting that should work, especially since many "user
fiendly" (misspelling intentional) MUAs don't display the address.
Avoiding that requires code changes, I guess?
> > 2.It is inconvenient for users to use. Many people will
> > mistakenly think that after clicking the subscribe button, it
> > will be successful, and they will not process the email.
> step, you can set Subscription Policy to Open, but then anyone can
> maliciously subscribe third parties to the list.
*Please* do not do this if your server is exposed to the Internet.
There are bots that search for such servers, and sell lists of them to
people who want to DOS mailboxes. You may find yourself banned across
the Internet.
The only real alternative to confirmation is approval by moderators.
I believe Mailman 3 offers that alternative as Mailman 2 did, can't
check at the moment.
> > Regarding the discussion on the subscription method of the
> > mailing list, should we consider optimizing the subscription
> > method of the mailing list, for example:
> > 1. use the method of sending a verification code to subscribe by
> > email，When you receive the verification code sent by the
> > mailing list and fill it in again, you can subscribe
> > successfully。
>
> How is this different from the present method.
Just wording. I think they are thinking of the "invite" option that
was provided in Mailman 2 mass subscribe (again, I think Mailman 3
retains it but I never ever used it so I don't recall for sure). But
it doesn't make sense for a user-initiated interaction.
> If you want, you can make
> a custom template and include something like:
>
> Alternatively, you can confirm by going to the URL
>
> https://www.example.com/mailman3/lists/$list_id/confirm/?token=$token
If I understand the OP's PoV correctly, I think I would change the
list information page to say "Send a personal invitation to join the
list to [email address here]. If you aren't a subscriber, you can
invite yourself." And change the word "confirm" in the confirmation
template to "subscribe".
> We don't include that in the default template because Mailman core
> doesn't know if you even have a web UI or how to access it if you
> do.
We should fix that, at least if the web interface is Postorius
(wishlist, assigned to me):
https://gitlab.com/mailman/mailman/-/issues/1055
@mark
https://gitlab.com/mailman/mailman/-/issues/901
Maybe we can close this? I'm not sure why it was left open since
there's no action proposed for Mailman improvement.
> > 2. use the email to send a connection to confirm the
> > subscription，People click the link to be redirected to a
> > confirmation page to subscribe to the email。
>
> See above.
Indeed, AFAICS almost everything the OP is suggesting amounts to an
identical procedure as far as the Mailman server is concerned, it's
just described to the subscriber differently. The one exception is
the suggestion to eliminate the confirmation step, and while that is
supported, it's quite likely to cause problems for third parties and
eventually for the Mailman site unless you have an alternative ground
source of truth such as an employee database.
Steve
3 years, 9 months
[MM3-users] Re: Postorius claims 2 messages held when there are none - out of sync
by Ken Alker
--On Monday, June 26, 2023 8:24 AM -0700 Mark Sapiro <mark(a)msapiro.net>
wrote:
> On 6/26/23 12:01 AM, Ken Alker wrote:
>> When I view a list via Postorius the "Held Messages" menu item has an
>> inverse-red-"2" next to it, as if there are two held messages. There
>> are, in fact, no messages in "/opt/mailman/mm/var/queue/*" (assuming the
>> held messages should be in "pipeline" subdirectory). When I click on
>> "Held Messages" I get "Something went wrong" followed by "HTTP Error
>> 500: {"title": "500 Internal Server Error"}". How do I fix this?
>
> The held messages are not in any queue. They are stored in Mailman's
> var/messages/ hierarchy. They can be examined with `mailman qfile`
Is the var/messages hierarchy also where all the archived emails are stored
for future viewing via the web interface?
> Also, there is a long thread at
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/threa
> d/UWCUQYU7Q5X5LWNII57ZZ4DDXE4DEFX7/ and maybe others that might be
> relevant.
I have never used the mailman shell before, but based on the referenced
thread I learned to do this (note that the thread also taught me how to
rewrite it to narrow down to just the list with issues so I didn't delete
held messages in other lists, if my case):
for verification:
venv) mailman@speedy:~/web/logs$ mailman shell -l
sbarc-list(a)lists.netlojix.com
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list
>>> requests = IListRequests(m)
>>> first = True
>>> for request in requests.held_requests:
... key, data = requests.get_request(request.id)
... if first:
... first = False
... print(m.list_id)
... print (f"""\
... Sender: {data['_mod_sender']}
... Subject: {data['_mod_subject']}
... Date: {data['_mod_hold_date']}
... Reason: {data['_mod_reason']}
... """)
...
and then to delete the two messages:
(venv) mailman@speedy:~/web/logs$ mailman shell -l
sbarc-list(a)lists.netlojix.com
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list
>>> requests = IListRequests(m)
>>> first = True
>>> for request in requests.held_requests:
... key, data = requests.get_request(request.id)
... if first:
... first = False
... print(m.list_id)
... print(f'request id: {request.id}, deleting')
... requests.delete_request(request.id)
...
sbarc-list.lists.netlojix.com
request id: 27, deleting
request id: 28, deleting
>>>
The result of this, however, was that the web interface still claimed I had
two held messages. I wasn't sure if there was some cron-type "sweeper"
process I had to wait to auto-run or if I had to manually do something to
create an update. I then realized that the sample script had a ' >>>
commit ()" ' at the end, which I'd missed. So, I re-entered the script
(but it didn't spit out that it deleted anything this time) and got an
error when I did the commit. So, I figured the messages HAD been delete
and the commit wasn't really necessary. When I went back to the web
interface, the number of held messages was at 0 and I no longer get the
original error when clicking on "Held messages", so my issue is solved.
But this does leave me with some educational questions:
1) Did I affect something somehow with my "commit", or was there a job that
cleaned things up (if so, what job)?
2) Is the "commit" even necessary (I saw it in someone else's script in the
thread)?
3) Is there a way to save a script and then execute it via shell rather
than typing it in every time (it appears from the examples that people are
typing them in every time, which is super tedious; I didn't paste for fear
of doing something damaging, but maybe that would work)?
4) Are the request.id's of the messages dynamic/changing, or are they fixed
for life? I thought I'd delete based on exact request.id without looping
but I was afraid maybe they were changing over time and that this wouldn't
be a good idea. Might be nice to know for the future.
Thanks for the guidance!
Ken
3 years, 4 months
[MM3-users] Re: Attachement processings questions
by Lars Schimmer
On 2020-02-03 20:24, Mark Sapiro wrote:
> On 2/3/20 4:27 AM, Lars Schimmer wrote:
>>
>> In a mailinglist it seems some mails are not forwarded to a member, but
>> is listed in archive in hyperkitty.
>> That mail did include a PDF attachement, but HyperKitty does not list
>> any attachement.
>
>
> There are two issues here. Why wasn't the message delivered to list
> members?, and why is there no attachment in Hyperkitty?
>
> For the first issue, you need to examine mailman.log for the time of the
> post looking for errors and tracebacks. Also, the MTA logs may have
> useful info if the messages were sent but not delivered.
Thank you so far, I maye found the issue:
Uncaught runner exception: 'ascii' codec can't encode character '\xe9'
in position 109: ordinal not in range(128)
Uncaught runner exception: 'ascii' codec can't encode character '\xa0'
in position 37: ordinal not in range(128)
Seems to be the problemm, as it happens the same moment as the
problematic email arrives in exim4:
2020-01-31 03:02:41 1ixLdp-0003Mo-5z => user(a)abc.de R=mailman3_router
T=mailman3_transport H=localhost [127.0.0.1] C="250 Ok"
2020-01-31 03:02:41 1ixLdp-0003Mo-5z Completed
and nothing more. No sign of mailman3 sending out any mail via this list.
Other emails getting to that list will be send out via user-bounces(a)abc.de.
Those mails are in the mailinglist archives, not on hold or anything.
Just the attachement missing.
So, please, howto fix the ascii encoding error in mailman3?
Full trace:
Jan 31 03:02:43 2020 (32591) Uncaught runner exception: 'ascii' codec
can't encode character '\xe9' in position 109: ordinal not in range(128)
Jan 31 03:02:43 2020 (32591) Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line
173, in _one_iteration
self._process_one_file(msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line
266, in _process_one_file
keepqueued = self._dispose(mlist, msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/runners/pipeline.py",
line 37, in _dispose
process(mlist, msg, msgdata, pipeline)
File "/usr/lib/python3/dist-packages/mailman/core/pipelines.py", line
50, in process
handler.process(mlist, msg, msgdata)
File "/usr/lib/python3/dist-packages/mailman/handlers/to_digest.py",
line 48, in process
mbox.add(msg)
File "/usr/lib/python3.7/mailbox.py", line 603, in add
self._toc[self._next_key] = self._append_message(message)
File "/usr/lib/python3.7/mailbox.py", line 758, in _append_message
offsets = self._install_message(message)
File "/usr/lib/python3.7/mailbox.py", line 830, in _install_message
self._dump_message(message, self._file, self._mangle_from_)
File "/usr/lib/python3.7/mailbox.py", line 215, in _dump_message
gen.flatten(message)
File "/usr/lib/python3.7/email/generator.py", line 116, in flatten
self._write(msg)
File "/usr/lib/python3.7/email/generator.py", line 181, in _write
self._dispatch(msg)
File "/usr/lib/python3.7/email/generator.py", line 214, in _dispatch
meth(msg)
File "/usr/lib/python3.7/email/generator.py", line 432, in _handle_text
super(BytesGenerator,self)._handle_text(msg)
File "/usr/lib/python3.7/email/generator.py", line 249, in _handle_text
self._write_lines(payload)
File "/usr/lib/python3.7/email/generator.py", line 155, in _write_lines
self.write(line)
File "/usr/lib/python3.7/email/generator.py", line 406, in write
self._fp.write(s.encode('ascii', 'surrogateescape'))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in
position 109: ordinal not in range(128)
> For the second, there are many content filtering settings not (yet)
> exposed in Postorius. These include filter_extensions, filter_types,
> pass_extensions and pass_types which control which MIME parts are
> removed. For a newly created Mailman 3 list, these four settings are
> empty, but importing a Mailman 2.1 list will set them according to the
> corresponding MM 2.1 list's settings.
>
>
>> All I found was "convert HTML to plaintext", but does this setting
>> remove attachements?
>
>
> It would only affect HTML parts, not pdfs.
>
>
>> Is there any generel documentation about attachement processing in MM3?
>
>
> <https://mailman.readthedocs.io/en/latest/src/mailman/handlers/docs/filterin…>.
>
>
>
>> I would like this mailinglist to accept mails with attachements, send it
>> to the members and archvie the attachments in HyperKitty, to.
>
>
> You can set Settings -> Alter Messages -> Filter content to No in
> Postorius to not do any content filtering at all. This will override the
> Collapse alternatives and Convert html to plaintext settings and pass
> the message without removing any MIME parts.
>
> But, I think you must have other issues, either with this mail in
> particular or in general as evidenced by the mail not being received by
> list members.
Ok, first fix the ascii encode error for me. Any idea how?
thank you.
MfG,
Lars Schimmer
--
-------------------------------------------------------------
TU Graz, Institut für ComputerGraphik & WissensVisualisierung
Tel: +43 316 873-5405 E-Mail: l.schimmer(a)cgv.tugraz.at
Fax: +43 316 873-5402 PGP-Key-ID: 0x4A9B1723
6 years, 8 months
[MM3-users] Re: integrating mm3 with postfix / lmtp
by Thor Atle Rustad
I will try to get a grip on uid namespaces then...
2017-11-13 10:49 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
> On Nov 13 2017, at 1:34 am, Thor Atle Rustad <thor.rustad(a)gmail.com>
> wrote:
>
>> There is a way around it!
>>
>> I have had two issues with the the maxking docker image. One is that the
>> regexp is not working properly. I reported that, and it has been fixed in
>> newer code. My other problem is that the docker image creates a user,
>> mailman, that receives uid 103. Well, uid 103 on my system is already taken
>> by systemd-bus-proxy (grep 103 /etc/passwd returns
>> "systemd-bus-proxy:x:103:105:systemd Bus Proxy,,,:/run/systemd:/bin/
>> false").
>>
>
> Containers usually run with uid namespace, so it doesn't really matter
> what uid is used outside of container.
>
> Unless, you mount /etc/shadow from host in to the container, which isn't
> really needed for the images.
>
>
>> My solution includes downloading the corrected postfix.py, and replacing
>> the Dockerfile. I put the postfix.py in <docker-mailman>/core/assets/.
>>
>> My Dockerfile:
>> FROM maxking/mailman-core
>>
>> RUN grep mailman /etc/passwd && grep mailman /etc/group \
>> && deluser mailman \
>> && addgroup -S -g900 mailman \
>> && adduser -S -u900 mailman mailman \
>> && grep mailman /etc/passwd && grep mailman /etc/group
>> COPY assets/postfix.py /usr/local/lib/python3.6/site-
>> packages/mailman/mta/postfix.py
>>
>> I then run docker build (with -t parameter, you must look up that
>> yourself). I use a different name for my images, so I end up with (note,
>> there are two tags per image):
>> root@mailer:/home/mailman/docker/docker-mailman_mods/core# docker images
>> REPOSITORY TAG IMAGE ID CREATED
>> SIZE
>> local/mailman_core_900 20171110_2 9649e84767e1 2 days
>> ago 176MB
>> local/mailman_core_900 latest 9649e84767e1 2 days
>> ago 176MB
>> local/mailman_web_900 20171110_2 07a9b3d7ddd6 2 days
>> ago 247MB
>> local/mailman_web_900 latest 07a9b3d7ddd6 2 days
>> ago 247MB
>>
>> I do the same with the web image, as I need to change the user there, too.
>>
>> Then, in docker-compose.yaml, I change the line(s) referring to the image
>> name(s):
>>
>> services:
>> mailman-core:
>> image: local/mailman_core_900
>> container_name: mailman-core
>> hostname: mailman-core
>>
>>
>> mailman-web:
>> image: local/mailman_web_900
>> container_name: mailman-web
>> hostname: mailman-web
>>
>> I don't know if this is a good solution, but at least it fixes some
>> serious issues with the 3.1's postfix integration that wouldn't otherwise
>> be fixed until the 3.2 release. The bottom line is that it works for me,
>> but it adds an additional layer of complication.
>>
>> 2017-11-03 19:40 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>:
>>
>> On Fri, Nov 3, 2017, at 08:29 AM, Fabian A. Santiago wrote:
>> > October 26, 2017 11:07 PM, "Mark Sapiro" <mark(a)msapiro.net> wrote:
>> >
>> > > On October 26, 2017 7:30:35 PM PDT, "Fabian A. Santiago" <
>> fsantiago(a)garbage-juice.com> wrote:
>> > >
>> > >> That was it. Perfect. I manually modified my regexp map and it works
>> > >> now. Excellent and Thank you. You're the man. Does mm3 ever refresh
>> > >> those maps or only as I add new domains / lists?
>> > >
>> > > Only when you make changes to domains or lists.
>> > >
>> > > --
>> > > Mark Sapiro <mark(a)msapiro.net>
>> > > Sent from my Not_an_iThing with standards compliant, open source
>> software.
>> >
>> > Mark,
>> >
>> > I've noticed that even simply restarting the mm3 components those alias
>> > maps get rewritten and the problem returns until I can manually edit it.
>>
>> Yeah, that is true. Transport maps are re-generated everytime the
>> container restarts.
>>
>> I don't think think there is any way around this right now :(
>>
>>
>> --
>> Abhilash Raj
>> maxking(a)asynchronous.in
>> _______________________________________________
>> Mailman-users mailing list
>> mailman-users(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
>>
>>
8 years, 11 months
[MM3-users] Re: Errors while importing mm2 list archives
by Odhiambo Washington
On Thu, Dec 8, 2022 at 10:41 PM Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/8/22 07:23, Odhiambo Washington wrote:
> > MM3-3.3.7, Hyperkitty-1.3.6
> >
> > While importing, I encountered several of this error:
> >
> > <CUT>
> > MySQLdb.OperationalError: (1366, "Incorrect string value:
> > '\\xF0\\x9F\\x98\\x8A, ...' for column 'content' at row 1")
> > The above exception was the direct cause of the following exception:
> ...
> > File
> >
> "/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/backends/mysql/base.py",
>
>
> That's a 4-byte utf-8 encoding for an emoji. MySQL and MariaDB have to be
> told to accept 4-byte encodings.
>
> In the DATABASES['default'] definition, you need to set `'OPTIONS':
> {'charset': 'utf8mb4'}`
>
I added that:
[database]
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://mailman_user:XXXXXXXXX@localhost
/mailmansuite?charset=utf8mb4&use_unicode=1
> Also, although it doesn't affect this, see
>
> https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/datab…
I read this too. Did not pick much from there.
> You might also need to modify the tables, particularly hyperkitty_email
> with something like
> ```
> ALTER TABLE hyperkitty_email
> DEFAULT CHARACTER SET utf8mb4,
> MODIFY subject varchar(512)
> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
> MODIFY content
> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
>
So I removed my list, dropped the database, and proceeded as follows:
create database mailmansuite CHARACTER SET utf8mb4 COLLATE
utf8mb4_0900_ai_ci;
ALTER TABLE hyperkitty_email DEFAULT CHARACTER SET utf8mb4;
ALTER TABLE hyperkitty_email MODIFY subject varchar(512) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY content longtext CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY sender_name varchar(255) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE hyperkitty_email MODIFY in_reply_to varchar(255) CHARACTER SET
utf8mb4 COLLATE utf8mb4_unicode_ci;
I did all those (although looks overzealous) just to be on the safe side. I
have seen people putting emojis in their names.
After that, I did the mailman-web migrate, created the list, imported the
config from MM2 and then proceeded to import the archives.
However, the errors still are there, and several messages were not imported
- I have several "failed to import" warnings.
<CUT>
django.db.utils.OperationalError: (1366, "Incorrect string value:
'\\xF0\\x9F\\x98\\x8A, ...' for column 'content' at row 1")
Message 253d9d44-cc3e-fed1-144b-34da88c52b39(a)emailplus.org failed to
import, skipping *<================== several of these "failed to import"
messages*
Computing thread structure
Synchronizing properties with Mailman
905 emails left to refresh, checked 0
765 emails left to refresh, checked 0
Warming up cache
Traceback (most recent call last):
File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module>
sys.exit(execute_from_command_line())
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_import.py",
line 411, in handle
call_command("hyperkitty_warm_up_cache", list_address)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/__init__.py",
line 198, in call_command
return command.execute(*args, **defaults)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_warm_up_cache.py",
line 53, in handle
self.warm_up_mlist(mlist, options)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/management/commands/hyperkitty_warm_up_cache.py",
line 61, in warm_up_mlist
for thread in mlist.recent_threads:
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 162, in recent_threads
return self.cached_values["recent_threads"]()
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/common.py",
line 58, in __call__
return self.get_or_set(*args, **kwargs)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 317, in get_or_set
return [Thread.objects.get(pk=pk) for pk in thread_ids]
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/hyperkitty/models/mailinglist.py",
line 317, in <listcomp>
return [Thread.objects.get(pk=pk) for pk in thread_ids]
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/manager.py",
line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File
"/opt/mailman/mm/venv/lib/python3.9/site-packages/django/db/models/query.py",
line 650, in get
raise self.model.DoesNotExist(
hyperkitty.models.thread.Thread.DoesNotExist: Thread matching query does
not exist.
(venv) [mailman@gw ~/mm]$
It therefore means I have several broken threads in the archives.
I am still willing to try other solutions if they are suggested.
And what do I make of the final error -
*hyperkitty.models.thread.Thread.DoesNotExist:
Thread matching query does not exist*. ??
--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
3 years, 10 months