Re: Migrating mailman3 to latest ubuntu lts
by Helio Loureiro
Hi,
No luck :(
(venv) mailman@new-server ~ (v3.3.9)> *pip freeze | egrep
"mailman-web|django-mailman3|django-allauth"*
django-allauth==0.59.0
django-mailman3==1.3.11
mailman-web==0.0.8
(venv) mailman@new-server ~ (v3.3.9)> *pip install -U
django-allauth==0.58.0*
Collecting django-allauth==0.58.0
Downloading django-allauth-0.58.0.tar.gz (861 kB)
---------------------------------------- 861.7/861.7 KB 9.4 MB/s eta
0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: requests-oauthlib>=0.3.0 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (1.3.1)
Requirement already satisfied: Django>=3.2 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (4.1.13)
Requirement already satisfied: pyjwt[crypto]>=1.7 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.8.0)
Requirement already satisfied: requests>=2.0.0 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (2.31.0)
Requirement already satisfied: python3-openid>=3.0.8 in
./venv/lib/python3.10/site-packages (from django-allauth==0.58.0) (3.2.0)
Requirement already satisfied: asgiref<4,>=3.5.2 in
./venv/lib/python3.10/site-packages (from
Django>=3.2->django-allauth==0.58.0) (3.7.2)
Requirement already satisfied: sqlparse>=0.2.2 in
./venv/lib/python3.10/site-packages (from
Django>=3.2->django-allauth==0.58.0) (0.4.4)
Requirement already satisfied: cryptography>=3.4.0 in
./venv/lib/python3.10/site-packages (from
pyjwt[crypto]>=1.7->django-allauth==0.58.0) (41.0.7)
Requirement already satisfied: defusedxml in
./venv/lib/python3.10/site-packages (from
python3-openid>=3.0.8->django-allauth==0.58.0) (0.7.1)
Requirement already satisfied: urllib3<3,>=1.21.1 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (2.1.0)
Requirement already satisfied: charset-normalizer<4,>=2 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (3.6)
Requirement already satisfied: certifi>=2017.4.17 in
./venv/lib/python3.10/site-packages (from
requests>=2.0.0->django-allauth==0.58.0) (2023.11.17)
Requirement already satisfied: oauthlib>=3.0.0 in
./venv/lib/python3.10/site-packages (from
requests-oauthlib>=0.3.0->django-allauth==0.58.0) (3.2.2)
Requirement already satisfied: typing-extensions>=4 in
./venv/lib/python3.10/site-packages (from
asgiref<4,>=3.5.2->Django>=3.2->django-allauth==0.58.0) (4.9.0)
Requirement already satisfied: cffi>=1.12 in
./venv/lib/python3.10/site-packages (from
cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0) (1.16.0)
Requirement already satisfied: pycparser in
./venv/lib/python3.10/site-packages (from
cffi>=1.12->cryptography>=3.4.0->pyjwt[crypto]>=1.7->django-allauth==0.58.0)
(2.21)
Building wheels for collected packages: django-allauth
Building wheel for django-allauth (pyproject.toml) ... done
Created wheel for django-allauth:
filename=django_allauth-0.58.0-py3-none-any.whl size=1157319
sha256=a430c552101d1ad47bc00b16d1c1d6df728afacdd13823927b4cbfb02c35dbfc
Stored in directory:
/local/mailman/.cache-ubuntu-22.04/pip/wheels/55/0a/79/e199827a18f310906c2a90b0e92b89c41daf21d2a502db6710
Successfully built django-allauth
Installing collected packages: django-allauth
Attempting uninstall: django-allauth
Found existing installation: django-allauth 0.59.0
Uninstalling django-allauth-0.59.0:
Successfully uninstalled django-allauth-0.59.0
Successfully installed django-allauth-0.58.0
(venv) mailman@new-server ~ (v3.3.9)> *mailman-web migrate*
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you don't care
about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't care
about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying account.0004_alter_emailaddress_drop_unique_email...Traceback
(most recent call last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 89, in _execute
return self.cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 257, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (2013, 'Lost connection to MySQL server during
query')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
sys.exit(main())
File
"/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
line 90, in main
execute_from_command_line(sys.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 96, in wrapped
res = handle_func(*args, **kwargs)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py",
line 349, in handle
post_migrate_state = executor.migrate(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 135, in migrate
state = self._migrate_all_forwards(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 167, in _migrate_all_forwards
state = self.apply_migration(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 252, in apply_migration
state = migration.apply(state, schema_editor)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py",
line 130, in apply
operation.database_forwards(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py",
line 235, in database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 788, in alter_field
self._alter_field(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 858, in _alter_field
self.execute(self._delete_unique_sql(model, constraint_name))
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 199, in execute
cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 67, in execute
return self._execute_with_wrappers(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 84, in _execute
with self.db.wrap_database_errors:
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line
91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 89, in _execute
return self.cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 257, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server
during query')
(venv) mailman@new-server ~ (v3.3.9) [0|1]> *more /etc/mailman3/settings.py*
# Mailman Web configuration file.
# /etc/mailman3/settings.py
# Get the default settings.
from mailman_web.settings.base import *
from mailman_web.settings.mailman import *
# Settings below supplement or override the defaults.
#: Default list of admins who receive the emails from error logging.
ADMINS = (
('Mailman Suite Admin', 'root@localhost'),
)
# Postgresql database setup.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mailman3web',
'USER': 'mailman3web',
# TODO: Replace this with the password.
'PASSWORD': '***********',
'HOST': 'localhost',
# PORT: set to empty string for default.
'PORT': '3306',
# OPTIONS: Extra parameters to use when connecting to the database.
#'OPTIONS': {
# Set sql_mode to 'STRICT_TRANS_TABLES' for MySQL. See
# https://docs.djangoproject.com/en/1.11/ref/
# databases/#setting-sql-mode
# 'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
# 'charset': 'utf8mb4',
#},
}
}
# 'collectstatic' command will copy all the static files here.
# Alias this location from your webserver to `/static`
STATIC_ROOT = '/local/mailman/web/static'
# enable the 'compress' command.
COMPRESS_ENABLED = True
# Make sure that this directory is created or Django will fail on start.
LOGGING['handlers']['file']['filename'] =
'/local/mailman/web/logs/mailmanweb.log'
#: See https://docs.djangoproject.com/en/dev/ref/settings/#allowed-hosts
ALLOWED_HOSTS = [
"localhost", # Archiving API from Mailman, keep it.
"127.0.0.1",
# "lists.your-domain.org",
# Add here all production domains you have.
"*"
]
#: See
https://docs.djangoproject.com/en/dev/ref/settings/#csrf-trusted-origins
(venv) mailman@new-server ~ (v3.3.9)> *mysql -umailman3web -p -h localhost
mailman3web*
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 32
Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input
statement.
MariaDB [mailman3web]> show tables;
+-------------------------------+
| Tables_in_mailman3web |
+-------------------------------+
| account_emailaddress |
| account_emailconfirmation |
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| django_admin_log |
| django_content_type |
| django_mailman3_maildomain |
| django_mailman3_profile |
| django_migrations |
| django_q_ormq |
| django_q_schedule |
| django_q_task |
| django_session |
| django_site |
| hyperkitty_attachment |
| hyperkitty_email |
| hyperkitty_favorite |
| hyperkitty_lastview |
| hyperkitty_mailinglist |
| hyperkitty_profile |
| hyperkitty_sender |
| hyperkitty_tag |
| hyperkitty_tagging |
| hyperkitty_thread |
| hyperkitty_threadcategory |
| hyperkitty_vote |
| socialaccount_socialaccount |
| socialaccount_socialapp |
| socialaccount_socialapp_sites |
| socialaccount_socialtoken |
+-------------------------------+
34 rows in set (0.000 sec)
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
On Mon, 18 Dec 2023 at 17:11, Mark Sapiro <mark(a)msapiro.net> wrote:
> On 12/18/23 6:24 AM, Helio Loureiro wrote:
> > Hi,
> >
> > Indeed it was the configuration. It was placed into
> > /etc/mailman3/mailman-web.py. After a I changed to
> > /etc/mailman3/settings.py a few things advanced a little bit more.
> >
> > I had to figure out how to fix mysqlclient installation since there
> isn't a
> > mention about it and the simple "pip install mysqclient" was breaking
> with
> > pkg-config issues. But it did work at the end.
> >
> > Now I can see further messages on mailman3-web than before.
> >
> > (venv) mailman@new-server ~ (v3.3.9)> mailman-web migrate
> > System check identified some issues:
> >
> > WARNINGS:
> > account.EmailAddress: (models.W036) MariaDB does not support unique
> > constraints with conditions.
> > HINT: A constraint won't be created. Silence this warning if you don't
> care
> > about it.
> > account.EmailAddress: (models.W043) MariaDB does not support indexes on
> > expressions.
> > HINT: An index won't be created. Silence this warning if you don't care
> > about it.
> > Operations to perform:
> > Apply all migrations: account, admin, auth, contenttypes,
> > django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
> > socialaccount
> > Running migrations:
> > Applying account.0004_alter_emailaddress_drop_unique_email...Traceback
> > (most recent call last):
>
>
> I'm not sure why there would be an issue with this migration, but there
> is a possible compatibility issue depending on how you installed things.
>
> django-mailman3<=1.3.11 is not compatible with django-allauth>=0.58.
>
> In your venv, try
> ```
> pip install django-allauth\<0.58
> ```
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
>
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> Archived at:
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…
>
> This message sent to helio(a)loureiro.eng.br
>
5 months
Re: admin/login/ cannot be accessed
by jean-christophe manciot
- Django version is 2.2.6-1ubuntu1
- Disabling HTTP2 in nginx means disabling it for all server blocks
listening on the same IP, which would degrade all other servers.
- Doing so leads to another error:
This page isn’t working
<mysite> didn’t send any data.
ERR_EMPTY_RESPONSE
- I cannot run nginx and apache on the same <ip_address>:443 port either.
I found no error in mailman3 or syslog logs.
In ```/etc/mailman3/mailman.cfg```, I have:
```
[logging.debian]
format: %(asctime)s (%(process)d) %(message)s
datefmt: %b %d %H:%M:%S %Y
propagate: no
level: debug
path: mailman.log
```
Yet, mailman.log does not seem to show debug level information.
On Wed, Dec 11, 2019 at 7:04 PM Abhilash Raj <maxking(a)asynchronous.in>
wrote:
>
>
> On Wed, Dec 11, 2019, at 9:25 AM, jean-christophe manciot wrote:
> > Ubuntu 20.04
> > python3-django 2:2.2.6-1ubuntu1
> > python3-django-hyperkitty 1.3.1 (built from sources)
> > mailman3-full 3.2.2-1
>
> Which version of Django are you using?
>
> >
> > Nginx server configuration:
> > ```
> > ...
> > ########
> > # Static
> > ########
> > location /favicon.ico
> > {
> > alias <mysite_dir>/static/hyperkitty/img/favicon.ico;
> > }
> > location /static/favicon.ico
> > {
> > alias <mysite_dir>/static/postorius/img/favicon.ico;
> > }
> > location /static/
> > {
> > alias <mysite_dir>/static/;
> > }
> >
> > #######################
> > # Upstream uwsgi server
> > #######################
> > location /
> > {
> > include /etc/nginx/uwsgi_params;
> > uwsgi_pass 127.0.0.1:<uwsgi_server_port>;
> > }
> > ...
> > ```
> > where:
> > - <mysite_dir> is a symlink to <django_dir>/static
> > - <uwsgi_server_port> matches the one defined in
> ```/etc/mailman3/uwsgi.ini```:
> > ```
> > [uwsgi]
> > # Port on which uwsgi will be listening.
> > uwsgi-socket = 127.0.0.1:<uwsgi_server_port>
> > ```
> >
>
> The config looks good to me in a quick glance.
>
>
> > All 3 systemd services run fine:
> > - mailman3
> > - mailman3-web
> > - qcluster
> >
> > I'm trying to login to the django administration pages.
> > I get the django administration login page at:
> > https://mysite/admin/login/
> > Logging in with the admin credentials leads to:
> > ```
> > This site can’t be reached
> > The webpage at https://mysite/admin/login/ might be temporarily down or
> > it may have moved permanently to a new web address.
> > ERR_HTTP2_PROTOCOL_ERROR
> > ```
> > This is very strange because it is the URL which I used to get the
> > login page in the first place.
>
>
> Looking at the error, it seems like something somewhere is re-directing to
> HTTP/2 or the request is based off of HTTP/2 and all the components in the
> stack don't support HTTP/2, leading to the error message.
>
> I haven't played a lot with HTTP/2 yet so I am not sure which specific
> component in the stack could be incompatible here.
>
> >
> > If I launch a test web server at another port with:
> > ```
> > <django_dir># python3 manage.py runserver <mysite_ip_address>:8080
> > Performing system checks...
> >
> > System check identified no issues (0 silenced).
> > December 11, 2019 - 17:50:48
> > Django version 2.2.6, using settings 'settings'
> > Starting development server at http://<mysite_ip_address>:8080/
> > Quit the server with CONTROL-C.
> > ```
> > and access it at ```http://<mysite_ip_address>:8080/admin/login/``` to
> > login with the same credentials as before, I get through and all the
> > django administration lines appear, although in a degraded layout:
> > ```
> > Site administration
> > Accounts
> > Email addresses Add Change
> > Authentication and Authorization
> > Groups Add Change
> > Users Add Change
> > Django Mailman 3
> > Mail domains Add Change
> > Profiles Add Change
> > ...
> > ```
> > Any idea what could be happening here?
>
>
> Degraded layout is due to missing static files since the development
> server that you spun off doesn't serve static files. So, that is okay.
> > _______________________________________________
> > Mailman-users mailing list -- mailman-users(a)mailman3.org
> > To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
> >
>
> --
> thanks,
> Abhilash Raj (maxking)
>
--
Jean-Christophe
4 years, 5 months
Re: Importing mbox files into archive defect with lines with From
by Stephen J. Turnbull
Alex Schuilenburg via Mailman-users writes:
> Nope, apologies.� I'm happy to add the list back in - I thought I
> hit reply-to-list.
No apologies necessary.
> As an email separator agreed, provided the body has all "First "
> (preceded by a blank line and nothing preceding on the same line)
> suitably escaped.
That's not the way this works. You don't get to choose, only to
defend your system. See
https://www.jwz.org/doc/content-length.html
> I have to move the lists onto a new Debian 12 server using the
> native mailman 3.3.8 & mailman-web 0+20200530-2 packages.
I think the relevant package version is HyperKitty's. Mailman-Web
should just be a wrapper around HyperKitty and Postorius.
> > The preferred way is to dump the database to SQL, and then load
> > it in to the new database directly rather than downloading the
> > mbox files and importing.
>
> Thats what I thought initially, but that failed as per
> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message….
>
> As my old installation appears to have the django_migrations table
> inconsistent with the state of the database, and Debian have been
> unresponsive so far.
Hm. I think it's more likely that the load overwrote the
django_migrations table with the old migrations table, but the
Debian-supplied database already had migrations applied on the
assumption that you're either creating a new Mailman instance or
upgrading in place. When you load the dumped database, that is
probably smart enough to delete tables before creating them (or
perhaps you just get lucky that the load doesn't try to delete or
rename columns) BUT new tables do NOT get that treatment. They just
sit around waiting for you to apply the migration that creates them
and "what migration doing?" KA-BOOM.
I suspect that "DROP DATABASE mailmanweb;" and then loading the dumped
database, followed by `mailman-web migrate` will work. (Usual caveat
of you should have a backup onsite, a backup in a bank vault, and a
backup store on the dark side of the moon before trying this!)
Thank you for going to Debian first (at about the same time is fine),
by the way. We really appreciate that. We try hard to make sure
Mailman works in all *our* common use cases, but distros have a
different set. It's very common that a distro will do something that
makes sense for their usual use cases that just fail badly for cases
they didn't anticipate.
> Then I guess that is the case in Debian 12.
Maybe, it's often hard to tell where distros go wrong. If my
guessalasys above is correct, it's simply the assumption that the user
is either doing a greenfield installation or an upgrade in place.
Surely those are the great majority of cases.
> I thought that ">From " would be escaped to ">>From ", and so on,
> so the escape could easily be reversed when imported.
Ah, you are an honest person. Do not commit crimes, my friend, you
will get caught. More devious thinkers quote messages by prepending
only ">" to the line. Or, knowing about From-stuffing when sending
signed mail then might pre-stuff From lines so that signature
validation succeeds by default. Either way if you unstuff you will
break the message. Maybe ChatGPT-10 will get it right. ;-)
The only way to win is to not play the mbox game.
> After all, I would expect that an export of the archive to mbox,
> followed by a delete of the archive, followed by a
> hyperkitty_import of the archive, should leave you at the same
> place.
You would expect, for sure. You would be wrong, because mbox is a
lossy format by design. (Or by lack of design, if you prefer.)
> Not with ">From " escapes in the new archives.� In fact I
> also had a number of messages with "Message-ID: <>" and worse: all
> messages with attachments had the text/plain content empty.
I don't EVEN want to think why that might be.
> The following dump and import worked.
>
> oldhost> mysqldump --no-create-info --no-create-db --disable-keys
> --complete-insert mailman3web > mailman3web.sql
>
> newhost> mysql
> MariaDB [(none]> use mailman3web
> MariaDB [mailman3web]> source mailman3web.sql
Yeah!!
Except I forgot how to update the FAQ. Now I have to learn again!
:-)
Steve
7 months, 4 weeks
Re: Weird response to postings after subscriptions and accounts
by tokyoprogressive＠mailbox.org
Thank you Mark. I will need to find time tomorrow to create two totally new email accounts that I can use in tests so I do not inadvertently sign up again with the same address. Also will double check my settings.
Best regards
Paul
Sent from my iPhone
> On Sep 17, 2019, at 14:39, Mark Sapiro <mark(a)msapiro.net> wrote:
>
>> On 9/16/19 9:22 PM, Paul Arenson wrote:
>> In my test of the two forms of signup, I am getting these unexpected results. Is it my settings or is something weird within the program itself?
>>
>> 1) https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres…
>>
>> 2) Choose "You can also subscribe without creating an account. If you wish to do so, please use the form below"
>>
>> 3) Then I have to approve it. They get "Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org
>
>
> You have to approve it or the user has to confirm her email address?
> Maybe you have to approve it because the lists Subscription Policy is
> Moderate.
>
>> 4) But I have to approve it.I assume this is because they do not yet have a real account. And they should be listed as a NON MEMBER?
>
>
> No. They will be a member, they just will not have a Django account that
> they can log in to.
>
>
>> 5) But even when they sign up here: https://list.tokyoprogressive.org/accounts/signup/?next=%2Fpostorius%2Flist… they get an email from my provider (NOT MY LIST????) as follows:
>>
>> "Hello from mailman3.emwd.com! You're receiving this e-mail because user iCloud has given yours as an e-mail address to connect their account.
>> To confirm this is correct, go to https://list.tokyoprogressive.org/accounts/confirm-email/MTE3:1iA44k:E7w7xh… Thank you from mailman3.emwd.com! mailman3.emwd.com"
>
>
> Because when you create a Django account, you have to confirm your email
> address.
>
>
>> 6) And this is where it gets weird. If they send from their email program and get the welcome (Welcome to the "Discuss" mailing list!
>> To post to this list, send your email to: discuss(a)list.tokyoprogressive.org You can unsubscribe or make adjustments to your options via email by
>> sending a message to....... I then get a notice:
>>
>> "Your mail to 'discuss(a)list.tokyoprogressive.org' with the subject test notice Is being held until the list moderator can review it for approval. The message is being held because: The message is not from a list member"
>
>
> What is Message Acceptance -> Default action to take when a member posts
> to the list ?
>
> You get the notice or the user gets the notice?
>
>
>> 7) While if they send FROM THE interface it goes through.
>>
>> 8) What I am confused about.
>>
>> Signing up for an account should make them a member, right? Subscribing should make them a non member, no?
>
> Signing up for an account is just signing up for a Django account. You
> can sign up for an account and log in and then subscribe to the list.
>
> Subscribing makes one a member whether or not you subscribe from a
> logged in account or subscribe without creating an account.
>
>
>> But here it seems that both make them a non member.
>
> If an address which is not a member posts to the list, that address
> becomes a non-member. This is the only way other than being explicitly
> added by an admin that an address becomes a non-member.
>
>
>> And worse, someone with an account sending from the Interface has their message get through while the same person sending from an email program has me get a notice that a non member is trying to post.
>
>
> If by sending from the interface, you mean posting via hyperkitty, the
> user must have an account and be logged in to do that.
>
> Your "same person" may have multiple entities within Mailman, possibly
> subtly different email addresses, one of which is a member and one of
> which isn't.
>
>
> --
> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
> San Francisco Bay Area, California better use your sense - B. Dylan
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
4 years, 8 months
Re: KeyError: 'subject_prefix'
by Torge Riedel
Am 15.11.20 um 12:20 schrieb Stephen J. Turnbull:
> This isn't very helpful. Does this happen for all lists? A
> particular list? How long have you been running this version of
> Mailman 3 (time since last update)?
This happens when daily cron is executed (/opt/mailman/web/bin/django-admin.sh runjobs daily) and I received the mail by cron.
I think I have updated to 3.3.1 two or three weeks after release.
But 3 days ago I recognized that mails from cron for user "mailman" where stored in /var/mail/mailman instead forwarding them to my admin inbox. Missed an alias. I changed this and checked the mail file for entries and found it is flooded with errors due to a wrong script (not the one mentioned here) in cron job. Maybe I have overseen this error happened already before, between all those other errors.
Looks like it is happening only for one list, see mailman core log below. I replaced the list name by <listid>@<listdomain>.de
A - maybe important - info: Archiving is disabled at all.
>
> Is there anything in the logs from Mailman core (this error is coming
> from the client side of the REST API module)?
[2020-11-15 00:00:10 +0100] [9949] [ERROR] Error handling request /3.1/lists/<listid>@<listdomain>.de/config
Traceback (most recent call last):
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 779, in _getter
getter = self._metadata._getter
AttributeError: 'NoneType' object has no attribute '_getter'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/mailman/core/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 134, in handle
self.handle_request(listener, req, client, addr)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 175, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper
rtn = function(*args, **kws)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__
return super().__call__(environ, start_response)
File "falcon/api.py", line 274, in falcon.api.API.__call__
File "falcon/api.py", line 269, in falcon.api.API.__call__
File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/listconf.py", line 279, in on_get
value = getter.get(self._mlist, attribute)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/rest/listconf.py", line 52, in get
return sorted(aliases.aliases)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/mailman/model/mailinglist.py", line 563, in aliases
for alias in aliases:
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 101, in instances
cursor.close()
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
exc_value, with_traceback=exc_tb,
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 61, in instances
for query_entity in query._entities
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 61, in <listcomp>
for query_entity in query._entities
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 4375, in row_processor
polymorphic_discriminator=self._polymorphic_discriminator,
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 422, in _instance_processor
getter = result._getter(col, False)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 781, in _getter
return self._non_result(None, err)
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1241, in _non_result
replace_context=err,
File "/opt/mailman/core/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
sqlalchemy.exc.ResourceClosedError: This result object does not return rows. It has been closed automatically.
[15/Nov/2020:00:00:10 +0100] "GET /3.1/lists/<listid>@<listdomain>.de/config HTTP/1.1" 500 0 "-" "-"
>
> > I checked the database,
>
> Which database? It looks to me like subject_prefix is declared in
> both Mailman core and HyperKitty (I haven't checked Postorius). I'm
> not real familiar with this code, so it could be that eventually
> everything boils down to REST calls to Mailmman core, but it looks
> like Django's database is involved for HyperKitty.
I checked both databases: mailman-web.hyperkitty_mailinglist and mailman-core.mailinglist
>
> > there is a column "subject_prefix", so I feel it is something in
> > the code causing the error.
>
> Sure, but what? We eventually see this:
>
> > File "/opt/mailman/web/venv/lib/python3.6/site-packages/mailmanclient/restbase/connection.py", line 112, in call
> > error_msg, response, None)
> > urllib.error.HTTPError: HTTP Error 500: <html>
> > <head>
> > <title>Internal Server Error</title>
> > </head>
> > <body>
> > <h1><p>Internal Server Error</p></h1>
> >
> > </body>
> > </html>
>
> which is mailmanclient passing on the REST API's report that Mailman
> core is very confused. There should be a log message from core for
> this.
>
Hope I provided all the information you requested for.
Kind regards
Torge
3 years, 6 months
Re: Trying to configure a list (hosted)
by Abhilash Raj
On Sat, Aug 31, 2019, at 7:05 PM, paul(a)arenson.org wrote:
> Thank you for the very quick response, Abhilash Raj. Being non
> technical (and therefore opting for hosted Mailman 3), I can partly
> understand your kind response.
>
> 1) I can understand that I cannot disable the different ways to sign up.
>
> 2) I am still struggling to understand the difference between the two
> web choices as shown here.
>
> WAY # 1 TO GET AN ACCOUNT
> https://list.tokyoprogressive.org/accounts/signup/?next=%2Fpostorius%2Flist…
>
> VS
>
> WAY # 2 YOU CAN GET AN ACCOUNT WITHOUT SIGNING UP
> at the bottom of
> https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres…
>
> (I was under the impression making only WAY #1 allows one to post, but
> are you saying that the bottom option also allows one to post, but only
> by email?)
That's right. Mailman is a mailing list manager and is primarily based around discussions using Emails. You can subscribe to mailing lists and send email to the list address, which then gets distributed to all the members. All the emails sent to the mailing lists are also archived usually in an archiver.
The official archiver, a.k.a the one supported by us, also additionally supports posting to the mailing list via the Web interface. If anyone wants to use the web ui to post, they need to have an account and need to verify that they own the email address they are signing up with.
Then, anything they post is sent as an email, on their behalf, to the mailing list and that's how you can get 2 ways to post.
You don't need any account on the web interface if all you want is to post through email.
> If I am confused, I know my users will also be confused. I think I
> understand I can qualify their subscription in the ways you suggest:
> "Confirm where all users would have to confirm their subscription using
> an email sent to them. Optionally, you can also enable moderation where
> each subscription needs explicit approval too."
>
> So to put it simply:
>
> 1) I want to make it a discussion list. Not available to people who are
> not signed up?
It is slightly confusing, but let's separate sign-up into two terms and let me know if that is still confusing, "sign up to web interface", "subscribe to mailing list".
- sign up to web interface: This is optional. This allows you, as an administrator to operate the mailing list and it's configuration. A user can sign up to the web interface and manage all the subscriptions they have, and also subscribe to mailing lists.
- subscribe to mailing list: This is a user participating in a Mailing List. For this, they don't necessarily need to sign up via web interface as there are other ways to subscribe to a mailing list, like anonymous subscribe (which is your #2 above). Once they are subscribed, they are called "subscribers".
Users will always have to "subscribe" before they can participate on a mailing list.
There is currently no way to mandate "sign up on the web interface" for all "subscribers".
>
> 2) This is dependent on all having access to the archives so they can
> see what they are responding to
If the archives are set to be private, only the "subscribers" can view the archives.
>
> 3) What settings do I make to insure that all who sign up via WAY #1
> can see the archives and post.
You have to set the archives to be private and that's all.
You can do that from:
List -> Settings -> Archiving
Options on that page should be self-explanatory
>
> 4) What settings do I make to insure that all who sign up via way #2
> can do the same?
They will receive emails and will be able to reply to them.
If they want to post via the web interface, they will have to create an account and sign up, same as #1.
>
> Last, in my account settings what is the difference between a member
> and a non member?
"non-members" are people Mailman remembers who interacted with a MailingList in past. For example, I can ask an list owner to let my post through, without actually subscribing(because I don't want to get all the emails, just send a single email) by allowing me to be a "non-member" who is allowed to post.
>
> Thanks and sorry I was not clear the first time.
> _______________________________________________
> Mailman-users mailing list -- mailman-users(a)mailman3.org
> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>
--
thanks,
Abhilash Raj (maxking)
4 years, 8 months
Re: Hyperkitty CPU usage
by Alain Kohli
Well, I understand that it might take a while, but the process is
running for over 200h now, I doubt it can really take _that_ long if
everything is working properly. Especially because the rebuild_index
command is only taking a few hours at most, doesn't that essentially do
the same?
We have ~700k messages in ~300 mailing lists in case that helps you
estimate how long it could take. I can take a look at alternative
backends, though.
On 5/7/19 4:32 PM, Abhilash Raj wrote:
> On Sat, Apr 27, 2019, at 6:22 PM, Alain Kohli wrote:
>> I'm running a custom image which is based on an older version of the one
>> here: https://github.com/maxking/docker-mailman. I attached it below.
>> But I separated postorius and hyperkitty, so hyperkitty is running in
>> its own container. I'm deploying the image with a plain 'docker run'
>> behind nginx. I made fulltext_index persistent now, but it didn't get
>> populated with anything yet. I don't really have an error traceback
>> because there is never an error thrown. The only thing with some content
>> is uwsgi-error.log, which you can find below. I'm also still getting the
>> "A string literal cannot contain NUL (0x00) characters." messages. I
>> also noticed that it takes incredibly long for the webinterface to load
>> (several minutes) even though there doesn't seem to be any process
>> consuming notable resources apart from the minutely job.
>>
>> Funnily enough, I have the exact same image deployed on a second server
>> as well for testing. On that one everything works fine. The only
>> difference is that on the problematic one I have a lot more mailing
>> lists/archives and that I imported them from mailman2. Could something
>> have gone wrong during the import? I used the regular hyperkitty_import
>> command.
> Yes, this is because `whoosh`, the library set by default to run fulltext
> indexing is a pure python implementation and quite slow in busy lists.
>
> We do support more backends though, see [1] for a list of all the supported
> search backends. Something like Xapian(C++) or Elasticsearch/Solr(Java)
> should be much better in terms of performance.
>
> [1]: https://django-haystack.readthedocs.io/en/master/backend_support.html
>
>> uwsgi-error.log:
>>
>> *** Starting uWSGI 2.0.18 (64bit) on [Sat Apr 27 22:50:17 2019] ***
>> compiled with version: 6.4.0 on 27 April 2019 22:48:42
>> os: Linux-4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19)
>> nodename: hyperkitty.docker
>> machine: x86_64
>> clock source: unix
>> detected number of CPU cores: 4
>> current working directory: /home/hyperkitty
>> detected binary path: /usr/local/bin/uwsgi
>> !!! no internal routing support, rebuild with pcre support !!!
>> setgid() to 82
>> setuid() to 82
>> chdir() to /home/hyperkitty
>> your memory page size is 4096 bytes
>> detected max file descriptor number: 1048576
>> lock engine: pthread robust mutexes
>> thunder lock: disabled (you can enable it with --thunder-lock)
>> uwsgi socket 0 bound to TCP address 0.0.0.0:8081 fd 8
>> uwsgi socket 1 bound to TCP address 0.0.0.0:8080 fd 9
>> Python version: 3.6.8 (default, Jan 30 2019, 23:54:38) [GCC 6.4.0]
>> Python main interpreter initialized at 0x55dfaa41c980
>> python threads support enabled
>> your server socket listen backlog is limited to 100 connections
>> your mercy for graceful operations on workers is 60 seconds
>> [uwsgi-cron] command "./manage.py runjobs minutely" registered as
>> cron task
>> [uwsgi-cron] command "./manage.py runjobs quarter_hourly" registered
>> as cron task
>> [uwsgi-cron] command "./manage.py runjobs hourly" registered as cron
>> task
>> [uwsgi-cron] command "./manage.py runjobs daily" registered as cron task
>> [uwsgi-cron] command "./manage.py runjobs monthly" registered as
>> cron task
>> [uwsgi-cron] command "./manage.py runjobs weekly" registered as cron
>> task
>> [uwsgi-cron] command "./manage.py runjobs yearly" registered as cron
>> task
>> mapped 208576 bytes (203 KB) for 4 cores
>> *** Operational MODE: threaded ***
>> WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter
>> 0x55dfaa41c980 pid: 1 (default app)
>> *** uWSGI is running in multiple interpreter mode ***
>> spawned uWSGI master process (pid: 1)
>> spawned uWSGI worker 1 (pid: 40, cores: 4)
>> Sat Apr 27 22:50:18 2019 - [uwsgi-cron] running "./manage.py runjobs
>> minutely" (pid 45)
>> [uwsgi-daemons] spawning "./manage.py qcluster" (uid: 82 gid: 82)
>> 22:50:21 [Q] INFO Q Cluster-47 starting.
>> 22:50:21 [Q] INFO Process-1:1 ready for work at 59
>> 22:50:21 [Q] INFO Process-1:2 ready for work at 60
>> 22:50:21 [Q] INFO Process-1:3 ready for work at 61
>> 22:50:21 [Q] INFO Process-1:4 ready for work at 62
>> 22:50:21 [Q] INFO Process-1:5 monitoring at 63
>> 22:50:21 [Q] INFO Process-1 guarding cluster at 58
>> 22:50:21 [Q] INFO Process-1:6 pushing tasks at 64
>> 22:50:21 [Q] INFO Q Cluster-47 running.
>> 22:59:31 [Q] INFO Enqueued 3403
>> 22:59:31 [Q] INFO Process-1:1 processing [update_from_mailman]
>> 22:59:33 [Q] INFO Processed [update_from_mailman]
>> Sat Apr 27 23:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> quarter_hourly" (pid 73)
>> Sat Apr 27 23:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> hourly" (pid 74)
>> [uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
>> with pid 73 exited after 64 second(s)
>> 23:01:28 [Q] INFO Enqueued 3404
>> 23:01:29 [Q] INFO Process-1:2 processing
>> [rebuild_mailinglist_cache_recent]
>> [uwsgi-cron] command "./manage.py runjobs hourly" running with pid
>> 74 exited after 91 second(s)
>> Sat Apr 27 23:01:36 2019 - uwsgi_response_write_body_do(): Broken
>> pipe [core/writer.c line 341] during GET / (212.203.58.154)
>> OSError: write error
>> 23:01:36 [Q] INFO Processed [rebuild_mailinglist_cache_recent]
>> Sat Apr 27 23:15:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> quarter_hourly" (pid 88)
>> [uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
>> with pid 88 exited after 4 second(s)
>> 23:28:24 [Q] INFO Enqueued 3405
>> 23:28:24 [Q] INFO Process-1:3 processing [update_from_mailman]
>> 23:28:25 [Q] INFO Processed [update_from_mailman]
>> Sat Apr 27 23:30:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> quarter_hourly" (pid 96)
>> [uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
>> with pid 96 exited after 4 second(s)
>> 23:44:40 [Q] INFO Enqueued 3406
>> 23:44:40 [Q] INFO Process-1:4 processing [update_from_mailman]
>> 23:44:41 [Q] INFO Processed [update_from_mailman]
>> Sat Apr 27 23:45:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> quarter_hourly" (pid 104)
>> [uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
>> with pid 104 exited after 4 second(s)
>> Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> quarter_hourly" (pid 113)
>> Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> hourly" (pid 114)
>> Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> daily" (pid 115)
>> Sun Apr 28 00:00:00 2019 - [uwsgi-cron] running "./manage.py runjobs
>> weekly" (pid 116)
>> [uwsgi-cron] command "./manage.py runjobs quarter_hourly" running
>> with pid 113 exited after 55 second(s)
>> [uwsgi-cron] command "./manage.py runjobs weekly" running with pid
>> 116 exited after 55 second(s)
>> 00:01:36 [Q] INFO Enqueued 3407
>> 00:01:36 [Q] INFO Process-1:1 processing
>> [rebuild_mailinglist_cache_recent]
>> [uwsgi-cron] command "./manage.py runjobs hourly" running with pid
>> 114 exited after 99 second(s)
>> 00:01:50 [Q] INFO Processed [rebuild_mailinglist_cache_recent]
>> 00:04:52 [Q] INFO Enqueued 3408
>> 00:04:52 [Q] INFO Process-1:2 processing [update_from_mailman]
>> 00:04:54 [Q] INFO Processed [update_from_mailman]
>>
>> Dockerfile:
>>
>> FROM python:3.6-alpine3.7 # Add startup script to container COPY
>> assets/docker-entrypoint.sh /usr/local/bin/ # Install packages and
>> dependencies for hyperkitty and add user for executing apps. # It's
>> important that the user has the UID/GID 82 so nginx can access the
>> files. RUN set -ex \&& apk add --no-cache --virtual .build-deps gcc
>> libc-dev linux-headers git \postgresql-dev \&& apk add --no-cache
>> --virtual .mailman-rundeps bash sassc mailcap \postgresql-client
>> curl \&& pip install -U django==2.2 \&& pip install
>> git+https://gitlab.com/eestec/mailmanclient
>>
>> \git+https://gitlab.com/mailman/hyperkitty@c9fa4d4bfc295438d3e01cd93090064d004cf44d
>> \git+https://gitlab.com/eestec/django-mailman3 \whoosh \uwsgi
>> \psycopg2 \dj-database-url \typing \&& apk del .build-deps \&&
>> addgroup -S -g 82 hyperkitty \&& adduser -S -u 82 -G hyperkitty
>> hyperkitty \&& chmod u+x /usr/local/bin/docker-entrypoint.sh# Add
>> needed files for uwsgi server + settings for django COPY
>> assets/__init__.py /home/hyperkittyCOPY assets/manage.py
>> /home/hyperkittyCOPY assets/urls.py /home/hyperkittyCOPY
>> assets/wsgi.py /home/hyperkittyCOPY assets/uwsgi.ini
>> /home/hyperkittyCOPY assets/settings.py /home/hyperkitty# Change
>> ownership for uwsgi+django files and set execution rights for
>> management script RUN chown -R hyperkitty /home/hyperkitty && chmod
>> u+x /home/hyperkitty/manage.py# Make sure we are in the correct
>> working dir WORKDIR /home/hyperkittyEXPOSE 8080 8081# Use stop
>> signal for uwsgi server STOPSIGNAL SIGINTENTRYPOINT
>> ["docker-entrypoint.sh"]CMD ["uwsgi", "--ini",
>> "/home/hyperkitty/uwsgi.ini"]
>>
>> On 4/27/19 7:58 PM, Abhilash Raj wrote:
>>> On Sat, Apr 27, 2019, at 9:40 AM, Alain Kohli wrote:
>>>> I have run "python manage.py rebuild_index" before, doesn't that do
>>>> clear_index as well? Apart from that, I run hyperkitty in a docker
>>>> container and didn't know fulltext_index should be persistent, so that
>>>> got deleted after every version update for sure.
>>> Which images are you using and how are you deploying them?
>>>
>>> You should persist fulltext_index, yes, and possibly logs if you need
>>> them for debugging later.
>>>
>>> Can you paste the entire error traceback?
>>>
>>>> On 4/26/19 10:18 PM, Mark Sapiro wrote:
>>>>> On 4/26/19 11:14 AM, Alain Kohli wrote:
>>>>>> I see loads of "A string literal cannot contain NUL (0x00) characters."
>>>>>> messages, but I haven't found missing messages in the archives yet. Not
>>>>>> sure how that could be related, though. Apart from that I don't see
>>>>>> anything unusual. The other jobs (quarter_hourly, hourly, etc.) seem to
>>>>>> run and finish normally.
>>>>> Did you upgrade from a Python 2.7 version of HyperKitty to a Python 3
>>>>> version? The Haystack/Whoosh search engine databases are not compatible
>>>>> between the two and "A string literal cannot contain NUL (0x00)
>>>>> characters." is the symptom.
>>>>>
>>>>> You need to run 'python manage.py clear_index' or just remove all the
>>>>> files from the directory defined as 'PATH' under HAYSTACK_CONNECTIONS in
>>>>> your settings file (normally 'fulltext_index' in the same directory that
>>>>> contains your settings.py.
>>>>>
>>>> _______________________________________________
>>>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>>>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>>>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>>>
>> _______________________________________________
>> Mailman-users mailing list -- mailman-users(a)mailman3.org
>> To unsubscribe send an email to mailman-users-leave(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
5 years
Re: config incoming email in my domain cpanel
by tlhackque
You've said you're working with labbrands.com.
labbrands.com has address 192.185.51.89
mail.labbrands.com has address 192.185.51.89
www.labbrands.com is an alias for labbrands.com.
telnet proves that Exim is running on that IP address. Either you're
running it, or hostgator is redirecting the SMTP ports.
You added:
lists.labbrands.com has address 190.145.27.66
Presumably that's a different machine.
So that now needs an MX record, and a firewall routing. And the SPF
records ... and spam and virus filters, and all the other stuff you need
to setup an independent, public-facing smtp server.
But at least you won't have to relay.
Good luck.
On 03-Aug-17 16:55, Rafael Mora wrote:
> El jue., 3 ago. 2017 a las 15:49, tlhackque via Mailman-users (<
> mailman-users(a)mailman3.org>) escribió:
>
>> I think you're confused.
>>
>> You already have mail.labbrands.com set up as the MX record for
>> labbrands.com. And it has an A record with the same address as your
>> webserver.
>>
> I'm working with the hostgator mailserver, we are not running a local
> mailserver.
>
> As suggested I added an A record like this:
> [image: image.png]
>
> Is it correct? is it redirecting to my Ip so I can redirect it to my
> postfix/mm3 server?
>
>
>> So if you're getting e-mail on that domain, there's another e-mail
>> server running on that IP address. You can't have 2 servers on port
>> 25. In that case, as has been noted before, you'll need to setup a
>> relay in that server, not a firewall redirect. Depending on your MTA,
>> you would need to relay to your internal server. And make sure your
>> firewall setup allows your MTA to do this.
>>
>> We can see it's EXIM:
>>
>> telnet mail.labbrands.com 25
>> Trying 192.185.51.89...
>> Connected to mail.labbrands.com (192.185.51.89).
>> Escape character is '^]'.
>> help
>> 220-gator4137.hostgator.com ESMTP Exim 4.87 #1 Thu, 03 Aug 2017 15:47:48
>> -0500
>> 220-We do not authorize the use of this system to transport unsolicited,
>> 220 and/or bulk e-mail.
>> 214-Commands supported:
>> 214 AUTH STARTTLS HELO EHLO MAIL RCPT DATA NOOP QUIT RSET HELP
>> quit
>> 221 gator4137.hostgator.com closing connection
>> Connection closed by foreign host.
>>
>> Or, consolidate all your e-mail to one server - which is a lot easier to
>> manage unless you have a really big operation. Postfix is probably the
>> right choice, but requires more setup.
>>
>> Although Mailman3 configuration is not well documented (as you've
>> discovered), you may want to get help from someone with more general
>> network and mail experience. You're now into territory that is, as
>> Simon indicated, not Mailman-specific.
>>
>> On 03-Aug-17 16:28, Rafael Mora wrote:
>>> El jue., 3 ago. 2017 a las 15:26, Mark Sapiro (<mark(a)msapiro.net>)
>> escribió:
>>>> On 08/03/2017 01:22 PM, Rafael Mora wrote:
>>>>> El jue., 3 ago. 2017 a las 15:18, Mark Sapiro (<mark(a)msapiro.net>)
>>>> escribió:
>>>>>> You need to forward port 25 for SMTP mail delivery and if you want the
>>>>>> web UI (Postorius and HyperKitty) accessible from the outside, port 80
>>>>>> for http and/or port 443 for https
>>>>>> <
>>>>>>
>> https://www.iana.org/assignments/service-names-port-numbers/service-names-p…
>>>>>>> .
>>>>> I mean for incoming mail redirected from my hostgator hosting, because
>>>> when
>>>>> I suscribe an email address MM3 sends a confirmation email, so I have
>> to
>>>>> reply to be suscribed to the list.
>>>> As I said, for mail delivery you need to forward port 25 to the Mailman
>>>> server.
>>>>
>>> Ok so I'll redirect in my zentyal firewall the port 25 to my local
>> centos7
>>> with postfix and MM3 with IP 192.168.1.42. Thanks Mark.
>>>
>>>
>>>> --
>>>> Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers,
>>>> San Francisco Bay Area, California better use your sense - B. Dylan
>>>> _______________________________________________
>>>> Mailman-users mailing list
>>>> mailman-users(a)mailman3.org
>>>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>>>
>> --
>> This communication may not represent my employer's views,
>> if any, on the matters discussed.
>>
>> _______________________________________________
>> Mailman-users mailing list
>> mailman-users(a)mailman3.org
>> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
>>
--
This communication may not represent my employer's views,
if any, on the matters discussed.
6 years, 9 months
Migration from old server to new server failed
by Helio Loureiro
Hi,
So we tried to the migration following the steps:
- stop mailman in the old machine (v3.1.1 from ubuntu packages)
- dump DB mailman3 and mailman3web
- backup /var/lib/mailman3 content
On the new machine:
- checkout from git the version 3.1.1 to match production and install the
services
- stop mailman3 and mailman3-web on the new machine
- load DBs into new machine
- copy content from /var/lib/mailman3 into /local/mailman/var (which
means: archives/ cache/ data/ ext/ lists/ locks/ messages/
public_suffix_list.dat queue/ templates/ web/)
- start mailman3 and mailman-3web
- run mailman3-web migrate
And I stopped here. Because it crashed.
(venv) mailman@new-mailman3-server ~ (v3.1.1) [0|SIGINT]> mailman-web
migrate
System check identified some issues:
WARNINGS:
account.EmailAddress: (models.W036) MariaDB does not support unique
constraints with conditions.
HINT: A constraint won't be created. Silence this warning if you don't care
about it.
account.EmailAddress: (models.W043) MariaDB does not support indexes on
expressions.
HINT: An index won't be created. Silence this warning if you don't care
about it.
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes,
django_mailman3, django_q, hyperkitty, postorius, sessions, sites,
socialaccount
Running migrations:
Applying account.0003_alter_emailaddress_create_unique_verified_email...
OK
Applying account.0004_alter_emailaddress_drop_unique_email... OK
Applying account.0005_emailaddress_idx_upper_email... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying django_mailman3.0003_sessions... OK
Applying django_q.0010_auto_20200610_0856... OK
Applying django_q.0011_auto_20200628_1055... OK
Applying django_q.0012_auto_20200702_1608... OK
Applying django_q.0013_task_attempt_count... OK
Applying django_q.0014_schedule_cluster... OK
Applying hyperkitty.0016_auto_20180309_0056... OK
Applying hyperkitty.0017_file_attachments... OK
Applying hyperkitty.0018_threadcategory_color... OK
Applying hyperkitty.0019_auto_20190127_null_description... OK
Applying hyperkitty.0020_auto_20190907_1927... OK
Applying hyperkitty.0021_add_owners_mods...Traceback (most recent call
last):
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 261, in query
_mysql.connection.query(self, query)
MySQLdb.OperationalError: (1050, "Table 'hyperkitty_mailinglist_moderators'
already exists")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/local/mailman/venv/bin/mailman-web", line 8, in <module>
sys.exit(main())
File
"/local/mailman/venv/lib/python3.10/site-packages/mailman_web/manage.py",
line 90, in main
execute_from_command_line(sys.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 446, in execute_from_command_line
utility.execute()
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/__init__.py",
line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 402, in run_from_argv
self.execute(*args, **cmd_options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 448, in execute
output = self.handle(*args, **options)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/base.py",
line 96, in wrapped
res = handle_func(*args, **kwargs)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py",
line 349, in handle
post_migrate_state = executor.migrate(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 135, in migrate
state = self._migrate_all_forwards(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 167, in _migrate_all_forwards
state = self.apply_migration(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/executor.py",
line 252, in apply_migration
state = migration.apply(state, schema_editor)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/migration.py",
line 130, in apply
operation.database_forwards(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py",
line 108, in database_forwards
schema_editor.add_field(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/schema.py",
line 104, in add_field
super().add_field(model, field)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 636, in add_field
return self.create_model(field.remote_field.through)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 447, in create_model
self.execute(sql, params or None)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py",
line 199, in execute
cursor.execute(sql, params)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 67, in execute
return self._execute_with_wrappers(
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 84, in _execute
with self.db.wrap_database_errors:
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/utils.py", line
91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/utils.py",
line 87, in _execute
return self.cursor.execute(sql)
File
"/local/mailman/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py",
line 75, in execute
return self.cursor.execute(query, args)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
179, in execute
res = self._query(mogrified_query)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line
330, in _query
db.query(q)
File
"/local/mailman/venv/lib/python3.10/site-packages/MySQLdb/connections.py",
line 261, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1050, "Table
'hyperkitty_mailinglist_moderators' already exists")
I tried to drop the table, but then it complained about another one. I
dropped all the complained tables and as result hypperkitty never
started. So I went back to see why the first error happened.
Any clue what to do next?
Best Regards,
Helio Loureiro
https://helio.loureiro.eng.br
https://github.com/helioloureiro
https://mastodon.social/@helioloureiro
3 months
Time Stand Still
by Barry Warsaw
Somewhen in the dark recesses of intarweb history, I found myself as the project leader for both Jython (née JPython) and GNU Mailman. I'd been involved with Jython since it was invented by Jim Hugunin around the time he came to work with us at Pythonlabs. I'd been contributing to Mailman since we inherited John Viega's Python-based Dave Matthews Band list server, and put it to use replacing python.org's Majordomo installation.
I'd enjoyed both projects, but knew I could not lead both, so I had to make a choice. I chose to turn over Jython to a team that's done a much better job over the years than I ever could have. Something about email, and especially the communication and collaboration patterns that it facilitates, really fascinated me. I know, I know, but we all have our lapses of sanity. Mine has lasted almost 20 years, a bit more than "momentary" perhaps.
I've rarely gotten paid to work on Mailman, but it did provide me some great opportunities. Most notably it led to my 10 year stint at Canonical. I was originally hired on there to integrate mailing lists with Launchpad, and Mailman was the obvious choice. I learned a ton doing that project, and working within the constraints of integrating the two Python-based systems, especially since Launchpad was originally not free software and Mailman was GPL'd. Later, the Zope-based Launchpad source code was released under the AGPL, making much of the monkeypatching unnecessary, but by then the system was solid and reliable, and you don't fix what's not broken.
Except, I guess I did. I took a lot of the lessons from that work, along with a good hard look at all the problems with Mailman 2, and began to break another cardinal rule of software development: second system syndrome. The result is Mailman 3. It took forever, and we're still not at complete feature parity with Mailman 2, but at least it's Real Enough to be used at many Real Sites, including python.org and lists.fedoraprojects.org.
It would be ridiculous for me to take significant credit for this. I have to acknowledge the amazing user community -- you! -- for all the support, patches, suggestions, feedback, patience, criticism, donations, and contributions that you've given to the project, and to me personally over the years. And my deepest gratitude goes to all the core developers that have stayed or come and gone, but most especially the current Cabal: Abhilash Raj, Aurelien Bompard, Florian Fuchs, Mark Sapiro, Stephen J. Turnbull, Terri Oda. You should know that each and every one of them is truly awesome, both in what they contribute technically, and in their amazing friendships. Mailman is infinitely better because of their involvement, and I've loved spending time with them over the years at the Pycon sprints, making releases and sharing teas and meals.
My blog is called We Fear Change, and that's humorously taken from a 90's bit in Mike Myer's excellent Wayne's World movie (a phrase actually uttered by the brilliant Dana Carvey as Garth). The irony of course is that while we all may fear change, it's the one constant thing we can count on. And in fact, we *require* change to thrive, because if you aren't changing, you aren't alive. Time, and being engaged with life's vagaries, means there's no alternative to change; it must be embraced.
And so, with a vague reference to the many (good!) changes in my personal and professional life, I'm announcing that I'm stepping down from the project leadership role of GNU Mailman, effective... nowish! And it's with unanimous agreement among the GNU Mailman Steering Committee (a.k.a. the Mailman Cabal), that we are announcing Abhilash Raj as the new project leader.
If you don't recognize Abhilash's name, you probably aren't paying attention, at least to Mailman 3. Abhilash came to us in 2013 as a Google Summer of Code student, and he's become one of the project's most valuable contributors. His list of accomplishments is long, and it includes everything from redesigning the website, to integrating CI with our GitLab build system, porting our code to the SQLAlchemy ORM, adding MySQL support, revving up adoption through his Docker images, along with his great coding work on Core, Postorius, HyperKitty, and mailmanclient.
This transition is good for the project too. Email, its defining protocols and standards, and its role in our daily lives, has changed profoundly since the early days of Mailman. A fresh perspective and enthusiasm will help keep Mailman relevant to the changing ways we -- especially the FLOSS and tech communities -- communicate.
Please join me in supporting Abhilash in every way possible as he takes over in this new role as project leader. I'll be here when and if needed, even as I create space in my "spare" time for... Something Else. I look forward to the vision that Abhilash will bring to the project, and I know that he will do a great job. To me, Mailman has always been about collaboration, and the best
way for it to succeed is for you to continue to contribute your insights, experiences, opinions, and skills with positive intention.
-Barry
https://www.wefearchange.org/
6 years, 5 months