makemigrations --merge error after Debian upgrade
Hi,
I upgraded my debian version, and since, it seems that I have a problem. I tried mailman-web migrate (and mailman-web collectstatic etc.) and I get the error message:
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0010_auto_20220618_2315, 0014_schedule_cluster in django_q; 0014_auto_20220618_2315, 0018_alter_emailtemplate_language in postorius).
To fix them run 'python manage.py makemigrations --merge'
So I tried mailman-web makemigrations --merge that ended with
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/postorius/migrations/0019_merge_20240930_1159.py'
There is no such a file in the migration directory. Any idea ?
Thanks in advance, All the best
Maxime Chupin
On 9/30/24 03:00, Chupin Maxime via Mailman-users wrote:
Hi, I upgraded my debian version,
Issues with the Debian Mailman packages should be reported to Debian. Please see https://wiki.list.org/x/12812344 and https://mail.python.org/archives/list/mailman-users@python.org/message/T6LB2...
and since, it seems that I have a problem. I tried
mailman-web migrate(andmailman-web collectstaticetc.) and I get the error message:CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0010_auto_20220618_2315, 0014_schedule_cluster in django_q; 0014_auto_20220618_2315, 0018_alter_emailtemplate_language in postorius). To fix them run 'python manage.py makemigrations --merge'So I tried
mailman-web makemigrations --mergethat ended withPermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/postorius/migrations/0019_merge_20240930_1159.py'
Did you run this as the Mailman user (list in the Debian packages)?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Oct 1, 2024 at 12:36 PM Chupin Maxime via Mailman-users < mailman-users@mailman3.org> wrote:
Mark Sapiro wrote:
Did you run this as the Mailman user (list in the Debian packages)?
Thank you, I tried, but the system asked me a Password for this user... I do not understand.
sudo -u list mailman-web makemigrations --merge Password:
The user executing sudo is yourself so enter your own password.
It's not the password for user "list" that is wanted. It's your own login
password.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Odhiambo Washington wrote:
The user executing sudo is yourself so enter your own password. It's not the password for user "list" that is wanted. It's your own login password.
No, because I’m root and I can do :
root@machine: sudo -u list bash
and then
list@machine: mailman-web makemigrations --merge
Password:
Hello,
On Tue, Oct 01, 2024 at 11:08:19AM -0000, Chupin Maxime via Mailman-users wrote:
root@machine: sudo -u list bash
If you are root already, you could also use su:
su - -u list -s /bin/bash -c "mailman-web makemigrations --merge"
(the -s /bin/bash is because the list account has a /usr/sbin/nologin shell in /etc/passwd)
Also note that Debian has a wrapper for mailman commands (but not mailman-web commands), for example:
root@ds-services-mailman:~# mailman-wrapper version GNU Mailman 3.3.8 (Tom Sawyer)
This wrapper, when run from root (sudo mailman-wrapper or just mailman-wrapper under the root user) changes the user to the list user:
root@ds-services-mailman:~# cat $(which mailman-wrapper) #!/bin/sh
su -s /bin/sh -c "mailman $*" list
Not very different from my su above.
On 10/1/24 06:49, Chupin Maxime via Mailman-users wrote:
Thank you, but still asking for Password to execute mailman-web...
Look at the contents of the mailman-web script. My guess is it contains
sudo -u list ...
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, Oct 01, 2024 at 08:10:54PM -0700, Mark Sapiro wrote:
Look at the contents of the mailman-web script. My guess is it contains
sudo -u list ...
Not exactly:
(debian bookworm)
su -s /bin/sh -c "python3 /usr/share/mailman3-web/manage.py $*" www-data
if run from root, it should not require a password.
Thank you. So, using mailman-web makemigrations --merge as root is the right way to do it (as it call user www-data). Unfortunately, I get the error wrote in my first message:
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/django_q/migrations/0015_merge_0010_auto_20220618_2315_0014_schedule_cluster.py'
Hello,
On Wed, Oct 02, 2024 at 08:05:44AM -0000, Chupin Maxime via Mailman-users wrote:
Thank you. So, using
mailman-web makemigrations --mergeasrootis the right way to do it (as it call userwww-data). Unfortunately, I get the error wrote in my first message: PermissionError: [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/django_q/migrations/0015_merge_0010_auto_20220618_2315_0014_schedule_cluster.py'
I did not use makemigrations so far.
It is well possible that files that were prepared were on the wrong UID, check
ls -la /usr/lib/python3/dist-packages/django_q/
(especially since debian separates mailman3 and mailman3-web users)
You could change the owner manually (chown www-data '/usr/lib/python3/dist-packages/django_q/*), but I prefer you get advice from others on the list.
On 10/2/24 05:55, Chupin Maxime via Mailman-users wrote:
That does not change anything... :(
What user ran the makemigrations --merge?
What does ls -la /usr/lib/python3/dist-packages/django_q/migrations/ show?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro wrote:
What user ran the makemigrations --merge?
cat /usr/bin/mailman-web
#!/bin/sh
su -s /bin/sh -c "python3 /usr/share/mailman3-web/manage.py $*" www-data
What does ls -la /usr/lib/python3/dist-packages/django_q/migrations/ show?
that all the files belong to www-data... but know I get the error for /usr/lib/python3/dist-packages/postorius/migrations. So I change the owner and everything works know.
I’ve done :
root@machine: mailman-web makemigration --merge
and then
root@machine: mailman-web migrate
and now, hyperkitty is working well!!!
Thank you to all of you.
Another error message from the cron executions:
data-www@machine: /usr/share/mailman3-web/manage.py runjobs minutely
/usr/lib/python3/dist-packages/django_q/conf.py:139: UserWarning: Retry and timeout are misconfigured. Set retry larger than timeout,
failure to do so will cause the tasks to be retriggered before completion.
See https://django-q.readthedocs.io/en/latest/configure.html#retry for details.
warn(
On Tue, Oct 01, 2024 at 11:29:52AM -0000, Chupin Maxime via Mailman-users wrote:
Another error message from the cron executions:
Under Debian, I wrote a script for fixing up a few things, you can find it here:
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
You might be interested by:
# work-around for django_q
# (does not seem to work, and DEFAULT_AUTO_FIELD should be also
# configured to django.db.models.BigAutoField)
dpkg-divert --rename /usr/lib/python3/dist-packages/django_q/conf.py
sed < /usr/lib/python3/dist-packages/django_q/conf.py.distrib
> /usr/lib/python3/dist-packages/django_q/conf.py
's%TIMEOUT = conf.get("timeout", None)%TIMEOUT = conf.get("timeout", 59)%'
Thanks. Unfortunately, whereas TIMEOUT is set to 59 and RETRY to 60, I still get the warn... Very weird.
On 10/1/24 06:46, Chupin Maxime via Mailman-users wrote:
Thanks. Unfortunately, whereas TIMEOUT is set to 59 and RETRY to 60, I still get the warn... Very weird.
We recommend
Q_CLUSTER = { 'timeout': 300, 'retry': 360, 'save_limit': 100, 'orm': 'default', }
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Tried to add these lines before the test in the conf.pyand nothing changes. Should I remove the test on TIMEOUT ?
# Number of seconds to wait for a worker to finish.
TIMEOUT = conf.get("timeout", 58)
# Whether to acknowledge unsuccessful tasks.
# This causes failed tasks to be considered delivered, thereby removing them from
# the task queue. Defaults to False.
ACK_FAILURES = conf.get("ack_failures", False)
# Number of seconds to wait for acknowledgement before retrying a task
# Only works with brokers that guarantee delivery. Defaults to 60 seconds.
RETRY = conf.get("retry", 60)
Q_CLUSTER = {
'timeout': 300,
'retry': 360,
'save_limit': 100,
'orm': 'default',
}
# Verify if retry and timeout settings are correct
if not TIMEOUT or (TIMEOUT > RETRY):
warn(
"""Retry and timeout are misconfigured. Set retry larger than timeout,
failure to do so will cause the tasks to be retriggered before completion.
See https://django-q.readthedocs.io/en/latest/configure.html#retry for details."""
)
On 10/2/24 05:58, Chupin Maxime via Mailman-users wrote:
Tried to add these lines before the test in the
conf.pyand nothing changes. Should I remove the test on TIMEOUT ?
You should be reporting these issues (this and the migrations issue) to Debian. We don't know how to deal with Debian packaging issues.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (4)
-
Chupin Maxime -
Marc SCHAEFER -
Mark Sapiro -
Odhiambo Washington