Has MM3 been tested with Python3.11? I updated a Debian VM to Python3.11, installed MM3 and migrated the configs. However, I am unable to get past the following error:
root@debian:/home/wash# less /opt/mailman/web/logs/uwsgi-error.log worker 2 buried after 1 seconds goodbye to uWSGI. *** Starting uWSGI 2.0.21 (64bit) on [Mon Mar 6 14:27:20 2023] *** compiled with version: 10.2.1 20210110 on 21 February 2023 07:16:19 os: Linux-6.1.0-5-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.12-1 (2023-02-15) nodename: debian machine: x86_64 clock source: unix detected number of CPU cores: 16 current working directory: / writing pidfile to /opt/mailman/mm/var/uwsgi.pid detected binary path: /opt/mailman/venv/bin/uwsgi !!! no internal routing support, rebuild with pcre support !!! your processes number limit is 31427 your memory page size is 4096 bytes detected max file descriptor number: 1024 lock engine: pthread robust mutexes thunder lock: disabled (you can enable it with --thunder-lock) uwsgi socket 0 bound to TCP address 0.0.0.0:8000 fd 8 uwsgi socket 1 bound to UNIX address /opt/mailman/mm/var/uwsgi.sock fd 9 Python version: 3.11.2 (main, Feb 21 2023, 02:03:40) [GCC 10.2.1 20210110] PEP 405 virtualenv detected: /opt/mailman/venv/ Set PythonHome to /opt/mailman/venv/ Python main interpreter initialized at 0x560130dd7df8 python threads support enabled your server socket listen backlog is limited to 100 connections your mercy for graceful operations on workers is 60 seconds mapped 250128 bytes (244 KB) for 4 cores *** Operational MODE: preforking+threaded *** Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.11/site-packages/mailman_web/wsgi.py", line 10, in <module> from django.core.wsgi import get_wsgi_application File "/opt/mailman/venv/lib/python3.11/site-packages/django/__init__.py", line 1, in <module> from django.utils.version import get_version File "/opt/mailman/venv/lib/python3.11/site-packages/django/utils/version.py", line 1, in <module> import datetime File "/usr/lib/python3.11/datetime.py", line 12, in <module> import math as _math ModuleNotFoundError: No module named 'math' unable to load app 0 (mountpoint='') (callable not found or import error) *** no app loaded. going in full dynamic mode *** *** uWSGI is running in multiple interpreter mode *** spawned uWSGI master process (pid: 117115) spawned uWSGI worker 1 (pid: 117118, cores: 2) spawned uWSGI worker 2 (pid: 117119, cores: 2) [uwsgi-daemons] spawning "/opt/mailman/venv/bin/mailman-web qcluster" (uid: 1001 gid: 1001) 11:27:20 [Q] INFO Q Cluster vegan-arizona-massachusetts-texas starting. 11:27:20 [Q] INFO Process-1:1 ready for work at 117129 11:27:20 [Q] INFO Process-1:2 ready for work at 117130 11:27:20 [Q] INFO Process-1:3 ready for work at 117131 11:27:20 [Q] INFO Process-1:4 ready for work at 117132 11:27:20 [Q] INFO Process-1:5 ready for work at 117133 11:27:20 [Q] INFO Process-1:6 ready for work at 117134 11:27:20 [Q] INFO Process-1:7 ready for work at 117135 11:27:20 [Q] INFO Process-1:8 ready for work at 117136 11:27:20 [Q] INFO Process-1:9 ready for work at 117137 11:27:20 [Q] INFO Process-1:10 ready for work at 117138 11:27:20 [Q] INFO Process-1:11 ready for work at 117139 11:27:20 [Q] INFO Process-1:12 ready for work at 117140 11:27:20 [Q] INFO Process-1:13 ready for work at 117141 11:27:20 [Q] INFO Process-1:14 ready for work at 117142 11:27:20 [Q] INFO Process-1:15 ready for work at 117143 11:27:20 [Q] INFO Process-1:16 ready for work at 117144 11:27:20 [Q] INFO Process-1:17 monitoring at 117145 11:27:20 [Q] INFO Process-1 guarding cluster vegan-arizona-massachusetts-texas 11:27:20 [Q] INFO Process-1:18 pushing tasks at 117146 11:27:20 [Q] INFO Q Cluster vegan-arizona-massachusetts-texas running. --- no python application found, check your startup logs for errors --- --- no python application found, check your startup logs for errors ---
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 3/6/23 03:38, Odhiambo Washington wrote:
Has MM3 been tested with Python3.11?
Yes, but not with uWSGI which is not part of Mailman per se.
I updated a Debian VM to Python3.11, installed MM3 and migrated the configs. However, I am unable to get past the following error:
...
Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.11/site-packages/mailman_web/wsgi.py", line 10, in <module> from django.core.wsgi import get_wsgi_application File "/opt/mailman/venv/lib/python3.11/site-packages/django/__init__.py", line 1, in <module> from django.utils.version import get_version File "/opt/mailman/venv/lib/python3.11/site-packages/django/utils/version.py", line 1, in <module> import datetime File "/usr/lib/python3.11/datetime.py", line 12, in <module> import math as _math ModuleNotFoundError: No module named 'math' unable to load app 0 (mountpoint='') (callable not found or import error)
This error says /usr/lib/python3.11/datetime.py can't import math. math is part of the python 3.11 standard library <https://docs.python.org/3/library/math.html>. There is something wrong with your Python 3.11 installation.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, Mar 6, 2023 at 7:10 PM Mark Sapiro <mark@msapiro.net> wrote:
On 3/6/23 03:38, Odhiambo Washington wrote:
Has MM3 been tested with Python3.11?
Yes, but not with uWSGI which is not part of Mailman per se.
Oh. I can easily switch between uWSGI, Gunicorn, or mod_wsgi.
I updated a Debian VM to Python3.11, installed MM3 and migrated the
configs. However, I am unable to get past the following error:
...
Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.11/site-packages/mailman_web/wsgi.py", line 10, in <module> from django.core.wsgi import get_wsgi_application File "/opt/mailman/venv/lib/python3.11/site-packages/django/__init__.py", line 1, in <module> from django.utils.version import get_version File "/opt/mailman/venv/lib/python3.11/site-packages/django/utils/version.py", line 1, in <module> import datetime File "/usr/lib/python3.11/datetime.py", line 12, in <module> import math as _math ModuleNotFoundError: No module named 'math' unable to load app 0 (mountpoint='') (callable not found or import error)
This error says /usr/lib/python3.11/datetime.py can't import math. math is part of the python 3.11 standard library <https://docs.python.org/3/library/math.html>. There is something wrong with your Python 3.11 installation.
So I will redo the venv by invoking the python binaries explicitly, in case I have conflicts. Thanks.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On Mon, Mar 6, 2023 at 7:10 PM Mark Sapiro <mark@msapiro.net> wrote:
On 3/6/23 03:38, Odhiambo Washington wrote:
Has MM3 been tested with Python3.11?
Yes, but not with uWSGI which is not part of Mailman per se.
Are there any specific tests that are required to be done with respect to uWSGI and mod_wsgi? I could run some of those in my VM. I have switched from uWSGI to mod_wsgi and I don't have any errors - so far. Web UI running without issues in both cases. I believe I do not need to test anything with Gunicorn as that is what the Developers are using all the time :)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 3/7/23 05:37, Odhiambo Washington wrote:
Are there any specific tests that are required to be done with respect to uWSGI and mod_wsgi?
If your web UI is working to access Postorius, HyperKitty and user accounts, your wsgi interface works.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Odhiambo Washington writes:
Has MM3 been tested with Python3.11?
Not heavily.
PEP 405 virtualenv detected: /opt/mailman/venv/
Have you recreated the venv from scratch, or did you try to upgrade it (or did you not upgrade it at all)?
import math as _mathModuleNotFoundError: No module named 'math'
This is not a version incompatibility in the sense that Python 3.11 does something different from earlier versions with the same code. This is Python 3.11 saying that your installation Python 3.11 is broken because it can't find stdlib modules that are bundled with it.
Try "python3.11 -c 'import math; print(dir(math))" (with the venv deactivated). It should produce a list of about 60 attribute names. If that fails, your Python 3.11 itself is broken. If that succeeds, python3.11 is incompatible with the venv. My only guess is that the venv contains the math module from an earlier version of Python and 3.11 won't load it. (Note, "python -m math" is not expected to work, I forget why not.)
On Mon, Mar 6, 2023 at 7:11 PM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Odhiambo Washington writes:
Has MM3 been tested with Python3.11?
Not heavily.
Okay. So a novice is trying it out. On a volatile platform too - Debian12, and Postgresql15.
PEP 405 virtualenv detected: /opt/mailman/venv/
Have you recreated the venv from scratch, or did you try to upgrade it (or did you not upgrade it at all)?
I have created it from scratch. Only the files in /etc/mailman3/ are constantly being reused.
import math as _mathModuleNotFoundError: No module named 'math'
This is not a version incompatibility in the sense that Python 3.11 does something different from earlier versions with the same code. This is Python 3.11 saying that your installation Python 3.11 is broken because it can't find stdlib modules that are bundled with it.
I install Python3.11 manually on Debian12 (maybe this is the issue!) VM using the script below: #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations make altinstall
Try "python3.11 -c 'import math; print(dir(math))" (with the venv deactivated). It should produce a list of about 60 attribute names.
wash@debian:~$ python3.11 -c 'import math; print(dir(math))' ['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'cbrt', 'ceil', 'comb', 'copysign', 'cos', 'cosh', 'degrees', 'dist', 'e', 'erf', 'erfc', 'exp', 'exp2', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'isqrt', 'lcm', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'nextafter', 'perm', 'pi', 'pow', 'prod', 'radians', 'remainder', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc', 'ulp']
If that fails, your Python 3.11 itself is broken. If that succeeds, python3.11 is incompatible with the venv. My only guess is that the venv contains the math module from an earlier version of Python and 3.11 won't load it. (Note, "python -m math" is not expected to work, I forget why not.)
I will retry installing the venv by calling pip3.11 explicitly and see what happens. I believe that eliminates the change of conflicting Python versions, yes?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 3/6/23 08:23, Odhiambo Washington wrote:
I install Python3.11 manually on Debian12 (maybe this is the issue!) VM using the script below: #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations
You also want --enable-shared
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, Mar 6, 2023 at 7:32 PM Mark Sapiro <mark@msapiro.net> wrote:
On 3/6/23 08:23, Odhiambo Washington wrote:
I install Python3.11 manually on Debian12 (maybe this is the issue!) VM using the script below: #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations
You also want --enable-shared
I had done that, but somehow the build had failed with it. I will try it again.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On Mon, Mar 6, 2023 at 7:52 PM Odhiambo Washington <odhiambo@gmail.com> wrote:
On Mon, Mar 6, 2023 at 7:32 PM Mark Sapiro <mark@msapiro.net> wrote:
On 3/6/23 08:23, Odhiambo Washington wrote:
I install Python3.11 manually on Debian12 (maybe this is the issue!) VM using the script below: #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations
You also want --enable-shared
I had done that, but somehow the build had failed with it. I will try it again.
I managed to reinstall Python3.11 the correct way:
<CUT> #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations --enable-shared make altinstall # Set Python 3.11 as the Default update-alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3.11 1 # Install PIP using the following command if you do not have it. curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11 </CUT>
After this, I installed a fresh venv, activated and installed everything according to
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-install
MM3 is running fine with uwsgi as documented. I have not deviated even a single step from the documentation above.
Now I will test with mod_wsgi and gunicorn and report back.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 2023-03-06 19:23:48 +0300 (+0300), Odhiambo Washington wrote: [...]
I install Python3.11 manually on Debian12 (maybe this is the issue!) VM using the script below: #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations make altinstall [...]
Are you sure the python3.11 you built is on your path, and is the one getting used? Keep in mind that Debian 12 (bookworm) comes with Python 3.11 as its default python3 already, so I'm curious why you needed to compile it yourself:
https://packages.debian.org/bookworm/python3
-- Jeremy Stanley
On Mon, Mar 6, 2023 at 7:47 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2023-03-06 19:23:48 +0300 (+0300), Odhiambo Washington wrote: [...]
I install Python3.11 manually on Debian12 (maybe this is the issue!) VM using the script below: #!/bin/bash cd Python-3.11.2 ./configure --enable-optimizations make altinstall [...]
Are you sure the python3.11 you built is on your path, and is the one getting used? Keep in mind that Debian 12 (bookworm) comes with Python 3.11 as its default python3 already, so I'm curious why you needed to compile it yourself:
I installed Bullseye, then upgraded to Bookworm. That's why. A little mix in the process as Bookworm is not yet released, or is it?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
On 2023-03-06 23:47:16 +0300 (+0300), Odhiambo Washington wrote: [...]
Bookworm is not yet released, or is it?
Not yet, no. It's well into the soft freeze, transitioning to hard freeze next week but no official target date set for the full freeze and release yet (that's at least a month out still, I expect).
Jeremy Stanley
participants (4)
-
Jeremy Stanley -
Mark Sapiro -
Odhiambo Washington -
Stephen J. Turnbull