Mailman web Stopped working after upgrade
Hi.
I have a VERY strange behaviour. I upgraded mailman3 and mailman web this weekend. It works fine (apart from ARC which don't work correctly in my setup but that is ok). But tonight the web user interface stopped working and I have not been logged in since it worked last time so something very strange is going on. I get the dreadful error message about psycopg2._psycopg is missing although it is installed both in ubuntu and in pip in the venv I run mailman. Mailman core still works fine, it is just Django being flaky.
I think that this might be due to some kind of python misfeature where compiled code behaves slightly differently to interpreted code, that might explain the delay in it working initially before breaking. I have no idea of how to solve this. I have tried to uninstall and reinstlall psycopg2 in different versions without any luck.
Any ideas? Logs below.
I get the following in the web server error log (removing the web things that frankly are not interesting):
mod_wsgi (pid=4488): Target WSGI script 'MAILMANROOT/bin/mailman-web20200229.wsgi' cannot be loaded as Python module. mod_wsgi (pid=4488): Exception occurred processing WSGI script 'MAILMANROOT/bin/mailman-web20200229.wsgi'. Traceback (most recent call last): File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/backends/postgresql/base.py", line 20, in <module> import psycopg2 as Database File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/psycopg2/__init__.py", line 50, in <module> from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID ImportError: No module named 'psycopg2._psycopg'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "MAILMANROOT/bin/mailman-web20200229.wsgi", line 47, in <module> application = get_wsgi_application() File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/core/wsgi.py", line 12, in get_wsgi_application django.setup(set_prefix=False) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/apps/registry.py", line 114, in populate app_config.import_models() File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 986, in _gcd_import File "<frozen importlib._bootstrap>", line 969, in _find_and_load File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 673, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 665, in exec_module File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/postorius-1.3.2-py3.7.egg/postorius/models.py", line 27, in <module> from django.contrib.auth.models import User File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/contrib/auth/models.py", line 2, in <module> from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/contrib/auth/base_user.py", line 47, in <module> class AbstractBaseUser(models.Model): File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/models/base.py", line 117, in __new__ new_class.add_to_class('_meta', Options(meta, app_label)) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/models/base.py", line 321, in add_to_class value.contribute_to_class(cls, name) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/models/options.py", line 204, in contribute_to_class self.db_table = truncate_name(self.db_table, connection.ops.max_name_length()) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/__init__.py", line 28, in __getattr__ return getattr(connections[DEFAULT_DB_ALIAS], item) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/utils.py", line 201, in __getitem__ backend = load_backend(db['ENGINE']) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/utils.py", line 110, in load_backend return import_module('%s.base' % backend_name) File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "MAILMANROOT/venv-3.7_2020/lib/python3.7/site-packages/Django-2.2.10-py3.7.egg/django/db/backends/postgresql/base.py", line 24, in <module> raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e) django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named 'psycopg2._psycopg'
Regards // David
On 3/3/20 12:33 PM, David Krantz wrote:
I think that this might be due to some kind of python misfeature where compiled code behaves slightly differently to interpreted code, that might explain the delay in it working initially before breaking. I have no idea of how to solve this. I have tried to uninstall and reinstlall psycopg2 in different versions without any luck.
You need to activate your virtualenv and do
pip install psycopg2-binary
it must be psycopg2-binary, not just psycopg2. It's probably best to do
pip uninstall psycopg2
first.
As far as it working for a while, did you restart your wsgi service (gunicorn, uwsgi, ??) after upgrading?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi,
I tried psycopg2-binary and a few other rehashes of that. Now I got it working by downloading the source for psycopg2 manually and building it myself, then I installed it into my venv like any other self-built package. https://pypi.org/project/psycopg2/#files
I run mod_wsgi in Apache so I thought it should be enough to restart Apache. I am quite convinced that I did that (and for that matter restart the machine at a few points). But I might have been sloppy and reversed the order in the end when I got it running, this process is a bit to manual. I will use another solution when I rebuild the server one day. The problem with rebuilding the server is that I'd have to install a MTA with all its modules, milters, setups and what not, and mailman. To just reinstall on a fresh machine would be easy, the trick is that I want to rebuild some things in a better way which will take me some time. E.g. using a separate wsgi service and a more modern spam filter than Spamassassin. Otherwise it is easy. But my situation now is a bit short on time so if I can postpone that a bit I will.
Thanks. // David
// David
On Tue, Mar 3, 2020 at 10:02 PM Mark Sapiro mark@msapiro.net wrote:
On 3/3/20 12:33 PM, David Krantz wrote:
I think that this might be due to some kind of python misfeature where compiled code behaves slightly differently to interpreted code, that might explain the delay in it working initially before breaking. I have no idea of how to solve this. I have tried to uninstall and reinstlall psycopg2 in different versions without any luck.
You need to activate your virtualenv and do
pip install psycopg2-binary
it must be psycopg2-binary, not just psycopg2. It's probably best to do
pip uninstall psycopg2
first.
As far as it working for a while, did you restart your wsgi service (gunicorn, uwsgi, ??) after upgrading?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
But now that it is working the order of the mailing lists on the index page are in random order again. When it worked initially they seemed sorted by name. Whey I follow the link "see all lists" the order is correct, though. Annoying.
BR // David
On Tue, Mar 3, 2020 at 10:26 PM David Krantz gazorg@gmail.com wrote:
Hi,
I tried psycopg2-binary and a few other rehashes of that. Now I got it working by downloading the source for psycopg2 manually and building it myself, then I installed it into my venv like any other self-built package. https://pypi.org/project/psycopg2/#files
I run mod_wsgi in Apache so I thought it should be enough to restart Apache. I am quite convinced that I did that (and for that matter restart the machine at a few points). But I might have been sloppy and reversed the order in the end when I got it running, this process is a bit to manual. I will use another solution when I rebuild the server one day. The problem with rebuilding the server is that I'd have to install a MTA with all its modules, milters, setups and what not, and mailman. To just reinstall on a fresh machine would be easy, the trick is that I want to rebuild some things in a better way which will take me some time. E.g. using a separate wsgi service and a more modern spam filter than Spamassassin. Otherwise it is easy. But my situation now is a bit short on time so if I can postpone that a bit I will.
Thanks. // David
// David
On Tue, Mar 3, 2020 at 10:02 PM Mark Sapiro mark@msapiro.net wrote:
On 3/3/20 12:33 PM, David Krantz wrote:
I think that this might be due to some kind of python misfeature where compiled code behaves slightly differently to interpreted code, that might explain the delay in it working initially before breaking. I have no idea of how to solve this. I have tried to uninstall and reinstlall psycopg2 in different versions without any luck.
You need to activate your virtualenv and do
pip install psycopg2-binary
it must be psycopg2-binary, not just psycopg2. It's probably best to do
pip uninstall psycopg2
first.
As far as it working for a while, did you restart your wsgi service (gunicorn, uwsgi, ??) after upgrading?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
On 3/3/20 1:52 PM, David Krantz wrote:
But now that it is working the order of the mailing lists on the index page are in random order again. When it worked initially they seemed sorted by name. Whey I follow the link "see all lists" the order is correct, though. Annoying.
They should be sorted by name regardless of the filter. I don't know what the cause of the issue might be.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 3/3/20 1:26 PM, David Krantz wrote:
Hi,
I tried psycopg2-binary and a few other rehashes of that. Now I got it working by downloading the source for psycopg2 manually and building it myself, then I installed it into my venv like any other self-built package. https://pypi.org/project/psycopg2/#files
I'm glad you got it working. I think installing psycopg2-binary with pip should have worked, so I suspect there was some residue of psycopg2 somewhere in the search path ahead of the psycopg2-binary that was the issue.
I run mod_wsgi in Apache so I thought it should be enough to restart Apache.
Yes, that should be sufficient.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
David Krantz -
Mark Sapiro