User settings triggers error
If one of my users (including admin) goes to their user drop-down in the upper-right of Postorius and clicks Mailman Settings, it triggers an error:
Here’s the error log:
Internal Server Error: /mailman3/postorius/accounts/subscriptions/
KeyError at /postorius/accounts/subscriptions/ 'file'
Request Method: GET Request URL: https://mail.pdc-racing.net/mailman3/postorius/accounts/subscriptions/ <https://mail.pdc-racing.net/mailman3/postorius/accounts/subscriptions/> Django Version: 2.2.12 Python Executable: /usr/bin/uwsgi-core Python Version: 3.8.2 Python Path: ['.', '', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Tue, 16 Jun 2020 10:31:56 -0700 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view 21. return view_func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/postorius/views/user.py" in user_subscriptions 307. memberships = [m for m in mm_user.subscriptions]
File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/user.py" in subscriptions 62. for address in self.addresses:
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/base.py" in __iter__ 221. for entry in self.rest_data:
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/base.py" in rest_data 197. response, content = self._connection.call(self._url)
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None)
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request)
File "/usr/lib/python3/dist-packages/postorius/middleware.py" in __call__ 39. return self.get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 36. response = response_for_exception(request, exc)
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in response_for_exception 91. log_response(
File "/usr/lib/python3/dist-packages/django/utils/log.py" in log_response 222. getattr(logger, level)(
File "/usr/lib/python3.8/logging/__init__.py" in error 1463. self._log(ERROR, msg, args, **kwargs)
File "/usr/lib/python3.8/logging/__init__.py" in _log 1577. self.handle(record)
File "/usr/lib/python3.8/logging/__init__.py" in handle 1587. self.callHandlers(record)
File "/usr/lib/python3.8/logging/__init__.py" in callHandlers 1649. hdlr.handle(record)
File "/usr/lib/python3.8/logging/__init__.py" in handle 950. self.emit(record)
File "/usr/lib/python3/dist-packages/django/utils/log.py" in emit 119. reporter = ExceptionReporter(request, is_email=True, *exc_info)
File "/usr/lib/python3/dist-packages/django/views/debug.py" in __init__ 254. self.template_info = getattr(self.exc_value, 'template_debug', None)
File "/usr/lib/python3.8/tempfile.py" in __getattr__ 607. file = self.__dict__['file']
Exception Type: KeyError at /postorius/accounts/subscriptions/ Exception Value: 'file' Request information: USER: admin
GET: No GET data
POST: No POST data
FILES: No FILES data
Anyone got a solution to this?
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/16/20 10:34 AM, Mark Dadgar wrote:
If one of my users (including admin) goes to their user drop-down in the upper-right of Postorius and clicks Mailman Settings, it triggers an error:
Here’s the error log:
Internal Server Error: /mailman3/postorius/accounts/subscriptions/
KeyError at /postorius/accounts/subscriptions/ 'file'
The KeyError is bogus. It is an error that occurs in Django's logging when trying to log the real error.
Request Method: GET Request URL: https://mail.pdc-racing.net/mailman3/postorius/accounts/subscriptions/ ... Traceback:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 34. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 115. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 113. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view 21. return view_func(request, *args, **kwargs)
File "/usr/lib/python3/dist-packages/postorius/views/user.py" in user_subscriptions 307. memberships = [m for m in mm_user.subscriptions]
Here we're trying to grt all the user's subscriptions.
Se we call mailmanclient for that.
File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/user.py" in subscriptions 62. for address in self.addresses:
client is getting all the user's addresses
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/base.py" in __iter__ 221. for entry in self.rest_data:
It got the addresses and is now iterating over them.
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/base.py" in rest_data 197. response, content = self._connection.call(self._url)
Which calls the REST API
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None)
And gets a 500 from core.
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
Which we aren't interested in.
The next step is to look in core's mailman.log to see what the rest call was and the exception and hopefully traceback the caused the 500 response.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 16, 2020, at 11:47 AM, Mark Sapiro <mark@msapiro.net> wrote:
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None)
And gets a 500 from core.
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
Which we aren't interested in.
The next step is to look in core's mailman.log to see what the rest call was and the exception and hopefully traceback the caused the 500 response.
Here’s what’s logged during that error:
Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/mark@pdc-racing.net HTTP/1.1" 200 432 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 131, in find attribute = getattr(resource, name, MISSING) File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/usermanager.py", line 94, in get_user_by_id if users.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT "user".password AS user_password, "user".id AS user_id, "user".display_name AS user_display_name, "user"._user_id AS user__user_id, "user"._created_on AS user__created_on, "user".is_server_owner AS user_is_server_owner, "user"._preferred_address_id AS user__preferred_address_id, "user".preferences_id AS user_preferences_id FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/05dcbeb4b78445eb96f86f9e4b7ab6cb/addresses HTTP/1.1" 500 59 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 131, in find attribute = getattr(resource, name, MISSING) File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/usermanager.py", line 94, in get_user_by_id if users.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT "user".password AS user_password, "user".id AS user_id, "user".display_name AS user_display_name, "user"._user_id AS user__user_id, "user"._created_on AS user__created_on, "user".is_server_owner AS user_is_server_owner, "user"._preferred_address_id AS user__preferred_address_id, "user".preferences_id AS user_preferences_id FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/05dcbeb4b78445eb96f86f9e4b7ab6cb/addresses HTTP/1.1" 500 59 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 131, in find attribute = getattr(resource, name, MISSING) File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/usermanager.py", line 94, in get_user_by_id if users.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT "user".password AS user_password, "user".id AS user_id, "user".display_name AS user_display_name, "user"._user_id AS user__user_id, "user"._created_on AS user__created_on, "user".is_server_owner AS user_is_server_owner, "user"._preferred_address_id AS user__preferred_address_id, "user".preferences_id AS user_preferences_id FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/05dcbeb4b78445eb96f86f9e4b7ab6cb/addresses HTTP/1.1" 500 59
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/16/20 12:18 PM, Mark Dadgar wrote:
On Jun 16, 2020, at 11:47 AM, Mark Sapiro <mark@msapiro.net> wrote:
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None)
And gets a 500 from core.
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
Which we aren't interested in.
The next step is to look in core's mailman.log to see what the rest call was and the exception and hopefully traceback the caused the 500 response.
Here’s what’s logged during that error:
Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/mark@pdc-racing.net HTTP/1.1" 200 432 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 131, in find attribute = getattr(resource, name, MISSING) File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/usermanager.py", line 94, in get_user_by_id if users.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT "user".password AS user_password, "user".id AS user_id, "user".display_name AS user_display_name, "user"._user_id AS user__user_id, "user"._created_on AS user__created_on, "user".is_server_owner AS user_is_server_owner, "user"._preferred_address_id AS user__preferred_address_id, "user".preferences_id AS user_preferences_id FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/05dcbeb4b78445eb96f86f9e4b7ab6cb/addresses HTTP/1.1" 500 59 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
This looks like another manifestation of the issue reported at https://gitlab.com/mailman/django-mailman3/-/issues/35 and https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
If you are running django-mailman3 1.3.3 and haven't upgraded to django-mailman3 1.3.4, see the above list post for a workaround.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 16, 2020, at 12:47 PM, Mark Sapiro <mark@msapiro.net> wrote:
This looks like another manifestation of the issue reported at https://gitlab.com/mailman/django-mailman3/-/issues/35 and https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
If you are running django-mailman3 1.3.3 and haven't upgraded to django-mailman3 1.3.4, see the above list post for a workaround.
I am running whatever the current ubuntu packages are (for the moment - my Postgres migrations was in preparation to move to something more current).
I will try the cache clear and see what happens.
Thanks, Mark.
- Mark
mark@pdc-racing.net | 408-348-2878
On Jun 16, 2020, at 12:56 PM, Mark Dadgar <mark@pdc-racing.net> wrote:
This looks like another manifestation of the issue reported at https://gitlab.com/mailman/django-mailman3/-/issues/35 and https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
If you are running django-mailman3 1.3.3 and haven't upgraded to django-mailman3 1.3.4, see the above list post for a workaround.
I am running whatever the current ubuntu packages are (for the moment - my Postgres migrations was in preparation to move to something more current).
I will try the cache clear and see what happens.
Clearing the cache had no effect.
I am running a down-rev version of django-mailman:
mark@mail:/usr/share/mailman3-web$ pip3 list | grep mailman
django-mailman3 1.3.2
mailman 3.2.2
mailman-hyperkitty 1.1.0
mailmanclient 3.2.2
I will likely punt on this problem until I migrate to current mailman3 code.
- Mark
mark@pdc-racing.net | 408-348-2878
On Tue, Jun 16, 2020, at 12:47 PM, Mark Sapiro wrote:
On 6/16/20 12:18 PM, Mark Dadgar wrote:
On Jun 16, 2020, at 11:47 AM, Mark Sapiro <mark@msapiro.net> wrote:
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None)
And gets a 500 from core.
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
Which we aren't interested in.
The next step is to look in core's mailman.log to see what the rest call was and the exception and hopefully traceback the caused the 500 response.
Here’s what’s logged during that error:
Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/mark@pdc-racing.net HTTP/1.1" 200 432 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 131, in find attribute = getattr(resource, name, MISSING) File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/usermanager.py", line 94, in get_user_by_id if users.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT "user".password AS user_password, "user".id AS user_id, "user".display_name AS user_display_name, "user"._user_id AS user__user_id, "user"._created_on AS user__created_on, "user".is_server_owner AS user_is_server_owner, "user"._preferred_address_id AS user__preferred_address_id, "user".preferences_id AS user_preferences_id FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/05dcbeb4b78445eb96f86f9e4b7ab6cb/addresses HTTP/1.1" 500 59 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
This looks like another manifestation of the issue reported at https://gitlab.com/mailman/django-mailman3/-/issues/35 and https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
If you are running django-mailman3 1.3.3 and haven't upgraded to django-mailman3 1.3.4, see the above list post for a workaround.
This is a different issue actually, seems like a bug in Core.
Interesting part of the above traceback is:
File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id)
and
FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}]
Which makes me think that we need to call str() on the UUID before passing it to sqlalchemy.
I wonder why it didn't show up already on our Mailman 3 instances, which also runs postgres. Perhaps, a different version of sqlalchemy triggers this. We'd need to look more into the specific version of sqlalchemy that causes this.
Mark Dadgar, Can you please let us know the version of sqlalchemy installed with your Mailman?
-- thanks, Abhilash Raj (maxking)
On Jun 16, 2020, at 4:53 PM, Abhilash Raj <maxking@asynchronous.in> wrote:
I wonder why it didn't show up already on our Mailman 3 instances, which also runs postgres. Perhaps, a different version of sqlalchemy triggers this. We'd need to look more into the specific version of sqlalchemy that causes this.
So one thing I realized: this is the server where I migrated the mm3 databases from sqlite to postgres via pgloader and had the truncated index name warnings that I posted about a while back.
Don’t know if that’s a factor but I thought I’d mention it.
Mark Dadgar, Can you please let us know the version of sqlalchemy installed with your Mailman?
python3-sqlalchemy/focal,now 1.3.12+ds1-1ubuntu2
- Mark
mark@pdc-racing.net | 408-348-2878
On Jun 16, 2020, at 6:52 PM, Mark Dadgar <mark@pdc-racing.net> wrote:
I wonder why it didn't show up already on our Mailman 3 instances, which also runs postgres. Perhaps, a different version of sqlalchemy triggers this. We'd need to look more into the specific version of sqlalchemy that causes this.
So one thing I realized: this is the server where I migrated the mm3 databases from sqlite to postgres via pgloader and had the truncated index name warnings that I posted about a while back.
Don’t know if that’s a factor but I thought I’d mention it.
Mark Dadgar, Can you please let us know the version of sqlalchemy installed with your Mailman?
python3-sqlalchemy/focal,now 1.3.12+ds1-1ubuntu2
There is definitely something sketchy going on here. As an admin, I am not getting accurate subscription status about people on a list. For example, I search for a list member, find their listing, check the box and click Unsubscribe Selected, and get an error because it turns out that they had already unsubscribed themselves previously. But Postorius thought they were still subscribed.
mailman.log errors:
-- Jun 18 14:18:47 2020 (1302) 127.0.0.1 - - "GET /3.1/lists/trackjunkies.pdc-racing.net HTTP/1.1" 200 422 Jun 18 14:18:47 2020 (1302) 127.0.0.1 - - "GET /3.1/lists/trackjunkies.pdc-racing.net/roster/owner HTTP/1.1" 200 625 Jun 18 14:18:47 2020 (1302) 127.0.0.1 - - "GET /3.1/lists/trackjunkies.pdc-racing.net/roster/moderator HTTP/1.1" 200 90 Jun 18 14:18:48 2020 (1302) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE member._member_id = '88b9844d-0791-44eb-8cf4-615a03b5c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 165, in find result = attribute( File "/usr/lib/python3/dist-packages/mailman/rest/lists.py", line 216, in member return AMember(member.member_id) File "/usr/lib/python3/dist-packages/mailman/rest/members.py", line 114, in __init__ else service.get_member(member_id)) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/subscriptions.py", line 71, in get_member if members.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE member._member_id = '88b9844d-0791-44eb-8cf4-615a03b5c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT member.id AS member_id, member._member_id AS member__member_id, member.role AS member_role, member.list_id AS member_list_id, member.moderation_action AS member_moderation_action, member.address_id AS member_address_id, member.preferences_id AS member_preferences_id, member.user_id AS member_user_id FROM member WHERE member._member_id = %(member_id_1)s) AS anon_1] [parameters: {'member_id_1': UUID('88b9844d-0791-44eb-8cf4-615a03b5cc53')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 18 14:18:48 2020 (1302) 127.0.0.1 - - "DELETE /3.1/lists/trackjunkies.pdc-racing.net/member/XXXXXXXX@gmail.com HTTP/1.1" 500 59 —
mailman-web.log errors:
-- ERROR 2020-06-18 21:21:06,399 1268 django.request Internal Server Error: /mailman3/postorius/lists/trackjunkies.pdc-racing.net/members/member/ Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py", line 321, in unsubscribe self._connection.call(path, method='DELETE') File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 500: b'A server error occurred. Please contact the administrator.'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/contrib/auth/mixins.py", line 52, in dispatch return super().dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/contrib/auth/mixins.py", line 109, in dispatch return super().dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/generic.py", line 65, in dispatch return super(MailingListView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 97, in dispatch return handler(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 169, in post return self._member_post(request, role) File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 120, in _member_post self.mailing_list.unsubscribe(member) File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py", line 324, in unsubscribe raise ValueError('%s is not a member address of %s' % ValueError: XXXXXXX@gmail.com is not a member address of trackjunkies@pdc-racing.net ERROR 2020-06-18 21:21:06,399 1268 django.request Internal Server Error: /mailman3/postorius/lists/trackjunkies.pdc-racing.net/members/member/ Traceback (most recent call last): File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py", line 321, in unsubscribe self._connection.call(path, method='DELETE') File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py", line 99, in call raise HTTPError(url, response.status, content, response, None) urllib.error.HTTPError: HTTP Error 500: b'A server error occurred. Please contact the administrator.'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 71, in view return self.dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/contrib/auth/mixins.py", line 52, in dispatch return super().dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/contrib/auth/mixins.py", line 109, in dispatch return super().dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/generic.py", line 65, in dispatch return super(MailingListView, self).dispatch(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/django/views/generic/base.py", line 97, in dispatch return handler(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 169, in post return self._member_post(request, role) File "/usr/lib/python3/dist-packages/postorius/views/list.py", line 120, in _member_post self.mailing_list.unsubscribe(member) File "/usr/lib/python3/dist-packages/mailmanclient/restobjects/mailinglist.py", line 324, in unsubscribe raise ValueError('%s is not a member address of %s' % ValueError: XXXXXXXXX@gmail.com is not a member address of trackjunkies@pdc-racing.net [pid: 1268|app: 0|req: 18686/18686] 76.102.110.193 () {68 vars in 1625 bytes} [Thu Jun 18 21:21:06 2020] POST /mailman3/postorius/lists/trackjunkies.pdc-racing.net/members/member/ => generated 1156 bytes in 162 msecs (HTTP/1.1 500) 5 headers in 182 bytes (1 switches on core 1) ——
Could this somehow be related to my recent migration from sqlite3 to postgres?
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/18/20 2:23 PM, Mark Dadgar wrote:
There is definitely something sketchy going on here. As an admin, I am not getting accurate subscription status about people on a list. For example, I search for a list member, find their listing, check the box and click Unsubscribe Selected, and get an error because it turns out that they had already unsubscribed themselves previously. But Postorius thought they were still subscribed.
...
Could this somehow be related to my recent migration from sqlite3 to postgres?
I think it's quite likely that that migration is involved and your database is now corrupted in some way. Exactly how or in what way, I don't know, but it seems likely to me that that's the issue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 18, 2020, at 3:27 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 6/18/20 2:23 PM, Mark Dadgar wrote:
There is definitely something sketchy going on here. As an admin, I am not getting accurate subscription status about people on a list. For example, I search for a list member, find their listing, check the box and click Unsubscribe Selected, and get an error because it turns out that they had already unsubscribed themselves previously. But Postorius thought they were still subscribed.
...
Could this somehow be related to my recent migration from sqlite3 to postgres?
I think it's quite likely that that migration is involved and your database is now corrupted in some way. Exactly how or in what way, I don't know, but it seems likely to me that that's the issue.
Oh good.
Any thoughts on how to salvage ~20 years of list archives?
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/18/20 3:33 PM, Mark Dadgar wrote:
Any thoughts on how to salvage ~20 years of list archives?
The archives shouldn't be an issue. I don't think they are involved in the corruption, but anyway, hopefully you still have the archive mbox you imported. Then for the messages since then, you should be able to get a mbox from hyperkitty with something like
with the obvious substitutions and appropriate start and end dates.
The list is trickier. If you are going to start over with a clean install, and if you have a config.pck that you imported from MM 2.1, I suggest the following:
Use the mailman members command to get a list of all members. Then
depending on how much work you want to do, also get separate lists of
plaintext and mime digest members and members with delivery disabled.
Then on the new install, import the config.pck with mailman import21
and import the archive mboxes with hyperkitty_import.
Run mailman members --sync with the full current membership list, and
record the [ADD]s. Then you can match the [ADD]s against the plain and
mime digest members and those with delivery disabled and set their options.
That's a lot to do, but it should get you close.
On the other hand, if you can wait, we might be able to figure out how to fix this, and it's even possible the underlying issue isn't database corruption.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 18, 2020, at 4:08 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 6/18/20 3:33 PM, Mark Dadgar wrote:
Any thoughts on how to salvage ~20 years of list archives?
The archives shouldn't be an issue. I don't think they are involved in the corruption, but anyway, hopefully you still have the archive mbox you imported. Then for the messages since then, you should be able to get a mbox from hyperkitty with something like
with the obvious substitutions and appropriate start and end dates.
The list is trickier. If you are going to start over with a clean install, and if you have a config.pck that you imported from MM 2.1, I suggest the following:
Use the
mailman memberscommand to get a list of all members. Then depending on how much work you want to do, also get separate lists of plaintext and mime digest members and members with delivery disabled.Then on the new install, import the config.pck with
mailman import21and import the archive mboxes with hyperkitty_import.Run
mailman members --syncwith the full current membership list, and record the [ADD]s. Then you can match the [ADD]s against the plain and mime digest members and those with delivery disabled and set their options.That's a lot to do, but it should get you close.
I would probably be satisfied with just exporting the current list of member email addresses + names and letting people go back and change their list settings to switch to digest, etc. if necessary.
On the other hand, if you can wait, we might be able to figure out how to fix this, and it's even possible the underlying issue isn't database corruption.
Sure - I can wait. Thank you.
- Mark
mark@pdc-racing.net | 408-348-2878
On Jun 18, 2020, at 4:25 PM, Mark Dadgar <mark@pdc-racing.net> wrote:
Any thoughts on how to salvage ~20 years of list archives?
The archives shouldn't be an issue. I don't think they are involved in the corruption, but anyway, hopefully you still have the archive mbox you imported. Then for the messages since then, you should be able to get a mbox from hyperkitty with something like
with the obvious substitutions and appropriate start and end dates.
The list is trickier. If you are going to start over with a clean install, and if you have a config.pck that you imported from MM 2.1, I suggest the following:
Use the
mailman memberscommand to get a list of all members. Then depending on how much work you want to do, also get separate lists of plaintext and mime digest members and members with delivery disabled.Then on the new install, import the config.pck with
mailman import21and import the archive mboxes with hyperkitty_import.Run
mailman members --syncwith the full current membership list, and record the [ADD]s. Then you can match the [ADD]s against the plain and mime digest members and those with delivery disabled and set their options.That's a lot to do, but it should get you close.
I would probably be satisfied with just exporting the current list of member email addresses + names and letting people go back and change their list settings to switch to digest, etc. if necessary.
On the other hand, if you can wait, we might be able to figure out how to fix this, and it's even possible the underlying issue isn't database corruption.
Sure - I can wait. Thank you.
I am probably going to go ahead with the list wipe and recreate in the next couple of days. Lack of access to the administration stuff is starting to become a problem.
Sigh.
So far, migrating from sqlite3 to postgres is looking like a bit of a bust.
- Mark
mark@pdc-racing.net | 408-348-2878
On Jun 18, 2020, at 4:08 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 6/18/20 3:33 PM, Mark Dadgar wrote:
Any thoughts on how to salvage ~20 years of list archives?
The archives shouldn't be an issue. I don't think they are involved in the corruption, but anyway, hopefully you still have the archive mbox you imported. Then for the messages since then, you should be able to get a mbox from hyperkitty with something like
with the obvious substitutions and appropriate start and end dates.
So in trying to download the recent archives ("export/trackjunkies-new.mbox.gz?start=2019-10-10&end=2020-06-28”), I end up with an 8.5MB file with one day’s worth of posts in it (2019-10-10).
I see these errors in the mailman-web.log:
Traceback (most recent call last): File "/usr/lib/python3.8/email/_header_value_parser.py", line 2060, in get_msg_id token, value = get_dot_atom_text(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1325, in get_dot_atom_text raise errors.HeaderParseError("expected atom at a start of " email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found ' <BYAPR02MB5976132151FF19510269C44CB3940@BYAPR02MB5976.namprd02.prod.outlook.com>>'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/hyperkitty/views/mlist.py", line 332, in stream_mbox msg = email.as_message() File "/usr/lib/python3/dist-packages/hyperkitty/models/email.py", line 175, in as_message msg["Message-ID"] = "<%s>" % self.message_id File "/usr/lib/python3.8/email/message.py", line 409, in __setitem__ self._headers.append(self.policy.header_store_parse(name, val)) File "/usr/lib/python3.8/email/policy.py", line 148, in header_store_parse return (name, self.header_factory(name, value)) File "/usr/lib/python3.8/email/headerregistry.py", line 602, in __call__ return self[name](name, value) File "/usr/lib/python3.8/email/headerregistry.py", line 197, in __new__ cls.parse(value, kwds) File "/usr/lib/python3.8/email/headerregistry.py", line 530, in parse kwds['parse_tree'] = parse_tree = cls.value_parser(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2117, in parse_message_id token, value = get_msg_id(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 2064, in get_msg_id token, value = get_obs_local_part(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1509, in get_obs_local_part obs_local_part[1].token_type=='dot'): IndexError: list index out of range [pid: 1359|app: 0|req: 12933/12933] 76.102.110.193 () {60 vars in 1504 bytes} [Sun Jun 28 19:52:57 2020] GET /mailman3/hyperkitty/list/trackjunkies@pdc-racing.net/export/trackjunkies-new.mbox.gz?start=2019-10-10&end=2020-06-28 => generated 5572382 bytes in 4271 msecs (HTTP/1.1 200) 5 headers in 203 bytes (377 switches on core 1)
Thoughts? It would be nice to salvage the last 8 months of archives.
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/28/20 12:55 PM, Mark Dadgar wrote:
So in trying to download the recent archives ("export/trackjunkies-new.mbox.gz?start=2019-10-10&end=2020-06-28”), I end up with an 8.5MB file with one day’s worth of posts in it (2019-10-10).
I see these errors in the mailman-web.log:
Traceback (most recent call last): File "/usr/lib/python3.8/email/_header_value_parser.py", line 2060, in get_msg_id token, value = get_dot_atom_text(value) File "/usr/lib/python3.8/email/_header_value_parser.py", line 1325, in get_dot_atom_text raise errors.HeaderParseError("expected atom at a start of " email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found ' <BYAPR02MB5976132151FF19510269C44CB3940@BYAPR02MB5976.namprd02.prod.outlook.com>>'
If I'm reading the above correctly, it appears that there is a message in the archive database whose message_id value is 'BYAPR02MB5976132151FF19510269C44CB3940@BYAPR02MB5976.namprd02.prod.outlook.com>'. The trailing '>' is causing HyperKitty's Email.as_message() method to try to set a value of '<BYAPR02MB5976132151FF19510269C44CB3940@BYAPR02MB5976.namprd02.prod.outlook.com>>'. Or maybe there's leading white space in the value, or both. Try the following patch. --- a/hyperkitty/models/email.py +++ b/hyperkitty/models/email.py @@ -24,7 +24,7 @@ import logging import os import re from email.message import EmailMessage -from email.utils import formataddr +from email.utils import formataddr, make_msgid from django.conf import settings from django.db import IntegrityError, models @@ -175,7 +175,12 @@ class Email(models.Model): header_date = self.date.astimezone(tz).replace(microsecond=0) # Date format: http://tools.ietf.org/html/rfc5322#section-3.3 msg["Date"] = header_date.strftime("%a, %d %b %Y %H:%M:%S %z") - msg["Message-ID"] = "<%s>" % self.message_id + try: + msg["Message-ID"] = "<%s>" % re.sub('[<>\s]', '', + self.message_id) + except: + msg["Message-ID"] = make_msgid() if self.in_reply_to: msg["In-Reply-To"] = unfold(self.in_reply_to) Also note that if you got 8.5.MB for one day's messages, probably the entire range will time out and you'll have to get it in pieces. -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 28, 2020, at 4:22 PM, Mark Sapiro <mark@msapiro.net> wrote:
If I'm reading the above correctly, it appears that there is a message in the archive database whose message_id value is 'BYAPR02MB5976132151FF19510269C44CB3940@BYAPR02MB5976.namprd02.prod.outlook.com>'. The trailing '>' is causing HyperKitty's Email.as_message() method to try to set a value of '<BYAPR02MB5976132151FF19510269C44CB3940@BYAPR02MB5976.namprd02.prod.outlook.com>>'.
Or maybe there's leading white space in the value, or both.
Try the following patch.
--- a/hyperkitty/models/email.py +++ b/hyperkitty/models/email.py @@ -24,7 +24,7 @@ import logging import os import re from email.message import EmailMessage -from email.utils import formataddr +from email.utils import formataddr, make_msgid
from django.conf import settings from django.db import IntegrityError, models @@ -175,7 +175,12 @@ class Email(models.Model): header_date = self.date.astimezone(tz).replace(microsecond=0) # Date format: http://tools.ietf.org/html/rfc5322#section-3.3 msg["Date"] = header_date.strftime("%a, %d %b %Y %H:%M:%S %z") - msg["Message-ID"] = "<%s>" % self.message_id + try: + msg["Message-ID"] = "<%s>" % re.sub('[<>\s]', '', + self.message_id) + except: + msg["Message-ID"] = make_msgid() if self.in_reply_to: msg["In-Reply-To"] = unfold(self.in_reply_to)
The patch does not apply cleanly against /usr/lib/python3/dist-packages/hyperkitty/models/email.py patching file email.py patch: **** malformed patch at line 4: import os I am assuming that this is because I am running the Ubuntu packages, which are a year down-rev: Mailman Core Version GNU Mailman 3.2.2 (La Villa Strangiato) Mailman Core API Version 3.0 Mailman Core Python Version 3.8.2 (default, Apr 27 2020, 15:53:34) [GCC 9.3.0] - Mark ----- mark@pdc-racing.net | 408-348-2878
On 6/28/20 4:39 PM, Mark Dadgar wrote:
The patch does not apply cleanly against /usr/lib/python3/dist-packages/hyperkitty/models/email.py
patching file email.py patch: **** malformed patch at line 4: import os
That probably isn't because of version mismatch. That would give applied at offset or with fuzz or a reject. malformed patch means the patch file is not formatted correctly. Here it is again as an attachment. That may help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 28, 2020, at 4:56 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 6/28/20 4:39 PM, Mark Dadgar wrote:
The patch does not apply cleanly against /usr/lib/python3/dist-packages/hyperkitty/models/email.py
patching file email.py patch: **** malformed patch at line 4: import os
That probably isn't because of version mismatch. That would give applied at offset or with fuzz or a reject. malformed patch means the patch file is not formatted correctly. Here it is again as an attachment. That may help.
That gives a slightly different error:
mark@mail:/usr/lib/python3/dist-packages/hyperkitty/models$ sudo patch < /tmp/patch.txt patching file email.py Hunk #1 FAILED at 24. patch: **** malformed patch at line 23:
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/28/20 5:20 PM, Mark Dadgar wrote:
That gives a slightly different error:
mark@mail:/usr/lib/python3/dist-packages/hyperkitty/models$ sudo patch < /tmp/patch.txt patching file email.py Hunk #1 FAILED at 24. patch: **** malformed patch at line 23:
I don't know about the failure. That probably is due to the version mismatch. The malformation is my fault. There should be an empty line following the last line.
You didn't say what your HyperKitty version is, but here's a patch against 1.3.0. If this doesn't work, you could just try patching it by hand.
I'm going fishing tomorrow so I won't be here much longer.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Jun 28, 2020, at 6:13 PM, Mark Sapiro <mark@msapiro.net> wrote:
I don't know about the failure. That probably is due to the version mismatch. The malformation is my fault. There should be an empty line following the last line.
You didn't say what your HyperKitty version is, but here's a patch against 1.3.0. If this doesn't work, you could just try patching it by hand.
OK, the patch against 1.3.0 appears to be working. Thank you.
I'm going fishing tomorrow so I won't be here much longer.
Enjoy the fishing!
- Mark
mark@pdc-racing.net | 408-348-2878
On 6/28/20 6:37 PM, Mark Dadgar wrote:
OK, the patch against 1.3.0 appears to be working. Thank you.
Good.
I'm going fishing tomorrow so I won't be here much longer.
Enjoy the fishing!
Thanks, I will.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Abhilash Raj -
Mark Dadgar -
Mark Sapiro