On Tue, Jun 16, 2020, at 12:47 PM, Mark Sapiro wrote:
On 6/16/20 12:18 PM, Mark Dadgar wrote:
On Jun 16, 2020, at 11:47 AM, Mark Sapiro <mark@msapiro.net> wrote:
File "/usr/lib/python3/dist-packages/mailmanclient/restbase/connection.py" in call 99. raise HTTPError(url, response.status, content, response, None)
And gets a 500 from core.
During handling of the above exception (HTTP Error 500: b'A server error occurred. Please contact the administrator.'), another exception occurred:
Which we aren't interested in.
The next step is to look in core's mailman.log to see what the rest call was and the exception and hopefully traceback the caused the 500 response.
Here’s what’s logged during that error:
Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/mark@pdc-racing.net HTTP/1.1" 200 432 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/lib/python3.8/wsgiref/handlers.py", line 137, in run self.result = application(self.environ, self.start_response) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 218, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 232, in falcon.api.API.__call__ File "falcon/api.py", line 229, in falcon.api.API.__call__ File "falcon/api.py", line 582, in falcon.api.API._get_responder File "/usr/lib/python3/dist-packages/mailman/rest/wsgiapp.py", line 131, in find attribute = getattr(resource, name, MISSING) File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/usermanager.py", line 94, in get_user_by_id if users.count() == 0: File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3630, in count return self.from_self(col).scalar() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3355, in scalar ret = self.one() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3325, in one ret = self.one_or_none() File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3294, in one_or_none ret = list(self) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3367, in __iter__ return self._execute_and_instances(context) File "/usr/lib/python3/dist-packages/sqlalchemy/orm/query.py", line 3392, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 982, in execute return meth(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement ret = self._execute_context( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context self._handle_dbapi_exception( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception util.raise_from_cause(sqlalchemy_exception, exc_info) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
[SQL: SELECT count(*) AS count_1 FROM (SELECT "user".password AS user_password, "user".id AS user_id, "user".display_name AS user_display_name, "user"._user_id AS user__user_id, "user"._created_on AS user__created_on, "user".is_server_owner AS user_is_server_owner, "user"._preferred_address_id AS user__preferred_address_id, "user".preferences_id AS user_preferences_id FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}] (Background on this error at: http://sqlalche.me/e/f405) Jun 16 12:16:54 2020 (1326) 127.0.0.1 - - "GET /3.1/users/05dcbeb4b78445eb96f86f9e4b7ab6cb/addresses HTTP/1.1" 500 59 Jun 16 12:16:54 2020 (1326) REST request handler error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1245, in _execute_context self.dialect.do_execute( File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 581, in do_execute cursor.execute(statement, parameters) psycopg2.errors.UndefinedFunction: operator does not exist: text = uuid LINE 4: WHERE "user"._user_id = '05dcbeb4-b784-45eb-96f8-6f9e4b7ab6c... ^ HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
This looks like another manifestation of the issue reported at https://gitlab.com/mailman/django-mailman3/-/issues/35 and https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
If you are running django-mailman3 1.3.3 and haven't upgraded to django-mailman3 1.3.4, see the above list post for a workaround.
This is a different issue actually, seems like a bug in Core.
Interesting part of the above traceback is:
File "/usr/lib/python3/dist-packages/mailman/rest/users.py", line 203, in _user return user_manager.get_user_by_id(user_id)
and
FROM "user" WHERE "user"._user_id = %(user_id_1)s) AS anon_1] [parameters: {'user_id_1': UUID('05dcbeb4-b784-45eb-96f8-6f9e4b7ab6cb')}]
Which makes me think that we need to call str() on the UUID before passing it to sqlalchemy.
I wonder why it didn't show up already on our Mailman 3 instances, which also runs postgres. Perhaps, a different version of sqlalchemy triggers this. We'd need to look more into the specific version of sqlalchemy that causes this.
Mark Dadgar, Can you please let us know the version of sqlalchemy installed with your Mailman?
-- thanks, Abhilash Raj (maxking)