Internal Server Error: Reattach this thread
Hi,
In the archive, clicking "Reconnect this thread" causes an internal server error. I received the following email, but I don't know how to solve it, please help me.
Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest
Internal Server Error: /archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest
AttributeError at /archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest 'function' object has no attribute 'lower'
Request Method: GET Request URL: http://mailman.example.com/archives/list/testml@list1.example.com/thread/LQ6... Django Version: 3.0.14 Python Executable: /opt/mailman/venv/bin/uwsgi Python Version: 3.10.6 Python Path: ['.', '', '/etc/mailman3', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/opt/mailman/venv/lib/python3.10/site-packages'] Server time: 水, 19 10月 2022 01:07:28 +0000 Installed Applications: ['hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount'] Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')
Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/mailman/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/lib/view_helpers.py", line 134, in inner return func(request, *args, **kwargs) File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/views/thread.py", line 413, in reattach_suggest default_search_query = stripped_subject( File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/lib/utils.py", line 155, in stripped_subject if subject.lower().startswith(mlist.subject_prefix.lower()):
Exception Type: AttributeError at /archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest Exception Value: 'function' object has no attribute 'lower' Request information: USER: admin
GET: q = ''
POST: No POST data
FILES: No FILES data
COOKIES: csrftoken = 'cBq9h8x7bG6BMboc89DB8ucQiToTRTpwijt0uJbVXq47ucV2cAGYh05TsaVARobK' sessionid = 'upuj3rv0wkxafklk633lx2jppwgrco1t'
META:
CSRF_COOKIE = 'cBq9h8x7bG6BMboc89DB8ucQiToTRTpwijt0uJbVXq47ucV2cAGYh05TsaVARobK'
HTTP_ACCEPT = '*/*'
HTTP_ACCEPT_ENCODING = 'gzip, deflate'
HTTP_ACCEPT_LANGUAGE = 'ja,en-US;q=0.7,en;q=0.3'
HTTP_CONNECTION = 'close'
HTTP_COOKIE = 'csrftoken=cBq9h8x7bG6BMboc89DB8ucQiToTRTpwijt0uJbVXq47ucV2cAGYh05TsaVARobK; sessionid=upuj3rv0wkxafklk633lx2jppwgrco1t'
HTTP_HOST = 'mailman.example.com'
HTTP_REFERER = 'http://mailman.example.com/archives/list/testml@list1.example.com/thread/LQ6...'
HTTP_USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0'
HTTP_X_FORWARDED_FOR = '192.168.30.94'
HTTP_X_REQUESTED_WITH = 'XMLHttpRequest'
PATH_INFO = '/archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest'
QUERY_STRING = 'q='
REMOTE_ADDR = '127.0.0.1'
REQUEST_METHOD = 'GET'
REQUEST_URI = '/archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest?q='
SCRIPT_NAME = ''
SERVER_NAME = 'ubuntu22'
SERVER_PORT = '8000'
SERVER_PROTOCOL = 'HTTP/1.0'
uwsgi.core = 0
uwsgi.node = b'ubuntu22'
uwsgi.version = b'2.0.20'
wsgi.errors = <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.file_wrapper = ''
wsgi.input =
Settings:
Using settings module settings
ABSOLUTE_URL_OVERRIDES = {}
ACCOUNT_AUTHENTICATION_METHOD = 'username_email'
ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'http'
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_EMAIL_VERIFICATION = 'mandatory'
ACCOUNT_UNIQUE_EMAIL = True
ADMINS = "(('Mailman Suite Admin', 'admin1@example.org'),)"
ALLOWED_HOSTS = ['127.0.0.1', 'mailman.example.com']
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = "('django.contrib.auth.backends.ModelBackend', 'allauth.account.auth_backends.AuthenticationBackend')"
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'auth.User'
BASE_DIR = PosixPath('/opt/mailman/web')
CACHES = {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'}
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_FILTERS = {'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'], 'js': ['compressor.filters.jsmin.rJSMinFilter']}
COMPRESS_JINJA2_GET_ENVIRONMENT =
Thank you.
hasimoto@designet.co.jp writes:
File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/lib/utils.py", line 155, in stripped_subject if subject.lower().startswith(mlist.subject_prefix.lower()):
Exception Type: AttributeError at /archives/list/testml@list1.example.com/thread/LQ6F7D7EVVUSIJJ7B5S675ZVSU5ZZCBU/reattach-suggest Exception Value: 'function' object has no attribute 'lower'
This indicates that either subject (from the message) or mlist.subject_prefix (from the mailing list's configuration) is a function, while both should be strings. This shouldn't happen (subject gets its value by extracting text from the message's header, and subject_prefix is actually a proxy object that acts like a string but "under the hood" fetches that string from the PostgreSQL database row for that mailing list), and I don't see offhand how such a bug would occur.
To help us diagnose the bug, please tell us (1) how you installed Mailman 3, as distribution packages, using pip, or from source (2) if this list was created in Mailman 3 or migrated from an existing Mailman 2.1 installation (3) what the versions of Mailman, Django, and HyperKitty installed are (if you can't get that from the OS package manager, you can just ls /usr/lib/python3.10/site-packages -- for most packages there will be two entries, one ending with a version number and .egg, .egg-info, or .dist-info).
Regards, Steve
Thank you for your reply.
(1) I installed while watching "Virtualenv Installation". (https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...)
Specifically, it is installed with the following command.
# su mailman $ pip install wheel mailman psycopg2-binary\<2.9
(2) This list was newly created with Mailman 3. Specifically, it is created with the following command.
# su mailman $ mailman create testml@list1.example.com
(3)The installed versions are as follows.
mailman-3.3.5 postorius-1.3.6 HyperKitty-1.3.5 mailmanclient-3.3.3 mailman_hyperkitty-1.2.0 Django-3.0.14 django_mailman3-1.3.7
It may be irrelevant, but the thread uses Japanese(multibyte). Please let me know if any other information is needed.
Thank you very much.
On 10/19/22 17:20, hasimoto@designet.co.jp wrote:
It may be irrelevant, but the thread uses Japanese(multibyte). Please let me know if any other information is needed.
That might be relevant.
Can you query the database? Find the message_id_hash of the thread. This is the 32 character string at the end of the URL to that thread in the archive. Then
SELECT subject FROM hyperkitty_email WHERE message_id_hash = that string
;
and see what you get.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hello, Mark.
thanks for the advice. I tried running it.
mailmanweb=# SELECT subject FROM hyperkitty_email WHERE message_id_hash = '6V34IXUPG7SVEG3TAQ3KZEBSK6F3A34Y'; subject
[Testml] 質問です (1 row)
I also created an English thread to see if multibyte is the culprit.
mailmanweb=# SELECT subject FROM hyperkitty_email WHERE message_id_hash = 'AWBTXSMCOIDUGO426ZWGBUESBQC3NP53'; subject
[Testml] hello (1 row)
Similarly, clicking "Reconnect this thread" in the archive resulted in an internal server error...
On 10/18/22 21:38, hasimoto@designet.co.jp wrote:
Hi,
In the archive, clicking "Reconnect this thread" causes an internal server error. I received the following email, but I don't know how to solve it, please help me.
How big is the archive in which you are doing this?
Are you running the periodic django-admin jobs via cron? See https://gitlab.com/mailman/hyperkitty/-/blob/master/example_project/crontab
I ask because I've now seen this issue in a test installation, but only with an archive containing only one message. The particular steps I followed somewhat by accident were:
Start with an archive containing only two messages in separate threads.
Open one thread and Delete this thread
leaving only one thread.
Open that thread and click Reattach this thread
- this produces the
error.
Restore the deleted message/thread with hyperkitty_import
Clicking Reattach this thread
in either thread produces the error.
After running the hourly and daily django-admin jobs, clicking Reattach this thread
in either thread works as expected.
That scenario with deleting and restoring a thread was somewhat
accidental, but did produce the error. in other list archives on the
same server, even one containing only one message, clicking Reattach this thread
in either thread works as expected.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hey.
Looking at other code, "subject" is a method.
I fixed it by correcting the code. is this a bug? python version issue?
- /opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/views/thread.py
411 @check_mlist_private 412 def reattach_suggest(request, mlist_fqdn, threadid): 413 mlist = get_object_or_404(MailingList, name=mlist_fqdn) 414 thread = get_object_or_404(Thread, mailinglist=mlist, thread_id=threadid) 415 416 default_search_query = stripped_subject( # change: thread.subject to thread.subject() 417 mlist, thread.subject()).lower().replace("re:", "")
- /opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/models/thread.py
69 def __str__(self): # change: thread.subject to thread.subject() 70 return self.subject()
On 10/25/22 17:08, cold00water@gmail.com wrote:
Hey.
Looking at other code, "subject" is a method.
I fixed it by correcting the code. is this a bug? python version issue?
- /opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/views/thread.py
411 @check_mlist_private 412 def reattach_suggest(request, mlist_fqdn, threadid): 413 mlist = get_object_or_404(MailingList, name=mlist_fqdn) 414 thread = get_object_or_404(Thread, mailinglist=mlist, thread_id=threadid) 415 416 default_search_query = stripped_subject( # change: thread.subject to thread.subject() 417 mlist, thread.subject()).lower().replace("re:", "")
- /opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/models/thread.py
69 def __str__(self): # change: thread.subject to thread.subject() 70 return self.subject()
Did you test this?, How? I.e. where you able to produce the error without your change and then verify that your change fixed it?
The existing code works in all but rare situations, and I think those are due to the thread object being instantiated with missing arguments.
In any case, your changes cause the ReattachTestCase.test_suggestions test in hyperkitty/tests/views/test_thread.py to fail as follows.
hyperkitty/tests/views/test_thread.py:66: in test_suggestions
response = self.client.get(reverse('hk_thread_reattach_suggest',
.tox/py39-django32/lib/python3.9/site-packages/django/test/client.py:742:
in get
response = super().get(path, data=data, secure=secure, **extra)
.tox/py39-django32/lib/python3.9/site-packages/django/test/client.py:396:
in get
return self.generic('GET', path, secure=secure, **{
.tox/py39-django32/lib/python3.9/site-packages/django/test/client.py:473:
in generic
return self.request(**r)
.tox/py39-django32/lib/python3.9/site-packages/django/test/client.py:719:
in request
self.check_exception(response)
.tox/py39-django32/lib/python3.9/site-packages/django/test/client.py:580:
in check_exception
raise exc_value
.tox/py39-django32/lib/python3.9/site-packages/django/core/handlers/exception.py:47:
in inner
response = get_response(request)
.tox/py39-django32/lib/python3.9/site-packages/django/core/handlers/base.py:181:
in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
hyperkitty/lib/view_helpers.py:137: in inner
return func(request, *args, **kwargs)
hyperkitty/views/thread.py:445: in reattach_suggest
mlist, thread.subject()).lower().replace("re:", "")
E TypeError: 'str' object is not callable
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I have only tested the operation on the web screen. I ran c, but it doesn't work because of another error.
$ python test_thread.py Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/tests/views/test_thread.py", line 29, in <module> from django.contrib.auth.models import User File "/opt/mailman/venv/lib/python3.10/site-packages/django/contrib/auth/models.py", line 2, in <module> from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/opt/mailman/venv/lib/python3.10/site-packages/django/contrib/auth/base_user.py", line 47, in <module> class AbstractBaseUser(models.Model): File "/opt/mailman/venv/lib/python3.10/site-packages/django/db/models/base.py", line 107, in __new__ app_config = apps.get_containing_app_config(module) File "/opt/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 252, in get_containing_app_config self.check_apps_ready() File "/opt/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 134, in check_apps_ready settings.INSTALLED_APPS File "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 76, in __getattr__ self._setup(name) File "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 57, in _setup raise ImproperlyConfigured( django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Please tell me how to test.
On 10/26/22 02:29, cold00water@gmail.com wrote:
I have only tested the operation on the web screen.
What exactly did you do to test this?
I ran c, but it doesn't work because of another error.
$ python test_thread.py Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.10/site-packages/hyperkitty/tests/views/test_thread.py", line 29, in <module> from django.contrib.auth.models import User File "/opt/mailman/venv/lib/python3.10/site-packages/django/contrib/auth/models.py", line 2, in <module> from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager File "/opt/mailman/venv/lib/python3.10/site-packages/django/contrib/auth/base_user.py", line 47, in <module> class AbstractBaseUser(models.Model): File "/opt/mailman/venv/lib/python3.10/site-packages/django/db/models/base.py", line 107, in __new__ app_config = apps.get_containing_app_config(module) File "/opt/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 252, in get_containing_app_config self.check_apps_ready() File "/opt/mailman/venv/lib/python3.10/site-packages/django/apps/registry.py", line 134, in check_apps_ready settings.INSTALLED_APPS File "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 76, in __getattr__ self._setup(name) File "/opt/mailman/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 57, in _setup raise ImproperlyConfigured( django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Please tell me how to test.
You need to get the source. E.g.,
git clone https://gitlab.com/mailman/hyperkitty.git
This will create a hyperkitty
directory in the current directory. Then
you can test with tox. Ensure you have tox and then something like
tox -e py310-django32 -- hyperkitty/tests/views/test_thread.py
will run those tests. You can probably also test with pytest
, but when
I tried that I ran into issues with Django not finding settings.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
In the web screen test, it was only confirmed that the "Reconnect this thread" error disappeared and threads could be added.
Something might be wrong with the environment, so I'll try again in vanilla.
In the web screen test, it was only confirmed that the "Reconnect this thread" error disappeared and threads could be added. Did you actually see the "Reconnect this thread" error before you made
On 10/26/22 19:07, cold00water@gmail.com wrote: the changes, and after you medae the changes were you able to successfully reconnect the thread to another thread?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (5)
-
cold00water@gmail.com
-
hasimoto dg
-
hasimoto@designet.co.jp
-
Mark Sapiro
-
Stephen J. Turnbull