Anyone seen this before? This is the result of any search query I run. I installed ElasticSearch as the search engine yesterday and this is the result. It is still indexing (large archives).
Mailman 3.2.1 from the debian packages in Ubuntu 19.10.
- Mark
mark@pdc-racing.net | 408-348-2878
Begin forwarded message:
From: root@pdc-racing.net Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/search Date: October 22, 2019 at 7:44:49 AM PDT To: mark@pdc-racing.net
Internal Server Error: /mailman3/hyperkitty/search
UnboundLocalError at /hyperkitty/search local variable 'search_exception' referenced before assignment
Request Method: GET Request URL: https://mail.pdc-racing.net/mailman3/hyperkitty/search?q=Ken Django Version: 1.11.22 Python Executable: /usr/bin/uwsgi-core Python Version: 3.7.5 Python Path: ['.', '', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] Server time: Tue, 22 Oct 2019 14:44:49 +0000 Installed Applications: ('hyperkitty', 'postorius', 'django_mailman3', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'rest_framework', 'django_gravatar', 'compressor', 'haystack', 'django_extensions', 'django_q', 'allauth', 'allauth.account', 'allauth.socialaccount', 'django_mailman3.lib.auth.fedora') Installed Middleware: ('django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.middleware.security.SecurityMiddleware', 'django_mailman3.middleware.TimezoneMiddleware', 'postorius.middleware.PostoriusMiddleware')
Traceback:
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in count 79. return self.object_list.count()
File "/usr/lib/python3/dist-packages/haystack/query.py" in count 522. return len(self)
File "/usr/lib/python3/dist-packages/haystack/query.py" in __len__ 86. self._result_count = self.query.get_count()
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in get_count 619. self.run()
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in run 951. results = self.backend.search(final_query, **search_kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in wrapper 33. return func(obj, query_string, *args, **kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in search 524. _source=True)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch/client/utils.py" in _wrapped 84. return func(*args, params=params, **kwargs)
During handling of the above exception (search() got an unexpected keyword argument 'doc_type'), another exception occurred:File "/usr/lib/python3/dist-packages/hyperkitty/views/search.py" in search 90. request.GET.get('count'),
File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in paginate 72. objects = paginator.page(page_num)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in page 57. number = self.validate_number(number)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in validate_number 46. if number > self.num_pages:
File "/usr/lib/python3/dist-packages/django/utils/functional.py" in __get__ 35. res = instance.__dict__[self.name] = self.func(instance)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in num_pages 91. if self.count == 0 and not self.allow_empty_first_page:
File "/usr/lib/python3/dist-packages/django/utils/functional.py" in __get__ 35. res = instance.__dict__[self.name] = self.func(instance)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in count 84. return len(self.object_list)
File "/usr/lib/python3/dist-packages/haystack/query.py" in __len__ 86. self._result_count = self.query.get_count()
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in get_count 619. self.run()
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in run 951. results = self.backend.search(final_query, **search_kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in wrapper 33. return func(obj, query_string, *args, **kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in search 524. _source=True)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch/client/utils.py" in _wrapped 84. return func(*args, params=params, **kwargs)
During handling of the above exception (search() got an unexpected keyword argument 'doc_type'), another exception occurred:File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/search.py" in search 100. if not isinstance(e, search_exception):
Exception Type: UnboundLocalError at /hyperkitty/search Exception Value: local variable 'search_exception' referenced before assignment Request information: USER: admin
GET: q = 'Ken'
POST: No POST data
FILES: No FILES data
On 10/22/19 7:51 AM, Mark Dadgar wrote:
Anyone seen this before? This is the result of any search query I run. I installed ElasticSearch as the search engine yesterday and this is the result. It is still indexing (large archives).
How did you install ElasticSearch. What ElasticSearch version? This list uses elasticsearch 5.5.3 installed from the .deb as described at <https://www.elastic.co/guide/en/elasticsearch/reference/5.5/deb.html#install-deb>. You can't use a later version because there are no python bindings for versions later than 5.x In addition we use the python bindings installed with `pip install elasticsearch5` <https://pypi.org/project>/elasticsearch5/ This version also requires haystack > 2.8.1 which we installed via `pip install -e git+https://github.com/django-haystack/django-haystack.git@master#egg=django-haystack` Finally, we have these patches to the python bindings
--- elasticsearch5_backend.py.orig 2019-05-24 21:26:04.031867202 -0700 +++ elasticsearch5_backend.py 2019-05-25 08:45:42.521374647 -0700 @@ -28,6 +28,15 @@ installation of 'elasticsearch>=5.0.0,<6.0.0'. \ Please refer to the documentation." ) +# We need to override some elasticsearch_backend settings as string type is +# gone in elasticsearch 5. +from haystack.backends import elasticsearch_backend +elasticsearch_backend.DEFAULT_FIELD_MAPPING = { + "type": "text", "analyzer": "snowball"} +elasticsearch_backend.FIELD_MAPPINGS["edge_ngram"] = { + "type": "text", "analyzer": "edgengram_analyzer"} +elasticsearch_backend.FIELD_MAPPINGS["ngram"] = { + "type": "text", "analyzer": "ngram_analyzer"}
class Elasticsearch5SearchBackend(ElasticsearchSearchBackend): --- elasticsearch_backend.py.orig 2019-06-16 10:21:42.441334321 -0700 +++ elasticsearch_backend.py 2019-06-16 10:24:20.802264202 -0700 @@ -78,8 +78,10 @@ )
# Settings to add an n-gram & edge n-gram analyzer. + # And, set max_result_window. DEFAULT_SETTINGS = { "settings": { + "max_result_window": "100000", "analysis": { "analyzer": { "ngram_analyzer": {
The patch to elasticsearch_backend.py may require an even bigger value for max_result_window. The issue is the monthly Django hyperkitty - update_and_clean_index job runs into <https://stackoverflow.com/questions/41677198/result-window-is-too-large-from-size-must-be-less-than-or-equal-to-10000-b/41677310>
Mailman 3.2.1 from the debian packages in Ubuntu 19.10.
- Mark ----- mark@pdc-racing.net | 408-348-2878
Begin forwarded message:
From: root@pdc-racing.net Subject: [Django] ERROR (EXTERNAL IP): Internal Server Error: /mailman3/hyperkitty/search Date: October 22, 2019 at 7:44:49 AM PDT To: mark@pdc-racing.net
Internal Server Error: /mailman3/hyperkitty/search
UnboundLocalError at /hyperkitty/search local variable 'search_exception' referenced before assignment
Request Method: GET Request URL: https://mail.pdc-racing.net/mailman3/hyperkitty/search?q=Ken ... Traceback:
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in count 79. return self.object_list.count()
File "/usr/lib/python3/dist-packages/haystack/query.py" in count 522. return len(self)
File "/usr/lib/python3/dist-packages/haystack/query.py" in __len__ 86. self._result_count = self.query.get_count()
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in get_count 619. self.run()
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in run 951. results = self.backend.search(final_query, **search_kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in wrapper 33. return func(obj, query_string, *args, **kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in search 524. _source=True)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch/client/utils.py" in _wrapped 84. return func(*args, params=params, **kwargs)
During handling of the above exception (search() got an unexpected keyword argument 'doc_type'), another exception occurred:
I suspect this error is because indexing is still in process. I'm not certain, but I think so.
File "/usr/lib/python3/dist-packages/hyperkitty/views/search.py" in search 90. request.GET.get('count'),
File "/usr/lib/python3/dist-packages/django_mailman3/lib/paginator.py" in paginate 72. objects = paginator.page(page_num)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in page 57. number = self.validate_number(number)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in validate_number 46. if number > self.num_pages:
File "/usr/lib/python3/dist-packages/django/utils/functional.py" in __get__ 35. res = instance.__dict__[self.name] = self.func(instance)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in num_pages 91. if self.count == 0 and not self.allow_empty_first_page:
File "/usr/lib/python3/dist-packages/django/utils/functional.py" in __get__ 35. res = instance.__dict__[self.name] = self.func(instance)
File "/usr/lib/python3/dist-packages/django/core/paginator.py" in count 84. return len(self.object_list)
File "/usr/lib/python3/dist-packages/haystack/query.py" in __len__ 86. self._result_count = self.query.get_count()
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in get_count 619. self.run()
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in run 951. results = self.backend.search(final_query, **search_kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/__init__.py" in wrapper 33. return func(obj, query_string, *args, **kwargs)
File "/usr/lib/python3/dist-packages/haystack/backends/elasticsearch_backend.py" in search 524. _source=True)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch/client/utils.py" in _wrapped 84. return func(*args, params=params, **kwargs)
During handling of the above exception (search() got an unexpected keyword argument 'doc_type'), another exception occurred:
File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request)
File "/usr/lib/python3/dist-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python3/dist-packages/hyperkitty/views/search.py" in search 100. if not isinstance(e, search_exception):
Exception Type: UnboundLocalError at /hyperkitty/search Exception Value: local variable 'search_exception' referenced before assignment
This is a bug in HyperKitty. We've never encountered it, but the code at <https://gitlab.com/mailman/hyperkitty/blob/master/hyperkitty/views/search.py#L93> only considers the whoosh and xapian backends and needs to be updated for elasticsearch. I've reported that at <https://gitlab.com/mailman/hyperkitty/issues/263>. -- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 10/22/19 6:44 PM, Mark Sapiro wrote:
This list uses elasticsearch 5.5.3 installed from the .deb as described at <https://www.elastic.co/guide/en/elasticsearch/reference/5.5/deb.html#install-deb>. You can't use a later version because there are no python bindings for versions later than 5.x
Apparently there are now bindings for elasticsearch 6, but I haven't tried to use them
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Oct 22, 2019, at 6:44 PM, Mark Sapiro <mark@msapiro.net> wrote:
On 10/22/19 7:51 AM, Mark Dadgar wrote:
Anyone seen this before? This is the result of any search query I run. I installed ElasticSearch as the search engine yesterday and this is the result. It is still indexing (large archives).
How did you install ElasticSearch. What ElasticSearch version?
From .deb, version 5.6.16.
This is a bug in HyperKitty. We've never encountered it, but the code at <https://gitlab.com/mailman/hyperkitty/blob/master/hyperkitty/views/search.py#L93> only considers the whoosh and xapian backends and needs to be updated for elasticsearch. I've reported that at <https://gitlab.com/mailman/hyperkitty/issues/263>.
Hmm, maybe I’ll take another shot at getting Xapian running then.
Whoosh ran for 5+ days trying to index my archive of 556,000+ messages before I gave up.
Thanks for your help, Mark. Much appreciated. If we ever bump into each other here in the Valley, I owe you a beer or 9 for all the value I’ve gotten from mailman over the years.
- Mark
mark@pdc-racing.net | 408-348-2878
On 10/22/19 9:41 PM, Mark Dadgar wrote:
Hmm, maybe I’ll take another shot at getting Xapian running then.
Whoosh ran for 5+ days trying to index my archive of 556,000+ messages before I gave up.
I haven't tried Xapian. On mail.python.org, I dont recall how many total messages there were at the time I switched from Whoosh to ElasticSearch, but a full index rebuild with Whoosh would take several hours, whereas ElasticSearch took about 10 minutes.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Oct 23, 2019, at 7:48 AM, Mark Sapiro <mark@msapiro.net> wrote:
I haven't tried Xapian. On mail.python.org, I dont recall how many total messages there were at the time I switched from Whoosh to ElasticSearch, but a full index rebuild with Whoosh would take several hours, whereas ElasticSearch took about 10 minutes.
ElasticSearch has been indexing for ~36 hours now. /var/lib/elasticsearch is currently 5.5GB.
- Mark
mark@pdc-racing.net | 408-348-2878
On Oct 22, 2019, at 9:41 PM, Mark Dadgar <mark@pdc-racing.net> wrote:
Hmm, maybe I’ll take another shot at getting Xapian running then.
So I did get Xapian running this morning and OMG it is fast. It indexed the entire archive in less than an hour and search from the archives page is lightning quick.
This second install attempt was pretty painless, mostly because I now know that I needed to install the appropriate haystack bindings for xapian via pip and also more critically that pip != pip3 - they are different animals.
Thank you again for the help, Mark. Mailman3 is a HUGE upgrade over 2.1.x.
- Mark
mark@pdc-racing.net | 408-348-2878
participants (2)
-
Mark Dadgar -
Mark Sapiro