Mailman 3.2.0 Hyperkitty fulltext index error
Hi,
I've just upgraded to Mailman 3.2.0 (finally everything Python 3 now, thanks!) and it appears to work. However, there's a repeating error message in mailman-suite's logfile for Hyperkitty that looks like this:
ERROR 2018-07-12 14:28:33,895 2742 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found
Traceback (most recent call last):
File "/home/mailman/env3/lib/python3.5/site-packages/hyperkitty/lib/utils.py", line 179, in run_with_lock
fn(*args, **kwargs)
File "/home/mailman/env3/lib/python3.5/site-packages/hyperkitty/search_indexes.py", line 115, in update_index
update_cmd.update_backend("hyperkitty", "default")
File "/home/mailman/env3/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 276, in update_backend
last_max_pk=max_pk)
File "/home/mailman/env3/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 97, in do_update
backend.update(index, current_qs, commit=commit)
File "/home/mailman/env3/lib/python3.5/site-packages/haystack/backends/whoosh_backend.py", line 178, in update
self.setup()
File "/home/mailman/env3/lib/python3.5/site-packages/haystack/backends/whoosh_backend.py", line 126, in setup
self.index = self.storage.open_index(schema=self.schema)
File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/filedb/filestore.py", line 176, in open_index
return indexclass(self, schema=schema, indexname=indexname)
File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/index.py", line 421, in __init__
TOC.read(self.storage, self.indexname, schema=self._schema)
File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/index.py", line 632, in read
check_size("int", _INT_SIZE)
File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/index.py", line 626, in check_size
sz = stream.read_varint()
File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/filedb/structfile.py", line 191, in read_varint
return read_varint(self.read)
File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/util/varints.py", line 102, in read_varint
b = ord(readfn(1))
TypeError: ord() expected a character, but string of length 0 found
The error repeats every minute, so I suppose it comes from the minutely cronjob. This cronjob looks like this:
- mailman bash -c 'cd /home/mailman && . /home/mailman/env3/bin/activate && django-admin runjobs minutely --pythonpath /home/mailman/mailman-suite/mailman-suite_project --settings settings'
What's wrong here?
Marvin
-- Blog: https://mg.guelker.eu PGP/GPG ID: F1D8799FBCC8BC4F
On 7/12/18 7:34 AM, Marvin Gülker wrote:
Hi,
I've just upgraded to Mailman 3.2.0 (finally everything Python 3 now, thanks!) and it appears to work. However, there's a repeating error message in mailman-suite's logfile for Hyperkitty that looks like this:
ERROR 2018-07-12 14:28:33,895 2742 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/home/mailman/env3/lib/python3.5/site-packages/hyperkitty/lib/utils.py", line 179, in run_with_lock fn(*args, **kwargs) File "/home/mailman/env3/lib/python3.5/site-packages/hyperkitty/search_indexes.py", line 115, in update_index update_cmd.update_backend("hyperkitty", "default") File "/home/mailman/env3/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 276, in update_backend last_max_pk=max_pk) File "/home/mailman/env3/lib/python3.5/site-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/home/mailman/env3/lib/python3.5/site-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/home/mailman/env3/lib/python3.5/site-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/home/mailman/env3/lib/python3.5/site-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1)) TypeError: ord() expected a character, but string of length 0 found
The error repeats every minute, so I suppose it comes from the minutely cronjob. This cronjob looks like this:
This is due the the change from Python 2.7 to Python 3. The existing search index is not compatible with Python 3 whoosh.
I fixed this by deleting all the files in the index directory which is defined as PATH in HAYSTACK_CONNECTIONS in your settings and running 'django_admin update_index', but just running 'django_admin rebuild_index' should do it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi,
Am 12. Juli 2018 um 08:14 Uhr -0700 schrieb Mark Sapiro:
This is due the the change from Python 2.7 to Python 3. The existing search index is not compatible with Python 3 whoosh.
Ah, I see.
I fixed this by deleting all the files in the index directory which is defined as PATH in HAYSTACK_CONNECTIONS in your settings and running 'django_admin update_index', but just running 'django_admin rebuild_index' should do it.
I simply followed your first suggestion, and it worked. I deleted all files in the index directory (which was "fulltext_index" in the "mailman-suite_project" directory, as we use https://gitlab.com/mailman/mailman-suite/tree/master/mailman-suite_project) and then ran update_index. That took a while, but succeeded. Since then the errors are gone.
Thank you! Marvin
-- Blog: https://mg.guelker.eu PGP/GPG ID: F1D8799FBCC8BC4F
for me, it's been enough to just delete the files, the minutely job has rebuilt them. Root cause seems to have been a server reboot that didn't wait for the mailman processes to finish correctly.
participants (3)
-
Jörg Schulz
-
Mark Sapiro
-
Marvin Gülker