hyperkitty.lib.utils Failed to update the fulltext index
hello there,
i migrated to a new host last night and i've been getting this error from a cronjob mail every minute. anyone have any ideas what would cause this?
ERROR 2019-04-17 14:26:33,056 27205 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/hyperkitty/search_indexes.py", line 87, in update_index update_cmd.update_backend("hyperkitty", "default") File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 276, in update_backend last_max_pk=max_pk) File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/lib/python2.7/dist-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/lib/python2.7/dist-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1))
thanks, ben
On Wed, Apr 17, 2019, at 7:28 AM, Ben Harris wrote:
hello there,
i migrated to a new host last night and i've been getting this error from a cronjob mail every minute. anyone have any ideas what would cause this?
Which version of Hyperkitty are you using?
Did you upgrade the version of Hyperkitty when you migrated to the new host?
ERROR 2019-04-17 14:26:33,056 27205 hyperkitty.lib.utils Failed to update the fulltext index: ord() expected a character, but string of length 0 found Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/hyperkitty/lib/utils.py", line 186, in run_with_lock fn(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/hyperkitty/search_indexes.py", line 87, in update_index update_cmd.update_backend("hyperkitty", "default") File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 276, in update_backend last_max_pk=max_pk) File "/usr/lib/python2.7/dist-packages/haystack/management/commands/update_index.py", line 97, in do_update backend.update(index, current_qs, commit=commit) File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 178, in update self.setup() File "/usr/lib/python2.7/dist-packages/haystack/backends/whoosh_backend.py", line 126, in setup self.index = self.storage.open_index(schema=self.schema) File "/usr/lib/python2.7/dist-packages/whoosh/filedb/filestore.py", line 176, in open_index return indexclass(self, schema=schema, indexname=indexname) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 421, in __init__ TOC.read(self.storage, self.indexname, schema=self._schema) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 632, in read check_size("int", _INT_SIZE) File "/usr/lib/python2.7/dist-packages/whoosh/index.py", line 626, in check_size sz = stream.read_varint() File "/usr/lib/python2.7/dist-packages/whoosh/filedb/structfile.py", line 191, in read_varint return read_varint(self.read) File "/usr/lib/python2.7/dist-packages/whoosh/util/varints.py", line 102, in read_varint b = ord(readfn(1))
thanks, ben
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
April 17, 2019 11:03 AM, "Abhilash Raj" <maxking@asynchronous.in> wrote:
Which version of Hyperkitty are you using?
Did you upgrade the version of Hyperkitty when you migrated to the new host?
I'm using the mailman3-full package in ubuntu (i'm aware it's behind a couple releases), so it's hyperkitty 1.1.4.
I didn't upgrade anything with the move.
Thanks, Ben
On Wed, Apr 17, 2019, at 8:06 AM, Ben Harris wrote:
April 17, 2019 11:03 AM, "Abhilash Raj" <maxking@asynchronous.in> wrote:
Which version of Hyperkitty are you using?
Did you upgrade the version of Hyperkitty when you migrated to the new host?
I'm using the mailman3-full package in ubuntu (i'm aware it's behind a couple releases), so it's hyperkitty 1.1.4.
I didn't upgrade anything with the move.
I am not sure where does Debian packaged version store it's fulltext index, but I am assuming the move somehow makes Hyperkitty not be able to read the index.
Can you try removing and re-building the entire index? I don't know again
the right command for Django there, but the hyperkitty command you want
to run is called rebuild_index. For source package, something like:
$ python manage.py rebuild_index
Do note that this will take a lot of time if you have a big archive.
Thanks, Ben
-- thanks, Abhilash Raj (maxking)
On 4/17/19 8:10 AM, Abhilash Raj wrote:
On Wed, Apr 17, 2019, at 8:06 AM, Ben Harris wrote:
April 17, 2019 11:03 AM, "Abhilash Raj" <maxking@asynchronous.in> wrote:
Which version of Hyperkitty are you using?
Did you upgrade the version of Hyperkitty when you migrated to the new host?
I'm using the mailman3-full package in ubuntu (i'm aware it's behind a couple releases), so it's hyperkitty 1.1.4.
I didn't upgrade anything with the move.
I am not sure where does Debian packaged version store it's fulltext index, but I am assuming the move somehow makes Hyperkitty not be able to read the index.
I think this is a known incompatibility between the Python 2.7 and Python 3 versions of Whoosh.
From the tracebacks, the current HyperKitty is running under Python 2.7. If the prior server was running HyperKitty with Python 3, that would explain this.
Can you try removing and re-building the entire index? I don't know again the right command for Django there, but the hyperkitty command you want to run is called
rebuild_index. For source package, something like:$ python manage.py rebuild_index
Do note that this will take a lot of time if you have a big archive.
If this is the incompatibility, rebuilding the index will fix it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, Apr 17, 2019, at 8:18 AM, Mark Sapiro wrote:
On 4/17/19 8:10 AM, Abhilash Raj wrote:
On Wed, Apr 17, 2019, at 8:06 AM, Ben Harris wrote:
April 17, 2019 11:03 AM, "Abhilash Raj" <maxking@asynchronous.in> wrote:
Which version of Hyperkitty are you using?
Did you upgrade the version of Hyperkitty when you migrated to the new host?
I'm using the mailman3-full package in ubuntu (i'm aware it's behind a couple releases), so it's hyperkitty 1.1.4.
I didn't upgrade anything with the move.
I am not sure where does Debian packaged version store it's fulltext index, but I am assuming the move somehow makes Hyperkitty not be able to read the index.
I think this is a known incompatibility between the Python 2.7 and Python 3 versions of Whoosh.
I *think* 1.1.x releases were based on Python 2.7, so that part looks right to me.
From the tracebacks, the current HyperKitty is running under Python 2.7. If the prior server was running HyperKitty with Python 3, that would explain this.
Given that 1.1.x supports only 2.7 and Ubuntu doesn't have 1.2 release, it is unlikely Python 3 was used.
Can you try removing and re-building the entire index? I don't know again the right command for Django there, but the hyperkitty command you want to run is called
rebuild_index. For source package, something like:$ python manage.py rebuild_index
Do note that this will take a lot of time if you have a big archive.
If this is the incompatibility, rebuilding the index will fix it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
April 17, 2019 11:10 AM, "Abhilash Raj" <maxking@asynchronous.in> wrote:
I am not sure where does Debian packaged version store it's fulltext index, but I am assuming the move somehow makes Hyperkitty not be able to read the index.
For anyone interested/posterity, I've found the index in /var/lib/mailman3/web/fulltext_index.
Can you try removing and re-building the entire index? I don't know again the right command for Django there, but the hyperkitty command you want to run is called
rebuild_index. For source package, something like:
$ python manage.py rebuild_index
manage.py is in /usr/share/mailman3-web (with settings symlinked to /etc/mailman3).
I'm getting the same TypeError: ord() expected a character, but string of length 0 found when running rebuild_index manually.
It turns out that MAIN_WRITELOCK got copied over in the migration and removing all files from /var/lib/mailman3/web/fulltext_index/ solved the issue.
Thanks again, Ben
Ben, just a quick thank you - I found this post in the archives and it solved a problem that had me banging my head against the wall.
š
- Mark
also thank you for this pointer here, this time a debian v12 setup saved :)
the issue after migrating lists from mailman21 to v3 and following the guide was that the index was rebuilt as root and therefore the cronjob running as www-data could not access these files - chown'ed the files to www-data and rebuilt the index as this very user - everything went fine.
thx!
participants (5)
-
Abhilash Raj -
Ben Harris -
hkļ¼ kapper.net -
Mark Dadgar -
Mark Sapiro