Hyperkitty tracebacks: possible DB corruption?
Hi Folks, Just recently I've started seeing backtraces from Django when trying to view old threads in hyperkitty. They say that more than one LastView is being returned, and at a later date cannot delete a LastView.
I'm using the Debian-packaged hyperkitty and postorius, but this code hasn't changed from upstream.
The backtraces come when a logged in used revisits a thread.
So I see:
Internal Server Error: /hyperkitty/list/devel@sel4.systems/thread/XP2GFYGXOXBC3QBKKZQLZ6STSHQHK6K2/
MultipleObjectsReturned at /hyperkitty/list/devel@sel4.systems/thread/XP2GFYGXOXBC3QBKKZQLZ6STSHQHK6K2/ get() returned more than one LastView -- it returned 2!
Then later on I see: LastView.objects.delete(pk=del_id) ^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'Manager' object has no attribute 'delete'
Here are the backtraces:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/lib/view_helpers.py", line 137, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/thread.py", line 162, in thread_index last_view_obj, created = LastView.objects.get_or_create( File "/usr/lib/python3/dist-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 581, in get_or_create return self.get(**kwargs), False File "/usr/lib/python3/dist-packages/django/db/models/query.py", line 439, in get raise self.model.MultipleObjectsReturned(
and
Internal Server Error: /hyperkitty/list/devel@sel4.systems/2022/1/
AttributeError at /hyperkitty/list/devel@sel4.systems/2022/1/ 'Manager' object has no attribute 'delete'
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/lib/view_helpers.py", line 137, in inner return func(request, *args, **kwargs) File "/usr/lib/python3/dist-packages/hyperkitty/views/mlist.py", line 127, in archives return _thread_list(request, mlist, threads, extra_context=extra_context) File "/usr/lib/python3/dist-packages/hyperkitty/views/mlist.py", line 173, in _thread_list LastView.objects.delete(pk=del_id)
On this, I note that in https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/models/thread... there is explicit check for the multiple LastView issue, and one of them is deleted; but in
https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/thread....
that check is not made.
-- Peter C
On the other error, the 'Manager' object has no attribute 'delete' one, I suspect you need: LastView.objects.get(pk=del_id).delete() instead of LastView.objects.delete(pk=del_id)
at https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/mlist.p...
I tried this and it seems to work.
-- Peter C
On 3/12/24 2:47 PM, Peter Chubb via Mailman-users wrote:
On the other error, the 'Manager' object has no attribute 'delete' one, I suspect you need: LastView.objects.get(pk=del_id).delete() instead of LastView.objects.delete(pk=del_id)
at https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/mlist.p...
I tried this and it seems to work.
Thank you for your analysis. I have filed https://gitlab.com/mailman/hyperkitty/-/issues/498 and will fix it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
FOr the other issue, I changed https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/thread....
From: if request.user.is_authenticated: last_view_obj, created = LastView.objects.get_or_create( thread=thread, user=request.user) if not created: last_view = last_view_obj.view_date last_view_obj.save() # update timestamp
to: if request.user.is_authenticated: try: last_view_obj, created = LastView.objects.get_or_create( thread=thread, user=request.user) except LastView.MultipleObjectsReturned: last_view_duplicate, last_view_obj = LastView.objects.filter( thread=thread, user=request.user).order_by("view_date").all() last_view_duplicate.delete() created = False
if not created:
last_view = last_view_obj.view_date
last_view_obj.save() # update timestampand that seems to have fixed it.
I've been thinking though that it'd be better to fix duplicates in the LastView class rather than on every get() or get_or_create() call.
Peter C
On 3/12/24 11:12 PM, peter--- via Mailman-users wrote:
FOr the other issue, I changed https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/views/thread....
From:
> if request.user.is_authenticated:
> last_view_obj, created = LastView.objects.get_or_create(
> thread=thread, user=request.user)
> if not created:
> last_view = last_view_obj.view_date
> last_view_obj.save() # update timestamp
to:
> if request.user.is_authenticated:
> try:
> last_view_obj, created = LastView.objects.get_or_create(
> thread=thread, user=request.user)
> except LastView.MultipleObjectsReturned:
> last_view_duplicate, last_view_obj = LastView.objects.filter(
> thread=thread, user=request.user).order_by("view_date").all()
> last_view_duplicate.delete()
> created = False
>
> if not created:
> last_view = last_view_obj.view_date
> last_view_obj.save() # update timestamp
and that seems to have fixed it.
Wrapping the last_view_obj, created = LastView.objects.get_or_create() in a try, except seems appropriate, but your code in the except clause assumes there will be exactly 2 objects returned. Something like
except LastView.MultipleObjectsReturned:
last_view_objects = LastView.objects.filter(
thread=thread,
user=request.user).order_by("view_date").all()
last_view_obj = last_view_objects[-1]
for object in last_view_objects[:-1]:
object.delete()
might be better.
I've been thinking though that it'd be better to fix duplicates in the LastView class rather than on every get() or get_or_create() call.
That might be good, but these exceptions are rare. I'm not sure how they even occur, so catching them on the get() or get_or_create() calls seems OK.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
"Mark" == Mark Sapiro <mark@msapiro.net> writes:
Mark> On 3/12/24 11:12 PM, peter--- via Mailman-users wrote:
Mark> Wrapping the last_view_obj, created = Mark> LastView.objects.get_or_create() in a try, except seems Mark> appropriate, but your code in the except clause assumes there Mark> will be exactly 2 objects returned.
I was copying code from https://gitlab.com/mailman/hyperkitty/-/blob/master/hyperkitty/models/thread...
Peter C
participants (3)
-
Mark Sapiro -
Peter Chubb -
peter＠chubb.wattle.id.au