I'm having an issue with the mailman web logs (/opt/mailman/web/logs/uwsgi-error.log) spamming the below:
20:53:44 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. : Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.6/site-packages/django_q/cluster.py", line 436, in worker res = f(*task["args"], **task["kwargs"]) File "/opt/mailman/venv/lib64/python3.6/site-packages/hyperkitty/tasks.py", line 79, in _rebuild_mailinglist_cache_for_month mlist = MailingList.objects.get(name=mlist_name) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/db/models/query.py", line 417, in get self.model._meta.object_name hyperkitty.models.mailinglist.MailingList.DoesNotExist: MailingList matching query does not exist.
the best match I can find in the archives is
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
I have used suggested command there (/opt/mailman/venv/bin/mailman-web runjobs recent_threads_cache) as well as manually running the hourly, daily, weekly, and monthly crons manually and all run successfully but I'm still getting the same entries spamming the logs to the point that when I discovered this the logs were 9.6GB in size.
I do not have a list named "MailingList".
On 6/28/21 2:02 PM, Sarah H wrote:
I'm having an issue with the mailman web logs (/opt/mailman/web/logs/uwsgi-error.log) spamming the below:
20:53:44 [Q] ERROR Failed [rebuild_mailinglist_cache_for_month] - MailingList matching query does not exist. : Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.6/site-packages/django_q/cluster.py", line 436, in worker res = f(*task["args"], **task["kwargs"]) File "/opt/mailman/venv/lib64/python3.6/site-packages/hyperkitty/tasks.py", line 79, in _rebuild_mailinglist_cache_for_month mlist = MailingList.objects.get(name=mlist_name) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/db/models/query.py", line 417, in get self.model._meta.object_name hyperkitty.models.mailinglist.MailingList.DoesNotExist: MailingList matching query does not exist. ... I do not have a list named "MailingList".
hyperkitty.models.mailinglist.MailingList.DoesNotExist is the name of the exception. MailingList refers to the hyperkitty.models.mailinglist.MailingList class. It is not the name of a list.
The actual code is
def _rebuild_mailinglist_cache_for_month(mlist_name, year, month):
try:
mlist = MailingList.objects.get(name=mlist_name)
except MailingList.DoesNotExist:
log.warning('MailingList with name %s does not exist', mlist_name)
return
mlist.cached_values["participants_count_for_month"].rebuild(year,
month)
Thus, there should be a warning level log message
MailingList with name xxx does not exist
Identifying the list name. This will be in Django's log which is probably not /opt/mailman/web/logs/uwsgi-error.log, but may be /opt/mailman/web/logs/mailmansuitelog. It is defined in settings(_local).py as the value of LOGGING['handlers']['file']['filename']
I'm not sure how this issue comes about. I suspect it comes from a task
that was created before some list was deleted and the task is being
continuously retried because it fails. What does your Django admin
qinfo command show?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
from the Django settings the log is here:
Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/opt/mailman/web/logs/mailmanweb.log'
but the only new entry in the log is from when I attempted to run the runjobs as the wrong user and the log entries for unable to connect to DB from when the uwsgi log filled the drive and the db process died, which is what originally alerted me to the issue.
ERROR 2021-06-28 20:29:46,227 1974245 hyperkitty.lib.utils Failed to update the fulltext index: [Errno 13] Permission denied: 'fulltext_index' Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.6/site-packages/hyperkitty/lib/utils.py", line 169, in run_with_lock fn(*args, **kwargs) File "/opt/mailman/venv/lib64/python3.6/site-packages/hyperkitty/search_indexes.py", line 118, in update_index update_cmd.update_backend("hyperkitty", "default") File "/opt/mailman/venv/lib64/python3.6/site-packages/haystack/management/commands/update_index.py", line 329, in update_backend last_max_pk=max_pk, File "/opt/mailman/venv/lib64/python3.6/site-packages/haystack/management/commands/update_index.py", line 111, in do_update backend.update(index, current_qs, commit=commit) File "/opt/mailman/venv/lib64/python3.6/site-packages/haystack/backends/whoosh_backend.py", line 262, in update self.setup() File "/opt/mailman/venv/lib64/python3.6/site-packages/haystack/backends/whoosh_backend.py", line 144, in setup os.makedirs(self.path) File "/usr/lib64/python3.6/os.py", line 220, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: 'fulltext_index'
On 6/29/21 8:44 AM, Sarah H wrote:
from the Django settings the log is here:
Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/opt/mailman/web/logs/mailmanweb.log'
but the only new entry in the log is from when I attempted to run the runjobs as the wrong user and the log entries for unable to connect to DB from when the uwsgi log filled the drive and the db process died, which is what originally alerted me to the issue.
The catching of the "MailingList.DoesNotExist" exception and logging "MailingList with name xxx does not exist" was recently added and will be released in HyperKitty 1.3.5.
Apply the patch to hyperkitty/tasks.py from https://gitlab.com/mailman/hyperkitty/-/merge_requests/341/diffs which should replace the uwsgi-error.log entries with the one line entries in mailmanweb.log.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Mark Sapiro -
Sarah H