Hello --
I'm stuck trying to integrate Hyperkitty. I see the following errors in mailman.log and mailmansuite.log respectively:
<snipped> [2020-05-27 16:42:11 -0400] [5140] [ERROR] Error handling request /3.1/lists/bry_test4.lists-test.rutgers.edu/archivers Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 134, in handle self.handle_request(listener, req, client, addr) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 175, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 274, in falcon.api.API.__call__ File "falcon/api.py", line 269, in falcon.api.API.__call__ File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/rest/lists.py", line 406, in on_get for archiver in archiver_set.archivers File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/rest/lists.py", line 407, in <dictcomp> if archiver.system_archiver.is_enabled} File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/model/mailinglist.py", line 587, in system_archiver raise AssertionError('Archiver not found: {}'.format(self.name)) AssertionError: Archiver not found: hyperkitty
and
<snipped> Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/postorius/middleware.py", line 42, in __call__ return self.get_response(request) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 36, in inner response = response_for_exception(request, exc) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 125, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/views/debug.py", line 89, in technical_500_response reporter = ExceptionReporter(request, exc_type, exc_value, tb) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/django/views/debug.py", line 254, in __init__ self.template_info = getattr(self.exc_value, 'template_debug', None) File "/opt/mailman/mm/venv/lib/python3.6/tempfile.py", line 479, in __getattr__ file = self.__dict__['file'] KeyError: 'file'
Note that I've kinda followed the Virtualenv route along with using Mark's helpful webpage re: his experiences.
Some relevant? config files...
mailman.cfg:
<snip> [archiver.prototype] enable: yes
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /opt/mailman/mm/mailman-hyperkitty.cfg <snip>
mailman-hyperkitty.cfg: [general] base_url: https://a_real_domain/mm3/hyperkitty api_key: <movealong>
I can provide additional conf file setups if warranted? I am curious (and perhaps this is in fact the issue) why when I run mailman conf I don't see the archiver.hyperkittty section even though its in the mailman.cfg file? While at the same time I do see archiver.{mail_archive,mhonarc}? I know it's reading my mailman.cfg based on other sections I've already setup/manipulated.
Another data point:
$ grep -i hyperkitty requirements.txt HyperKitty==1.3.2 mailman-hyperkitty==1.1.0
Any suggestions?
Bryan Loniewski Rutgers University OIT:EAS:Messaging and Collaboration Services System Administrator
Hello --
I'm stuck trying to integrate Hyperkitty. I see the following errors in mailman.log and mailmansuite.log respectively:
<snipped> [2020-05-27 16:42:11 -0400] [5140] [ERROR] Error handling request /3.1/lists/bry_test4.lists-test.rutgers.edu/archivers Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 134, in handle self.handle_request(listener, req, client, addr) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/gunicorn/workers/sync.py", line 175, in handle_request respiter = self.wsgi(environ, resp.start_response) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/database/transaction.py", line 50, in wrapper rtn = function(*args, **kws) File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", line 193, in __call__ return super().__call__(environ, start_response) File "falcon/api.py", line 274, in falcon.api.API.__call__ File "falcon/api.py", line 269, in falcon.api.API.__call__ File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/rest/lists.py", line 406, in on_get for archiver in archiver_set.archivers File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/rest/lists.py", line 407, in <dictcomp> if archiver.system_archiver.is_enabled> File "/opt/mailman/mm/venv/lib/python3.6/site-packages/mailman/model/mailinglist.py",
On 5/27/20 2:24 PM, Bryan Loniewski wrote: line 587, in system_archiver
raise AssertionError('Archiver not found: {}'.format(self.name))AssertionError: Archiver not found: hyperkitty
I've been looking at the code, and I think this can occur if the mailing list has a configured archiver which is no longer known in mailman.config.config.archivers. I.e., the list has hyperkitty as an archiver, but hyperkitty is no longer configured as an archiver in Mailman.
and
<snipped> Traceback (most recent call last): ... KeyError: 'file'
This traceback is useless. All it says is a Django logging error occurred in trying to log the real exception. The interesting one is immediately before this in the log.
Note that I've kinda followed the Virtualenv route along with using Mark's helpful webpage re: his experiences.
Some relevant? config files...
mailman.cfg:
<snip> [archiver.prototype] enable: yes
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver enable: yes configuration: /opt/mailman/mm/mailman-hyperkitty.cfg <snip>
That looks good.
mailman-hyperkitty.cfg: [general] base_url: https://a_real_domain/mm3/hyperkitty api_key: <movealong>
I can provide additional conf file setups if warranted? I am curious (and perhaps this is in fact the issue) why when I run mailman conf I don't see the archiver.hyperkittty section even though its in the mailman.cfg file?
I don't either and I don't know why., For example on lists.mailman3.org
mailman conf|grep [[]archiver
[archiver.mail_archive] class: mailman.archiving.mailarchive.MailArchive
[archiver.mail_archive] clobber_date: maybe
[archiver.mail_archive] clobber_skew: 1d
[archiver.mail_archive] configuration: python:mailman.config.mail_archive
[archiver.mail_archive] enable: no
[archiver.mhonarc] class: mailman.archiving.mhonarc.MHonArc
[archiver.mhonarc] clobber_date: maybe
[archiver.mhonarc] clobber_skew: 1d
[archiver.mhonarc] configuration: python:mailman.config.mhonarc
[archiver.mhonarc] enable: no
[archiver.prototype] class: mailman.archiving.prototype.Prototype
[archiver.prototype] clobber_date: maybe
[archiver.prototype] clobber_skew: 1d
[archiver.prototype] configuration: changeme
[archiver.prototype] enable: yes
doesn't show hyperkitty and neither does
mailman conf -k enable
[archiver.mail_archive] enable: no
[archiver.mhonarc] enable: no
[archiver.prototype] enable: yes
but
/mailman conf -s archiver.hyperkitty
[archiver.hyperkitty] class: mailman_hyperkitty.Archiver
[archiver.hyperkitty] clobber_date: maybe
[archiver.hyperkitty] clobber_skew: 1d
[archiver.hyperkitty] configuration:
/opt/mailman/mm/deployment/mailman-hyperkitty.cfg
[archiver.hyperkitty] enable: yes
So I don't think this per se is the issue.
Any suggestions?
Try
mailman shell
May 28 00:35:47 2020 (15162) patched 1/1 modules (sqlalchemy)
Welcome to the GNU Mailman shell
Use commit() to commit changes.
Use abort() to discard changes since the last commit.
Exit with ctrl+D does an implicit commit() but exit() does not.
>>> from mailman.config import config
>>> list(config.archivers)
[<mailman.archiving.mhonarc.MHonArc object at 0x7fdd9ccff6a0>,
<mailman.archiving.mailarchive.MailArchive object at 0x7fdd9ccffac8>,
<mailman.archiving.prototype.Prototype object at 0x7fdd9ccffb38>,
<mailman_hyperkitty.Archiver object at 0x7fdd9ccffe80>]
>>>
Does yours list hyperkitty?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Yes it does list hyperkitty:
mailman shell Welcome to the GNU Mailman shell
from mailman.config import config list(config.archivers) [<mailman.archiving.mhonarc.MHonArc object at 0x7fa3c3f13358>, <mailman.archiving.mailarchive.MailArchive object at 0x7fa3c3f13828>, <mailman.archiving.prototype.Prototype object at 0x7fa3c3f13908>, <mailman_hyperkitty.Archiver object at 0x7fa3c3f13c18>] quit()
Any other suggestions to try?
On 5/28/20 6:46 AM, brylon@oit.rutgers.edu wrote:
Yes it does list hyperkitty:
mailman shell Welcome to the GNU Mailman shell
from mailman.config import config list(config.archivers) [<mailman.archiving.mhonarc.MHonArc object at 0x7fa3c3f13358>, <mailman.archiving.mailarchive.MailArchive object at 0x7fa3c3f13828>, <mailman.archiving.prototype.Prototype object at 0x7fa3c3f13908>, <mailman_hyperkitty.Archiver object at 0x7fa3c3f13c18>] quit()
Any other suggestions to try?
Just grasping at straws, but have you stopped and started Mailman.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Yes, I've stopped/started Mailman (quite a few times).
Is there a particular order to undo Hyperkitty and re-do? Or perhaps add some debug lines to a particular file to get at the root cause?
On 5/28/20 8:06 AM, brylon@oit.rutgers.edu wrote:
Yes, I've stopped/started Mailman (quite a few times).
Is there a particular order to undo Hyperkitty and re-do? Or perhaps add some debug lines to a particular file to get at the root cause?
I'm not certain, but I don't think your issue has anything to do with the installation of HyperKitty itself. From the OP, the issue occurs in REST upon a get of /3.1/lists/bry_test4.lists-test.rutgers.edu/archivers which is probably coming from Postorius -> Settings -> Archiving.
The actual error occurs in Core trying to process that request, and while it clearly has to do with archiving configuration, I don't think the installation of Hyperkitty is the issue.
What happens if you set
[archiver.hyperkitty] enable: no
and restart Mailman?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Okay, so that worked. I no longer see the errors in the logs nor in the GUI crashing when navigating to the list(s) settings and I even see [] prototype as a checkbox (unchecked) under Setting -> Archiving -> Active archivers unlike previous iterations.
I then took it an obvious step further and re-enabled hyperkitty and restarted Mailman and sure enough the logs look good, the GUI looks proper and the Active archivers now have 2 checkboxes [x]hyperkitty (checked) and []prototype (unchecked) for my test lists.
<some time passes>
I seem to have archives working on posts now, however, during my troubleshooting I moved away some archives/hyperkitty/spool/*.pck files and was wondering if/when I put them back is there a django-admin/hyperkitty command to run and process those again? (and if not I'm pretty confident the next post to any list? would slurp them all into hyperkitty anyway?)
On 5/28/20 3:00 PM, brylon@oit.rutgers.edu wrote:
I seem to have archives working on posts now, however, during my troubleshooting I moved away some archives/hyperkitty/spool/*.pck files and was wondering if/when I put them back is there a django-admin/hyperkitty command to run and process those again? (and if not I'm pretty confident the next post to any list? would slurp them all into hyperkitty anyway?)
Your confidence is not misplaced. Every time mailman_hyperkitty receives a post from Mailman to be archived, it first tries to reprocess all the messages queued in archives/hyperkitty/spool/.
There is no Django command because neither Django nor HyperKitty is directly involved. This is all mailman_hyperkitty which is a shim between Mailman and HyperKitty. Mailman talks to mailman_hyperkitty directly via the configured mailman_hyperkitty.Archiver class
[archiver.hyperkitty]
class: mailman_hyperkitty.Archiver
and mailman_hyperkitty talks to HyperKitty via the API at the base_url configured in mailman-hyperkitty.cfg.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (3)
-
Bryan Loniewski -
brylon＠oit.rutgers.edu
-
Mark Sapiro