The MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header.
Hi everybody,
as per subject, I'm getting this error in the hyperkitty log when mailman daemon tries to archive a mail. I've setup the hyperkitty.cfg file with api_key = MAILMAN_ARCHIVER_KEY in settings_local.py but still nothing. I've seen here and in GitLab issues some errors concerning a wrong api key but here it says that it wasn't sent at all. All packages installed via pip install + latest .zip release, including mailman-hyperkitty. Currently running on apache with mod_wsgi_py3 without https.
Any clue? All the best, Marco Marinello
On 10/22/21 8:51 AM, Marco Marinello wrote:
Hi everybody,
as per subject, I'm getting this error in the hyperkitty log when mailman daemon tries to archive a mail. I've setup the hyperkitty.cfg file with api_key = MAILMAN_ARCHIVER_KEY in settings_local.py but still nothing. I've seen here and in GitLab issues some errors concerning a wrong api key but here it says that it wasn't sent at all. All packages installed via pip install + latest .zip release, including mailman-hyperkitty. Currently running on apache with mod_wsgi_py3 without https.
The following combinations should work:
mailman-hyperkitty 1.2.0 with HyperKitty 1.3.5
mailman-hyperkitty <1.2.0 with HyperKitty <1.3.5
The following will not work:
mailman-hyperkitty 1.2.0 with HyperKitty <1.3.5
mailman-hyperkitty <1.2.0 with HyperKitty 1.3.5
Do you see a message like 'The MAILMAN_ARCHIVER_KEY is now required to be sent over the Authorization HTTP header, you need to upgrade the mailman-hyperkitty package to 1.2.0 or newer.'
In your Django log and/or your mailman.log? These will only be logged with HyperKitty 1.3.5 and mailman-hyperkitty < 1.2.0
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Dear Mark,
had # pip freeze ... HyperKitty==1.3.5 ... mailman-hyperkitty==1.2.0 ...
and got the described behaviour. Downgraded to
HyperKitty @ https://gitlab.com/mailman/hyperkitty/-/archive/1.3.4/hyperkitty-1.3.4.zip mailman-hyperkitty @ https://gitlab.com/mailman/mailman-hyperkitty/-/archive/v1.1.0/mailman-hyper...
and now archiving works.
Best, Marco
On 10/22/21 12:53 PM, Marco Marinello wrote:
Dear Mark,
had # pip freeze ... HyperKitty==1.3.5 ... mailman-hyperkitty==1.2.0 ...
and got the described behaviour. Downgraded to
HyperKitty @ https://gitlab.com/mailman/hyperkitty/-/archive/1.3.4/hyperkitty-1.3.4.zip mailman-hyperkitty @ https://gitlab.com/mailman/mailman-hyperkitty/-/archive/v1.1.0/mailman-hyper...
and now archiving works.
Actually, with HyperKitty==1.3.5 and mailman-hyperkitty <1.2.0, the message is not exactly as in this message's Subject:. It is 'The MAILMAN_ARCHIVER_KEY is now required to be sent over the Authorization HTTP header, you need to upgrade the mailman-hyperkitty package to 1.2.0 or newer.
Perhaps you paraphrased the message. If not, that exact message was not in any version of HyperKitty leading up to 1.3.5.
And with mailman-hyperkitty==1.2.0, it should work.
I see what pip freeze reported, but something was wrong somewhere because that combination should work and the 'The MAILMAN_ARCHIVER_KEY ...' message would only be logged with a version of mailman-hyperkitty <1.2.0.
Perhaps you did not realize you had to restart Mailman core and skipped that step. The mailman-hyperkitty module is imported by Mailman core, thus core needs to be restarted if it is changed.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Dear Mark,
configuration needed much troubleshooting so I'm sure I always restarted mailman core along apache2 but since it's still a testing environment, I took the liberty to test it again for you.
# pip install https://gitlab.com/mailman/hyperkitty/-/archive/1.3.5/hyperkitty-1.3.5.zip # pip install https://gitlab.com/mailman/mailman-hyperkitty/-/archive/1.2.0/mailman-hyperk... # systemctl restart mailman apache2 # tail -f /var/log/mailman/mailman.log /opt/hyperkitty_django/hyperkitty.log ==> /opt/hyperkitty_django/hyperkitty.log <== ERROR 2021-10-22 23:03:19,225 25973 hyperkitty.views.mailman The MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. WARNING 2021-10-22 23:03:19,226 25973 django.request Unauthorized: /hyperkitty/api/mailman/archive WARNING 2021-10-22 23:03:19,226 25973 django.request Unauthorized: /hyperkitty/api/mailman/archive
==> /var/log/mailman/mailman.log <== Oct 22 21:03:19 2021 (26091) HyperKitty failure on http://127.0.0.1:8080/hyperkitty/api/mailman/archive: <html><title>Auth required</title><body> <h1>Authorization Required</h1><p>The archiver key must be sent over the Authorization HTTP header. </p></body></html> (401) Oct 22 21:03:19 2021 (26091) Exception in the HyperKitty archiver: <html><title>Auth required</title><body> <h1>Authorization Required</h1><p>The archiver key must be sent over the Authorization HTTP header. </p></body></html> Oct 22 21:03:19 2021 (26091) Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.9/site-packages/mailman_hyperkitty/__init__.py", line 151, in _archive_message url = self._send_message(mlist, msg) File "/opt/mailman/venv/lib/python3.9/site-packages/mailman_hyperkitty/__init__.py", line 207, in _send_message raise ValueError(result.text) ValueError: <html><title>Auth required</title><body> <h1>Authorization Required</h1><p>The archiver key must be sent over the Authorization HTTP header. </p></body></html> # pip freeze | grep yperk HyperKitty @ https://gitlab.com/mailman/hyperkitty/-/archive/1.3.5/hyperkitty-1.3.5.zip mailman-hyperkitty @ https://gitlab.com/mailman/mailman-hyperkitty/-/archive/1.2.0/mailman-hyperk...
All the best, Marco
On 10/22/21 2:10 PM, Marco Marinello wrote:
Dear Mark,
configuration needed much troubleshooting so I'm sure I always restarted mailman core along apache2 but since it's still a testing environment, I took the liberty to test it again for you.
# pip install https://gitlab.com/mailman/hyperkitty/-/archive/1.3.5/hyperkitty-1.3.5.zip # pip install https://gitlab.com/mailman/mailman-hyperkitty/-/archive/1.2.0/mailman-hyperk...
It appears that https://gitlab.com/mailman/mailman-hyperkitty/-/archive/1.2.0/mailman-hyperk... is the latest mailman-hyperkitty, but https://gitlab.com/mailman/hyperkitty/-/archive/1.3.5/hyperkitty-1.3.5.zip is missing some commits. It has
https://gitlab.com/mailman/hyperkitty/-/commit/79b22a23d3390db741a8233fa1ff6...
but is missing
https://gitlab.com/mailman/hyperkitty/-/commit/ec9c8ed18798cf8f7e89dfaba0014...
which is very strange because it does seem to have
https://gitlab.com/mailman/hyperkitty/-/commit/705bdbb2fb27be60d17ada7cba848...
I Suggest you do
pip install -U HyperKitty pip install -U mailman-hyperkitty
which will get the latest packages from PyPI or if you want the heads from gitlab,
pip install git+https://gitlab.com/mailman/HyperKitty.git pip install git+https://gitlab.com/mailman/mailman-hyperkitty.git
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Dear Mark,
I upgraded the pypi packages as you suggested but the problem remains the same. Tried also installing from git heads but always get
==> /opt/hyperkitty_django/hyperkitty.log <== ERROR 2021-10-23 09:42:42,523 30765 hyperkitty.views.mailman The MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. WARNING 2021-10-23 09:42:42,523 30765 django.request Unauthorized: /hyperkitty/api/mailman/urls WARNING 2021-10-23 09:42:42,523 30765 django.request Unauthorized: /hyperkitty/api/mailman/urls
==> /var/log/mailman/mailman.log <== Oct 23 07:42:42 2021 (30888) HyperKitty failure on http://127.0.0.1:8080/hyperkitty/api/mailman/urls: <html><title>Auth required</title><body> <h1>Authorization Required</h1><p>The archiver key must be sent over the Authorization HTTP header. </p></body></html> (401)
For information, how does your release cycle work? I prefer installing from a source rather than pypi. Is the master normally stable? When a packages lands on pypi? I'll stick on 1.3.4 / 1.1.0 for the moment.
All the best, Marco
On 10/23/21 12:43 AM, Marco Marinello wrote:
Dear Mark,
I upgraded the pypi packages as you suggested but the problem remains the same. Tried also installing from git heads but always get
==> /opt/hyperkitty_django/hyperkitty.log <== ERROR 2021-10-23 09:42:42,523 30765 hyperkitty.views.mailman The MAILMAN_ARCHIVER_KEY was not sent as the Authorization HTTP header. WARNING 2021-10-23 09:42:42,523 30765 django.request Unauthorized: /hyperkitty/api/mailman/urls WARNING 2021-10-23 09:42:42,523 30765 django.request Unauthorized: /hyperkitty/api/mailman/urls
==> /var/log/mailman/mailman.log <== Oct 23 07:42:42 2021 (30888) HyperKitty failure on http://127.0.0.1:8080/hyperkitty/api/mailman/urls: <html><title>Auth required</title><body> <h1>Authorization Required</h1><p>The archiver key must be sent over the Authorization HTTP header. </p></body></html> (401)
Those messages look correct for HyperKitty 1.3.5, if mailman-hyperkitty is not sending the key as expected. The particular condition causing those messages is there is no "Authorization" header in the headers of the request from mailman-hyperkitty and there is also no "key" parameter in the request as there would have been with mailman-hyperkitty 1.1.0
This would seem to say that both HyperKitty 1.3.5 and mailman-hyperkitty 1.2.0 are doing the right thing, but somehow the request headers are not getting through in the http request from mailman-hyperkitty to HyperKitty.
What is your setting for base_url in mailman-hyperkitty.cfg? Is there some redirect or other situation going on in serving the request to HyperKitty that could cause headers to get lost?
For information, how does your release cycle work? I prefer installing from a source rather than pypi. Is the master normally stable? When a packages lands on pypi?
We are a small, all volunteer project and we don't have a fixed release schedule. We make releases when we feel enough change has been accumulated to warrant on or when there is a serious issue to fix.
The heads of the gitlab branches are generally usable. There was actually a situation briefly where the mailman-hyperkitty changes for this issue were merged before the HyperKitty changes. See https://gitlab.com/mailman/hyperkitty/-/merge_requests/354#note_693162778 but situations like this are rare. The servers at lists.mailman3.org and mail.python.org run the heads of the gitlab branches (I update them regularly) so you can generally be confident that they work.
I'll stick on 1.3.4 / 1.1.0 for the moment.
There are several changes in HyperKitty 1.3.5. See https://docs.mailman3.org/projects/hyperkitty/en/latest/news.html#news-1-3-5
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Finally have found my issue... Apache automatically removes the header, see https://stackoverflow.com/questions/26549250/apache-strips-down-authorizatio... . Hope it can be useful for somebody else!
Thank you for all the informations shared with me, Mark. Hope to hear from you soon! And kudos to you and the group of volunteers.
On 10/25/21 5:18 AM, Marco Marinello wrote:
Finally have found my issue... Apache automatically removes the header, see https://stackoverflow.com/questions/26549250/apache-strips-down-authorizatio... . Hope it can be useful for somebody else!
I'm glad you found this, but I'm a bit confused. I run a few servers with apache 2.4 proxying to gunicorn and I don't see the issue. Perhaps this is a uWSGI or mod_wsgi issue.
In any case, it can possibly be avoided by setting base_url in mailman-hyperkitty.cfg to something like
base_url: https://127.0.0.1:8000/archives
or whatever localhost port the WSGI application is listening on to bypass apache.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (2)
-
Marco Marinello -
Mark Sapiro