New Mailman 3 installation issues
I have been trying to get a mailman 3 installation working for longer than I'd like to admit. Core appears to be installed and working as desired but Postorius (and hyperkitty, but I haven't even started troubleshooting it while trying to get the web interface working) is only loading the main page with no lists shown and all remaining links return a server error (specific text is: "Server error An error occurred while processing your request." with the error text highlighted). I think I followed all of the installation instructions, including the updated pythonpath line in the systemd module, but it's entirely possible that there is an error somewhere along the way.
I have had to make two minor tweaks, the /static/ location was moved from /opt/mailman/web/static to /var/www/mailman/ (owner mailman:nginx) as nginx was giving permissions errors (even if it owned the files or with the mod set to 777) with the default location and wouldn't load the css, and installed postgres directly from their repo as the method in the guide doesn't create the user. Aside from that everything should be exactly as laid out in the installation guide.
The logs are showing everything running and the DB logs show all the queries going through successfully, there are no errors in the web logs corresponding with the errors being served on the site aside an entry for a 500 in the access logs
[18/May/2021:12:13:01] "GET /archives/ HTTP/1.1" 500 1102 "http://mailman3/mailman3/lists/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.91 Safari/537.36" "-"
when I took a look in the DB tables many of them appear to be empty, I have a few lists in core that were imported from 2.1 but no new lists created.
Mailman: 3.3.4 Postorius: 1.3.4 OS: Oracle Linux 8.3 Python: 3.6 postgres: 13.2
On 5/18/21 12:30 PM, sarah.hartt@ionica.ca wrote:
I have been trying to get a mailman 3 installation working for longer than I'd like to admit. Core appears to be installed and working as desired but Postorius (and hyperkitty, but I haven't even started troubleshooting it while trying to get the web interface working) is only loading the main page with no lists shown and all remaining links return a server error (specific text is: "Server error An error occurred while processing your request." with the error text highlighted). I think I followed all of the installation instructions, including the updated pythonpath line in the systemd module, but it's entirely possible that there is an error somewhere along the way.
Is there more information in logs? (nginx and django)
I have had to make two minor tweaks, the /static/ location was moved from /opt/mailman/web/static to /var/www/mailman/ (owner mailman:nginx) as nginx was giving permissions errors (even if it owned the files or with the mod set to 777) with the default location and wouldn't load the css, and installed postgres directly from their repo as the method in the guide doesn't create the user. Aside from that everything should be exactly as laid out in the installation guide.
The permissions is probably a SELinux or apparmor issue. Which installation guide are you following? I'm guessing https://docs.mailman3.org/en/latest/install/virtualenv.html.
Are you saying that
sudo apt install postgresql
doesn't create a postgres user? It should.
The logs are showing everything running and the DB logs show all the queries going through successfully, there are no errors in the web logs corresponding with the errors being served on the site aside an entry for a 500 in the access logs
[18/May/2021:12:13:01] "GET /archives/ HTTP/1.1" 500 1102 "http://mailman3/mailman3/lists/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.91 Safari/537.36" "-"
This is a GET of /archives/ which is HyperKitty. Django should log something for this with more detail.
when I took a look in the DB tables many of them appear to be empty, I have a few lists in core that were imported from 2.1 but no new lists created.
Empty tables per se is not necessarily a problem depending on which ones.
Mailman: 3.3.4 Postorius: 1.3.4 OS: Oracle Linux 8.3 Python: 3.6 postgres: 13.2
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark,
Yes, that is the installation guide that I used. The nginx logs are clear aside from what I have listed and the now resolved permission denied entries from prior to moving static assets.
I am getting new entries in the django logs.
/opt/mailman/web/logs/mailmanweb.log
ERROR 2021-05-18 20:08:38,893 3078109 django.request Internal Server Error: /accounts/login/ Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/handlers/base.py", line 145, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/core/handlers/base.py", line 143, in _get_response response = response.render() File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/response.py", line 105, in render self.content = self.rendered_content File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/response.py", line 83, in rendered_content return template.render(context, self._request) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/backends/django.py", line 61, in render return self.template.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 171, in render return self._render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 163, in _render return self.nodelist.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/loader_tags.py", line 150, in render return compiled_parent._render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 163, in _render return self.nodelist.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/loader_tags.py", line 150, in render return compiled_parent._render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 163, in _render return self.nodelist.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 936, in render bit = node.render_annotated(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/django/template/base.py", line 903, in render_annotated return self.render(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/compressor/templatetags/compress.py", line 143, in render return self.render_compressed(context, self.kind, self.mode, forced=forced) File "/opt/mailman/venv/lib64/python3.6/site-packages/compressor/templatetags/compress.py", line 97, in render_compressed return self.render_offline(context) File "/opt/mailman/venv/lib64/python3.6/site-packages/compressor/templatetags/compress.py", line 82, in render_offline 'is the original content:\n\n%s' % (key, original_content)) compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "fa788b08b65fd318d68cfc5afebc18acd3e93cebffadb54d863572ad1754f368" is missing from offline manifest. You may need to run "python manage.py compress". Here is the original content:
<link rel="stylesheet" href="/static/hyperkitty/libs/fonts/icomoon/icomoon.css" type="text/css" media="all" />
<link rel="stylesheet" href="/static/hyperkitty/libs/fonts/droid/droid.css" type="text/css" media="all" />
<link rel="stylesheet" href="/static/django-mailman3/css/main.css" />
<link rel="stylesheet" type="text/x-scss" media="all" href="/static/hyperkitty/sass/hyperkitty.scss" />SELinux is disabled and when installed via dnf in oracle 8 RHEL it was not creating the postgres user correctly so I used their repo directly to avoid having to do so manually. This was suggested by a much more experienced admin when she took a look at the guide after I encountered issues.
If there is anything else I can provide to help diagnose the issue please let me know. if there are specific tables that I can check to ensure that they are/are not being populated to rule out a DB related issue or anything else you would suggest I try I greatly appreciate any assistance.
Sarah
On 5/18/21 1:21 PM, sarah.hartt@ionica.ca wrote:
I am getting new entries in the django logs.
/opt/mailman/web/logs/mailmanweb.log
ERROR 2021-05-18 20:08:38,893 3078109 django.request Internal Server Error: /accounts/login/ Traceback (most recent call last):
...
File "/opt/mailman/venv/lib64/python3.6/site-packages/compressor/templatetags/compress.py", line 82, in render_offline 'is the original content:\n\n%s' % (key, original_content)) compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "fa788b08b65fd318d68cfc5afebc18acd3e93cebffadb54d863572ad1754f368" is missing from offline manifest. You may need to run "python manage.py compress".
have you run the commands:
mailman-web migrate
mailman-web collectstatic
mailman-web compress
The last of these is not in the doc - I'll fix that - and may be the issue.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I had run the first 2, but not the third. I'm getting proper page loads after running compress! I'm going to migrate all of my remaining lists to the new server and muck about but it looks like it's working now!
On 5/18/21 1:50 PM, Sarah H wrote:
I had run the first 2, but not the third. I'm getting proper page loads after running compress! I'm going to migrate all of my remaining lists to the new server and muck about but it looks like it's working now!
Good!
I have added the compress command (and also compilemessages) to the doc https://docs.mailman3.org/en/latest/install/virtualenv.html#compress-css-fil...
Sorry about the omission.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark,
I wanted to make another note on the documentation, the migration documents (https://docs.mailman3.org/en/latest/migration.html#fn2) reference using
'python manage.py hyperkitty_import'
it should be
'mailman-web hyperkitty_import'
this is the same for the index rebuild.
On 5/19/21 11:50 AM, Sarah H wrote:
Hi Mark,
I wanted to make another note on the documentation, the migration documents (https://docs.mailman3.org/en/latest/migration.html#fn2) reference using
'python manage.py hyperkitty_import'
it should be
'mailman-web hyperkitty_import'
That is a can of worms. It all depends on how you installed Mailman 3.
If you follow the doc at
https://docs.mailman3.org/en/latest/install/virtualenv.html, the django
management command will be mailman-web, but installation in other ways
can result in its being manage.py or django-admin or almost anything.
For example,
https://docs.mailman3.org/en/latest/pre-installation-guide.html#what-do-i-ne...
refers to a typical Django project in which the management command is
manage.py.
https://docs.mailman3.org/en/latest/config-web.html is inconsistent in
itself in that it uses mailman-web in most places except the note
about qcluster at
https://docs.mailman3.org/en/latest/config-web.html#running-the-task-queue
which also links to
http://hyperkitty.readthedocs.io/en/latest/install.html#asynchronous-tasks
which uses django-admin.
I'm thinking that we should put a note about this near the top of https://docs.mailman3.org/en/latest/ and let it go at that.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I think you've just hit the nail on the head as far as my biggest issue with this project. The documentation lays out an install method that should be roughly followable for anyone (though it needs a bit of fine tuning here and there) and then the rest of the configuration guide and documentation assumes that you have installed the software in a completely different way and doesn't work the same as the guide laid out.
The documentation should be uniform, if you want users to install it one way then write the guide and documentation so that's the "default". Sure present alternatives or notes "if you've installed via git, your commands will look like this xxx" but ultimately the biggest issue I've had with getting this deployed is inconsistent or lacking documentation.
On 5/19/21 1:26 PM, Sarah H wrote:
I think you've just hit the nail on the head as far as my biggest issue with this project. The documentation lays out an install method that should be roughly followable for anyone (though it needs a bit of fine tuning here and there) and then the rest of the configuration guide and documentation assumes that you have installed the software in a completely different way and doesn't work the same as the guide laid out.
Because virtually all that documentation was written by different people and long before the https://docs.mailman3.org/en/latest/install/virtualenv.html was added.
The documentation should be uniform, if you want users to install it one way then write the guide and documentation so that's the "default". Sure present alternatives or notes "if you've installed via git, your commands will look like this xxx" but ultimately the biggest issue I've had with getting this deployed is inconsistent or lacking documentation.
I'm not trying to make excuses. I know the docs need improvement, but we are a very small, all volunteer project doing the best we can. If you feel you can offer improvements to the documentation, we'd love to have your help. You are more than welcome to submit merge requests to update the docs.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Sorry, what I wrote may have been a bit overly negative, I'm grateful for the work you and everyone else has put into the project and documenting it and for the help you've provided here. It was directed solely at highlighting an issue that was encountered by an inexperienced admin, I've never worked with django previously or very extensively with python outside just following the directions that were laid before me and suggesting a method of making it easier for others in my shoes.
I'm not experienced enough that I could offer much as far as improvements go outside sharing feedback.
Sarah H writes:
I'm not experienced enough that I could offer much as far as improvements go outside sharing feedback.
Thank you for the report and the feedback.
Of course the advertised suggestion for "how to help" is a properly cleaned up merge request on GitLab. But the bare minimum is a "this doc <paste URL> says X but that doc <paste URL> assumes Y and it just didn't make sense" as a bug report on GitLab. Anything that makes it easy for us to find the problem text in the source really speeds up the process a lot.
Titles are important and worth effort to be clear and concise because they're the primary way we find issues of interest, but the content can be stream of consciousness to be untangled later. Issues on GitLab are far preferable to list reports because they're not mixed in with a ton of other stuff, and GitLab has tools the make triage easier.
Steve
participants (4)
-
Mark Sapiro -
Sarah H -
sarah.hartt＠ionica.ca
-
Stephen J. Turnbull