Still unable to get mailman3 working on Ubuntu 20.04 LTS
Hi all, after many more attempts, I've still failed to find any set of instructions that can install mailman3 on an Ubuntu 20.04 LTS server of mine. I've recently switched to a new server without any previous installation (apart from an apache web server) so that I can follow the instructions provided in
https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10
to the letter, if possible. But still, I run into an error I do not understand.
I have followed the instructions until this point:
# pip install importlib_resources==1.1.0 Once the above packages are installed run “mailman info” in your virtual environment. That will install Mailman core into your /opt/mailman/mm/var directory.
(I had to add "apt install libmemcached-dev" to the instruction for "pip install psycopg2-binary" to succeed.)
Then, I run "mailman info", but get an error message:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 508, in connect return self.dbapi.connect(*cargs, **cparams) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed > for user "mailman" connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed for user "mailman"
This suggests that the connection to postgresql fails, but if I test this using
# psql --user mailman
and type in the password listed in settings_local.py, I can login fine. I've also changed the port number in that configuration from '' to '2222', but the connection is still made to port 5432, which suggests to me that my call to
mailman info
uses another set of connection parameters provided elsewhere.
Where can this be found? How will I be able to run "mailman info" w/o error and continue the installation as detailed in https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 ?
Any help is appreciated.
Happy new year, Michael.
We've set up several Mailman 3 servers on Ubuntu 20.04 successfully. We've always followed the virtualenv instructions: https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
I realise those aren't the instructions you've been going through but I hope that at least confirmation that the virtualenv docs work is of some use to you.
Philip
On Sun, 2 Jan 2022 at 10:07, Michael Heseltine <heseltine.michael@gmail.com> wrote:
Hi all, after many more attempts, I've still failed to find any set of instructions that can install mailman3 on an Ubuntu 20.04 LTS server of mine. I've recently switched to a new server without any previous installation (apart from an apache web server) so that I can follow the instructions provided in
https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10
to the letter, if possible. But still, I run into an error I do not understand.
I have followed the instructions until this point:
# pip install importlib_resources==1.1.0 Once the above packages are installed run “mailman info” in your virtual environment. That will install Mailman core into your /opt/mailman/mm/var directory.
(I had to add "apt install libmemcached-dev" to the instruction for "pip install psycopg2-binary" to succeed.)
Then, I run "mailman info", but get an error message:
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 508, in connect return self.dbapi.connect(*cargs, **cparams) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed > for user "mailman" connection to server at "localhost" (::1), port 5432 failed: FATAL: password authentication failed for user "mailman"
This suggests that the connection to postgresql fails, but if I test this using
# psql --user mailman
and type in the password listed in settings_local.py, I can login fine. I've also changed the port number in that configuration from '' to '2222', but the connection is still made to port 5432, which suggests to me that my call to
mailman info
uses another set of connection parameters provided elsewhere.
Where can this be found? How will I be able to run "mailman info" w/o error and continue the installation as detailed in https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 ?
Any help is appreciated.
Happy new year, Michael.
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Michael Heseltine writes:
and type in the password listed in settings_local.py,
All of the settings*.py files are for Django, that is, for Postorius and HyperKitty.
Configuration of Mailman 3 core is mostly done in .cfg files, typically in /etc/mailman but I don't know about the instructions you're following. From docs/database.rst:
You would then need to set both the `class` and `url` variables in
`mailman.cfg` like so::
[database]
class: mailman.database.postgresql.PostgreSQLDatabase
url: postgres://myuser:mypassword@mypghost/mailmanThe "class" line is verbatim. In the "url" line the words beginning with "my" are dummies that you need to replace with the appropriate values.
Howto_Install_Mailman3_On_Debian10 provides a template for mailman.cfg which you may want to copy and customize at least the passwords)
Steve
Thanks Philip and Steven, with your indications, I've found out that the mailman user password was listed in the settings.py files, but not in the mailman.cfg. I copied the samples from the Howto site and customized it for my local system, but did miss that config item.
Now, get a proper reply (I think) from "mailman info"
GNU Mailman 3.3.5 (Tom Sawyer) Python 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0] config file: /opt/mailman/mm/mailman.cfg db url: postgres://mailman:<REDACTED>@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:<REDACTED>
and have created the necessary files under /etc/systemd/service/, but run into problems when trying to execute "/opt/mailman/mm/bin/mailman-post-update" (I've "chmod +x /opt/mailman/mm/bin/*" before that in order to be able to execute the script files):
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0 Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in > execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in > execute django.setup() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate app_config = AppConfig.create(entry) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/apps/config.py", line 224, in create import_module(entry) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 848, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/mm/venv/lib/python3.8/site-packages/haystack/__init__.py", line 43, in <module> raise ImproperlyConfigured("The HAYSTACK_CONNECTIONS setting is required.") django.core.exceptions.ImproperlyConfigured: The HAYSTACK_CONNECTIONS setting is required.
I have no idea whatsoever what this HAYSTACK_CONNECTIONS thing is, nor have I found anything in the HowTo.
Can you help me any further?
Cheers, Michael.
Hi Philip, Stephen, anyone, I've read a little further, and from what I gather this HAYSTACK play some part in providing a full text search for mailing list messages. But I cannot find any indication whatsoever how to set it. As I understand it, HAYSTACK_CENNECTIONS is a variable to be set in settings.py, but no matter what I set there be that (from https://django-haystack.readthedocs.io/en/master/settings.html)
HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack.backends.solr_backend.SolrEngine', 'URL': 'http://localhost:9001/solr/default', 'TIMEOUT': 60 * 5, 'INCLUDE_SPELLING': True, 'BATCH_SIZE': 100, 'EXCLUDED_INDEXES': ['thirdpartyapp.search_indexes.BarIndex'], }, 'autocomplete': { 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': '/opt/mailman/mm/whoosh_index', 'STORAGE': 'file', 'POST_LIMIT': 128 * 1024 * 1024, 'INCLUDE_SPELLING': True, 'BATCH_SIZE': 100, 'EXCLUDED_INDEXES': ['thirdpartyapp.search_indexes.BarIndex'], }, 'slave': { 'ENGINE': 'xapian_backend.XapianEngine', 'PATH': '/opt/mailman/mm/xapian_index', 'INCLUDE_SPELLING': True, 'BATCH_SIZE': 100, 'EXCLUDED_INDEXES': ['thirdpartyapp.search_indexes.BarIndex'], }, 'db': { 'ENGINE': 'haystack.backends.simple_backend.SimpleEngine', 'EXCLUDED_INDEXES': ['thirdpartyapp.search_indexes.BarIndex'], } }
(which seems wrong to me because the mailman3 HowTo makes no mention of anything else than whoosh)
or just the default from https://docs.mailman3.org/projects/mailman-web/en/latest/settings.html :
{'default': {'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', 'PATH': 'fulltext_index'}}
I seem to get the same error message:
# /opt/mailman/mm/bin/mailman-post-update
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0 Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute django.setup() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate app_config.import_models() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models self.models_module = import_module(models_module_name) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 848, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/models.py", line 1, in <module> from compressor.conf import CompressorConf # noqa File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/conf.py", line 9, in <module> class CompressorConf(AppConf): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/appconf/base.py", line 73, in __new__ new_class._configure() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/appconf/base.py", line 102, in _configure value = callback(value) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/conf.py", line 115, in configure_url if not value.endswith('/'): AttributeError: 'NoneType' object has no attribute 'endswith'
which looks to me like a parsing error (though I don't know which error, nor which file is being parsed).
What am I doing wrong here? Is there any default or sample setting for HAYSTACK_CONNECTIONS that I can use? Or else, can I disable haystack/whoosh altogether? I really just want to get two small mailing lists up and running on this site, with no bells and whistles whatsoever. But these error messages wouldn't be any less clearer to me if they were written in Mandarin. :-(
Cheers, Michael.
On 1/2/22 4:03 AM, Michael Heseltine wrote:
I have no idea whatsoever what this HAYSTACK_CONNECTIONS thing is, nor have I found anything in the HowTo.
The HowTo you are following under Setup Directories and Files for Mailman 3 includes
# touch /opt/mailman/mm/settings.py (copy of the mailman-suite settings.py file but with DEBUG = False)
While I would have written this section differently to emphasize you
actually need to copy the content and not just touch (i.e. create) the
file, mailman-suite settings.py is a link to
https://gitlab.com/mailman/mailman-suite/blob/master/mailman-suite_project/s...
and that in turn contains:
#
# Full-text search engine
#
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
'PATH': os.path.join(BASE_DIR, "fulltext_index"),
# You can also use the Xapian engine, it's faster and more
accurate,
# but requires another library.
#
http://django-haystack.readthedocs.io/en/v2.4.1/installing_search_engines.html#xapian
# Example configuration for Xapian:
#'ENGINE': 'xapian_backend.XapianEngine'
},
}
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, I was confused by the wording of that segment and ended up using another file as template for the settings.py. I have now copied the Gitlab settings.py to my installation, just changed the "DEBUG = True" to "DEBUG = False", and nothing else.
But that did not change anything, I still get the following error message when running "/opt/mailman/mm/bin/mailman-post-update" in the venv:
~# /opt/mailman/mm/bin/mailman-post-update
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0 Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute django.setup() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate app_config.import_models() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models self.models_module = import_module(models_module_name) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 848, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/models.py", line 1, in <module> from compressor.conf import CompressorConf # noqa File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/conf.py", line 9, in <module> class CompressorConf(AppConf): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/appconf/base.py", line 73, in __new__ new_class._configure() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/appconf/base.py", line 102, in _configure value = callback(value) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/conf.py", line 115, in configure_url if not value.endswith('/'): AttributeError: 'NoneType' object has no attribute 'endswith'
Do I need to repeat any steps in between the creation of the settings.py file and the running of "/opt/mailman/mm/bin/mailman-post-update"?
Cheers, Michael.
On 1/3/22 2:44 AM, Michael Heseltine wrote:
But that did not change anything, I still get the following error message when running "/opt/mailman/mm/bin/mailman-post-update" in the venv:
...
File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/conf.py", line 115, in configure_url if not value.endswith('/'): AttributeError: 'NoneType' object has no attribute 'endswith'
That code is
def configure_url(self, value):
# Uses Django's STATIC_URL by default
if value is None:
value = settings.STATIC_URL
if not value.endswith('/'):
raise ImproperlyConfigured("URL settings (e.g. COMPRESS_URL) "
"must have a trailing slash")
return value
which is basically saying you have no setting for STATIC_URL, but https://gitlab.com/mailman/mailman-suite/blob/master/mailman-suite_project/s... sets it, so I don't know what the issue is.
What does
grep STATIC /opt/mailman/mm/settings*
show?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thanks Mark, I made an error this morning and copied over the settings_local.py (instead of settings.py) file; after correcting this mistake, I now get the > following error message from "mailman-post-update":
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0
- /opt/mailman/mm/bin/django-admin compress Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/utils.py", line 66, in __getitem__ return self._engines[alias] KeyError: 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 121, in get_package_libraries module = import_module(entry[1]) File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 848, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/templatetags/decorate.py", line 4, in <module> from hyperkitty.lib.renderer import markdown_renderer, text_renderer File "/opt/mailman/mm/venv/lib/python3.8/site-packages/hyperkitty/lib/renderer.py", line 7, in <module> from mistune.scanner import escape_html, escape_url ImportError: cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/mm/venv/lib/python3.8/site-packages/mistune/scanner.py)
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/management/commands/compress.py", line 296, in handle self.handle_inner(**options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/management/commands/compress.py", line > 319, in handle_inner offline_manifest, block_count, results = self.compress(engine, extensions, verbosity, follow_links, log) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/management/commands/compress.py", line > 101, in compress if not self.get_loaders(): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/compressor/management/commands/compress.py", line > 51, in get_loaders for e in engines.all(): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/utils.py", line 90, in all return [self[alias] for alias in self] File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/utils.py", line 90, in <listcomp> return [self[alias] for alias in self] File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/utils.py", line 81, in __getitem__ engine = engine_cls(params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 25, in __init__ options['libraries'] = self.get_templatetag_libraries(libraries) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 43, in get_templatetag_libraries libraries = get_installed_libraries() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 108, in get_installed_libraries for name in get_package_libraries(pkg): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/template/backends/django.py", line 123, in get_package_libraries raise InvalidTemplateLibrary( django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/mm/venv/lib/python3.8/site-packages/mistune/scanner.py)
What does that mean?
Cheers, Michael.
On 1/3/22 10:10 AM, Michael Heseltine wrote:
raise InvalidTemplateLibrary(django.template.library.InvalidTemplateLibrary: Invalid template library specified. ImportError raised when trying to load 'hyperkitty.templatetags.decorate': cannot import name 'escape_html' from 'mistune.scanner' (/opt/mailman/mm/venv/lib/python3.8/site-packages/mistune/scanner.py)
What does that mean?
See https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/5...
You need to install https://gitlab.com/mailman/hyperkitty/-/merge_requests/379 or downgrade mistune in your venv with
pip install mistune==2.0.0rc1
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, I tried out "pip install mistune==2.0.0rc1" (as I got hyperkitty via pip) which worked and let me move forward a little further once more, but still I get an error from "mailman-post-update":
# /opt/mailman/mm/bin/mailman-post-update
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0
- /opt/mailman/mm/bin/django-admin compress Compressing... done Compressed 2 block(s) from 104 template(s) for 1 context(s).
- /opt/mailman/mm/bin/django-admin compilemessages Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/commands/compilemessages.py", line 113, in handle self.compile_messages(locations) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/commands/compilemessages.py", line 130, in compile_messages self.stdout.write( File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 155, in write self._out.write(style_func(msg)) UnicodeEncodeError: 'latin-1' codec can't encode character '\u201c' in position 5: ordinal not in range(256)
Cheers, Michael.
On 1/3/22 4:01 PM, Michael Heseltine wrote:
Hi Mark, I tried out "pip install mistune==2.0.0rc1" (as I got hyperkitty via pip) which worked and let me move forward a little further once more, but still I get an error from "mailman-post-update":
# /opt/mailman/mm/bin/mailman-post-update
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0
- /opt/mailman/mm/bin/django-admin compress Compressing... done Compressed 2 block(s) from 104 template(s) for 1 context(s).
- /opt/mailman/mm/bin/django-admin compilemessages Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/commands/compilemessages.py", line 113, in handle self.compile_messages(locations) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/commands/compilemessages.py", line 130, in compile_messages self.stdout.write( File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 155, in write self._out.write(style_func(msg)) UnicodeEncodeError: 'latin-1' codec can't encode character '\u201c' in position 5: ordinal not in range(256)
This looks like an issue with some message catalog which is declared as
character set latin-1 but contains a unicode left_double_quote character.
Unfortunately, django-admin compilemessages compiles, at least in my
test installation, 1583 message catalogs from 19 different projects, so
finding the problem file is difficult and I don't know how to narrow it
down.
However, this is the next to last step in mailman-post-update and isn't important if your installation is English language only and the last step is optional and runs
/opt/mailman/mm/bin/django-admin update_index
to rebuild the search index for HyperKitty and you can just run that one step if you want. At this point, you probably have no archives so this should take almost no time.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, sadly, skipping this step does not work either, as I then get the following error message from "/opt/mailman/mm/bin/django-admin update_index":
# /opt/mailman/mm/bin/django-admin update_index [ERROR/MainProcess] Error updating hyperkitty using default Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) psycopg2.errors.UndefinedTable: relation "hyperkitty_email" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "hyperkitty_email" ^
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/haystack/management/commands/update_index.py", line 297, in handle self.update_backend(label, using) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/haystack/management/commands/update_index.py", line 324, in update_backend total = qs.count() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/query.py", line 412, in count return self.query.get_count(using=self.db) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/sql/query.py", line 519, in get_count number = obj.get_aggregation(using, ['__count'])['__count'] File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/sql/query.py", line 504, in get_aggregation result = compiler.execute_sql(SINGLE) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql cursor.execute(sql, params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 98, in execute return super().execute(sql, params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) django.db.utils.ProgrammingError: relation "hyperkitty_email" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "hyperkitty_email" ^
Traceback (most recent call last): File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) psycopg2.errors.UndefinedTable: relation "hyperkitty_email" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "hyperkitty_email" ^
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 8, in <module> sys.exit(execute_from_command_line()) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/haystack/management/commands/update_index.py", line 297, in handle self.update_backend(label, using) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/haystack/management/commands/update_index.py", line 324, in update_backend total = qs.count() File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/query.py", line 412, in count return self.query.get_count(using=self.db) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/sql/query.py", line 519, in get_count number = obj.get_aggregation(using, ['__count'])['__count'] File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/sql/query.py", line 504, in get_aggregation result = compiler.execute_sql(SINGLE) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql cursor.execute(sql, params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 98, in execute return super().execute(sql, params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/opt/mailman/mm/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) django.db.utils.ProgrammingError: relation "hyperkitty_email" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "hyperkitty_email"
Cheers, Michael.
On 1/3/22 11:31 PM, Michael Heseltine wrote:
Hi Mark, sadly, skipping this step does not work either, as I then get the following error message from "/opt/mailman/mm/bin/django-admin update_index": ...
django.db.utils.ProgrammingError: relation "hyperkitty_email" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "hyperkitty_email"
That's because you have no archives yet. Just ignore that too.
You are now ready to create lists.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Actually, I misspoke. There is one more step from mailman-post-update that is important. You need to run
/opt/mailman/mm/bin/django-admin migrate
to run all the django migrations. This may also avoid the
django.db.utils.ProgrammingError: relation "hyperkitty_email" does not exist
LINE 1: SELECT COUNT(*) AS "__count" FROM "hyperkitty_email"
error.
Hi all, I've done the additional steps
/opt/mailman/mm/bin/django-admin migrate /opt/mailman/mm/bin/django-admin createsuperuser
without error, and also started the services mailman, qcluster and gunicorn without error messages.
However, I just get an internal server error when I try to access https://lists.mydomain.com/ (which is served by a local apache virthost).
The apache log shows just one error:
[Thu Jan 06 10:01:39.201833 2022] [proxy:warn] [pid 3627279] [client 333.333.333.333:59328] AH01144: No protocol handler was valid for the URL / (scheme 'http'). If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.
The virtual host is configured like this:
| <VirtualHost lists.mydomain.com:443> | | ServerName lists.mydomain.com | Include /etc/letsencrypt/options-ssl-apache.conf | SSLCertificateFile /etc/letsencrypt/live/myhost.mydomain.com/fullchain.pem | SSLCertificateKeyFile /etc/letsencrypt/live/myhost.mydomain.com/privkey.pem | ServerAdmin mailman@lists.mydomain.com | Alias /static /opt/mailman/mm/web/static | Alias /favicon.ico /opt/mailman/mm/web/static/hyperkitty/img/favicon.ico | ProxyPassMatch ^/mm/static/ ! | ProxyPass / http://localhost:8000/ | ProxyPassReverse / http://localhost:8000/ | | <Directory /> | Allow from all | Require all granted | </Directory> | | <Directory /opt/mailman/mm/web/static/hyperkitty/img/> | Allow from all | Require all granted | Options FollowSymLinks | </Directory> | | <Directory /opt/mailman/mm/web/static/> | Options FollowSymLinks | Allow from all | Require all granted | </Directory> | | </VirtualHost>
Can anybody tell me why this doesn't work?
Cheers, Michael.
On 1/6/22 1:20 AM, Michael Heseltine wrote:
Hi all, I've done the additional steps
/opt/mailman/mm/bin/django-admin migrate /opt/mailman/mm/bin/django-admin createsuperuser
without error, and also started the services mailman, qcluster and gunicorn without error messages.
Did you see my post at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/... and did you run
/opt/mailman/mm/bin/django-admin migrate
before starting services?
However, I just get an internal server error when I try to access https://lists.mydomain.com/ (which is served by a local apache virthost).
The apache log shows just one error:
[Thu Jan 06 10:01:39.201833 2022] [proxy:warn] [pid 3627279] [client 333.333.333.333:59328] AH01144: No protocol handler was valid for the URL / (scheme 'http'). If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.
Do you have all of these apache modules enabled, proxy.conf, proxy_http.load and proxy.load, in particular proxy_http.load?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Did you see my post at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/... and did you run
/opt/mailman/mm/bin/django-admin migrate
before starting services?
As mentioned above: yes, I did; and yes, I did it before starting the services.
Do you have all of these apache modules enabled, proxy.conf, proxy_http.load and proxy.load, in particular proxy_http.load?
Well, mostly:
# a2enmod proxy_http.load Considering dependency proxy for proxy_http: Module proxy already enabled Module proxy_http already enabled # a2enmod proxy.load Module proxy already enabled # a2enmod proxy.conf ERROR: Module proxy.conf does not exist!
Did a "systemctl restart apache2", but nothing has changed - still get an internal Server error.
Btw, I the ProxyPass(Reverse) supposed to point at the URL listed in MAILMAN_REST_API_URL? Because that one is set to port 8001 in settings.py, and if I try to "telnet localhost 8000", i get "Connection refused".
Cheers, Michael
On 1/6/22 10:21 AM, Michael Heseltine wrote:
Btw, I the ProxyPass(Reverse) supposed to point at the URL listed in MAILMAN_REST_API_URL? Because that one is set to port 8001 in settings.py, and if I try to "telnet localhost 8000", i get "Connection refused".
No. You probably don't need ProxyPassReverse at all, but it doesn't hurt and it should be the same URL as in ProxyPass.
Port 8001 is Mailman core's REST server and that is what's in MAILMAN_REST_API_URL. Port 8000 is your wsgi server. The HowTo you're following tells you to install and configure gunicorn and it should be running and listening on port 8000 as it's what receives the proxied requests from Apache.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Michael Heseltine writes:
Do you have all of these apache modules enabled, proxy.conf, proxy_http.load and proxy.load, in particular proxy_http.load?
Well, mostly:
# a2enmod proxy_http.load Considering dependency proxy for proxy_http: Module proxy already enabled Module proxy_http already enabled # a2enmod proxy.load Module proxy already enabled # a2enmod proxy.conf ERROR: Module proxy.conf does not exist!
In my Debian configuration that module is needed because it contains the necessary directives to enable proxying. Do you have a "ProxyRequests On" and appropriate <Proxy> blocks elsewhere in your configuration? Is it possible that you have a location control in the <Proxy> block that blocks http proxying for Mailman's URL? (I agree that this doesn't make sense given the "protocol not found" error, but I can't see how this is a Mailman- side problem, either.) Is it possible you have an ftp or other non-http proxy and shut off the http proxy for security's sake?
Is it possible that the mod_proxy_http.load file is corrupt (or empty)? The a2*mod scripts do a thorough job of examining the file system, but don't check runtime functionality as far as I can see (but the last time I needed Perl I let Jeff Friedl do it for me, a loong time ago :-).
The bottom line, though, is that that error message has to do with the internal configuration of Apache, and as far as I can see it has nothing to do with Mailman itself.
Steve
On 1/6/22 6:28 PM, Stephen J. Turnbull wrote:
Michael Heseltine writes:
Do you have all of these apache modules enabled, proxy.conf, proxy_http.load and proxy.load, in particular proxy_http.load?
Well, mostly:
# a2enmod proxy_http.load Considering dependency proxy for proxy_http: Module proxy already enabled Module proxy_http already enabled # a2enmod proxy.load Module proxy already enabled # a2enmod proxy.conf ERROR: Module proxy.conf does not exist!
In my Debian configuration that module is needed because it contains the necessary directives to enable proxying.
Actually though, I think that's enabled as part of proxy.
The bottom line, though, is that that error message has to do with the internal configuration of Apache, and as far as I can see it has nothing to do with Mailman itself.
However, see https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
I now suspect the real issue is there's no wsgi server listening on port 8000.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mark Sapiro writes:
On 1/6/22 6:28 PM, Stephen J. Turnbull wrote:
In my Debian configuration [mod_proxy_http.conf] is needed because it contains the necessary directives to enable proxying.
Actually though, I think that's enabled as part of proxy.
The Apache documentation strongly implies that this needs to be explicitly enabled by the admin:
The forward proxy is activated using the ProxyRequests
directive. Because forward proxies allow clients to access
arbitrary sites through your server and to hide their true origin,
it is essential that you secure your server so that only
authorized clients can access the proxy before activating a
forward proxy.By contrast for a reverse proxy:
A reverse proxy is activated using the ProxyPass directive or the
[P] flag to the RewriteRule directive. It is not necessary to turn
ProxyRequests on in order to configure a reverse proxy.Both are from https://httpd.apache.org/docs/2.4/mod/mod_proxy.html.
However, see https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
I now suspect the real issue is there's no wsgi server listening on port 8000.
Could be. Apache docs notwithstanding, I'm happy to trust your intuition, and that would make things simple for the OP. :-)
Steve
Hi Mark, Stephen, in fact, there isn't anything running on port 8000, even though I added the service description files under "/lib/systemd/system" as detailed in the HowTo.
Looking at "journalctl -xe", both the "gunicorn" and the "qcluster" service run into errors i don't understand:
(qcluster)
Jan 08 21:22:23 lists.mydomain.com systemd[1]: qcluster.service: Start request repeated too quickly. Jan 08 21:22:23 lists.mydomain.com systemd[1]: qcluster.service: Failed with result 'exit-code'. -- Subject: Unit failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- The unit qcluster.service has entered the 'failed' state with result 'exit-code'. Jan 08 21:22:23 lists.mydomain.com systemd[1]: Failed to start HyperKitty async tasks runner. -- Subject: A start job for unit qcluster.service has failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- A start job for unit qcluster.service has finished with a failure.
-- The job identifier is 734836 and the job result is failed.
(gunicorn)
Jan 08 21:24:58 lists.mydomain.com systemd[1]: gunicorn.service: Failed with result 'exit-code'. -- Subject: Unit failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- The unit gunicorn.service has entered the 'failed' state with result 'exit-code'. ...skipping... Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/bin/gunicorn", line 14, in> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: sys.exit(gunicorn.app.wsgiapp.run()) Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: WSGIApplication("%(prog)s [OPTIONS] [APP_MODUL> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: super().run() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: Arbiter(self).run() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: self.halt(reason=inst.reason, exit_status=inst> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: self.stop() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: time.sleep(0.1) Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: self.reap_workers() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: raise HaltServer(reason, self.WORKER_BOOT_ERRO> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: gunicorn.errors.HaltServer: <HaltServer 'Worker fa> Jan 08 21:24:58 lists.mydomain.com systemd[1]: gunicorn.service: Main process exited, code=exited, statu> -- Subject: Unit process exited -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- An ExecStart= process belonging to unit gunicorn.service has exited.
-- The process' exit code is 'exited' and its exit status is 1.
the qcluster.service and guncorn.service are 1:1 copies of the sample files provided in the base howto.
Do you know what is the problem now?
Cheers, Michael.
On 1/8/22 12:31 PM, Michael Heseltine wrote:
Hi Mark, Stephen, in fact, there isn't anything running on port 8000, even though I added the service description files under "/lib/systemd/system" as detailed in the HowTo.
Looking at "journalctl -xe", both the "gunicorn" and the "qcluster" service run into errors i don't understand:
(qcluster)
Jan 08 21:22:23 lists.mydomain.com systemd[1]: qcluster.service: Start request repeated too quickly. Jan 08 21:22:23 lists.mydomain.com systemd[1]: qcluster.service: Failed with result 'exit-code'. -- Subject: Unit failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- The unit qcluster.service has entered the 'failed' state with result 'exit-code'. Jan 08 21:22:23 lists.mydomain.com systemd[1]: Failed to start HyperKitty async tasks runner. -- Subject: A start job for unit qcluster.service has failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- A start job for unit qcluster.service has finished with a failure.
-- The job identifier is 734836 and the job result is failed.
This isn't your immediate problem, but it is an issue. The above isn't too informative. What do you get if you run
/opt/mailman/mm/bin/django-admin qcluster
manually?
(gunicorn)
Jan 08 21:24:58 lists.mydomain.com systemd[1]: gunicorn.service: Failed with result 'exit-code'. -- Subject: Unit failed -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- The unit gunicorn.service has entered the 'failed' state with result 'exit-code'. ...skipping... Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/bin/gunicorn", line 14, in> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: sys.exit(gunicorn.app.wsgiapp.run()) Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: WSGIApplication("%(prog)s [OPTIONS] [APP_MODUL> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: super().run() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: Arbiter(self).run() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: self.halt(reason=inst.reason, exit_status=inst> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: self.stop() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: time.sleep(0.1) Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: self.reap_workers() Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-pa> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: raise HaltServer(reason, self.WORKER_BOOT_ERRO> Jan 08 21:24:57 lists.mydomain.com gunicorn[3707354]: gunicorn.errors.HaltServer: <HaltServer 'Worker fa> Jan 08 21:24:58 lists.mydomain.com systemd[1]: gunicorn.service: Main process exited, code=exited, statu> -- Subject: Unit process exited -- Defined-By: systemd -- Support: http://www.ubuntu.com/support
-- An ExecStart= process belonging to unit gunicorn.service has exited.
-- The process' exit code is 'exited' and its exit status is 1.
It would help to see the full traceback, not truncated at the right. You can scroll the display to the right with the right arrow key or just find the messages in /var/log/syslog.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
This isn't your immediate problem, but it is an issue. The above isn't too informative. What do you get if you run
/opt/mailman/mm/bin/django-admin qcluster
manually?
I get the following:
# /opt/mailman/mm/bin/django-admin qcluster 11:50:41 [Q] INFO Q Cluster autumn-sixteen-tango-march starting. 11:50:41 [Q] INFO Process-1:1 ready for work at 3725610 11:50:41 [Q] INFO Process-1:2 ready for work at 3725611 11:50:42 [Q] INFO Process-1:3 ready for work at 3725612 11:50:42 [Q] INFO Process-1:4 ready for work at 3725613 11:50:42 [Q] INFO Process-1:5 monitoring at 3725614 11:50:42 [Q] INFO Process-1 guarding cluster autumn-sixteen-tango-march 11:50:42 [Q] INFO Process-1:6 pushing tasks at 3725615 11:50:42 [Q] INFO Q Cluster autumn-sixteen-tango-march running.
And then nothing until I end the command with Ctrl-C.
It would help to see the full traceback, not truncated at the right. You can scroll the display to the right with the right arrow key or just find the messages in /var/log/syslog.
Sorry, I didn't realize that they were truncated. Here are ( I hope) the full logs from /var/log/syslog:
Jan 8 21:24:57 lists systemd[1]: Started GNU Mailman web interfaces. Jan 8 21:24:57 lists gunicorn[3707354]: !!! Jan 8 21:24:57 lists gunicorn[3707354]: !!! WARNING: configuration file should have a valid Python extension. Jan 8 21:24:57 lists gunicorn[3707354]: !!! Jan 8 21:24:57 lists gunicorn[3707354]: Traceback (most recent call last): Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 209, in run Jan 8 21:24:57 lists gunicorn[3707354]: self.sleep() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 357, in sleep Jan 8 21:24:57 lists gunicorn[3707354]: ready = select.select([self.PIPE[0]], [], [], 1.0) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 242, in handle_chld Jan 8 21:24:57 lists gunicorn[3707354]: self.reap_workers() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 525, in reap_workers Jan 8 21:24:57 lists gunicorn[3707354]: raise HaltServer(reason, self.WORKER_BOOT_ERROR) Jan 8 21:24:57 lists gunicorn[3707354]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3> Jan 8 21:24:57 lists gunicorn[3707354]: During handling of the above exception, another exception occurred: Jan 8 21:24:57 lists gunicorn[3707354]: Traceback (most recent call last): Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/bin/gunicorn", line 14, in <module> Jan 8 21:24:57 lists gunicorn[3707354]: sys.exit(gunicorn.app.wsgiapp.run()) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 67, in run Jan 8 21:24:57 lists gunicorn[3707354]: WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 231, in run Jan 8 21:24:57 lists gunicorn[3707354]: super().run() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 72, in run Jan 8 21:24:57 lists gunicorn[3707354]: Arbiter(self).run() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 229, in run Jan 8 21:24:57 lists gunicorn[3707354]: self.halt(reason=inst.reason, exit_status=inst.exit_status) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 342, in halt Jan 8 21:24:57 lists gunicorn[3707354]: self.stop() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 393, in stop Jan 8 21:24:57 lists gunicorn[3707354]: time.sleep(0.1) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 242, in handle_chld Jan 8 21:24:57 lists gunicorn[3707354]: self.reap_workers() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 525, in reap_workers Jan 8 21:24:57 lists gunicorn[3707354]: raise HaltServer(reason, self.WORKER_BOOT_ERROR) Jan 8 21:24:57 lists gunicorn[3707354]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3> Jan 8 21:24:58 lists systemd[1]: gunicorn.service: Main process exited, code=exited, status=1/FAILURE Jan 8 21:24:58 lists systemd[1]: gunicorn.service: Failed with result 'exit-code'.
On 1/9/22 3:26 AM, Michael Heseltine wrote:
This isn't your immediate problem, but it is an issue. The above isn't too informative. What do you get if you run
/opt/mailman/mm/bin/django-admin qcluster
manually?
I get the following:
# /opt/mailman/mm/bin/django-admin qcluster 11:50:41 [Q] INFO Q Cluster autumn-sixteen-tango-march starting. 11:50:41 [Q] INFO Process-1:1 ready for work at 3725610 11:50:41 [Q] INFO Process-1:2 ready for work at 3725611 11:50:42 [Q] INFO Process-1:3 ready for work at 3725612 11:50:42 [Q] INFO Process-1:4 ready for work at 3725613 11:50:42 [Q] INFO Process-1:5 monitoring at 3725614 11:50:42 [Q] INFO Process-1 guarding cluster autumn-sixteen-tango-march 11:50:42 [Q] INFO Process-1:6 pushing tasks at 3725615 11:50:42 [Q] INFO Q Cluster autumn-sixteen-tango-march running.
And then nothing until I end the command with Ctrl-C.
This is what I would expect. It seems to be working. I'm not sure why it fails when run from systemd. Are you running memcached?
systemctl status memcached.service
That needs to be installed and running for qcluster.
Sorry, I didn't realize that they were truncated. Here are ( I hope) the full logs from /var/log/syslog:
Jan 8 21:24:57 lists systemd[1]: Started GNU Mailman web interfaces. Jan 8 21:24:57 lists gunicorn[3707354]: !!! Jan 8 21:24:57 lists gunicorn[3707354]: !!! WARNING: configuration file should have a valid Python extension. Jan 8 21:24:57 lists gunicorn[3707354]: !!! Jan 8 21:24:57 lists gunicorn[3707354]: Traceback (most recent call last): Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 209, in run Jan 8 21:24:57 lists gunicorn[3707354]: self.sleep() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 357, in sleep Jan 8 21:24:57 lists gunicorn[3707354]: ready = select.select([self.PIPE[0]], [], [], 1.0) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 242, in handle_chld Jan 8 21:24:57 lists gunicorn[3707354]: self.reap_workers() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 525, in reap_workers Jan 8 21:24:57 lists gunicorn[3707354]: raise HaltServer(reason, self.WORKER_BOOT_ERROR) Jan 8 21:24:57 lists gunicorn[3707354]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3> Jan 8 21:24:57 lists gunicorn[3707354]: During handling of the above exception, another exception occurred: Jan 8 21:24:57 lists gunicorn[3707354]: Traceback (most recent call last): Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/bin/gunicorn", line 14, in <module> Jan 8 21:24:57 lists gunicorn[3707354]: sys.exit(gunicorn.app.wsgiapp.run()) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 67, in run Jan 8 21:24:57 lists gunicorn[3707354]: WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 231, in run Jan 8 21:24:57 lists gunicorn[3707354]: super().run() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 72, in run Jan 8 21:24:57 lists gunicorn[3707354]: Arbiter(self).run() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 229, in run Jan 8 21:24:57 lists gunicorn[3707354]: self.halt(reason=inst.reason, exit_status=inst.exit_status) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 342, in halt Jan 8 21:24:57 lists gunicorn[3707354]: self.stop() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 393, in stop Jan 8 21:24:57 lists gunicorn[3707354]: time.sleep(0.1) Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 242, in handle_chld Jan 8 21:24:57 lists gunicorn[3707354]: self.reap_workers() Jan 8 21:24:57 lists gunicorn[3707354]: File "/opt/mailman/mm/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 525, in reap_workers Jan 8 21:24:57 lists gunicorn[3707354]: raise HaltServer(reason, self.WORKER_BOOT_ERROR) Jan 8 21:24:57 lists gunicorn[3707354]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3> Jan 8 21:24:58 lists systemd[1]: gunicorn.service: Main process exited, code=exited, status=1/FAILURE Jan 8 21:24:58 lists systemd[1]: gunicorn.service: Failed with result 'exit-code'.
I'm not sure about this one either. You might want to look at https://stackoverflow.com/questions/24488891/gunicorn-errors-haltserver-halt... or other hits from https://www.google.com/search?q=gunicorn+worker+failed+to+boot
Are your files /etc/systemd/system/gunicorn.service /opt/mailman/mm/bin/gunicorn opt/mailman/mm/gunicorn.conf
the same as recommended at
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (4)
-
Mark Sapiro -
Michael Heseltine -
Philip Colmer -
Stephen J. Turnbull