A little stuck with installation of MM3 - ModuleNotFoundError: No module named 'flufl.lock'
I decided to try this again, on FreeBSD-12.1
I still decided to follow https://wiki.list.org/DOC/Mailman%203%20installation%20experience which I know is not the main documentation, but I find the process easier to follow. I am also taking notes to see if I could share if I manage to succeed. This is important for me because Python-2.7 is being removed from FreeBSD in Dec 2020. I need to bail out of mailman-2.x before that happens so I am trying my hand again om MM3. I have failed before :-)
I am stuck at this step:
(venv) [root@gw /opt/mailman/mm]# /opt/mailman/mm/bin/mailman-post-update line 401, in execute_from_command_line
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0 Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 33, in <module> sys.exit(load_entry_point('Django==3.0.8', 'console_scripts', 'django-admin')()) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/management/__init__.py",
utility.execute()File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/management/__init__.py", line 377, in execute django.setup() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/apps/registry.py", line 114, in populate app_config.import_models() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/models/__init__.py", line 26, in <module> from .email import Attachment, Email File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/models/email.py", line 35, in <module> from .mailinglist import MailingList File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/models/mailinglist.py", line 37, in <module> from hyperkitty.lib.utils import pgsql_disable_indexscan File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/lib/utils.py", line 42, in <module> from flufl.lock import Lock *ModuleNotFoundError: No module named 'flufl.lock'* (venv) [root@gw /opt/mailman/mm]# pip install flufl.lock Requirement already satisfied: flufl.lock in ./venv/lib/python3.7/site-packages/flufl.lock-4.0-py3.7.egg (4.0) Requirement already satisfied: atpublic in ./venv/lib/python3.7/site-packages/atpublic-1.0-py3.7.egg (from flufl.lock) (1.0) Requirement already satisfied: psutil in ./venv/lib/python3.7/site-packages/psutil-5.7.2-py3.7-freebsd-12.1-RELEASE-p7-amd64.egg (from flufl.lock) (5.7.2)
Even after the 'pip install flufl.lock' it still doesn't find it!
I have had a similar problem earlier with installing pylibmc. It could not find libmemcache.h despite it being inside /usr/local/include somewhere and I luckily solved that with export LIBMEMCACHED=/usr/local && pip install pylibmc. I am wondering if there is an equivalent to resolve this one with flufl.lock.
Thanks in advance.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/25/20 6:36 AM, Odhiambo Washington wrote:
I decided to try this again, on FreeBSD-12.1
I still decided to follow https://wiki.list.org/DOC/Mailman%203%20installation%20experience which I know is not the main documentation, but I find the process easier to follow. I am also taking notes to see if I could share if I manage to succeed. This is important for me because Python-2.7 is being removed from FreeBSD in Dec 2020. I need to bail out of mailman-2.x before that happens so I am trying my hand again om MM3. I have failed before :-)
Even if it's not in FreeBSD, you can download Python 2.1.18 from https://www.python.org/downloads/release/python-2718/ and install it.
(venv) [root@gw /opt/mailman/mm]# pip install flufl.lock ... Even after the 'pip install flufl.lock' it still doesn't find it!
This is a packaging glitch with the flufl modules. Remove and reinstall them.
pip uninstall flufl.bounce flufl.i18n flufl.lock pip install flufl.bounce flufl.i18n flufl.lock
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat, 25 Jul 2020 at 19:58, Mark Sapiro mark@msapiro.net wrote:
On 7/25/20 6:36 AM, Odhiambo Washington wrote:
I decided to try this again, on FreeBSD-12.1
I still decided to follow https://wiki.list.org/DOC/Mailman%203%20installation%20experience which I know is not the main documentation, but I find the process easier to follow. I am also taking notes to see if I could share if I manage to succeed. This is important for me because Python-2.7 is being removed from FreeBSD in Dec 2020. I need to bail out of mailman-2.x before that happens so I am trying my hand again om MM3. I have failed before :-)
Even if it's not in FreeBSD, you can download Python 2.1.18 from https://www.python.org/downloads/release/python-2718/ and install it.
(venv) [root@gw /opt/mailman/mm]# pip install flufl.lock ... Even after the 'pip install flufl.lock' it still doesn't find it!
This is a packaging glitch with the flufl modules. Remove and reinstall them.
pip uninstall flufl.bounce flufl.i18n flufl.lock pip install flufl.bounce flufl.i18n flufl.lock
It would appear that luck isn't just on my side:
root@gw:/opt/mailman/mm # pip uninstall flufl.bounce flufl.i18n flufl.lock WARNING: Skipping flufl.bounce as it is not installed. WARNING: Skipping flufl.i18n as it is not installed. Found existing installation: flufl.lock 4.0 Uninstalling flufl.lock-4.0: Would remove: /usr/local/lib/python3.7/site-packages/flufl.lock-4.0-py3.7-nspkg.pth /usr/local/lib/python3.7/site-packages/flufl.lock-4.0.dist-info/* /usr/local/lib/python3.7/site-packages/flufl/lock/* Proceed (y/n)? y Successfully uninstalled flufl.lock-4.0 root@gw:/opt/mailman/mm # pip install flufl.bounce flufl.i18n flufl.lock Collecting flufl.bounce Downloading flufl.bounce-3.0.1-py3-none-any.whl (175 kB) |████████████████████████████████| 175 kB 18 kB/s Collecting flufl.i18n Downloading flufl.i18n-3.0.tar.gz (21 kB) Processing /root/.cache/pip/wheels/5a/de/bb/5e6448d5923f53f07f181ec225bf7b1d778d42384869dcf261/flufl.lock-4.0-py3-none-any.whl Requirement already satisfied: atpublic in /usr/local/lib/python3.7/site-packages (from flufl.bounce) (1.0) Collecting zope.interface Downloading zope.interface-5.1.0.tar.gz (225 kB) |████████████████████████████████| 225 kB 35 kB/s Requirement already satisfied: psutil in /usr/local/lib/python3.7/site-packages (from flufl.lock) (5.7.2) Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from zope.interface->flufl.bounce) (40.6.2) Building wheels for collected packages: flufl.i18n, zope.interface Building wheel for flufl.i18n (setup.py) ... done Created wheel for flufl.i18n: filename=flufl.i18n-3.0-py3-none-any.whl size=27279 sha256=7adc834e27423865de03bcad268b86d9df1d24c59110c7e4d4b8bb5863fb4e53 Stored in directory: /root/.cache/pip/wheels/5c/85/09/5b346688f1283d8622ac06adf4f2682bb13d36c2f410c52eb6 Building wheel for zope.interface (setup.py) ... done Created wheel for zope.interface: filename=zope.interface-5.1.0-cp37-cp37m-freebsd_12_1_RELEASE_p7_amd64.whl size=194660 sha256=51f326627a9121d8b4df989714bdcdf516f02d55f99de38bb9d7cc9867d93146 Stored in directory: /root/.cache/pip/wheels/84/93/fd/6bcb85b4bbf697c15b066350384cf3651feb47dcd5029a0b8d Successfully built flufl.i18n zope.interface Installing collected packages: zope.interface, flufl.bounce, flufl.i18n, flufl.lock Successfully installed flufl.bounce-3.0.1 flufl.i18n-3.0 flufl.lock-4.0 zope.interface-5.1.0 root@gw:/usr/home/wash # cd /opt/mailman/mm/ root@gw:/opt/mailman/mm # virtualenv venv created virtual environment CPython3.7.8.final.0-64 in 285ms creator CPython3Posix(dest=/opt/mailman/mm/venv, clear=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv) added seed packages: PyMySQL==0.10.0, Whoosh==2.7.4, gunicorn==20.0.4, mod_wsgi==4.7.1, mysqlclient==2.0.1, pip==20.1.1, psycopg2_binary==2.8.5, pylibmc==1.6.1, setuptools==49.2.0, wheel==0.34.2 activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator root@gw:/opt/mailman/mm # bash [root@gw /opt/mailman/mm]# source /opt/mailman/mm/venv/bin/activate (venv) [root@gw /opt/mailman/mm]# /opt/mailman/mm/bin/mailman-post-update line 401, in execute_from_command_line
- '[' False == False ']'
- mkdir -p /opt/mailman/mm/static
- /opt/mailman/mm/bin/django-admin collectstatic --clear --noinput --verbosity 0 Traceback (most recent call last): File "/opt/mailman/mm/venv/bin/django-admin", line 33, in <module> sys.exit(load_entry_point('Django==3.0.8', 'console_scripts', 'django-admin')()) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/management/__init__.py",
utility.execute()File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/management/__init__.py", line 377, in execute django.setup() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/__init__.py", line 24, in setup apps.populate(settings.INSTALLED_APPS) File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/apps/registry.py", line 114, in populate app_config.import_models() File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/apps/config.py", line 211, in import_models self.models_module = import_module(models_module_name) File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1006, in _gcd_import File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/models/__init__.py", line 26, in <module> from .email import Attachment, Email File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/models/email.py", line 35, in <module> from .mailinglist import MailingList File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/models/mailinglist.py", line 37, in <module> from hyperkitty.lib.utils import pgsql_disable_indexscan File "/opt/mailman/mm/venv/lib/python3.7/site-packages/HyperKitty-1.3.4-py3.7.egg/hyperkitty/lib/utils.py", line 42, in <module> from flufl.lock import Lock ModuleNotFoundError: No module named 'flufl.lock' (venv) [root@gw /opt/mailman/mm]#
What else should I try?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/25/20 11:37 AM, Odhiambo Washington wrote:
On Sat, 25 Jul 2020 at 19:58, Mark Sapiro mark@msapiro.net wrote:
This is a packaging glitch with the flufl modules. Remove and reinstall them.
pip uninstall flufl.bounce flufl.i18n flufl.lock pip install flufl.bounce flufl.i18n flufl.lock
It would appear that luck isn't just on my side:
root@gw:/opt/mailman/mm # pip uninstall flufl.bounce flufl.i18n flufl.lock ...
You need to run these pip commands in your venv. I.e. do this
[root@gw /opt/mailman/mm]# source /opt/mailman/mm/venv/bin/activate
before the pip uninstall and pip install.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat, 25 Jul 2020 at 22:11, Mark Sapiro mark@msapiro.net wrote:
On 7/25/20 11:37 AM, Odhiambo Washington wrote:
On Sat, 25 Jul 2020 at 19:58, Mark Sapiro mark@msapiro.net wrote:
This is a packaging glitch with the flufl modules. Remove and reinstall them.
pip uninstall flufl.bounce flufl.i18n flufl.lock pip install flufl.bounce flufl.i18n flufl.lock
It would appear that luck isn't just on my side:
root@gw:/opt/mailman/mm # pip uninstall flufl.bounce flufl.i18n flufl.lock ...
You need to run these pip commands in your venv. I.e. do this
[root@gw /opt/mailman/mm]# source /opt/mailman/mm/venv/bin/activate
before the pip uninstall and pip install. https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
That has solved the problem. Thank you very much. I have taken note of that. for my NOTES.
Finally, I have adapted your init script to get some rudimentary one that I could use on FreeBSD. I have to change /opt/mailman to be owned by mailman3:mailman3 (because I have "live" MLs on this server, using mailman-2.1.34. I am not sure if they can co-exist. I suppose they could, but what might be the security implication, if any?? This is what the init script looks like (rudimentary!!): (venv) [root@gw /usr/local/etc/rc.d]# less mailman3 ### BEGIN INIT INFO # Provides: GNU Mailman # Short-Description: Mailman3 Service # Description: service control for Mailman3 ### END INIT INFO
PATH=/opt/mailman/mm/bin:/opt/mailman/mm/venv/bin:/usr/sbin:/usr/bin:/bin:/sbin: DESC="GNU Mailman service" DAEMON=/opt/mailman/mm/bin/mailman NAME=mailman USER=mailman3 GROUP=mailman3
# Needed by click export LANG=en_US.UTF-8
# Exit if the package is not installed [ -x "$DAEMON" ] || exit 0
# Load the VERBOSE setting and other rcS variables #. /lib/init/vars.sh
# Define LSB log_* functions. # Depend on lsb-base (>= 3.2-14) to ensure that this file is present # and status_of_proc is working. #. /lib/lsb/init-functions
case "$1" in start) [ "$VERBOSE" != no ] && echo "Starting $DESC" "$NAME" # use --force to remove a stale lock. /usr/local/bin/sudo -u $USER $DAEMON start --force ;; stop) [ "$VERBOSE" != no ] && echo "Stopping $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON stop ;; status) /usr/local/bin/sudo -u $USER $DAEMON status ;; reopen) /usr/local/bin/sudo -u $USER $DAEMON reopen ;; restart) log_daemon_msg "Restarting $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON restart ;; *) echo "Usage: $SCRIPTNAME {start|stop|status|reopen|restart}" >&2 exit 3 ;; esac
It does start mailman3 for sure, but also complains after a few seconds with the message:
(venv) [root@gw /usr/local/etc/rc.d]# */opt/mailman/mm/venv/lib/python3.7/site-packages/mailman-3.3.2b1-py3.7.egg/mailman/rest/wsgiapp.py:180: DeprecatedWarning: Call to deprecated function __init__(...). API class may be removed in a future release, use falcon.App instead.*
- **kws)*
But:
(venv) [root@gw /usr/local/etc/rc.d]# ps ax | grep mailman 11677 - Is 0:00.01 /usr/local/bin/python /usr/local/mailman/bin/mailmanctl -s -q start (python2.7) 15860 - S 0:07.86 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=ArchRunner:0:1 -s (python2.7) 16310 - S 0:07.43 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=BounceRunner:0:1 -s (python2.7) 16928 - S 0:07.07 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=CommandRunner:0:1 -s (python2.7) 17335 - S 0:07.47 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s (python2.7) 17975 - S 0:07.02 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=NewsRunner:0:1 -s (python2.7) 18419 - S 0:20.06 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s (python2.7) 19015 - S 0:07.13 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=VirginRunner:0:1 -s (python2.7) 19131 - I 0:00.15 /usr/local/bin/python /usr/local/mailman/bin/qrunner --runner=RetryRunner:0:1 -s (python2.7) 27273 - Is 0:01.64 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/master --force -C /opt/mailman/mm/mailman.cfg (python3.7) 28526 - S 0:01.70 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=archive:0:1 (python3.7) 28627 - I 0:01.77 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=bounces:0:1 (python3.7) 28988 - S 0:01.70 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=command:0:1 (python3.7) 29341 - S 0:01.69 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=in:0:1 (python3.7) 29978 - S 0:01.68 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=lmtp:0:1 (python3.7) 30843 - S 0:01.70 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=nntp:0:1 (python3.7) 31541 - S 0:01.70 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=out:0:1 (python3.7) 31675 - S 0:01.70 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=pipeline:0:1 (python3.7) 32181 - S 0:01.77 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 (python3.7) 33166 - I 0:01.65 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=retry:0:1 (python3.7) 33694 - S 0:01.70 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=virgin:0:1 (python3.7) 34600 - S 0:01.69 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=digest:0:1 (python3.7) 41250 - S 0:00.01 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 (python3.7) 41533 - S 0:00.01 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/runner -C /opt/mailman/mm/mailman.cfg --runner=rest:0:1 (python3.7) 98450 - I 0:00.45 httpd: mailman-web (httpd) 59886 0 S+ 0:00.00 grep mailm
Assuming that I am using mod_wsgi with Apache, and that I have configured apache right using https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt .. should I be able to access the MM3 web UI?? At that point, I am feeling somewhat confused as to what to try next.
(venv) [root@gw /opt/mailman/mm/var/logs]# less mailman.log Jul 25 22:53:16 2020 (40670) Master started Jul 25 22:53:18 2020 (42036) bounces runner started. Jul 25 22:53:18 2020 (45205) out runner started. Jul 25 22:53:19 2020 (41191) archive runner started. Jul 25 22:53:19 2020 (46670) retry runner started. Jul 25 22:53:19 2020 (44567) nntp runner started. Jul 25 22:53:19 2020 (43069) in runner started. Jul 25 22:53:19 2020 (46960) virgin runner started. Jul 25 22:53:20 2020 (45720) pipeline runner started. Jul 25 22:53:20 2020 (47326) digest runner started. Jul 25 22:53:20 2020 (43755) lmtp runner started. Jul 25 22:53:20 2020 (45922) rest runner started. [2020-07-25 22:53:20 +0300] [45922] [INFO] Starting gunicorn 20.0.4 [2020-07-25 22:53:20 +0300] [45922] [INFO] Listening at: http://127.0.0.1:8001 (45922) [2020-07-25 22:53:20 +0300] [45922] [INFO] Using worker: sync [2020-07-25 22:53:20 +0300] [54732] [INFO] Booting worker with pid: 54732 [2020-07-25 22:53:20 +0300] [55467] [INFO] Booting worker with pid: 55467 Jul 25 22:53:21 2020 (42743) command runner started.
How do I access the web UI for MM3 now?
Thanks in advance.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/25/20 12:54 PM, Odhiambo Washington wrote:
Finally, I have adapted your init script to get some rudimentary one that I could use on FreeBSD. I have to change /opt/mailman to be owned by mailman3:mailman3 (because I have "live" MLs on this server, using mailman-2.1.34. I am not sure if they can co-exist. I suppose they could, but what might be the security implication, if any??
I manage more than one server that supports both Mailman 2.1 and Mailman
3 running as user mailman. I'm not aware of any security issues.
This is what the init script looks like (rudimentary!!): (venv) [root@gw /usr/local/etc/rc.d]# less mailman3 ### BEGIN INIT INFO # Provides: GNU Mailman # Short-Description: Mailman3 Service # Description: service control for Mailman3 ### END INIT INFO
PATH=/opt/mailman/mm/bin:/opt/mailman/mm/venv/bin:/usr/sbin:/usr/bin:/bin:/sbin: DESC="GNU Mailman service" DAEMON=/opt/mailman/mm/bin/mailman NAME=mailman USER=mailman3 GROUP=mailman3
# Needed by click export LANG=en_US.UTF-8
# Exit if the package is not installed [ -x "$DAEMON" ] || exit 0
# Load the VERBOSE setting and other rcS variables #. /lib/init/vars.sh
# Define LSB log_* functions. # Depend on lsb-base (>= 3.2-14) to ensure that this file is present # and status_of_proc is working. #. /lib/lsb/init-functions
case "$1" in start) [ "$VERBOSE" != no ] && echo "Starting $DESC" "$NAME" # use --force to remove a stale lock. /usr/local/bin/sudo -u $USER $DAEMON start --force
T don't recommend using --force in init scripts. It *shouldn't* matter because even with --force the master shouldn't break the lock if the pip that set it still exists, but I prefer not to use it.
;;stop) [ "$VERBOSE" != no ] && echo "Stopping $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON stop ;; status) /usr/local/bin/sudo -u $USER $DAEMON status ;; reopen) /usr/local/bin/sudo -u $USER $DAEMON reopen ;; restart) log_daemon_msg "Restarting $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON restart ;; *) echo "Usage: $SCRIPTNAME {start|stop|status|reopen|restart}" >&2 exit 3 ;; esac
It does start mailman3 for sure, but also complains after a few seconds with the message:
(venv) [root@gw /usr/local/etc/rc.d]# */opt/mailman/mm/venv/lib/python3.7/site-packages/mailman-3.3.2b1-py3.7.egg/mailman/rest/wsgiapp.py:180: DeprecatedWarning: Call to deprecated function __init__(...). API class may be removed in a future release, use falcon.App instead.*
- **kws)*
Hmm... strange, My various MM 3 installs are running falcon 1.4.0, 2.0.0 and 3.0.0a1 and I don't see this warning, but it is only a deprecation warning. It doesn't mean it won't work.
But:
(venv) [root@gw /usr/local/etc/rc.d]# ps ax | grep mailman ...
It seems Mailman core and the rest runner and its workers are all running.
Assuming that I am using mod_wsgi with Apache, and that I have configured apache right using https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt .. should I be able to access the MM3 web UI??
Yes, I think so.
At that point, I am feeling somewhat confused as to what to try next.
(venv) [root@gw /opt/mailman/mm/var/logs]# less mailman.log Jul 25 22:53:16 2020 (40670) Master started Jul 25 22:53:18 2020 (42036) bounces runner started. Jul 25 22:53:18 2020 (45205) out runner started. Jul 25 22:53:19 2020 (41191) archive runner started. Jul 25 22:53:19 2020 (46670) retry runner started. Jul 25 22:53:19 2020 (44567) nntp runner started. Jul 25 22:53:19 2020 (43069) in runner started. Jul 25 22:53:19 2020 (46960) virgin runner started. Jul 25 22:53:20 2020 (45720) pipeline runner started. Jul 25 22:53:20 2020 (47326) digest runner started. Jul 25 22:53:20 2020 (43755) lmtp runner started. Jul 25 22:53:20 2020 (45922) rest runner started. [2020-07-25 22:53:20 +0300] [45922] [INFO] Starting gunicorn 20.0.4 [2020-07-25 22:53:20 +0300] [45922] [INFO] Listening at: http://127.0.0.1:8001 (45922) [2020-07-25 22:53:20 +0300] [45922] [INFO] Using worker: sync [2020-07-25 22:53:20 +0300] [54732] [INFO] Booting worker with pid: 54732 [2020-07-25 22:53:20 +0300] [55467] [INFO] Booting worker with pid: 55467 Jul 25 22:53:21 2020 (42743) command runner started.
The above is all fine, but it is just Mailman core.
How do I access the web UI for MM3 now?
The Apache config points to /opt/mailman/mm/wsgi.py, a sampole of which is at https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=wsgi.py. Do you have that?
If you installed the Apache config literally, the location for Mailman 3 is defined by https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt#CA-29567efd6c2bbc2b7bb6c84828f419edb0e127f8_19 and the URL would be http(s)://your.server/mm3 - have you tried that?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat, Jul 25, 2020, at 2:17 PM, Mark Sapiro wrote:
On 7/25/20 12:54 PM, Odhiambo Washington wrote:
Finally, I have adapted your init script to get some rudimentary one that I could use on FreeBSD. I have to change /opt/mailman to be owned by mailman3:mailman3 (because I have "live" MLs on this server, using mailman-2.1.34. I am not sure if they can co-exist. I suppose they could, but what might be the security implication, if any??
I manage more than one server that supports both Mailman 2.1 and Mailman 3 running as user
mailman. I'm not aware of any security issues.This is what the init script looks like (rudimentary!!): (venv) [root@gw /usr/local/etc/rc.d]# less mailman3 ### BEGIN INIT INFO # Provides: GNU Mailman # Short-Description: Mailman3 Service # Description: service control for Mailman3 ### END INIT INFO
PATH=/opt/mailman/mm/bin:/opt/mailman/mm/venv/bin:/usr/sbin:/usr/bin:/bin:/sbin: DESC="GNU Mailman service" DAEMON=/opt/mailman/mm/bin/mailman NAME=mailman USER=mailman3 GROUP=mailman3
# Needed by click export LANG=en_US.UTF-8
# Exit if the package is not installed [ -x "$DAEMON" ] || exit 0
# Load the VERBOSE setting and other rcS variables #. /lib/init/vars.sh
# Define LSB log_* functions. # Depend on lsb-base (>= 3.2-14) to ensure that this file is present # and status_of_proc is working. #. /lib/lsb/init-functions
case "$1" in start) [ "$VERBOSE" != no ] && echo "Starting $DESC" "$NAME" # use --force to remove a stale lock. /usr/local/bin/sudo -u $USER $DAEMON start --force
T don't recommend using --force in init scripts. It *shouldn't* matter because even with --force the master shouldn't break the lock if the pip that set it still exists, but I prefer not to use it.
;;stop) [ "$VERBOSE" != no ] && echo "Stopping $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON stop ;; status) /usr/local/bin/sudo -u $USER $DAEMON status ;; reopen) /usr/local/bin/sudo -u $USER $DAEMON reopen ;; restart) log_daemon_msg "Restarting $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON restart ;; *) echo "Usage: $SCRIPTNAME {start|stop|status|reopen|restart}" >&2 exit 3 ;; esac
It does start mailman3 for sure, but also complains after a few seconds with the message:
(venv) [root@gw /usr/local/etc/rc.d]# */opt/mailman/mm/venv/lib/python3.7/site-packages/mailman-3.3.2b1-py3.7.egg/mailman/rest/wsgiapp.py:180: DeprecatedWarning: Call to deprecated function __init__(...). API class may be removed in a future release, use falcon.App instead.*
- **kws)*
Hmm... strange, My various MM 3 installs are running falcon 1.4.0, 2.0.0 and 3.0.0a1 and I don't see this warning, but it is only a deprecation warning. It doesn't mean it won't work.
I have seen this during development and there is an issue open for this1.
But:
(venv) [root@gw /usr/local/etc/rc.d]# ps ax | grep mailman ...
It seems Mailman core and the rest runner and its workers are all running.
Assuming that I am using mod_wsgi with Apache, and that I have configured apache right using https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt .. should I be able to access the MM3 web UI??
Yes, I think so.
At that point, I am feeling somewhat confused as to what to try next.
(venv) [root@gw /opt/mailman/mm/var/logs]# less mailman.log Jul 25 22:53:16 2020 (40670) Master started Jul 25 22:53:18 2020 (42036) bounces runner started. Jul 25 22:53:18 2020 (45205) out runner started. Jul 25 22:53:19 2020 (41191) archive runner started. Jul 25 22:53:19 2020 (46670) retry runner started. Jul 25 22:53:19 2020 (44567) nntp runner started. Jul 25 22:53:19 2020 (43069) in runner started. Jul 25 22:53:19 2020 (46960) virgin runner started. Jul 25 22:53:20 2020 (45720) pipeline runner started. Jul 25 22:53:20 2020 (47326) digest runner started. Jul 25 22:53:20 2020 (43755) lmtp runner started. Jul 25 22:53:20 2020 (45922) rest runner started. [2020-07-25 22:53:20 +0300] [45922] [INFO] Starting gunicorn 20.0.4 [2020-07-25 22:53:20 +0300] [45922] [INFO] Listening at: http://127.0.0.1:8001 (45922) [2020-07-25 22:53:20 +0300] [45922] [INFO] Using worker: sync [2020-07-25 22:53:20 +0300] [54732] [INFO] Booting worker with pid: 54732 [2020-07-25 22:53:20 +0300] [55467] [INFO] Booting worker with pid: 55467 Jul 25 22:53:21 2020 (42743) command runner started.
The above is all fine, but it is just Mailman core.
How do I access the web UI for MM3 now?
The Apache config points to /opt/mailman/mm/wsgi.py, a sampole of which is at https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=wsgi.py. Do you have that?
If you installed the Apache config literally, the location for Mailman 3 is defined by https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt#CA-29567efd6c2bbc2b7bb6c84828f419edb0e127f8_19 and the URL would be http(s)://your.server/mm3 - have you tried that?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
-- thanks, Abhilash Raj (maxking)
On Sun, 26 Jul 2020 at 00:17, Mark Sapiro mark@msapiro.net wrote:
On 7/25/20 12:54 PM, Odhiambo Washington wrote:
Finally, I have adapted your init script to get some rudimentary one
that I
could use on FreeBSD. I have to change /opt/mailman to be owned by mailman3:mailman3 (because I have "live" MLs on this server, using mailman-2.1.34. I am not sure if they can co-exist. I suppose they could, but what might be the security implication, if any??
I manage more than one server that supports both Mailman 2.1 and Mailman 3 running as user
mailman. I'm not aware of any security issues.This is what the init script looks like (rudimentary!!): (venv) [root@gw /usr/local/etc/rc.d]# less mailman3 ### BEGIN INIT INFO # Provides: GNU Mailman # Short-Description: Mailman3 Service # Description: service control for Mailman3 ### END INIT INFO
PATH=/opt/mailman/mm/bin:/opt/mailman/mm/venv/bin:/usr/sbin:/usr/bin:/bin:/sbin:
DESC="GNU Mailman service" DAEMON=/opt/mailman/mm/bin/mailman NAME=mailman USER=mailman3 GROUP=mailman3
# Needed by click export LANG=en_US.UTF-8
# Exit if the package is not installed [ -x "$DAEMON" ] || exit 0
# Load the VERBOSE setting and other rcS variables #. /lib/init/vars.sh
# Define LSB log_* functions. # Depend on lsb-base (>= 3.2-14) to ensure that this file is present # and status_of_proc is working. #. /lib/lsb/init-functions
case "$1" in start) [ "$VERBOSE" != no ] && echo "Starting $DESC" "$NAME" # use --force to remove a stale lock. /usr/local/bin/sudo -u $USER $DAEMON start --force
T don't recommend using --force in init scripts. It *shouldn't* matter because even with --force the master shouldn't break the lock if the pip that set it still exists, but I prefer not to use it.
;;stop) [ "$VERBOSE" != no ] && echo "Stopping $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON stop ;; status) /usr/local/bin/sudo -u $USER $DAEMON status ;; reopen) /usr/local/bin/sudo -u $USER $DAEMON reopen ;; restart) log_daemon_msg "Restarting $DESC" "$NAME" /usr/local/bin/sudo -u $USER $DAEMON restart ;; *) echo "Usage: $SCRIPTNAME {start|stop|status|reopen|restart}" >&2 exit 3 ;; esac
It does start mailman3 for sure, but also complains after a few seconds with the message:
(venv) [root@gw /usr/local/etc/rc.d]#
*/opt/mailman/mm/venv/lib/python3.7/site-packages/mailman-3.3.2b1-py3.7.egg/mailman/rest/wsgiapp.py:180:
DeprecatedWarning: Call to deprecated function __init__(...). API class may be removed in a future release, use falcon.App instead.*
- **kws)*
Hmm... strange, My various MM 3 installs are running falcon 1.4.0, 2.0.0 and 3.0.0a1 and I don't see this warning, but it is only a deprecation warning. It doesn't mean it won't work.
But:
(venv) [root@gw /usr/local/etc/rc.d]# ps ax | grep mailman ...
It seems Mailman core and the rest runner and its workers are all running.
Assuming that I am using mod_wsgi with Apache, and that I have configured apache right using
.. should I be able to access the MM3 web UI??
Yes, I think so.
At that point, I am feeling somewhat confused as to what to try next.
(venv) [root@gw /opt/mailman/mm/var/logs]# less mailman.log Jul 25 22:53:16 2020 (40670) Master started Jul 25 22:53:18 2020 (42036) bounces runner started. Jul 25 22:53:18 2020 (45205) out runner started. Jul 25 22:53:19 2020 (41191) archive runner started. Jul 25 22:53:19 2020 (46670) retry runner started. Jul 25 22:53:19 2020 (44567) nntp runner started. Jul 25 22:53:19 2020 (43069) in runner started. Jul 25 22:53:19 2020 (46960) virgin runner started. Jul 25 22:53:20 2020 (45720) pipeline runner started. Jul 25 22:53:20 2020 (47326) digest runner started. Jul 25 22:53:20 2020 (43755) lmtp runner started. Jul 25 22:53:20 2020 (45922) rest runner started. [2020-07-25 22:53:20 +0300] [45922] [INFO] Starting gunicorn 20.0.4 [2020-07-25 22:53:20 +0300] [45922] [INFO] Listening at: http://127.0.0.1:8001 (45922) [2020-07-25 22:53:20 +0300] [45922] [INFO] Using worker: sync [2020-07-25 22:53:20 +0300] [54732] [INFO] Booting worker with pid: 54732 [2020-07-25 22:53:20 +0300] [55467] [INFO] Booting worker with pid: 55467 Jul 25 22:53:21 2020 (42743) command runner started.
The above is all fine, but it is just Mailman core.
How do I access the web UI for MM3 now?
The Apache config points to /opt/mailman/mm/wsgi.py, a sampole of which is at < https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=wsgi.py
. Do you have that?
Yes, I have that file.
If you installed the Apache config literally, the location for Mailman 3 is defined by < https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt#CA-29567efd6c2bbc2b7bb6c84828f419edb0e127f8_19
and the URL would be http(s)://your.server/mm3 - have you tried that?
There is still some confusion on my part about the directives in the file so allow me to seek some clarifications.
The following are the contents of a file I have created and placed in my Apache Includes/ directory. I was hoping that with it, I can now access http://lists.my.server/mm3 and get the UI.
<CUT>
# Global section
WSGIDaemonProcess mailman-web display-name=mailman-web
maximum-requests=1000 umask=0002 user=mailman3
group=mailman3
python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var
WSGIRestrictSignal Off
# (I'm not sure that WSGIRestrictSignal Off is required, but it was in the # provided example so I kept it. I also made changes to WSGIDaemonProcess # based on my own mod_wsgi experience elsewhere.)
ErrorLog /var/log/myserver-error.log LogLevel debug
# This goes in the VirtualHost block for the domain.
# Mailman 3 stuff
Alias /static "/var/spool/mailman-web/static" *<----- Where is this
directory supposed to be and what/who creates it and with what permissions?*
</CUT-HERE>
I end up with an "Internal server error" and from the error log I see:
[Sun Jul 26 12:04:43.444127 2020] [wsgi:info] [pid 6444] [remote 197.232.81.246:53383] mod_wsgi (pid=6444, process='mailman-web', application='lists.my.server|/mm3'): Loading Python script file '/opt/mailman/mm/wsgi.py'. [Sun Jul 26 12:04:44.091922 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] mod_wsgi (pid=6444): Failed to exec Python script file '/opt/mailman/mm/wsgi.py'. [Sun Jul 26 12:04:44.092006 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] mod_wsgi (pid=6444): Exception occurred processing WSGI script '/opt/mailman/mm/wsgi.py'. [Sun Jul 26 12:04:44.092940 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] Traceback (most recent call last): [Sun Jul 26 12:04:44.093019 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/wsgi.py", line 38, in <module> [Sun Jul 26 12:04:44.093029 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] application = get_wsgi_application() [Sun Jul 26 12:04:44.093048 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/wsgi.py", line 12, in get_wsgi_application [Sun Jul 26 12:04:44.093057 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] django.setup(set_prefix=False) [Sun Jul 26 12:04:44.093071 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/__init__.py", line 19, in setup [Sun Jul 26 12:04:44.093090 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) [Sun Jul 26 12:04:44.093108 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 76, in __getattr__ [Sun Jul 26 12:04:44.093117 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] self._setup(name) [Sun Jul 26 12:04:44.093130 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 63, in _setup [Sun Jul 26 12:04:44.093138 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] self._wrapped = Settings(settings_module) [Sun Jul 26 12:04:44.093158 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 142, in __init__ [Sun Jul 26 12:04:44.093166 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] mod = importlib.import_module(self.SETTINGS_MODULE) [Sun Jul 26 12:04:44.093179 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module [Sun Jul 26 12:04:44.093187 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] return _bootstrap._gcd_import(name[level:], package, level) [Sun Jul 26 12:04:44.093210 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "<frozen importlib._bootstrap>", line 994, in _gcd_import [Sun Jul 26 12:04:44.093224 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "<frozen importlib._bootstrap>", line 971, in _find_and_load [Sun Jul 26 12:04:44.093239 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked [Sun Jul 26 12:04:44.093265 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] ModuleNotFoundError: No module named 'settings'
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/26/20 3:15 AM, Odhiambo Washington wrote:
There is still some confusion on my part about the directives in the file so allow me to seek some clarifications.
The following are the contents of a file I have created and placed in my Apache Includes/ directory. I was hoping that with it, I can now access http://lists.my.server/mm3 and get the UI.
<CUT>
# Global section WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3
group=mailman3 python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var WSGIRestrictSignal Off
ServerName lists.my.server ServerAdmin odhiambo@gmail.com # (I'm not sure that WSGIRestrictSignal Off is required, but it was in the # provided example so I kept it. I also made changes to WSGIDaemonProcess # based on my own mod_wsgi experience elsewhere.)
ErrorLog /var/log/myserver-error.log LogLevel debug
# This goes in the VirtualHost block for the domain.
# Mailman 3 stuff Alias /static "/var/spool/mailman-web/static" *<----- Where is this directory supposed to be and what/who creates it and with what permissions?*
It is created/updated by the mailman-post-update script based on CONFDIR and STATIC_DIR.
<Directory "/var/spool/mailman-web/static">
With the settings in the sample, the above should be
<Directory "/opt/mailman/mm/static">and the alias
Alias /static "/opt/mailman/mm/static"
Clearly https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt doesn't match some other files. I'll review abd update that.
Require all granted </Directory>WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py
<Files wsgi.py> Order deny,allow Allow from all Require all granted </Files> WSGIProcessGroup mailman-web </Directory> </VirtualHost> </CUT-HERE>
I end up with an "Internal server error" and from the error log I see:
...
[Sun Jul 26 12:04:44.093265 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] ModuleNotFoundError: No module named 'settings'
Django is trying to import your settings.py file and can't find it. Is it at /opt/mailman/mm/settings.py and also local settings at /opt/mailman/mm/settings_local.py?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, 26 Jul 2020 at 18:48, Mark Sapiro mark@msapiro.net wrote:
On 7/26/20 3:15 AM, Odhiambo Washington wrote:
There is still some confusion on my part about the directives in the file so allow me to seek some clarifications.
The following are the contents of a file I have created and placed in my Apache Includes/ directory. I was hoping that with it, I can now access http://lists.my.server/mm3 and get the UI.
<CUT>
# Global section WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3
group=mailman3
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var WSGIRestrictSignal Off
ServerName lists.my.server ServerAdmin odhiambo@gmail.com # (I'm not sure that WSGIRestrictSignal Off is required, but it was in
# provided example so I kept it. I also made changes to WSGIDaemonProcess # based on my own mod_wsgi experience elsewhere.)
ErrorLog /var/log/myserver-error.log LogLevel debug
# This goes in the VirtualHost block for the domain.
# Mailman 3 stuff Alias /static "/var/spool/mailman-web/static" *<----- Where is this directory supposed to be and what/who creates it and with what
python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7 the permissions?*
It is created/updated by the mailman-post-update script based on CONFDIR and STATIC_DIR.
The fact that I have run the mailman-post-update script and these directories aren't created means there is a problem in my setup, yes?
<Directory "/var/spool/mailman-web/static">With the settings in the sample, the above should be
<Directory "/opt/mailman/mm/static">and the alias
Alias /static "/opt/mailman/mm/static"
doesn't match some other files. I'll review abd update that.
Okay. I am anxiously waiting.
Require all granted </Directory>WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py
<Files wsgi.py> Order deny,allow Allow from all Require all granted </Files> WSGIProcessGroup mailman-web </Directory> </VirtualHost> </CUT-HERE>
I end up with an "Internal server error" and from the error log I see:
...
[Sun Jul 26 12:04:44.093265 2020] [wsgi:error] [pid 6444] [remote 197.232.81.246:53383] ModuleNotFoundError: No module named 'settings'
Django is trying to import your settings.py file and can't find it. Is it at /opt/mailman/mm/settings.py and also local settings at /opt/mailman/mm/settings_local.py?
Yes, I have these files: (venv) [root@gw /opt/mailman/mm]# ls -al /opt/mailman/mm/settings* -rw-r--r-- 1 mailman3 wheel 15518 Jul 25 14:34 /opt/mailman/mm/settings.py -rw-r--r-- 1 mailman3 wheel 15773 Jul 26 10:33 /opt/mailman/mm/settings_local.py
PS: I made my /opt/mailman to be owned by mailman3:wheel because of a confusion with the instructions at some point in the reading of some documentation previously (in my previous attempt last year!) and had noted that in my NOTES. I'll change this once things get clear for me. Also, the fact that one of the init scripts https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=init.d_mailman.txt in your "reflection" page was calling sudo :)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/26/20 9:12 AM, Odhiambo Washington wrote:
On Sun, 26 Jul 2020 at 18:48, Mark Sapiro mark@msapiro.net wrote:
On 7/26/20 3:15 AM, Odhiambo Washington wrote:
There is still some confusion on my part about the directives in the file so allow me to seek some clarifications.
The following are the contents of a file I have created and placed in my Apache Includes/ directory. I was hoping that with it, I can now access http://lists.my.server/mm3 and get the UI.
<CUT>
# Global section WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3
group=mailman3
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var WSGIRestrictSignal Off
ServerName lists.my.server ServerAdmin odhiambo@gmail.com # (I'm not sure that WSGIRestrictSignal Off is required, but it was in
# provided example so I kept it. I also made changes to WSGIDaemonProcess # based on my own mod_wsgi experience elsewhere.)
ErrorLog /var/log/myserver-error.log LogLevel debug
# This goes in the VirtualHost block for the domain.
# Mailman 3 stuff Alias /static "/var/spool/mailman-web/static" *<----- Where is this directory supposed to be and what/who creates it and with what
python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7 the permissions?*
It is created/updated by the mailman-post-update script based on CONFDIR and STATIC_DIR.
The fact that I have run the mailman-post-update script and these directories aren't created means there is a problem in my setup, yes?
If your mailman-post-update script is essentially the same as https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mailman-post-update.txt and it ran successfully, it would have created opt/mailman/mm/static and populated it. What was the output when you ran it?
Yes, I have these files: (venv) [root@gw /opt/mailman/mm]# ls -al /opt/mailman/mm/settings* -rw-r--r-- 1 mailman3 wheel 15518 Jul 25 14:34 /opt/mailman/mm/settings.py -rw-r--r-- 1 mailman3 wheel 15773 Jul 26 10:33 /opt/mailman/mm/settings_local.py
Is there an empty /opt/mailman/mm/__init__.py file?
PS: I made my /opt/mailman to be owned by mailman3:wheel because of a confusion with the instructions at some point in the reading of some documentation previously (in my previous attempt last year!) and had noted that in my NOTES. I'll change this once things get clear for me. Also, the fact that one of the init scripts https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=init.d_mailman.txt in your "reflection" page was calling sudo :)
It does sudo -u $USER ... so that it runs Mailman as the Mailman user
as it should. Have you set
USER=mailman3
in yours?
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, 27 Jul 2020 at 02:49, Mark Sapiro mark@msapiro.net wrote:
On Sun, 26 Jul 2020 at 18:48, Mark Sapiro mark@msapiro.net wrote:
On 7/26/20 3:15 AM, Odhiambo Washington wrote:
There is still some confusion on my part about the directives in the file so allow me to seek some clarifications.
The following are the contents of a file I have created and placed in my Apache Includes/ directory. I was hoping that with it, I can now access http://lists.my.server/mm3 and get the UI.
<CUT>
# Global section WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3
group=mailman3On 7/26/20 9:12 AM, Odhiambo Washington wrote: python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var WSGIRestrictSignal Off
ServerName lists.my.server ServerAdmin odhiambo@gmail.com # (I'm not sure that WSGIRestrictSignal Off is required, but it was in the # provided example so I kept it. I also made changes to WSGIDaemonProcess # based on my own mod_wsgi experience elsewhere.)
ErrorLog /var/log/myserver-error.log LogLevel debug
# This goes in the VirtualHost block for the domain.
# Mailman 3 stuff Alias /static "/var/spool/mailman-web/static" *<----- Where is this directory supposed to be and what/who creates it and with what permissions?*
It is created/updated by the mailman-post-update script based on CONFDIR and STATIC_DIR.
The fact that I have run the mailman-post-update script and these directories aren't created means there is a problem in my setup, yes?
If your mailman-post-update script is essentially the same as < https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mailman-post-update.txt
and it ran successfully, it would have created opt/mailman/mm/static and populated it. What was the output when you ran it?
Yes, I have these files: (venv) [root@gw /opt/mailman/mm]# ls -al /opt/mailman/mm/settings* -rw-r--r-- 1 mailman3 wheel 15518 Jul 25 14:34 /opt/mailman/mm/settings.py -rw-r--r-- 1 mailman3 wheel 15773 Jul 26 10:33 /opt/mailman/mm/settings_local.py
Is there an empty /opt/mailman/mm/__init__.py file?
No.
root@gw:/usr/local/mailman # ls -al /opt/mailman/mm/ total 88 drwxr-xr-x 8 mailman3 wheel 512 Jul 26 14:55 . drwxr-xr-x 6 mailman3 wheel 512 Jul 26 18:38 .. drwxr-xr-x 8 root wheel 512 Jul 25 22:16 .sass-cache drwxr-xr-x 2 mailman3 wheel 512 Jul 26 16:35 __pycache__ drwxr-xr-x 2 mailman3 wheel 512 Jul 25 14:42 bin -rw-r--r-- 1 mailman3 wheel 248 Jul 25 14:37 gunicorn.conf -rw-r--r-- 1 mailman3 wheel 266 Jul 26 15:34 gunicorn.conf.py lrwxr-xr-x 1 mailman3 wheel 24 Jul 25 14:39 logs -> /opt/mailman/mm/var/logs -rw-r--r-- 1 mailman3 wheel 370 Jul 25 14:32 mailman-hyperkitty.cfg -rw-r--r-- 1 mailman3 wheel 1291 Jul 26 17:12 mailman.cfg -rw-r--r-- 1 mailman3 wheel 15518 Jul 25 14:34 settings.py -rw-r--r-- 1 mailman3 wheel 15805 Jul 26 21:22 settings_local.py drwxr-xr-x 9 mailman3 wheel 512 Jul 25 22:16 static -rw-r--r-- 1 mailman3 wheel 1395 Jul 25 14:35 urls.py drwxr-xr-x 12 mailman3 wheel 512 Jul 26 19:30 var drwxr-xr-x 5 mailman3 wheel 512 Jul 26 17:11 venv -rw-r--r-- 1 mailman3 wheel 1170 Jul 26 11:39 wsgi.py
PS: I made my /opt/mailman to be owned by mailman3:wheel because of a confusion with the instructions at some point in the reading of some documentation previously (in my previous attempt last year!) and had noted that in my NOTES. I'll change this once things get clear for me. Also, the fact that one of the init scripts < https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=init.d_mailman.txt
in your "reflection" page was calling sudo :)
It does
sudo -u $USER ...so that it runs Mailman as the Mailman user as it should. Have you setUSER=mailman3
in yours?
Yes. However, that also means the /opt/mailman should be owned by mailman3, right? So I wasn't quite mistaken.
PS: There is the issue with qcluster, which doesn't seem to have been documented in details from your "experience". I see the configuration file and the init script, but not how to install it, or whether the procedure already installed it. Kindly clarify.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/27/20 4:29 AM, Odhiambo Washington wrote:
On Mon, 27 Jul 2020 at 02:49, Mark Sapiro mark@msapiro.net wrote:
Is there an empty /opt/mailman/mm/__init__.py file?
No.
There has to be one. Create it and then Django will be able to import your settings.
See the first line after
Create directories: ... and files:
at https://wiki.list.org/DOC/Mailman%203%20installation%20experience#line-69
Yes. However, that also means the /opt/mailman should be owned by mailman3, right? So I wasn't quite mistaken.
It doesn't matter what user actually owns this directory and its
subordinates as long as the Mailman user, mailman3 in your case, can
read and write there. Of course, making everything owned by the Mailman
user is the easiest way to ensure this.
Note also that any commands in /opt/mailman/mm/bin need to be run as the Mailman user and not root. Running them as root can create things owned by root the can't be read by the Mailman user.
PS: There is the issue with qcluster, which doesn't seem to have been documented in details from your "experience". I see the configuration file and the init script, but not how to install it, or whether the procedure already installed it.
qcluster is part of Django. If you have this
# # Asynchronous tasks # Q_CLUSTER = { 'timeout': 300, 'save_limit': 100, 'orm': 'default', }
or similar uncommented in your Django settings, and also 'django_q' in your INSTALLED_APPS, it's there and all you need to do is run
/opt/mailman/mm/bin/django-admin qcluster
to run it, but it should be run as a service via init or ??
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, 28 Jul 2020 at 06:50, Mark Sapiro mark@msapiro.net wrote:
On 7/27/20 4:29 AM, Odhiambo Washington wrote:
On Mon, 27 Jul 2020 at 02:49, Mark Sapiro mark@msapiro.net wrote:
Is there an empty /opt/mailman/mm/__init__.py file?
No.
There has to be one. Create it and then Django will be able to import your settings.
See the first line after
Create directories: ... and files:
at <https://wiki.list.org/DOC/Mailman%203%20installation%20experience#line-69
I don't know why I didn't see that initially :-)
Yes. However, that also means the /opt/mailman should be owned by mailman3, right? So I wasn't quite mistaken.
It doesn't matter what user actually
ownsthis directory and its subordinates as long as the Mailman user, mailman3 in your case, can read and write there. Of course, making everything owned by the Mailman user is the easiest way to ensure this.
True.
Note also that any commands in /opt/mailman/mm/bin need to be run as the Mailman user and not root. Running them as root can create things owned by root the can't be read by the Mailman user.
So, now I remember why I had to chown -R mailman3 /opt/mailman Then now it seems like I have been doing things incorrectly, because I have been doing a lot of operations as root by simply: cd /opt/mailman/mm virtualenv venv source venv/bin/activate Looks like the correct way is to start with su - mailman3
PS: There is the issue with qcluster, which doesn't seem to have been documented in details from your "experience". I see the configuration file and the init script, but not how to install it, or whether the procedure already installed it.
qcluster is part of Django. If you have this
# # Asynchronous tasks # Q_CLUSTER = { 'timeout': 300, 'save_limit': 100, 'orm': 'default', }
or similar uncommented in your Django settings, and also 'django_q' in your INSTALLED_APPS, it's there and all you need to do is run
/opt/mailman/mm/bin/django-admin qcluster
to run it, but it should be run as a service via init or ??
I have run it from supervisord. It has forked 30 processes and refuses to kill them when I stop supervisord. I will look for options around it.
Answering my own question I asked earlier, I have found "django-admin help --commands" to answer it.
PS: Did you by any chance find time to look into the issue about wsgi.py that you had mentioned - the one that has me stuck using it??
Thanks again.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/28/20 2:34 AM, Odhiambo Washington wrote:
So, now I remember why I had to chown -R mailman3 /opt/mailman Then now it seems like I have been doing things incorrectly, because I have been doing a lot of operations as root by simply: cd /opt/mailman/mm virtualenv venv source venv/bin/activate Looks like the correct way is to start with su - mailman3
Yes. All those things should be done as the Mailman user.
PS: There is the issue with qcluster, which doesn't seem to have been ... I have run it from supervisord. It has forked 30 processes and refuses to kill them when I stop supervisord. I will look for options around it.
I know nothing about supervisord, but there should be a parent process. E.g., here
msapiro@mail:~$ ps -fwwu mailman|grep qcluster mailman 2885 1 0 Jul18 ? 00:00:00 /bin/bash /opt/mailman/mm/bin/mailman-web-django-admin qcluster mailman 2889 2885 0 Jul18 ? 00:01:57 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 2930 2889 0 Jul18 ? 01:21:03 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 2944 2930 0 Jul18 ? 00:04:17 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 2945 2930 3 Jul18 ? 07:11:14 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 6467 2930 0 15:39 ? 00:00:03 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 12337 2930 0 15:00 ? 00:00:07 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 12639 2930 0 11:08 ? 00:00:11 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 14658 2930 0 Jul27 ? 00:00:22 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 15860 2930 0 01:41 ? 00:00:21 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 19205 2930 0 Jul27 ? 00:00:29 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster msapiro@mail:~$
pid 2885 is the parent which forked 2889 which in turn forked 2930 which forked all the others. stopping (sigterm) 2885 in this case will stop them all.
I have seen an issue on one server where some of the qcluster workers become orphaned and their parent pid is 1. These are left when I stop the upstart service so I have this script.
#! /bin/bash
if echo status qcluster | grep -q 'running' ; then
echo qcluster is running
exit
fi
ps -fwwu mailman|grep '[a]dmin qcluster'|awk '{print $2}'|xargs kill
to kill them.
PS: Did you by any chance find time to look into the issue about wsgi.py that you had mentioned - the one that has me stuck using it??
If you're referring to the issue at the end of https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/..., I think that was caused by your not having the __init__.py file in /opt/mailman/mm/.
If not, I probably need more information.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, 29 Jul 2020 at 02:58, Mark Sapiro mark@msapiro.net wrote:
On 7/28/20 2:34 AM, Odhiambo Washington wrote:
So, now I remember why I had to chown -R mailman3 /opt/mailman Then now it seems like I have been doing things incorrectly, because I
have
been doing a lot of operations as root by simply: cd /opt/mailman/mm virtualenv venv source venv/bin/activate Looks like the correct way is to start with su - mailman3
Yes. All those things should be done as the Mailman user.
PS: There is the issue with qcluster, which doesn't seem to have been ... I have run it from supervisord. It has forked 30 processes and refuses to kill them when I stop supervisord. I will look for options around it.
I know nothing about supervisord, but there should be a parent process. E.g., here
msapiro@mail:~$ ps -fwwu mailman|grep qcluster mailman 2885 1 0 Jul18 ? 00:00:00 /bin/bash /opt/mailman/mm/bin/mailman-web-django-admin qcluster mailman 2889 2885 0 Jul18 ? 00:01:57 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 2930 2889 0 Jul18 ? 01:21:03 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 2944 2930 0 Jul18 ? 00:04:17 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 2945 2930 3 Jul18 ? 07:11:14 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 6467 2930 0 15:39 ? 00:00:03 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 12337 2930 0 15:00 ? 00:00:07 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 12639 2930 0 11:08 ? 00:00:11 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 14658 2930 0 Jul27 ? 00:00:22 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 15860 2930 0 01:41 ? 00:00:21 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster mailman 19205 2930 0 Jul27 ? 00:00:29 /opt/mailman/mm/venv/bin/python /opt/mailman/mm/venv/bin/django-admin qcluster msapiro@mail:~$
pid 2885 is the parent which forked 2889 which in turn forked 2930 which forked all the others. stopping (sigterm) 2885 in this case will stop them all.
I have seen an issue on one server where some of the qcluster workers become orphaned and their parent pid is 1. These are left when I stop the upstart service so I have this script.
#! /bin/bash if echo
status qcluster| grep -q 'running' ; then echo qcluster is running exit fi ps -fwwu mailman|grep '[a]dmin qcluster'|awk '{print $2}'|xargs killto kill them.
I found some obscure option for supervisord that enabled the killing of the process(es). Obscure because it's not documented in the sample configuration for supervisord, but it's out there in the wild.
PS: Did you by any chance find time to look into the issue about wsgi.py that you had mentioned - the one that has me stuck using it??
If you're referring to the issue at the end of < https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
, I think that was caused by your not having the __init__.py file in /opt/mailman/mm/.
If not, I probably need more information.
Yes, the same problem. I created the file __init__.py in /opt/mailman/mm However, when I run using the wsgi.py I ended up (still do!) with the error:
[Wed Jul 29 10:08:51.869144 2020] [wsgi:info] [pid 79132] [remote 197.232.81.246:64709] mod_wsgi (pid=79132, process='mailman-web', application='mm3-lists.server.name|/mm3'): Loading Python script file '/opt/mailman/mm/wsgi.py'. [Wed Jul 29 10:08:51.872461 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] mod_wsgi (pid=79132): Failed to exec Python script file '/opt/mailman/mm/wsgi.py'. [Wed Jul 29 10:08:51.872507 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] mod_wsgi (pid=79132): Exception occurred processing WSGI script '/opt/mailman/mm/wsgi.py'. [Wed Jul 29 10:08:51.872754 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] Traceback (most recent call last): [Wed Jul 29 10:08:51.872827 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/wsgi.py", line 38, in <module> [Wed Jul 29 10:08:51.872835 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] application = get_wsgi_application() [Wed Jul 29 10:08:51.872847 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/wsgi.py", line 12, in get_wsgi_application [Wed Jul 29 10:08:51.872852 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] django.setup(set_prefix=False) [Wed Jul 29 10:08:51.872862 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/__init__.py", line 19, in setup [Wed Jul 29 10:08:51.872868 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) [Wed Jul 29 10:08:51.872878 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 76, in __getattr__ [Wed Jul 29 10:08:51.872884 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] self._setup(name) [Wed Jul 29 10:08:51.872893 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 63, in _setup [Wed Jul 29 10:08:51.872899 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] self._wrapped = Settings(settings_module) [Wed Jul 29 10:08:51.872909 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/conf/__init__.py", line 142, in __init__ [Wed Jul 29 10:08:51.872914 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] mod = importlib.import_module(self.SETTINGS_MODULE) [Wed Jul 29 10:08:51.872923 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module [Wed Jul 29 10:08:51.872929 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] return _bootstrap._gcd_import(name[level:], package, level) [Wed Jul 29 10:08:51.872938 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "<frozen importlib._bootstrap>", line 994, in _gcd_import [Wed Jul 29 10:08:51.872949 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "<frozen importlib._bootstrap>", line 971, in _find_and_load [Wed Jul 29 10:08:51.872960 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked [Wed Jul 29 10:08:51.872984 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] ModuleNotFoundError: No module named 'settings'
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/29/20 12:15 AM, Odhiambo Washington wrote:
Yes, the same problem. I created the file __init__.py in /opt/mailman/mm However, when I run using the wsgi.py I ended up (still do!) with the error:
[Wed Jul 29 10:08:51.869144 2020] [wsgi:info] [pid 79132] [remote 197.232.81.246:64709] mod_wsgi (pid=79132, process='mailman-web', application='mm3-lists.server.name|/mm3'): Loading Python script file '/opt/mailman/mm/wsgi.py'.
What is the contend of the file /opt/mailman/mm/wsgi.py?
You previously posted the setting of WSGIDaemonProcess in your Apache config as:
WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3
group=mailman3 python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/var
Is it still that?
...
197.232.81.246:64709] mod_wsgi (pid=79132): Exception occurred processing WSGI script '/opt/mailman/mm/wsgi.py'. [Wed Jul 29 10:08:51.872754 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] Traceback (most recent call last): [Wed Jul 29 10:08:51.872827 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/wsgi.py", line 38, in <module> [Wed Jul 29 10:08:51.872835 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] application = get_wsgi_application() [Wed Jul 29 10:08:51.872847 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] File "/opt/mailman/mm/venv/lib/python3.7/site-packages/Django-3.0.8-py3.7.egg/django/core/wsgi.py", line 12, in get_wsgi_application ... [Wed Jul 29 10:08:51.872984 2020] [wsgi:error] [pid 79132] [remote 197.232.81.246:64709] ModuleNotFoundError: No module named 'settings'
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 30 Jul 2020 at 00:13, Mark Sapiro mark@msapiro.net wrote:
On 7/29/20 12:15 AM, Odhiambo Washington wrote:
Yes, the same problem. I created the file __init__.py in /opt/mailman/mm However, when I run using the wsgi.py I ended up (still do!) with the
error:
[Wed Jul 29 10:08:51.869144 2020] [wsgi:info] [pid 79132] [remote 197.232.81.246:64709] mod_wsgi (pid=79132, process='mailman-web', application='mm3-lists.server.name|/mm3'): Loading Python script file '/opt/mailman/mm/wsgi.py'.
What is the contend of the file /opt/mailman/mm/wsgi.py?
[wash@gw ~]$ cd /opt/mailman/mm/ [wash@gw /opt/mailman/mm]$ less wsgi.py """ WSGI config for HyperKitty project.
It exposes the WSGI callable as a module-level variable named
application.
For more information on this file, see https://docs.djangoproject.com/en/{{ docs_version }}/howto/deployment/wsgi/ """
import os
# import sys # import site
# For some unknown reason, sometimes mod_wsgi fails to set the python paths to # the virtualenv, with the 'python-path' option. You can do it here too. # # # Remember original sys.path. #prev_sys_path = list(sys.path) # # Add here, for the settings module #site.addsitedir(os.path.abspath(os.path.dirname(__file__))) # # Add the virtualenv #venv = os.path.join(os.path.abspath(os.path.dirname(__file__)), # '..', 'lib', 'python2.6', 'site-packages') # site.addsitedir(venv) # # Reorder sys.path so new directories at the front. # new_sys_path = [] # for item in list(sys.path): # if item not in prev_sys_path: # new_sys_path.append(item) # sys.path.remove(item) # sys.path[:0] = new_sys_path
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
application = get_wsgi_application()
You previously posted the setting of WSGIDaemonProcess in your Apache config as:
WSGIDaemonProcess mailman-web display-name=mailman-web maximum-requests=1000 umask=0002 user=mailman3
group=mailman3 python-path=/opt/mailman/mm/venv/lib/python3.7/site-packages:/opt/mailman/mm/venv/lib/python3.7
python-home=/opt/mailman/mm/venv home=/opt/mailman/mm/varIs it still that?
This is from wsgi.conf in my Apache Includes/. It's a copy/paste of your version, with the only modification being the change pf python version to 3.7
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/30/20 1:53 AM, Odhiambo Washington wrote:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
Try changing this one line in wsgi.py to
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
If that doesn't work, try adding these two lines following the line
import os
import sys sys.path.insert(0, '/opt/mailman/mm')
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 30 Jul 2020 at 20:45, Mark Sapiro mark@msapiro.net wrote:
On 7/30/20 1:53 AM, Odhiambo Washington wrote:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
Try changing this one line in wsgi.py to
os.environ["DJANGO_SETTINGS_MODULE"] = "settings"
That option did not work.
If that doesn't work, try adding these two lines following the line
import os
import sys sys.path.insert(0, '/opt/mailman/mm')
This works - I mean changes the error message to
[Thu Jul 30 18:11:50.269667 2020] [wsgi:error] [pid 7627] [remote 197.232.81.246:63962] /opt/mailman/mm/venv/lib/python3.7/site-packages/readme_renderer-26.0-py3.7.egg/readme_renderer/markdown.py:38: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering. [Thu Jul 30 18:11:50.269700 2020] [wsgi:error] [pid 7627] [remote 197.232.81.246:63962] warnings.warn(_EXTRA_WARNING) [Thu Jul 30 21:11:50.573233 2020] [authz_core:error] [pid 10846] [client 197.232.81.246:63962] AH01630: client denied by server configuration: /var/spool/mailman-web, referer: http://mm3-lists.my.server/mm3/mailman3/lists/
There is NO /var/spool/mailman-web anywhere. It has not been created. You mentioned that it should be created automatically, IIRC.
root@gw:*/opt/mailman/mm* # ls -al total 116 drwxr-xr-x 8 mailman3 wheel 512 Jul 29 10:06 . drwxr-xr-x 7 mailman3 wheel 512 Jul 28 14:17 .. drwxr-xr-x 8 mailman3 wheel 512 Jul 25 22:16 .sass-cache -rw-r--r-- 1 mailman3 wheel 0 Jul 29 10:06 __init__.py drwxr-xr-x 2 mailman3 wheel 512 Jul 30 21:12 __pycache__ -rw-r--r-- 1 mailman3 wheel 2372 Jul 28 13:16 _settings_local.py drwxr-xr-x 2 mailman3 wheel 512 Jul 25 14:42 bin -rw-r--r-- 1 mailman3 wheel 248 Jul 25 14:37 gunicorn.conf -rw-r--r-- 1 mailman3 wheel 266 Jul 26 15:34 gunicorn.conf.py -rw-r--r-- 1 mailman3 wheel 0 Jul 28 12:19 init.py lrwxr-xr-x 1 mailman3 wheel 24 Jul 25 14:39 logs -> /opt/mailman/mm/var/logs -rw-r--r-- 1 mailman3 wheel 445 Jul 28 11:37 mailman-hyperkitty.cfg -rw-r--r-- 1 mailman3 wheel 1291 Jul 26 17:12 mailman.cfg -rw-r--r-- 1 mailman3 wheel 15516 Jul 27 16:32 settings.py -rw-r--r-- 1 mailman3 wheel 16069 Jul 28 13:15 settings_local.py -rw-r----- 1 mailman3 wheel 5914 Jul 27 17:35 settings_local.py.2 -rw-r--r-- 1 mailman3 wheel 15831 Jul 27 16:21 settings_local.py.BAK drwxr-xr-x 9 mailman3 wheel 512 Jul 25 22:16 static -rw-r--r-- 1 mailman3 wheel 1395 Jul 25 14:35 urls.py drwxr-xr-x 12 mailman3 wheel 512 Jul 30 21:12 var drwxr-xr-x 5 mailman3 wheel 512 Jul 26 17:11 venv -rw-r--r-- 1 mailman3 wheel 1271 Jul 30 21:03 wsgi.py
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
On 7/30/20 11:24 AM, Odhiambo Washington wrote:
This works - I mean changes the error message to
[Thu Jul 30 18:11:50.269667 2020] [wsgi:error] [pid 7627] [remote 197.232.81.246:63962] /opt/mailman/mm/venv/lib/python3.7/site-packages/readme_renderer-26.0-py3.7.egg/readme_renderer/markdown.py:38: UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering. [Thu Jul 30 18:11:50.269700 2020] [wsgi:error] [pid 7627] [remote 197.232.81.246:63962] warnings.warn(_EXTRA_WARNING)
I'm not sure why you are getting this warning, and it's non-fatal, but you can get rid of it by
pip install cmarkgfm
in your virtualenv.
[Thu Jul 30 21:11:50.573233 2020] [authz_core:error] [pid 10846] [client 197.232.81.246:63962] AH01630: client denied by server configuration: /var/spool/mailman-web, referer: http://mm3-lists.my.server/mm3/mailman3/lists/
There is NO /var/spool/mailman-web anywhere. It has not been created. You mentioned that it should be created automatically, IIRC.
No. I think I told you that the references to /var/spool/mailman-web/static in your Apache config should be changed to /opt/mailman/mm/static and that directory would be created/populated by mailman-post-update.
I think you still have
Alias /static "/var/spool/mailman-web/static"
in your Apache config and it needs to be
Alias /static "/opt/mailman/mm/static"
That may be my fault as when I recently edited https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt, I missed that line.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 30 Jul 2020 at 22:02, Mark Sapiro mark@msapiro.net wrote:
On 7/30/20 11:24 AM, Odhiambo Washington wrote:
This works - I mean changes the error message to
[Thu Jul 30 18:11:50.269667 2020] [wsgi:error] [pid 7627] [remote 197.232.81.246:63962]
/opt/mailman/mm/venv/lib/python3.7/site-packages/readme_renderer-26.0-py3.7.egg/readme_renderer/markdown.py:38:
UserWarning: Markdown renderers are not available. Install 'readme_renderer[md]' to enable Markdown rendering. [Thu Jul 30 18:11:50.269700 2020] [wsgi:error] [pid 7627] [remote 197.232.81.246:63962] warnings.warn(_EXTRA_WARNING)
I'm not sure why you are getting this warning, and it's non-fatal, but you can get rid of it by
pip install cmarkgfm
in your virtualenv.
(venv) [mailman3@gw ~]$ pip install cmarkgfm Requirement already satisfied: cmarkgfm in ./mm/venv/lib/python3.7/site-packages/cmarkgfm-0.4.2-py3.7-freebsd-12.1-RELEASE-p7-amd64.egg (0.4.2) Requirement already satisfied: cffi>=1.0.0 in ./mm/venv/lib/python3.7/site-packages/cffi-1.14.0-py3.7-freebsd-12.1-RELEASE-p7-amd64.egg (from cmarkgfm) (1.14.0) Requirement already satisfied: pycparser in ./mm/venv/lib/python3.7/site-packages/pycparser-2.20-py3.7.egg (from cffi>=1.0.0->cmarkgfm) (2.20) (venv) [mailman3@gw ~]$
[Thu Jul 30 21:11:50.573233 2020] [authz_core:error] [pid 10846] [client 197.232.81.246:63962] AH01630: client denied by server configuration: /var/spool/mailman-web, referer: http://mm3-lists.my.server/mm3/mailman3/lists/
There is NO /var/spool/mailman-web anywhere. It has not been created. You mentioned that it should be created automatically, IIRC.
No. I think I told you that the references to /var/spool/mailman-web/static in your Apache config should be changed to /opt/mailman/mm/static and that directory would be created/populated by mailman-post-update.
I think you still have
Alias /static "/var/spool/mailman-web/static"
in your Apache config and it needs to be
Alias /static "/opt/mailman/mm/static"
That may be my fault as when I recently edited < https://wiki.list.org/DOC/Mailman%203%20installation%20experience?action=AttachFile&do=view&target=mpo_wsgi.txt
, I missed that line.
After changing the Alias line and running the mailman-post-update, all errors are gone and I have a web UI. Yayi!!!
I have been able to create a domain, a list for testing. Now I just need to go through the analysis of what I have been doing so that I can clean up my notes. Then I can come back on migration issues.
A BIG THANK YOU for all the support up to this point.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)
participants (3)
-
Abhilash Raj -
Mark Sapiro -
Odhiambo Washington