About to Install MM3 -- Seeking clarification
Good Morning Everyone,
I installed and managed mailman 2 for about two years. I am now ready to try Mailman 3. The documentation for version 2 was lucid and this forum was quite helpful in the past.
Now I'd like to proceed with installating MM3 but I doubt my understanding of the docs. It seems to assume that users will know what to do.
Okay, don't laugh! I just want to be sure what I am doing. Does the following sequence resemble the proper steps?
- Download the source from a git repo (I understood that one)
- Create a virtual environment (venv perhaps ... not stated explicitly)
- activate the virtual environment
- from the git project root, run: ./mailman start
I know, its stupid. I had to ask. It would really build my confidence someone would confirm the above steps or counter then with something very explicit. What about process ownership? Do I need to have a unix mailman user? What else do I need to get mailman up and running before proceeding to configuration. Ahaa! I perused the config docs a bit. Where can I find the .cfg file?
I hope these questions are not too annoying.
Regards Onyeibo
Hello Onyeibo,
nobody is laughing here!
If your server runs a Debian/Ubuntu like OS, this guide is really helpful:
https://docs.mailman3.org/en/latest/install/virtualenv.html[1]
I used it to set up my mailman3, and it worked quite well. If you have Apache instead of Nginx (like me), you must do some modifications for that.
Cheers, Eggert
Am Mittwoch, 12. Oktober 2022, 10:24:29 CEST schrieb Onyeibo Oku:
Good Morning Everyone,
I installed and managed mailman 2 for about two years. I am now ready to try Mailman 3. The documentation for version 2 was lucid and this forum was quite helpful in the past.
Now I'd like to proceed with installating MM3 but I doubt my understanding of the docs. It seems to assume that users will know what to do.
Okay, don't laugh! I just want to be sure what I am doing. Does the following sequence resemble the proper steps?
- Download the source from a git repo (I understood that one)
- Create a virtual environment (venv perhaps ... not stated explicitly)
- activate the virtual environment
- from the git project root, run: ./mailman start
I know, its stupid. I had to ask. It would really build my confidence someone would confirm the above steps or counter then with something very explicit. What about process ownership? Do I need to have a unix mailman user? What else do I need to get mailman up and running before proceeding to configuration. Ahaa! I perused the config docs a bit. Where can I find the .cfg file?
I hope these questions are not too annoying.
Regards Onyeibo
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
[1] https://docs.mailman3.org/en/latest/install/virtualenv.html
On 10/12/22 01:24, Onyeibo Oku wrote:
Good Morning Everyone,
I installed and managed mailman 2 for about two years. I am now ready to try Mailman 3. The documentation for version 2 was lucid and this forum was quite helpful in the past.
Now I'd like to proceed with installating MM3 but I doubt my understanding of the docs. It seems to assume that users will know what to do.
Okay, don't laugh! I just want to be sure what I am doing. Does the following sequence resemble the proper steps?
As Eggert says, we aren't laughing ...
- Download the source from a git repo (I understood that one)
- Create a virtual environment (venv perhaps ... not stated explicitly)
- activate the virtual environment
- from the git project root, run: ./mailman start
Step 4 is premature. More below.
I know, its stupid. I had to ask.
Not stupid ...
It would really build my confidence someone would confirm the above steps or counter then with something very explicit. What about process ownership? Do I need to have a unix mailman user? What else do I need to get mailman up and running before proceeding to configuration. Ahaa! I perused the config docs a bit. Where can I find the .cfg file?
As Eggert suggests, see
https://docs.mailman3.org/en/latest/install/virtualenv.html and follow
that. You don't need to download the source from GitLab. Following that
doc will get the latest releases from PyPI via pip. Those are still
somewhat out of date - we really should make new releases - so if you
want the latest from GitLab, there are a couple of ways to do it.
Perhaps the easiest is to just run the various pip install commands as
documented and then upgrade Mailman core in your venv with
pip install --upgrade git+https://gitlab.com/mailman/mailman@master
and update the various mailman web packages with
pip install --upgrade git+https://gitlab.com/mailman/xxx@master
where xxx is mailman-web, mailman-hyperkitty, django-mailman3, postorius, hyperkitty. I.e., run it 5 times, once for each value of xxx.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, 12 Oct 2022 17:52:47 -0700 Mark Sapiro <mark@msapiro.net> wrote:
Thanks Eggert and Sapiro.
As Eggert suggests, see https://docs.mailman3.org/en/latest/install/virtualenv.html and follow that.
I already started out on the above route. Its an adaptation because I am running Fedora. I had no problem with system-wide installations via dnf package management. However, some parts of the documentation does not add up.
You don't need to download the source from GitLab. Following that doc will get the latest releases from PyPI via pip. Those are still somewhat out of date - we really should make new releases - so if you want the latest from GitLab, there are a couple of ways to do it. Perhaps the easiest is to just run the various
pip installcommands as documented and then upgrade Mailman core in your venv with ``` pip install --upgrade git+https://gitlab.com/mailman/mailman@masterand update the various mailman web packages withpip install --upgrade git+https://gitlab.com/mailman/xxx@master
where xxx is mailman-web, mailman-hyperkitty, django-mailman3, postorius, hyperkitty. I.e., run it 5 times, once for each value of xxx.
I will come to this eventually (assuming that does not break things later). I aim to get it all working with regular pip packages first. I have installed mailman core already.
Fedora expects mailman files to reside at /usr/lib/mailman. It appears to ship with a mailman unix user. I adapted to those system defaults.
Now I observe that "su - mailman" gives me the proper prompt but .bashrc doesn't kick in. I still have to manually change to the virtual environment despite setting the automatic change in .bashrc. However, "su mailman" runs .bashrc but the prompt says "bash X.XX $". Is this normal?
Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" which should correspond to "/usr/lib/mailman/mm/var/*" on my instance. However, I don't see any instructions on how to get the "/mm" folder and the associated sub-folders. I just have "/usr/lib/mailman"
I am lost
"pip list" within the venv environment confirms that I have "mailman 3.3.5". What am I missing?
Regards Onyeibo
On 10/12/22 22:03, Onyeibo Oku wrote:
Fedora expects mailman files to reside at /usr/lib/mailman. It appears to ship with a mailman unix user. I adapted to those system defaults.
I would suggest you not do that and just install mailman in
/opt/mailman per
https://docs.mailman3.org/en/latest/install/virtualenv.html. If you
don't want to do that, you will have to replace all occurrences of
/opt/mailman in the doc with /usr/lib/mailman.
Now I observe that "su - mailman" gives me the proper prompt but .bashrc doesn't kick in. I still have to manually change to the virtual environment despite setting the automatic change in .bashrc. However, "su mailman" runs .bashrc but the prompt says "bash X.XX $". Is this normal?
Yes. The suggested change to .bashrc assumes you are logging in as the
Mailman user, not switching via su. You might try sudo -u mailman bash or sudo -u mailman -i instead of su, but I'm sot sure if that
will help.
Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" which should correspond to "/usr/lib/mailman/mm/var/*" on my instance. However, I don't see any instructions on how to get the "/mm" folder and the associated sub-folders. I just have "/usr/lib/mailman"
I think if you have var_dir: /usr/lib/mailman/mm/var in the
/etc/mailman3/mailman.cfg file you create per
https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-mailm...,
then starting Mailman core /usr/lib/mailman/venv/bin/mailman start
will create it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Thu, 13 Oct 2022 12:12:32 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/12/22 22:03, Onyeibo Oku wrote:
Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" which should correspond to "/usr/lib/mailman/mm/var/*" on my instance. However, I don't see any instructions on how to get the "/mm" folder and the associated sub-folders. I just have "/usr/lib/mailman"
I think if you have
var_dir: /usr/lib/mailman/mm/varin the /etc/mailman3/mailman.cfg file you create per https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-mailm..., then starting Mailman core/usr/lib/mailman/venv/bin/mailman startwill create it.
Thanks Mark, that was very helpful. I now have "mm" folder.
I have reached the section "Starting Mailman automatically". After creating the cfg file. Here is what I get:
# systemctl status mailman3
mailman3.service - GNU Mailing List Manager Loaded: loaded (/etc/systemd/system/mailman3.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2022-10-19 19:52:03 UTC; 4min 51s ago Process: 782 ExecStart=/opt/mailman/venv/bin/mailman start (code=exited, status=1/FAILURE) CPU: 156ms
Oct 19 19:52:02 mail.server.tld systemd[1]: Starting mailman3.service - GNU Mailing List Manager... Oct 19 19:52:03 mail.server.tld mailman[782]: Traceback (most recent call last): Oct 19 19:52:03 mail.server.tld mailman[782]: File "/opt/mailman/venv/bin/mailman", line 33, in <module> Oct 19 19:52:03 mail.server.tld mailman[782]: sys.exit(load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')()) Oct 19 19:52:03 mail.server.tld mailman[782]: File "/opt/mailman/venv/bin/mailman", line 25, in importlib_load_entry_point Oct 19 19:52:03 mail.server.tld mailman[782]: return next(matches).load() Oct 19 19:52:03 mail.server.tld mailman[782]: StopIteration Oct 19 19:52:03 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE Oct 19 19:52:03 mail.server.tld systemd[1]: mailman3.service: Failed with result 'exit-code'. Oct 19 19:52:03 mail.server.tld systemd[1]: Failed to start mailman3.service - GNU Mailing List Manager.
Mailman starts well from virtual environment though, ... but systemd doesn't replicate that success.
Regards Onyeibo
Onyeibo Oku writes:
Mailman starts well from virtual environment though, ... but systemd doesn't replicate that success.
systemd doesn't know anything about venv. If it's possible to source (not call as a command!) the venv bin/activate script via systemd, that should fix it. If not, we'll have to figure out what it is that the venv provides that the current systemd config does not. I would guess that after sourcing the venv's bin/activate
echo PATH: $PATH echo -n 'sys.path: '; python -c 'import sys; print (sys.path)'
will tell you what you need to know. You will have to translate sys.path into the form that is appropriate for PYTHON_PATH (I believe that's colon-separated, like PATH), and then tell systemd to set those environment variables. I can't help with that, I know nothing about systemd configs.
Steve
On 10/19/22 21:23, Stephen J. Turnbull wrote:
Onyeibo Oku writes:
Mailman starts well from virtual environment though, ... but systemd doesn't replicate that success.
systemd doesn't know anything about venv. If it's possible to source (not call as a command!) the venv bin/activate script via systemd, that should fix it. If not, we'll have to figure out what it is that the venv provides that the current systemd config does not. I would guess that after sourcing the venv's bin/activate
Our suggested systemd script at https://docs.mailman3.org/en/latest/install/virtualenv.html#starting-mailman... has
ExecStart=/opt/mailman/venv/bin/mailman start ExecReload=/opt/mailman/venv/bin/mailman restart ExecStop=/opt/mailman/venv/bin/mailman stop
/opt/mailman/venv/bin/mailman should work whether or not a venv is active.
Judging from the traceback, Onyeibo Oku's /opt/mailman/venv/bin/mailman is quite different from mine which is just
#!/opt/mailman/venv/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman'
__requires__ = 'mailman==3.3.5'
import re
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(
load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')()
)
What is yours? and can you start mailman as the mailman user with
/opt/mailman/venv/bin/mailman start
without activating the venv?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I have a file
/opt/mailman/.bashrc
file with this line:
source /opt/mailman/venv/bin/activate
works quite well with systemd.
My service file looks like this:
[Unit] Description=GNU Mailing List Manager After=syslog.target network.target mysqld.service
[Service] Type=forking PIDFile=/opt/mailman/mm/var/master.pid Environment="MAILMAN_CONFIG_FILE=/etc/mailman3/mailman.cfg" User=mailman Group=mailman ExecStart=/opt/mailman/venv/bin/mailman start ExecReload=/opt/mailman/venv/bin/mailman restart ExecStop=/opt/mailman/venv/bin/mailman stop
[Install] WantedBy=multi-user.target
Eggert
Am Donnerstag, 20. Oktober 2022, 08:38:05 CEST schrieb Mark Sapiro:
On 10/19/22 21:23, Stephen J. Turnbull wrote:
Onyeibo Oku writes:
Mailman starts well from virtual environment though, ... but systemd doesn't replicate that success.
systemd doesn't know anything about venv. If it's possible to source (not call as a command!) the venv bin/activate script via systemd, that should fix it. If not, we'll have to figure out what it is that the venv provides that the current systemd config does not. I would guess that after sourcing the venv's bin/activate
Our suggested systemd script at https://docs.mailman3.org/en/latest/install/virtualenv.html#starting-mailman -automatically has
ExecStart=/opt/mailman/venv/bin/mailman start ExecReload=/opt/mailman/venv/bin/mailman restart ExecStop=/opt/mailman/venv/bin/mailman stop
/opt/mailman/venv/bin/mailman should work whether or not a venv is active.
Judging from the traceback, Onyeibo Oku's /opt/mailman/venv/bin/mailman is quite different from mine which is just
#!/opt/mailman/venv/bin/python3 # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman' __requires__ = 'mailman==3.3.5' import re import sys from pkg_resources import load_entry_point if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit( load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')() )What is yours? and can you start mailman as the mailman user with
/opt/mailman/venv/bin/mailman startwithout activating the venv?
On Wed, 19 Oct 2022 23:38:05 -0700 Mark Sapiro <mark@msapiro.net> wrote:
Judging from the traceback, Onyeibo Oku's /opt/mailman/venv/bin/mailman is quite different from mine which is just ``` #!/opt/mailman/venv/bin/python3 # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman' __requires__ = 'mailman==3.3.5' import re import sys from pkg_resources import load_entry_point
if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit( load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')() )
What is yours? and can you start mailman as the mailman user with/opt/mailman/venv/bin/mailman start
without activating the venv?
True mine is different:
**************** /opt/mailman/venv/bin/mailman ************
#!/opt/mailman/venv/bin/python # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman' import re import sys
# for compatibility with easy_install; see #2198 __requires__ = 'mailman==3.3.5'
try: from importlib.metadata import distribution except ImportError: try: from importlib_metadata import distribution except ImportError: from pkg_resources import load_entry_point
def importlib_load_entry_point(spec, group, name): dist_name, _, _ = spec.partition('==') matches = ( entry_point for entry_point in distribution(dist_name).entry_points if entry_point.group == group and entry_point.name == name ) return next(matches).load()
globals().setdefault('load_entry_point', importlib_load_entry_point)
if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit(load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')())
*************** end of /opt/mailman/venv/bin/mailman ***********
I have tried setting environment path (PYTHONPATH) by specifying either "Environment" or "EnvironmentFile" in /etc/systemd/system/mailmain3.service. None works.
I had: EnvironmentFile=/opt/mailman/.systemd_vars
where the contents of .systemd_vars are: PYTHONPATH=/usr/lib64/python310.zip:/usr/lib64/python3.10:/usr/lib64/python3.10/lib-dynload:/usr/lib64/python3.10/site-packages:/usr/lib/python3.10/site-packages:/opt/mailman/venv/lib64/python3.10/site-packages:/opt/mailman/venv/lib/python3.10/site-packages
At this point, I am frustrated. Is there something else I can do to get mailman3 on this server?
from a frustrated Onyeibo
On Fri, 21 Oct 2022 06:56:42 +0100 Onyeibo Oku <onyeibo@schemefusion.com> wrote:
True mine is different:
**************** /opt/mailman/venv/bin/mailman ************
#!/opt/mailman/venv/bin/python # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman' import re import sys
# for compatibility with easy_install; see #2198 __requires__ = 'mailman==3.3.5'
try: from importlib.metadata import distribution except ImportError: try: from importlib_metadata import distribution except ImportError: from pkg_resources import load_entry_point
def importlib_load_entry_point(spec, group, name): dist_name, _, _ = spec.partition('==') matches = ( entry_point for entry_point in distribution(dist_name).entry_points if entry_point.group == group and entry_point.name == name ) return next(matches).load()
globals().setdefault('load_entry_point', importlib_load_entry_point)
if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit(load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')())
*************** end of /opt/mailman/venv/bin/mailman ***********
Would it be better to upgrade to the git instance instead? Something is not resolving in importlib_load_entry_point. Is there somewhere else I should look for errors ... perhaps to better understand what is happening in importlib_load_entry_point?
In any case, it works when virtual environment is active. Any idea how to activate that environment via systemd then?
Regards Onyeibo
On 10/21/22 01:14, Onyeibo Oku wrote:
In any case, it works when virtual environment is active. Any idea how to activate that environment via systemd then?
Can you successfully start mailman (or run any mailman commands) with, e.g.
/opt/mailman/venv/bin/mailman start
or
/opt/mailman/venv/bin/mailman info
without activating the venv?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, 21 Oct 2022 09:52:03 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/21/22 01:14, Onyeibo Oku wrote:
In any case, it works when virtual environment is active. Any idea how to activate that environment via systemd then?
Can you successfully start mailman (or run any mailman commands) with, e.g. ``` /opt/mailman/venv/bin/mailman start
or
/opt/mailman/venv/bin/mailman info
without activating the venv?
Yes. All those are working without activating venv.
On 10/21/22 09:56, Onyeibo Oku wrote:
On Fri, 21 Oct 2022 09:52:03 -0700 Mark Sapiro <mark@msapiro.net> wrote:
Can you successfully start mailman (or run any mailman commands) with, e.g. ``` /opt/mailman/venv/bin/mailman start
or
/opt/mailman/venv/bin/mailman info
without activating the venv?Yes. All those are working without activating venv.
On my test installation with /etc/systemd/system/mailman3.service installed per https://docs.mailman3.org/en/latest/install/virtualenv.html#starting-mailman...
systemctl start|stop|status mailman3
all work. Assuming
python3 -c 'from pkg_resources import load_entry_point'
(not in your venv) returns no error, try replacing /opt/mailman/venv/bin/mailman with
#!/opt/mailman/venv/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman'
__requires__ = 'mailman==3.3.5'
import re
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(
load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')()
)
Or in yours posted at
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
replace everything from the first try: through
globals().setdefault(...) with
from pkg_resources import load_entry_point
and see if that helps. Actually, if I replace mine with yours, it still works with systemctl, but this may be dependent on the python version (mine is 3.8.10). I'm really only guessing here, but this in your traceback
Oct 19 19:52:03 mail.server.tld mailman[782]: File
"/opt/mailman/venv/bin/mailman", line 25, in importlib_load_entry_point
Oct 19 19:52:03 mail.server.tld mailman[782]: return
next(matches).load()
Oct 19 19:52:03 mail.server.tld mailman[782]: StopIteration
Oct 19 19:52:03 mail.server.tld systemd[1]: mailman3.service: Control
process exited, code=exited, status=1/FAILURE
seems to say that the return next(matches).load() is raising
StopIteration and that is the issue, so removing that stuff from
/opt/mailman/venv/bin/mailman may help.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Fri, 21 Oct 2022 12:23:44 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/21/22 09:56, Onyeibo Oku wrote:
On Fri, 21 Oct 2022 09:52:03 -0700 Mark Sapiro <mark@msapiro.net> wrote:
Can you successfully start mailman (or run any mailman commands) with, e.g. ``` /opt/mailman/venv/bin/mailman start
or
/opt/mailman/venv/bin/mailman info
without activating the venv?Yes. All those are working without activating venv.
On my test installation with /etc/systemd/system/mailman3.service installed per https://docs.mailman3.org/en/latest/install/virtualenv.html#starting-mailman...
systemctl start|stop|status mailman3
all work. Assuming
python3 -c 'from pkg_resources import load_entry_point'(not in your venv) returns no error, try replacing /opt/mailman/venv/bin/mailman with
#!/opt/mailman/venv/bin/python3 # EASY-INSTALL-ENTRY-SCRIPT: 'mailman==3.3.5','console_scripts','mailman' __requires__ = 'mailman==3.3.5' import re import sys from pkg_resources import load_entry_point if __name__ == '__main__': sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0]) sys.exit( load_entry_point('mailman==3.3.5', 'console_scripts', 'mailman')() )Or in yours posted at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/... replace everything from the first
try:throughglobals().setdefault(...)withfrom pkg_resources import load_entry_pointand see if that helps.
I did the second alternative and replaced from "try" to the "globals ...". It works when I started it directly. However, I get a detailed traceback when systemd tries to start it.
Here is what I got:
Oct 22 04:13:55 mail.server.tld mailman[9253]: Traceback (most recent call last): Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/opt/mailman/venv/bin/mailman", line 9, in <module> Oct 22 04:13:55 mail.server.tld mailman[9253]: from pkg_resources import load_entry_point Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3267, in <module> Oct 22 04:13:55 mail.server.tld mailman[9253]: def _initialize_master_working_set(): Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3241, in _call_aside Oct 22 04:13:55 mail.server.tld mailman[9253]: f(*args, **kwargs) Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3279, in _initialize_master_working_set Oct 22 04:13:55 mail.server.tld mailman[9253]: working_set = WorkingSet._build_master() Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 564, in _build_master Oct 22 04:13:55 mail.server.tld mailman[9253]: ws = cls() Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 557, in __init__ Oct 22 04:13:55 mail.server.tld mailman[9253]: self.add_entry(entry) Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 613, in add_entry Oct 22 04:13:55 mail.server.tld mailman[9253]: for dist in find_distributions(entry, True): Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2077, in find_on_path Oct 22 04:13:55 mail.server.tld mailman[9253]: for dist in factory(fullpath): Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2142, in distributions_from_metadata Oct 22 04:13:55 mail.server.tld mailman[9253]: yield Distribution.from_location( Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2606, in from_location Oct 22 04:13:55 mail.server.tld mailman[9253]: )._reload_version() Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3004, in _reload_version Oct 22 04:13:55 mail.server.tld mailman[9253]: md_version = self._get_version() Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2785, in _get_version Oct 22 04:13:55 mail.server.tld mailman[9253]: version = _version_from_file(lines) Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2569, in _version_from_file Oct 22 04:13:55 mail.server.tld mailman[9253]: line = next(iter(version_lines), '') Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2780, in _get_metadata Oct 22 04:13:55 mail.server.tld mailman[9253]: for line in self.get_metadata_lines(name): Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1422, in get_metadata_lines Oct 22 04:13:55 mail.server.tld mailman[9253]: return yield_lines(self.get_metadata(name)) Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1412, in get_metadata Oct 22 04:13:55 mail.server.tld mailman[9253]: value = self._get(path) Oct 22 04:13:55 mail.server.tld mailman[9253]: File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1616, in _get Oct 22 04:13:55 mail.server.tld mailman[9253]: with open(path, 'rb') as stream: Oct 22 04:13:55 mail.server.tld mailman[9253]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO' Oct 22 04:13:55 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE
It seems one of the dependent packages (zope) requires certain privileges that systemd does not have. This may be what was failing initially. How might one solve this?
Regards Onyeibo
Onyeibo Oku writes:
(backtrace reformatted for readability)
Oct 22 04:13:55 mail.server.tld mailman[9253]:
File "/usr/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1616, in _get Oct 22 04:13:55 mail.server.tld mailman[9253]: with open(path, 'rb') as stream: Oct 22 04:13:55 mail.server.tld mailman[9253]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO'
That's just an ordinary attempt to read a file, nothing special about zope. It seems that that package was installed with the wrong ownership or permissions. Most likely you installed the package as root rather than the mailman user.
Everything in a venv should be owned by the user:group it is intended for.[1] Assuming that the user that Mailman processes are running under is "mailman" with group "mailman", then
chown -R mailman:mailman /opt/mailman/venv
should do the trick.
Steve
Footnotes: [1] Probably there are applications where this is not true. But it is true for Mailman.
On Sat Oct 22, 2022 at 1:40 PM WAT, Stephen J. Turnbull wrote:
Onyeibo Oku writes:
(backtrace reformatted for readability)
Oct 22 04:13:55 mail.server.tld mailman[9253]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO'
That's just an ordinary attempt to read a file, nothing special about zope. It seems that that package was installed with the wrong ownership or permissions. Most likely you installed the package as root rather than the mailman user.
Everything in a venv should be owned by the user:group it is intended for.[1] Assuming that the user that Mailman processes are running under is "mailman" with group "mailman", then
chown -R mailman:mailman /opt/mailman/venv
should do the trick.
Steve
Strangely, all files in /opt/mailman/venv belong to the mailman group
ls -lh /opt/mailman/venv/lib/python3.10/site-packages/zope* -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.component-5.0.1-py3.8-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.configuration-4.4.1-py3.9-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.event-4.5.0-py3.6-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.hookable-5.2-py3.10-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.interface-5.5.0-py3.10-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.schema-6.2.1-py3.8-nspkg.pth
and so it goes in the subdirectories.
Yet I set "mailman" as the User and Group in the systemd service file.
Onyeibo
On 10/22/22 09:35, Onyeibo Oku wrote:
Strangely, all files in /opt/mailman/venv belong to the mailman group
ls -lh /opt/mailman/venv/lib/python3.10/site-packages/zope* -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.component-5.0.1-py3.8-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.configuration-4.4.1-py3.9-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.event-4.5.0-py3.6-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.hookable-5.2-py3.10-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.interface-5.5.0-py3.10-nspkg.pth -rw-r--r--. 1 mailman mailman 529 Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.schema-6.2.1-py3.8-nspkg.pth
and so it goes in the subdirectories.
The specific error you reported refers tp /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
what does
ls -l
/opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
show?
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat Oct 22, 2022 at 7:44 PM WAT, Mark Sapiro wrote:
The specific error you reported refers tp /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
what does
ls -l /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFOshow?
See below:
command: ls -lh /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
output: -rw-r--r--. 1 mailman mailman 7.8K Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
-Onyeibo
On 10/22/22 13:07, Onyeibo Oku wrote:
On Sat Oct 22, 2022 at 7:44 PM WAT, Mark Sapiro wrote:
The specific error you reported refers tp /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
what does
ls -l /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFOshow?
See below:
command: ls -lh /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
output: -rw-r--r--. 1 mailman mailman 7.8K Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
This is actually strange. What does pip --version in your venv show?
Did you do anything to install mailman and other packages in your venv
other than pip install.
I ask because I don't think there should be *egg-info directories in /opt/mailman/venv/lib/python3.10/site-packages/ at all. In particular, in addition to the actual code directories, I only see files and directories with names ending in nspkg.pth and dist-info.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sat Oct 22, 2022 at 9:56 PM WAT, Mark Sapiro wrote:
On 10/22/22 13:07, Onyeibo Oku wrote:
On Sat Oct 22, 2022 at 7:44 PM WAT, Mark Sapiro wrote:
The specific error you reported refers tp /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
what does
ls -l /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFOshow?
See below:
command: ls -lh /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
output: -rw-r--r--. 1 mailman mailman 7.8K Oct 19 18:31 /opt/mailman/venv/lib/python3.10/site-packages/zope.i18nmessageid-5.0.1-py3.10.egg-info/PKG-INFO
This is actually strange. What does
pip --versionin your venv show? Did you do anything to install mailman and other packages in your venv other thanpip install.I ask because I don't think there should be *egg-info directories in /opt/mailman/venv/lib/python3.10/site-packages/ at all. In particular, in addition to the actual code directories, I only see files and directories with names ending in nspkg.pth and dist-info.
It is strange indeed.
So I decided to start all over again. I deleted both "mm" and "venv" folders and recreated "venv" first. I noted that after reinstalling the virtual environment pip was at version 21.3.1. I assumed that version may be buggy and upgraded to pip 22.3 before performing subsequent installations.
Next came the wheel package, followed by psycopg2. I checked the /opt/mailman/venv/lib64/python3.10/site-packages/ path to confirm that things were clean. All good -- no egg-info/* folders. I then installed mailman.
Now Mailman starts, and stops as desired when running as mailman user (both in virtual and non-virtual environments). However, running the service via systemd still fails. I now get a different traceback:
Oct 23 03:43:00 mail.server.tld mailman[19410]: Traceback (most recent call last): Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/bin/mailman", line 8, in <module> Oct 23 03:43:00 mail.server.tld mailman[19410]: sys.exit(main()) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 1130, in __call__ Oct 23 03:43:00 mail.server.tld mailman[19410]: return self.main(*args, **kwargs) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 1054, in main Oct 23 03:43:00 mail.server.tld mailman[19410]: with self.make_context(prog_name, args, **extra) as ctx: Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 920, in make_context Oct 23 03:43:00 mail.server.tld mailman[19410]: self.parse_args(ctx, args) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 1613, in parse_args Oct 23 03:43:00 mail.server.tld mailman[19410]: rest = super().parse_args(ctx, args) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 1378, in parse_args Oct 23 03:43:00 mail.server.tld mailman[19410]: value, args = param.handle_parse_result(ctx, opts, args) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 2360, in handle_parse_result Oct 23 03:43:00 mail.server.tld mailman[19410]: value = self.process_value(ctx, value) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/click/core.py", line 2322, in process_value Oct 23 03:43:00 mail.server.tld mailman[19410]: value = self.callback(ctx, self, value) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/bin/mailman.py", line 95, in initialize_config Oct 23 03:43:00 mail.server.tld mailman[19410]: initialize(value) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/core/initialize.py", line 217, in initialize Oct 23 03:43:00 mail.server.tld mailman[19410]: initialize_1(config_path) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/core/initialize.py", line 122, in initialize_1 Oct 23 03:43:00 mail.server.tld mailman[19410]: mailman.config.config.load(config_path) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/config/config.py", line 142, in load Oct 23 03:43:00 mail.server.tld mailman[19410]: self.push(filename, user_config.read()) Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/config/config.py", line 148, in push Oct 23 03:43:00 mail.server.tld mailman[19410]: self._post_process() Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/config/config.py", line 160, in _post_process Oct 23 03:43:00 mail.server.tld mailman[19410]: self.ensure_directories_exist() Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/mailman/config/config.py", line 262, in ensure_directories_exist Oct 23 03:43:00 mail.server.tld mailman[19410]: with Lock(lock_file): Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/flufl/lock/_lockfile.py", line 461, in __enter__ Oct 23 03:43:00 mail.server.tld mailman[19410]: self.lock() Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/flufl/lock/_lockfile.py", line 328, in lock Oct 23 03:43:00 mail.server.tld mailman[19410]: self._write() Oct 23 03:43:00 mail.server.tld mailman[19410]: File "/opt/mailman/venv/lib64/python3.10/site-packages/flufl/lock/_lockfile.py", line 492, in _write Oct 23 03:43:00 mail.server.tld mailman[19410]: with open(self._claimfile, 'w') as fp: Oct 23 03:43:00 mail.server.tld mailman[19410]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/mm/var/locks/mailman-cfg.lck|server.tld|19410|619270043958240354' Oct 23 03:43:00 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE
On 10/22/22 21:29, Onyeibo Oku wrote:
Now Mailman starts, and stops as desired when running as mailman user (both in virtual and non-virtual environments). However, running the service via systemd still fails. I now get a different traceback:
Oct 23 03:43:00 mail.server.tld mailman[19410]: Traceback (most recent call last):
...
Oct 23 03:43:00 mail.server.tld mailman[19410]: with open(self._claimfile, 'w') as fp: Oct 23 03:43:00 mail.server.tld mailman[19410]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/mm/var/locks/mailman-cfg.lck|server.tld|19410|619270043958240354' Oct 23 03:43:00 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE
What are ownership and permissions on /opt/mailman/mm/var/locks/
Assuming they are
drwxr-xr-x mailman mailman
it must be the case that systemd is not running the
/opt/mailman/venv/bin/mailman start command as the mailman user. I
suspect this is a Fedora issue. I note that
https://docs.fedoraproject.org/en-US/quick-docs/understanding-and-administer...
does not mention User or Group. You might try replacing
ExecStart=/opt/mailman/venv/bin/mailman start
ExecReload=/opt/mailman/venv/bin/mailman restart
ExecStop=/opt/mailman/venv/bin/mailman stop
with
ExecStart=sudo -u mailman /opt/mailman/venv/bin/mailman start
ExecReload=sudo -u mailman /opt/mailman/venv/bin/mailman restart
ExecStop=sudo -u mailman /opt/mailman/venv/bin/mailman stop
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Sun, 23 Oct 2022 08:00:19 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/22/22 21:29, Onyeibo Oku wrote:
Now Mailman starts, and stops as desired when running as mailman user (both in virtual and non-virtual environments). However, running the service via systemd still fails. I now get a different traceback:
Oct 23 03:43:00 mail.server.tld mailman[19410]: Traceback (most recent call last):
...
Oct 23 03:43:00 mail.server.tld mailman[19410]: with open(self._claimfile, 'w') as fp: Oct 23 03:43:00 mail.server.tld mailman[19410]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/mm/var/locks/mailman-cfg.lck|server.tld|19410|619270043958240354' Oct 23 03:43:00 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE
What are ownership and permissions on /opt/mailman/mm/var/locks/
Assuming they are
drwxr-xr-x mailman mailman
it must be the case that systemd is not running the
/opt/mailman/venv/bin/mailman startcommand as themailmanuser. I suspect this is a Fedora issue. I note that https://docs.fedoraproject.org/en-US/quick-docs/understanding-and-administer... does not mention User or Group. You might try replacingExecStart=/opt/mailman/venv/bin/mailman start ExecReload=/opt/mailman/venv/bin/mailman restart ExecStop=/opt/mailman/venv/bin/mailman stop with ExecStart=sudo -u mailman /opt/mailman/venv/bin/mailman start ExecReload=sudo -u mailman /opt/mailman/venv/bin/mailman restart ExecStop=sudo -u mailman /opt/mailman/venv/bin/mailman stop
This does not work. Systemd says it cannot find sudo (file). After updating in the commands "sudo" to "/bin/sudo" afterwards Systemd throws a different error message: PermissionError on /bin/sudo
This is tiring. At this moment I am left with the possibility that SELinux may be in the way. I cannot think of a way to prove this hypothesis. This is speculation. Perhaps I should try another installation method other than Virtual. I found Mailman3 in Fedora repository. Yes, it is packaged. May be I should try that or perhaps I should try the git version. I'd be happy to have a functioning Mailman before Friday.
Regards Onyeibo
Onyeibo Oku writes:
This is tiring. At this moment I am left with the possibility that SELinux may be in the way. I cannot think of a way to prove this hypothesis.
How about the obvious, disable SELinux temporarily? To dynamically disable it before the reboot, run the following command: # setenforce 0
Alternatively disable persistently:
- Open the SELinux configuration file: /etc/selinux/config. Locate the following line: SELINUX=enforcing
- Change the value to disabled: SELINUX=disabled
- Save your changes and close the file. On the next reboot, SELinux is permanently disabled.[1]
Or, if it is SELinux, the denial should be logged in /var/log/audit/audit.log.
In step 2, you could also set SELINUX=permissive, which permits but logs accesses that would be denied under SELINUX=enforcing.
See also https://www.tecmint.com/selinux-essentials-and-control-filesystem-access/ for a SELinux tutorial that explains all of the above in greater detail without going entirely off into the weeds.
This is speculation. Perhaps I should try another installation method other than Virtual.
If the permissions and ownerships are in fact correct as you report, that's going to be a lot of work for a very low probability of success.
Footnotes: [1] Thanks to IBM for this summary.
On 10/24/22 23:56, Stephen J. Turnbull wrote:
Onyeibo Oku writes:
This is speculation. Perhaps I should try another installation method other than Virtual.
If the permissions and ownerships are in fact correct as you report, that's going to be a lot of work for a very low probability of success.
I agree. I think at this point if you can't solve this by disabling or
configuring SELinux, you should be using Fedora support resources to ask
why your systemd service does not seem to honor User=.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, 25 Oct 2022 15:56:08 +0900 "Stephen J. Turnbull" <stephenjturnbull@gmail.com> wrote:
Onyeibo Oku writes:
This is tiring. At this moment I am left with the possibility that SELinux may be in the way. I cannot think of a way to prove this hypothesis.
How about the obvious, disable SELinux temporarily? To dynamically disable it before the reboot, run the following command: # setenforce 0
Thank you.
Sure enough Systemd succeeded when SELinux was in permissive mode (ignoring series of AVCs, of course). With SELinux in Enforcing mode, we are back to our previous mileage.
Conclusion: SELinux is in the way!
I must be missing necessary policy modules for mailman or I am missing relevant parts of the tutorial that deals with SELinux. Where do I go from here?
On 10/25/22 12:56, Onyeibo Oku wrote:
I must be missing necessary policy modules for mailman or I am missing relevant parts of the tutorial that deals with SELinux. Where do I go from here?
Create the necessary SELinux rules to allow mailman to access what it
needs to.
I don't know if anyone on this list can help you with that.
Note that if mailman is currently a normal user as opposed to a
system user making it a system user might help. See man usermod
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, 25 Oct 2022 14:46:06 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/25/22 12:56, Onyeibo Oku wrote:
I must be missing necessary policy modules for mailman or I am missing relevant parts of the tutorial that deals with SELinux. Where do I go from here?
Create the necessary SELinux rules to allow
mailmanto access what it needs to.
Roger that.
There is progress beyond the previous report (after setting mailman_lock_t on '/opt/mailman/mm/var/locks'. Systemd fails again on '/opt/mailman/mm/var/logs/mailman.log'
I applied mailman_log_t SELinux policy to that folder. Unlike the previous AVC, that didn't fix it.
PermissionError: [Errno 13] Permission denied: '/opt/mailman/mm/var/logs/mailman.log'
AVC avc: denied { append } for pid=774 comm="mailman" name="mailman.log" dev="sda" ino=2070696 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:mailman_log_t:s0 tclass=file permissive=0
I don't know if anyone on this list can help you with that.
Note that if
mailmanis currently anormal useras opposed to asystem usermaking it a system user might help. Seeman usermod
Mailman user is a System User here.
Regards Onyeibo
On Tue, 25 Oct 2022 20:56:31 +0100 Onyeibo Oku <onyeibo@schemefusion.com> wrote:
On Tue, 25 Oct 2022 15:56:08 +0900 "Stephen J. Turnbull" <stephenjturnbull@gmail.com> wrote:
Onyeibo Oku writes:
This is tiring. At this moment I am left with the possibility that SELinux may be in the way. I cannot think of a way to prove this hypothesis.
How about the obvious, disable SELinux temporarily? To dynamically disable it before the reboot, run the following command: # setenforce 0
Thank you.
Sure enough Systemd succeeded when SELinux was in permissive mode (ignoring series of AVCs, of course). With SELinux in Enforcing mode, we are back to our previous mileage.
Conclusion: SELinux is in the way!
I am happy to inform you that Systemd now starts and stops Mailman without interruptions. After series of SELinux file context labeling, setting the nis boolean, and a few custom policies, it resolved. Thank you for the responses (and your patience).
Now at https://docs.mailman3.org/en/latest/install/virtualenv.html#setting-up-a-wsg...
I have questions. I have a running instance of uwsgi serving a Django website on the target machine.
Will there be a conflict of resource (ports, threads, etc) running another instance?
Can one have multiple instances of uwsgi service? In whichcase, it doesn't hurt to follow the doc to its logical conclusion?
Regards Onyeibo
Onyeibo Oku writes:
I am happy to inform you that Systemd now starts and stops Mailman without interruptions.
Yay!
After series of SELinux file context labeling, setting the nis boolean, and a few custom policies, it resolved. Thank you for the responses (and your patience).
Perhaps you could post these to our wiki (https://wiki.list.org) if you don't consider them sensitive?
- Will there be a conflict of resource (ports, threads, etc) running another instance [of uwsgi]?
There can be. That's your problem, you need to configure each application to use non-conflicting ports and locations (URL paths, if the applications share externally-visible domains and ports). They can also conflict over internal resources such as file locks and I/O channels, but that depends on the specific applications. There are no conflicts among Mailman suite applications if configured as we recommend, but to know what conflicts might arise you need to explain what other apps will be running, and how you plan to configure them.
- Can one have multiple instances of uwsgi service?
You mean the same app running in parallel? That's probably not a good idea. If you mean several apps running in parallel, the answer is yes, provided you configure them not to conflict over resources.
In whichcase, it doesn't hurt to follow the doc to its logical conclusion?
I don't understand what you mean by logical conclusion.
On Sat, 29 Oct 2022 18:21:06 +0900 "Stephen J. Turnbull" <stephenjturnbull@gmail.com> wrote:
After series of SELinux file context labeling, setting the nis boolean, and a few custom policies, it resolved. Thank you for the responses (and your patience).
Perhaps you could post these to our wiki (https://wiki.list.org) if you don't consider them sensitive?
No problem. Although I suspect some of the Custom SELinux policies may be unnecessary, assuming I missed certain files/folders that require specific labels. I tried my best using the following pages:
- https://www.systutorials.com/docs/linux/man/8-mailman_queue_selinux/
- https://linux.die.net/man/8/mailman_selinux
- Will there be a conflict of resource (ports, threads, etc) running another instance [of uwsgi]?
There can be. That's your problem, you need to configure each application to use non-conflicting ports and locations (URL paths, if the applications share externally-visible domains and ports). They can also conflict over internal resources such as file locks and I/O channels, but that depends on the specific applications. There are no conflicts among Mailman suite applications if configured as we recommend, but to know what conflicts might arise you need to explain what other apps will be running, and how you plan to configure them.
I am not sure I understand what you mean by apps. Is that Django apps (./manage.py startapp app)? Do you mean a Django Project instead (./manage.py startproject app)?
What I have here is an active website project (live) done on Django framework. It runs from a different virtual environment (venv).
hence: mailman --> /opt/mailman/venv website --> /srv/website/venv (has a running uwsgi instance)
In whichcase, it doesn't hurt to follow the doc to its logical conclusion?
I don't understand what you mean by logical conclusion.
That is to follow the guide to the end of the page (disregarding the possibility of conflicts). This guide here: https://docs.mailman3.org/en/latest/install/virtualenv.html#setting-up-a-wsg...
--Onyeibo
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Onyeibo Oku writes:
I am not sure I understand what you mean by apps. Is that Django apps (./manage.py startapp app)? Do you mean a Django Project instead (./manage.py startproject app)?
An app for my purpose is everything configured via the settings.py used by the Django application created in a wsgi.py file. I guess you would think of them as projects.
What I have here is an active website project (live) done on Django framework. It runs from a different virtual environment (venv).
hence: mailman --> /opt/mailman/venv website --> /srv/website/venv (has a running uwsgi instance)
The venvs isolate the Python code used by each app from the other. But the apps can still have conflicts over ports (this is mentioned in the Django docs), filesystem objects, and other resources. For example, a lot of applications use the /static/ location as a place to keep content that is supplied by your project and doesn't change. This is problematic if two different apps each wants to install a default.css, but they're different. One of the apps is going to look funny.
Only the person who configures the apps can guess whether this kind of conflict will arise. Neither Mailman nor Django itself tries to protect you from such conflicts (I don't see how they could, in fact).
In whichcase, it doesn't hurt to follow the doc to its logical conclusion?
I don't understand what you mean by logical conclusion.
That is to follow the guide to the end of the page (disregarding the possibility of conflicts).
No, you can't disregard the possibility of conflicts. My understanding from what you've posted so far is that you have an app "website" that is already implemented and running. Most likely that app has configured resources (specifically ports and website locations in particular /static/) that are likely to conflict. You need to manage those conflicts, most likely by changing the ports used by Mailman, Postorius, and Hyperkitty and by mapping a different location such as /mailman/static/ to the /var directory that actually contains the static files managed by "manage.py collectstatic".
There may be other possible conflicts, but I can't help you with those because they depend on your website code.
Steve
I have this in my mailman3.service file:
[Service] Type=forking PIDFile=/opt/mailman/mm/var/master.pid Environment="MAILMAN_CONFIG_FILE=/etc/mailman3/mailman.cfg" User=mailman Group=mailman ExecStart=/opt/mailman/venv/bin/mailman start ExecReload=/opt/mailman/venv/bin/mailman restart ExecStop=/opt/mailman/venv/bin/mailman stop That works fine.
Am Dienstag, 25. Oktober 2022, 07:06:57 CEST schrieb Onyeibo Oku:
On Sun, 23 Oct 2022 08:00:19 -0700
Mark Sapiro <mark@msapiro.net> wrote:
On 10/22/22 21:29, Onyeibo Oku wrote:
Now Mailman starts, and stops as desired when running as mailman user (both in virtual and non-virtual environments). However, running the service via systemd still fails. I now get a different traceback:
Oct 23 03:43:00 mail.server.tld mailman[19410]: Traceback (most
recent call last): ...
Oct 23 03:43:00 mail.server.tld mailman[19410]: with open(self._claimfile, 'w') as fp: Oct 23 03:43:00 mail.server.tld mailman[19410]: PermissionError: [Errno 13] Permission denied: '/opt/mailman/mm/var/locks/mailman-cfg.lck|server.tld|19410|619270043958 240354' Oct 23 03:43:00 mail.server.tld systemd[1]: mailman3.service: Control process exited, code=exited, status=1/FAILURE
What are ownership and permissions on /opt/mailman/mm/var/locks/
Assuming they are
drwxr-xr-x mailman mailman
it must be the case that systemd is not running the
/opt/mailman/venv/bin/mailman startcommand as themailmanuser. I suspect this is a Fedora issue. I note that https://docs.fedoraproject.org/en-US/quick-docs/understanding-and-administ ering-systemd/#_service_parameters does not mention User or Group. You might try replacingExecStart=/opt/mailman/venv/bin/mailman start ExecReload=/opt/mailman/venv/bin/mailman restart ExecStop=/opt/mailman/venv/bin/mailman stop with ExecStart=sudo -u mailman /opt/mailman/venv/bin/mailman start ExecReload=sudo -u mailman /opt/mailman/venv/bin/mailman restart ExecStop=sudo -u mailman /opt/mailman/venv/bin/mailman stopThis does not work. Systemd says it cannot find sudo (file). After updating in the commands "sudo" to "/bin/sudo" afterwards Systemd throws a different error message: PermissionError on /bin/sudo
This is tiring. At this moment I am left with the possibility that SELinux may be in the way. I cannot think of a way to prove this hypothesis. This is speculation. Perhaps I should try another installation method other than Virtual. I found Mailman3 in Fedora repository. Yes, it is packaged. May be I should try that or perhaps I should try the git version. I'd be happy to have a functioning Mailman before Friday.
Regards Onyeibo
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Help!
Regards Onyeibo
On Thu Oct 13, 2022 at 6:03 AM WAT, Onyeibo Oku wrote:
On Wed, 12 Oct 2022 17:52:47 -0700 Mark Sapiro <mark@msapiro.net> wrote:
Thanks Eggert and Sapiro.
As Eggert suggests, see https://docs.mailman3.org/en/latest/install/virtualenv.html and follow that.
I already started out on the above route. Its an adaptation because I am running Fedora. I had no problem with system-wide installations via dnf package management. However, some parts of the documentation does not add up.
You don't need to download the source from GitLab. Following that doc will get the latest releases from PyPI via pip. Those are still somewhat out of date - we really should make new releases - so if you want the latest from GitLab, there are a couple of ways to do it. Perhaps the easiest is to just run the various
pip installcommands as documented and then upgrade Mailman core in your venv with ``` pip install --upgrade git+https://gitlab.com/mailman/mailman@masterand update the various mailman web packages withpip install --upgrade git+https://gitlab.com/mailman/xxx@master
where xxx is mailman-web, mailman-hyperkitty, django-mailman3, postorius, hyperkitty. I.e., run it 5 times, once for each value of xxx.I will come to this eventually (assuming that does not break things later). I aim to get it all working with regular pip packages first. I have installed mailman core already.
Fedora expects mailman files to reside at /usr/lib/mailman. It appears to ship with a mailman unix user. I adapted to those system defaults.
Now I observe that "su - mailman" gives me the proper prompt but .bashrc doesn't kick in. I still have to manually change to the virtual environment despite setting the automatic change in .bashrc. However, "su mailman" runs .bashrc but the prompt says "bash X.XX $". Is this normal?
Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" which should correspond to "/usr/lib/mailman/mm/var/*" on my instance. However, I don't see any instructions on how to get the "/mm" folder and the associated sub-folders. I just have "/usr/lib/mailman"
I am lost
"pip list" within the venv environment confirms that I have "mailman 3.3.5". What am I missing?
Regards Onyeibo
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
Onyeibo Oku writes:
I already started out on the above route. Its an adaptation because I am running Fedora. I had no problem with system-wide installations via dnf package management. However, some parts of the documentation does not add up.
As Mark already wrote, "adapting" to Fedora is a bad idea. It makes it harder to read our documentation, which as you note is imperfect. Also, it interferes with Fedora's own package management. You should not be installing anything but Fedora packages into /usr and /var (among others).
Now I observe that "su - mailman" gives me the proper prompt but .bashrc doesn't kick in. I still have to manually change to the virtual environment despite setting the automatic change in .bashrc. However, "su mailman" runs .bashrc but the prompt says "bash X.XX $". Is this normal?
Yes. Exec summary: Different startup files are read depending on whether the "login" and "interactive" flags are set, and the bin/activate script is not a normal script, invoking it is very finicky. I don't know exactly what's going on with the automatic invocation of the venv, but I suspect it's related to that.
A quick workaround for the difference between "su mailman" and "su - mailman" would be to link .bash_profile -> .bashrc, or source one from the other. See bash docs for details.
Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" which should correspond to "/usr/lib/mailman/mm/var/*" on my instance. However, I don't see any instructions on how to get the "/mm" folder and the associated sub-folders. I just have "/usr/lib/mailman"
That's undoubtedly related to "adapting" to Fedora practice. Where pip packaged files ended up depends on where you put your venv.
My recommendation is to delete everything you've done to this point and start over from the beginning, following our instructions to the letter. This has two benefits: (1) you don't need to continuously translate from Mailman instructions to Fedora practice, and (2) in case our instructions are inaccurate or poorly adapted to the Fedora environment, it's *much* easier for us to advise you on what to do.
Steve
On Thu Oct 13, 2022 at 6:03 AM WAT, Onyeibo Oku wrote:
On Wed, 12 Oct 2022 17:52:47 -0700 Mark Sapiro <mark@msapiro.net> wrote:
Thanks Eggert and Sapiro.
As Eggert suggests, see https://docs.mailman3.org/en/latest/install/virtualenv.html and follow that.
So far, this has been an interesting exercise. Mailman3 (core) is running but not Hyperkitty and Postorious.
My setup is different though. I am running a system-wide instance of uwsgi. At the moment, uwsgi is serving a website. It is running in Emperor-tyrant mode. Hence, it can serve multiple django projects. I only need to add the relevant .ini file at /etc/uwsgi.d/ and it spawns a service for that.
I have added mailman3.ini (taken from the suggested page above) into the /etc/uwsgi.d/ directory. This is what "systemctl status uwsgi" says:
Oct 31 06:57:18 mail.server.tld uwsgi[20676]: [uWSGI] getting INI configuration from mailman3.ini Oct 31 06:57:18 mail.server.tld uwsgi[20676]: unable to find logger file Oct 31 06:57:18 mail.server.tld uwsgi[20261]: Mon Oct 31 06:57:18 2022 - [emperor] curse the uwsgi instance mailman3.ini (pid: 20676) Oct 31 06:57:21 mail.server.tld uwsgi[20261]: Mon Oct 31 06:57:21 2022 - [emperor] removed uwsgi instance mailman3.ini Oct 31 06:58:30 mail.server.tld uwsgi[20684]: [emperor-tyrant] dropping privileges to 973 972 for instance mailman3.ini Oct 31 06:58:30 mail.server.tld uwsgi[20684]: *** has_emperor mode detected (fd:9) *** Oct 31 06:58:30 mail.server.tld uwsgi[20684]: [uWSGI] getting INI configuration from mailman3.ini Oct 31 06:58:30 mail.server.tld uwsgi[20684]: unable to find logger file Oct 31 06:58:30 mail.server.tld uwsgi[20261]: Mon Oct 31 06:58:30 2022 - [emperor] curse the uwsgi instance mailman3.ini (pid: 20684) Oct 31 06:58:33 mail.server.tld uwsgi[20261]: Mon Oct 31 06:58:33 2022 - [emperor] removed uwsgi instance mailman3.ini
It is not finding the logger file. The other django project on this machine is running fine. I can confirm that the logger plugin is installed. I must be missing something.
Regards Onyeibo
On 10/31/22 00:17, Onyeibo Oku wrote:
It is not finding the logger file. The other django project on this machine is running fine. I can confirm that the logger plugin is installed. I must be missing something.
If you are following https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configur... take note of
# Make sure that this directory is created or Django will fail on start.
LOGGING['handlers']['file']['filename'] =
'/opt/mailman/web/logs/mailmanweb.log'
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, 31 Oct 2022 09:38:37 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/31/22 00:17, Onyeibo Oku wrote:
It is not finding the logger file. The other django project on this machine is running fine. I can confirm that the logger plugin is installed. I must be missing something.
If you are following https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configur... take note of
# Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/opt/mailman/web/logs/mailmanweb.log'
The file and exact path is existing. I made sure of that.
Regards Onyeibo
On Mon, 31 Oct 2022 09:38:37 -0700 Mark Sapiro <mark@msapiro.net> wrote:
On 10/31/22 00:17, Onyeibo Oku wrote:
It is not finding the logger file. The other django project on this machine is running fine. I can confirm that the logger plugin is installed. I must be missing something.
If you are following https://docs.mailman3.org/en/latest/install/virtualenv.html#initial-configur... take note of
# Make sure that this directory is created or Django will fail on start. LOGGING['handlers']['file']['filename'] = '/opt/mailman/web/logs/mailmanweb.log'
` [Reviving this thread].
I abandoned the hyperkitty setup for a while and returned to it again today. I made progress. It turns out that I needed the uwsgi-logger-file rpm package and It wasn't installed with the uwsgi server. I adapted the settings also, since I was using Emperor mode (recall that there are other Django projects on the box). Currently, mailman, uwsgi and nginx are all running. They all seem to be happy until I assess the mailman-web home URL (e.g. http://my.server.com).
The outcome: "502 Bad Gateway"
Nothing shows in the logs except at /var/log/nginx/error.log:
2022/12/08 15:35:10 [error] 5879#5879: *7 upstream prematurely closed connection while reading response header from upstream, client: 105.113.33.89, server: my.server.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "my.server.com", referrer: "https://my.server.com/".
Something seems to be wrong between uwsgi and mailman, but I don't know where to look.
Regards Onyeibo
On Thu, 8 Dec 2022 17:27:13 +0100 Onyeibo Oku <onyeibo@schemefusion.com> wrote:
[Reviving this thread].
I abandoned the hyperkitty setup for a while and returned to it again today. I made progress. It turns out that I needed the uwsgi-logger-file rpm package and It wasn't installed with the uwsgi server. I adapted the settings also, since I was using Emperor mode (recall that there are other Django projects on the box). Currently, mailman, uwsgi and nginx are all running. They all seem to be happy until I assess the mailman-web home URL (e.g. http://my.server.com).
The outcome: "502 Bad Gateway"
Nothing shows in the logs except at /var/log/nginx/error.log:
2022/12/08 15:35:10 [error] 5879#5879: *7 upstream prematurely closed connection while reading response header from upstream, client: 105.113.33.89, server: my.server.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "my.server.com", referrer: "https://my.server.com/".
Something seems to be wrong between uwsgi and mailman, but I don't know where to look.
Regards Onyeibo
Happy New Year to all the Mailman People here (nice rhyme).
I pulled the plugs on uwsgi and installed Gunicorn. Gunicorn gave more mileage but something is still not right. When I visit the mailmanweb home with a browser, I see the following:
"Server Error. An error occurred while processing your request."
Mailmanweb and mailman3 services are running. I perused the log file at /opt/mailman/web/logs/mailmanweb.log and found the following:
raise MailmanConnectionError( mailmanclient.restbase.connection.MailmanConnectionError: ('Could not connect to Mailman API: ', 'ConnectionError(MaxRetryError("HTTPConnectionPool(host=\'localhost\', port=8001): Max retries exceeded with url: /3.1/lists?advertised=true&count=0&page=1 (Caused by NewConnectionError(\'<urllib3.connection.HTTPConnection object at 0x7f7f4b128c40>: Failed to establish a new connection: [Errno 111] 0x7f7f4b128c40>Connection refused\'))"))')
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/base.py", line 199, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/base.py", line 362, in process_exception_by_middleware response = middleware_method(request, exception) File "/opt/mailman/venv/lib64/python3.10/site-packages/postorius/middleware.py", line 48, in process_exception return utils.render_api_error(request) File "/opt/mailman/venv/lib64/python3.10/site-packages/postorius/utils.py", line 36, in render_api_error return render(request, 'postorius/errors/generic.html', File "/opt/mailman/venv/lib64/python3.10/site-packages/django/shortcuts.py", line 24, in render content = loader.render_to_string(template_name, context, request, using=using) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/loader.py", line 62, in render_to_string return template.render(context, request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/backends/django.py", line 62, in render return self.template.render(context) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/base.py", line 173, in render with context.bind_template(self): File "/usr/lib64/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/context.py", line 254, in bind_template updates.update(processor(self.request)) File "/opt/mailman/venv/lib64/python3.10/site-packages/django_mailman3/context_processors.py", line 32, in common context["site_name"] = get_current_site(request).name File "/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/shortcuts.py", line 16, in get_current_site return Site.objects.get_current(request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/models.py", line 59, in get_current return self._get_site_by_id(site_id) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/models.py", line 30, in _get_site_by_id site = self.get(pk=site_id) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/query.py", line 650, in get raise self.model.DoesNotExist( django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist.
Can someone point me in the right direction please?
Regards Onyeibo
On 1/2/23 03:54, Onyeibo Oku wrote:
I pulled the plugs on uwsgi and installed Gunicorn. Gunicorn gave more mileage but something is still not right. When I visit the mailmanweb home with a browser, I see the following:
"Server Error. An error occurred while processing your request."
Mailmanweb and mailman3 services are running. I perused the log file at /opt/mailman/web/logs/mailmanweb.log and found the following:
raise MailmanConnectionError( mailmanclient.restbase.connection.MailmanConnectionError: ('Could not connect to Mailman API: ', 'ConnectionError(MaxRetryError("HTTPConnectionPool(host=\'localhost\', port=8001): Max retries exceeded with url: /3.1/lists?advertised=true&count=0&page=1 (Caused by NewConnectionError(\'<urllib3.connection.HTTPConnection object at 0x7f7f4b128c40>: Failed to establish a new connection: [Errno 111] 0x7f7f4b128c40>Connection refused\'))"))')
Postorius is unable to connect to Mailman core's REST API. Are your settings for accessing Mailman's REST API correct? E.g., does
MAILMAN_REST_API_URL = 'http://localhost:8001' MAILMAN_REST_API_USER = 'restadmin' MAILMAN_REST_API_PASS = 'restpass'
match
$ mailman info ... REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:restpass
Note the 3.1/ in REST root url is not included in MAILMAN_REST_API_URL.
What do you get from, e.g.
curl -urestadmin:restpass http://localhost:8001/3.1/lists
It should return JSON with information about your lists.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Mon, 2 Jan 2023 09:57:45 -0800 Mark Sapiro <mark@msapiro.net> wrote:
MAILMAN_REST_API_URL = 'http://localhost:8001' MAILMAN_REST_API_USER = 'restadmin' MAILMAN_REST_API_PASS = 'restpass'
I've not had any occasion to set these while following the documentation.
match
$ mailman info ... REST root url: http://localhost:8001/3.1/ REST credentials: restadmin:restpass
Note the 3.1/ in REST root url is not included in MAILMAN_REST_API_URL.
What do you get from, e.g.
curl -urestadmin:restpass http://localhost:8001/3.1/lists
It should return JSON with information about your lists.
I get the following: {"start": 0, "total_size": 0, "http_etag": "\"678678567346533573674563786538\""}
On 1/2/23 11:56, Onyeibo wrote:
On Mon, 2 Jan 2023 09:57:45 -0800 Mark Sapiro <mark@msapiro.net> wrote:
curl -urestadmin:restpass http://localhost:8001/3.1/lists
It should return JSON with information about your lists.
I get the following: {"start": 0, "total_size": 0, "http_etag": "\"678678567346533573674563786538\""}
That is the expected response if you have no lists yet. So connection to Mailman's REST API works via curl, so why doesn't it work via mailmanclient? If Mailman core was running, it should work. I don't know why it doesn't. Perhaps you have some kind of firewall that's blocking it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi Mark, thanks for your inputs. I am making progress.
On January 2, 2023 9:15:59 PM GMT+01:00, Mark Sapiro <mark@msapiro.net> wrote:
On 1/2/23 11:56, Onyeibo wrote:
On Mon, 2 Jan 2023 09:57:45 -0800 Mark Sapiro <mark@msapiro.net> wrote:
curl -urestadmin:restpass http://localhost:8001/3.1/lists
It should return JSON with information about your lists.
I get the following: {"start": 0, "total_size": 0, "http_etag": "\"678678567346533573674563786538\""}
That is the expected response if you have no lists yet. So connection to Mailman's REST API works via curl, so why doesn't it work via mailmanclient? If Mailman core was running, it should work. I don't know why it doesn't. Perhaps you have some kind of firewall that's blocking it.
It turns out to be a fault from my settings. Postfix was not happy with the default email for the admin (<root@localhost>) since I had an operational virtual map for users. That connection error disappeared when I added a functioning admin email address.
There are other issues, however.
(1) Mailman refuses to resolve to a preferred subdomain. If the server's domain is "website.tld", I'd like mailman's homepage to be at "list.website.tld". Mailman insists that I add "website.tld" to the ALLOWED HOSTS, whereas "lists.website.tld" is already there as an ALLOWED HOST (same VPS/IP address nonetheless). The result is that the homepage ends up at "https://website.tld/mailman3/lists" when the desired url is "https://lists.website.tld/mailman3/lists". This is a conflict because the main domain is intended for another service. How do I approach this?
(2) See the traceback below: ERROR 2023-01-03 07:40:32,078 27926 django.request Internal Server Error: /mailman3/lists/ Traceback (most recent call last): File "/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailman/venv/lib64/python3.10/site-packages/postorius/views/list.py", line 980, in list_index return render(request, template, File "/opt/mailman/venv/lib64/python3.10/site-packages/django/shortcuts.py", line 24, in render content = loader.render_to_string(template_name, context, request, using=using) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/loader.py", line 62, in render_to_string return template.render(context, request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/backends/django.py", line 62, in render return self.template.render(context) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/base.py", line 173, in render with context.bind_template(self): File "/usr/lib64/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/template/context.py", line 254, in bind_template updates.update(processor(self.request)) File "/opt/mailman/venv/lib64/python3.10/site-packages/django_mailman3/context_processors.py", line 32, in common context["site_name"] = get_current_site(request).name File "/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/shortcuts.py", line 16, in get_current_site return Site.objects.get_current(request) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/models.py", line 59, in get_current return self._get_site_by_id(site_id) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/contrib/sites/models.py", line 30, in _get_site_by_id site = self.get(pk=site_id) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/query.py", line 650, in get raise self.model.DoesNotExist( django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist.
It seems a migration is missing. How should I approach this?
On 1/3/23 00:31, Onyeibo Oku wrote:
(1) Mailman refuses to resolve to a preferred subdomain. If the server's domain is "website.tld", I'd like mailman's homepage to be at "list.website.tld". Mailman insists that I add "website.tld" to the ALLOWED HOSTS, whereas "lists.website.tld" is already there as an ALLOWED HOST (same VPS/IP address nonetheless). The result is that the homepage ends up at "https://website.tld/mailman3/lists" when the desired url is "https://lists.website.tld/mailman3/lists". This is a conflict because the main domain is intended for another service. How do I approach this?
This is all configured in your web server. It needs to proxy requests for lists.website.tld to gunicorn and depending on the web server, may need more. E.g. see https://docs.mailman3.org/en/latest/install/virtualenv.html#nginx-configurat... for nginx.
(2) See the traceback below: ERROR 2023-01-03 07:40:32,078 27926 django.request Internal Server Error: /mailman3/lists/ Traceback (most recent call last): ... "/opt/mailman/venv/lib64/python3.10/site-packages/django/db/models/query.py", line 650, in get raise self.model.DoesNotExist( django.contrib.sites.models.Site.DoesNotExist: Site matching query does not exist.
It seems a migration is missing. How should I approach this?
Set SITE_ID = 0 in your Django settings. Also see https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Tue, 3 Jan 2023 09:46:00 -0800 Mark Sapiro <mark@msapiro.net> wrote:
This is all configured in your web server. It needs to proxy requests for lists.website.tld to gunicorn and depending on the web server, may need more. E.g. see https://docs.mailman3.org/en/latest/install/virtualenv.html#nginx-configurat... for nginx.
I have fixed this.
Set SITE_ID = 0 in your Django settings. Also see https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
Actually "SITE_ID = 1" solved that instead. I also had to go into the database and update the values (django provided 'example.com' by default and that was problematic).
Yaaay!
The mailing-list is up and running. I am super grateful to the mailman support here in this list. Thank you so much Mark, Stephen J, and Eggert.
I have more questions but I think it is better to start a new thread for that.
Best regards Onyeibo
On Mon, Jan 30, 2023 at 12:39 PM Onyeibo Oku <onyeibo@schemefusion.com> wrote:
On Tue, 3 Jan 2023 09:46:00 -0800 Mark Sapiro <mark@msapiro.net> wrote:
This is all configured in your web server. It needs to proxy requests for lists.website.tld to gunicorn and depending on the web server, may need more. E.g. see
https://docs.mailman3.org/en/latest/install/virtualenv.html#nginx-configurat...
for nginx.
I have fixed this.
Set SITE_ID = 0 in your Django settings. Also see
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
Actually "SITE_ID = 1" solved that instead. I also had to go into the database and update the values (django provided 'example.com' by default and that was problematic).
The common practice I have see is that one creates their domain on the web interface, which then displays the SITE_ID = N. You then take the value into settings.py to replace the one that's there.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
participants (6)
-
Eggert Ehmke -
Mark Sapiro -
Odhiambo Washington -
Onyeibo -
Onyeibo Oku
-
Stephen J. Turnbull