Setting up on ubuntu server 22.04
Hi guys, I have an Ubuntu 22.04 server running on a Hetzner machine, with HestiaCP (Exim4) on it. I cannot for the life of me start the service of mailman3 core.
Following the installation tutorial I have tried with the venv approach and I am having the following issues:
- It is not clear to me whether the mailman configuration should be outside venv (and outside the mailman shell).
- For the above reason I have created a config file: /etc/mailman3/mailman.cfg and the mailman3 service outside venv. When starting the service I get the following:
ImportError: cannot import name 'path' from 'importlib_resources' (/opt/mailman/venv/lib/python3.10/site-packages/importlib_resources/__init__.py)
I've set up the database, the mailman user and the virtual environment.
I know I am missing something here (e.g. Python version? other dependencies?).
In general would it be better or/and easier to set this up on a new machine in my Hetzner cloud e.g. on Debian which seems to have packages of mailman available?
Thanks in advance for all your help.
Kyriakos
Kyriakos Terzopoulos writes:
ImportError: cannot import name 'path' from 'importlib_resources'
Try putting the error message into Google with "" around it. (This is generic advice for the future, obviously I already know what it is and will tell you. :-)
This is a known issue where Python changed some APIs that Mailman uses. Discussion is here: https://gitlab.com/mailman/mailman/-/issues/1093 There's a simple patch to requirements.txt at the end of the comments. Edit it, and delete and reinstall the mailman package, it will take 10 minures. If you're familiar with Python and pip you can also force remove that package, then reinstall with the additional version constraint.
In general would it be better or/and easier to set this up on a new machine in my Hetzner cloud e.g. on Debian which seems to have packages of mailman available?
That depends. Do you want to get your Mailman help from Debian (the answer is "NO!!"[1] unless you mostly expect to need no help -- that probably works for 70% of Debian + Mailman users) or from us? Unless your use case is exactly what the distro maintainer envisions, you will occasionally run into issues that are impossible for them to figure out, and rather expensive for us to help you with since they depend on old and modified versions. We strongly prefer you use a venv install, and if Mailman is at all important to your site's mission, you probably will too.
Footnotes: [1] At least for servers, I prefer Debian myself. Distros are the best thing since sliced bread. But Mailman is still a relatively fast-moving target[2] and none of the distros keeps up.
[2] In part because Gmail, Yahoo!, and O365 hate mailing lists, almost as much as the spammers hate us, and we keep tweaking things to deal with both kinds of Internet mischief.
Hi Stephen, all,
Firstly, thank you for your kind support. It seems that there is some misunderstanding there from my side, since I did not use the github repo to install mailman - I did it through pip install.
I will try to set it up with github and come back with any issues.
Thank you. Regards
*Kyriakos Terzopoulos*
Web developer / e-learning expert
*Tel:*+30 211 213 9858
*Mobile:* +30 694 526 4512
- E-mail: *kyriakos.terzopoulos@gmail.com
- Skype:* kyriakos.terzopoulos Find me on Facebook <http://www.facebook.com/cirrus3d> Follow me on Twitter <http://twitter.com/#%21/cirrus3d>
On Mon, 2 Oct 2023 at 18:59, Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Kyriakos Terzopoulos writes:
ImportError: cannot import name 'path' from 'importlib_resources'
Try putting the error message into Google with "" around it. (This is generic advice for the future, obviously I already know what it is and will tell you. :-)
This is a known issue where Python changed some APIs that Mailman uses. Discussion is here: https://gitlab.com/mailman/mailman/-/issues/1093 There's a simple patch to requirements.txt at the end of the comments. Edit it, and delete and reinstall the mailman package, it will take 10 minures. If you're familiar with Python and pip you can also force remove that package, then reinstall with the additional version constraint.
In general would it be better or/and easier to set this up on a new machine in my Hetzner cloud e.g. on Debian which seems to have packages of mailman available?
That depends. Do you want to get your Mailman help from Debian (the answer is "NO!!"[1] unless you mostly expect to need no help -- that probably works for 70% of Debian + Mailman users) or from us? Unless your use case is exactly what the distro maintainer envisions, you will occasionally run into issues that are impossible for them to figure out, and rather expensive for us to help you with since they depend on old and modified versions. We strongly prefer you use a venv install, and if Mailman is at all important to your site's mission, you probably will too.
Footnotes: [1] At least for servers, I prefer Debian myself. Distros are the best thing since sliced bread. But Mailman is still a relatively fast-moving target[2] and none of the distros keeps up.
[2] In part because Gmail, Yahoo!, and O365 hate mailing lists, almost as much as the spammers hate us, and we keep tweaking things to deal with both kinds of Internet mischief.
On Tue, Oct 3, 2023 at 10:36 AM Kyriakos Terzopoulos < kyriakos.terzopoulos@gmail.com> wrote:
Hi Stephen, all,
Firstly, thank you for your kind support. It seems that there is some misunderstanding there from my side, since I did not use the github repo to install mailman - I did it through pip install.
I will try to set it up with github and come back with any issues.
Not sure what you mean by github, but it will be a lot easier to follow this guide: https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Hi Odhiambo,
Installing this way I get the following error (Note that I have installed all dependencies):
(venv) mailman@hestia:~$ mailman info
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman", line 33, in <module>
sys.exit(load_entry_point('mailman==3.3.8', 'console_scripts','mailman')())
File "/opt/mailman/venv/bin/mailman", line 25, in importlib_load_entry_point
return next(matches).load()File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 23, in <module>
from mailman.config import configFile "/opt/mailman/venv/lib/python3.10/site-packages/mailman/config/__init__.py", line 20, in <module>
from mailman.config.config import ConfigurationFile "/opt/mailman/venv/lib/python3.10/site-packages/mailman/config/config.py", line 27, in <module>
from importlib_resources import path, read_textImportError: cannot import name 'path' from 'importlib_resources' (/opt/mailman/venv/lib/python3.10/site-packages/importlib_resources/__init__.py)
*Kyriakos Terzopoulos *Web developer / e-learning expert
*Tel:*+30 211 213 9858
*Mobile:* +30 694 526 4512
- E-mail: *kyriakos.terzopoulos@gmail.com
- Skype:* kyriakos.terzopoulos Find me on Facebook <http://www.facebook.com/cirrus3d> Follow me on Twitter <http://twitter.com/#%21/cirrus3d>
On Tue, 3 Oct 2023 at 12:52, Odhiambo Washington <odhiambo@gmail.com> wrote:
On Tue, Oct 3, 2023 at 10:36 AM Kyriakos Terzopoulos < kyriakos.terzopoulos@gmail.com> wrote:
Hi Stephen, all,
Firstly, thank you for your kind support. It seems that there is some misunderstanding there from my side, since I did not use the github repo to install mailman - I did it through pip install.
I will try to set it up with github and come back with any issues.
Not sure what you mean by github, but it will be a lot easier to follow this guide:
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
On Tue, Oct 3, 2023 at 2:12 PM Kyriakos Terzopoulos < kyriakos.terzopoulos@gmail.com> wrote:
Hi Odhiambo,
Installing this way I get the following error (Note that I have installed all dependencies):
(venv) mailman@hestia:~$ mailman info
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman", line 33, in <module>
sys.exit(load_entry_point('mailman==3.3.8', 'console_scripts','mailman')())
File "/opt/mailman/venv/bin/mailman", line 25, in importlib_load_entry_point
return next(matches).load()File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 23, in <module>
from mailman.config import configFile "/opt/mailman/venv/lib/python3.10/site-packages/mailman/config/__init__.py", line 20, in <module>
from mailman.config.config import ConfigurationFile "/opt/mailman/venv/lib/python3.10/site-packages/mailman/config/config.py", line 27, in <module>
from importlib_resources import path, read_textImportError: cannot import name 'path' from 'importlib_resources' (/opt/mailman/venv/lib/python3.10/site-packages/importlib_resources/__init__.py)
Looks like this is related to this issue: https://gitlab.com/mailman/mailman/-/issues/1093 Did you really start afresh?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Yes - I completely removed the *venv* directory after deactivating it and started from scratch.
However, I cannot find the mailman/setup.py anywhere in order to patch it, since the installation was done through pip from here: https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
Maybe I can try upgrading python from 3.10 to 3.11 and re-install
*Kyriakos Terzopoulos*
Web developer / e-learning expert
*Tel:*+30 211 213 9858
*Mobile:* +30 694 526 4512
- E-mail: *kyriakos.terzopoulos@gmail.com
- Skype:* kyriakos.terzopoulos Find me on Facebook <http://www.facebook.com/cirrus3d> Follow me on Twitter <http://twitter.com/#%21/cirrus3d>
On Tue, 3 Oct 2023 at 15:56, Odhiambo Washington <odhiambo@gmail.com> wrote:
On Tue, Oct 3, 2023 at 2:12 PM Kyriakos Terzopoulos < kyriakos.terzopoulos@gmail.com> wrote:
Hi Odhiambo,
Installing this way I get the following error (Note that I have installed all dependencies):
(venv) mailman@hestia:~$ mailman info
Traceback (most recent call last):
File "/opt/mailman/venv/bin/mailman", line 33, in <module>
sys.exit(load_entry_point('mailman==3.3.8', 'console_scripts','mailman')())
File "/opt/mailman/venv/bin/mailman", line 25, in importlib_load_entry_point
return next(matches).load()File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
module = import_module(match.group('module'))File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/bin/mailman.py", line 23, in <module>
from mailman.config import configFile "/opt/mailman/venv/lib/python3.10/site-packages/mailman/config/__init__.py", line 20, in <module>
from mailman.config.config import ConfigurationFile "/opt/mailman/venv/lib/python3.10/site-packages/mailman/config/config.py", line 27, in <module>
from importlib_resources import path, read_textImportError: cannot import name 'path' from 'importlib_resources' (/opt/mailman/venv/lib/python3.10/site-packages/importlib_resources/__init__.py)
Looks like this is related to this issue: https://gitlab.com/mailman/mailman/-/issues/1093 Did you really start afresh?
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
On 10/3/23 06:00, Kyriakos Terzopoulos wrote:
However, I cannot find the mailman/setup.py anywhere in order to patch it, since the installation was done through pip from here: https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
Maybe I can try upgrading python from 3.10 to 3.11 and re-install
Upgrading Python won't help. There is currently an incompatibility issue
when following
https://docs.mailman3.org/en/latest/install/virtualenv.html because that
doc is based on the head of the GitLab branch, but pip install mailman3 installs the 3.3.8 released version of core and the latest
6.1.0 version of importlib_resources and these are incompatible.
There are two ways to fix this.
The easy way is to downgrade importlib_resources in your venv with
pip install importlib_resources\<6.0
The slightly more complex way is to upgrade mailman core to 3.3.9b1 from GitLab with
pip install -U git+https://gitlab.com/mailman/mailman.git
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On Wed, Oct 4, 2023 at 3:45 AM Mark Sapiro <mark@msapiro.net> wrote:
On 10/3/23 06:00, Kyriakos Terzopoulos wrote:
However, I cannot find the mailman/setup.py anywhere in order to patch
it,
since the installation was done through pip from here:
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
Maybe I can try upgrading python from 3.10 to 3.11 and re-install
Upgrading Python won't help. There is currently an incompatibility issue when following https://docs.mailman3.org/en/latest/install/virtualenv.html because that doc is based on the head of the GitLab branch, but
pip install mailman3installs the 3.3.8 released version of core and the latest 6.1.0 version of importlib_resources and these are incompatible.There are two ways to fix this.
The easy way is to downgrade importlib_resources in your venv with
pip install importlib_resources\<6.0The slightly more complex way is to upgrade mailman core to 3.3.9b1 from GitLab with
pip install -U git+https://gitlab.com/mailman/mailman.git
And I think the 'slightly more complex way' is a better choice as it lets one not get stuck with old software versions :)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
Odhiambo Washington writes:
And I think the 'slightly more complex way' is a better choice as it lets one not get stuck with old software versions :)
It may also cause other dependency problems, though. Unless you know what you're doing enough to be willing to try to diagnose and handle them yourself, we'd generally prefer you stick to the currently released Mailman suite.
I guess Mark's recommendation signals that he believes that 3.3.9b1 is pretty much ready for prime time, so in this particular case it's a reasonable option. I wouldn't say it's "better" though.
Steve
Thanks Mark,
That seemed to work:
(venv) mailman@hestia:~/venv$ mailman info GNU Mailman 3.3.8 (Tom Sawyer) Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] config file: /etc/mailman3/mailman.cfg db url: postgresql://mailman:XXXXXXXXXXXXX>@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ REST credentials: XXXXX:XXXXXX
Now, off to handle other issues like nginx, exim4, etc. I will search or create another thread for this if I'm stuck. It would be nice to somehow update the documentation for https://docs.mailman3.org/en/latest/install/virtualenv.html.
Thank you all for your efforts.
*Kyriakos*
On Wed, 4 Oct 2023 at 03:44, Mark Sapiro <mark@msapiro.net> wrote:
On 10/3/23 06:00, Kyriakos Terzopoulos wrote:
However, I cannot find the mailman/setup.py anywhere in order to patch
it,
since the installation was done through pip from here:
https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-insta...
Maybe I can try upgrading python from 3.10 to 3.11 and re-install
Upgrading Python won't help. There is currently an incompatibility issue when following https://docs.mailman3.org/en/latest/install/virtualenv.html because that doc is based on the head of the GitLab branch, but
pip install mailman3installs the 3.3.8 released version of core and the latest 6.1.0 version of importlib_resources and these are incompatible.There are two ways to fix this.
The easy way is to downgrade importlib_resources in your venv with
pip install importlib_resources\<6.0The slightly more complex way is to upgrade mailman core to 3.3.9b1 from GitLab with
pip install -U git+https://gitlab.com/mailman/mailman.git-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ Archived at: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
This message sent to kyriakos.terzopoulos@gmail.com
Kyriakos Terzopoulos writes:
Firstly, thank you for your kind support. It seems that there is some misunderstanding there from my side, since I did not use the github repo to install mailman - I did it through pip install.
Right, I understood that. The point is you can create a requirements.txt file by using "python -m pip freeze > requirements.txt", edit that file as suggested in the GitLab issue, and run "python -m pip install -r requirements.txt". I prefer this method because it documents exactly what I had in the venv at any given point in time.
I think that should only reinstall the package with the new constraint, but it may refuse to do any installation because everything is already installed. In that case you have to either use "--force-reinstall" in the pip command (which will reinstall *all* packages) or you can use "python -m pip remove importlib_resources" followed by "python -m pip install 'importlib_resources>=1.1.0,<6.0'".
Steve
This worked for me. Now my mailman3.service is up and running.
I created a _requirements.in_ file with the following content: mailman flufl.lock<8.0 importlib-resources<6.0 psycopg2-binary<2.9 flufl.i18n==3.2
By using pip-tools, you can compile this file and get the requirements.txt
participants (5)
-
Kyriakos Terzopoulos -
Mark Sapiro -
Odhiambo Washington -
Rami Boutassghount -
Stephen J. Turnbull