Upgrade Mailmam Core from
Hello,
I have newly set up Mailman 3 on Ubuntu 20.04. Sending and receiving mails between list members does work, but I am currently experiencing two issues:
- a lists non-members who send mails to the list don't get notified about the message being held for moderation. Moderation itself doesn't seem to work. The moderation list is always empty.
- Archiving is activated within list settings, but there isn't any archive created.
I found this thread, where the first of these issues is also described: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/K...
Here there is an incompatibility between Mailmain 3 and Python depicted, which also seems to affect me. I am running Mailman 3.2.2 on Python 3 .8.10.
So I need to either upgrade Mailman Core or downgrade Python. I already did
apt-get upgrade apt-get update Unfortunately this didn't upgrade Mailman Core's version.
Which steps do I have to undertake to upgrade Mailman Core?
Regards, Sascha.
On 9/24/22 10:57, sascha.rissel@gmail.com wrote:
Hello,
I have newly set up Mailman 3 on Ubuntu 20.04. Sending and receiving mails between list members does work, but I am currently experiencing two issues:
- a lists non-members who send mails to the list don't get notified about the message being held for moderation. Moderation itself doesn't seem to work. The moderation list is always empty.
- Archiving is activated within list settings, but there isn't any archive created.
I found this thread, where the first of these issues is also described: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/K...
Here there is an incompatibility between Mailmain 3 and Python depicted, which also seems to affect me. I am running Mailman 3.2.2 on Python 3 .8.10.
This is only one of the issues with the Debian/Ubuntu packages.
So I need to either upgrade Mailman Core or downgrade Python. I already did
apt-get upgrade apt-get update Unfortunately this didn't upgrade Mailman Core's version.
Which steps do I have to undertake to upgrade Mailman Core?
I suggest first removing the Debian/Ubuntu packages and installing the PyPI packages following docs at https://docs.mailman3.org/en/latest/install/virtualenv.html. There are also other takes on this at https://wiki.list.org/x/17891998; https://wiki.list.org/x/17892066 and https://docs.google.com/document/d/1xIcSsoNFp2nHi7r4eQys00s9a0k2sHhu1V5Plant...
This will require some effort, but you will be freed from dependence on out of date packages.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Thanks, Mark!
I am just following the Virtualenv Installation, up to the point when I run
pip install wheel mailman psycopg2-binary\<2.9 from within virtualenv.
This command runs a little time, eventually ending with (the following is printed about 5 or 6 times): ERROR: Command errored out with exit status 1: command: /opt/mailman/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rdlaw8e1/lazr.delegates/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rdlaw8e1/lazr.delegates/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-049shc9h cwd: /tmp/pip-install-rdlaw8e1/lazr.delegates/ Complete output (6 lines): usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: invalid command 'bdist_wheel'
ERROR: Failed building wheel for lazr.delegates
Running setup.py clean for lazr.delegates
Failed to build mailman authheaders authres lazr.config dkimpy lazr.delegates
Installing collected packages: wheel, atpublic, attrs, aiosmtpd, six, python-dateutil, python-editor, sqlalchemy, MarkupSafe, Mako, alembic, authres, dnspython, dkimpy, publicsuffix2, authheaders, click, falcon, zope.interface, flufl.bounce, flufl.i18n, psutil, flufl.lock, gunicorn, zipp, importlib-resources, nose, lazr.delegates, lazr.config, passlib, urllib3, certifi, idna, charset-normalizer, requests, zope.event, zope.hookable, zope.component, zope.i18nmessageid, zope.schema, zope.configuration, mailman, psycopg2-binary
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/opt/mailman/venv/lib/python3.8/site-packages/wheel-0.37.1.dist-info'
Consider using the --user
option or check the permissions.
Did I miss to grant some permissions to the "mailman" user/group?
I just noticed that I forgot to run
sudo su mailman before activating virtualenv.
Now I did this. Mailman installation process runs to the end, not yielding "permission denied", but still printing out 5 times: ERROR: Command errored out with exit status 1: command: /opt/mailman/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-i1bp8cl9/dkimpy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-i1bp8cl9/dkimpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-liw7r9lt cwd: /tmp/pip-install-i1bp8cl9/dkimpy/ Complete output (6 lines): usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: invalid command 'bdist_wheel'
ERROR: Failed building wheel for dkimpy Running setup.py clean for dkimpy Building wheel for lazr.delegates (setup.py) ... error ERROR: Command errored out with exit status 1: command: /opt/mailman/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-i1bp8cl9/lazr.delegates/setup.py'"'"'; __file__='"'"'/tmp/pip-install-i1bp8cl9/lazr.delegates/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-f6q_xexm cwd: /tmp/pip-install-i1bp8cl9/lazr.delegates/ Complete output (6 lines): usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: invalid command 'bdist_wheel'
ERROR: Failed building wheel for lazr.delegates
Nevertheless it ends with:
Successfully installed Mako-1.2.3 MarkupSafe-2.1.1 aiosmtpd-1.4.2 alembic-1.6.5 atpublic-3.1.1 attrs-22.1.0 authheaders-0.15.1 authres-1.2.0 certifi-2022.9.24 charset-normalizer-2.1.1 click-8.1.3 dkimpy-1.0.5 dnspython-2.2.1 falcon-3.1.0 flufl.bounce-4.0 flufl.i18n-4.1.1 flufl.lock-7.1.1 gunicorn-20.1.0 idna-3.4 importlib-resources-5.9.0 lazr.config-2.2.3 lazr.delegates-2.0.4 mailman-3.3.5 nose-1.3.7 passlib-1.7.4 psutil-5.9.2 psycopg2-binary-2.8.6 publicsuffix2-2.20191221 python-dateutil-2.8.2 python-editor-1.0.4 requests-2.28.1 six-1.16.0 sqlalchemy-1.3.24 urllib3-1.26.12 wheel-0.37.1 zipp-3.8.1 zope.component-5.0.1 zope.configuration-4.4.1 zope.event-4.5.0 zope.hookable-5.2 zope.i18nmessageid-5.0.1 zope.interface-5.4.0 zope.schema-6.2.1
I suppose there are some packages missing, probably the one it was not able to build.
Thanks, Steven. I did run
pip install wheel additionally.
Then I uninstalled the mailman packages, via running
pip uninstall wheel mailman psycopg2-binary\<2.9 to be able to begin anew.
Now I run
pip install wheel mailman psycopg2-binary\<2.9
It runs through but still prints out this one error:
ERROR: Command errored out with exit status 1: command: /opt/mailman/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-rnm7lm08/mailman/setup.py'"'"'; __file__='"'"'/tmp/pip-install-rnm7lm08/mailman/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-jky3c7_5 cwd: /tmp/pip-install-rnm7lm08/mailman/ Complete output (6 lines): usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help
error: invalid command 'bdist_wheel'
ERROR: Failed building wheel for mailman
For the moment, I ignored this error error: invalid command 'bdist_wheel'
ERROR: Failed building wheel for mailman
and carried on, with creating a configuration file at /etc/mailman3/mailman.cfg for Mailman Core. I notice that directory /etc/mailman3 has not been creating during mailman setup from with venv.
Is this correct? Shall I manually create this directory or persist mailman.cfg at some other location where system user mailman has permissions to?
On 9/27/22 05:42, sascha.rissel@gmail.com wrote:
For the moment, I ignored this error error: invalid command 'bdist_wheel'
ERROR: Failed building wheel for mailman
and carried on, with creating a configuration file at /etc/mailman3/mailman.cfg for Mailman Core. I notice that directory /etc/mailman3 has not been creating during mailman setup from with venv.
Is this correct? Shall I manually create this directory or persist mailman.cfg at some other location where system user mailman has permissions to?
I suggest you remove the venv and start over ensuring you do sudo su mailman
before creating the venv.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hi all,
as Mark suggested, I delete the venv and started anew, making sure that I do
sudo su mailman before creating the venv again.
Executing
pip install wheel mailman psycopg2-binary\<2.9 led to the same error message as in my post https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
saying
error: invalid command 'bdist_wheel' six times, namely for: ERROR: Failed building wheel for mailman ERROR: Failed building wheel for authheaders ERROR: Failed building wheel for authres ERROR: Failed building wheel for lazr.config ERROR: Failed building wheel for dkimpy ERROR: Failed building wheel for lazr.delegates
pip install wheel
After this, I deleted the venv again and startet anew, again. But this time, I explicitly did (as Stephen suggested above) prior to
pip install wheel mailman psycopg2-binary\<2.9
This time, it ran through without errors, although "wheel" is also contained in the latter command. Can you explain that?
Second question: There is still no directory "/etc/mailman3" created, where I can create mailman.cfg in. Does this maybe result from the fact that in first instance, I tried to install mailman3-full via "apt-get install" and removed it again via "apt-get remove"; with manually deleting /etc/mailman3 afterwards.
Shall I create /etc/mailman3 manually and adding "mailman" system user permissions?
In advance, thanks again for your help!
On Thu, Sep 29, 2022 at 9:18 AM <sascha.rissel@gmail.com> wrote:
Hi all,
as Mark suggested, I delete the venv and started anew, making sure that I do
sudo su mailman before creating the venv again.
Executing
pip install wheel mailman psycopg2-binary\<2.9 led to the same error message as in my post https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message/...
saying
error: invalid command 'bdist_wheel' six times, namely for: ERROR: Failed building wheel for mailman ERROR: Failed building wheel for authheaders ERROR: Failed building wheel for authres ERROR: Failed building wheel for lazr.config ERROR: Failed building wheel for dkimpy ERROR: Failed building wheel for lazr.delegates
pip install wheel
After this, I deleted the venv again and startet anew, again. But this time, I explicitly did (as Stephen suggested above) prior to
pip install wheel mailman psycopg2-binary\<2.9
This time, it ran through without errors, although "wheel" is also contained in the latter command. Can you explain that?
Second question: There is still no directory "/etc/mailman3" created, where I can create mailman.cfg in. Does this maybe result from the fact that in first instance, I tried to install mailman3-full via "apt-get install" and removed it again via "apt-get remove"; with manually deleting /etc/mailman3 afterwards.
Shall I create /etc/mailman3 manually and adding "mailman" system user permissions?
In advance, thanks again for your help
Can you possibly blow away everything and start afresh again - using the following howto - https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 ? And IF you get stuck, feel free to ping me even offlist. PS: I don't use Postfix and Nginx so I will not be able to assist you with those two. However, I will do my best to hand-hold you on the rest, as has been done to me on this list too.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-)
Odhiambo Washington writes:
Can you possibly blow away everything and start afresh again - using the following howto - https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10 ?
I appreciate your efforts to help, and with a different set of facts your approach *would* be helpful. So don't stop! But please be more careful. Sascha already has successfully installed Mailman (in fact more times than they know :-), has experience with this particular approach to installation, and was already interacting with two core developers. Starting from scratch at this point doesn't make sense.
After this, I deleted the venv again and startet anew, again.
Unknown to Sascha, this venv was already a successful install. The error messages can be ignored.
pip install wheel
But this time, I explicitly did (as Stephen suggested above) prior to
pip install wheel mailman psycopg2-binary\<2.9
This time, it ran through without errors, although "wheel" is also contained in the latter command. Can you explain that?
Note: this second run is also successful.
Explanation: "Failed to build wheel" is actually not an error. *wheels* are the preferred packaging format for Python for several years now, but others are available, and pip used one of those instead. I suspect the "Error" tag comes from a separate external command (bdist_wheel), but pip reads the error status and tries an alternative method to build a package.
Sascha: I don't understand why pip doesn't install wheel before starting on mailman in "pip wheel mailman psycopg2-binary\<2.9. I think that's a bug, because this is an obvious way to use pip. Perhaps you have an older version of pip? Have you tried pip install --upgrade pip?
Second question: There is still no directory "/etc/mailman3" created, where I can create mailman.cfg in.
That's right. There are a number of different philosophies about handling files like mailman.cfg, and the Mailman install scripts can't be sure which one you want to follow. /etc/mailman3/mailman.cfg probably works for 90% or 95% of installations nowadays, but we don't know. If you're not going to put mailman.cfg there, we shouldn't create the directory.
Does this maybe result from the fact that in first instance, I tried to install mailman3-full via "apt-get install" and removed it again via "apt-get remove"; with manually deleting /etc/mailman3 afterwards.
That's what I consider correct behavior as implemented by apt.
Shall I create /etc/mailman3 manually and adding "mailman" system user permissions?
Mark wrote earlier, but let me confirm that yes, that's what you should do. Note that Debian and Ubuntu prefer to gather all possible mailing list users into a single user 'list'. I don't think this causes trouble for Mailman, but in many cases if you create a different user (as you will if you follow our instructions), you will have problems if the system-created directory with system-standard owner and permissions is left behind. This is one of several reasons why you should purge (not just delete) packages supplied by distros if you are going to switch to upstream's code. (Obviously keep copies!)
Regards, Steve
Stephen, many thanks for your precise answer. I am still working on the installation process, having paused it for the last two or three days. I am currently busy with configuring hyperkitty, mailman-web and so on.
I wonder a bit how deep I have to dive into these individual configuration files. Is there some minimal config for a standard Ubuntu/Debian system?
Regards, Sascha.
Sascha Rissel writes:
I wonder a bit how deep I have to dive into these individual configuration files. Is there some minimal config for a standard Ubuntu/Debian system?
I think that if you follow the "venv" process as described in <https://docs.mailman3.org/en/latest/install/virtualenv.html?>, it covers everything you must configure. There are about a dozen places that you must check in mailman.cfg, such as the location of your MTA and its port (for the simplest installations, host=127.0.0.1, port=25 will do), mailman's LMTP port (usually the default of 8024 will do), and the credentials for accessing the REST interface (restadmin and restpass: change these!!) Pretty sure there are others. There are quite a few decisions that need to be made in mailman-web settings.py, including setting the REST credentials to the ones you chose in mailman.cfg, and setting up your database (sqlite3 is great for quick experiments or tiny installations with 10 posts/year ;-). And I don't think there are any MTAs or webservers that are plug and play with Mailman: you have to configure the MTA to forward mailman list addresses to Mailman's LMTP port, and usually there's nontrivial configuration of the namespaces that are to be forwarded to Mailman/Django and those that are served directly (such as CSS style files) by the webserver. Django also has its own access credentials that must be coordinated with mailman.cfg I think.
So "minimal" is still pretty complex.
While I would be *extremely* cautious about using any *values* for parameters that are specified in Brian Carpenter's HOWTO <https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10> (this is the one that Odhiambo suggested, and it's a really valuable resource), I'm pretty sure it accurately covers all the steps you need to take in configuration. You may find it easier to read than our manual. Going through that and checking your venv configuration against the steps in the HOWTO would be a good way to confirm that you've got a good configuration.
On 9/28/22 23:18, sascha.rissel@gmail.com wrote:
pip install wheel
After this, I deleted the venv again and startet anew, again. But this time, I explicitly did (as Stephen suggested above) prior to
pip install wheel mailman psycopg2-binary\<2.9
This time, it ran through without errors, although "wheel" is also contained in the latter command. Can you explain that?
Apparently, if you run pip install wheel mailman psycopg2-binary\<2.9
pip attempts to build lazr.delegates
which is a mailman
dependency
before it fully installs wheel
. Thus the need to pip install wheel
first.
However, with both virtualenv 20.0.17/python 3.8.10
and virtualenv 20.4.2/python 3.9.10
virtualenv venv
installs wheel
in
site-packages in the venv as does python3 -m virtualenv venv
, but
python3 -m venv venv
does not. But, at least in my case, with venv
built with python3 -m venv venv
and not containing wheel
in
site-packages
pip install wheel mailman psycopg2-binary\<2.9
runs
without error.
Second question: There is still no directory "/etc/mailman3" created, where I can create mailman.cfg in.
Does this maybe result from the fact that in first instance, I tried to install mailman3-full via "apt-get install" and removed it again via "apt-get remove"; with manually deleting /etc/mailman3 afterwards.
Shall I create /etc/mailman3 manually and adding "mailman" system user permissions?
Yes, as the doc says at https://docs.mailman3.org/en/latest/install/virtualenv.html#installing-mailm..., "After this, create a configuration file at /etc/mailman3/mailman.cfg for Mailman Core:"
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
participants (5)
-
Mark Sapiro
-
Odhiambo Washington
-
Sascha Rissel
-
sascha.rissel@gmail.com
-
Stephen J. Turnbull