Managing mailman3 via curl (using bash-scripts or perl-scripts)
Hello Mark, hello Stephen,
sorry for bothering again...but the amount of documentation is overwhelming... Some info about my setup: I run mailman3 in an virtual environment with postgresql-Server (all in recent versions) - operating system Ubuntu 24.04-LTS.
My aim is to provide one high-level script for the daily task of list creation.
- list creation via command-line script with two/three parameters like: ./create-list --listname myfirst.list@lists.example.org --admin-email admin-user@example.org [--admin-name "Peter Rabbit"]
The create-list script itself should be a bash- or perl-script (I am not very familiar with python - SORRY ;-) ) I would like the scripts to trigger some curl-calls on the mailman server (from outside the mailman shell and even from outside the virtual environment) Reading the RestAPI documentation here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/u... several questions arise:
- Can I succeed with this plan? (Are curl-calls sufficient? Is it possible from outside the virtual environment?)
- Is this a viable curl-call to create a user?: curl -X POST -u 'admin-user:<password>' -H "Content-Type: application/json" -d '{ "email": "peter.rabbit@example.org", "display_name": "Peter Rabbit", "password": "secret" }' http://localhost:8001/3.1/users
- and in a second step can I assign this user to a list as owner with a curl-call like this?: curl -X POST -u 'admin-user:<password>' -H "Content-Type: application/json" -d '{ "list_id": "newsletter.lists.example.org", "subscriber": "peter.rabbit@example.org", "role": "owner" }' http://localhost:8001/3.1/members
- What about the commits that are used while operating in mailman shell? (Do I have to trigger commits while operating via curl-calls? And how can I trigger commits via curl?)
- In the unlikely event that I change from the dark side to the good side i.e. "python programming" ;-) is there a "to-the-point" reference for IListManager, IUserManager etc. giving available functions (with parameters) in a kind of list way? (the example kind of documentation referenced above is good, but where can I find a list of all functions?)
- I think the starting point for managing lists with python is here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/commands/do... right?
- Going the python way will require to enter the virtual environment, right? (otherwise the libraries, environment variables etc. are not available...I think)
Thank you for some guidance here... :-)
Kind regards
Chris
On 2025-09-29 13:05:27 -0000 (-0000), Christian Schneider wrote: [...]
- list creation via command-line script with two/three parameters like: ./create-list --listname myfirst.list@lists.example.org --admin-email admin-user@example.org [--admin-name "Peter Rabbit"]
The create-list script itself should be a bash- or perl-script (I am not very familiar with python - SORRY ;-) ) I would like the scripts to trigger some curl-calls on the mailman server (from outside the mailman shell and even from outside the virtual environment) [...]
We're creating lists using Ansible to make calls to Mailman's REST API, but it should be trivial to translate this example to any scripting language of your choice:
https://opendev.org/opendev/system-config/src/commit/79fc523/playbooks/roles...
Hopefully that gets you on the right track, but if you do decide to delve into Python you'll have a lot more flexibility since the existing Mailman CLI is essentially a Python module.
Note that "activating" a venv is typically not necessary (I never do it), you just have to make sure to type the full path to executables in the bin directory of the venv when running them, or add that directory to PATH in your shell environment.
Jeremy Stanley
Jeremy Stanley writes:
Note that "activating" a venv is typically not necessary (I never do it), you just have to make sure to type the full path to executables in the bin directory of the venv when running them, or add that directory to PATH in your shell environment.
I don't recommend the latter to folks who aren't experienced with Python. It can be confusing if you get a different Python from the one you expect due to upgrades/installations of other Python versions. Typing full paths or creating shell aliases/functions containing the full path is safe.
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
On 2025-09-30 00:15:09 +0900 (+0900), Stephen J. Turnbull wrote:
Jeremy Stanley writes:
Note that "activating" a venv is typically not necessary (I never do it), you just have to make sure to type the full path to executables in the bin directory of the venv when running them, or add that directory to PATH in your shell environment.
I don't recommend the latter to folks who aren't experienced with Python. It can be confusing if you get a different Python from the one you expect due to upgrades/installations of other Python versions. Typing full paths or creating shell aliases/functions containing the full path is safe.
Yes, I should have included the caveat that adding a venv's bin/
directory to your PATH should only be for temporary measure or e.g.
if you have a dedicated account you want to log into specifically
for running those commands. This is part of what the activate script
in a venv does too, FWIW, it just gets undone again by the
deactivate command if/when you remember to run it later.
While understanding executable search order helps avoid accidentally shadowing other commands like your Python interpreter executable (append vs prepend), obviously altering your shell's PATH should not be done unless you fully comprehend the risks, which is why I suggested using the full path to the executable as the first recommendation.
A middle ground I take between those two is to add ~/bin to my PATH and then populate it with symlinks to executables in various venvs that contain my Python-based tools. This gives me the flexibility to only include the specific executables I want in my command search while omitting the rest.
Jeremy Stanley
Christian Schneider writes:
My aim is to provide one high-level script for the daily task of list creation.
I'll try to get you started. Mark is likely to have comments or improvements.
If you're on-host, list creation can be done with the 'mailman' utility. The point of the REST API is to allow "arms length" operation by arbitrary software, not necessarily on the same host.
The create-list script itself should be a bash- or perl-script (I am not very familiar with python - SORRY ;-) )
The REST API is script-language-agnostic. It's a bit clumsy compared to using the mailmanclient Python API, but using the REST API, any scripting or programming language would suffer from that.
I would like the scripts to trigger some curl-calls on the mailman server (from outside the mailman shell and even from outside the virtual environment)
It's your call, but are you sure the commands provided by the 'mailman' utility don't do what you need?
- Can I succeed with this plan? (Are curl-calls sufficient? Is it possible from outside the virtual environment?)
Yes, you can do anything using curl that the REST API can do. The virtual environment is only relevant to controlling Mailman from Python. If you are using the REST API, you need a running Mailman, which will already be running in the correct virtual environment.
However, calling curl from a script is very clumsy. You will need to trap errors such as "user exists" and so on. You probably need to parse the response as well as check the HTTP status.
- Is this a viable curl-call to create a user?: curl -X POST -u 'admin-user:<password>' -H "Content-Type: application/json" -d '{ "email": "peter.rabbit@example.org", "display_name": "Peter Rabbit", "password": "secret" }' http://localhost:8001/3.1/users
The "admin-user" in this command is the "restadmin" user, not the list-owner. I think in the 'create-list' command you meant the --admin-user to be the list owner.
Also, the "password" attribute is not very useful in Mailman 3. The only authorized user of the REST API is the restadmin user. There is no provision for access by other users. To authenticate other users, you will need to provide an alternative authentication database, such as .htpasswd validated by a front-end such as Apache. In a full Mailman suite installation, administrative and subscriber authentication is managed by Django, not by Mailman core.
- and in a second step can I assign this user to a list as owner with a curl-call like this?: curl -X POST -u 'admin-user:<password>' -H "Content-Type: application/json" -d '{ "list_id": "newsletter.lists.example.org", "subscriber": "peter.rabbit@example.org", "role": "owner" }' http://localhost:8001/3.1/members
Yes.
- What about the commits that are used while operating in mailman shell?
Not relevant. "mailman shell" is intended for experimentation, so an explicit commit function is provided to trigger a backend database commit. This allows you to fix typos before committing, or even to exit without changing the database if you think you made a big mistake.
In the REST API, each mutating REST call is a single database transaction and the commit is triggered automatically. If you have a multiple call script and something goes wrong in a later call, you will have to unwind any calls that succeeded by hand.
- In the unlikely event that I change from the dark side to the good side i.e. "python programming" ;-) is there a "to-the-point" reference for IListManager, IUserManager etc. giving available functions (with parameters) in a kind of list way? (the example kind of documentation referenced above is good, but where can I find a list of all functions?)
No. https://docs.mailman3.org/projects/mailman/en/release-3.1/src/mailman/rest/d... is fairly complete, but the only truly complete lists are in the source. You can also go into the Mailman shell, import the relevant modules and use the help() function.
- I think the starting point for managing lists with python is here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/commands/do... right?
Yes.
- Going the python way will require to enter the virtual environment, right? (otherwise the libraries, environment variables etc. are not available...I think)
For development, yes. It would be very annoying. However, in production, you would most likely use "mailman shell -r script". In that case you can invoke the script with the full path "/path/to/venv/bin/mailman" and place "script" in the venv/bin directory. This will automatically invoke the Python portion of the virtual environment (eg, sys.path will be set appropriately), but not the shell environment. Then you can use a shell script or function to abbreviate the command.
-- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan
Hello,
thank you very much to everyone who took part in this discussion!
I got a very interesting insight!
@Jeremy: The ansible approach is fabulous! My team plans to go the ansible way for various automation and configuration tasks! So this is a good perspective for the future! And with your link to the ansible file I am prepared for the change to come... :-) Also the remark on using the venv installed binaries, libraries etc. from outside is very helpful.
@Stephen: Thank you for your warning! I will keep it in mind while doing such kind of operations from outside the environment. (I am familiar with the shell and shell variables like PATH, LIBRARY_PATH etc...so I hope I can get along... :-))
Your comments on my curl-command are right and this is due to my "anonymization"-effort:
Is this a viable curl-call to create a user?: curl -X POST -u 'admin-user:<password>' -H "Content-Type: application/json" -d '{ "email": "peter.rabbit@example.org", "display_name": "Peter Rabbit", "password": "secret" }' http://localhost:8001/3.1/users
The "admin-user" in this command is the "restadmin" user, not the list-owner. I think in the 'create-list' command you meant the --admin-user to be the list owner. Also, the "password" attribute is not very useful in Mailman 3. The only authorized user of the REST API is the restadmin user. There is no provision for access by other users. To authenticate other users, I should have used the alias "restadmin" and the password "restadmin-password" for the "-u"-parameter to be more precise...so the curl-call should look like this:
curl -X POST -u 'restadmin:restadmin-password' -H "Content-Type: application/json" -d '{ "email": "peter.rabbit@example.org", "display_name": "Peter Rabbit", "password": "secret" }' http://localhost:8001/3.1/users
And for describing the call of my high-level script I should have used: ./create-list --listname myfirst.list@lists.example.org --owner-email peter.rabbit@example.org [--owner-name "Peter Rabbit"]
Using python...I think your suggestion:
However, in production, you would most likely use "mailman shell -r script". is the recommended way according to the docs here: https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/commands/do...
Once again: Thank you very much to everyone for this enlightening and helpful discussion!
Kind regards
Chris
participants (3)
-
Christian Schneider -
Jeremy Stanley -
Stephen J. Turnbull