Re: mailman3 postorius cannot retrieve template
Dear Mark,
you wrote:
"When you create a template in Postorius, that uri becomes something like
https://example.com/mailman3/api/templates/list/<list_id>/<template_name>
which basically says get the template from Postorius, and the template if
any in var/templates is ignored."
In my case that is not true.
I get an db-entry in the mailman3 table template which doesnt lead to a suitable link: https://localhost/postorius/api/templates/list/testliste01.mydomain.de/list: member:regular:footer
In the next stept it leads to a flood of entries in my syslog because django is asking for a certificate match for "localhost": Nov 2 11:47:49 myserver mailman3[175334]: Nov 02 11:47:49 2021 (175334) Certificate did not match expected hostname: localhost. Certificate: ...
Letsencrypt cannot deliver this match for "localhost", it is impossible to configurate letsencrypt to do so.
The running script must create a postgres INSERT database command which includes the correct domain-name including the correct path .../mailman3/api/templates .... and not ... postorius/api/....
I dont know, where is the place to correct this. the [paths.debian] section
in mailman-web.py ?? ... ALLOWED_HOSTS ... '*' or MAILMAN_REST_API_URL = 'http://localhost:8001' or POSTORIUS_TEMPLATE_BASE_URL = 'https://localhost/mailman3/'
in mailman.cfg ??
the [webservice] section
- or elsewhere
Thanks in advance for your reply.
Wolfgang
-----Ursprüngliche Nachricht----- Von: Mark Sapiro mark@msapiro.net Gesendet: Sonntag, 31. Oktober 2021 19:15 An: mailman-users@mailman3.org Betreff: [MM3-users] Re: mailman3 postorius cannot retrieve template at ... (<no authorization>)
On 10/29/21 3:41 AM, Wolfgang Bock via Mailman-users wrote:
the question is: postgres - mailman3web-postorius_emailtemplate table vs mailman3 - templates table
I installed a brand new mailman3 with mailman Core 3.3.3, Api 3.1, Core Python 3.9.2 postgres12 nginx on debian bullseye.
I created a template via postorius -> lists -> (testliste01) -> templates -> new template -> [list:member:regular:footer] and [list:member:digest:footer]
They are shown in the postgres database mailman3 table templates: id name context uri username password 14 list:member:digest:footer testliste01.mydomain.de http://localhost/postorius/api/templates/list/testliste01.mydomain.de/ list:m ember:digest:footer 15 list:member:regular:footer testliste01.mydomain.de http://localhost/postorius/api/templates/list/testliste01.mydomain.de/ list:m ember:regular:footer
First there where no templates in the /var/lib/mailman3/templates folder, which was empty after installation.
This is expected. That is one place where you can put custom templates.
Later I created in /var/lib/mailman3/templates theses folders and the files /list/testliste01.mydomain.de/list:member:digest:footer + list:member:regular:footer owned by list:list 644
A testmail went through but instead of the default footer no footer was shown.
By default Mailman searches for templates first in the var/templates directory and then in the templates directory in the Mailman installation. The details are described at https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/utilities/i18n. py#L44.
However, the default search is only applied if there is matching entry for
the template name and context in the templates table in the database or if
the uri in the entry is a mailman:// uri.
When you create a template in Postorius, that uri becomes something like
https://example.com/mailman3/api/templates/list/<list_id>/<template_name>
which basically says get the template from Postorius, and the template if
any in var/templates is ignored.
In the sys.log I found the following error message: Oct 29 11:28:04 myserver mailman3[33015]: Message: 'Cannot retrieve template at http://localhost/postorius/api/templates/list/testliste01.mydomain.de/ list:m ember:regular:footer (<no authorization>)'
So Mailman is trying to get the Postorius template. What is in the web server logs for this retrieval?
How can I solve this problem? What kind of authorisation is meant? list? restadmin?
I read in https://docs.mailman3.org/en/latest/config-core.html#configure-templates ... list specific templates invar/templates/lists/LIST-ID/LC/ ... I guess the LC means Language Code. But in the database table templates no LC subdirectory is mentioned (in my case must be de).
If you delete the Postorius template, The entry in the templates table will be removed and the search will revert to var/templates, etc.
And I read (dont know where) .... The templates created in Postorius are created in the postorius_emailtemplate table in Mailman's database and referenced via URLs like
'https://example.com/mailman3/api/templates/list/<list-id>/<template-name>'.
I.e., they are not stored in the file system. ...
That's true. I found in postgres database mailman3web in the table mailman3web postorius_emailtemplate my
id name data language created_at modified_at context identifier 15 list:member:digest:footer postorius footer digest 1x linefeed 2021-10-29 11:17:40.664689+02 2021-10-29 11:26:48.692628+02 list testliste01.mydomain.de 16 list:member:regular:footer postorius footer non-digest 1x linefeed 2021-10-29 11:18:29.166241+02 2021-10-29 11:27:24.519024+02 list testliste01.mydomain.de
But my question is how to integrate this into the mailman3-system. The language isn't set either.
It is integrated. The language isn't set because you are getting the template from Postorius and Postorius knows the language.
On 10/31/21 4:13 AM, Wolfgang Bock via Mailman-users wrote:
regarding my mail before, here an additional information.
I can show the templates with following command (anonymized):
https://active_domain.de/postorius/api/templates/list/testliste01.active_dom ain.de/list:member:regular:footer
As it should be.
The complete syslog
(edited and reformatted by me)
> --- Logging error ---
> Traceback (most recent call last): Oct 30 11:49:52 active_server
mailman3[107630]: File
"/usr/lib/python3/dist-packages/mailman/model/template.py", line 110, in get
> contents = protocols.get(actual_uri, **auth)
> File
"/usr/lib/python3/dist-packages/mailman/utilities/protocols.py", line
39, in get
> response.raise_for_status()
> File "/usr/lib/python3/dist-packages/requests/models.py", line
943, in raise_for_status
> raise HTTPError(http_error_msg, response=self)
> requests.exceptions.HTTPError: 404 Client Error: Not Found for url:
http://localhost/postorius/api/templates/list/testliste01.active_domain.de/l
ist:member:regular:footer
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/lib/python3.9/logging/__init__.py", line 430, in format
> return self._format(record)
> File "/usr/lib/python3.9/logging/__init__.py", line 426, in _format
> return self._fmt % record.__dict__
> KeyError: 't'
> During handling of the above exception, another exception occurred:
Your template is malformed. You have $ strings that are not defined as
Mailman substitution variables. See
https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/t
emplates.html#templated-texts
--
Mark Sapiro <mark@msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
_______________________________________________
Mailman-users mailing list -- mailman-users@mailman3.org
To unsubscribe send an email to mailman-users-leave@mailman3.org
https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
On 11/2/21 3:28 PM, Wolfgang Bock via Mailman-users wrote:
I get an db-entry in the mailman3 table template which doesnt lead to a suitable link: https://localhost/postorius/api/templates/list/testliste01.mydomain.de/list: member:regular:footer
In the next stept it leads to a flood of entries in my syslog because django is asking for a certificate match for "localhost": Nov 2 11:47:49 myserver mailman3[175334]: Nov 02 11:47:49 2021 (175334) Certificate did not match expected hostname: localhost. Certificate: ...
Letsencrypt cannot deliver this match for "localhost", it is impossible to configurate letsencrypt to do so.
The running script must create a postgres INSERT database command which includes the correct domain-name including the correct path .../mailman3/api/templates .... and not ... postorius/api/....
I dont know, where is the place to correct this.
- in mailman-web.py ?? ... ALLOWED_HOSTS ... '*' or MAILMAN_REST_API_URL = 'http://localhost:8001' or POSTORIUS_TEMPLATE_BASE_URL = 'https://localhost/mailman3/'
If you normally access Postorius on your site at something like https://www.example.com/postorius, than set that as POSTORIUS_TEMPLATE_BASE_URL
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Hello Mark,
I reconstruct the configuration problems regarding the creation of templates in mm3:
I installaed mailman3 via the debian bullseye package(s).
BTW: My experience is to install it step by step, not as mailman3-full, but as mailman-3, mailman3-doc, mailman3-web and python3-django-mailman3. In my case the full installation did not run clean, the step-by-step variant run clean.
The debian bullseye variant comes with a nginx conf-file in /etc/mailman3/nginx.conf ... # This nginx config file is part of the mailman3-web package. # # This nginx configuration file is a vhost configuration. ... # Nginx SSL snippet. To enable it, please uncomment and update the server_name and the ... server { listen 443; listen [::]:443; server_name mailman.example.com; server_tokens off; # # ## Strong SSL Security # ## https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/ ssl on; ssl_certificate /etc/letsencrypt/live/mailman.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/mailman.example.com/privkey.pem; # ... The configuration "ssl on" is outdated as nginx -t comments.
The actual configuration should be:
/etc/nginx/ /etc/nginx/sites-available/nginx.lists.mydomain.ssl (and /sites-enabled) .... server { listen 443 ssl http2; listen [::]:443 ssl http2; server_name lists.mydomain.com; # for example server_tokens off;
ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;... If you - as proposed in the mailman3 nginx.conf - include in nginx.lists.mydomain.ssl the following: .... upstream mailman3 { server unix:/run/mailman3-web/uwsgi.sock fail_timeout=0; } .... you cannot in include this in another vhost-nginx configuration (for example): /etc/nginx/sites-available/nginx.lists.mydomain_2.ssl as nginx -t says something like "no double entries please".
So I put this in another file like /etc/nginx/sites-available/nginx.00.mailman3.upstream (and via symlink in .../enabled/...) and both nginx-domain-files work.
The debian bullseye also serves a file mailman-web.py with an entry ... POSTORIUS_TEMPLATE_BASE_URL = 'https://localhost/mailman3/' ... As mentioned in a mail before that could not work:
- letsencrypt doesnt serve an entry match for localhost. That floods your syslog.
- "localhost" doesnt match with the nginx.conf-proposal.
It must be something like /etc/mailman3/mailman-web.py .... POSTORIUS_TEMPLATE_BASE_URL = 'lists.mydomain.com/' ....
In this combination the creating of templates on the postorius website works.
My conclusion: I ask you to consider these experiences in an update for the mailman3 packages in debian bullseye. It would save time, questions and nerves.
Regards
Wolfgang
-----Ursprüngliche Nachricht----- Von: Mark Sapiro mark@msapiro.net Gesendet: Dienstag, 2. November 2021 23:44 An: mailman-users@mailman3.org Betreff: [MM3-users] Re: mailman3 postorius cannot retrieve template
On 11/2/21 3:28 PM, Wolfgang Bock via Mailman-users wrote:
I get an db-entry in the mailman3 table template which doesnt lead to a suitable link:
https://localhost/postorius/api/templates/list/testliste01.mydomain.de/list:
member:regular:footer
In the next stept it leads to a flood of entries in my syslog because django is asking for a certificate match for "localhost": Nov 2 11:47:49 myserver mailman3[175334]: Nov 02 11:47:49 2021 (175334) Certificate did not match expected hostname: localhost. Certificate: ...
Letsencrypt cannot deliver this match for "localhost", it is impossible to configurate letsencrypt to do so.
The running script must create a postgres INSERT database command which includes the correct domain-name including the correct path .../mailman3/api/templates .... and not ... postorius/api/....
I dont know, where is the place to correct this.
- in mailman-web.py ?? ... ALLOWED_HOSTS ... '*' or MAILMAN_REST_API_URL = 'http://localhost:8001' or POSTORIUS_TEMPLATE_BASE_URL = 'https://localhost/mailman3/'
If you normally access Postorius on your site at something like https://www.example.com/postorius, than set that as POSTORIUS_TEMPLATE_BASE_URL
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/
On 11/4/21 5:04 AM, Wolfgang Bock via Mailman-users wrote:
My conclusion: I ask you to consider these experiences in an update for the mailman3 packages in debian bullseye. It would save time, questions and nerves.
Issues with Debian packaging should be directed to Debian.
-- Mark Sapiro mark@msapiro.net The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
--On Thursday, November 4, 2021 8:25 AM -0700 Mark Sapiro mark@msapiro.net wrote:
On 11/4/21 5:04 AM, Wolfgang Bock via Mailman-users wrote:
My conclusion: I ask you to consider these experiences in an update for the mailman3 packages in debian bullseye. It would save time, questions and nerves.
Issues with Debian packaging should be directed to Debian.
Or as previously discussed, the debian packages should be avoided entirely.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
- Quanah Gibson-Mount (quanah@symas.com) [211104 22:59]:
--On Thursday, November 4, 2021 8:25 AM -0700 Mark Sapiro mark@msapiro.net wrote:
On 11/4/21 5:04 AM, Wolfgang Bock via Mailman-users wrote:
My conclusion: I ask you to consider these experiences in an update for the mailman3 packages in debian bullseye. It would save time, questions and nerves.
Issues with Debian packaging should be directed to Debian.
Or as previously discussed, the debian packages should be avoided entirely.
I strongly disagree with that statement.
Distribution packages have their own up- and downsides. Of course, how they are done is to be discussed with the distribution, so: if something is wrong with a debian package, please follow the instructions on bugs.debian.org on reporting bugs.
However, they have the advantage that they're integrated with the distribution. They have the disadvantage that they're stable for some time - that is why it's called a stable release. If they have all the features someone wants, they usually work fine (and my first mm3-installation was the default debian package). If one looks for bleeding edge, of course that's not what one can expect from a distribution package - but that's also a feature.
Andi
On Nov 4, 2021, at 3:05 PM, Andreas Barth aba@ayous.org wrote:
- Quanah Gibson-Mount (quanah@symas.com) [211104 22:59]:
--On Thursday, November 4, 2021 8:25 AM -0700 Mark Sapiro mark@msapiro.net wrote:
On 11/4/21 5:04 AM, Wolfgang Bock via Mailman-users wrote:
My conclusion: I ask you to consider these experiences in an update for the mailman3 packages in debian bullseye. It would save time, questions and nerves.
Issues with Debian packaging should be directed to Debian.
Or as previously discussed, the debian packages should be avoided entirely.
I strongly disagree with that statement.
Distribution packages have their own up- and downsides. Of course, how they are done is to be discussed with the distribution, so: if something is wrong with a debian package, please follow the instructions on bugs.debian.org on reporting bugs.
However, they have the advantage that they're integrated with the distribution. They have the disadvantage that they're stable for some time - that is why it's called a stable release. If they have all the features someone wants, they usually work fine (and my first mm3-installation was the default debian package). If one looks for bleeding edge, of course that's not what one can expect from a distribution package - but that's also a feature.
You left out the fact that the distro edition of mailman3 has very poor support because Debian doesn’t know how to do it and the setup is different enough from vanilla that this community has a hard time supporting it as well.
You can certainly try to run it, but I’m willing to bet that you’ll regret it.
Having tried it myself, I can’t in good conscience recommend it to anyone.
The bottom line is that, IMHO, distro works great for fairly mature projects. MM3 isn’t that yet.
YMMV, my $0.02, you get what you pay for, etc.
- Mark
mark@pdc-racing.net | 408-348-2878
Andreas Barth writes:
Distribution packages have their own up- and downsides.
I'm far from absolutist about distro packaging of Mailman, but historically, the downsides have very much outweighed the upsides for most users who come to us on these lists, unfortunately. It took many iterations of Mailman 2 at Debian to get to the point where I was willing to use the Debian packages at all, and for Mailman 3 I've never used them. Even Brian Carpenter's Install Guide avoids Debian's Mailman packages.
Of course, how they are done is to be discussed with the distribution, so: if something is wrong with a debian package, please follow the instructions on bugs.debian.org on reporting bugs.
This certainly is the definition of good citizenship. I hope all distribution users will file packaging bugs with their distros, and take it in good humor when we tell them, "works for me, so you should ask <distro> about it."
They have the disadvantage that they're stable for some time
If people have been using Mailman 2, they have had a rather better experience for the last few years, though. Since we only fix security issues in Mailman 2, Mailman itself is stable, and to the extent that I pay attention, I think Debian at least has been prompt about tracking Mailman 2 releases. Not so for Mailman 3, though.
If they have all the features someone wants,
But this is precisely the problem for Mailman 3. Mailman 3 is not feature-complete compared to Mailman 2, and even two years ago it wasn't close. (Sorry!) Yes, if all you want is a vanilla mail exploder, maybe with a footer added, Mailman 3 is great. But then there's no need to upgrade to Mailman 3 from Mailman 2, and if you're happy being distro-bound, you'll have a better experience with Mailman 2.
I'm sure there are quite a few Mailman 3 admins happy with their distro releases from 2 years or 6 months ago. But to someone who comes to us asking for advice, I am definitely going to recommend the Docker containers or a venv or /usr/local installation, because I expect that kind of user to be back for more advice in the future.
Steve
Teilnehmer (6)
-
Andreas Barth -
Mark Dadgar -
Mark Sapiro -
Quanah Gibson-Mount -
Stephen J. Turnbull -
Wolfgang Bock