Hi,
I'm unable to update the custom welcome template (list:user:notice:welcome) after it's initial creation.
When the custom welcome template is created (in Postorius), and a user then subscribed by the list-owner (using mass subscribe), the welcome message is received as expected.
However when I update the template, the databases are updated correctly but the Welcome message is still sent with the old template content.
mailman.templates mailmanweb.postorius_emailtemplate
Additionally, if I delete the template and create a new one the old content still gets sent.
I cannot find where this content is written or is being cached.
Any clues on tracking this bugbear down gratefully received.
Regards, Mark
$ mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] config file: /etc/mailman3/mailman.cfg db url: postgresql://******:******@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/
Installed with venv.
===================== Query to DB - mailman
SELECT URI FROM TEMPLATE WHERE NAME = 'list:user:notice:welcome' AND context = 'testlist.example.com';
====================== Result (all directories and file owned by 'mailman' with '765' permissions)
file:///templates/list/testlist.example.com/list:user:notice:welcome
====================== Content of above Welcome file
This is the second test Welcome message.
====================== Text of the welcome message actually received. (It's the content of the first custom template.)
This is the first test message to Welcome new members.
On 5/27/25 18:54, Mark wrote:
However when I update the template, the databases are updated correctly but the Welcome message is still sent with the old template content.
mailman.templates mailmanweb.postorius_emailtemplate
Additionally, if I delete the template and create a new one the old content still gets sent.
I cannot find where this content is written or is being cached.
The content is stored in the mailmanweb.postorius_emailtemplate table. As far as Mailman core and Postorius are concerned, that's the only place.
I'm guessing, but my guess is it's cached in your wsgi server. I.e., upon initial creation an entry is made in the mailman.template table with a uri like https://example.com/mailman3/api/templates/list/list.example.com/list:user:n... and when Mailman core wants the template, it gets that uri which I'm guessing is then cached in the wsgi server. Then, if the content is changed, the GET returns the old cached value instead of getting the new value from Postorius. If the template is deleted, the mailman.template table entry is deleted and at that point Mailman core will get the template from the file system following it's normal search rules, but if the template is recreated in Postorius, a new table entry is made with the same uri which may still get a cached template from the wsgi server.
If this is correct, restarting mailman-web or whatever controls the wsgi server should fix it.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 5/27/25 18:54, Mark wrote:
However when I update the template, the databases are updated correctly but the Welcome message is still sent with the old template content.
mailman.templates mailmanweb.postorius_emailtemplate
Additionally, if I delete the template and create a new one the old content still gets sent.
I cannot find where this content is written or is being cached.
On 2025-05-28 02:34, Mark Sapiro wrote:
The content is stored in the mailmanweb.postorius_emailtemplate table. As far as Mailman core and Postorius are concerned, that's the only place.
... If the template is deleted, the mailman.template table entry is deleted and at that point Mailman core will get the template from the file system following it's normal search rules,
Where does Mailman core search for the file if the table entry is deleted?
On 5/27/25 23:09, Mark wrote:
Where does Mailman core search for the file if the table entry is deleted?
The search order is described in the docstring at https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/utilities/i18n....
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 5/28/25 01:14, Mark wrote:
Then I created a template using Postorius and updated the uri in mailman.templates to:
file:///opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt - and that works.
Which says get the template from the file system at that location and don't ask for it from Postorius at all. This would be the default action if there were no entry for this template in the mailman.template table at all. I.e. you now have a template in Postorius, but it is not used.
It's looking promising. Now I just need to modify my script that periodically copies the data from the "mailmanweb.postorius_emailtemplate" table to a file and updates the uri.
Why it won't handle the templates in the database I don't know. But at least I can get a work-around solution in place now.
If you want to continue to use the template at /opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt just delete the template from Postorius and update that file directly.
If Postorius template updates don't work because of web server caching or whatever, just don't use them and use templates in the /opt/mailman/mm/var/templates/ hierarchy instead. You can create sitewide templates at /opt/mailman/mm/var/templates/site/en/*, domain specific templates in /opt/mailman/mm/var/templates/domains/en/* and list specific templates in /opt/mailman/mm/var/templates/lists/<list-id>/en/ without any entries in the mailman.template table.
Note that sitewide templates can be the most useful and can't be created in Postorius anyway.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
On 5/28/25 01:14, Mark wrote:
Then I created a template using Postorius and updated the uri in mailman.templates to:
file:///opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt
- and that works.
It's looking promising. Now I just need to modify my script that periodically copies the data from the "mailmanweb.postorius_emailtemplate" table to a file and updates the uri.
On 2025-05-28 19:17, Mark Sapiro wrote:
Which says get the template from the file system at that location and don't ask for it from Postorius at all. This would be the default action if there were no entry for this template in the mailman.template table at all. I.e. you now have a template in Postorius, but it is not used.
I have jumped the gun on thinking that the template as a file was working. It's not. It's the same story: the initial file template is good, but it's for sure getting cached somewhere, and an updated file is ignored. The same as when the template was being drawn from the database.
Restarting mailman3 and mailmanweb didn't help, neither did rebooting the server.
Looking at Gunicorn status, I see a failing connection to 127.0.0.1 which is the error message I was getting when adding a new subscriber - with the custom Welcome template in the database (which is why I moved it to file).
127.0.0.1 and locahost are in the ALLOWED_HOSTS and
CSRF_TRUSTED_ORIGINS of the settings.py file
I'm at a loss with this caching problem. Any other clues happily received.
$ gunicorn --version gunicorn (version 21.2.0)
$ gunicorn status [2025-05-29 02:29:41 +0000] [17838] [INFO] Starting gunicorn 21.2.0 [2025-05-29 02:29:41 +0000] [17838] [ERROR] Connection in use: ('127.0.0.1', 8000) [2025-05-29 02:29:41 +0000] [17838] [ERROR] Retrying in 1 second. [2025-05-29 02:29:42 +0000] [17838] [ERROR] Connection in use: ('127.0.0.1', 8000)
# ps -ef | grep gunicorn root 1522331 1521736 0 22:57 pts/1 00:00:00 grep gunicorn mailman 1608073 1 0 May20 ? 00:01:24 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application mailman 1608075 1608073 1 May20 ? 02:24:20 /opt/mailman/venv/bin/python3 /opt/mailman/venv/bin/gunicorn -c /etc/mailman3/gunicorn.conf mailman_web.wsgi:application
On 5/28/25 19:40, Mark wrote:
I have jumped the gun on thinking that the template as a file was working. It's not. It's the same story: the initial file template is good, but it's for sure getting cached somewhere, and an updated file is ignored. The same as when the template was being drawn from the database.
Are you just updating the file /opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt with nothing in Postorius and either no entry at all in the mailman.template table with
name = 'list:user:notice:welcome' context = 'testlist.example.com'
or if there is such an entry, with
uri = 'file:///opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt'
If so, editing the file /opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt should work.
Restarting mailman3 and mailmanweb didn't help, neither did rebooting the server.
Looking at Gunicorn status, I see a failing connection to 127.0.0.1 which is the error message I was getting when adding a new subscriber - with the custom Welcome template in the database (which is why I moved it to file).
127.0.0.1 and locahost are in the ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS of the settings.py file
I'm at a loss with this caching problem. Any other clues happily received.
If as I say above, there is no entry in the mailman.template table for the name and context or if there is an entry with uri = 'file:///opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt', then gunicorn and mailman-web are not involved at all - mailman core will get the template from the file directly.
-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan
I have jumped the gun on thinking that the template as a file was working. It's not. It's the same story: the initial file template is good, but it's for sure getting cached somewhere, and an updated file is ignored. The same as when the template was being drawn from the database.
Are you just updating the file /opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt with nothing in Postorius and either no entry at all in the mailman.template table with
name = 'list:user:notice:welcome' context = 'testlist.example.com'
or if there is such an entry, with
uri = 'file:///opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt'
If so, editing the file /opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt should work.
Progress!
It now works if the template information is removed from BOTH database tables (mailmanweb.postorius_emailtemplate and mailman.templates).
Just removing the row from the "mailmanweb.postorius_emailtemplate" table and changing the URI to "file:///opt/mailman/mm/var/templates/lists/testlist.example.com/en/list:user:notice:welcome.txt" in mailman.templates didn't help.
So now, with both databases cleared, mailman looks for the updated file without any problems.
Thank you very much Mark, once again, for your solid advice and patience.
Best, Mark
participants (2)
-
Mark -
Mark Sapiro