Search results for "Postorius" - Mailman-users

Re: Hyperkitty performance problem

by Greg Newby

Hi monochromec. I have a similar installation and was also having various problems and delays. I'm on a similarly sized system (Ubuntu 24.04 with venv installations of mailman3; 32GB memory). What I found was that tuning of the resources made a very significant impact. Out of the box tuning parameters for the various components were not sufficient to keep mail flowing. The "system info" tab in the web interface tells you where things are held up, if they are. I was having trouble with the 'out' queue getting slow, and the postfix 'mailq' was showing thousands of messages waiting to be delivered. I was running out of smtpd processes. Archiving was delayed (though not as much as yours). Some of the things I did: - Increase PostgreSQL resource limits. Some of the parameters in my configuration files: # Tuning, via https://pgtune.leopard.in.ua/ for 'data warehouse' app. max_connections = 100 # default shared_buffers = 8GB effective_cache_size = 24GB maintenance_work_mem = 2GB # wal_buffers = 16MB default_statistics_target = 500 effective_io_concurrency = 2 work_mem = 26214kB min_wal_size = 4GB max_wal_size = 16GB max_parallel_workers_per_gather = 4 max_parallel_maintenance_workers = 4 - Increase the number of mailman processes in mailman.cfg # Note: Values need to be a power of 2. Make sure PostgreSQL allows more connections than the # of instances [runner.in] class: mailman.runners.incoming.IncomingRunner instances: 16 # instances: 8 [runner.out] class: mailman.runners.outgoing.OutgoingRunner instances: 64 The "out" setting, notably, decreased mail delivery times. - Increase postfix resources. Notably in main.cf: # https://www.postfix.org/TUNING_README.html # default_process_limit = 100 # default default_process_limit = 250 Maybe these will point you in the right direction. I realize you are focused on hyperkitty, and my experience was that all the major components (database, MTA and mailman3) all needed more resources. ~ Greg On Sat, Dec 28, 2024 at 7:26 AM monochromec via Mailman-users < mailman-users(a)mailman3.org> wrote: > Following the root cause analysis which Tobias started all those weeks ago > we (the admin team behind the installation) are still struggling with the > following phenomenon: messages on average take more than 24 hours to be > processed, more precisely, the average lifetime of a pickled message object > in `/var/lib/mailman3/queue/pipeline` clocks in at around 26 hours). > > Couple of stats of the installation: standard installation from Bookworm > OS repos, Hyperkitty as archiver and Postorius as web frontend as explained > above, running Python 3.11.2 from the standard systemd service as packaged > with the Bookworm deb file. All backends (Core + Hyperkitty) are supported > by Postgres version 15+248. The MTA is a standard Postfix installation, > again from OS packages. > > The underlying VM has 7 cores with just under 24 GB of main memory. This > production instance is handling less than 130 mailing list (MLs) with an > average of less than 10 postings per day per ML. CPU core utilisation > hovers around 50% with the lion share allocated to the four pipeline > runners as part of the MM configuration. > > OS resource utilisation is well below bounds (approx. 8 GB of main memory > allocated to running processes), plenty of available sockets space (I > noticed some transient `Connection lost during _handle_client()` warnings > in the logs so I checked that the SMTP runner can connect to Postfix for > delivering the messages after processing by checked the socket allocation > of the running processes). > > Cursory review of the corresponding Core classes (runner + pipeline > implementation in addition to `posting_pipeline`) didn't reveal any further > pointers. What I did notice though that increasing the logging levels of > the components (namely `smtp`, `pipeline` and `lmtp` to `debug`) in > `/etc/mailman3/mailman.cfg` didn't add any useful information to the logs > as configured after restarting the Core. > > As outlined above, Hyperkitty doesn't seem to do a check based on ML and > message ID before archiving a message in the database. But this only add a > REST roundtrip and Postgres communication through Hyperkitty's underlying > Django framework to the overall system load and the driving UWSGI instance > is well within CPU cycle bounds. > > Any pointers are appreciated - more than happy to provide more info if > required. > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to gbnewby(a)petascale.org >

8 months, 2 weeks

Re: New installation, new issue: mailman3 migrate -- no module CommonMark

by Mark Sapiro

On 10/26/24 16:20, Bill Oliver wrote: > > (venv) mailman@mail:~$ mailman-web migrate > Traceback (most recent call last): > File "/opt/mailman/venv/lib/python3.10/site- > packages/django/template/backends/django.py", line 128, in > get_package_libraries > module = import_module(entry[1]) > File "/usr/lib/python3.10/importlib/__init__.py", line 126, in > import_module > return _bootstrap._gcd_import(name[level:], package, level) > File "<frozen importlib._bootstrap>", line 1050, in _gcd_import > File "<frozen importlib._bootstrap>", line 1027, in _find_and_load > File "<frozen importlib._bootstrap>", line 1006, in > _find_and_load_unlocked > File "<frozen importlib._bootstrap>", line 688, in _load_unlocked > File "<frozen importlib._bootstrap_external>", line 883, in > exec_module > File "<frozen importlib._bootstrap>", line 241, in > _call_with_frames_removed > File "/opt/mailman/venv/lib/python3.10/site- > packages/postorius/templatetags/markdown.py", line 21, in <module> > from readme_renderer import markdown as md_render > File "/opt/mailman/venv/lib/python3.10/site- > packages/readme_renderer/markdown.py", line 16, in <module> > from CommonMark import commonmark > ModuleNotFoundError: No module named 'CommonMark' What is your version of readme_renderer? I don't see readme_renderer importing from CommonMark in any recent version. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

10 months, 3 weeks

When mass subscribing using format John Doe <john.doe@example.com>, "Members" show some with real name, others without.

by Henrik Rasmussen

GNU Mailman 3.2.0 (La Villa Strangiato): A list administrator asked me why some people is subscribed with full name and some are not, when he is mass subscribing people using the format John Doe john.doe(a)example.com. He had experienced this a lot of times. I tested this several times on my test lists, and one member, who already had his name registrered in Account (Postorius top right), and thus also in Django, kept being subscribed without real name, while the others was subscribed with full name in Members -> Subscribers. It seems (I’m not certain about this) that the one’s without real name is the one’s already having an account and thus registred in Django (admin/auth/user/), while the ones subscribed with full name associated is the ones not having an account already. However, as soon as the member (the one with the account) went to Account and changed his real name, the name was immediately associated with the subscription in Members -> Subscribers. I expect this to be by design, but why isn’t the name associated with the subscription when mass subscribing in the first place, taken from the account? Henrik Rasmussen

5 years, 10 months

Re: Re-enable member after bouce processed disable

by Mark Sapiro

On 12/9/21 6:07 AM, Henrik Rasmussen wrote: > Recently I upgraded Mailman to GNU Mailman 3.3.4 (Tom Sawyer) (Postorius: 1.3.5 and HyperKitty: 1.3.4) which is now processing bounces. > > When a member have been disabled due to bounces, how is the member re-enabled? I would guess it would be under the "Users -> Members -> Member options -> Delivery status", but theses are both (Enabled and disabled) unchecked, not indicating whether or not the member is either or. You are looking at the user's list based preferences. If one of these is unset, it falls back to address based preferences and from there to the user's global preferences. You can see all of these for yourself at the Mailman settings choice in the dropdown at your user name, but for another list member you only see the list based preferences. > How do I or the list admins re-enable a bounce-disabled member? Just set the member's delivery status on the page you see to Enabled. Even though it was not previously shown as Disabled by Bounces, it will be enabled. Likewise the member can set her Delivery status to Enabled. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 9 months

Auto discard after max_days_to_hold not working.

by Stephen J. Turnbull

Hitashi mancho via Mailman-users writes: > I have migrated around 300 mailing lists from Mailman 2 to Mailman > 3. In most of the mailing lists we have configured > "max_days_to_hold" to 2. It was working fine in Mailman 2. However, > it is not working in Mailman 3. This doesn't seem to be exposed in Postorius. It's not all that old in Mailman 3 (April 2020) so it's possible that it is not moved from Mailman 2 to Mailman 3 by the migration scripts. You can use "mailman withlist" to access that variable: Use "mailman withlist -l LIST(a)DOMAIN.TLD", then at the prompt: >>> m.max_days_to_hold to examine the value for LIST(a)DOMAIN.TLD. To do a mass change, something like for ml in `mailman lists`; do mailman withlist -l $ml -r fix_mdth done where you put in Mailman's bin directory the file 'fix_mdth.py' containing def fix_mdth(ml): ml.max_days_to_hold = 2 commit() Of course if there are list where you want a longer or indefinite hold you'll need to change those back, since this approach changes all of them. You can get more information with "mailman withlist --detail". -- GNU Mailman consultant (installation, migration, customization) Sirus Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan

4 months, 2 weeks

Re: Post installation issues (recipients refused) with Exim4 in a virtual host environment

by Kyriakos Terzopoulos

Hi, I have the following in my mailman log [11/Oct/2023:12:31:39 +0300] "GET /3.1/lists/ testlist(a)myproject.eu/held/count HTTP/1.1" 200 73 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:43 +0300] "GET /3.1/lists/testlist.myproject.eu HTTP/1.1" 200 425 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:43 +0300] "GET /3.1/lists/testlist(a)myproject.eu/config HTTP/1.1" 200 3111 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:43 +0300] "GET /3.1/lists/testlist.myproject.eu/archivers HTTP/1.1" 200 100 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist.myproject.eu HTTP/1.1" 200 425 "-" "GNU Mailman REST client v3.3.5" Oct 11 12:31:51 2023 (1534165) deque: do_confirm_verify Traceback (most recent call last): File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/app/workflow.py", line 69, in __next__ return step() File "/opt/mailman/venv/lib/python3.10/site-packages/mailman/app/subscriptions.py", line 382, in _step_send_confirmation raise StopIteration StopIteration [11/Oct/2023:12:31:51 +0300] "POST /3.1/members HTTP/1.1" 202 143 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist.myproject.eu HTTP/1.1" 200 425 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist(a)myproject.eu/config HTTP/1.1" 200 3111 "-" "GNU Mailman REST client v3.3.5" [11/Oct/2023:12:31:51 +0300] "GET /3.1/lists/testlist.myproject.eu/archivers HTTP/1.1" 200 100 "-" "GNU Mailman REST client v3.3.5" *Kyriakos Terzopoulos *Web developer / e-learning expert *Tel:*+30 211 213 9858 *Mobile:* +30 694 526 4512 * E-mail: *kyriakos.terzopoulos(a)gmail.com * Skype:* kyriakos.terzopoulos Find me on Facebook <http://www.facebook.com/cirrus3d> Follow me on Twitter <http://twitter.com/#%21/cirrus3d> On Wed, 11 Oct 2023 at 11:54, Odhiambo Washington <odhiambo(a)gmail.com> wrote: > On Wed, Oct 11, 2023 at 11:15 AM Kyriakos Terzopoulos < > kyriakos.terzopoulos(a)gmail.com> wrote: > > > Hi, > > > > I have set up (I believe successfully) and configured the following in a > > Virtualhost environment (with HestiaCP): > > > > - A record for lists.mydomain.com > > - Venv > > - Mailman core > > - Mailman web (run migrate, collectstatic, compress, compilemessages) > > Postorius > > - gunicorn, hyperkitty > > - Nginx > > - SMTP exim4 conf > > > > I can see Posrorius and I have logged in and created my first mailing > list: > > > > [image: Screenshot 2023-10-11 at 10.57.31 AM.png] > > > > My issue is that even though I have subscribed to my list as you can see > > above, I am not receiving the confirmation email. I get a "recipients > > refused" error. The smtp log for mailman is as follows: > > > > Oct 11 10:41:20 2023 (1533666) > > <169701007904.1534165.14569329382406660776@localhost> recipients > refused: > > {' > > kyriakos.terzopoulos(a)gmail.com': (550, b'Verification failed for > > <testlist-bounces+kyriakos.terzopoulos=gmail.com(a)myproject.eu > > >\nUnrouteable > > address\nSender verify failed')} > > > > Oct 11 10:41:20 2023 (1533666) > > <169701007904.1534165.14569329382406660776@localhost> smtp to > > testlist(a)myproject.eu for 1 recips, completed in 0.26796579360961914 > > seconds > > > > Oct 11 10:41:20 2023 (1533666) > > <169701007904.1534165.14569329382406660776@localhost> post to > > testlist(a)myproject.eu from > > testlist-confirm+c9b20c5290bec43147a7e96a6b8cd825daf913d7(a)myproject.eu, > > 1667 bytes, 1 failures > > > > Oct 11 10:41:20 2023 (1533666) > > <169701007904.1534165.14569329382406660776@localhost> delivery to > > kyriakos.terzopoulos(a)gmail.com failed with code 550, b'Verification > failed > > for <testlist-bounces+kyriakos.terzopoulos=gmail.com(a)myproject.eu > > >\nUnrouteable > > address\nSender verify failed' > > > > I have used the configuration for exim that is here > > < > > > https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/docs/mta.h… > > > > > and > > I have also tested with the alternative approach that is documented in > > > > What do you have in the Exim's mainlog? > > > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) > [How to ask smart questions: > http://www.catb.org/~esr/faqs/smart-questions.html] > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Archived at: > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > This message sent to kyriakos.terzopoulos(a)gmail.com >

1 year, 11 months

Re: Easiest way to install a new mailman3 deployment?

by Stephen J. Turnbull

ieso(a)johnwillson.com writes: > Here's an interesting suggestion I received that I'm going to kick > around a bit.. has anyone tried using Ansible to assist the > installation process? > > https://github.com/rivimey/ansible-mailman3 rivimey is very competent. That would undoubtedly be a good place to start. However, her setup may be more complicated than most folks need (IIRC, she has a multi-homed setup with Mailman core, Postorius, and HyperKitty on three separate IPs on a separate subnet in the DMZ, while Mailman was really designed for three Mailman applications running on the same host, along with the database server and MTA). > I hadn't heard of Ansible before, but it looks perfect for this > kind of situation. By the way: Ansible is not the only such application, but a Friend of Mailman used to work there (I think he's since been transferred elsewhere after they got acquired by RedHat), so if people want to work on this kind of thing, we may have more resources vs. Ansible than say Chef or Puppet. I'm not sure we want to maintain an Ansible configuration, but I imagine we could stick it in contrib as "an example that works for somebody YMMV unsupported NO WARRANTY". :-) Steve

4 years, 8 months

Re: Mailman3 doesn't send Mails to users (Debian12 with Plesk)

by Mark Sapiro

On 5/24/24 2:18 AM, marcus--- via Mailman-users wrote: > > The problem, however, is that it does not send out any mails, nor do they appear in the archive. > > When I look in the Mailman-Log I find: > > May 16 01:44:55 2024 (6833) Cannot connect to SMTP server localhost on port 25 This is a misleading error message. It can occur when Mailman's out runner can't read a template. You need to check ownership and permissions on mailman/templates in the code tree and Mailman's var/templates directory which probably has permissions errors from running inport21 as root. Also, if you've set any templates in Postorius, they may not be accessable if your setting for POSTORIUS_TEMPLATE_BASE_URL is not good. > May 16 01:48:15 2024 (61252) error opening connection to nntp_host: localhost > [Errno 111] Connection refused > May 16 01:48:15 2024 (61252) NNTP error for list vereinsmitglieder(a)ludothek.de: > [Errno 111] Connection refused Are you gating messages to/from usenet for this list? If so, you have to set host: and port: and maybe user: and password in the nntp section of mailman.cfg -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 3 months

Unable to update templates

by Mark

Hi, I'm unable to update the custom welcome template (list:user:notice:welcome) after it's initial creation. When the custom welcome template is created (in Postorius), and a user then subscribed by the list-owner (using mass subscribe), the welcome message is received as expected. However when I update the template, the databases are updated correctly but the Welcome message is still sent with the old template content. mailman.templates mailmanweb.postorius_emailtemplate Additionally, if I delete the template and create a new one the old content still gets sent. I cannot find where this content is written or is being cached. Any clues on tracking this bugbear down gratefully received. Regards, Mark $ mailman info GNU Mailman 3.3.9 (Tom Sawyer) Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] config file: /etc/mailman3/mailman.cfg db url: postgresql://******:******@localhost/mailman devmode: DISABLED REST root url: http://localhost:8001/3.1/ -- Installed with venv. ===================== Query to DB - mailman ===================== SELECT URI FROM TEMPLATE WHERE NAME = 'list:user:notice:welcome' AND context = 'testlist.example.com'; ====================== Result (all directories and file owned by 'mailman' with '765' permissions) ====================== file:///templates/list/testlist.example.com/list:user:notice:welcome ====================== Content of above Welcome file ====================== This is the second test Welcome message. ====================== Text of the welcome message actually received. (It's the content of the first custom template.) ====================== This is the first test message to Welcome new members.

3 months, 3 weeks

Re: default language ignored

by Mark Sapiro

On 10/02/2017 05:46 AM, Marc Sluiter wrote: > Hi, > > one last question for now: how do switch the language of the mails sent by mailman? > I have this in my configuration, but without effect: > > [language.de] charset: iso-8859-1 > [language.de] description: German > [language.de] enabled: yes > > [mailman] default_language: de It has no effect because there are as yet no translations in Mailman 3. You need to create templates in the desired language. See <http://mailman.readthedocs.io/en/latest/src/mailman/rest/docs/templates.html> That refers to setting URLs for templates which does work, but the easiest way is to put them in the default search path in Mailman's file hierarchy at $var-dir/templates/. Sitewide ones in $var-dir/templates/site/, Domain specific in $var-dir/templates/domains/ and list specific ones in $var-dir/templates/lists/. For example, a list specific, English language regular (non-digest) message footer might go in $var-dir/templates/lists/mymaillist.example.com/en/list:member:regular:footer > Is it also possible to change the language for a single list? Yes, although that is not yet exposed in Postorius. The list has a preferred_language attribute which you can set via REST or 'mailman withlist' aka 'mailman shell'. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

7 years, 11 months

Mailman-users search results for query "Postorius"