Search results for "Postorius" - Mailman-users

Re: Affinity Mailman Is Now Beta

by Mark Sapiro

On 6/18/20 12:44 PM, Brian Carpenter wrote: > Dear Mailman Admins and Users, > > Affinity Mailman is our new admin interface for Mailman 3 and we intend > to use it to replace Postorius. It has been under development for about > 3 months now and it recently reached beta. Just to be clear, Brian has the full support of the GNU Mailman project in his endeavor to create a new web UI for Mailman 3, but "our" above means Brian and EMWD, not the GNU Mailman project. We of the GNU Mailman project are delighted that Brian is working on an alternative web UI, but we have no plan to abandon Postorius and HyperKitty. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 5 months

Re: Held messages: Long time waiating for response of the Mailman API

by Mark Sapiro

On 1/24/22 15:06, Mark Sapiro wrote: > On 1/18/22 06:24, Stephan Krinetzki wrote: >> Hello, >> >> We are using Mailman 3.3.5 with Hyperkitty 1.3.5. Mailman 3 is >> installed in a venv and deployed using Apache and uwsgi. This works >> quite well so far, only withheld messages cause us a problem: If we >> try to approve them via Postorius, the message is delivered to the >> list immediately, but the success message in Postorius takes a long >> time - on average the process hangs for three minutes until the >> message is no longer displayed. We have already adjusted the timeout >> in gunicorn.cfg, but this does not improve the situation. Are there >> any tips on how we can speed up the REST API? Or is it even a >> configuration error? > > > There are two steps in handling the POST for a held message. The first > step is accepting, rejecting or discarding the message. This is > happening immediately. > > The second step is handling the poster's moderation action based on the > choice from the Set member moderation dropdown in the view of the held > message. This is processed by the code at > <https://gitlab.com/mailman/postorius/-/blob/master/src/postorius/views/list…>. > If no selection has been made, the default value POSTed for > moderation_choice is `no-action` and that code is skipped. If it is one > of the choices `Hold for moderation`, `Reject (with notification)`, > `Discard (no notification)`, `Accept immediately (bypass other rules)` > or `Default processing` that is processed. So presumably a selection has > been made and processing is stuck on getting the member or setting the > member's moderation action. > > Possibly the logs of your database manager will have information. > Some additional info. By gunicorn.cfg, do you mean the file at mailman/config/gunicorn.cfg? If you are using gunicorn as a wsgi server, that is separate from the gunicorn serving the REST API and its config doesn't affect the REST API. The relevant config is mailman/config/gunicorn.cfg?. Also, I doubt the issue is with the REST API per se, but rather it is in Mailman core's processing of the particular endpoint and if that is slow, it is most likely some kind of database issue. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 10 months

Re: Database running VERY hot

by Abhilash Raj

Hi Darren, On 3/8/18 9:43 AM, Darren Smith wrote: > Hello all, > > We had an old installation of Mailman 2.1 across three servers with over > 32,000 mailing lists on them. Due to circumstances beyond my control, we > had to shut them down and quickly set up Mailman 3 on a new server. > > We have Mailman3 set up on a single server this time, as we were having > problems with a 3-server setup and Hyperkitty and archives. Long story > short, it is up and running, and our users are starting to come back. > > However, we do not have very many users hitting Postorius and Hyperkitty > yet, and we are seeing a distressing number of 504 gateway timeouts. > > We noticed that there is a significant number of queries to the API > whenever hitting Postorius's front page to load the mailing lists, and then > on every load. It looks as if it makes a single call to the API to get > information about a single list, and does this 10 times - once for each > list - to render the page. This gets much worse when users set the view to > 200 per page, as they are inclined to do with 32,000 mailing lists... > > What we've done, actually, is to override the front page and point the > users to a different set of static pages where we have all of our lists > categorized and searchable, and when they find the list they are looking > for, we have a link back to postorius. It has taken a lot of the stress off > of the server. > > However, I still see where the database at times can run VERY hot. > > We have set up this server to run with PostgreSQL. When I do a ps aux | > grep postgres, I find a lot of processes that have been spawned to take > care of mailman - a dozen or so. However, one of the threads is taking up > about 85% of the load, another thread about 14%, and little dribbles here > and there from the rest of the threads. > > Here are the questions: > > 1) Has anyone run into problems like these? I would think that maybe this > is an issue having so many mailing lists, but honestly there aren't that > many people using the system yet. It is going to ramp up as the users > realize that we are live again, but we aren't getting that much traffic. I can say that in general, Postorius doesn't really optimize REST API queries and so loading a single page can sometimes make things bad. Currently, the largest installation of Mailman 3 that we know of runs at lists.fedoraproejct.org and the administrators of the list would know how well does our Web UI scale. However, I don't think we have anyone actively working on trying to fix this particular issue. Pull Requests and fixes are welcome :) > > 2) Is there any configuration that I can set, especially with regards to > the database, so that it can be used more efficiently and not throttle > through one or two threads? There isn't any configuration that would change how databases are used in Mailman Core. Django *may* have some, but I don't know about any of them of the top of my head. thanks, Abhilash

6 years, 8 months

Re: Postorius no connection to REST API

by Mark Sapiro

On 8/11/21 4:18 AM, Richard Rosner wrote: > Mark Sapiro wrote: >>> You are in a better position to answer that than am I. >> What does sudo netstat -lntp show? > > A lot. But since most of that isn't relevant here: > Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name > tcp 0 0 0.0.0.0:465 0.0.0.0:* LISTEN 20241/master > tcp 0 0 127.0.0.1:8024 0.0.0.0:* LISTEN 14076/python3 > tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 20241/master > tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 14080/python3 So some python3 process is listening on port 8001 > tcp 0 0 0.0.0.0:587 0.0.0.0:* LISTEN 20241/master > tcp6 0 0 :::80 :::* LISTEN 13882/apache2 > tcp6 0 0 :::465 :::* LISTEN 20241/master > tcp6 0 0 :::25 :::* LISTEN 20241/master > tcp6 0 0 :::443 :::* LISTEN 13882/apache2 > tcp6 0 0 :::587 :::* LISTEN 20241/master > >> What does ps -fwwa|grep rest show? > > root 15055 14843 0 12:58 pts/1 00:00:00 grep rest > > So whatever it's supposed to find, it's not there But whatever is listening on port 8001 is apparently not Mailman's rest server. What does ps -fww 14080 or whatever PID is currently listening on port 8001 show. >>> mailman3-web.service must also run as list. > > I changed that. It didn't like it. OK. Perhaps instead you should have uid: list gid: list in your uwsgi configuration if you don't already. > >>> So, what do you have in your apache config for proxying to uwsgi and >> what's your uwsgi configuration. > > lists-ssl.conf: > <VirtualHost *:443> > ServerAdmin admin(a)domain.de > ServerName lists.domain.de > > Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico > Alias /mailman3/static /var/lib/mailman3/web/static > > <Directory "/var/lib/mailman3/web/static"> > Require all granted > </Directory> > > <IfModule mod_proxy_uwsgi.c> > ProxyPass /mailman3/favicon.ico ! > ProxyPass /mailman3/static ! > ProxyPass "/mailman3" "unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost:8001/" I'm not familiar enough with this form of ProxyPass using sockets to understand what the localhost:8001 does in this context, but uwsgi should not be doing anything with port 8001. In a configuration using TCP, it would listen on port 8000. Port 8001 is where Mailman's REST API server listens. uwsgi should receive connects in your case via the unix socket and then pass them to Django via the application in wsgi.py Also, you may want other paths proxied to uwsgi, namely at least some of hyperkitty, postorius, archives, accounts, admin and user-profile. > </IfModule> ... > I guess with uwsgi config you mean the /etc/mailman3/uwsgi.ini file? > [uwsgi] > # Port on which uwsgi will be listening. > uwsgi-socket = /run/mailman3-web/uwsgi.sock > > #Enable threading for python > enable-threads = true > > # Move to the directory wher the django files are. > chdir = /usr/share/mailman3-web > > # Use the wsgi file provided with the django project. > wsgi-file = wsgi.py > > # Setup default number of processes and threads per process. > master = true > process = 2 > threads = 2 > > # Drop privielges and don't run as root. > uid = www-data > gid = www-data These should be 'list', not 'www-data'. That's your permissions issue on settings.py -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 3 months

Installed but mailman reports missing hyperkitty module

by jbernal.web.dev＠gmail.com

I've opted for the Mailman3 package installation on Ubuntu 22 LTS on top of a manually installed Mailman3 core package from the Ubuntu 20 Debian repository. Everything is working great on my installation. However, if I try to enable hyperkitty in /etc/mailman.cfg postorius reports 500 status codes on all pages. /var/log/mailman3/mailman.log reports this error: ModuleNotFoundError: No module named 'mailman3_hyperkitty' Any idea why postorius can't seem to find the installed hyperkitty module? Stack trace: 2023-02-08 12:48:14 [FALCON] [ERROR] GET /3.1/lists/test.listdev.ocdla.org/archivers => Traceback (most recent call last): File "falcon/app.py", line 361, in falcon.app.App.__call__ File "/usr/lib/python3/dist-packages/mailman/rest/lists.py", line 405, in on_get archiver_set = IListArchiverSet(self._mlist) File "/usr/lib/python3/dist-packages/zope/component/_api.py", line 156, in adapter_hook return sitemanager.queryAdapter(object, interface, name, default) File "/usr/lib/python3/dist-packages/zope/interface/registry.py", line 354, in queryAdapter return self.adapters.queryAdapter(object, interface, name, default) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/mailinglist.py", line 614, in __init__ for archiver in config.archivers: File "/usr/lib/python3/dist-packages/mailman/config/config.py", line 280, in archivers archiver = call_name(class_path) File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 69, in call_name named_callable = find_name(dotted_name) File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 52, in find_name module = import_module(module_path) File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked ModuleNotFoundError: No module named 'mailman3_hyperkitty'

1 year, 9 months

Re: Apache Conf

by Mail Person

Ok, progress with tinkering some. Using the below configuration I can manually navigate to lists.example.com/mailman3/postorius/lists/ and all is working fine. If I go to lists.example.com/mailman3 it for some reason redirects to lists.example.com/mailman/postorius/lists/ which generates a 404. Going to lists.example.com also hits a 404. So closer, but not perfect. Any ideas? The current, sort of working config: <VirtualHost *:80> ServerAdmin admin(a)example.com ServerName lists.example.com ServerAlias www.lists.example.com Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico Alias /mailman3/static /var/lib/mailman3/web/static <Directory "/var/lib/mailman3/web/static"> Require all granted </Directory> <IfModule mod_proxy_uwsgi.c> ProxyPass /mailman3/favicon.ico ! ProxyPass /mailman3/static ! ProxyPass /mailman3 unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost </IfModule> RewriteEngine on RewriteCond %{SERVER_NAME} =lists.example.com [OR] RewriteCond %{SERVER_NAME} =www.lists.example.com RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] </VirtualHost>

2 years, 4 months

Re: Splitting hosts: mta&core vs postorius

by Mark Sapiro

On 6/21/20 9:38 AM, Ruth Ivimey-Cook wrote: > > > Problem is, if I do that (which I did in the beginning), the core rest > service listens on localhost / 127.0.0.1 (it obviously recognises that > as the , and so cannot be connected to by anything. That is why I set it > to 0.0.0.0 to begin with, and what the documentation implies (fairly > strongly) that this parameter is about. What documentation? I do understand what you are saying and I see the issue. I don't understand why it works for this poster <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> but maybe that's specific to Docker containers. > IMO, the hostname parameter needs to be split up into a "what address > should the rest of the world use to address me" and separately "what > address(es) should I listen on". Multiple listen addresses enable > multihomed hosts to get the right connectivity, too. Try something like this and let us know if it works? If so, we can do something like add a config setting for this. > --- a/src/mailman/rest/gunicorn.py > +++ b/src/mailman/rest/gunicorn.py > @@ -53,7 +53,7 @@ def make_gunicorn_server(): > # We load some options from config.webservice, while other extended options > # for gunicorn can be defined in configuration: section. We also load up > # some logging options since gunicorn sets up it's own loggers. > - host = config.webservice.hostname > + host = '0.0.0.0' > port = int(config.webservice.port) > log_path = os.path.join(config.LOG_DIR, config.logging.http['path']) > options = { -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 5 months

Mailman 3 install questions: HAYSTACK_CONNECTIONS

by heller＠deepsoft.com

I am using the debian 12 repo versions of mailman3, django, posterious, hyperkitty, and xapian: sharky5% dpkg-query -l python3-{django,django-mailman3,django-hyperkitty,django-postorius,xapian} mailman3\* | grep '^ii' ii mailman3 3.3.8-2~deb12u2 all Mailing list management system ii mailman3-doc 3.3.8-2~deb12u2 all Mailing list management system documentation ii mailman3-web 0+20200530-2.1 all Django project integrating Mailman3 Postorius and HyperKitty ii python3-django 3:3.2.19-1+deb12u1 all High-level Python web development framework ii python3-django-hyperkitty 1.3.7-1 all Web user interface to access GNU Mailman3 archives ii python3-django-mailman3 1.3.9-1 all Django library to help interaction with Mailman3 (Python 3 version) ii python3-django-postorius 1.3.8-3 all Web user interface to access GNU Mailman3 ii python3-xapian 1.4.22-1 amd64 Xapian search engine interface for Python3 Before adding the HAYSTACK_CONNECTIONS options, I have this: sharky5% sudo mailman-web haystack_info --no-color --skip-checks Number of handled 1 index(es). - Model: Email by Index: <hyperkitty.search_indexes.EmailIndex object at 0x7fd0de1b0470> When I add this to /etc/mailman3/maymail_web.py (adapted from the section titled: "Setting up Fulltext search (xapian)" on https://docs.mailman3.org/en/latest/install/virtualenv.html#set-up-xapian HAYSTACK_CONNECTIONS = { 'default': { 'PATH': "/var/lib/mailman3/web/fulltext_index", 'ENGINE': 'xapian_backend.XapianEngine' }, } I get: File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1206, in _gcd_import File "<frozen importlib._bootstrap>", line 1178, in _find_and_load File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked ModuleNotFoundError: No module named 'xapian_backend' So, do I use Xapian or not? Is something missing somewhere? Is this a documentation bug or something else? -- Robert Heller -- Cell: 413-658-7953 GV: 978-633-5364 Deepwoods Software -- Custom Software Services http://www.deepsoft.com/ -- Linux Administration Services heller(a)deepsoft.com -- Webhosting Services

4 months, 3 weeks

Re: More Mailman releases!

by Wikinaut

before upgrade (half-way upgrade works → thus: Postorius Version 1.3.10 ; GNU Mailman 3.3.9 (Tom Sawyer) works; the problem pops up when upgrading hyperkitty/mailman-web): https://mydomain/mailman3/lists/mylist.mydomain/ => works https://mydomain/postorius/lists/mylist.mydomain/ => does not work (404 not found) apache2: <VirtualHost *:443> ServerAdmin webmaster@localhost DocumentRoot /var/www/html ServerName mydomain Alias /static "/opt/mailman/web/static" <Directory "/opt/mailman/web/static"> Require all granted </Directory> <IfModule mod_headers.c> RequestHeader unset X-Forwarded-Proto <If "%{HTTPS} =~ /on/"> RequestHeader set X-Forwarded-Proto "https" </If> </IfModule> <IfModule mod_proxy.c> ProxyPreserveHost On ProxyPass "/mailman3" "http://127.0.0.1:8000/mailman3" ProxyPass "/archives" "http://127.0.0.1:8000/archives" ProxyPass "/accounts" "http://127.0.0.1:8000/accounts" ProxyPass "/admin" "http://127.0.0.1:8000/admin" ProxyPass "/user-profile" "http://127.0.0.1:8000/user-profile" </IfModule> Include /etc/letsencrypt/options-ssl-apache.conf SSLCertificateFile /etc/letsencrypt/live/mydomain/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/mydomain/privkey.pem </VirtualHost>

1 year, 1 month

Re: Postorius claims 2 messages held when there are none - out of sync

by Mark Sapiro

On 6/26/23 4:19 PM, Ken Alker wrote: > > What else is stored in /var/messages, if not archives? I see 4711 > messages in that directory (which I presume would be as many messages > are in the archive database). If there is a write-up on this, feel free > to point me to it. I found > <https://docs.mailman3.org/projects/mailman/en/latest/build/lib/mailman/mode…> but this isn't really answering my question. These are probably artifacts from your old Debian package. There have been issues in the past with handled messages not being removed from the message store. See https://gitlab.com/mailman/mailman/-/issues/257 This was fixed in 3.3.5 and the task runner implemented in 3.3.5 removes orphaned messages from the message store, but if they are still referenced in pendings they won't be deleted. > I didn't think so, but I'll certainly buy that. So, I'm assuming by > this you are implying that there is no helper function that needed to > run and the number of held messages should have dropped to zero > immediately upon a refresh of the web page? Correct, assuming your changes were committed. > So if I run the script, exist the shell, then come back in again, I > can't rely on the request.id being the same? I'd need to re-run the > script and get a new request.id? No. The request id for a given held message won't change. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 5 months

Mailman-users search results for query "Postorius"