Search results for "Postorius" - Mailman-users

Re: Database running VERY hot

by Abhilash Raj

On Fri, Mar 9, 2018, at 9:11 AM, Darren Smith wrote: > Abhilash, > > Thanks for the response. I'm going over to lists.fedoraproject.org to > see if they have done any fine tuning.> > Our installation is at https://lists.rootsweb.ancestry.com, if you're > interested in seeing an installation with over 32,000 mailing lists. > As I mentioned, it was big enough that we have disabled the browse > experience in Postorius and are hosting our own pages for browsing to > a new list. Please keep us posted if you have other unique challenges for operating this high number of lists, other than this particular one. We'd love to be able to support high number of lists efficiently :) > If we get time to work on this we will certainly create a pull > request. :)> > > On Thu, Mar 8, 2018 at 11:49 AM, Abhilash Raj > <maxking(a)asynchronous.in> wrote:>> Hi Darren, >> >> >> On 3/8/18 9:43 AM, Darren Smith wrote: >> >>> Hello all, >>> >>> We had an old installation of Mailman 2.1 across three servers >>> with over>>> 32,000 mailing lists on them. Due to circumstances beyond my >>> control, we>>> had to shut them down and quickly set up Mailman 3 on a new server.>>> >>> We have Mailman3 set up on a single server this time, as we were >>> having>>> problems with a 3-server setup and Hyperkitty and archives. Long >>> story>>> short, it is up and running, and our users are starting to come >>> back.>>> >>> However, we do not have very many users hitting Postorius and >>> Hyperkitty>>> yet, and we are seeing a distressing number of 504 gateway >>> timeouts.>>> >>> We noticed that there is a significant number of queries to the API>>> whenever hitting Postorius's front page to load the mailing lists, >>> and then>>> on every load. It looks as if it makes a single call to the API >>> to get>>> information about a single list, and does this 10 times - once >>> for each>>> list - to render the page. This gets much worse when users set the >>> view to>>> 200 per page, as they are inclined to do with 32,000 mailing >>> lists...>>> >>> What we've done, actually, is to override the front page and >>> point the>>> users to a different set of static pages where we have all of our >>> lists>>> categorized and searchable, and when they find the list they are >>> looking>>> for, we have a link back to postorius. It has taken a lot of the >>> stress off>>> of the server. >>> >>> However, I still see where the database at times can run VERY hot.>>> >>> We have set up this server to run with PostgreSQL. When I do a ps >>> aux |>>> grep postgres, I find a lot of processes that have been spawned >>> to take>>> care of mailman - a dozen or so. However, one of the threads is >>> taking up>>> about 85% of the load, another thread about 14%, and little >>> dribbles here>>> and there from the rest of the threads. >>> >>> Here are the questions: >>> >>> 1) Has anyone run into problems like these? I would think that >>> maybe this>>> is an issue having so many mailing lists, but honestly there >>> aren't that>>> many people using the system yet. It is going to ramp up as the >>> users>>> realize that we are live again, but we aren't getting that much >>> traffic.>> >> I can say that in general, Postorius doesn't really optimize REST API >> queries and so loading a single page can sometimes make things bad.>> >> Currently, the largest installation of Mailman 3 that we know of >> runs at lists.fedoraproejct.org and the administrators of the list >> would know how well does our Web UI scale.>> >> However, I don't think we have anyone actively working on trying to >> fix this particular issue. Pull Requests and fixes are welcome :) >> >>> >>> 2) Is there any configuration that I can set, especially with >>> regards to the database, so that it can be used more efficiently >>> and not throttle through one or two threads?>> >> There isn't any configuration that would change how databases are >> used in Mailman Core. Django *may* have some, but I don't know about >> any of them of the top of my head.>> >> thanks, >> Abhilash -- Abhilash Raj maxking(a)asynchronous.in

7 years, 3 months

Re: Link to account

by Mark Sapiro

On 4/11/23 23:08, Lars Bjørndal wrote: > > I would like to have postorius at .../mailman3 and admin, accounts etc. under .../mailman3/admin .../mailman3/accounts etc. Try this. In urls.py ``` urlpatterns = [ path( '', RedirectView.as_view(url=reverse_lazy('list_index'), permanent=True), ), path('mailman3/archives/', include('hyperkitty.urls')), path('', include('django_mailman3.urls')), path('mailman3/accounts/', include('allauth.urls')), path('mailman3/admin/', admin.site.urls), path('mailman3/', include('postorius.urls')), ] ``` In apache ``` <IfModule mod_proxy.c> ProxyPreserveHost On ProxyPass "/mailman3/archives" "http://127.0.0.1:8000/mailman3/archives" ProxyPass "/mailman3/accounts" "http://127.0.0.1:8000/mailman3/accounts" ProxyPass "/mailman3/admin" "http://127.0.0.1:8000/mailman3/admin" ProxyPass "user-profile" "http://127.0.0.1:8000/user-profile" ProxyPass "/mailman3" "http://127.0.0.1:8000/mailman3" </IfModule> ``` -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 2 months

requiring questionnaire from new users

by Stephen J. Turnbull

Ethan Furman writes: > Is there a way to require new users to answer a questionnaire when > they subscribe? You could have it in the Postorius signup page template itself (see discussion below), but there currently is no way in Mailman to communicate that information to the moderator. Without modifying Mailman, you could have a separate signup page that invokes a Python program that notifies the admins and collects arbitrary data, as well as passing the normal subscription information to Mailman core for the normal address validation and moderator approval process. I have not done this myself, but a number of organizations have reported successfully creating alternative signup mechanisms (I haven't heard of this one). Such a signup page is the approach I'd recommend if the questions you have in mind would get free-form responses. Unfortunately, I do not believe the questionnaire responses could be displayed in the normal list admin interface without patching both Postorius and Mailman core, so this communication would have to be outside the normal procedure (but you would still use Postorius/core for the email validation and moderator approval steps). Perhaps one of the other devs has an idea about integrating it with the normal interface without patching. > I suspect that one of the lists I manage is getting a lot of > subscribers who don't understand what that particular list is for, > and having to answer a question or two would allow me to direct > them to better resources. I'm not sure how you want this to work. I guess you're getting this suspicion because new posters are posting a lot of off-topic stuff? So you want to have a subscription process where the moderators approve all subscriptions, and they would see the questionnaire responses in the subscription approval request? Not sure how well a simple questionnaire would capture this problem (see below), but it could help a lot, too. An alternative that doesn't inform the moderators is to modify the signup page template to (1) require the user check off the description of the list point by point (like a TOS notice, this is obviously not an improvement from the well-informed subscriber's point of view, though) and (2) provide links to commonly suggested alternative resources in the page template (these might be useful even to well-informed subscribers). The page templates are standard Django, so I'm sure that there's some facility for this. For example, if you were talking about python-dev, you could have Do you want to participate in implementation-focused discussions about work in progress to improve the Python language and its standard library? ( ) yes ( ) no If you answered 'yes', welcome! Do note that the focus is narrowly on implementation, and Python provides <a>many other venues</a> for new <a>ideas</a> and special interests like <a>distribution</a> and <a>documentation</a>. If you answered 'no', consider others of our <a>many lists</a> and our <a>Discourse channels</a>. - Do you want to discuss the best way to write a program in Python? ( ) yes ( ) no If you answered 'yes', <a>python-list</a> is probably a better venue for that. - Do you have a bug to report? ( ) yes ( ) no If you answered 'yes', please report it on the <a>issue tracker</a>. Discussion will take place there in any case, and you can attach a patch there if you have one. - Do you want to discuss a new idea to include in the Python language or standard library? ( ) yes ( ) no If you answered 'yes' but don't have a complete patch yet, <a>python-ideas</a> is probably a better venue for you. If you do have a patch, you should post it as a merge request on <a>GitHub</a>, too. This approach could get REALLY tedious (core-mentorship, scads of SIGs, etc). I think you'd have to add your own Javascript if you wanted to hide each recommendation until the new subscriber checks 'yes', but that would alleviate the annoyance quite a bit, I think. The only hitch is that I'm not sure Postorius wouldn't glitch on the extra information from the page, but even if so, that can probably be dealt with (a form without a submit button?) Regards, Steve

3 years, 10 months

Deleting archives

by Henrik Rasmussen

Docker-mailman 0.1: I have deleted some lists that I used for testing, but Hyperkitty seems to preserve the archives in the Available lists. Clicking on a list archive gets me to the archive of that list (with no posts because I haven't posted anything to the list). The URL is https://[WEBSITE]/hyperkitty/list/[LISTNAME](a)[LISTDOMAIN]/<https://[WEBSITE]/hyperkitty/list/%5bLISTNAME%5d(a)%5bLISTDOMAIN%5d/> (with a (a)), but when I click Manage subscription, the URI changes to https://[WEBSITE]/postorius/lists/[LISTNAME].[LIST<https://[WEBSITE]/postorius/lists/%5bLISTNAME%5d.%5bLIST>DOMAIN/ (with a dot instead of (a) which seems normal behavior) and I get a "Not Found The requested URL /postorius/lists/[LISTNAME]DOT[LISTDOMAIN]/ was not found on this server. " error. Of course I can't manage the subscription for a list I have deleted, but shouldn't the archive for a list be deleted along deleting the list. Henrik Rasmussen

7 years, 10 months

Re: Moderated unsub requests are not provided by mailman3 service

by Joachim Schrod

Mark Sapiro writes: > On 11/29/24 4:47 PM, Joachim Schrod wrote: > > > System information: mailman3 3.3.3; Postorius (mailman3-web) 1.3.4; > > both on Debian 11 (bullseye). > > > > I know that this is out of date; thus a definitive info "this is fixed > > by now" is quite OK for me. (Debian upgrade is planned as the next > > action after getting Mailman3 to work for that use case.) > > Yes, this is fixed. Thanks for the information that it's fixed in Postorius 1.3.6; that information is a great help, it means that I don't need to spend more time on debugging resolved issues. I tried to gather data if an update to Debian 12 (bookworm) would help. But packages.debian.org is not helpful at all, for me. I know that this is beyond the realm of this mailing list, but perhaps someone here has that information and can supply it: Debian 12 (bookworm) package mailman3-web has the version "0+20200530-2.1". This is supposed to be from some source package mailman-suite that is a salsa project that I don't grok at 1st and 2nd sight. Can anybody please provide me with information what versions of mailman3 core and mailman3-web are in this package; i.e., in Debian 12 (bookworm)? Sorry, but I'm not versed enough to understand the information about Debian packages that my search delivers. Cheers, Joachim -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Joachim Schrod Email: joachim(a)schrod.org Roedermark, Germany ``How do we persuade new users that spreading fonts across the page like peanut butter across hot toast is not necessarily the route to typographic excellence?'' -- Peter Flynn

6 months, 2 weeks

Re: Affinity Mailman Is Now Beta

by Mark Sapiro

On 6/18/20 12:44 PM, Brian Carpenter wrote: > Dear Mailman Admins and Users, > > Affinity Mailman is our new admin interface for Mailman 3 and we intend > to use it to replace Postorius. It has been under development for about > 3 months now and it recently reached beta. Just to be clear, Brian has the full support of the GNU Mailman project in his endeavor to create a new web UI for Mailman 3, but "our" above means Brian and EMWD, not the GNU Mailman project. We of the GNU Mailman project are delighted that Brian is working on an alternative web UI, but we have no plan to abandon Postorius and HyperKitty. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years

Re: Held messages: Long time waiating for response of the Mailman API

by Mark Sapiro

On 1/24/22 15:06, Mark Sapiro wrote: > On 1/18/22 06:24, Stephan Krinetzki wrote: >> Hello, >> >> We are using Mailman 3.3.5 with Hyperkitty 1.3.5. Mailman 3 is >> installed in a venv and deployed using Apache and uwsgi. This works >> quite well so far, only withheld messages cause us a problem: If we >> try to approve them via Postorius, the message is delivered to the >> list immediately, but the success message in Postorius takes a long >> time - on average the process hangs for three minutes until the >> message is no longer displayed. We have already adjusted the timeout >> in gunicorn.cfg, but this does not improve the situation. Are there >> any tips on how we can speed up the REST API? Or is it even a >> configuration error? > > > There are two steps in handling the POST for a held message. The first > step is accepting, rejecting or discarding the message. This is > happening immediately. > > The second step is handling the poster's moderation action based on the > choice from the Set member moderation dropdown in the view of the held > message. This is processed by the code at > <https://gitlab.com/mailman/postorius/-/blob/master/src/postorius/views/list…>. > If no selection has been made, the default value POSTed for > moderation_choice is `no-action` and that code is skipped. If it is one > of the choices `Hold for moderation`, `Reject (with notification)`, > `Discard (no notification)`, `Accept immediately (bypass other rules)` > or `Default processing` that is processed. So presumably a selection has > been made and processing is stuck on getting the member or setting the > member's moderation action. > > Possibly the logs of your database manager will have information. > Some additional info. By gunicorn.cfg, do you mean the file at mailman/config/gunicorn.cfg? If you are using gunicorn as a wsgi server, that is separate from the gunicorn serving the REST API and its config doesn't affect the REST API. The relevant config is mailman/config/gunicorn.cfg?. Also, I doubt the issue is with the REST API per se, but rather it is in Mailman core's processing of the particular endpoint and if that is slow, it is most likely some kind of database issue. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 4 months

Re: Database running VERY hot

by Abhilash Raj

Hi Darren, On 3/8/18 9:43 AM, Darren Smith wrote: > Hello all, > > We had an old installation of Mailman 2.1 across three servers with over > 32,000 mailing lists on them. Due to circumstances beyond my control, we > had to shut them down and quickly set up Mailman 3 on a new server. > > We have Mailman3 set up on a single server this time, as we were having > problems with a 3-server setup and Hyperkitty and archives. Long story > short, it is up and running, and our users are starting to come back. > > However, we do not have very many users hitting Postorius and Hyperkitty > yet, and we are seeing a distressing number of 504 gateway timeouts. > > We noticed that there is a significant number of queries to the API > whenever hitting Postorius's front page to load the mailing lists, and then > on every load. It looks as if it makes a single call to the API to get > information about a single list, and does this 10 times - once for each > list - to render the page. This gets much worse when users set the view to > 200 per page, as they are inclined to do with 32,000 mailing lists... > > What we've done, actually, is to override the front page and point the > users to a different set of static pages where we have all of our lists > categorized and searchable, and when they find the list they are looking > for, we have a link back to postorius. It has taken a lot of the stress off > of the server. > > However, I still see where the database at times can run VERY hot. > > We have set up this server to run with PostgreSQL. When I do a ps aux | > grep postgres, I find a lot of processes that have been spawned to take > care of mailman - a dozen or so. However, one of the threads is taking up > about 85% of the load, another thread about 14%, and little dribbles here > and there from the rest of the threads. > > Here are the questions: > > 1) Has anyone run into problems like these? I would think that maybe this > is an issue having so many mailing lists, but honestly there aren't that > many people using the system yet. It is going to ramp up as the users > realize that we are live again, but we aren't getting that much traffic. I can say that in general, Postorius doesn't really optimize REST API queries and so loading a single page can sometimes make things bad. Currently, the largest installation of Mailman 3 that we know of runs at lists.fedoraproejct.org and the administrators of the list would know how well does our Web UI scale. However, I don't think we have anyone actively working on trying to fix this particular issue. Pull Requests and fixes are welcome :) > > 2) Is there any configuration that I can set, especially with regards to > the database, so that it can be used more efficiently and not throttle > through one or two threads? There isn't any configuration that would change how databases are used in Mailman Core. Django *may* have some, but I don't know about any of them of the top of my head. thanks, Abhilash

7 years, 3 months

Re: Postorius no connection to REST API

by Mark Sapiro

On 8/11/21 4:18 AM, Richard Rosner wrote: > Mark Sapiro wrote: >>> You are in a better position to answer that than am I. >> What does sudo netstat -lntp show? > > A lot. But since most of that isn't relevant here: > Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name > tcp 0 0 0.0.0.0:465 0.0.0.0:* LISTEN 20241/master > tcp 0 0 127.0.0.1:8024 0.0.0.0:* LISTEN 14076/python3 > tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 20241/master > tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 14080/python3 So some python3 process is listening on port 8001 > tcp 0 0 0.0.0.0:587 0.0.0.0:* LISTEN 20241/master > tcp6 0 0 :::80 :::* LISTEN 13882/apache2 > tcp6 0 0 :::465 :::* LISTEN 20241/master > tcp6 0 0 :::25 :::* LISTEN 20241/master > tcp6 0 0 :::443 :::* LISTEN 13882/apache2 > tcp6 0 0 :::587 :::* LISTEN 20241/master > >> What does ps -fwwa|grep rest show? > > root 15055 14843 0 12:58 pts/1 00:00:00 grep rest > > So whatever it's supposed to find, it's not there But whatever is listening on port 8001 is apparently not Mailman's rest server. What does ps -fww 14080 or whatever PID is currently listening on port 8001 show. >>> mailman3-web.service must also run as list. > > I changed that. It didn't like it. OK. Perhaps instead you should have uid: list gid: list in your uwsgi configuration if you don't already. > >>> So, what do you have in your apache config for proxying to uwsgi and >> what's your uwsgi configuration. > > lists-ssl.conf: > <VirtualHost *:443> > ServerAdmin admin(a)domain.de > ServerName lists.domain.de > > Alias /mailman3/favicon.ico /var/lib/mailman3/web/static/postorius/img/favicon.ico > Alias /mailman3/static /var/lib/mailman3/web/static > > <Directory "/var/lib/mailman3/web/static"> > Require all granted > </Directory> > > <IfModule mod_proxy_uwsgi.c> > ProxyPass /mailman3/favicon.ico ! > ProxyPass /mailman3/static ! > ProxyPass "/mailman3" "unix:/run/mailman3-web/uwsgi.sock|uwsgi://localhost:8001/" I'm not familiar enough with this form of ProxyPass using sockets to understand what the localhost:8001 does in this context, but uwsgi should not be doing anything with port 8001. In a configuration using TCP, it would listen on port 8000. Port 8001 is where Mailman's REST API server listens. uwsgi should receive connects in your case via the unix socket and then pass them to Django via the application in wsgi.py Also, you may want other paths proxied to uwsgi, namely at least some of hyperkitty, postorius, archives, accounts, admin and user-profile. > </IfModule> ... > I guess with uwsgi config you mean the /etc/mailman3/uwsgi.ini file? > [uwsgi] > # Port on which uwsgi will be listening. > uwsgi-socket = /run/mailman3-web/uwsgi.sock > > #Enable threading for python > enable-threads = true > > # Move to the directory wher the django files are. > chdir = /usr/share/mailman3-web > > # Use the wsgi file provided with the django project. > wsgi-file = wsgi.py > > # Setup default number of processes and threads per process. > master = true > process = 2 > threads = 2 > > # Drop privielges and don't run as root. > uid = www-data > gid = www-data These should be 'list', not 'www-data'. That's your permissions issue on settings.py -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years, 10 months

Installed but mailman reports missing hyperkitty module

by jbernal.web.dev＠gmail.com

I've opted for the Mailman3 package installation on Ubuntu 22 LTS on top of a manually installed Mailman3 core package from the Ubuntu 20 Debian repository. Everything is working great on my installation. However, if I try to enable hyperkitty in /etc/mailman.cfg postorius reports 500 status codes on all pages. /var/log/mailman3/mailman.log reports this error: ModuleNotFoundError: No module named 'mailman3_hyperkitty' Any idea why postorius can't seem to find the installed hyperkitty module? Stack trace: 2023-02-08 12:48:14 [FALCON] [ERROR] GET /3.1/lists/test.listdev.ocdla.org/archivers => Traceback (most recent call last): File "falcon/app.py", line 361, in falcon.app.App.__call__ File "/usr/lib/python3/dist-packages/mailman/rest/lists.py", line 405, in on_get archiver_set = IListArchiverSet(self._mlist) File "/usr/lib/python3/dist-packages/zope/component/_api.py", line 156, in adapter_hook return sitemanager.queryAdapter(object, interface, name, default) File "/usr/lib/python3/dist-packages/zope/interface/registry.py", line 354, in queryAdapter return self.adapters.queryAdapter(object, interface, name, default) File "/usr/lib/python3/dist-packages/mailman/database/transaction.py", line 85, in wrapper return function(args[0], config.db.store, *args[1:], **kws) File "/usr/lib/python3/dist-packages/mailman/model/mailinglist.py", line 614, in __init__ for archiver in config.archivers: File "/usr/lib/python3/dist-packages/mailman/config/config.py", line 280, in archivers archiver = call_name(class_path) File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 69, in call_name named_callable = find_name(dotted_name) File "/usr/lib/python3/dist-packages/mailman/utilities/modules.py", line 52, in find_name module = import_module(module_path) File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked ModuleNotFoundError: No module named 'mailman3_hyperkitty'

2 years, 4 months

Mailman-users search results for query "Postorius"