Search results for "Postorius" - Mailman-users

Re: Digest Formatting

by Stephen J. Turnbull

mailman(a)manygoodideas.com writes: > 1) Is there a way to trigger a digest for a user (for testing > purposes)? Specifically, is there a function that can be > revealed via the REST api to an interface like Postorius and > Affinity? You can't trigger a digest to an individual member (unless they're the only active digest subscriber to the list, of course, so you can experiment on a test list if it's not specific to a particular list). I thought there was a way to trigger immediate distribution of a digest immediately, but I guess I'm remembering that from Mailman 2. Neither the Postorius management interface nor the core documentation suggests there's such a trigger in Mailman 3. > > 2) I have customized list:member:digest:header, but am not seeing > the custom text where I thought it should be. The documentation > seems to imply that both list:member:digest:header and > list:member:digest:footer surround each message in the digest Which documentation do you mean? As far as I know (Mark will have authoritative information if I'm wrong), the digest header and footer appear at the top and bottom of the *whole* digest message (ie, once each per digest), not surrounding or in the bodies of the component messages.

4 years, 7 months

Re: Messages disappearing into hole??

by Mark Sapiro

On 8/30/20 6:15 AM, Ruth Ivimey-Cook wrote: > > I looked around the interface and decided to set up the sender as a > non-member, as that appeared to permit them to be "special" and outside > of the normal access rules for the list. Before I did that I could send > a message from this sender and that would be sent to me as list owner > for moderation, so both incoming (from my mailserver) and outgoing (to > it) can be completed. > > However, with a non-member who is auto-accepted the message is passed > from my mailserver to mailman and then disappears. I cannot find > anything in the logs to indicate what happened next. I have tried > setting the default non-member policy to both 'hold' and 'default' with > no change in that behaviour. The logs for the mailserver show the > message being sent to mailman; mailman's "smtp.log" shows an LMTP > message being received ok, and then nothing. > > Any ideas how to find out what is going on? I can share config if that > helps but not sure what would be useful. First, this is a bad way to configure an announce list because anyone can post by spoofing the authorized posting address. The better way is to set a list moderator password and post using an Approved: <password> header or pseudo header. See <https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rules/docs…> for details. As far as the disappearing message is concerned, it's hard to say. What is the non-member's moderation action? Unless it is 'list default' setting the list's default non-member policy has no effect. Also, if the non-member's address or a pattern matching it is in accept_these_nonmembers, that takes precedence over whatever the non-member's moderation action is. One guess about the disappearing message is content filtering may be removing all the message. What is the setting for Filter Action? > As a side-issue, at one point I added a regex matching the sender > address to the acceptable members list, but find that I cannot now > delete it (using postorius): If I delete the field and Save, it returns > on page refresh. There is an issue with Postorius. You can't empty these settings. If you clear the box or or have it contain only whitespace, Postorius won't change the prior setting. In order to remove the only entry in Postorius, you have to replace it with a bogus entry or you can remove it with `mailman shell`. I've just filed <https://gitlab.com/mailman/postorius/-/issues/444> for this. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 9 months

Re: Verification Failed Error

by Odhiambo Washington

On Fri, 21 Dec 2018 at 03:25, Abhilash Raj <maxking(a)asynchronous.in> wrote: > > > On Thu, Dec 20, 2018, at 4:05 PM, brian(a)emwd.com wrote: > > Abhilash Raj wrote: > > > It should be pointing to Exim running on host, so > > > > > > EMAIL_HOST = 172.19.199.1 > > > > That fixed the network not found error. Thank you for your help there. > > > > Now I am back to the verification error: > > > > "smtplib.SMTPRecipientsRefused: {'brian(a)emwd.com': (550, b'Verification > > failed for <postorius(a)host.name.com>\nUnknown user\nSender verify > > failed')}" > > > > I am at a lost on how to get exim to allow mail sent from > > postorius(a)host.name.com. Obviously host.name.com is the mm3 domain that > > I am using. > > A quick search says that Exim is trying to verify that ` > postorius(a)host.name.com` > is a real mailbox (which I am assuming it is not), and rejecting the email > coming > from Postorius. > > You probably have > > verify = sender > > somewhere in your exim.conf, which is causing this. You can either modify > the > > DEFAULT_FROM_EMAIL > > in your settings_local.py so that it is a real address that Exim can > verify, or, > you can remove verify = sender from your Exim configuration. 2nd option > might > not be the best if you want to properly combat spam. > > Exim experts would be able to help you better with this. > > > > > Hi Brian, In postorius(a)host.name.com, please change the host.name.com to match the MM3_DOMAINS or whatever it is called in your MACROS. Remember, it has to be a VALID hostname that you will be using for Mailman3 lists. That, coupled with the IPs 172.19.199.x being in relay_from_hosts should let the address to pass verification with Exim. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

6 years, 5 months

Re: Any known limit of number of lists that one can be owner/member of?

by Mark Sapiro

On 5/8/19 3:33 AM, Alan So wrote: > We have just migrated hundreds of lists from Mailman 2 to Mailman 3. It was smooth in most of the cases. However, we just notice that there are some abnormal behaviour of Postorius for particular users who are owner/member of a large number of lists. We further look into a particular user who is owner of 84 lists. When the user login Postorius, only 50 lists are displayed no matter what the number of "Results per page" is chosen. Prev/Next page cannot be clicked as there is only 1 page. See <https://gitlab.com/mailman/postorius/issues/335> > When I use the following mailman client snippet, it also displays the same 50 lists: > > lists = client.find_lists('xxx(a)yyy.com', 'owner') > for m_list in lists: > print(m_list.fqdn_listname) > > Did we hit any limit of find_lists? Is it a bug or are something wrong during the migration? find_lists defaults to count=50. What happens if you do lists = client.find_lists('xxx(a)yyy.com', 'owner', count=100) -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 1 month

Re: Held messages: Long time waiating for response of the Mailman API

by Stephen J. Turnbull

Thanks for your patience with our team; I had already received your second "ping" and Mark's response, but due to the way my MUA presents my INBOX I didn't see it (aside to self: *don't* answer Mailman traffic from the inbox, use the "Mailman 3 virtual folder" view! :-). I would have written very differently if I had seen it.... Stephan Krinetzki writes: > Then we should maybe move to gunicorn, if this is the better way > (or at least better supported way) That's definitely a possibility. Among other things, gunicorn is a Python application, so all the Mailman devs have some hope of debugging it or at least localizing it. That's not true of uwsgi. The other possibility if you don't need high performance is to use the Apache mod_wsgi plugin which seems to be quite stable (that's what I do out of laziness -- I really should have uwsgi and gunicorn setups for testing -- my own throughput and admin needs are very modest). > After we enabled the "slow query log", we got in the postgresql > database the following output: > > 2022-01-25 12:49:41.538 CET [16126] LOG: unexpected EOF on > client connection with an open transaction Ouch. That is a database problem but I don't think it's a problem with the database. There are two database connections involved, one is Mailman core, the other is Django, but this > > > the message is delivered to the list immediately, says that core has finished at least some of its business. I guess it's possible that core drops the transaction on the floor, but that seems weird because I don't think this is a common problem. I'm not going to guess further, maybe Mark or Abhilash has something. ISTR other delay issues that have to do with very large databases. Do you have huge numbers of users, lists, or subscriptions to particular lists? Of course it could be Postorius/Django, but I think Postorius delegates all the logic and database work to core for this. Django's databases are user- and authentication-oriented, I don't recall any list-management logic in it that would induce this: > 2022-01-25 15:34:36.047 CET [16864] LOG: duration: 30760.028 > ms statement: DELETE FROM pendedkeyvalue WHERE > pendedkeyvalue.id = XXX Next log: > ERROR 2022-01-25 18:55:55,037 2599 django.request Service Unavailable: > /postorius/lists/games.lists.rwth-aachen.de/held_messages > ERROR 2022-01-25 18:58:24,325 2588 postorius Mailman REST API > not available Not sure about this. It could indicate a deadlock where each of core and the backend database is waiting for the other, so core isn't able to respond to Postorius. It looks to me like the rest of the traceback is a natural consequence of unavailability of the REST API. > * The venv is in /opt/mailman/mailman-venv > * The settings are in /opt/mailman/mailman-venv/mailman-suite/mailman-suite_project I don't see much to flag here. I don't recall whether allauth.socialaccount is supposed to be enabled or disabled if you have no allauth.socialaccount.providers enabled, but it's probably OK (and I don't see how it could be related to this problem): > contenten of settings.py > # Application definition > > INSTALLED_APPS = ( [...] > 'allauth', > 'allauth.account', > 'allauth.socialaccount', Database looks fine: > # Database > # https://docs.djangoproject.com/en/1.8/ref/settings/#databases > > DATABASES = { > #'default': { > # Example for PostgreSQL (recommanded for production): > 'default': { > 'ENGINE': 'django.db.backends.postgresql_psycopg2', > 'NAME': 'mailman', > 'USER': 'mailman', > 'PASSWORD': '<PASS>', > 'HOST': 'localhost', > } > } This shouldn't have anything to do with slow Postorius (full-text search is relevant only to archives, ie, HyperKitty), but Whoosh is known to be slower than Xapian: > # > # Full-text search engine > # > HAYSTACK_CONNECTIONS = { > 'default': { > 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine', > 'PATH': os.path.join(BASE_DIR, "fulltext_index"), > # You can also use the Xapian engine, it's faster and more accurate, > # but requires another library. > # http://django-haystack.readthedocs.io/en/v2.4.1/installing_search_engines.h… > # Example configuration for Xapian: > #'ENGINE': 'xapian_backend.XapianEngine' > }, > } > Nope, the virtual machine is dedicated to Mailman3. Mailman3 as > venv, uwsgi as Webservice, Apache as Webserver, Postfix as > Mailserver and PostgreSQL 10 als Database, all on the same host Oh, well, I guess the case against Mailman gets a little stronger.... > At moment i think its a slow database. Because of the slow > database, the Mailman API is slow and so on. We are trying now to > solve the database part with postgresqltuner - maybe we will see > more details then. OK, keep us posted. I will say that it looks from the logs discussed above that PostgreSQL is waiting for Mailman, not vice versa (but that doesn't rule out a deadlock where both are waiting for the other). I hope my message trims things down enough that it rings a bell for somebody. Steve

3 years, 4 months

Re: Apache+mod_wsgi issue

by Mark Sapiro

On 12/24/22 00:00, Odhiambo Washington wrote: > > These are my files: > 1. urls.py > <CUT > > from django.conf.urls import include > from django.urls import re_path > from django.contrib import admin > from django.urls import reverse_lazy > from django.views.generic import RedirectView > > urlpatterns = [ > re_path(r'^$', RedirectView.as_view( > url=reverse_lazy('list_index'), > permanent=True)), > re_path(r'^postorius/', include('postorius.urls')), > re_path(r'^hyperkitty/', include('hyperkitty.urls')), > re_path(r'', include('django_mailman3.urls')), > re_path(r'^accounts/', include('allauth.urls')), > # Django admin > re_path(r'^admin/', admin.site.urls), > re_path(r'^mailman3/', include('postorius.urls')), > re_path(r'^archives/', include('hyperkitty.urls')), > ] > </CUT> You don't need all of those. ^postorius/ and ^mailman3/ are two different ways to access Postorius, and ^hyperkitty/ and ^archives/ are two different ways to access HypereKitty. You only need one of each, but having both is OK. > 2. apache.conf > <CUT> > ... > Alias /static "/opt/mailman/mm/static" > <Directory "/opt/mailman/mm/static"> > Require all granted > </Directory> > # WSGIScriptAlias /mailman3 /opt/mailman/mm/wsgi.py > # WSGIScriptAlias /accounts /opt/mailman/mm/wsgi.py > # WSGIScriptAlias /admin /opt/mailman/mm/wsgi.py > # WSGIScriptAlias /archives /opt/mailman/mm/wsgi.py > # WSGIScriptAlias /user-profile /opt/mailman/mm/wsgi.py > # WSGIScriptAlias /hyperkitty /opt/mailman/mm/wsgi.py > # WSGIScriptAlias /postorius /opt/mailman/mm/wsgi.py > WSGIScriptAlias / /opt/mailman/mm/wsgi.py This will send every http request to mod_wsgi except for requests to /static. This is OK if there are no other web sites on this server and the only http requests are for Django related things. ... > I have tested with creating all those commented out aliases and that ended > up still giving me /mailman3/mailman3/* which I did not > like, although everything appeared to work. > I then ended up with only "WSGIScriptAlias / /opt/mailman/mm/wsgi.py" > and now when I type https://mm3-lists.kictanet.or.ke I end > up at https://mm3-lists.kictanet.or.ke/mailman3/lists/. Of course something > has appended "/mailman3" > So this has resulted in all URLs having /mailman3/ appended, except for the > /archives link. How about https://mm3-lists.kictanet.or.ke/user_profile/ ? > Now my question is about what could be adding the /mailman3. Is it possible > that I installed something out of the ordinary? It's a mod_wsgi thing. I haven't used mod_wsgi for Mailman for some time (I now use gunicorn on all the sites I manage), but when I did I think I had WSGIScriptAlias /mm3 /opt/mailman/mm/wsgi.py and all the URLs were like https://example.com/mm3/mailman3/..., https://example.com/mm3/archives/..., etc. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 5 months

Re: Database running VERY hot

by Darren Smith

Abhilash, Thanks for the response. I'm going over to lists.fedoraproject.org to see if they have done any fine tuning. Our installation is at https://lists.rootsweb.ancestry.com, if you're interested in seeing an installation with over 32,000 mailing lists. As I mentioned, it was big enough that we have disabled the browse experience in Postorius and are hosting our own pages for browsing to a new list. If we get time to work on this we will certainly create a pull request. :) On Thu, Mar 8, 2018 at 11:49 AM, Abhilash Raj <maxking(a)asynchronous.in> wrote: > Hi Darren, > > > On 3/8/18 9:43 AM, Darren Smith wrote: > >> Hello all, >> >> We had an old installation of Mailman 2.1 across three servers with over >> 32,000 mailing lists on them. Due to circumstances beyond my control, we >> had to shut them down and quickly set up Mailman 3 on a new server. >> >> We have Mailman3 set up on a single server this time, as we were having >> problems with a 3-server setup and Hyperkitty and archives. Long story >> short, it is up and running, and our users are starting to come back. >> >> However, we do not have very many users hitting Postorius and Hyperkitty >> yet, and we are seeing a distressing number of 504 gateway timeouts. >> >> We noticed that there is a significant number of queries to the API >> whenever hitting Postorius's front page to load the mailing lists, and >> then >> on every load. It looks as if it makes a single call to the API to get >> information about a single list, and does this 10 times - once for each >> list - to render the page. This gets much worse when users set the view to >> 200 per page, as they are inclined to do with 32,000 mailing lists... >> >> What we've done, actually, is to override the front page and point the >> users to a different set of static pages where we have all of our lists >> categorized and searchable, and when they find the list they are looking >> for, we have a link back to postorius. It has taken a lot of the stress >> off >> of the server. >> >> However, I still see where the database at times can run VERY hot. >> >> We have set up this server to run with PostgreSQL. When I do a ps aux | >> grep postgres, I find a lot of processes that have been spawned to take >> care of mailman - a dozen or so. However, one of the threads is taking up >> about 85% of the load, another thread about 14%, and little dribbles here >> and there from the rest of the threads. >> >> Here are the questions: >> >> 1) Has anyone run into problems like these? I would think that maybe this >> is an issue having so many mailing lists, but honestly there aren't that >> many people using the system yet. It is going to ramp up as the users >> realize that we are live again, but we aren't getting that much traffic. >> > > I can say that in general, Postorius doesn't really optimize REST API > queries and so loading a single page can sometimes make things bad. > > Currently, the largest installation of Mailman 3 that we know of runs at > lists.fedoraproejct.org and the administrators of the list would know how > well does our Web UI scale. > > However, I don't think we have anyone actively working on trying to fix > this particular issue. Pull Requests and fixes are welcome :) > > >> 2) Is there any configuration that I can set, especially with regards to >> the database, so that it can be used more efficiently and not throttle >> through one or two threads? >> > > There isn't any configuration that would change how databases are used in > Mailman Core. Django *may* have some, but I don't know about any of them of > the top of my head. > > thanks, > Abhilash >

7 years, 3 months

Re: FILTER_VHOST not working

by Mark Sapiro

On 3/22/21 1:03 PM, Johannes Rohr wrote: > In /etc/mailman3/mailman-web.py I have added > FILTER_VHOST = True > so that only the list for the current domain are shown. > > But this has no effect. The current situation with Postorius >=1.3.3 and Hyperkitty >=1.3.3 is for HyperKitty, FILTER_VHOST = True works. For Postorius, it works only if in the domains view, each mail host has a unique web host. It doesn't work. See this post <https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message…> and others in that thread for more info. Prior to Postorius 1.3.3 and Hyperkitty 1.3.3 it would only work if the mail host and web host values were identical for the domain. > > What's more, in hyperkitty I see at the top left the lable for the > default domain, even though I am visiting from another domain. That has nothing to do with FILTER_VHOST. The displayed there is the site referenced in the SITE_ID setting. > The web domains are exactly the same as the mail domans > (lists.[domain].[tld]) In that case, I'm not sure what's going on. > In the Nginx config I have added > > location / { > proxy_set_header X-Forwarded-Proto $scheme; > proxy_set_header X-Forwarded-Host $host; > proxy_pass http://localhost:8000/; > [....] > > } > > to make sure that mailman3 knows from which URL it is visited. > > Am I overlooking something? > > I am using the distribution packages on Ubuntu 20.04: > mailman3 3.2.2-1 > mailman3-web 0+20180916-10 > > I see, the mailman3-web package is quite dated. Is it recommended to > install via virtualenv instead? Yes, the issue is at least partially that that mailman3-web package doesn't have any of the FILTER_VHOST fixes. And yes, if you don't want to be stuck with dated packages tha don't include recent fixes, we recommend installation as documented at <https://docs.mailman3.org/en/latest/install/virtualenv.html> -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

4 years, 2 months

Re: 400 error on held messages

by Dan Caballero

Here is the stack trace from the mailmansuite.log. ERROR 2023-10-04 10:59:03,587 2091 postorius Un-handled exception: HTTP Error 400: invalid literal for int() with base 10: 'oAAAANSUhEUgAACfYAAAiVBORw0KGgoAAAANSUhEUgAACfYAAAYKCAIAAAAQvT9CAAAACXBIWXMAAC4jAAAuIwF4pT92AAAg\r\n' Traceback (most recent call last): File "/opt/mailmanve/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/mailmanve/lib/python3.9/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view return view_func(request, *args, **kwargs) File "/opt/mailmanve/lib/python3.9/site-packages/postorius/auth/decorators.py", line 63, in wrapper return fn(*args, **kwargs) File "/opt/mailmanve/lib/python3.9/site-packages/postorius/views/list.py", line 864, in list_moderation held_messages = paginate( File "/opt/mailmanve/lib/python3.9/site-packages/django_mailman3/lib/paginator.py", line 71, in paginate objects = paginator.page(page_num) File "/opt/mailmanve/lib/python3.9/site-packages/django_mailman3/lib/paginator.py", line 47, in page result = self.function(count=self.per_page, page=number) File "/opt/mailmanve/lib/python3.9/site-packages/mailmanclient/restobjects/mailinglist.py", line 164, in get_held_page return Page(self._connection, url, HeldMessage, count, page) File "/opt/mailmanve/lib/python3.9/site-packages/mailmanclient/restbase/page.py", line 37, in __init__ self._create_page() File "/opt/mailmanve/lib/python3.9/site-packages/mailmanclient/restbase/page.py", line 62, in _create_page response, content = self._connection.call(self._build_url()) File "/opt/mailmanve/lib/python3.9/site-packages/mailmanclient/restbase/connection.py", line 160, in call raise HTTPError(params.get('url'), response.status_code, urllib.error.HTTPError: HTTP Error 400: invalid literal for int() with base 10: 'oAAAANSUhEUgAACfYAAAiVBORw0KGgoAAAANSUhEUgAACfYAAAYKCAIAAAAQvT9CAAAACXBIWXMAAC4jAAAuIwF4pT92AAAg\r\n'

1 year, 8 months

Re: Changing subscription address

by Stephen J. Turnbull

Abhilash Raj writes: > Well a couple of stars have to align for it to work unfortunately. I'm not going to wait for the stars; I have backward-incompatible surgery in mind. > Add to that you can’t set an address as primary using just email > commands, you end up with what you are seeing. Right. The idea is that the first time an address is used, a User will be created and that address will be primary for that User. When the person wants to claim that address, they go to Postorius, create a Postorius account, do the address verification dance and set credentials, which links the Postorius account to the User via the Address. This is why the ability to merge Users becomes an urgent need: people who subscribe via email are going to end up with multiple Users. > We don’t necessarily need the flag on user to set an unverified primary > address, we might be able to get away without that. We could set the > address as primary *after* we verify it. We always verify addresses > regardless of the choice of the subscription mode (user or address). I'm not clear what you're saying. In general admins can't read your mail, so they can't verify you on a mass subscribe or an import. > Right now, P is explicit about it. You can choose if you want Primary > Address or one of the explicit addresses added to your account. The > primary address is the pre-selected choice, so if you go and just click > on subscribe button, you get the primary address being subscribed. If > you choose the explicit address then we subscribe the explicit address. > > Is that something you think needs modifying? No. The option to set a subscription to the literal primary address should still be available. I don't see a good reason to complicate the UI logic by comparing primary to literal addresses just to remove that one. I just want the primary address to be populated automatically, and inherit-from-primary be the default subscription address when the subscription address matches primary, *unless* the subscriber explicitly chooses that literal address in Postorius. This is backward-incompatible, but I think it will be overwhelmingly popular with new users and old. > The email join command does the first part you suggested, (if the lookup > succeed and matches primary address, use inherit) but doesn’t do the > second part (it will create a user, add the address to it and then use the > address to subscribe). We can and should fix that. I think that's all we need to do going forward. We do need a better UI for setting options (including address) for a selection of subscriptions, but that's an independent task. Thanks for looking at this, Abhilash! Steve

4 years, 2 months

Mailman-users search results for query "Postorius"