Search results for "Postorius" - Mailman-users

Re: Mailman not able to process "big" requests

by Abhilash Raj

On Mon, Apr 23, 2018, at 10:41 PM, Jonas Felber wrote: > Hi, > > We migrated from mailman2 to mailman3 a week ago. Since then we are > unable to send messages the our students through the mailing list. > Here are some symptoms: > > 1. I go to mass subscribe and subscribe 400 emails > 2. I click subscribe, the browser (in my case safari) tries to load the > page for a few seconds > 3. The request times out > 4. Once the service becomes responsible again, I see that only part of > my emails got subscribed (~50 in this case). > > For me this really looks like the emails are sent on the "web-thread" > that gets killed once the request times out. > Looking at this > https://gitlab.com/mailman/postorius/blob/master/src/postorius/views/list.p… > <https://gitlab.com/mailman/postorius/blob/master/src/postorius/views/list.p…> > seems to confirm my suspicions. Yes, you are correct. Mass Subscription current just subscribes each address one-by-one and thus times-out. I have created an issue for this https://gitlab.com/mailman/postorius/issues/264. > > Some quick overview of our infrastructure: The TCP endpoints are: > browser -> haproxy in kubernetes-entry-router -> nginx in kubernetes- > ingress -> nginx in mailman -> mailman I would very much appreciate if you'd share you configuration and/or a blog post on your setup. I have been trying to set Mailman use Kubernetes for a time, but haven't had time to do it. > We mitigated the issue by giving it more CPU power and increasing the > timeouts in one of the nginx endpoints. > We also have a problem with sending out emails to "large" (>50) groups, > as they seem to not getting sent out at all. Do you have any idea what > the problem here could be? Which MTA are you using? What do you see in mailman's logs and MTA's logs? I haven't seen this kind of problem before with Mailman 3. This very list is hosted using Mailman 3 and I believe has more than 50 subscribers. > > We would be really glad if you can help us getting our mailman up and > running, as I currently have to send emails out by "hand". > I will happily provide you with more details about our configuration, > just let me know what you need. > > Kind regards, > > Jonas > > Jonas Felber <jonas.felber(a)vis.ethz.ch <mailto:jonas.felber@vis.ethz.ch>> > Aktuar & CTF Präsident > > VIS - Verein der Informatik Studierenden an der ETH Zürich > CAB E 31, Universitätstr. 6, ETH Zentrum, CH-8092 Zürich > https://www.vis.ethz.ch <https://www.vis.ethz.ch/> > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > Email had 1 attachment: > + signature.asc > 1k (application/pgp-signature) -- Abhilash Raj maxking(a)asynchronous.in

8 years, 6 months

Re: Where should Postfix, Django, Hyperkitty, Postorius, Dart-Sass, etc. be installed if MM3 is in a python virtual environment?

by Andrew Hodgson

Ron / BCLUG <admin(a)bclug.ca> wrote: >Keith Christian wrote on 2024-05-24 16:41: >> pip install wheel mailman psycopg2-binary >> Should, or must, any of Django, Hyperkitty, Postorius, Dart-Sass, or >> any other supporting programs be installed into this virtual >> environment ? >As I understand it, mailman-hyperkitty get installed into the venv, but django and postorius are pulled in as a dependency of mailman-web (I >believe?) Yes that is correct. I've actually a slightly different approach and when creating the env choose to bring in system packages as well: python3 -m venv --system-site-packages --symlinks venv The packages I install on a fresh system are: sudo apt install python3-dev python3-wheel python3-psycopg2 sassc python3-rcssmin python3-venv python3-certbot-nginx postgresql exim4-daemon-light Then in venv I just run pip install mailman mailman-hyperkitty mailman-web uwsgi My goal here is that I want as minimum compilation tools on the system as possible. I installed the Ubuntu version of the psycopg2 package as it's the same version of the Postgresql software also maintained by Ubuntu. I seem to remember rcsmin needing compilation tools so installed the Ubuntu maintained package. I typically don't install Links on my systems but remember it was a requirement at some point so will probably include it on a new Ubuntu 24.04 host I am planning. The Certbott installation does itself pull in a few Python libraries that Mailman uses so when installing Mailman in the venv it doesn't have to pull as many packages. I did try and see what packages I could include from the system in this way but when we start adding more packages Apt starts to get crazy with dependencies, so I made the goal of just pulling in packages which required compilation. Originally I used Gunicorn as the wsgi backend for the Django stuff from Nginx. This is because Gunicorn gets installed as a dependency when you pull in Mailman. When I did it this way I didn't need to install Python3-Dev, I only needed to install the Python3-Venv package. However my old setup using Gunicorn didn't use Mailman-Web, and when I migrated to using Mailman-Web as part of a Ubuntu 22.04 setup I had trouble getting Gunicorn to pick up the Django backend: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… I discussed this here: https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/… Apparently the installation doc is now updated with a Gunicorn example so when I go Ubuntu 24.04 I will try that method. I have configured a system in this way in the past using Xapian and pulled in the Ubuntu Xapian packages. My new system uses Woosh as I migrated to Mailman-Web and that installed Woosh as a dependency, and I had forgotten to back up the Xapian indexes. Andrew.

2 years, 5 months

[MM3-users]Re: mailman 2 vs. mailman 3: what happened to multipart/digest?

by Simon Hanna

On 05/12/2016 09:26 PM, Barry Warsaw wrote: > On May 12, 2016, at 12:18 PM, Mark Sapiro wrote: > >> It is the intent that Postorius will eventually expose all these >> settings. We're not there yet. At present you could possibly do this by >> interacting with mailmanclient, although I'm not certain the settings are >> exposed there. > > I'm not sure if they're exposed in mailmanclient, but they are exposed in the > core's REST API, on the member resource. It's the `delivery_mode` attribute > and it takes as legal values one of 'regular', 'plaintext_digests', > 'mime_digests', and 'summary_digests' (IOW, the enum item names for the > DeliveryMode enum). They are exposed in mailmanclient and are used in postorius. They can be found in the users personal subscription settings https://lists.mailman3.org/mailman3/accounts/mailmansettings/ > > Cheers, > -Barry > _______________________________________________ > Mailman-users mailing list > mailman-users(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >

10 years, 5 months

Re: Starting new thread in Postorius -> disappears

by Thor Atle Rustad

> > > > When I send a message from the Postorius interface, it is neither sent > out > > or stored. I briefly see this message: "Your reply has been sent and is > > being processed.", and then nothing happens. > > When you say nothing happens, you just mean that you don't see anything in > the > UI or you don't see anything in the logs? > In the UI, I get a green line with text saying the message has been sent. However, the new message is not added to the the thread in any way. In the logs there is nothing that pertains to that particular action > > Logs for mailman-web are in /opt/mailman/web/logs/mailmanweb.log (on the > host). > The log mailmanweb.log is empty. uwsgi.log is not empty. Interestingly, in the mailman-web container, there is a mailmanweb.log in the root of the server which gets updated like this: INFO 2017-11-21 10:53:33,765 36 hyperkitty.views.mailman Archived message <20171121115328.2dc8970f@thor-E6530> to https://mailer.domain.com/ hyperkitty/list/aaa(a)klubb.domain.com/message/2XYZL6G35IJYOW5WG6IGXI424Q44DG FK/ but nothing is written when I try to send message from Postorius. > > > > > I have installed maxking's docker-mailman and copied my old setup with > > basic stuff like passwords and such. The only new thing is that I'm now > > running in userspace by putting in /etc/subuid and subguid: > > > > mailman:10000:65536 > > This ideally shouldn't affect anything, I haven't tried running this way > though. > > > > > > > and in /etc/docker/daemon.json: > > > > { > > "userns-remap": "mailman" > > } > > > > So I get the following ownership in /opt/mailman: > > > > > > drwxrwsr-x 3 10100 10000 4096 nov. 21 09:00 core > > drwx------ 19 10070 10000 4096 nov. 21 09:27 database > > drwxr-sr-x 5 10100 10101 4096 nov. 21 09:24 web > > > > (I have created a user named mailman with uid 10100 so, in reality, it > says > > "mailman" there instead of 10100 but I guess that is irrelevant here) > > If would be interesting to see what processes are running inside each > containers > and who their owners are, in case this is at all relevant to the issue, > which I > am not so sure about. > > Web container: bash-4.3# ps aux PID USER TIME COMMAND 1 mailman 0:00 uwsgi --ini /opt/mailman-web/uwsgi.ini 36 mailman 0:00 uwsgi --ini /opt/mailman-web/uwsgi.ini 37 mailman 0:00 /bin/sh -c ./manage.py qcluster 41 mailman 0:00 python ./manage.py qcluster 54 mailman 0:00 python ./manage.py qcluster 55 mailman 0:00 python ./manage.py qcluster 56 mailman 0:00 python ./manage.py qcluster 57 mailman 0:00 python ./manage.py qcluster bash-4.3# ls -l total 96 -rw-rw-r-- 1 mailman root 0 Mar 29 2017 __init__.py -rw-rw-r-- 1 mailman root 42308 Nov 21 10:53 mailmanweb.log -rwxrwxr-x 1 mailman root 242 Mar 29 2017 manage.py -rw-rw-r-- 1 mailman root 12171 Oct 1 22:26 settings.py -rw-r--r-- 1 root root 9643 Nov 21 09:21 settings.pyc -rw-rw-r-- 1 mailman root 1777 May 23 21:31 settings_local.py -rw-r--r-- 1 root root 1196 Nov 21 09:21 settings_local.pyc -rw-rw-r-- 1 mailman root 1314 Mar 29 2017 urls.py -rw-r--r-- 1 root root 791 Nov 21 09:21 urls.pyc -rw-rw-r-- 1 mailman root 1425 Aug 6 17:04 uwsgi.ini -rwxrwxr-x 1 mailman root 1173 Mar 29 2017 wsgi.py core container: bash-4.3# ps aux PID USER TIME COMMAND 1 mailman 0:00 {master} /usr/local/bin/python /usr/local/bin/master 26 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=archive:0:1 -C /etc/mailman.cfg 27 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=bounces:0:1 -C /etc/mailman.cfg 28 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=command:0:1 -C /etc/mailman.cfg 29 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=in:0:1 -C /etc/mailman.cfg 30 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=lmtp:0:1 -C /etc/mailman.cfg 31 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=nntp:0:1 -C /etc/mailman.cfg 32 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg 33 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=pipeline:0:1 -C /etc/mailman.cfg 34 mailman 0:01 /usr/local/bin/python /usr/local/bin/runner --runner=rest:0:1 -C /etc/mailman.cfg 35 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=retry:0:1 -C /etc/mailman.cfg 36 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=virgin:0:1 -C /etc/mailman.cfg 37 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=digest:0:1 -C /etc/mailman.cfg 2017-11-21 11:18 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>: > Hi Thor, > > On Tue, 2017-11-21 at 10:53 +0100, Thor Atle Rustad wrote: > > I have had this working before, so I guess it's a simple setup error. > > > > When I send a message from the Postorius interface, it is neither sent > out > > or stored. I briefly see this message: "Your reply has been sent and is > > being processed.", and then nothing happens. > > When you say nothing happens, you just mean that you don't see anything in > the > UI or you don't see anything in the logs? > > Logs for mailman-web are in /opt/mailman/web/logs/mailmanweb.log (on the > host). > > > > > > I have installed maxking's docker-mailman and copied my old setup with > > basic stuff like passwords and such. The only new thing is that I'm now > > running in userspace by putting in /etc/subuid and subguid: > > > > mailman:10000:65536 > > This ideally shouldn't affect anything, I haven't tried running this way > though. > > > > > > > and in /etc/docker/daemon.json: > > > > { > > "userns-remap": "mailman" > > } > > > > So I get the following ownership in /opt/mailman: > > > > > > drwxrwsr-x 3 10100 10000 4096 nov. 21 09:00 core > > drwx------ 19 10070 10000 4096 nov. 21 09:27 database > > drwxr-sr-x 5 10100 10101 4096 nov. 21 09:24 web > > > > (I have created a user named mailman with uid 10100 so, in reality, it > says > > "mailman" there instead of 10100 but I guess that is irrelevant here) > > If would be interesting to see what processes are running inside each > containers > and who their owners are, in case this is at all relevant to the issue, > which I > am not so sure about. > > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > -- > thanks, > Abhilash Raj >

8 years, 11 months

m3 database contents

by Stephen J. Turnbull

Alexander Inzinger-Zrock via Mailman-users writes: > except for [1] i was not able to find anything about what kind of data > will be stored in the database(s) of a mailman3-system. I don't think there's a formal schema in the documentation. It's an object database backed by a relational database managed by SQLAlchemy. To get the SQL schema you can get it from your RDBMS, I suppose. Regarding the semantics of the data, it's in mailman/src/mailman/models. About sizes, I have a brand-new, nearly empty dedicated Mailman PostgreSQL installation that takes up 69MB for the whole cluster. (That's for 129 users, split across 6 lists of about 25 subscribers each, data for all three components: Postorius, HyperKitty, and core. HyperKitty usage is neglible, only 10 messages total so far.) A Mailman 3 installation with a 20 year history (recently upgraded from Mailman 2), somewhere between 500 and 1000 lists, and on the order of 100,000 users takes up 103MB as a text dump[1]. That's going to compress by a double-digit factor when loaded back into Postgres, I bet. This is just core and Postorius data.[2] In other words, the mailing list cost of Mailman is going to be insignificant compared to the overhead of a PostgreSQL installation unless you're really huge. There's a third system I don't have access to any more (another recent upgrade), and IIRC the (dedicated to Mailman) database cluster took up about 128MB on disk, with ~20k lists and ~50k users. I don't even want to think about mail throughput! Some of that database space was archives, but it's hard to say how much. 10s of megabytes at least, probably (the predecessor Mailman 2 installation was about 15 years old). The primary use case for lists was periodic automatically generated reports on system health and usage statistics, and notifications, which weren't archived. But there were hundreds of discussion and announce lists for customers and internal users, many of which are archived. The Mailman and Postorius tables in the database don't grow significantly with time. The deciding factors are the number of users and the number of lists. So the bottom line is, Mailman database size and growth is a rounding error on any modern system. Archives are another matter. But again, the database cost of the metadata is insignificant compared to message storage and (likely to be even bigger) full-text indexing. I'd estimate that the 500-list site adds about 5TB of message data a year and about the same in indexing.[2] But the rate of growth for any given site is going to be extremely case-dependent, and the part that's due to the database metadata is going to be a tiny fraction, unless your users are extremely economical with their messages. Steve [1] The PostgreSQL cluster supports several other applications, it takes up gigabytes, almost none of it Mailman. [2] It doesn't use HyperKitty for archiving so I don't know how much it would add in database metadata.

1 year, 9 months

Re: cc'd e-mail addresses rewritten with @invalid.domain

by Mark Sapiro

On 9/23/20 12:48 PM, pwilliam(a)fredhutch.org wrote: > Hi Mailman Users, > > I've done some more testing with this issue and have determined that this seems to be an issue with how commas within the display name in cc'd messages are being processed with messages sent from the Outlook client. > > Here are some test scenarios. Note I set the mailing list I was testing with to hold messages in moderation. Once the message was held I copied the information from the headers directly from Postorius held message information into the space below: > > Test Scenario 1: > Display name in Outlook: Williamss, Peterr C > Outcome: Does not work as intended. The CC address is formatted as Williamss(a)invalid.domain in messages received by mailing list members, oddly the "Peterr C" part of the display name is not present anywhere in the CC addresses received by mailing list members. > Header information from held message in Postorius > CC: "os-x-lion(a)lists.test.org" <os-x-lion(a)lists.test.org>, > "Williamss, > Peterr C" <peterbw(a)test.com> "Williamss, Peterr C" <peterbw(a)test.com> is correct for how this address spec should be represented. If in fact, it is split into two lines "Williamss, Peterr C" <peterbw(a)test.com> as it appears above, it looks to me as if the actual header should be > CC: "os-x-lion(a)lists.test.org" <os-x-lion(a)lists.test.org>, "Williamss, Peterr C" <peterbw(a)test.com> but is being folded on the whitespace between Williamss and Peterr. This is <https://gitlab.com/mailman/mailman/-/issues/725>, fixed at <https://gitlab.com/mailman/mailman/-/merge_requests/652>. The fix will be in the next (3.3.2) release of Mailman core. This is really an Outlook issue. See <https://gitlab.com/mailman/mailman/-/issues/725#note_351314868> for why folding within a quoted string is allowed, but in cases like this SHOULD not be done. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 1 month

Re: Show number of subscribers

by Mark Sapiro

On 6/28/19 3:19 AM, andrew.bernard(a)gmail.com wrote: > Would there be a way to show the number of subscribers to a list in the web UI? > > The number of participants per message is great, but this would be good too. My users seem very interested in this number. I guess they want to know how big the audience is that is reading their posts. I don't know how difficult it would be to add this to HyperKitty, and I'm not sure how useful it would be in general. HyperKitty is an archiver. It deals with archiving posts. It knows who has sent a post, but doesn't know or care if that person is, was or will be a member of the list. Postorius knows who the list members are and this information should be available to list members or the world depending on the list's member_roster_visibility setting, but see <https://gitlab.com/mailman/postorius/issues/369>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

7 years, 4 months

Re: Another interesting error...

by Darren Smith

Well, using the mailman shell we were able to find the request that was causing the problem - it was apparently a spam message that had some chinese characters in it. We used the shell to delete the held request... And postorius is still giving me an error 500. If I'm not mistaken the held messages are kept in .pck files - do you know if when postorius asks for held messages through the API, does mailman go to the database or to the .pck? I would actually suspect the latter, or removing the request should have fixed the problem. I'm hoping this makes sense... On Mon, Apr 23, 2018 at 3:39 PM, Darren Smith <silas.crutherton(a)gmail.com> wrote: > Very good, I'll give that a try and let you know! > > > On Mon, Apr 23, 2018 at 3:37 PM, Simon Hanna <simon(a)hannaweb.eu> wrote: > >> On 04/23/2018 11:29 PM, Darren Smith wrote: >> > "It's in Mailman's database in the 'message' table." >> > >> > I can see the message table - is there a way to see where it is marked >> as >> > "held"? All I'm seeing in that table is id, message_id, message_id_hash, >> > and path. >> > >> > I'm wanting to be able to query for held messages for a given mailing >> list, >> > as there are currently over 98,000 entries in the message table. >> This should be possible using the mailman shell >> http://mailman.readthedocs.io/en/latest/src/mailman/model/do >> cs/requests.html?highlight=held >> >> I haven't used the shell extensively and didn't read the whole docs, but >> it looks like it's the right place to look. If you can't do that using >> the shell, I'd say it's worth a bug report :-) >> _______________________________________________ >> Mailman-users mailing list >> mailman-users(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > >

8 years, 6 months

Re: Mm3 - new adventure

by Mark Sapiro

On 03/14/2017 09:14 AM, SP2L wrote: > > Lately started adventurous journey with Mailman3 > which is way, way different already from the beginning. > > Used Mailman bundler to install GNU Mailman, Postorius and HyperKitty > testing instance in virtual environment, mainly relaying on belows recipe: > > https://gitlab.com/mailman/mailman-bundler/ mailman-bundler is out of date and deprecated. You might find <https://wiki.list.org/x/17891998> of interest. > Present status. > > Above trio and MySQLServer are "talking" to each other > but because there isproper database on MySQLServer > I am facing strange and unexpected issues, Hi! > > Executing last two commands: > > (venv)$ ./bin/mailman start > (venv)$ ./bin/mailman-web-django-admin runserver 0.0.0.0:8000 & > > I see Postorius and HyperKitty up & runningbut... > > Belowvery condensed messages from the console: > > > /HyperKitty/ > > DEBUG = False > > [12/Mar/2017 18:59:43] "GET /archives/ HTTP/1.1" 200 8002 > WARNING Not Found: > /static/hyperkitty/libs/jquery/smoothness/jquery-ui-1.10.3.custom.min.css ... Have you run the script that bundler installs as bin/mailman-post-update? > /mailman3/ > > DEBUG = False > > Server Error (500) > > > DEBUG = True > > HTTP Error 500: A server error occurred. Please contact the administrator. > > Exception Location: > /var/lib/mailman3/mailman-bundler/eggs/mailmanclient-1.0.1-py2.7.egg/mailmanclient/_client.py > in call, line 109 > > Then VERY, verylong list... Without knowing what the exception was and at least the start of the traceback, it's hard to say what this might be. > By means of PhpMyAdmin I created manually mailman3 database > and few tables (looking at 0001_initial.py) , but it is not working O.K. > This handmade table is not doing it's job. You shouldn't need to do this. If you have the proper settings and credentials in the DATABASES setting in mailman-web/settings_local.py and the [database] section in deployment/mailman.cfg, the tables should be created for you. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

9 years, 7 months

Re: Postorius not HTML-decoding < to "<" characters

by Mark Sapiro

On 11/9/24 17:09, Mark Sapiro wrote: > > Here's what's going on. In normal text, mistune disables HTML tags by > converting `<` to the html entity `<`, but your browser renders the > HTML entity `<` as `<` so you see what was written. > > The issue here is the text you quoted was originally in a markdown code > block, i.e. preceded and followed by lines of three backticks. Code > blocks are intended to be seen literally as written so they are wrapped > in <pre><code> ... </code></pre> tags in the HTML and also HTML escaped. > The problem is the tag disabling converts `<` to `<` but then the > HTML escaping converts `<` to `&lt;` which your browser then > renders as `<` rather than `<`. > > (aside, it will be interesting to see how that is rendered in the archive) The backtick quoting causes the < to be rendered as `<` Here are the above two paragraphs written in a way (removing the backticks from around <) that I hope will render as I intended. Here's what's going on. In normal text, mistune disables HTML tags by converting < to the html entity `<`, but your browser renders the HTML entity `<` as < so you see what was written. The issue here is the text you quoted was originally in a markdown code block, i.e. preceded and followed by lines of three backticks. Code blocks are intended to be seen literally as written so they are wrapped in <pre><code> ... </code></pre> tags in the HTML and also HTML escaped. The problem is the tag disabling converts < to `<` but then the HTML escaping converts `<` to `&lt;` which your browser then renders as `<` rather than <. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years

Mailman-users search results for query "Postorius"