Search results for "Postorius" - Mailman-users

m3 database contents

by Stephen J. Turnbull

Alexander Inzinger-Zrock via Mailman-users writes: > except for [1] i was not able to find anything about what kind of data > will be stored in the database(s) of a mailman3-system. I don't think there's a formal schema in the documentation. It's an object database backed by a relational database managed by SQLAlchemy. To get the SQL schema you can get it from your RDBMS, I suppose. Regarding the semantics of the data, it's in mailman/src/mailman/models. About sizes, I have a brand-new, nearly empty dedicated Mailman PostgreSQL installation that takes up 69MB for the whole cluster. (That's for 129 users, split across 6 lists of about 25 subscribers each, data for all three components: Postorius, HyperKitty, and core. HyperKitty usage is neglible, only 10 messages total so far.) A Mailman 3 installation with a 20 year history (recently upgraded from Mailman 2), somewhere between 500 and 1000 lists, and on the order of 100,000 users takes up 103MB as a text dump[1]. That's going to compress by a double-digit factor when loaded back into Postgres, I bet. This is just core and Postorius data.[2] In other words, the mailing list cost of Mailman is going to be insignificant compared to the overhead of a PostgreSQL installation unless you're really huge. There's a third system I don't have access to any more (another recent upgrade), and IIRC the (dedicated to Mailman) database cluster took up about 128MB on disk, with ~20k lists and ~50k users. I don't even want to think about mail throughput! Some of that database space was archives, but it's hard to say how much. 10s of megabytes at least, probably (the predecessor Mailman 2 installation was about 15 years old). The primary use case for lists was periodic automatically generated reports on system health and usage statistics, and notifications, which weren't archived. But there were hundreds of discussion and announce lists for customers and internal users, many of which are archived. The Mailman and Postorius tables in the database don't grow significantly with time. The deciding factors are the number of users and the number of lists. So the bottom line is, Mailman database size and growth is a rounding error on any modern system. Archives are another matter. But again, the database cost of the metadata is insignificant compared to message storage and (likely to be even bigger) full-text indexing. I'd estimate that the 500-list site adds about 5TB of message data a year and about the same in indexing.[2] But the rate of growth for any given site is going to be extremely case-dependent, and the part that's due to the database metadata is going to be a tiny fraction, unless your users are extremely economical with their messages. Steve [1] The PostgreSQL cluster supports several other applications, it takes up gigabytes, almost none of it Mailman. [2] It doesn't use HyperKitty for archiving so I don't know how much it would add in database metadata.

1 year

Re: Another interesting error...

by Darren Smith

Well, using the mailman shell we were able to find the request that was causing the problem - it was apparently a spam message that had some chinese characters in it. We used the shell to delete the held request... And postorius is still giving me an error 500. If I'm not mistaken the held messages are kept in .pck files - do you know if when postorius asks for held messages through the API, does mailman go to the database or to the .pck? I would actually suspect the latter, or removing the request should have fixed the problem. I'm hoping this makes sense... On Mon, Apr 23, 2018 at 3:39 PM, Darren Smith <silas.crutherton(a)gmail.com> wrote: > Very good, I'll give that a try and let you know! > > > On Mon, Apr 23, 2018 at 3:37 PM, Simon Hanna <simon(a)hannaweb.eu> wrote: > >> On 04/23/2018 11:29 PM, Darren Smith wrote: >> > "It's in Mailman's database in the 'message' table." >> > >> > I can see the message table - is there a way to see where it is marked >> as >> > "held"? All I'm seeing in that table is id, message_id, message_id_hash, >> > and path. >> > >> > I'm wanting to be able to query for held messages for a given mailing >> list, >> > as there are currently over 98,000 entries in the message table. >> This should be possible using the mailman shell >> http://mailman.readthedocs.io/en/latest/src/mailman/model/do >> cs/requests.html?highlight=held >> >> I haven't used the shell extensively and didn't read the whole docs, but >> it looks like it's the right place to look. If you can't do that using >> the shell, I'd say it's worth a bug report :-) >> _______________________________________________ >> Mailman-users mailing list >> mailman-users(a)mailman3.org >> https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ >> > >

7 years, 9 months

Re: Starting new thread in Postorius -> disappears

by Thor Atle Rustad

> > > > When I send a message from the Postorius interface, it is neither sent > out > > or stored. I briefly see this message: "Your reply has been sent and is > > being processed.", and then nothing happens. > > When you say nothing happens, you just mean that you don't see anything in > the > UI or you don't see anything in the logs? > In the UI, I get a green line with text saying the message has been sent. However, the new message is not added to the the thread in any way. In the logs there is nothing that pertains to that particular action > > Logs for mailman-web are in /opt/mailman/web/logs/mailmanweb.log (on the > host). > The log mailmanweb.log is empty. uwsgi.log is not empty. Interestingly, in the mailman-web container, there is a mailmanweb.log in the root of the server which gets updated like this: INFO 2017-11-21 10:53:33,765 36 hyperkitty.views.mailman Archived message <20171121115328.2dc8970f@thor-E6530> to https://mailer.domain.com/ hyperkitty/list/aaa(a)klubb.domain.com/message/2XYZL6G35IJYOW5WG6IGXI424Q44DG FK/ but nothing is written when I try to send message from Postorius. > > > > > I have installed maxking's docker-mailman and copied my old setup with > > basic stuff like passwords and such. The only new thing is that I'm now > > running in userspace by putting in /etc/subuid and subguid: > > > > mailman:10000:65536 > > This ideally shouldn't affect anything, I haven't tried running this way > though. > > > > > > > and in /etc/docker/daemon.json: > > > > { > > "userns-remap": "mailman" > > } > > > > So I get the following ownership in /opt/mailman: > > > > > > drwxrwsr-x 3 10100 10000 4096 nov. 21 09:00 core > > drwx------ 19 10070 10000 4096 nov. 21 09:27 database > > drwxr-sr-x 5 10100 10101 4096 nov. 21 09:24 web > > > > (I have created a user named mailman with uid 10100 so, in reality, it > says > > "mailman" there instead of 10100 but I guess that is irrelevant here) > > If would be interesting to see what processes are running inside each > containers > and who their owners are, in case this is at all relevant to the issue, > which I > am not so sure about. > > Web container: bash-4.3# ps aux PID USER TIME COMMAND 1 mailman 0:00 uwsgi --ini /opt/mailman-web/uwsgi.ini 36 mailman 0:00 uwsgi --ini /opt/mailman-web/uwsgi.ini 37 mailman 0:00 /bin/sh -c ./manage.py qcluster 41 mailman 0:00 python ./manage.py qcluster 54 mailman 0:00 python ./manage.py qcluster 55 mailman 0:00 python ./manage.py qcluster 56 mailman 0:00 python ./manage.py qcluster 57 mailman 0:00 python ./manage.py qcluster bash-4.3# ls -l total 96 -rw-rw-r-- 1 mailman root 0 Mar 29 2017 __init__.py -rw-rw-r-- 1 mailman root 42308 Nov 21 10:53 mailmanweb.log -rwxrwxr-x 1 mailman root 242 Mar 29 2017 manage.py -rw-rw-r-- 1 mailman root 12171 Oct 1 22:26 settings.py -rw-r--r-- 1 root root 9643 Nov 21 09:21 settings.pyc -rw-rw-r-- 1 mailman root 1777 May 23 21:31 settings_local.py -rw-r--r-- 1 root root 1196 Nov 21 09:21 settings_local.pyc -rw-rw-r-- 1 mailman root 1314 Mar 29 2017 urls.py -rw-r--r-- 1 root root 791 Nov 21 09:21 urls.pyc -rw-rw-r-- 1 mailman root 1425 Aug 6 17:04 uwsgi.ini -rwxrwxr-x 1 mailman root 1173 Mar 29 2017 wsgi.py core container: bash-4.3# ps aux PID USER TIME COMMAND 1 mailman 0:00 {master} /usr/local/bin/python /usr/local/bin/master 26 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=archive:0:1 -C /etc/mailman.cfg 27 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=bounces:0:1 -C /etc/mailman.cfg 28 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=command:0:1 -C /etc/mailman.cfg 29 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=in:0:1 -C /etc/mailman.cfg 30 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=lmtp:0:1 -C /etc/mailman.cfg 31 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=nntp:0:1 -C /etc/mailman.cfg 32 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=out:0:1 -C /etc/mailman.cfg 33 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=pipeline:0:1 -C /etc/mailman.cfg 34 mailman 0:01 /usr/local/bin/python /usr/local/bin/runner --runner=rest:0:1 -C /etc/mailman.cfg 35 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=retry:0:1 -C /etc/mailman.cfg 36 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=virgin:0:1 -C /etc/mailman.cfg 37 mailman 0:00 /usr/local/bin/python /usr/local/bin/runner --runner=digest:0:1 -C /etc/mailman.cfg 2017-11-21 11:18 GMT+01:00 Abhilash Raj <maxking(a)asynchronous.in>: > Hi Thor, > > On Tue, 2017-11-21 at 10:53 +0100, Thor Atle Rustad wrote: > > I have had this working before, so I guess it's a simple setup error. > > > > When I send a message from the Postorius interface, it is neither sent > out > > or stored. I briefly see this message: "Your reply has been sent and is > > being processed.", and then nothing happens. > > When you say nothing happens, you just mean that you don't see anything in > the > UI or you don't see anything in the logs? > > Logs for mailman-web are in /opt/mailman/web/logs/mailmanweb.log (on the > host). > > > > > > I have installed maxking's docker-mailman and copied my old setup with > > basic stuff like passwords and such. The only new thing is that I'm now > > running in userspace by putting in /etc/subuid and subguid: > > > > mailman:10000:65536 > > This ideally shouldn't affect anything, I haven't tried running this way > though. > > > > > > > and in /etc/docker/daemon.json: > > > > { > > "userns-remap": "mailman" > > } > > > > So I get the following ownership in /opt/mailman: > > > > > > drwxrwsr-x 3 10100 10000 4096 nov. 21 09:00 core > > drwx------ 19 10070 10000 4096 nov. 21 09:27 database > > drwxr-sr-x 5 10100 10101 4096 nov. 21 09:24 web > > > > (I have created a user named mailman with uid 10100 so, in reality, it > says > > "mailman" there instead of 10100 but I guess that is irrelevant here) > > If would be interesting to see what processes are running inside each > containers > and who their owners are, in case this is at all relevant to the issue, > which I > am not so sure about. > > > _______________________________________________ > > Mailman-users mailing list > > mailman-users(a)mailman3.org > > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > > -- > thanks, > Abhilash Raj >

8 years, 2 months

Re: Mm3 - new adventure

by Mark Sapiro

On 03/14/2017 09:14 AM, SP2L wrote: > > Lately started adventurous journey with Mailman3 > which is way, way different already from the beginning. > > Used Mailman bundler to install GNU Mailman, Postorius and HyperKitty > testing instance in virtual environment, mainly relaying on belows recipe: > > https://gitlab.com/mailman/mailman-bundler/ mailman-bundler is out of date and deprecated. You might find <https://wiki.list.org/x/17891998> of interest. > Present status. > > Above trio and MySQLServer are "talking" to each other > but because there isproper database on MySQLServer > I am facing strange and unexpected issues, Hi! > > Executing last two commands: > > (venv)$ ./bin/mailman start > (venv)$ ./bin/mailman-web-django-admin runserver 0.0.0.0:8000 & > > I see Postorius and HyperKitty up & runningbut... > > Belowvery condensed messages from the console: > > > /HyperKitty/ > > DEBUG = False > > [12/Mar/2017 18:59:43] "GET /archives/ HTTP/1.1" 200 8002 > WARNING Not Found: > /static/hyperkitty/libs/jquery/smoothness/jquery-ui-1.10.3.custom.min.css ... Have you run the script that bundler installs as bin/mailman-post-update? > /mailman3/ > > DEBUG = False > > Server Error (500) > > > DEBUG = True > > HTTP Error 500: A server error occurred. Please contact the administrator. > > Exception Location: > /var/lib/mailman3/mailman-bundler/eggs/mailmanclient-1.0.1-py2.7.egg/mailmanclient/_client.py > in call, line 109 > > Then VERY, verylong list... Without knowing what the exception was and at least the start of the traceback, it's hard to say what this might be. > By means of PhpMyAdmin I created manually mailman3 database > and few tables (looking at 0001_initial.py) , but it is not working O.K. > This handmade table is not doing it's job. You shouldn't need to do this. If you have the proper settings and credentials in the DATABASES setting in mailman-web/settings_local.py and the [database] section in deployment/mailman.cfg, the tables should be created for you. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

8 years, 10 months

Re: Postorius not HTML-decoding < to "<" characters

by Mark Sapiro

On 11/9/24 17:09, Mark Sapiro wrote: > > Here's what's going on. In normal text, mistune disables HTML tags by > converting `<` to the html entity `<`, but your browser renders the > HTML entity `<` as `<` so you see what was written. > > The issue here is the text you quoted was originally in a markdown code > block, i.e. preceded and followed by lines of three backticks. Code > blocks are intended to be seen literally as written so they are wrapped > in <pre><code> ... </code></pre> tags in the HTML and also HTML escaped. > The problem is the tag disabling converts `<` to `<` but then the > HTML escaping converts `<` to `&lt;` which your browser then > renders as `<` rather than `<`. > > (aside, it will be interesting to see how that is rendered in the archive) The backtick quoting causes the < to be rendered as `<` Here are the above two paragraphs written in a way (removing the backticks from around <) that I hope will render as I intended. Here's what's going on. In normal text, mistune disables HTML tags by converting < to the html entity `<`, but your browser renders the HTML entity `<` as < so you see what was written. The issue here is the text you quoted was originally in a markdown code block, i.e. preceded and followed by lines of three backticks. Code blocks are intended to be seen literally as written so they are wrapped in <pre><code> ... </code></pre> tags in the HTML and also HTML escaped. The problem is the tag disabling converts < to `<` but then the HTML escaping converts `<` to `&lt;` which your browser then renders as `<` rather than <. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

1 year, 2 months

Re: restrict sending to owner/moderator

by Mark Sapiro

On 09/18/2017 02:28 AM, m.dohmen(a)xconnect.de wrote: > > I'd like to restrict message acceptance on a per-list basis to the owner or moderator, so that simple subscribers cannot send messages to the list. But I don't want the owner/moderator having to moderate own messages, so the member setting "hold for moderation" in Postorius isn't applicable. And on 09/19/2017 10:52 AM, David Krantz wrote: > > I have just tested a variant of this as I got curious: You find the > member you want to be able to send emails without moderation, then set > Administrative Options to "Accept Immediately (bypass other rules)". > That works. I have found no other ways, though. Or you may wish to set the owners/moderators subscriptions to "Default processing" which will still apply subsequent rule checks. I.e., set Default action to take when a member posts to the list to Reject (with notification) and set an appropriate notice (currently can't be set in Postorius - set it via 'mailman shell') and set Default action to take when a non-member posts to thelist to Discard or perhaps Reject but this creates backscatter. Then ordinary members have Moderation set to List default and owners/moderators have it set to Accept or Default. The problems with this are sall the owners/moderators need to be subscribers and there are maintenance issues with adding/removing owners/moderators. A better way to do this by sharing a "moderator password" for the list among the authorized posters. See <http://mailman.readthedocs.io/en/latest/src/mailman/rules/docs/approved.html> for how this is done to pre-approve posts. Then set the list to Discard/Reject all posts, and people who know the "moderator password" can still post. There is still an issue in that the password needs to be changed and redistributed when people become unauthorized, but this is more secure as regular moderation can be fooled by spoofing the sender. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

8 years, 4 months

Re: Default Setting for 'Results per Page' and other Web Pull-downs

by Mark Sapiro

On 5/5/23 14:25, Dave Hall via Mailman-users wrote: > > My question: when I go to any page that has a list (of lists, of members, > or archives), there are pulldowns like 'Results per page' that seem to > reset to the minimum number when I refresh the page. There are others like > 'Hide inactive' on the Archives page. What I'm looking for is a per-user > way to customize these values so I don't have to keep on setting them to > the value I prefer. > > If that is available in a later release then I have something to look > forward to. If not, it sure would be nice to have. No, it is not available. You can submit a wishlist issue at https://gitlab.com/mailman/postorius/-/issues/new for Postorius and https://gitlab.com/mailman/hyperkitty/-/issues/new for HyperKitty. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 8 months

Installing MM3 requires Django<1.11

by Henrik Rasmussen

Two questions regarding Mailman 3 installation: 1) The Mail server setup at http://docs.mailman3.org/en/latest/devsetup.html#set-up-a-mail-server isn't described yet. Instead it refers to https://docs.djangoproject.com/en/1.10/topics/email/#topic-email-backends, but it doesn't say which file I have to change. According to https://www.webforefront.com/django/setupdjangoemail.html the file is called settings.py, but the only settings.py I can find at this stage is the /opt/mailman/postorius/example_project/settings.py file. According to http://docs.mailman3.org/en/latest/devsetup.html#set-up-and-run-postorius, "Postorius and HyperKitty both come with example_project directories with basic configuration so you can try them out. For this tutorial, however, we'll be using a project that combines both instead.". In what file should I change the mail settings then? 2) While installing Hyperkitty*) I got the error message "Django 1.11rc1 is installed but Django<1.11 is required by {'django-haystack'}". As I have slavishly followed the installation guide including http://docs.mailman3.org/en/latest/devsetup.html#set-up-django-mailman3, I wonder why a non-supported Django version was installed. *) http://docs.mailman3.org/en/latest/devsetup.html#set-up-and-run-hyperkitty Regards Henrik

8 years, 9 months

Re: Adding user - Error 409 Subscription request already pending

by Mark Sapiro

On 4/20/19 8:55 AM, Kevin Hurtley wrote: > > It sounded on Github that some opinion was that it wasn't necessary to see these pending requests, but as "users will be users" I think would be useful to see what is in queue for subscription requests, it would be better yet to be able to either clear them out or accept them there. But at very least seeing them there would go a long way helping out my moderators troubleshoot. The issue with the way Postorius displayed all the subscription requests was that it wasn't possible for the moderator to distinguish request waiting moderator approval from those waiting user confirmation, and in general a moderator shouldn't approve a request waiting user confirmation, but this thread shows there is sometimes a need for a moderator to be able to deal with these. I have reopened <https://gitlab.com/mailman/postorius/issues/314>, and will look at fixing this. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 9 months

Re: integrating mm3 with postfix / lmtp

by Mark Sapiro

On 10/25/2017 05:41 PM, Fabian A. Santiago wrote: > > No to the domains part and I think virtual alias maps is needed: > > mail postfix/trivial-rewrite[19445]: warning: do not list domain xxx in BOTH virtual_mailbox_domains and relay_domains > Oct 25 20:39:19 mail postfix/trivial-rewrite[19445]: warning: do not list domain xxx in BOTH virtual_mailbox_domains and relay_domains > Oct 25 20:39:19 mail postfix/submission/smtpd[19441]: NOQUEUE: reject: RCPT from pool-108-53-239-102.nwrknj.fios.verizon.net[108.53.239.102]: 550 5.1.1 <test123@xxx>: Recipient address rejected: User unknown in virtual mailbox table; from=<fsantiago@xxx> to=<test123@xxx> proto=ESMTP helo=<[10.0.0.29]> (above slightly sanitized for return to the list) The issue here is your list domain cannot be a virtual_mailbox_domain or a virtual_alias_domain. I have the same issue on mail.python.org where we have both Mailman 2 and Mailman 3 lists @python.org. and python.org is a virtual_alias_domain. I have a group of MRs for a branch that implements "alias_domains" to deal with this. Basically it works by creating another (virtual_alias_maps) postfix_vmap mapping to map list addresses @python.org to addresses @x.python.org. Then the postfix_domains and postfix_lmtp maps have the x.python.org domain and there is a patch to mailman/runners/lmtp.py to map the x.python.org domain back to python.org. The various MRs are <https://gitlab.com/mailman/mailman/merge_requests/202> for the core, <https://gitlab.com/mailman/mailmanclient/merge_requests/18> for the REST bindings used by Postorius and <https://gitlab.com/mailman/postorius/merge_requests/186> to be able to set an alias_domain via Postorius. That's not all implemented at mail.python.org. What's there is the basic Postfix config, a bash script the generate the new mappings from the old and a simple patch to mailman/runners/lmtp.py. So, the bottom line is it can be done and it's in process, but it won't really be supported until Mailman 3.2. In the mean time, you just need to have a separate domain for your lists that isn't a virtual_*_domain. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

8 years, 2 months

Mailman-users search results for query "Postorius"