Search results for "Postorius" - Mailman-users

Re: Emergency moderation and clearing out a lot of held messages

by Abhilash Raj

On 12/14/22 03:31, Mark Sapiro wrote: > On 12/13/22 10:38, Dan Caballero wrote: >> Hi Mark, >> >> I ran the commands and it took about 35 seconds for the loop to run. >> >> Here's the result. >> >>>>> count >> 4168 > > > So that's at least part of the issue. How does 35 seconds compare to the > length of time to process one moderated message through Postorius? > Probably should've read more of the thread before previous reply ;-) So, the issue _could_ be due to latency to the remote database vs local database that we have on m.p.o. Mailman3 is not very efficient when it comes to total no. of database queries done per operation, which is something I have been (very slowly) tracking and fixing on a case-by-case basis. So, for example, handling 1 held message in m.p.o made 1.1k postgres database calls in total and even though they were each only few hundred micro-seconds each, the total added up to roughly ~2sec avg response time on the handle message API endpoint. If you handle more than 1, it scales linearly. My suspicion is that in your case Dan, due to a remote instance, the latency per call is higher (maybe in the order of few or 10s of milliseconds is my guess?) and then depending on the total no. of entries you have in pendings and pendedkeyvalue tables (with some filters, not _all_ entries), it adds up to a high value. The no. of database calls here is of order n^2, where n is the entries in pendings of type "held message" and "data" and their respective linked relationships in pendedkeyvalue table. My MR makes it such that we don't need to scan all entries of "type"="data", so that part will become constant time. And the pending of type "held message" will be limited to no. of held messages in a single MailingList(compared to all mailing lists like today), so it will help depending on the distribution of held messages in various lists. I am also exploring ways to make that 2nd query also constant time. -- thanks, Abhilash Raj (maxking)

2 years, 10 months

Re: MM3 Postfix FROM header issue

by eboltz＠lhtservices.com

Mark, Thanks for the assistance thus far. When I create a new Domain within Postorius. If I set the Mail Host to 'example.com' and the Alias Domain to 'lists.example.com', create a new list such as 'staff_test', it'll default to the 'example.com' domain (which is the real domain). Then when I subscribe, I'll get an email from 'staff_test(a)example.com', when I want it to be from 'staff_test(a)lists.example.com' instead, also if I reply back to the 'staff_test(a)example.com' then I"ll get a bounce-back that it doesn't exist, but if I send to 'staff_test(a)lists.example.com' it'll go to the newly created list. I'd like all defaults to be '@lists.example.com', if that makes sense. Here are the settings of my postfix_lmtp and postfix_vmap # AUTOMATICALLY GENERATED BY MAILMAN ON 2023-12-17 20:05:34 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Aliases which are visible only in the @lists.example.com domain. staff_test(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-bounces(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-confirm(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-join(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-leave(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-owner(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-request(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-subscribe(a)lists.example.com lmtp:[127.0.0.1]:8024 staff_test-unsubscribe(a)lists.example.com lmtp:[127.0.0.1]:8024 # AUTOMATICALLY GENERATED BY MAILMAN ON 2023-12-17 20:05:34 # # This file is generated by Mailman, and is kept in sync with the binary hash # file. YOU SHOULD NOT MANUALLY EDIT THIS FILE unless you know what you're # doing, and can keep the two files properly in sync. If you screw it up, # you're on your own. # Virtual mappings for the @example.com domain. staff_test(a)example.com staff_test(a)lists.example.com staff_test-bounces(a)example.com staff_test-bounces(a)lists.example.com staff_test-confirm(a)example.com staff_test-confirm(a)lists.example.com staff_test-join(a)example.com staff_test-join(a)lists.example.com staff_test-leave(a)example.com staff_test-leave(a)lists.example.com staff_test-owner(a)example.com staff_test-owner(a)lists.example.com staff_test-request(a)example.com staff_test-request(a)lists.example.com staff_test-subscribe(a)example.com staff_test-subscribe(a)lists.example.com staff_test-unsubscribe(a)example.com staff_test-unsubscribe(a)lists.example.com

1 year, 9 months

Re: Mailman3 in dockers on EC2 - running out of memory and puking ev.ry.where.

by Abhilash Raj

On Tue, Apr 9, 2019, at 2:34 PM, roger.hislop(a)is.co.za wrote: > Hi all -- really need some help. I'm tech literate, but definitely not > a sysadmin or dev. > > I'm following the tutorial at "Olay's Farmland" at > https://www.olay.xyz/2018/01/01/deploy-mailman-3-on-aws-using-docker/ > which seems to be a clone of the same content here > https://xiaoxing.us/2018/01/01/deploy-mailman-3-on-aws-using-docker/ The right place to get help regarding these guides would be the author of the posts, which AFAIK, have no affiliation with Mailman Core Team. On a (very) quick glance, it seems fine. > Easy enough to get most of the way ... set up EC2 instance, configure > SES and add DNS settings with DKIM etc. Set up Postfix, test it. All > working groovy. > > I run through the how to all the way through setting up the containers > and firing it up. This is where things start to get funky. > > Once the docker-compose is run, I run "curl > http://172.19.199.3:8000/postorius/lists/" to test. First time it said > it couldn't connect to database. So I nuked and restarted from scratch. > Second time it said the same thing... was reading around the place and > tried again, and then reran that curl and it worked -- gave me html > from a page. But the VM started running slowly --taking forever to > follow keypresses. > > Eventually became unresponsive. So I rebooted the machine via AWS > console, and docker ps showed only the postgres db running. > docker-compose ps shows > Name Command State > Ports > ------------------------------------------------------------------------------------------ > docker-mailman_database_1 docker-entrypoint.sh postgres Up > 5432/tcp > mailman-core docker-entrypoint.sh maste ... Exit 255 > 8001/tcp, 8024/tcp > mailman-web docker-entrypoint.sh uwsgi ... Exit 255 > 8000/tcp, 8080/tcp > > So rebuilt the container -- same thing. It looks and feels like its > running out of memory. > > I used the compose file from > https://gist.githubusercontent.com/Yexiaoxing/833bfcc5d3e4e0c06a8b7f0bac7c4… (with appropriate edits). How much memory do yo have in your VM? Have you tried using a bigger VM? thanks Abhilash

6 years, 6 months

Re: About to Install MM3 -- Seeking clarification

by Onyeibo Oku

On Wed, 12 Oct 2022 17:52:47 -0700 Mark Sapiro <mark(a)msapiro.net> wrote: Thanks Eggert and Sapiro. > > As Eggert suggests, see > https://docs.mailman3.org/en/latest/install/virtualenv.html and > follow that. I already started out on the above route. Its an adaptation because I am running Fedora. I had no problem with system-wide installations via dnf package management. However, some parts of the documentation does not add up. > You don't need to download the source from GitLab. > Following that doc will get the latest releases from PyPI via pip. > Those are still somewhat out of date - we really should make new > releases - so if you want the latest from GitLab, there are a couple > of ways to do it. Perhaps the easiest is to just run the various `pip > install` commands as documented and then upgrade Mailman core in your > venv with ``` > pip install --upgrade git+https://gitlab.com/mailman/mailman@master > ``` > and update the various mailman web packages with > ``` > pip install --upgrade git+https://gitlab.com/mailman/xxx@master > ``` > where xxx is mailman-web, mailman-hyperkitty, django-mailman3, > postorius, hyperkitty. I.e., run it 5 times, once for each value of > xxx. > I will come to this eventually (assuming that does not break things later). I aim to get it all working with regular pip packages first. I have installed mailman core already. Fedora expects mailman files to reside at /usr/lib/mailman. It appears to ship with a mailman unix user. I adapted to those system defaults. Now I observe that "su - mailman" gives me the proper prompt but .bashrc doesn't kick in. I still have to manually change to the virtual environment despite setting the automatic change in .bashrc. However, "su mailman" runs .bashrc but the prompt says "bash X.XX $". Is this normal? Subsequent parts of the doc show the path "/opt/mailman/mm/var/*" which should correspond to "/usr/lib/mailman/mm/var/*" on my instance. However, I don't see any instructions on how to get the "/mm" folder and the associated sub-folders. I just have "/usr/lib/mailman" I am lost "pip list" within the venv environment confirms that I have "mailman 3.3.5". What am I missing? Regards Onyeibo

3 years

hosted signup is producing unexpected results

by tokyoprogressive＠mac.com

In my test of the two forms of signup, I am getting these unexpected results. Is it my settings or is something weird within the program itself? 1) https://list.tokyoprogressive.org/postorius/lists/discuss.list.tokyoprogres… 2) Choose "You can also subscribe without creating an account. If you wish to do so, please use the form below" 3) Then I have to approve it. They get "Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org 4) But I have to approve it.I assume this is because they do not yet have a real account. And they should be listed as a NON MEMBER? 5) But even when they sign up here: https://list.tokyoprogressive.org/accounts/signup/?next=%2Fpostorius%2Flist… they get an email from my provider (NOT MY LIST????) as follows: "Hello from mailman3.emwd.com! You're receiving this e-mail because user iCloud has given yours as an e-mail address to connect their account. To confirm this is correct, go to https://list.tokyoprogressive.org/accounts/confirm-email/MTE3:1iA44k:E7w7xh… Thank you from mailman3.emwd.com! mailman3.emwd.com" 6) And this is where it gets weird. If they send from their email program and get the welcome (Welcome to the "Discuss" mailing list! To post to this list, send your email to: discuss(a)list.tokyoprogressive.org You can unsubscribe or make adjustments to your options via email by sending a message to....... I then get a notice: "Your mail to 'discuss(a)list.tokyoprogressive.org' with the subject test notice Is being held until the list moderator can review it for approval. The message is being held because: The message is not from a list member" 7) While if they send FROM THE interface it goes through. 8) What I am confused about. Signing up for an account should make them a member, right? Subscribing should make them a non member, no? But here it seems that both make them a non member. And worse, someone with an account sending from the Interface has their message get through while the same person sending from an email program has me get a notice that a non member is trying to post. Before I can move my actual members over from another mailing list provider, I want to make sure what works and what doesn't. Thanks. paul!!!

6 years

Re: [MM3-users]Re: [Mailman-Users] Mailman3 administration

by Barry Warsaw

On Jun 20, 2016, at 10:47 PM, Bill wrote: >And it leads me to think about timelines. > >I'm aware that the freeze for Debian 9 (Stretch) is scheduled for the end of >this year, and that we can expect it to be released sometime in >mid-2017. (Don't quote me on that though ;) And I'm not sure about Centos, I >didn't see a Mailman 3.0 package there or in Fedora. We have a Debian maintainer who did some work on packaging, but got stuck on a few issues. I think one of the biggest problems is also something that HK/Postorius is grappling with right now - the eminent demise of persona.org. Work is being done to switch over to django-social-auth, and that's a blocker for a full 3.1 suite release. For the core, the last big missing thing is the unsubscription workflow (see GL#213). I'll be looking at that, trying to adapt or finish Abhilash's work there. >But is there any chance that a Mailman 3.0 or 3.1 package is going to make it >into current release cycles? I'd like for it to happen; we'll have to see. >But we're in the evaluation/design phase of this project and from the >providers point of view it would be helpful to know if they should wait a few >months for Mailman 3.0/3.1 or just go ahead now with 2.1.x and upgrade to 3.1 >at some point down the road. While their list admins are experienced, they >are new to Mailman and the fewer burps the better. Although they may very >well take a run at 3.0 now and see how things work out. We just need some >sort of timeline to consider. I still think we can get a 3.1 release out by year's end, if not sooner. As I say, we have just a couple of blockers left, though they may or may not be small. I'd say, don't try to deploy 3.0, but either go with the git heads, or wait for hopefully some beta releases of 3.1 coming soon. I am seriously considering not releasing a 3.0.4 and just EOL'ing 3.0.x once 3.1 is released. It should be a smooth upgrade if you already have 3.0 deployed. Cheers, -Barry

9 years, 2 months

Re: UnicodeEncodeError: 'ascii' codec can't encode character

by Marvin Gülker

Hi, Am 10. Dezember 2019 um 17:09 Uhr +0900 schrieb Stephen J. Turnbull: > You're the "it's 2019, let's use SMTP UTF8 everywhere" guy, right? No, I'm not that guy. I've not said anything like that, nor have I knowingly configured Postfix to use that. I've looked up how to do it now that you mention it, and found the smtputf8_enable parameter[1]. I have not used it, but it turns out that it is enabled by default. I can turn it off and see if that makes any difference in the future. Testing with telnet, Postfix indeed announced "250 SMTPUTF8" on EHLO. > If you're right about the umlaut being the trigger, the first thing I'd look at is a feature-negotiation problem in the MTA delivering to Mailman. I'm pretty sure our LMTP does not offer the SMTP UTF8 feature. In that case the problem should go away with disabling SMTPUTF8, I suppose? > I would guess this is in the shunt queue. If there's only one file there, you can just delete it. There was a whole bunch of files in that directory. I've taken a look several of them, and as they were all spam, I've taken the liberty to delete them all (this list is so low volume that I know all the senders personally anyway). However, even after restarting mailman, the 500 error persists when visiting the held messages page in postorius. I've also looked into all the other queue directories, they were all empty. > Assuming you're right about your name being the trigger, U+FFFD REPLACEMENT CHARACTER is not in your name. I confess that I haven't visited the held messages page for a very long time (because I didn't have a reason to; I received no notifications of held messages). Only after I received the notification that my errorneous message was being held, I visited it, and I was faced with the 500 error. If Mailman failed to notify me of held messages previously, they might have been in hold status already and one of them might be the cause. Marvin [1]: http://www.postfix.org/postconf.5.html#smtputf8_enable PS: Sorry for the missing link [1] from the last mail. Here it is: https://gitlab.com/mailman/hyperkitty/issues/201 -- Blog: https://mg.guelker.eu

5 years, 10 months

Re: About to Install MM3 -- Seeking clarification

by Mark Sapiro

On 10/12/22 01:24, Onyeibo Oku wrote: > > Good Morning Everyone, > > I installed and managed mailman 2 for about two years. I am now > ready to try Mailman 3. The documentation for version 2 was lucid and > this forum was quite helpful in the past. > > Now I'd like to proceed with installating MM3 but I doubt my > understanding of the docs. It seems to assume that users will know what > to do. > > Okay, don't laugh! I just want to be sure what I am doing. Does the > following sequence resemble the proper steps? As Eggert says, we aren't laughing ... > 1. Download the source from a git repo (I understood that one) > 2. Create a virtual environment (venv perhaps ... not stated explicitly) > 3. activate the virtual environment > 4. from the git project root, run: ./mailman start Step 4 is premature. More below. > I know, its stupid. I had to ask. Not stupid ... > It would really build my confidence > someone would confirm the above steps or counter then with something > very explicit. What about process ownership? Do I need to have a unix > mailman user? What else do I need to get mailman up and running > before proceeding to configuration. Ahaa! I perused the config docs a > bit. Where can I find the .cfg file? As Eggert suggests, see https://docs.mailman3.org/en/latest/install/virtualenv.html and follow that. You don't need to download the source from GitLab. Following that doc will get the latest releases from PyPI via pip. Those are still somewhat out of date - we really should make new releases - so if you want the latest from GitLab, there are a couple of ways to do it. Perhaps the easiest is to just run the various `pip install` commands as documented and then upgrade Mailman core in your venv with ``` pip install --upgrade git+https://gitlab.com/mailman/mailman@master ``` and update the various mailman web packages with ``` pip install --upgrade git+https://gitlab.com/mailman/xxx@master ``` where xxx is mailman-web, mailman-hyperkitty, django-mailman3, postorius, hyperkitty. I.e., run it 5 times, once for each value of xxx. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

3 years

Re: Trying to save a new style

by pcjkollmorgen＠hotmail.com

Thanks. I was able to create a python module and install it which sets up 2 new styles, and I have one of those now set as the default list style in Postorius. I needed to include a few more interfaces but otherwise kept to the example plugin file (just made another file for the other list). ``` from public import public from mailman.interfaces.styles import IStyle from mailman.styles.default import LegacyDefaultStyle from zope.interface import implementer from mailman.interfaces.action import Action, FilterAction from mailman.interfaces.archiver import ArchivePolicy from mailman.interfaces.autorespond import ResponseAction from mailman.interfaces.bounce import UnrecognizedBounceDisposition from mailman.interfaces.digests import DigestFrequency from mailman.interfaces.mailinglist import ( DMARCMitigateAction, Personalization, ReplyToMunging, SubscriptionPolicy, ArchiveRenderingMode) from mailman.interfaces.nntp import NewsgroupModeration from mailman.model.roster import RosterVisibility # Documentation on List styles # https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/styles/doc… # also some was worked out from src/mailman/styles/base.py @public @implementer(IStyle) class MyPrivateStyle(LegacyDefaultStyle): # Provide a unique name to this style so it doesn't clash with the ones # defined by default. name = 'my-announce-style' # Provide a usable description that will be shown to the users in Web # Interface. description = 'My Announce mailing list style.' def apply(self, mailing_list): """See `IStyle`.""" # Set settings from super class. super().apply(mailing_list) # Make modifications on top. mailing_list.admin_immed_notify = True mailing_list.admin_notify_mchanges = True mailing_list.advertised = True mailing_list.allow_list_posts = False mailing_list.anonymous_list = False mailing_list.archive_policy = ArchivePolicy.private mailing_list.archive_rendering_mode = ArchiveRenderingMode.markdown mailing_list.collapse_alternatives = False mailing_list.convert_html_to_plaintext = False mailing_list.default_member_action = Action.defer mailing_list.default_nonmember_action = Action.hold mailing_list.digest_send_periodic = False mailing_list.digests_enabled = False mailing_list.include_rfc2369_headers = True mailing_list.max_message_size = "40000" mailing_list.max_num_recipients = "30" mailing_list.member_roster_visibility = RosterVisibility.moderators mailing_list.send_goodbye_message = True mailing_list.send_welcome_message = True mailing_list.subscription_policy = SubscriptionPolicy.moderate mailing_list.unsubscription_policy = SubscriptionPolicy.moderate ``` Having a docker-mailman setup means needing to install this any time that the mailmail-core container is rebuilt (not all that often), however the source code for the module can live in a host volume such as /opt/mailman/core/var/plugins , it is simply an execution of the install which is need after a container is rebuilt. eg: docker exec mailman-core pip install -E /opt/mailman/var/plugins/yourplugin I currently need to pip install django-allauth[saml] and apk add openldap-dev (this one should be in a new version of the container) on any new mailman-core container anyway so I'm not too worried.

1 year, 4 months

Re: LookupError: unknown encoding: _iso-2022-jp$esc

by Mark Sapiro

On 09/07/2018 08:30 AM, Thor Atle Rustad wrote: > This happens when I open a page of Held Messages containing an offending > message. I get Server Error (500) in Postorius. Here is the mailman.log: > > > Sep 07 15:25:39 2018 (32) REST request handler error: > Traceback (most recent call last): > File "/usr/local/lib/python3.6/wsgiref/handlers.py", line 137, in run > self.result = application(self.environ, self.start_response) > File > "/usr/local/lib/python3.6/site-packages/mailman/database/transaction.py", > line 50, in wrapper > rtn = function(*args, **kws) > File "/usr/local/lib/python3.6/site-packages/mailman/rest/wsgiapp.py", > line 214, in __call__ > return super().__call__(environ, start_response) > File "/usr/local/lib/python3.6/site-packages/falcon/api.py", line 244, in > __call__ > responder(req, resp, **params) > File > "/usr/local/lib/python3.6/site-packages/mailman/rest/post_moderation.py", > line 160, in on_get > resource = self._make_collection(request) > File "/usr/local/lib/python3.6/site-packages/mailman/rest/helpers.py", > line 159, in _make_collection > for resource in collection] > File "/usr/local/lib/python3.6/site-packages/mailman/rest/helpers.py", > line 159, in <listcomp> > for resource in collection] > File > "/usr/local/lib/python3.6/site-packages/mailman/rest/post_moderation.py", > line 150, in _resource_as_dict > resource = self._make_resource(request.id) > File > "/usr/local/lib/python3.6/site-packages/mailman/rest/post_moderation.py", > line 77, in _make_resource > resource['msg'] = msg.as_string() > File "/usr/local/lib/python3.6/site-packages/mailman/email/message.py", > line 54, in as_string > value = email.message.Message.as_string(self) > File "/usr/local/lib/python3.6/email/message.py", line 158, in as_string > g.flatten(self, unixfrom=unixfrom) > File "/usr/local/lib/python3.6/email/generator.py", line 116, in flatten > self._write(msg) > File "/usr/local/lib/python3.6/email/generator.py", line 181, in _write > self._dispatch(msg) > File "/usr/local/lib/python3.6/email/generator.py", line 214, in _dispatch > meth(msg) > File "/usr/local/lib/python3.6/email/generator.py", line 243, in > _handle_text > msg.set_payload(payload, charset) > File "/usr/local/lib/python3.6/email/message.py", line 315, in set_payload > payload = payload.encode(charset.output_charset) > LookupError: unknown encoding: _iso-2022-jp$esc I think this should be fixed in Mailman core 3.2.0. It is related to Mailman core issues 256, 437 and 470 and merge requests 197, 322, 350 and 375. In particular, it looks like you are missing at least <https://gitlab.com/mailman/mailman/merge_requests/375>. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

7 years, 1 month

Mailman-users search results for query "Postorius"