Search results for "Postorius" - Mailman-users

Re: Hyperkitty performance problem

by Tobias Diekershoff

Hey all, so far we could not find the reason behind the problem. However while reviewing the installation process one one our team mentioned that he read the installation instructions for Hyperkitty different then the rest of the team. Are Mailman3/Postorius and Hyperkitty supposed to run in separated databases, or are they supposed to use the same database? Thanks in advance for clarification! Tobias On 06.11.24 08:18, Tobias Diekershoff wrote: > Hey all, > > last week we caught some AI bots causing high LOAD on our mailman3 > servers archive, that we blocked with nginx rules. > > I'm not sure if it is related to this incident, but we noticed this > when analyzing the log files at the time. In the mailman.log > hyperkitty is listed 10k times and more to archive a single mail over > the time of 2h, seemingly all with the same process ID > > > (1186) HyperKitty archived message [...] > > The mailman3-web.log has matching log entries that the mail was archived > > > hyperkitty.views.mailman Archived message > > every 0.5 seconds. > > In the web interface to the archive, the mail is only listed once though. > > My suspicion is, that these archiving processes are now overloading > the system, preventing a timely catching up work of the queued > messages. I could not find anything in the Mailman3 nor Hyperkitty > documentation to better distribute the system resources to the Mailman > runner or prevent Hyperkitty to restart archiving a mail over and over > again. > > Any hint about where to look for the reason of this is highly > appreciated. > > Best, > Tobias > -- Tobias Diekershoff >>> System Hacker Free Software Foundation Europe Schönhauser Allee 6/7, 10119 Berlin, Germany | t +49-30-27595290 Registered at Amtsgericht Hamburg, VR 17030 | fsfe.org/support OpenPGP-Key ID ... 0x25FE376FF17694A1 Fingerprint ...... 23EE F484 FDF8 291C BA09 A406 25FE 376F F176 94A1

1 year, 5 months

Re: Changing subscription address

by Stephen J. Turnbull

Abhilash Raj writes: > > There does need to be a way to revert each option to inherit, and > > there currently doesn't seem to be one. The 'Preferences' interface I > > described in the previous post doesn't provide it either. I will > > think about that. > > The implementation of this isn’t very difficult, but the UX of this is > something I don’t know how to implement in that page to be very > honest. Oh, I can play Liz Warren on that one! :-) "I have a plan for that." > We do already have ability to switch from an explicit address to Primary > Address subscription. I would imagine it would be more than just a > button, but more like a dropdown to choose an email or primary address > and switch all subscriptions to that. Allan's situation is that for some reason he has a lot of users (or at least suspects he does) who would be happy to have only one address that there's only one place to change, but for some reason that address is copied to (not inherited by) each of several subscriptions. We should address that, because I'm pretty sure his suspicion is correct. For example, until I changed one of them -- and got two emails about it for my discard pile :-) -- all of my python.org Mailman 3 subscriptions were explicit. I still have about 8 to go. :-) > This is how Core actually does work[1] if you subscribe via Email > and you already have a primary address. It will lookup a user using > the address and if the address is the primary address of the user, > it will subscribe the primary address. If you say so, I guess it's true. Nevertheless, both Allan and I have observed users with only one address, with that address subscribed literally each time to several (~10) lists. > However, if it isn’t the primary address, and this is a new address, it will > subscribe the address instead of user because the address isn’t > verified at this point and hence can’t be a primary address. We could > extend the subscription workflow to add an additional step to set this > address as primary and subscribe the primary address. Yes, please, I think. Users who don't want this for subscriptions to the primary address are going to be (rare) edge cases. If we need an additional attribute on the user to handle the case of an unverified primary address, I think that's an acceptable cost. Note that to "claim" this User as her own, she has to verify that address. And if she *can* verify that address, we can't stop her from doing so, we can just make it more annoying. > Postorius has the full ability to define its own workflow because > REST API provides full expression capability and doesn’t enforce > anything. Previous versions didn’t had support for subscribing as > Primary Address, but that has since changed and is the default one > in the dropdown. > > This is for authenticated subscription, the anonymous subscribe works > similar to email -join command. Mass-subscribe is probably working the > same way. All of these should unify all subscriptions-by-this-address to one User, in my opinion. > Like mentioned above, it should be possible to do what you suggest. I > am getting into the implementation details here, but the Subscription > workflow would need an additional flag that would verify an address > and use the flag to determine if it should subscribe the address or > addresses’s user if an address is provided to the workflow. I don't think so. Just do the lookup, if it succeeds and matches primary address, use inherit. If it doesn't succeed, create a User and use inherit. If it does succeed, and doesn't match, use the explicit address. It seems to me that this flag would be impractical for email subscriptions (email commands with parameters are generally beyond the kind of user this is most important for) or mass subscriptions (how does the list owner know what the user wants?) anyway. If somebody really wants all their subscriptions via primary address to be copies of the primary address rather than user, I'm sorry, but they'll have to deal with that. I can't imagine that it's the usual case, and there's an easy workaround: make your primary address something other than your usual subscription address! If you disagree with me about the relative burden, then maybe we shouldn't do this at all, but I'm pretty sure this is something we should either do for all subscriptions, or for none. > Primary Address is also an attribute of the Mailman User, so it > should be possible for an admin to set that for a user if the user > doesn’t have an account. When they sign up, they’d be able to > manage and update accordingly. True, but AIUI Allan has a *lot* of users and on average they're relatively needy compared to, say, my user population. The point of his request is to smooth workflows for admins with user populations where the admin *wants* to do many things for their users, or to have smooth user workflows when they teach their users to do it for themselves. That's why I'm focusing on the "I have one address and if it changes I want to change it once" case. I think it's both extremely common, and extremely annoying :-) to admins who have to deal with the current situation. Steve

5 years, 1 month

Re: Using REST with CLI using curl .... help needed

by Michael Bakonyi

Dear Stephen, allright, thx a lot for your quick answer! After fiddling around some more time, I guess I had some misunderstandings regarding the REST-API. I guess if we reach the API correctly, we will not be confronted with the CSRF-Error. We will continue to try but nevertheless: What would be a cleaner approach to reach the API from an external IP? I didn't found any examples in the web so far. Cheers, Michael > Am 24.08.2022/ 34 um 14:27 schrieb Stephen J. Turnbull <stephenjturnbull(a)gmail.com>: > > kontakt(a)michaelbakonyi.de writes: > >> I have the same usecase as Krešimir and tried the example of >> Mark. So in mailman.cfg I set hostname to >> "https://lists.mydomain.org" and then fire this command > > I don't recall this, and unfortunately Mark is on vacation and > completely offline until mid-September, except for maybe a couple days > next week. > >> curl -urestadmin:mypw --referer https://lists.mydomain.org --request POST -H 'Content-Type: application/json' --data-binary '{"list_id": "thelist.lists.mydomain.org", "subscriber": "user(a)example.net", "display_name": "Joe User", "pre_verified": true, "pre_confirmed": true, "pre_approved": true}' https://lists.mydomain.org/mailman3/postorius/lists/thelist.lists.mydomain.… >> >> But then I get a CSRF-warning. Do I now have to get the token and >> put it in my POST or is there an easier way to do this? > > As far as *I* know there's no way to avoid the CSRF warning. If that > means you can't get the response data you need, you will have to do > the CSRF two-step. > > With luck maybe Mark will see this in a few days but I think he's > going to be busy preparing the 2d half of his trip. > > Regards, > Steve >

3 years, 8 months

Re: Implicit destination

by David Krantz

On Mon, Aug 27, 2018 at 2:50 AM Mark Sapiro <mark(a)msapiro.net> wrote: > > That is not the issue. Sending the held message notice to a spammer, if > it reaches the spammer, is probably inconsequential. The issue is that > virtually always in spam the sender address is either undeliverable or > is the address of an innocent 3rd party. Sending these notices to > innocent 3rd parties is called backscatter and is a bad thing. > > To avoid this backscatter, you can set the list's > respond_to_post_requests attribute to False. There are two problems > here. One is this setting is not currently exposed in Postorius, and the > other is this will also stop desired held message notices such as those > from list members. Aha. That was a simple solution although with a bad consequence. Will it stop moderation mails to the moderator as well? I'll try. Otherwise it will not be a problem in my case. The lists where I have this problem are more of function addresses for a number of persons (like a board or workgroup) so they accept posts from non-members. I use mailman as a simple forward gets legit mail caught in the receivers spam filters and might cause more black-list problems. You are of course entirely correct in that backscatter is the real problem, I should have thought of that. The thing in the spam I've received lately is somewhat sophisticated: the senders have correct (albeit strange) domains and correct SPF records so they try to seem legit although the senders are located in unexpected parts of the world. I just checked one example, in that case the MX pointed to the sender IP, the SPF was very sloppy (~all) and everything looked correct except for the fact that we usually do not receive mail from Ukraine and the cryptocurrency scam in the body. It almost got caught in the spam filter, but not quite. cheers // David

7 years, 8 months

Re: New releases

by Mark Sapiro

On 6/9/20 1:05 AM, Danil Smirnov wrote: > On Tue, 9 Jun 2020 at 00:05, Mark Sapiro <mark(a)msapiro.net> wrote: > >> After you upgraded did you run these django-admin commands? >> ``` >> django-admin collectstatic --clear --noinput --verbosity 0 >> django-admin compress >> django-admin compilemessages >> django-admin migrate >> ``` >> These 4 commands should be run after every update to Postorius, >> HyperKitty or django_mainlman3. >> > > Hm, interesting... Could you point me to some related docs please? The documentation is weak and should be updated, but see <https://docs.mailman3.org/en/latest/upgrade-3.2.html#migration-steps> > To upgrade, I've just bumped app versions in my Dockerfiles, built and > restarted... > Mailman-core worked right away with no issues. > Mailman-web (Hyperkitty) failed with the error: > > ---------- > ERROR 2020-06-08 19:46:41,942 14 django.request Internal Server Error: > /hyperkitty/ > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/compressor/base.py", line 282, in > precompile > mod = import_module(mod_name) > File "/usr/lib/python3.6/importlib/__init__.py", line 126, in > import_module > return _bootstrap._gcd_import(name[level:], package, level) > File "<frozen importlib._bootstrap>", line 994, in _gcd_import > File "<frozen importlib._bootstrap>", line 971, in _find_and_load > File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked > ModuleNotFoundError: No module named 'sassc -t compressed {infile} > {outfile}' ... > I've fixed the error by removing the older file which is > /opt/mailman-web-data/static/hyperkitty/libs/bootstrap/stylesheets/_bootstrap.scss > > After that everything seems to be working fine. I'm not sure about the above error. 'sassc -t compressed {infile} {outfile}' is defined in `settings` COMPRESS_PRECOMPILERS, but it is a command to be run as part of `django-admin compress`, not a module to be imported. Possibly running `django-admin compress` before starting Hyperkitty would have avoided this. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 10 months

Re: Re-send subscription response and file permission error in queue

by Mark Sapiro

On 9/11/20 2:18 PM, Mark Sapiro wrote: > On 9/8/20 10:56 PM, accounts-mailman(a)holbrook.no wrote: >> A user sent subscription request to the `..-join@` address of a list. Email domain wasn't set up properly, so response got rejected. Now, if the user tries to send another request, no reponse comes back. In the log I find a python stack trace saying subscription already pending. >> >> ``` >> Sep 08 10:32:44 2020 (19918) Traceback (most recent call last): >> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, in _one_iteration >> self._process_one_file(msg, msgdata) >> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, in _process_one_file >> keepqueued = self._dispose(mlist, msg, msgdata) >> File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 196, in _dispose >> mlist, msg, msgdata, parts, results) >> File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py", line 104, in process >> ISubscriptionManager(mlist).register(subscriber) >> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 520, in register >> list(workflow) >> File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, in __next__ >> return step() >> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 227, in _step_sanity_checks >> raise SubscriptionPendingError(self.mlist, self.address.email) >> mailman.interfaces.subscriptions.SubscriptionPendingError >> ``` > > > I'll look at this when I get some time. I'm still catching up after > being off line for 12 days. This exception should also send a response to the user's email saying "User <user(a)example.com has a pending subscription for list(a)example.net" Was this sent? To fix, >> How can I remove the pending subscription, or re-send the response email? > > In Postorius, pull down Subscription Requests and select Pending > Confirmation and delete the request. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 7 months

request to held messages yields set of email addresses

by Thomas Krichel

Hi gang, I probably have already developed an uneviable reputation of being able to produce the finest of mess. I stay true to form here. In my 3.12 installation when my admin clicks to view the held messages, he gets an internal server error. The error comes from postorius, where the log entry is 2026-04-15 12:45:25 [FALCON] [ERROR] GET /3.1/lists/bibnez(a)folks.email/held => Traceback (most recent call last): File "falcon/app.py", line 440, in falcon.app.App.__call__ File "/usr/local/mailman/venv/lib/python3.12/site-packages/mailman/rest/post_moderation.py", line 194, in on_get resource = self._make_collection(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/mailman/venv/lib/python3.12/site-packages/mailman/rest/helpers.py", line 189, in _make_collection [etag(resource) for resource in entries] ^^^^^^^^^^^^^^ File "/usr/local/mailman/venv/lib/python3.12/site-packages/mailman/rest/helpers.py", line 104, in etag return json.dumps(resource, cls=ExtendedEncoder, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/__init__.py", line 238, in dumps **kw).encode(obj) ^^^^^^^^^^^ File "/usr/lib/python3.12/json/encoder.py", line 200, in encode chunks = self.iterencode(o, _one_shot=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/encoder.py", line 258, in iterencode return _iterencode(o, 0) ^^^^^^^^^^^^^^^^^ File "/usr/local/mailman/venv/lib/python3.12/site-packages/mailman/rest/helpers.py", line 68, in default return super().default(obj) ^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/json/encoder.py", line 180, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type set is not JSON serializable Checking the actual set that is returned, by converting the set into a string and writing it to a file (by enhancing helpers.py), I discover that what is returned is as set of 53 email addresses, not of 24 messages as the web interface suggests. Intrestingly, my admin reports that the he is correctly informed about the held messages via email. The then forwards those that need posting to the list. I suspect that to get to the bottom of this problem, I need to use the low-level postorious functions that actually access the parse the held message data. -- Written by Thomas Krichel http://openlib.org/home/krichel on his 22230th day.

2 weeks, 1 day

Registration mail is not sent

by Stephen J. Turnbull

Christian H. Kuhn via Mailman-users writes: > i got mailman-web up and runnung. Is this a system package, or did you use the "Virtualenv install" (https://docs.mailman3.org/en/latest/install/virtualenv.html)? > Wanted to registrate the main user. Got an error ("Bei der > Verarbeitung Ihrer Anfrage ist ein Fehler aufgetreten."). Google says that means "An error occurred while processing your request." That's not a Postfix-like error message. Rather, that looks like a webserver. Did you try to register the "main user" in Postorius? Or in some web interface to your mail server? What do you mean by "main user"? The site administrator, also called the superuser? I don't think you can register "the" superuser through the web.[1] You need to use sudo -u mailman mailmanweb createsuperuser or similar. (If you are using a system package manager rather than pip, it would usually prompt you to do this.) > In the logs of postfix, i could see the connection from mailman to > postfix, but no mail was transmitted to postfix. It's always helpful to post the full log message. If you're concerned about security/privacy, you can redact hostnames and mailaddresses (but say what was redacted, and be prepared to answer questions about the redacted text). Did you send mail through a list? If not, why were you looking in the Postfix logs? It seems possible that some Mailman process raised an exception (a backtrace should be logged in error.log, mailmanweb.log or mailman.log), and tried to report to the site owner by email, but the address was unknown to Postfix. Without more information it's hard to way what you need to do. Regards, Steve Footnotes: [1] Once you have a superuser, it is possible to create additional superusers. But it is not possible to do that for the first one. -- GNU Mailman consultant (installation, migration, customization) Sirius Open Source https://www.siriusopensource.com/ Software systems consulting in Europe, North America, and Japan

1 month, 1 week

Re: Getting connection refused on postfix trying to connect to mailman-core:lmtp

by Abhilash Raj

On Thu, Oct 5, 2017, at 05:36 PM, Dmitry Makovey wrote: > On 10/05/2017 03:20 PM, Abhilash Raj wrote: > > On Thu, Oct 5, 2017, at 03:08 PM, Dmitry Makovey wrote: > >> > >> I've got a setup where postfix runs inside one VM(container) and mailman > >> runs inside another one (maxking containers). I've wired everything > >> according to docs yet I'm getting: > >> > >> postfix/lmtp[266]: 66A72800A87: to=<somelist(a)lists.here.stanford.edu>, > >> relay=none, delay=0.5, delays=0.48/0.01/0/0, dsn=4.4.1, status=deferred > >> (connect to mailman-01.stanford.edu[1.2.3.4]:8024: Connection refused) I found this in your settings: [mta] lmtp_host: mailman-01.stanford.edu [mta] lmtp_port: 8024 And the log message above. I believe that the LMTP runner died because it wasn't able to bind to `mailman-01.stanford.edu`, which I am assuming is the hostname assigned to the host running these containers. `MM_HOSTNAME` env variable in the docker containers should be something that the process inside mailman-core container can bind to and can be reached by postfix (which can run either on host or on another container). (Now that I read it myself, I agree that the name of the variable sounds not-so-intuitive.) The default configuration (and docker-compose.yaml) sets this value to the IP Address of the container (172.19.199.2), which is reachable from the host. If you set this value to whatever IP the mailman-core is assigned and re-create the containers (or just re-start and run `mailman aliases` in mailman-core to re-generate transport_maps), it would work out. Also, I see your docker-compose.yaml configuration (`MAILMAN_CORE_IP: 172.19.199.2`) is not consistent with your output of `mailman conf`([webservice] hostname: 172.19.199.5`). I am not sure how that happened though, just wanted to point it out. This *might* cause the rest runner to die to and thus Postorius/HK wouldn't work. Hope that was helpful! -- Abhilash Raj maxking(a)asynchronous.in

8 years, 6 months

Re: Issues with migrating from Mailman2 to Mailman 3

by Mark Sapiro

On 9/22/19 10:29 AM, Uwe Grasshoff wrote: > Hello all, > > I'm currently trying to convert a Debian based mailman2 installation to > a docker based mailman3 installation. > I successfully running the mailman-web and mailman-core containers > together with the database container but I'm struggling with the import > of the old mailing lists. In the core container I'm running this two > commands for every mailing list: > > mailman create mailman(a)example.com > mailman import21 mailman(a)example.com > /oldmailman/lists/mailman/config.pck > > Then I'm switching over to the mailman-web container and running this > two commands: > > python3.6 manage.py hyperkitty_import -l mailman(a)example.com > /oldmailman/archives/private/mailman.mbox/mailman.mbox > python3.6 manage.py update_index_one_list mailman(a)example.com > > I'm getting no errors or something else which could indicate a problem. > After that I'm looking into the web interface > http://example.com/postorius/lists/ > There I can only see the public mailing list and the private ones are > not shown there (as expected). Everything up to this point is expected behavior. > When I then try to login with the List > Manager credentials of the lists (from mailman2) I can't login in. So I > thought maybe I have to "register" myself again to than get access to > the list management. > So I tried that but when I filled out the form and click "Register" I'm > presented with an error page which titles "Serverfehler" (Server error > in german) and the message "An error occurred while processing your > request." I'm not sure about the error, but the import21 process registered you so what you need to do is go to the Login page, click "Forgot Password?" and then request a password reset. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 7 months

Mailman-users search results for query "Postorius"