Search results for "Postorius" - Mailman-users

Re: Hyperkitty performance problem

by Tobias Diekershoff

Hey all, so far we could not find the reason behind the problem. However while reviewing the installation process one one our team mentioned that he read the installation instructions for Hyperkitty different then the rest of the team. Are Mailman3/Postorius and Hyperkitty supposed to run in separated databases, or are they supposed to use the same database? Thanks in advance for clarification! Tobias On 06.11.24 08:18, Tobias Diekershoff wrote: > Hey all, > > last week we caught some AI bots causing high LOAD on our mailman3 > servers archive, that we blocked with nginx rules. > > I'm not sure if it is related to this incident, but we noticed this > when analyzing the log files at the time. In the mailman.log > hyperkitty is listed 10k times and more to archive a single mail over > the time of 2h, seemingly all with the same process ID > > > (1186) HyperKitty archived message [...] > > The mailman3-web.log has matching log entries that the mail was archived > > > hyperkitty.views.mailman Archived message > > every 0.5 seconds. > > In the web interface to the archive, the mail is only listed once though. > > My suspicion is, that these archiving processes are now overloading > the system, preventing a timely catching up work of the queued > messages. I could not find anything in the Mailman3 nor Hyperkitty > documentation to better distribute the system resources to the Mailman > runner or prevent Hyperkitty to restart archiving a mail over and over > again. > > Any hint about where to look for the reason of this is highly > appreciated. > > Best, > Tobias > -- Tobias Diekershoff >>> System Hacker Free Software Foundation Europe Schönhauser Allee 6/7, 10119 Berlin, Germany | t +49-30-27595290 Registered at Amtsgericht Hamburg, VR 17030 | fsfe.org/support OpenPGP-Key ID ... 0x25FE376FF17694A1 Fingerprint ...... 23EE F484 FDF8 291C BA09 A406 25FE 376F F176 94A1

1 year, 3 months

Re: Changing subscription address

by Stephen J. Turnbull

Abhilash Raj writes: > > There does need to be a way to revert each option to inherit, and > > there currently doesn't seem to be one. The 'Preferences' interface I > > described in the previous post doesn't provide it either. I will > > think about that. > > The implementation of this isn’t very difficult, but the UX of this is > something I don’t know how to implement in that page to be very > honest. Oh, I can play Liz Warren on that one! :-) "I have a plan for that." > We do already have ability to switch from an explicit address to Primary > Address subscription. I would imagine it would be more than just a > button, but more like a dropdown to choose an email or primary address > and switch all subscriptions to that. Allan's situation is that for some reason he has a lot of users (or at least suspects he does) who would be happy to have only one address that there's only one place to change, but for some reason that address is copied to (not inherited by) each of several subscriptions. We should address that, because I'm pretty sure his suspicion is correct. For example, until I changed one of them -- and got two emails about it for my discard pile :-) -- all of my python.org Mailman 3 subscriptions were explicit. I still have about 8 to go. :-) > This is how Core actually does work[1] if you subscribe via Email > and you already have a primary address. It will lookup a user using > the address and if the address is the primary address of the user, > it will subscribe the primary address. If you say so, I guess it's true. Nevertheless, both Allan and I have observed users with only one address, with that address subscribed literally each time to several (~10) lists. > However, if it isn’t the primary address, and this is a new address, it will > subscribe the address instead of user because the address isn’t > verified at this point and hence can’t be a primary address. We could > extend the subscription workflow to add an additional step to set this > address as primary and subscribe the primary address. Yes, please, I think. Users who don't want this for subscriptions to the primary address are going to be (rare) edge cases. If we need an additional attribute on the user to handle the case of an unverified primary address, I think that's an acceptable cost. Note that to "claim" this User as her own, she has to verify that address. And if she *can* verify that address, we can't stop her from doing so, we can just make it more annoying. > Postorius has the full ability to define its own workflow because > REST API provides full expression capability and doesn’t enforce > anything. Previous versions didn’t had support for subscribing as > Primary Address, but that has since changed and is the default one > in the dropdown. > > This is for authenticated subscription, the anonymous subscribe works > similar to email -join command. Mass-subscribe is probably working the > same way. All of these should unify all subscriptions-by-this-address to one User, in my opinion. > Like mentioned above, it should be possible to do what you suggest. I > am getting into the implementation details here, but the Subscription > workflow would need an additional flag that would verify an address > and use the flag to determine if it should subscribe the address or > addresses’s user if an address is provided to the workflow. I don't think so. Just do the lookup, if it succeeds and matches primary address, use inherit. If it doesn't succeed, create a User and use inherit. If it does succeed, and doesn't match, use the explicit address. It seems to me that this flag would be impractical for email subscriptions (email commands with parameters are generally beyond the kind of user this is most important for) or mass subscriptions (how does the list owner know what the user wants?) anyway. If somebody really wants all their subscriptions via primary address to be copies of the primary address rather than user, I'm sorry, but they'll have to deal with that. I can't imagine that it's the usual case, and there's an easy workaround: make your primary address something other than your usual subscription address! If you disagree with me about the relative burden, then maybe we shouldn't do this at all, but I'm pretty sure this is something we should either do for all subscriptions, or for none. > Primary Address is also an attribute of the Mailman User, so it > should be possible for an admin to set that for a user if the user > doesn’t have an account. When they sign up, they’d be able to > manage and update accordingly. True, but AIUI Allan has a *lot* of users and on average they're relatively needy compared to, say, my user population. The point of his request is to smooth workflows for admins with user populations where the admin *wants* to do many things for their users, or to have smooth user workflows when they teach their users to do it for themselves. That's why I'm focusing on the "I have one address and if it changes I want to change it once" case. I think it's both extremely common, and extremely annoying :-) to admins who have to deal with the current situation. Steve

4 years, 11 months

Re: Using REST with CLI using curl .... help needed

by Michael Bakonyi

Dear Stephen, allright, thx a lot for your quick answer! After fiddling around some more time, I guess I had some misunderstandings regarding the REST-API. I guess if we reach the API correctly, we will not be confronted with the CSRF-Error. We will continue to try but nevertheless: What would be a cleaner approach to reach the API from an external IP? I didn't found any examples in the web so far. Cheers, Michael > Am 24.08.2022/ 34 um 14:27 schrieb Stephen J. Turnbull <stephenjturnbull(a)gmail.com>: > > kontakt(a)michaelbakonyi.de writes: > >> I have the same usecase as Krešimir and tried the example of >> Mark. So in mailman.cfg I set hostname to >> "https://lists.mydomain.org" and then fire this command > > I don't recall this, and unfortunately Mark is on vacation and > completely offline until mid-September, except for maybe a couple days > next week. > >> curl -urestadmin:mypw --referer https://lists.mydomain.org --request POST -H 'Content-Type: application/json' --data-binary '{"list_id": "thelist.lists.mydomain.org", "subscriber": "user(a)example.net", "display_name": "Joe User", "pre_verified": true, "pre_confirmed": true, "pre_approved": true}' https://lists.mydomain.org/mailman3/postorius/lists/thelist.lists.mydomain.… >> >> But then I get a CSRF-warning. Do I now have to get the token and >> put it in my POST or is there an easier way to do this? > > As far as *I* know there's no way to avoid the CSRF warning. If that > means you can't get the response data you need, you will have to do > the CSRF two-step. > > With luck maybe Mark will see this in a few days but I think he's > going to be busy preparing the 2d half of his trip. > > Regards, > Steve >

3 years, 6 months

Re: Implicit destination

by David Krantz

On Mon, Aug 27, 2018 at 2:50 AM Mark Sapiro <mark(a)msapiro.net> wrote: > > That is not the issue. Sending the held message notice to a spammer, if > it reaches the spammer, is probably inconsequential. The issue is that > virtually always in spam the sender address is either undeliverable or > is the address of an innocent 3rd party. Sending these notices to > innocent 3rd parties is called backscatter and is a bad thing. > > To avoid this backscatter, you can set the list's > respond_to_post_requests attribute to False. There are two problems > here. One is this setting is not currently exposed in Postorius, and the > other is this will also stop desired held message notices such as those > from list members. Aha. That was a simple solution although with a bad consequence. Will it stop moderation mails to the moderator as well? I'll try. Otherwise it will not be a problem in my case. The lists where I have this problem are more of function addresses for a number of persons (like a board or workgroup) so they accept posts from non-members. I use mailman as a simple forward gets legit mail caught in the receivers spam filters and might cause more black-list problems. You are of course entirely correct in that backscatter is the real problem, I should have thought of that. The thing in the spam I've received lately is somewhat sophisticated: the senders have correct (albeit strange) domains and correct SPF records so they try to seem legit although the senders are located in unexpected parts of the world. I just checked one example, in that case the MX pointed to the sender IP, the SPF was very sloppy (~all) and everything looked correct except for the fact that we usually do not receive mail from Ukraine and the cryptocurrency scam in the body. It almost got caught in the spam filter, but not quite. cheers // David

7 years, 6 months

Re: New releases

by Mark Sapiro

On 6/9/20 1:05 AM, Danil Smirnov wrote: > On Tue, 9 Jun 2020 at 00:05, Mark Sapiro <mark(a)msapiro.net> wrote: > >> After you upgraded did you run these django-admin commands? >> ``` >> django-admin collectstatic --clear --noinput --verbosity 0 >> django-admin compress >> django-admin compilemessages >> django-admin migrate >> ``` >> These 4 commands should be run after every update to Postorius, >> HyperKitty or django_mainlman3. >> > > Hm, interesting... Could you point me to some related docs please? The documentation is weak and should be updated, but see <https://docs.mailman3.org/en/latest/upgrade-3.2.html#migration-steps> > To upgrade, I've just bumped app versions in my Dockerfiles, built and > restarted... > Mailman-core worked right away with no issues. > Mailman-web (Hyperkitty) failed with the error: > > ---------- > ERROR 2020-06-08 19:46:41,942 14 django.request Internal Server Error: > /hyperkitty/ > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/compressor/base.py", line 282, in > precompile > mod = import_module(mod_name) > File "/usr/lib/python3.6/importlib/__init__.py", line 126, in > import_module > return _bootstrap._gcd_import(name[level:], package, level) > File "<frozen importlib._bootstrap>", line 994, in _gcd_import > File "<frozen importlib._bootstrap>", line 971, in _find_and_load > File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked > ModuleNotFoundError: No module named 'sassc -t compressed {infile} > {outfile}' ... > I've fixed the error by removing the older file which is > /opt/mailman-web-data/static/hyperkitty/libs/bootstrap/stylesheets/_bootstrap.scss > > After that everything seems to be working fine. I'm not sure about the above error. 'sassc -t compressed {infile} {outfile}' is defined in `settings` COMPRESS_PRECOMPILERS, but it is a command to be run as part of `django-admin compress`, not a module to be imported. Possibly running `django-admin compress` before starting Hyperkitty would have avoided this. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 9 months

Re: Re-send subscription response and file permission error in queue

by Mark Sapiro

On 9/11/20 2:18 PM, Mark Sapiro wrote: > On 9/8/20 10:56 PM, accounts-mailman(a)holbrook.no wrote: >> A user sent subscription request to the `..-join@` address of a list. Email domain wasn't set up properly, so response got rejected. Now, if the user tries to send another request, no reponse comes back. In the log I find a python stack trace saying subscription already pending. >> >> ``` >> Sep 08 10:32:44 2020 (19918) Traceback (most recent call last): >> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 171, in _one_iteration >> self._process_one_file(msg, msgdata) >> File "/usr/lib/python3/dist-packages/mailman/core/runner.py", line 264, in _process_one_file >> keepqueued = self._dispose(mlist, msg, msgdata) >> File "/usr/lib/python3/dist-packages/mailman/runners/command.py", line 196, in _dispose >> mlist, msg, msgdata, parts, results) >> File "/usr/lib/python3/dist-packages/mailman/commands/eml_membership.py", line 104, in process >> ISubscriptionManager(mlist).register(subscriber) >> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 520, in register >> list(workflow) >> File "/usr/lib/python3/dist-packages/mailman/app/workflow.py", line 69, in __next__ >> return step() >> File "/usr/lib/python3/dist-packages/mailman/app/subscriptions.py", line 227, in _step_sanity_checks >> raise SubscriptionPendingError(self.mlist, self.address.email) >> mailman.interfaces.subscriptions.SubscriptionPendingError >> ``` > > > I'll look at this when I get some time. I'm still catching up after > being off line for 12 days. This exception should also send a response to the user's email saying "User <user(a)example.com has a pending subscription for list(a)example.net" Was this sent? To fix, >> How can I remove the pending subscription, or re-send the response email? > > In Postorius, pull down Subscription Requests and select Pending > Confirmation and delete the request. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

5 years, 6 months

Re: Getting connection refused on postfix trying to connect to mailman-core:lmtp

by Abhilash Raj

On Thu, Oct 5, 2017, at 05:36 PM, Dmitry Makovey wrote: > On 10/05/2017 03:20 PM, Abhilash Raj wrote: > > On Thu, Oct 5, 2017, at 03:08 PM, Dmitry Makovey wrote: > >> > >> I've got a setup where postfix runs inside one VM(container) and mailman > >> runs inside another one (maxking containers). I've wired everything > >> according to docs yet I'm getting: > >> > >> postfix/lmtp[266]: 66A72800A87: to=<somelist(a)lists.here.stanford.edu>, > >> relay=none, delay=0.5, delays=0.48/0.01/0/0, dsn=4.4.1, status=deferred > >> (connect to mailman-01.stanford.edu[1.2.3.4]:8024: Connection refused) I found this in your settings: [mta] lmtp_host: mailman-01.stanford.edu [mta] lmtp_port: 8024 And the log message above. I believe that the LMTP runner died because it wasn't able to bind to `mailman-01.stanford.edu`, which I am assuming is the hostname assigned to the host running these containers. `MM_HOSTNAME` env variable in the docker containers should be something that the process inside mailman-core container can bind to and can be reached by postfix (which can run either on host or on another container). (Now that I read it myself, I agree that the name of the variable sounds not-so-intuitive.) The default configuration (and docker-compose.yaml) sets this value to the IP Address of the container (172.19.199.2), which is reachable from the host. If you set this value to whatever IP the mailman-core is assigned and re-create the containers (or just re-start and run `mailman aliases` in mailman-core to re-generate transport_maps), it would work out. Also, I see your docker-compose.yaml configuration (`MAILMAN_CORE_IP: 172.19.199.2`) is not consistent with your output of `mailman conf`([webservice] hostname: 172.19.199.5`). I am not sure how that happened though, just wanted to point it out. This *might* cause the rest runner to die to and thus Postorius/HK wouldn't work. Hope that was helpful! -- Abhilash Raj maxking(a)asynchronous.in

8 years, 5 months

Re: Issues with migrating from Mailman2 to Mailman 3

by Mark Sapiro

On 9/22/19 10:29 AM, Uwe Grasshoff wrote: > Hello all, > > I'm currently trying to convert a Debian based mailman2 installation to > a docker based mailman3 installation. > I successfully running the mailman-web and mailman-core containers > together with the database container but I'm struggling with the import > of the old mailing lists. In the core container I'm running this two > commands for every mailing list: > > mailman create mailman(a)example.com > mailman import21 mailman(a)example.com > /oldmailman/lists/mailman/config.pck > > Then I'm switching over to the mailman-web container and running this > two commands: > > python3.6 manage.py hyperkitty_import -l mailman(a)example.com > /oldmailman/archives/private/mailman.mbox/mailman.mbox > python3.6 manage.py update_index_one_list mailman(a)example.com > > I'm getting no errors or something else which could indicate a problem. > After that I'm looking into the web interface > http://example.com/postorius/lists/ > There I can only see the public mailing list and the private ones are > not shown there (as expected). Everything up to this point is expected behavior. > When I then try to login with the List > Manager credentials of the lists (from mailman2) I can't login in. So I > thought maybe I have to "register" myself again to than get access to > the list management. > So I tried that but when I filled out the form and click "Register" I'm > presented with an error page which titles "Serverfehler" (Server error > in german) and the message "An error occurred while processing your > request." I'm not sure about the error, but the import21 process registered you so what you need to do is go to the Login page, click "Forgot Password?" and then request a password reset. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

6 years, 5 months

Re: Upgrade-path from Ubuntu18LTS to recent mailman release

by Stephen J. Turnbull

Stefan Bauer writes: > thanks alot. That looks promising. Can we just copy the current > mailman3-database and take the configurations or is there a specific > migration path we have to follow? Much of the documentation on mailman.readthedocs.io discusses migrations and upgrades. You should read those for general hints. The distro Mailman to venv Mailman case is more discussed here on the mailing list than in those documents. I don't have pointers to those threads or the time to look offhand, maybe over the next few weeks as I work on our GSoC project. If you do find any, and would post them here, I'll make sure they get into the docs. The main thing is to backup Mailman configs (probably in /etc/mailman or /etc/mailman3) and the backend database (core, Postorius, possibly HyperKitty depending on configuration, and the django-mailman auth stuff) before trying to migrate. (For the backend database I would just dump the RDBMS unless non-Mailman content is really huge.) The nature of the virtualenv install is such that it *will not* overwrite any of the distro files at all (as long as you put it in /usr/local or /opt). However, you want to preserve the configs and database and reuse them, so those *must* be backed up. > We are currently running 3.1.1-9 (Ubuntu 18). Ubuntu 18 is probably old enough that you need to be careful about MySQL and its derivatives. The specific problems are (1) you need to specify the right UTF-8 variant for the coding system, and (2) more recent versions of Mailman tend to make certain records overflow. If you're using PostgreSQL these caveats *do not* apply. If you're using sqlite3 (I doubt it, but just for completeness) you should change to PostgreSQL or MySQL. If you're using something else (same caveat), I can't really say. Probably somebody out there is using Oracle or whatever, but we don't know about that. :-) Steve

3 years, 10 months

Re: Digests not working correctly

by Mark Sapiro

On 6/4/23 18:35, Joel Lord wrote: > > The periodic digests do seem to be coming out. I also now have > confirmation that the one message in this morning's digest that was from > May 4th was also included in the last digest back on May 4th, so it > seems that the one message was left behind in the digest queue when the > periodic digest was sent. I don't see how that can happen. The process that sends a digest renames the var/lists/<list-id>/digest.mmdf mailbox file in which the messages are accumulated to var/lists/<list-id>/digest.<volume>.<issue>.mmdf, where <volume> and <issue> are the volume and issue numbers of that digest, and then queues a message in the `digest` queue to tell the digest runner to create the digest from the messages in that mbox and send it. Thus, it leaves no var/lists/<list-id>/digest.mmdf mailbox file behind and that is created anew when the next post arrives. Further, if there is a non-empty digest.mmdf file, its messages should be sent no later than the next 11 PM `cron digests`. > There was one earlier message to the list > back on May 4th, before the one that got duplicated, but I can't tell if > that triggered a size-based digest to be sent: the logs aren't clear > enough on that detail for me to tell. OK > Just to inform things: > > (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep mailman > django-mailman3==1.3.9 > mailman==3.3.8 > mailman-hyperkitty==1.2.1 > mailman-web==0.0.6 > mailmanclient==3.3.5 > (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep hyper > mailman-hyperkitty==1.2.1 Actually, it's HyperKitty, not hyperkitty, but I assume HyperKitty is up to date as are the others. > (venv) lists@host2:~/mailman/venv/bin$ pip freeze | grep post > postorius==1.3.8 > > -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 years, 9 months

Mailman-users search results for query "Postorius"