Search results for "Postorius" - Mailman-users

Re: Emergency moderation and clearing out a lot of held messages

by Caballero, Danny (Dan)

Thanks for some clarification. I cleared out 2800 held messages via Python shell "mailman -l" option and it took about 28 hours. Anything done via Mailman shell is a connection from Mailman core to the database, correct? Although I can't replicate that many messages via Postorius/DJango, I do see similar slowness there as well at a message per minute or more of processing. I recreated the above Mailman shell clean up via Docker container running MariaDB and noticed similar processing time for discarding held messages. That leads me to conclude that the issue isn't network related or specific to AWS RDS. I'm open to testing and switching over to Postgres if that is the straightforward solution. ---- Dan Caballero Systems Administrator Academic Computing Solutions IMSS - Caltech https://imss.caltech.edu ________________________________ From: Stephen J. Turnbull <stephenjturnbull(a)gmail.com> Sent: Friday, November 11, 2022 10:23 AM To: Caballero, Danny (Dan) <dancab(a)caltech.edu> Cc: mailman-users(a)mailman3.org <mailman-users(a)mailman3.org> Subject: [MM3-users] Re: Emergency moderation and clearing out a lot of held messages Caballero, Danny (Dan) writes: > I don't understand. We have a single database configured for > everything. I don't recall reading anything in the set-up > documentation about using more than 1 database for Mailman3. I was trying to help localize the database request to one of the three major components: core, Postorius, HyperKitty. If you use the two- database scheme documented here: https://docs.mailman3.org/en/latest/install/virtualenv.html#virtualenv-inst… knowing that database would at least identify whether the slowness is associated with core or with the web apps. The excruciating details: Conceptually, there are three separate databases: the one used by Mailman core to manage everything but authentication and archives, the one used by Django to manage authentication for HyperKitty and Postorius, and the one used by HyperKitty to manage archives. They don't share tables, and there are no name collisions for the tables so it doesn't really matter how the backend handles them. The two database configuration is not really a recommendation. I suppose the practice derives from an abundance of caution, as there are concepts like "User" that must be implemented in somewhat different ways across components (core doesn't authenticate users, while Postorius's primary function is authentication of users). I myself would worry that there might be name collisions among tables, which would cause mayhem, but that's probably excessive.

2 years

Re: Fresh installation mailman3-suite via "pipx" - any experiences yet?

by Stephen J. Turnbull

Mark Sapiro writes: > > On Thu, Jan 25, 2024 at 4:43 PM Wikinaut <mail(a)tgries.de> wrote: > > > >> I recently found the "pipx" wrapper very useful and wanted to > >> ask you if this could be facilitate the fresh installation of > >> the mailman3-suite. Has anyone in the list already experiences > >> installing mailman and mailmanweg through "pipx"? > pipx apparently does create virtual environments. See > https://pipx.pypa.io/stable/comparisons/ > > The thing there that concerns me is "pipx replaces a subset of > pip's functionality; it lets you install cli applications but NOT > libraries that you import in your code." > > What does this say about the multiple dependencies that we install. I'm pretty sure it will install dependencies. I think (everything I say below is based on reading the README.md from pipx's pypa.io repo, so caveat installer) that the complexity that will arise is that (for example) a naive approach like $ pipx install mailman $ pipx install mailmanweb $ pipx install postorius $ pipx install hyperkitty will create 4 separate venvs with 3 copies of mailmanclient, 3 copies of django-mailman3, and two copies of Django. Besides the space waste, there's also the small problem that mailmanweb won't have copies of postorius and hyperkitty in its venv. :-þ pipx has an "inject" command that allows adding other packages to a venv, but I think this means that pipx has no great advantage over the standard way of installing mailman into a venv. pipx also installs stuff into the user's $HOME, specifically into ~/.local/share/pipx/venvs which I think will be kind of annoying if you need to dig into package resources. (I expect that you will still install configs into /etc/mailman3 and various data into /var/lib/mailman3/var or /opt/mailman3 or whatever without a problem.) This would only be an issue for heavy-duty customizers. This can probably be customized. One of the big advantages pipx touts is "look ma, no sudo". That's not really an advantage for system daemons, though. So bottom line is you can do it, but you'll need to study up on pipx arcana, because Mailman is a bunch of daemon it buys you very little, and the result won't look like any Mailman installation you've seen described on mailman-users or StackExchange. Steve

10 months

Deactivate list without deleting it?

by Stephen J. Turnbull

Colin Watson writes: > Ideally, I'd like mailing lists to have a "deactivated" state in > addition to full deletion. In this state, the list would appear > nonexistent for most purposes: incoming email would be rejected, > Postorius would deny its existence, etc. [...] > Has there been any discussion of this in the past, or any advice as to > implementation strategy? I don't recall anybody discussing this. Your use case is your use case, but I think if we were going to implement this I'd probably want "partial" deactivation where the archives remain available but no mail is delivered, can't subscribe, etc. I'm also not sure I'd want Postorius to deny it ever existed. That seems rather unfriendly (again, I don't understand your use case, so that is not a comment on your requirements). So to get it into the distribution we'd need to make these things optional. (Perhaps if these options proliferate much, they could be incorporated into list "styles" so that instead of setting all the options manually, the primary use cases could be served by "intuitively" named styles.) It seems to me that dump, delete, and restore may be the best approach for the HyperKitty side because HyperKitty maintains all that state. Of course this could be done inside the database. Also, perhaps you can (ab)use the ArchivePolicy enum by adding an "inactive" element, but core has no way to enforce those on the archiver (which can be a third party host like mail-archive.com), and I'm not sure HyperKitty ever consults core about that once the archive is created. I'm not sure what you mean about IArchiver; as far as I can see that interface is sort of half-duplex: it provides the information core needs *about* the archiver to do core's side of archiving, not for core to communicate *with* the archiver. In fact all of the concrete code is callbacks provided by the archiver, and once a message is archived the core forgets about it. So this isn't "just" an extension; it's a fundamental redesign of IArchiver. I'm not sure that's not a good thing, but we do need to think about it. ISTM we're gradually converging back on the monolithic approach to MLM, with the distribution, administration, and archiving quite tightly interwoven, and I'm not sure I like that. OTOH, we do get a lot of requests for ever more bundling.

6 years, 10 months

Re: list of subscribers

by Simon Hanna

On 02/04/2018 08:37 PM, Peter Münster wrote: > On Sun, Feb 04 2018, Simon Hanna wrote: > >> You should be able to use mailmanclient to get a list of subscribers with >> names. > It seems, that mailmanclient does not know about the real name (the > first name and the last name in Django). But this seems to work: > > from django.contrib.auth.models import User > user = User.objects.get(email='address(a)of.user') > print(user.first_name) > print(user.last_name) > > So, I'll try to make some loop over a list with email addresses, that I > can get on the web interface. These are names that are saved in Postorius' (django) database and have nothing to do with mailman. Furthermore you will only find them if people have logins for the frontend. They don't have to. Also these names are optional. They aren't populated during signup and people have to provide them afterwards themselves. >> Names are optional though, if people signup without them you won't be able to >> get them. > My usage of mailman is perhaps a bit special: users can't sign up, only > the site owner signs up manually with create_user(email=, first_name=...) > Then the site owner creates the mailing lists and uses mass subscription > (for example all members of a basketball team). At one moment, the list > of subscribers should be verified by all members, just to be sure, that > nobody got forgotten. > > >> Currently there is no option to require names. I'll open an issue in >> postorius about showing the name in the member list if there is one >> associated. > Thanks. It would be nice, to see the first_name and the last_name of the > django-auth user object. > > >> IIRC one person(postorius account) can also signup at different lists using >> different names (regardless of address). > You mean the "display_name" of mailmanclient here? I won't use it. > For my usage, I prefer a clear association of email, first name and > last name in one account, no need for "display_name" here. Displaying the names stored in django probably won't happen. Matching the names in django with the members in lists would require lots of code where people can just supply their name when they signup for a list. Also you might want to be known by name on one list and not on others...

6 years, 9 months

Re: Postorius claims 2 messages held when there are none - out of sync

by Ken Alker

--On Monday, June 26, 2023 8:24 AM -0700 Mark Sapiro <mark(a)msapiro.net> wrote: > On 6/26/23 12:01 AM, Ken Alker wrote: >> When I view a list via Postorius the "Held Messages" menu item has an >> inverse-red-"2" next to it, as if there are two held messages. There >> are, in fact, no messages in "/opt/mailman/mm/var/queue/*" (assuming the >> held messages should be in "pipeline" subdirectory). When I click on >> "Held Messages" I get "Something went wrong" followed by "HTTP Error >> 500: {"title": "500 Internal Server Error"}". How do I fix this? > > The held messages are not in any queue. They are stored in Mailman's > var/messages/ hierarchy. They can be examined with `mailman qfile` Is the var/messages hierarchy also where all the archived emails are stored for future viewing via the web interface? > Also, there is a long thread at > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/threa > d/UWCUQYU7Q5X5LWNII57ZZ4DDXE4DEFX7/ and maybe others that might be > relevant. I have never used the mailman shell before, but based on the referenced thread I learned to do this (note that the thread also taught me how to rewrite it to narrow down to just the list with issues so I didn't delete held messages in other lists, if my case): for verification: venv) mailman@speedy:~/web/logs$ mailman shell -l sbarc-list(a)lists.netlojix.com Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list >>> requests = IListRequests(m) >>> first = True >>> for request in requests.held_requests: ... key, data = requests.get_request(request.id) ... if first: ... first = False ... print(m.list_id) ... print (f"""\ ... Sender: {data['_mod_sender']} ... Subject: {data['_mod_subject']} ... Date: {data['_mod_hold_date']} ... Reason: {data['_mod_reason']} ... """) ... and then to delete the two messages: (venv) mailman@speedy:~/web/logs$ mailman shell -l sbarc-list(a)lists.netlojix.com Welcome to the GNU Mailman shell Use commit() to commit changes. Use abort() to discard changes since the last commit. Exit with ctrl+D does an implicit commit() but exit() does not. The variable 'm' is the sbarc-list(a)lists.netlojix.com mailing list >>> requests = IListRequests(m) >>> first = True >>> for request in requests.held_requests: ... key, data = requests.get_request(request.id) ... if first: ... first = False ... print(m.list_id) ... print(f'request id: {request.id}, deleting') ... requests.delete_request(request.id) ... sbarc-list.lists.netlojix.com request id: 27, deleting request id: 28, deleting >>> The result of this, however, was that the web interface still claimed I had two held messages. I wasn't sure if there was some cron-type "sweeper" process I had to wait to auto-run or if I had to manually do something to create an update. I then realized that the sample script had a ' >>> commit ()" ' at the end, which I'd missed. So, I re-entered the script (but it didn't spit out that it deleted anything this time) and got an error when I did the commit. So, I figured the messages HAD been delete and the commit wasn't really necessary. When I went back to the web interface, the number of held messages was at 0 and I no longer get the original error when clicking on "Held messages", so my issue is solved. But this does leave me with some educational questions: 1) Did I affect something somehow with my "commit", or was there a job that cleaned things up (if so, what job)? 2) Is the "commit" even necessary (I saw it in someone else's script in the thread)? 3) Is there a way to save a script and then execute it via shell rather than typing it in every time (it appears from the examples that people are typing them in every time, which is super tedious; I didn't paste for fear of doing something damaging, but maybe that would work)? 4) Are the request.id's of the messages dynamic/changing, or are they fixed for life? I thought I'd delete based on exact request.id without looping but I was afraid maybe they were changing over time and that this wouldn't be a good idea. Might be nice to know for the future. Thanks for the guidance! Ken

1 year, 5 months

Re: Postorius not HTML-decoding < to "<" characters

by Mark Sapiro

> > Does that mean the DB has data with mismatched tags in it? The database has exactly what list members receive by email. What we are talking about is how this is rendered in HyperKitty. > Interestingly, the quoted part of my message has < where my message > has proper less-than and greater-than for the Apache tags. > > > https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… > > > It's inconsistent with the archive rendering mode at lists.mailman3.org. Here's what's going on. In normal text, mistune disables HTML tags by converting `<` to the html entity `<`, but your browser renders the HTML entity `<` as `<` so you see what was written. The issue here is the text you quoted was originally in a markdown code block, i.e. preceded and followed by lines of three backticks. Code blocks are intended to be seen literally as written so they are wrapped in <pre><code> ... </code></pre> tags in the HTML and also HTML escaped. The problem is the tag disabling converts `<` to `<` but then the HTML escaping converts `<` to `&lt;` which your browser then renders as `<` rather than `<`. (aside, it will be interesting to see how that is rendered in the archive) If HyperKitty is set to render as plain text rather than markdown it doesn't interpret code blocks so this doesn't happen. There are other anomalies with the markdown rendering, e.g., as pointed out by Jeremy at https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/message… Those notwithstanding, we prefer markdown rendering for this list because often people post Python snippets or `mailman shell` interactions and enclosing those in markdown code blocks renders them much better. > I haven't currently got an instance that I can test on but will keep > this in mind when the time comes to migrate. > > Should the rendering at lists.mailman3.org get modified, please share > the procedure. > > > > Testing: less-than and greater-than around a tag: > > <Location /mailman/accounts/signup> > > Testing: less-than and greater-than around a tag in a quoted block: > > > <Location /mailman/accounts/signup> Both the above tests look fine in the archive. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 weeks, 4 days

Re: Mailman3 throws 500 error (unicode encode) when loading held messages in Postorius

by Stephen J. Turnbull

Simon Handfield via Mailman-users writes: > I have been able to identify a file in the shunt queue that threw a > matching error when I performed mailman qfile on it. > I have run a loop over all remaining files in the shunt queue and > all the remaining files can be accessed by mailman qfile without > error. Good. But there are still a bunch of problems with those files since they got shunted. > I moved this file out of the queue in the hope that I would then be > able to load the held_messages page successfully. Unfortunately I > am still seeing a 500 error on loading the page. If this is a Mailman problem, I don't understand it. The shunt queue is not the moderation queue. The shunt queue is for messages that Mailman cannot handle without throwing an exception. Once a message is sent to the shunt queue, Mailman ignores it, so it shouldn't cause problems for the moderation page. Have you tried restarting Mailman since moving the qfile? If that doesn't work, and if it's the same error message and traceback, the easy suspect is browser caching (although we should be disabling that, I hope). Try a manual refresh (usually Ctrl-R or in the menu). If that doesn't work, try a private browsing window or another browser if you don't want to clear the cache or restart the browser. > I temporarily reenabled debug logging to confirm that the same > error was being thrown. > > Assuming that file was the cause are there other steps I need to > perform in addition to removing the file from the shunt queue > directory? The only one I can think of is the check for browser-side caching. Unfortunately, Mark's out of town (like, on top of a mountain with no bars on his phone), and I am not going to be able to do much with this until Thursday at the earliest. If you do get a new traceback, post it here (somebody else may have an idea). If you like you may send me the qfile at the address in From. I'll see if I can reproduce the bug, and I'm pretty sure I can tell you what's in the file so at least you'll know whether it's important to somebody who isn't a scammer/spammer. Steve

4 years, 2 months

Re: {Spam?} Server name in new account confirmation email

by Meehan, John

Thanks for your replies. From Mark's reply, and looking at django_allauth, the Subject of the confirmation email uses the site name from the django_site db table in brackets, but the URL in the message body comes from the HTTP header. Adding proxy_set_header Host $http_host before the proxy_pass line on the SSL proxy server solved the problem. ________________________________ From: Mark Sapiro <mark(a)msapiro.net> Sent: Thursday, December 3, 2020 10:25 AM To: mailman-users(a)mailman3.org <mailman-users(a)mailman3.org> Subject: [MM3-users] Re: {Spam?} Server name in new account confirmation email On 12/3/20 9:26 AM, Meehan, John wrote: > Hi, when someone creates a new account (Sign Up), an email is sent to them to confirm their email address. It has the form below. Does anyone know how I can Caution (External, mark(a)msapiro.net<mailto:mark@msapiro.net>) Sensitive Content, First-Time Sender Details<https://shared.outlook.inky.com/details?id=Y29tbXNjb3BlL2pvaG4ubWVlaGFuQGNv…> Report This Email<https://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL2pvaG4ubWVlaGFuQGNvb…> FAQ<https://www.inky.com/banner-faq/> Protection by INKY<https://www.inky.com> On 12/3/20 9:26 AM, Meehan, John wrote: > Hi, when someone creates a new account (Sign Up), an email is sent to them to confirm their email address. It has the form below. Does anyone know how I can change the server name in the URL (From server.example.com to something else)? I have an SSL proxy server that I want to direct users through. Thanks. - John This message comes from django_allauth. The domains come from the django_site database table. If you have Postorius >= 1.3.3 The Domains view will show you the SITE_ID for the domain and has an `Edit` link the the Django admin UI entry for that site. Otherwise, you can go to the Django admin `Sites` view and see the DOMAIN NAME and DISPLAY NAME for each defined site, and you can edit the domain name. You may also need to adjust the SITE_ID setting in your Django settings to match the site you want. The trick here is the SITE_IDs aren't displayed in the Django admin UI or in Postorius < 1.3.3, and they may not be sequential or start at 1. If you query the database with `SELECT * FROM django_site;` you'll see entries with columns id, domain and name which will tell you the SITE_ID. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan _______________________________________________ Mailman-users mailing list -- mailman-users(a)mailman3.org To unsubscribe send an email to mailman-users-leave(a)mailman3.org https://secure-web.cisco.com/1FBkrl8Fna83jeBpMzhcS-TGdx6ALOyBwi0uCPsqGDtm9J…

3 years, 11 months

Re: Changing subscription address

by Stephen J. Turnbull

Thank you for your comments, Allan! Allan Hansen writes: > On 3/15/21, 21:28 , "Stephen J. Turnbull" <turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: >>In the 'Mailman Settings' screen, List-based Preferences' tab, >You may mean the 'Subscriptions' tab. I did mean the 'List-Based Preferences' tab. The 'Subscriptions' tab is non-functional. It's in some sense aesthetically more pleasing than 'List-Based Preferences', but you have to do extra, unnecessary work to get anything done from the 'Subscriptions' tab. It may make sense to just get rid of the current 'Subscriptions' tab and rename 'List-based Preferences' to 'Subscriptions'. The only functionality of 'Subscriptions' over 'List-based Preferences' is that it lists you once for each role and address (member, non-member, moderator, owner) that you have for that list. This is possibly more confusing than helpful for non-owners. Moderators get informed of their role every time they need to do something. And it's possible to be both a member and a non-member of a list, which is pretty confusing. > Under 'User profile for <name>', under the 'E-mail Addresses', add > button to ensure that 'Primary E-mail' is used for all lists. If you define "ensure" strictly, this would be a fair amount of work because it would require overriding the normal "cascade" of preferences, and incidentally a way to turn off that override. I'd rather not make the implementation so complicated. There does need to be a way to revert each option to inherit, and there currently doesn't seem to be one. The 'Preferences' interface I described in the previous post doesn't provide it either. I will think about that. It should be normally unnecessary to use the 'Preferences' interface, because the default subscription should be "use primary address". For people using the current versions of Mailman 3, who are already have a bunch of subscriptions with the same *explicit* address, I think it is good enough. But if there's demand for a "revert all to use primary address" button, that would be easy enough to add. Here's why I think such a button would be unnecessary in future versions of Mailman. What should happen if you subscribe by an address not already known to Mailman without creating an account in Postorius is 1. Mailman registers the address. 2. Mailman creates a User object to "own" that address, and makes that address the primary address for the User. 3. Mailman creates a subscription which links the User (not the address) to the mailing list so that the primary address is used. For addresses already known to Mailman, Mailman should look up the User registered to that address, and if the subscription address is the primary address, set "use primary address" for the subscription. In this scheme, as far as I can see, your user doesn't need to do anything with the account until she wants to change the address. At that point she needs to activate the account: - visit her personal options page in Postorius - request a password reset (she doesn't have one yet, so she can't log in) - click the special link - set a password or link a social media account, and - finally change the address. Note that this is the bare minimum of work for her. She has to prove she can read mail at that address. Anything less, and *anybody* can change her subscriptions. We could omit requiring a new password, but it's not a lot of work once you've got to this point, while the "request, wait for email, click link, do work" dance is pretty annoying. Why make someone do it again next time? This is problematic if she's already lost access to the old address, of course, but in that case you're probably going to have to intervene anyway. I don't think Postorius requires her to do anything else, but if she wants to she can update her profile at that point. I have to check the code, and probably also ask Barry and the other folks who designed the current subscription mechanism in case there's some fatal flaw in the scheme above. But I think that should do what almost all "one address" users want, and it's probably what the majority of people who have multiple addresses and use different ones for different purposes want most of the time, too. Note to self of possible problems: (1) The primary address is where you would send administrative mail to the account holder and that might not be an appropriate address for the default subscription. (2) This is a pretty radical change from the current situation, so it might need to be an option for the list or domain owner. > And, indeed, new subscriptions should default to 'Use Primary.' I agree. I'm astonished that there are situations where they don't, to be quite honest. That's the whole point of having accounts in the first place -- having *one* source of information about the user rather than spreading it out over a bunch of lists. > Is that possible now? It is already the default for someone who subscribes by logging in to Postorius. I assume that it is *not* the default for someone who subscribes by email or mass subscription, otherwise you wouldn't have the problem. > Does a member have to have an account for me to set it up thus for > them? Yes. The primary address is an attribute of the account. Subscriptions can't know anything about a primary address unless there is an account. However, it should be possible to make this default for an account that hasn't been activated yet, such as in a mass subscription. Regards, Steve

3 years, 8 months

Re: REST API Question: possible to change user preferred email?

by Abhilash Raj

On Fri, Mar 15, 2019, at 10:59 AM, Nathan Dixon wrote: > Hi, > We are in need of being able to change a user's email address via the REST > API. After looking through the documentation and code, I can't see a way to > do this in a simple way. > > Although the documentation seems to show that if a user changes their > preferred address, it will track throughout all their subscriptions. It depends on how are you actually using Mailman. Postorius today subscribes people using their addresses, so the method you mentioned below is the correct way to change addresses. If you think you are going to do this more frequently, you could subscribe users with their user_ids and then when you need to change their memberships, you just update their preferred_address. See [1] here for how to join a mailing list using user_ids. Note that this isn't available today using Postorius (our Web UI), but I'd be happy to work with you if you want to contribute it. [1]: https://mailman.readthedocs.io/en/latest/src/mailman/rest/docs/membership.h… > > See: > https://gitlab.com/mailman/mailman/blob/master/src/mailman/rest/docs/member… > "When Gwen changes her preferred address, her subscription automatically > tracks the new address." > > The only way I can see to do this via the REST API would be to do the > following long winded way: > > *1. Create a new address:* > POST to http://<api_url>/3.1/users/<$OldEmail>/addresses > $postData: {email = "$NewEmail"} > > *2. Verify the new address:* > POST to http://<api_url>/3.1/addresses/<$NewEmail>/verify > $postData: { } > > *3. Get all the memberships for the old email address* > $memberships = GET to http://<api_url>/3.1/addresses/<$OldEmail>/memberships > > *4. Change users email for each membership individually* > foreach ($membership in $memberships) { > PATCH to http://<api_url>/3.1/members/<$membership.member_id> > $pathData: { address = $NewEmail } > } > > *5. Delete Old Address* > DELETE to http://<api_url>/3.1/addresses/<$OldEmail> > > Am I missing something (as in there is a way to do a single PATCH request > somewhere)? Or is this the only way to update someone's email address with > these 5 steps? > > Thanks, > Nathan Dixon > _______________________________________________ > Mailman-users mailing list -- mailman-users(a)mailman3.org > To unsubscribe send an email to mailman-users-leave(a)mailman3.org > https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/ > -- thanks, Abhilash Raj (maxking)

5 years, 8 months

Mailman-users search results for query "Postorius"