On 10/19/20 9:04 PM, Mark Sapiro wrote:

I suppose it could be considered a bug, but what's happening is your initial subscribe created a user and and address and set the user's preferred address to the created address. It also set the display_name of the address and it subscribed the user to the list or possibly the address to the list depending on what actually did the subscribe.

When the user/address was unsubscribed, it is removed from the list's membership and the list was removed from the user's memberships, but the user and address still existed.

Then when you resubscribed, the existing user/address was made a member of the list, but not updated so the "new" display_name was ignored. I think this is also the case if the original address had no display_name.

You could say this is a bug and the display_name associated with this new subscription should replace the prior one, but what if that address was subscribed to other lists and didn't want the display_name changed on those lists.

Ideally (perhaps) there could be multiple address records with the same email address and different display_names, but address records are keyed by email address, so this can't be.

One workaround would be for a user who wanted to have an address with different display_names for different lists to have multiple addresses likeaddress+list1@example.com,address+list2@example.com, etc.

Is this really an issue that should be fixed or just a curiosity?

No. No. No.

No one set their preferred address here at all. The first incident was an imported list member who was not a registered user. They wanted their real name changed to something else. The list owner tried to figured out how to do that, found no way to do it, figured out it was just easier to unsubscribe them, and resubscribed them to the list using a different name. The old name showed up again. This is a bug and please fix it soon. This has nothing to do with being subscribed to other lists. I can probably come up with a workaround using Affinity (one of the benefits of moving away from Postorius) but I can't imagine this not being a serious issue as Mailman 3/Postorius is more widely adopted.

-- Brian Carpenter Harmonylists.com Emwd.com

On 10/19/20 10:23 PM, Mark Sapiro wrote:

And the import21 created the user and address records for this user.

Does the same thing for a new subscriber as well. So there is no pathway to change a real name that is associated with an email address. None. Zilch. Mailman 2 made this so easy and Mailman 3 made it impossible. I will let someone else file the bug report. You don't really think that this is an issue which means it will be years before it is addressed. So I will save me some time. I will learn to live with it.

-- Brian Carpenter Harmonylists.com Emwd.com

On 10/19/20 11:32 PM, Mark Sapiro wrote:

On 10/19/20 7:42 PM, Brian Carpenter wrote:

...

...

And the import21 created the user and address records for this user. Does the same thing for a new subscriber as well. So there is no pathway to change a real name that is associated with an email address. None. Zilch. Mailman 2 made this so easy and Mailman 3 made it impossible. I will let someone else file the bug report. You don't really think that

On 10/19/20 10:23 PM, Mark Sapiro wrote: this is an issue which means it will be years before it is addressed. So I will save me some time. I will learn to live with it.

I didn't say I didn't think it was a real issue. I mostly questioned whether a new subscription should change an existing display_name. I agree that there should be a way for a user to change the display_name associated with her address. I'm not so sure about a list admin.

Respectively, I think you are asking the wrong question here. The real question is why isn't a display_name being removed when a list subscriber is unsubscribed. Also list members being forced into a powerless role by Mailman 3's architecture have no way of changing a name except through a list owner unless they register an account but then again, *even that doesn't allow them to change their display name*. There are also other use cases in which a list owner will want to identify portions of their membership roster through the use of the display_name.

Most list members interact with a Mailing list program via email. I know you understand that. However it is List owners that are the "real" power users of Mailman and so the admin interface should be designed in such a way that empowers List owners. Taking the ability away from List owners (and List members to make), what should be a simple name change, weakens them.

And in any case, I'm only one person. I'm not the only one deciding what's important and what's not. I only decide what I want to work on, not what anyone else thinks is important or wants to work on, so even if I think something is not worth doing, that doesn't mean it won't get done, and once again for emphasis, I do thing a user should be able to change the display name associated with her address(es).

Again I think we are missing something here from this conversation: data is not being removed. A list subscriber being removed (unsubscribed) should have their information totally removed. It is not. I now have one list owner who realizes this and it is not good. How does this not violate GDPR? This is why I think all the mailman developers ought to make this issue a high priority. It is creepy to see a name returned out of nowhere when someone resubscribes to a list without associating a name with a second resubscribing. I think this puts List owners at a disadvantage.

Mailman 3 is totally different from Mailman 2.1 in this respect. Mailman 2.1 had no concept of user. All it knew was addresses subscribed to lists and an address subscribed to one list had no connection to the same address subscribed to another list or being an owner or moderator of a list.

I think this should still be the case for non-registered list members. List owners/moderators are different. They HAVE to be registered users of Postorius/Affinity in order to manage/moderate a list. I wonder what the majority of list members' scenarios are. Is the majority scenario in play today the one where most list members are only subscribed to a single list on a single mailman instance? Or is it the scenario that is in majority use the one where a single list member is subscribed to multiple lists? I think that it is important to find out because it should govern the development process to a certain degree. This single approach of having registered user account with multiple associated email accounts elevates one scenario at the expense of the other. It also assumes/requires that list members have a registered user account and I can tell you from my experience that is not going to happen. The majority of mailman 2 users will not registered with a Mailman 3 admin interface. Some will of course, the majority will not. Maybe the behavior should and will change but it will take years for that to happen. So what I am challenging and questioning here is the approach where such assumptions are being made already that results in the weakening of list owners.

Mailman 3 does have a concept of user and addresses belonging to a user. This complicates things in some ways. In Mailman 2.1 we could have "The Boss <user@example.com" as a member of one list and "Just a Peon <user@example.com>" as a member of another list. In Mailman 3 that is not possible unless the addresses are tweaked in some way to make them different.

Perhaps there is a better approach here to handle single member w/multiple subscriptions that doesn't hurt the ability of List owners to serve the needs of list members who do not have a registered user account and prevents the removal of data when a SINGLE user unsubscribes from a list.

You don't seem to be concerned about the case where I subscribe to a second list with a different display name and am surprised to find my display name changed on the first list, but it's something that I have to consider.

I am not concerned because I think such cases are in the extreme minority. Are you saying that people are now going to be using different real names associated with their other subscriptions as well and that is going to be so needed it warrants this complex system that you have come up with? If someone whats to have a different name associated with a different subscription/email address then I am going to tell them create a new user account and use a good password management program to keep track of their logins. Right now I think this minority use case scenario is negatively impacting other administrative tasks of Mailman, particularly in the removal of user data when someone unsubscribes from a list. Its the use of a square peg being forced into a round hole. Except in this case most of the holes are round but by golly, we are going to use that square peg regardless!

As far as filing or not filing an issue, issues in the GitLab trackers are how we track these things. Threads on mailing lists are appropriate for discussion of issues, but if something is going to get changed or fixed, an issue in the tracker is the way to ensure it doesn't get put aside and forgotten. If this is as important to you as it seems to be, please file the issue.

I will but what is disturbing is that you don't think that the non-removal of user data in cases of unsubscribing is not important or even an issue.

--

Brian Carpenter Harmonylists.com Emwd.com

On Oct 20, 2020, at 8:19 PM, Mark Sapiro <mark@msapiro.net> wrote:

Even in mailman 2.1, while a list admin could go to a user's options page for the list and change things, the "change globally" check boxes only worked for the user, not for the list admin.

That was by design!?

As an admin, that was so irritating.

• Mark

mark@pdc-racing.net | 408-348-2878

So, the issue seems to be, like in the following scenario

We imagine a list server for the domains fruiteaters and meateaters. We have two listadmins, anna (fruit) and carl (meat). They don't share anything. We have a user who subscribes to fruiteaters and meateaters using his address hunger@eatall.com

He wants to show up in the fruiteaters list as fullname wormy wonder, but in the meateaters list as fritz the cat. While he can do so using two different email addresses, he cannot do so using

wormy wonder <hunger@eatall.com> fritz the cat <hunger@eatall.com>

as from: addresses because all requests will be matched to the one email address.

Worse, if Carl would be able to change the name to from wormy wonder to fritz the cat, Anna would see that name change and like to change it back because the user she had known, user has been identified as wormy wonder.

So, if these user properties are stored somewhere not related to the lists, we don't seem to have a chance of display name changes by list admins as long as the data model isn't changed. This doesn't seem to be easy.

So, only some kind of super-Admin should currently be able to change these display names, and these changes would be visible for all lists, and one email address can only have one display name for all lists.

Not optimal for some rare conditions - that's the way it seems to be???

I am a bit late to this thread, but I am going to try to respond to some of the points raised in the thread.

On 10/19/20 7:42 PM, Brian Carpenter wrote:

On 10/19/20 10:23 PM, Mark Sapiro wrote:

...

And the import21 created the user and address records for this user. Does the same thing for a new subscriber as well. So there is no pathway to change a real name that is associated with an email address. None. Zilch.

Yes, this is a limitation of the REST API, the display_name attribute can't be updated. Hence, Postorius can't provide a way for anyone to update the name. I have opened an issue for this1 and I think it should be allowed.

I am admin for a list. Users got subscribed with bulk import during transition from MM2 to MM3. So in a sense I am responsible for adding these to a db. BUT there is no way for me to remove them. Only mailman admin with access to the db can do it manually.

The fact that a user's memberships & User as an entity that owns email address is a design choice that Mailman developers made after some experience of using and administering Mailing lists running on Mailman 2. It was annoying to have to manage several different passwords and accounts to manage membership on same mailing list server.

The policy decision that Mailman would allow a user/address to exist even if there are no subscriptions on it isn't un-resonable in certain environments and maybe it is in some. I don't think anyone here denies that either of those behavior could be an expectation from someone using Mailman. But Mailman is an open source project with decent amount of extensibility via HTTP API & Python plugin system.

If the default policy doesn't match the expectations of some users, I don't think it is difficult to cleanup users/address with 0 subscriptions in a nightly cron script. It is trivial with some basic scripting and we are happy to help out with such a mechanism. I wouldn't be opposed to directly adding a command in Core that lets you clean up all sorts of stuff.

On Wed, Oct 21, 2020, at 10:20 AM, Apollinaris Schöll wrote:

There might be a handful of these.

But the bigger problem is a user can’t be removed entirely.

User can delete account and unsubscribe from all lists. But the email and user name is still kept in the db. Just tested with Affinity List admin can’t do it either.

Front-ends like Affinity should be able to implement such a policy by checking subscriptions of an address on every un-subscription call and delete the user & address if it comes out to be 0. The API is documented here2, but if there are gaps in there like I mentioned above in my email, please open issues so we know about them.

I think the correct action is to remove all records if a email is unsubscribed from all lists and that user has not registered for an account. If a user has registered for an account all information including display name must be removed from that particular list subscription. It should not matter if a user unsubscribes or a list admin unsubscribes. There shouldn’t be any data kept.

Also, like mentioned above, as a list admin, you can request the Site Admin to get the address fully scrubbed or if this is an expectation from all list admins, the Site Admin has the ability to automate such a task. The fact the Mailman doesn't do it by default is because the definition of "correct" action is different.

User should have the ability to delete themselves from the system and if it doesn't wipe everything yet from database, it should be considered a bug in Postorius and be fixed. Postorius track users separately than Core, so the it should make sure to delete in both places and there are appropriate APIs in core to delete Users and Addresses.

Now, the ability in Postorius to edit the display_name. The side effect of the model where User is a Site level thing, makes it impossible to customize user's display_name on a per-list basis, for both list admin and the user themselves. That is a trade-off and it is possible that there were use-cases where that was desirable in Mailman 2, but with the superior subscription management functionality in Postorius, I feel the decision is justified. I respect that some of y'all think that it probably isn't if it doesn't suit you particular use cases.

From the thread, I gather than folks mostly agree that a list admin shouldn't be allowed to edit a user's display name. Site Admins however should be able to edit that, which brings me to next point.

Like Mark brought up, there is definitely a big missing feature of User management in Postorius. This future management page would be for Site Admins (Superusers) to manage Users and have the ability to change pretty much everything about users including their addresses, memberships and preferences and also be able to wipe off the users from all the databases if they want to. If there are more features than that which folks think should be a part of this, please feel free to add to it. As it is hopefully implied from the description, it is a big task and needs substantial chunk of time, part of which this hasn't been picked up yet.

...

On Oct 21, 2020, at 1:43 AM, Jörg Schulz <info@joergschulz.de> wrote:

So, the issue seems to be, like in the following scenario

We imagine a list server for the domains fruiteaters and meateaters. We have two listadmins, anna (fruit) and carl (meat). They don't share anything. We have a user who subscribes to fruiteaters and meateaters using his address hunger@eatall.com

He wants to show up in the fruiteaters list as fullname wormy wonder, but in the meateaters list as fritz the cat. While he can do so using two different email addresses, he cannot do so using

wormy wonder <hunger@eatall.com> fritz the cat <hunger@eatall.com>

as from: addresses because all requests will be matched to the one email address.

Worse, if Carl would be able to change the name to from wormy wonder to fritz the cat, Anna would see that name change and like to change it back because the user she had known, user has been identified as wormy wonder.

So, if these user properties are stored somewhere not related to the lists, we don't seem to have a chance of display name changes by list admins as long as the data model isn't changed. This doesn't seem to be easy.

So, only some kind of super-Admin should currently be able to change these display names, and these changes would be visible for all lists, and one email address can only have one display name for all lists.

Not optimal for some rare conditions - that's the way it seems to be???

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

-- thanks, Abhilash Raj (maxking)

The main point here is not "poor Mailman developers", it's that we need input from users (subscribers too, although list owners are probably most important) about how to design these features. Unfortunately I don't have time or energy to fix the tone which is a little self-pitying, so please read with that main point in mind -- I'm posting because I really want to know what people need, and want.

And the bit about "re-using Mailman 2 scripts" too. I'm not sure how practical it is given the architecture changes, but we'll definitely take a look at requests.

Alex Schuilenburg via Mailman-users writes:

Yes, this is social media territory, but with the EU/UK DPA (data protection act) and record fines being issued,

Sure. It's also a pretty big ask of the Mailman developers. As you are now aware, the architecture of the application is not well-suited to this. If you're worried about paying fines, maybe a contribution of a fraction of the average fine would be good insurance? :-)

But it's not just the architecture. Even the requirements are controversial. I'm on record that it might be a good idea to remove the User (ie, the profile of PII in the database) when the last subscription is deleted, but should that be automatic unless inhibited, should it be an option offered but default to "keep", should it be an operation done only on specific request? In any of those options, since this is a major deletion of presumed unneeded but in some cases valuable data, we should provide a convenient but cautious UI. Should there be a temporary backup of the data in case the user changes their mind? Is that GDPR conformant? Maybe the user could download their profile for later upload? Maybe that could be portable across Mailman 3 sites, with some kind of semi-automated merge UI?

How about the user's archived posts? How about quotations of the user in *other's* archived posts? Just designing this is not easy, but because it might involve changes in the architecture, we do need to design in advance. This is not the kind of change where "move fast, lose user data, and break everything" makes sense.

I'm not saying we can't do it, but we also have our priorities, and limitations. Also, I don't think we can, let alone should, do that design ourselves. Even if it's not reasonable for you to contribute code or money, we do need to know the community's requirements, and because it's often pretty fuzzy stuff, I think it's reasonable to ask *why* a feature is needed -- we may be able to negotiate a change that's easier to code or less fragile in operation if we know what the underlying need is. So members of this list can help a lot by (1) answering the questions I asked above, (2) coming up with more questions (and answering them), and (3) turning the whole thread into concise requests for enhancement on the tracker.

Yes, it would be nice to re-use some of those old mm2 scripts built up over the years on mm3.

Feel free to post the scripts you want to reuse on the tracker, in request-for-enhancement issues. Without an accurate idea of what you want to do, specifically the data the scripts want to access, we'd have to reproduce the whole Mailman 2 API/UI. Some of that would be extremely tedious, such as anything that manipulates list config pickles -- they don't exist any more. But other things might be fairly easy.

On 10/28/20 11:11 AM, Apollinaris Schöll wrote:

Most important is to offer an UI to completely delete users for subscribers and list admins. Currently delete account shows this message "Are you sure you want to delete your account? This will remove your account along with all your subscriptions.” This is not true and needs to be done.

Why do you think it's not true? With current Mailman from the HEADs of the GitLab branches, the only thing not removed is posts from the archives. The Django account and the Mailman core user and address records are all removed.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 10/28/20 1:41 PM, Apollinaris Schöll wrote:

If this has been implemented already then that’s good. I have not tested with latest.

Does this also happen when a user unsubscribes or a list admin unsubscribes?

No, it only happens when a user deletes their account.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 10/28/20 1:42 PM, Brian Carpenter wrote:

This is a bug with Affinity which I have scheduled to be fixed. However I am confused. Data is completely removed from Mailman 3 core when you delete the account via Postorius but not when you unsubscribe from a list?

Why do you keep bringing this up? The current design of Mailman core does not remove the user when the user unsubscribes from a list, even if it is their only subscription. We know you don't like this behavior, but there are reasons for it.

On the other hand, when a user removes their account, the circumstance is different and we do remove the user's information in this case because that's what the user asked for.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On Wed, 2020-10-28 at 15:07 +0900, Stephen J. Turnbull wrote: [...]

Alex Schuilenburg via Mailman-users writes:

...

Yes, this is social media territory, but with the EU/UK DPA (data protection act) and record fines being issued,

Sure. It's also a pretty big ask of the Mailman developers. As you are now aware, the architecture of the application is not well-suited to this. If you're worried about paying fines, maybe a contribution of a fraction of the average fine would be good insurance? :-)

That contribution depends whether I was a large company providing a mailing list service, or an individual supporting breadline charities and a small open source company. I also think myself as technically competant enough to go under the hood and do what I suggested as a one off task, but its not something I would relish doing again - I have other things to spend my time on. Hence the suggestion. Like many others on this list, I tend to volunteer a fair amount of my time.

To address your other questions from an EU/UK perspective:

But it's not just the architecture. Even the requirements are controversial. I'm on record that it might be a good idea to remove the User (ie, the profile of PII in the database) when the last subscription is deleted, but should that be automatic unless inhibited, should it be an option offered but default to "keep", should it be an operation done only on specific request?

Its rare that someone would want their posts and posting history deleted (e.g. the death of a subscriber would mean the closing of their accounts - but you would not want to delete their legacy. In fact many families would prefer it remain.). So I'd leave their account alone when the last subscription is deleted so posts are still attributable.

In any of those options, since this is a major deletion of presumed unneeded but in some cases valuable data, we should provide a convenient but cautious UI. Should there be a temporary backup of the data in case the user changes their mind? Is that GDPR conformant?

No. The requirement is very clear. If you were asked to delete the data, and then later they came back and asked what data you held, you would have to disclose the temporary backup and then would be in breach.

FWIW, the EU courts have already ruled that automated periodic backups is permissible as long as the backups are diminishing. i.e. over time the data will eventually disappear - no need to search and remove from backups (unless restored of course).

Maybe the user could download their profile for later upload? Maybe that could be portable across Mailman 3 sites, with some kind of semi-automated merge UI?

Overkill. If a user's posts/data is to be deleted, that's final. Once its done, there is no recovery other than diminishing backup/disaster which you would not do unless it was disaster recovery. You just need to ensure the request is genuine.

How about the user's archived posts? How about quotations of the user in *other's* archived posts? Just designing this is not easy, but because it might involve changes in the architecture, we do need to design in advance. This is not the kind of change where "move fast, lose user data, and break everything" makes sense.

Actually, its not that tough a task - you have the information available so deleting posts from the archives should not be too hard. to delete the offending posts. I've done this with MM2 under the hood with perl scripts to delete the posts with all mentions in the mbox archive, deleted the web archive and used the cleaned mbox to regenerate the web archive.

Having migrated my company's MM" lists to MM3, and hit a couple of migration issues, I can see that it should still be possible to repeat this, buts its now become harder because of the underlying relational database and its dependencies. Its something MM3 developers need to keep in the back of their mind.

I'm not saying we can't do it, but we also have our priorities, and limitations. Also, I don't think we can, let alone should, do that design ourselves. Even if it's not reasonable for you to contribute code or money, we do need to know the community's requirements, and because it's often pretty fuzzy stuff, I think it's reasonable to ask *why* a feature is needed -- we may be able to negotiate a change that's easier to code or less fragile in operation if we know what the underlying need is. So members of this list can help a lot by (1) answering the questions I asked above, (2) coming up with more questions (and answering them), and (3) turning the whole thread into concise requests for enhancement on the tracker.

A bit more on my perspective: Mailing lists have become somewhat old- school (I'm old school, prefer it, and mailing lists are far more useful) as social media has grown but I prefer lists as most people have email, everyone who uses social media has email, while not everyone uses/likes fakebook/twitter/etc.

Unfortunately, mailing lists can be abused just like social media. Consider for example a self-help group for victims of sexual assult, most preferring to remain anonymous. Then a discussion turns nasty and details which are legally prohibited and morally wrong are published (albeit only to subscribers). Heated discussions ensure and it turns out the original poster was the convicted perpetrator and the victim underage.

Now your average list admin has no power to delete the offending posts this scenario?

• they can only take the list and archives offline(?), effectively killing the list. The site owner/admin has neither the finances nor technical capability to remove the offending posts/thread. Moderators are low/unpaid volunteers, so the list is not moderated. So the whole archives are removed - archives which hold valuable information and shared experiences for silent victims. Who is the winner and loser in

Yes, this is an extreme example, but illustrates why such a feature is needed, not just the ability to hide real email addresses.

Is MM3 a suitable tool to provide a list in the above example? I think it is the best candidate, but needs work. Fakebook certainly not - they would not even respond to appeals from police.

...

Yes, it would be nice to re-use some of those old mm2 scripts built up over the years on mm3.

Feel free to post the scripts you want to reuse on the tracker, in request-for-enhancement issues. Without an accurate idea of what you want to do, specifically the data the scripts want to access, we'd have to reproduce the whole Mailman 2 API/UI. Some of that would be extremely tedious, such as anything that manipulates list config pickles -- they don't exist any more. But other things might be fairly easy.

Most of my scripts operate on the mbox archives, removing attachments, obfiscating email addresses and contact details from certain posts, and regenerating the web archives & index. They wont work in the current architecture of MM3. For the rest, I've already tapped into the backend (sorry, I don't have time for REST etc) to extract the information I need for my new MM3 lists. But I wont be moving my other MM2 lists over to MM3 for a while.

-- Alex

On 29/10/2020 01:57, Brian Carpenter wrote:

With Mailman 3, every list owner has the ability to remove a reply in a thread, a complete thread, and even the entire archives. That is a welcome difference from MM2 in my opinion.

A very welcome difference, only I don't see this option in the web interface of my debian 3.2.1 installation.  Is this a shell only option, 3.3 feature or configuration option I have missed?

-- Alex

On 10/29/20 5:17 AM, Brian Carpenter wrote:

On 10/29/20 4:27 AM, Alex Schuilenburg via Mailman-users wrote:

...

On 29/10/2020 01:57, Brian Carpenter wrote:

...

With Mailman 3, every list owner has the ability to remove a reply in a thread, a complete thread, and even the entire archives. That is a welcome difference from MM2 in my opinion.

A very welcome difference, only I don't see this option in the web interface of my debian 3.2.1 installation.  Is this a shell only option, 3.3 feature or configuration option I have missed?

It is not a shell option. It can be done via Hyperkitty but I am not sure when that feature showed up.

Actually, with current HyperKitty, only site admins can remove messages or threads or the archive. The ability for list owners to do it is <https://gitlab.com/mailman/hyperkitty/-/issues/261>.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

I just wanted to let the community know that we have addressed the issue of user data being retained when they unsubscribe from a single list or delete their Affinity/Empathy registration account. The latter was a bug on our end. In both scenarios, all data is removed from the server when someone unsubscribes from a single list and/or deletes their registration. The only time data is retained is if the user still holds a subscription to one or more lists on the server for obvious reasons.

This is in regard to our Affinity/Empathy UIs. I believe the issue that was raised originally in this thread/post still exists with Postorius/Hyperkitty.

Brian Mailmanhost.com

I agree that deleting, without an explicit request to do so, user-entered information is a violation of the user’s data This applies when the user has set up an account on the system.

If he/she has never created an account, then I see no problem cleaning up the system when the user deletes the last subscription, as any remaining data presumably has been created by the server. Keeping that information on the server without the knowledge of the user is not right.

Yours,

Allan Hansen hansen@rc.org

On Nov 2, 2020, at 19:51 , Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Brian Carpenter writes:

...

This is in regard to our Affinity/Empathy UIs. I believe the issue [about data retention] that was raised originally in this thread/post still exists with Postorius/Hyperkitty.

I don't speak *for* the Mailman team on this, but my *impression* from these conversations is that we have a consensus on the team: deleting the User data (authentication and profile) is a very big deal that should be done only on explicit request, not as an automatic side effect of other changes. We understand that others have a different opinion, but we think that this is a difference of opinion, not a mistake on either side.

My own suggestion is that we *should* provide the *option* to delete the User data, and prompt for it, when that User's last subscription on the server is deleted. Note that there is a race condition where the user deletes the second-to-last subscription and an admin then deletes the last one. The user won't see the option, but the admin will. I can see this as a bad outcome, depending on user's intent, whether the admin deletes the User object or leaves it in place.

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

On 11/3/20 3:15 AM, Stephen J. Turnbull wrote:

Allan Hansen writes:

...

If he/she has never created an account, then I see no problem cleaning up the system when the user deletes the last subscription,

Without an account (ie, a User object), the user can't authenticate themselves as owner of the subscription, and therefore can't delete any subscriptions. What scenario are you thinking of here?

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

If you mean if they don't have a Django user account, they can't unsubscribe? I think that is true if they are wanting to unsubscribe via the web interface. But sending an email to listname-unsubscribe@listdomain allows them to unsubscribe without such user authentication. Isn't that correct?

My intent in our changes is to give the list owner and member exactly what they expect: when they explicitly remove a subscription and/or user profile, that they expect their data to be totally removed. I can easily picture a scenario where an older list has a number of list members move to on to other things in life and even abandon their email accounts totally. In those cases it is imperative that those list addresses are removed either via bounce processing or list owner intervention and that no legacy data on such members remain.

--

Brian Carpenter Harmonylists.com Emwd.com

On 11/4/20 12:04 PM, Stephen J. Turnbull wrote:

Brian Carpenter writes:

...

If you mean if they don't have a Django user account, they can't unsubscribe?

The Django user account is more or less a proxy for the underlying Mailman User object, which is what we're concerned with here because that's where the data you want deleted is stored.

...

I think that is true if they are wanting to unsubscribe via the web interface. But sending an email to listname-unsubscribe@listdomain allows them to unsubscribe without such user authentication. Isn't that correct?

They don't need a Django password or social auth, but they'll still have to do the OTK dance. I don't see a reason to distinguish the methods of authentication. They all reduce to the OTK dance to prove ownership of a mailbox, then optional *delegation* to a password in Django or social auth via Gmail etc.

Please speak in simple terms. I am pretty sure my questions were straightforward but I have a sneaky suspicion you convoluted the issue with these replies. I feel like my questions were not answered.

I understand that your users don't understand this or care to. Thing is, some of *our* users care about the features that this architecture enables.

I think everyone should care that data should be removed when there is an expectation that it is being removed.

...

My intent in our changes is to give the list owner and member exactly what they expect: when they explicitly remove a subscription and/or user profile, that they expect their data to be totally removed.

That's fine, and you're welcome to implement that in Affinity. In fact, as I understand it, you have already done so. *I'm* not saying *nobody* should implement that.

I'm saying that *Mailman* shouldn't, because a lot of subscribers to Mailman lists won't like it. The whole point of Mailman 3 is to cater to users who want powerful control over their subscriptions. For Mailman (Postorius), prompting the user "If you delete this subscription, you will have no subscriptions linked to this account. Would you like to delete the account as well?" (as I have proposed three times now) is the right way to go. This can be implemented via email with a second OTK.

Since the account is a django account, I don't see why unsubscribing from a list has anything to do with that account except removing the user's prefer settings for a list that they are no longer subscribed to. For me I made this issue about those list members who unsubscribe from a single list AND does not have a django user account. This discussion has expanded to all kinds of scenarios.

Those who are the power users are not in play here. I am sure they intend to to keep their subscriptions active and their django account open. I am talking about those who want to leave a list or wants to remove their django user account permanently.

...

I can easily picture a scenario where an older list has a number of list members move to on to other things in life

And I can picture a scenario where they take a summer break and reactivate a decade later. I would be *pissed off* if my data got deleted in that scenario. (Yes, I've done that in the five year variant, if not the full decade.)

Why would you be mad? Am I missing something here? The data that is being kept are a few fields of data (email address, username, password, real-name, and some list preferences). Why would anyone expect to have such inactive accounts kept perpetually? Regardless, I am not talking about removing accounts/subscriptions without a user's permission. I am talking about when a user either does it themselves or asks a List Owner to do so.

...

and even abandon their email accounts totally.

Which is a totally different scenario. I deny your "imperative" in the former case; inactivity from the point of view of a list owner is (usually) not abandonment. I question it in the latter, because "abandon" is an inference drawn from lack of activity or bounces, either of which could be inadvertant (respectively the "summer break" scenario, and separation from an employer).

I worded that poorly. But remember if an email address that no longer exists (because a list member never unsubscribed from a list and closed their email provider account anyways) is removed due to bounces (as it should), that user data still exists. I think that is not good and such a scenario happens a lot.

...

In those cases it is imperative that those list addresses are removed either via bounce processing or list owner intervention and that no legacy data on such members remain.

But how do you propose to identify "legacy data"? In Mailman 3, members are *people*, not addresses. Do you really want it to be the case that if Albert signs up with albert@example.com, and later gets fired by example.com, then all his subscriptions get bounce-cancelled, and his User profile gets irreversibly trashed? Or some large email provider hosting lots of posters decides to suddenly implement some spam protection that causes a ton of bounces (as DMARC p=reject did in 2014) and hundreds of users have their accounts trashed?

It worked for Mailman 2 right? What happened in those cases? They just got resubscribed again. As you said, unsubscribing doesn't impact a django account anyways. So I don't understand your point here. Again I am talking about the scenario where a user is requesting to be removed or does the removal themselves and expects their data to be removed.

You just can't know without asking Albert what he wants done in his case and you can be quite sure you're doing the wrong thing in the DMARC-like case. In either case, resubscribing is a click per list if his data is retained (and Albert needs an OTK confirmation of another address). It's an annoying session of duplicating his configuration (including passwords and social auth links) if not, and probably some months of discovering that a configuration that built up over years wasn't accurately reproduced for some lists.

Apples and oranges here. For public lists and a user who does not have a Affinity/Django account, it is just a click per list. For those who have a django/Affinity account, well there account should still be there since they never asked for it to be removed or did not remove it themselves.

Personally I think I am going to implement some sort of backup generation tool for an Affinity account holder where they can download their profile/list settings in a json or equivalent format before account deletion. Then if they wish to create their account, they can rebuild their profile from that backup.

List owners, of course, can do what they want with their lists. If they want to add automation for their subscribers that simplify the lives of people who have simple needs, that's not something Mailman can, should, or will try to stop. That's *why* I support efforts like Affinity, Empathy, and Harmony.

Steve

I personally intend to get into the "power-user" market. That is the #1 reason for creating my UIs. I have that flexibility.

--

Brian Carpenter Harmonylists.com Emwd.com

On 11/5/20 12:20 PM, Stephen J. Turnbull wrote:

Brian Carpenter writes:

...

I think everyone should care that data should be removed when there is an expectation that it is being removed.

I do care, and I agree that everyone should care.

The difference is that I also think that everyone should care that data *should not* be removed when there is an expectation that it *will be retained*, and, therefore, that the base Mailman we distribute should at least ask before deleting data as a side effect of some other action.

I don't see how I can make it plainer than that, so I'm done here.

Well I am having the same difficulty. *With your statement here, let me ask you this:*

1. If someone subscribes to a public list, are they being notified that a user account is being created (by Mailman 3 core) as part of the subscription process?

2. When someone unsubscribes from mailing list, are they being notified that their data is being retained when they unsubscribe (by Mailman 3 core)?

3. When someone subscribes to a public list, are they being made aware that there is no way, for them, the subscriber, to remove their user data *except* if they also register an account through Postorius and then proceeds to delete that account along with them unsubscribing from a list?

4. Am I correct to assume when someone unsubscribes from all lists, AND deletes their Postorius/Django account, that all data on them is being removed?

So please understand that questions 1-3 are for a scenario of a subscriber that is only subscribing to a SINGLE list and has not registered with Postorius/Django.

Question 4 is dealing with someone who has multiple subscriptions AND is also a registered Postorius/Django user.

*This is what I believe are valid reasons for an expectation that data being retained:*

1. A subscriber who is NOT a Postorius/Django user but has multiple subscriptions to various lists on the same Mailman 3 instance subscribes from a single list. They should expect their user data to be retained because they still have active subscriptions to other lists.

2. A subscriber who IS a registered Postorius/Django user is subscribed to a single list and unsubscribes from that single list but did NOT delete their Postorius/Django account.

3. A combination of 1&2.

All I am requesting is that for single list subscribers who are not registered users of Postorius/Django, to have their user data totally removed when they unsubscribe. I simply fail to see how someone could claim a right to data retention when they unsubscribe from the one list that they were subscribed to.

This is important to me because almost all of the list members of the lists that I host fall into this category: single list member who is not a registered user of Affinity/Postorius.

If you need help implementing the "delete everything" policy in Affinity because of some insufficiency in the REST API, let me know and I will make sure the REST API allows you to do that. But I'm pretty sure it's already possible.

It has already been done with Affinity and the REST API was sufficient to allow us to do that. Thanks for offering.

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

-- Brian Carpenter Harmonylists.com Emwd.com

My 2 cents:

1. I like the concept of an account held by the subscriber and that subscriptions are managed by the subscriber via that account.
2. The account should not necessarily go away if the subscriber unsubscribes from all lists. It should be optional, but it must go away if requested.
3. The subscriber should have the option to delete the account, which will automatically unsubscribe from all lists.
4. The account should have an account name and a password.
5. The subscriber should not necessarily be able to change the account name.
6. The subscriber should be able to change password, email address, subscriptions, options and all.
7. The moderator must be able to see list membership lists and subscribe/unsubscribe members.

What’s missing:

2., 3. : The subscriber must have full control of the account. Privacy concerns and privacy law compliance makes this necessary. 6.: The subscriber must be able to easily change email address for selected or all lists without moderator/admin assistance. 7.: Basic part of moderator duties because the moderator approves the subscription requests.

Yours,

Allan

On Oct 20, 2020, at 9:13 PM, Mark Sapiro <mark@msapiro.net> wrote:

On 10/20/20 8:53 PM, Mark Dadgar wrote:

...

On Oct 20, 2020, at 8:19 PM, Mark Sapiro <mark@msapiro.net> wrote:

...

Even in mailman 2.1, while a list admin could go to a user's options page for the list and change things, the "change globally" check boxes only worked for the user, not for the list admin.

That was by design!?

As an admin, that was so irritating.

Yes, it was by design. Do you really think that list owners should be able to enable or disable mail delivery, set digest format, user's password, etc. for a user's subscription on lists that they don't own?

Yeah, that makes sense for very large installations. Good point.

IIRC, mm2 had the concept of a site-admin. It would be nice if it had worked for them, though.

• Mark

mark@pdc-racing.net | 408-348-2878

On 10/20/20 6:54 AM, Brian Carpenter wrote:

On 10/19/20 11:32 PM, Mark Sapiro wrote:

...

Mailman 3 is totally different from Mailman 2.1 in this respect. Mailman 2.1 had no concept of user. All it knew was addresses subscribed to lists and an address subscribed to one list had no connection to the same address subscribed to another list or being an owner or moderator of a list.

I think this should still be the case for non-registered list members. List owners/moderators are different. They HAVE to be registered users of Postorius/Affinity in order to manage/moderate a list.

Here's a thought. For Affinity/Empathy at least, how about using Mailman core's user as the Affinity registered user. I.e., if someone's first interaction is to register with Affinity, have that registration create a core user to keep the Affinity registration data. This core user already accommodates a display_name and password and email address(es).

If the person already is a core user when registering for Affinity, just update the core user as needed, probably only a password reset or just setting it if it is currently None.

Then you don't have the issue of another, separate registration for Affinity.

This would be more difficult for Postorius/Hyperkitty because those users are Django users, but that isn't an issue for Affinity.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 10/23/20 9:37 AM, Brian Carpenter wrote:

How would a person already be a core user outside of a registration with Affinity? Via a list subscription or the import21 script?

Yes, or even just by sending a post to a list as a nonmember.

Everyone who is or has been a member, non-member, owner or moderator of a list has a user and one or more address records in Mailman core.

I do want to apologize for my tone Mark. I was wrong in the way I communicated.

Apology accepted. I understand that it can be frustrating to be dependent on something you don't control and that this can sometimes lead to "anomalous" behavior.

Here's a take on this that is intended to be humorous, but is actually quite frightening. https://xkcd.com/2347/

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

On 10/23/20 1:13 PM, Brian Carpenter wrote:

On 10/23/20 1:04 PM, Mark Sapiro wrote:

...

Everyone who is or has been a member, non-member, owner or moderator of a list has a user and one or more address records in Mailman core.

And those records are kept in the database? Is that correct?

Yes, the user record is in the user table and the address record(s) are in the address table. You can query these and create users and addresses via REST. You can also PATCH user attributes via REST, but not (yet) address attributes.

...

Here's a take on this that is intended to be humorous, but is actually quite frightening. https://xkcd.com/2347/

I actually came across that cartoon months ago and that one person reminded me of you. So the fact that you are sharing that cartoon is ironic and very funny!

Yes, quite.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

Victoriano Giralt writes:

El vie, 23-10-2020 a las 08:19 -0700, Mark Sapiro escribió:

...

This would be more difficult for Postorius/Hyperkitty because those users are Django users, but that isn't an issue for Affinity.

All of a sudden, an idea has dawned on me ... Wouldn't it be possible for Postorius/Hyperkitty, both Django applications, to use Mailman core user model as their User model? It is just a setting for Django (AUTH_USER_MODEL).

I find the "just a setting" claim a bit implausible. I suspect a certain amount of replumbing of Postorius and HyperKitty will be necessary to get it to work.

Most important, the "AUTH_" is pretty scary. Mailman core has *no* authentication built in. It is assumed to be firewalled off from the rest of the world with only access from Postorius and HyperKitty permitted. Can you expand on how hard it would be to do this? Remember that some Mailman instances are either big targets (eg, Apple) or require good security for content reasons (lists for doctors' patient support groups, domestic violence support groups).

This is not a great authn/authz model in the modern Internet, but as far as I can see changing it is going to be quite a lot of work, and none of us are authentication specialists (although Abhilash did study security, I don't think he specialized in authentication tech) -- providing reasonably reliable and secure authentication services, especially "social auth" (OpenAuth, OpenID etc) is an important reason we went with Django in the first place.

Steve

On 10/20/20 12:29 AM, Stephen J. Turnbull wrote:

Brian Carpenter writes:

...

You don't really think that this is an issue which means it will be years before it is addressed.

You're making the classic mistake of reporting an issue assuming that your *proposed *fix* is the *issue*. Mark replies that the immediate issue is not that a database record of the user is retained, it's that there's no way for an authenticated user or even an authenticated list admin to change that user's display name in the database. It's a database, there *is* a way to change it in Mailman, it's "just" that it requires site admin intervention. (Assume "just" is marked with the nonexistent sarcasm emoji.)

I reported a serious problem. The reply was to bring up assumptions that should not be made such as a list member is actually a registered user. As you said they are not. List members are treated differently but I think they are not. Regardless of whether they are just a list member or a list member that is also a registered user, there is NO MECHANISM in place where you can change their associated display name AND their data is not entirely removed when a reasonable assumption is being made that it has, such as list unsubscribing. Sorry but I can't see how this behavior can be justified and also not being made a high priority to fix.

It's arguable that there should *also* be a way for a person to delete their user entirely (in fact, some interpretation of GDPR say that "it's the law" :-). But that's not the immediate issue faced by *this* user and list admin, and I doubt deleting the user object is a big issue for users (but it might be for admins because of GDPR etc).

Whether it gets addressed this week or sometime in the next few months or in the summer is something you're more than welcome to advocate. Like Mark, I recommend filing an issue.

Steve

Some may think that retaining data when there is a reasonable assumption that it is being removed is immoral and not just a violation of GDPR. As I said in my reply to Mark, it is a big deal for list owners (admins) because they are the ones that are going to be interacting with Postorius/Affinity the most. Most list members, especially those migrating from MM2, will not be registering with Postorius/Affinity. List owners/moderators have no choice.

--

Brian Carpenter Harmonylists.com Emwd.com

just created an issue on gitlab https://gitlab.com/mailman/postorius/-/issues/451 regarding name changes, member deletions. Maybe this adds some coolness to this discussion.