On Wed, 2020-10-28 at 15:07 +0900, Stephen J. Turnbull wrote: [...]
Alex Schuilenburg via Mailman-users writes:
Yes, this is social media territory, but with the EU/UK DPA (data protection act) and record fines being issued,
Sure. It's also a pretty big ask of the Mailman developers. As you are now aware, the architecture of the application is not well-suited to this. If you're worried about paying fines, maybe a contribution of a fraction of the average fine would be good insurance? :-)
That contribution depends whether I was a large company providing a mailing list service, or an individual supporting breadline charities and a small open source company. I also think myself as technically competant enough to go under the hood and do what I suggested as a one off task, but its not something I would relish doing again - I have other things to spend my time on. Hence the suggestion. Like many others on this list, I tend to volunteer a fair amount of my time.
To address your other questions from an EU/UK perspective:
But it's not just the architecture. Even the requirements are controversial. I'm on record that it might be a good idea to remove the User (ie, the profile of PII in the database) when the last subscription is deleted, but should that be automatic unless inhibited, should it be an option offered but default to "keep", should it be an operation done only on specific request?
Its rare that someone would want their posts and posting history deleted (e.g. the death of a subscriber would mean the closing of their accounts - but you would not want to delete their legacy. In fact many families would prefer it remain.). So I'd leave their account alone when the last subscription is deleted so posts are still attributable.
In any of those options, since this is a major deletion of presumed unneeded but in some cases valuable data, we should provide a convenient but cautious UI. Should there be a temporary backup of the data in case the user changes their mind? Is that GDPR conformant?
No. The requirement is very clear. If you were asked to delete the data, and then later they came back and asked what data you held, you would have to disclose the temporary backup and then would be in breach.
FWIW, the EU courts have already ruled that automated periodic backups is permissible as long as the backups are diminishing. i.e. over time the data will eventually disappear - no need to search and remove from backups (unless restored of course).
Maybe the user could download their profile for later upload? Maybe that could be portable across Mailman 3 sites, with some kind of semi-automated merge UI?
Overkill. If a user's posts/data is to be deleted, that's final. Once its done, there is no recovery other than diminishing backup/disaster which you would not do unless it was disaster recovery. You just need to ensure the request is genuine.
How about the user's archived posts? How about quotations of the user in *other's* archived posts? Just designing this is not easy, but because it might involve changes in the architecture, we do need to design in advance. This is not the kind of change where "move fast, lose user data, and break everything" makes sense.
Actually, its not that tough a task - you have the information available so deleting posts from the archives should not be too hard. to delete the offending posts. I've done this with MM2 under the hood with perl scripts to delete the posts with all mentions in the mbox archive, deleted the web archive and used the cleaned mbox to regenerate the web archive.
Having migrated my company's MM" lists to MM3, and hit a couple of migration issues, I can see that it should still be possible to repeat this, buts its now become harder because of the underlying relational database and its dependencies. Its something MM3 developers need to keep in the back of their mind.
I'm not saying we can't do it, but we also have our priorities, and limitations. Also, I don't think we can, let alone should, do that design ourselves. Even if it's not reasonable for you to contribute code or money, we do need to know the community's requirements, and because it's often pretty fuzzy stuff, I think it's reasonable to ask *why* a feature is needed -- we may be able to negotiate a change that's easier to code or less fragile in operation if we know what the underlying need is. So members of this list can help a lot by (1) answering the questions I asked above, (2) coming up with more questions (and answering them), and (3) turning the whole thread into concise requests for enhancement on the tracker.
A bit more on my perspective: Mailing lists have become somewhat old- school (I'm old school, prefer it, and mailing lists are far more useful) as social media has grown but I prefer lists as most people have email, everyone who uses social media has email, while not everyone uses/likes fakebook/twitter/etc.
Unfortunately, mailing lists can be abused just like social media. Consider for example a self-help group for victims of sexual assult, most preferring to remain anonymous. Then a discussion turns nasty and details which are legally prohibited and morally wrong are published (albeit only to subscribers). Heated discussions ensure and it turns out the original poster was the convicted perpetrator and the victim underage.
Now your average list admin has no power to delete the offending posts this scenario?
- they can only take the list and archives offline(?), effectively killing the list. The site owner/admin has neither the finances nor technical capability to remove the offending posts/thread. Moderators are low/unpaid volunteers, so the list is not moderated. So the whole archives are removed - archives which hold valuable information and shared experiences for silent victims. Who is the winner and loser in
Yes, this is an extreme example, but illustrates why such a feature is needed, not just the ability to hide real email addresses.
Is MM3 a suitable tool to provide a list in the above example? I think it is the best candidate, but needs work. Fakebook certainly not - they would not even respond to appeals from police.
Yes, it would be nice to re-use some of those old mm2 scripts built up over the years on mm3.
Feel free to post the scripts you want to reuse on the tracker, in request-for-enhancement issues. Without an accurate idea of what you want to do, specifically the data the scripts want to access, we'd have to reproduce the whole Mailman 2 API/UI. Some of that would be extremely tedious, such as anything that manipulates list config pickles -- they don't exist any more. But other things might be fairly easy.
Most of my scripts operate on the mbox archives, removing attachments, obfiscating email addresses and contact details from certain posts, and regenerating the web archives & index. They wont work in the current architecture of MM3. For the rest, I've already tapped into the backend (sorry, I don't have time for REST etc) to extract the information I need for my new MM3 lists. But I wont be moving my other MM2 lists over to MM3 for a while.
-- Alex