All,

This disconnect between Mailman Core and Django and Postorius is, by far, the weakest and, for subscribers, most confusing aspect of MM3 (once it has been installed, which is extremely hard and confusing).

In a true account-centric system, you set up a web account and from there manage your emails and, for each email, your subscriptions. Subscription options would be hierarchical: account/email/list, with lower levels inheriting/overriding settings from/the level above.

That’s really what I had expected when I pushed for us to move our 50 email lists to MM3 and I was very excited about the prospect, as it would solve a lot of system/subscriber management issues I had managing MM2.

Alas, I was very surprised that that’s not how MM3 works. Some things got harder, even, such as the inability to change subscription addresses, which should have been a showstopper from the start! I still hope that MMxx OOTB will get there, but I guess it’ll be along haul. Would we be talking about MM4 to be released 20 years from now?

Some dedicated and skilled developers are currently investing heavily in a replacement for Postorius, but I really believe that any work is better spent transforming Mailman Core etc. into a true account-based system. Then, and only then, will it make sense to build new and better interfaces on top, as, without a healthy core, such interface will currently may have to inherit the confusion and disconnect that currently is so aggravating.

The above is not meant to aggravate the MM3 development team, for which I have utmost respect and gratitude, and which I admire for their tireless support given over decades to the community of admins. Software development is devilishly hard and time consuming.

Yours,

Allan Hansen hansen@rc.org

Begin forwarded message:

From: Mark Sapiro <mark@msapiro.net> Subject: [MM3-users] Re: Removing a mail addresses and users Date: June 10, 2020 at 12:46:01 PDT To: mailman-users@mailman3.org

On 6/10/20 10:23 AM, Abhijith PA via Mailman-users wrote:

...

Hello.

I tried to remove random email address signing ups (which I signed up while testing) via admin panel. After removing I tried to sign up again, but I am getting,

'A user is already registered with this e-mail address'

You removed the user's subscription from the list, but you didn't remove the user from the system.

Mailman3 has a concept of user which didn't exist in Mailman 2.1. User's have addresses and a user or one of the user's addresses can belong to a list with one or more roles (non-member, member, moderator or owner)

...

Combing through the database, I found auth_user table in mailman3web db still contain those mail address. How to wipe email address entirely from the db so I can signup again without the 'forget password' method

There are a couple of things. If you log in to the web UI as a django superuser, you can go to the django admin UI -> Users section and delete the User, but I think that will only delete the web user and not delete the user from Mailman core.

You can also log in to the web UI as the user and go to <https://lists.mailman3.org/user-profile/delete>, but that again only deletes the web user, and if you could do that, you could just subscribe once you're logged in, so I assume that's what you want to avoid.

I don't think there is a way for a user as opposed to a site admin to delete her user record from Mailman core and even for a site admin, there's no way in Postorius to delete a user from core. It has to be done via REST <https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/users.html#deleting-users-via-the-api> or mailman shell.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

Mark Sapiro writes:

Then the people who developed the web based management UI (Postorius) and archive UI (HyperKitty) chose to develop those within a Django framework and Django has its own concept of User separate from Mailman Core and that is where the disconnect occurs.

It's not that Mailman Core lacks what you want. It's that Django doesn't use it.

I think that's mostly right, in terms of the features that users miss. However, as far as I know, Mailman core does lack facilities for identification, authentication, and authorization of connections to the REST API. And that means that the front ends have to handle this. I would guess that's why the web interfaces are built around Django user authentication.

I think it would be possible to have somewhat tighter integration between the Django "web users" and the Mailman core User objects, but it's not necessarily going to be trivial.

I see that Abhilash is pretty optimistic, but I fear this this is going to be a long-tail situation where we're going to be seeing core user vs. web-gui user integration issues in 2030 (maybe by then only 1 every 450 days ;-). I have some ideas, maybe in a couple weeks I can sketch them out.

Steve

On Sun, Jun 14, 2020, at 12:45 PM, Allan Hansen wrote:

All,

Thank you for your thoughtful responses to my call for a removal of the core vs. Django disconnect. From your responses it appears that my suggestion my issues were caused by the Mailman Core was not based in reality. Instead, the issue is how Postorius interacts with the Core. My apologies for overreaching, as the cause of my issues is not an issue to me. Keeping Core a simple list manager is fine, if Postorius is easy to use and does the expected.

As a Subscriber, you are able to do switch emails in subscriptions from your options page. The URL to which is displayed in the List's Summary page when you are logged in. Yes, it requires an approval if the list's settings are set to moderate but that is going to be fixed, see this issue1. It is quite simple IMO to fix this one, if someone wants to take this up.

Well, it’s more than that. Based on the current setup, I have asked my subscribers who want to change their subscription addresses to: a. Create an account if you do not already have one. b. Go to the user profile page. c. Select ‘E-mail Addresses’. d. Add the new e-mail to the user profile. e. Wait for the verification notice and verify the new address. e1. If it does not show send email to hansen@rc.org <mailto:hansen@rc.org> to get the email verified manually. f. Sign in to the account again and to the user profile (or refresh the page if not signed out). g. Select the new address as the primary address. h. Click on ‘Manage Lists’ i. For each list you are subscribed to: i1. Select the list i2. Click ‘List Options Page.’ i3. Pull down the ’Select Email’ menu. i4 Select your new email. i5. Press ‘Change email used for subscription. i6. When the moderator contacts you, explain that you are just changing your email. i6.1. If the moderator is late, send a reminder or send email to hansen@rc.org <mailto:hansen@rc.org> to bypass the moderator.

So, a-f is something we still always want to keep since we aren't going to deal with un-verified email addresses.

Now, Primary Address is something interesting in Mailman 3, which wasn't utilized until now in Postorius. I recently added a new feature to Postorius which allows users to subscribe via their Primary Address and it switches the delivery address when you switch your primary address. You can play with how it looks at https://lists.mailman3.org (Mailman instance serving this list) where you will see "Primary Address (myprimaryemail@example.com)" as an option when subscribing/switching address.

This is still somewhat new feature and there are gaps to it, but it will grow over time. For example, as as admin, you can only subscribe a User's address and not their Primary Address (from Mass Subscription page, or through the CLI commands).

As a User, you are able to switch to using your primary address and then every time you switch your primary address, all your subscription shifts to that new address without any extra steps.

Also, I just created a fix for i61, so switching email addresses won't require approvals anymore. I am still debating if this should be configurable by list owners.

Most people choke on these instructions, so I have unfortunately resorted to just asking them to subscribe the new address and forget about the old. This is not good for our reputation when the old addresses start bouncing. So, in more detail:

I would like to ask my subscribers who want to change their address to: a. Create an account if you did not already have one. b. Go to the user profile page. c. Select ‘E-mail Addresses’. d. Select an existing address e-mail listed the user profile. e. Press new button ‘Change address’. f. When page refreshes, enter the new e-mail address. g. Press ‘Apply.' d. Wait for the verification notice. d1. If it does not show send email to hansen@rc.org <mailto:hansen@rc.org> to get the email verified manually.

In other words, in addition to ‘Make Primary’ and ‘Re-send Verification’ and ‘Remove’, another button says: ‘Change Address’ or’ ‘Replace Address’ or some such.

This button brings up another page that looks the same, but instead of ‘Add E-mail Adress’ is says : Enter New E-mail Address’ and instead of ‘Add E-mail’ is says: ‘Apply’. Then the page refreshes back to the original page, but the new address is now replacing the old address. Optionally, it can show the old address still with a ‘pending change’ until verification.

I am still thinking about a few implementation details of this one, I've created an issue2

I am thinking we could minimize changes by retaining the same workflow for adding and verifying new address, but add a new workflow of some sort to switch subscriptions to a new address with a single button click. This should simplify the Loop of going to each option page and switching addresses. Finally, the old address can be removed.

Another train of thought is that when you Delete your email address, you get an option to switch all the subscriptions on that address to some other address. I could do this by adding a page after you click on "Remove" button. Although, this would restrict the ability to switch subscriptions by only deleting the original one, which I don't want.

Maybe I'll end up implementing both of these.

What do others think about this?

Further, when the new address is verified, all lists get updated with the new address replacing the old address and all settings associated with the old address are now associated with the new address. No moderator or admin involvement needed (other than d1).

The fix1 I mentioned above should already resolve this issue.

You are able to sign in via web and manage your email and subscriptions. The preferences also work exactly how you described above where the lower level override the default and/or upper level settings.

To some extent that is true, Abhilash, and I appreciate that. The above (address change) is a missing piece in that picture and I look forward to seeing it added, if you have time.

Yours,

Allan Hansen hansen@rc.org

...

On Jun 14, 2020, at 0:19 , Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Mark Sapiro writes:

...

Then the people who developed the web based management UI (Postorius) and archive UI (HyperKitty) chose to develop those within a Django framework and Django has its own concept of User separate from Mailman Core and that is where the disconnect occurs.

It's not that Mailman Core lacks what you want. It's that Django doesn't use it.

I think that's mostly right, in terms of the features that users miss. However, as far as I know, Mailman core does lack facilities for identification, authentication, and authorization of connections to the REST API. And that means that the front ends have to handle this. I would guess that's why the web interfaces are built around Django user authentication.

I think it would be possible to have somewhat tighter integration between the Django "web users" and the Mailman core User objects, but it's not necessarily going to be trivial.

I see that Abhilash is pretty optimistic, but I fear this this is going to be a long-tail situation where we're going to be seeing core user vs. web-gui user integration issues in 2030 (maybe by then only 1 every 450 days ;-). I have some ideas, maybe in a couple weeks I can sketch them out.

Steve

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

-- thanks, Abhilash Raj (maxking)

On Wed, Jun 10, 2020, at 3:33 PM, Allan Hansen wrote:

All,

This disconnect between Mailman Core and Django and Postorius is, by far, the weakest and, for subscribers, most confusing aspect of MM3 (once it has been installed, which is extremely hard and confusing).

The disconnect was intentional to make sure that we are fully able to leverage the power of Django's ability to run multiple dis-connected applications in a single Site and Account system. And Mailman Core was intentionally designed to not be an account management system so it can be kept simple.

The disconnect however has caused more confusion than has helped, which we acknowledge. But this disconnect doesn't exist all that much right now, barring any bugs. We synchronize most of the state from Postorius down to Mailman Core including, any additional addresses that you add, primary address etc. Display Names don't trigger an update in Core, but it isn't that hard to do if you open a ticket.

In a true account-centric system, you set up a web account and from there manage your emails and, for each email, your subscriptions. Subscription options would be hierarchical: account/email/list, with lower levels inheriting/overriding settings from/the level above.

You are able to sign in via web and manage your email and subscriptions. The preferences also work exactly how you described above where the lower level override the default and/or upper level settings.

That’s really what I had expected when I pushed for us to move our 50 email lists to MM3 and I was very excited about the prospect, as it would solve a lot of system/subscriber management issues I had managing MM2.

Alas, I was very surprised that that’s not how MM3 works. Some things got harder, even, such as the inability to change subscription addresses, which should have been a showstopper from the start! I still hope that MMxx OOTB will get there, but I guess it’ll be along haul. Would we be talking about MM4 to be released 20 years from now?

As a Subscriber, you are able to do switch emails in subscriptions from your options page. The URL to which is displayed in the List's Summary page when you are logged in. Yes, it requires an approval if the list's settings are set to moderate but that is going to be fixed, see this issue1. It is quite simple IMO to fix this one, if someone wants to take this up.

Are you asking about switching emails as an administrator for your subscribers?

Some dedicated and skilled developers are currently investing heavily in a replacement for Postorius, but I really believe that any work is better spent transforming Mailman Core etc. into a true account-based system. Then, and only then, will it make sense to build new and better interfaces on top, as, without a healthy core, such interface will currently may have to inherit the confusion and disconnect that currently is so aggravating.

I don't know if we want to add account management to Core, but User management is missing from Postorius, that is true. We punted on it and delegated that part to Django's admin interface. Long long time ago, I remember seeing that there was a User's tab for managing users, but that has since changed I think.

If you and some others on this list would like to propose an RFC here2, it would be a great help for me. What I am basically looking for is what kind of "User management" are administrators looking for. It doesn't have to be too detailed but a single line example would be:

• As a Admin, I would like to delete a User completely from my system, including all their addresses and subscription
  • Also, it would be good to delete their account but retain their subscriptions
• As a Admin, I would like to update User's attributes in the system like:
  • Manually verify their email address
  • Manually update their Name
• As a User, I would like to delete my own account and all the subscriptions.

Maybe this could be a starting point, but something like this can be added to Postorius.

As for the question in this thread, you are able to delete the User from Django's admin page and that would delete all the related EmailAddress for them. Deleting an Email address doesn't delete the User, since the relationship exists that User owns Emails. There is currently no way to delete all the subscriptions for a User though.

The above is not meant to aggravate the MM3 development team, for which I have utmost respect and gratitude, and which I admire for their tireless support given over decades to the community of admins. Software development is devilishly hard and time consuming.

And thank you for posting what you think is an important enhancement to Postorius. We don't have a very strict process to choose what new features to work on, so email like this do help us prioritize. Other users could also help by upvoting or just adding "+1" to the bugs/feature enhancements that affects them the most so we can fix them sooner rather than later. I would be more than happy if more people participated in just being able to prioritize features, all the work really happens in the open on Gitlab.

As administrators of some 'open source' lists, the needs of us as admins are somewhat different from what would be in a closed organization and ability to kick someone completely out of the system is something never I had to do since folks are supposed to manage their own subscriptions. We do ban people from sending emails if they've been spamming, but that is a different thing, they are still able to login into Postorius AFAIK.

Yours,

Allan Hansen hansen@rc.org

...

Begin forwarded message:

From: Mark Sapiro <mark@msapiro.net> Subject: [MM3-users] Re: Removing a mail addresses and users Date: June 10, 2020 at 12:46:01 PDT To: mailman-users@mailman3.org

On 6/10/20 10:23 AM, Abhijith PA via Mailman-users wrote:

...

Hello.

I tried to remove random email address signing ups (which I signed up while testing) via admin panel. After removing I tried to sign up again, but I am getting,

'A user is already registered with this e-mail address'

You removed the user's subscription from the list, but you didn't remove the user from the system.

Mailman3 has a concept of user which didn't exist in Mailman 2.1. User's have addresses and a user or one of the user's addresses can belong to a list with one or more roles (non-member, member, moderator or owner)

...

Combing through the database, I found auth_user table in mailman3web db still contain those mail address. How to wipe email address entirely from the db so I can signup again without the 'forget password' method

There are a couple of things. If you log in to the web UI as a django superuser, you can go to the django admin UI -> Users section and delete the User, but I think that will only delete the web user and not delete the user from Mailman core.

You can also log in to the web UI as the user and go to <https://lists.mailman3.org/user-profile/delete>, but that again only deletes the web user, and if you could do that, you could just subscribe once you're logged in, so I assume that's what you want to avoid.

I don't think there is a way for a user as opposed to a site admin to delete her user record from Mailman core and even for a site admin, there's no way in Postorius to delete a user from core. It has to be done via REST <https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/rest/docs/users.html#deleting-users-via-the-api> or mailman shell.

-- Mark Sapiro <mark@msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

---

Mailman-users mailing list -- mailman-users@mailman3.org To unsubscribe send an email to mailman-users-leave@mailman3.org https://lists.mailman3.org/mailman3/lists/mailman-users.mailman3.org/

-- thanks, Abhilash Raj (maxking)