On Jul 22, 2017, at 06:58, tlhackque via Mailman-users <mailman-users@mailman3.org> wrote:
On 21-Jul-17 22:58, Barry Warsaw wrote:
One big project that I’d love to see gain more traction is Lemme, our authenticating REST proxy. We just haven’t had much collective resources to spend working on it.
The architectural problem with a separate proxy that provides separation of privilege is that the proxy has to be kept in sync with the core -
This can be true, as is the case for example in mailmanclient. I think that’s different than the lemme case though because mmclient is a language binding to a REST API, so it needs to know the details of the API in order to provide a high-fidelity binding. This could potentially be mitigated if we had a static representation of the API, and there have been discussions of that elsewhere, so let’s not dive too deeply into that discussion here.
Where I think lemme is different is that it would be a more pure HTTP command forwarding proxy with a bit of preprocessing. It wouldn’t need to know the details of the REST API and wouldn’t need to be explicitly kept in sync.
The general approach we hashed out a few Pycons ago is outlined here: https://gitlab.com/mailman/lemme/blob/master/OUTLINE.rst
In addition, as the recent issues indicate, the REST API already has scaling problems - adding a proxy in the path will exacerbate those.
Yes, the proxy will not be very fast, but it’s also targeting a very different audience.
The current REST API, what I call the “administrative” API needs to be fast (and I’m confident we can make improvements to the existing implementation) because it’s there to serve Postorius, HyperKitty, and any other custom, tightly integrated front-ends. It provides full access to the API with no authorization because it trusts those front-ends implicitly, and those front-ends need the ability to perform any necessary action.
Lemme must fundamentally *not* trust its clients. Authentication is only the first part of its paranoia; it must also authorize the authenticated user and determine whether they have permissions to perform the requested action. It’s the authorization part that makes this a “big project” to me.
Either way, the major work is (or should be) authorization tests in the core, with a touch of mapping an authenticated user to her(his) capabilities (for a list) in the database. Not a s"big" separate project.
I disagree about the bigness, but I can be convinced with code. :)
The core has no authorization or permission information built-in. It does have a user database, but I’ve always considered that to be tightly focused on the data needed to determine posting permissions, not access to the REST API. We also don’t have a model for how users map to API permissions. We’ve always recommended that the admin API be exposed only on trusted IPs (highly recommending localhost), but a proxy has to be exposed on the public internet for it to be useful. That, and because the admin API is default-open and the proxy must be default-closed, we want to segregate any possible vulnerabilities into an optional component. Not every site will want to allow their list operations to be scriptable.
To me, this all points to a separate proxy, developed as an official subproject. There we can experiment with permission models to find what works best and is the most understandable. We can allow others to experiment as well, and sites can opt into or out of running the proxy, or even the official proxy. Any vulnerabilities in the proxy can be fixed quickly, and rolled out on a different release schedule than Core.
Now, we’re careful, I think the bulk of lemme could be written with an eye toward some future integration with Core, although I would strongly recommend running it as a separate HTTP server than the admin API. That way, we can experiment with and develop lemme at its own pace, and once it stabilizes, we could consider bringing it in as a Core feature.
Cheers, -Barry