On Wed, Nov 8, 2017, at 04:38 AM, Simon Hanna wrote:
On 11/08/2017 12:03 AM, Abhilash Raj wrote:
New images are available on quay.io and, moving forward, the rest of the image builds will also be moved to Quay[4][5]. Since docker deployment is currently the recommended install, with manual installs using the master branches for "experts" I wonder why you are using Quay.
For those who don't know about it, Quay is a container registry offering from CoreOS. The main reason why I chose to move to Quay, from DockerHub that we are currently using, is security.
Some parts of both Dockerhub and Quay are open source and some aren't. There isn't a very clear boundary/transparency in what technologies they use. Both are free to use for open source projects. However, Quay provides an improved security model with RBAC to push images from build server, without having to put my account password on the build server. It also includes a (open source) security scanner for images, which looks for vulnerabilities in packages included in the image so that I can re-build them if one is found.
There are open source registries available, most popular of which I know is provided by Gitlab. However, I feel Quay is better for my current workflow, which involves lots of bash scripts for building and testing images on public CI system for rolling releases.
Given more free cycles, I do intend to have a better build pipeline and possibly use Gitlab's own registry to distribute images.
It looks like it's a non-free system. Why use Quay but refuse to use a non-free translation system like transifex?
These images are built using git-heads *only* if they are passing our test suite and are re-generated weekly. You should be aware that while all these components are tested with their individual test suites, their combination might sometimes not be stable. This will get you updates/bug-fixes much faster :) This contradicts what you said in my merge request for Postorius https://gitlab.com/mailman/postorius/merge_requests/232#note_43388756
You either use released versions, or make sure that the master branches are always compatible. You can't have both with the current development model.
Not exactly, I still believe all our git-heads should be compatible with released (i.e. stable) versions of dependencies, regardless of the fact that we maintain those dependencies.
Also, there is at least one test per-project which tests with git-heads of dependencies we maintain, so that we know if we make any incompatible changes.
I am not sure what in the current development model prevents us to do both?
The story for container images is different than testing with git-heads. We need better integration testing for the entire suite to be stable in conjunction, rather than independently. In fact, containers might be the perfect way to actually do the integration testing.
If this campaign succeeds, here is a road map of what I intend to get done: Sounds great, what is the limit where the campaign succeeds? What will happen to the funds if it doesn't?
- Add Admin Dashboard project from GSoC 2014 (maybe?) Didn't find anything using google. What is that project about?
https://www.google-melange.com/archive/gsoc/2015/orgs/gnu_mailman/projects/b...
- Add better testing of container images and provide deployment instructions for Kubernetes & Docker Swarm To be honest, I don't think we should all in on containers. Yes they are nice, but I guess most people would prefer distro packages. Even if people want containers, I for instance would prefer plain systemd-nspawn
Like Barry mentioned, it doesn't have to be all-or-nothing. There is work being done on distro packaging for Debian and I am ready to help others trying to do the same for other distros!
Systemd-nspawn isn't really a production grade tool to use containers. I am not sure about how much it adheres to the OCI specs, but if it does, it should not matter which deployment tool you are using with images.
I wouldn't mind adding configurations for systemd-nspawn in the documentation, contributions are most welcome.
- Improve the container images to work with new micro-services architecture, to achieve scaling and redundancy in services. The current bottleneck is the rest api from core. IMO bulk actions, filtering and sorting should be added there before trying to have multiple postorius/hyperkitty instances serving the same core...
I didn't say they will be served from the same core ;-)
Yes, the Core's API can be further improved to be more efficient. But that doesn't prevent us from scaling. I understand there are issues around multiple instances of containers, but I have ideas to get around most of them.
-- Abhilash Raj maxking@asynchronous.in