On Thu, Jul 09, 2020 at 01:28:27PM +0200, Andrea Bolognani wrote:
> On Thu, 2020-07-09 at 11:30 +0100, Daniel P. Berrangé wrote:
> > On Wed, Jul 08, 2020 at 10:46:57PM -0400, Cleber Rosa wrote:
> > > +- name: Installation of basic packages to build QEMU
> > > +  hosts: all
> > > +  vars_files:
> > > +    - vars.yml
> > > +  tasks:
> > > +    - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
> > > +      apt:
> > > +        update_cache: yes
> > > +        # This matches the packages on tests/docker/Dockerfiles/ubuntu1804.docker
> > 
> > I'd be inclined to actually use docker on the custom runners.
> > 
> > eg. instead of having separate physical machines or VMs for each
> > (distro, arch) pair, have a single host distro for the arch. Then
> > use docker to provide the build environment against each distro.
> > 
> > IOW, a RHEL-8 aarch64 host, running docker for ubuntu18.04, fedora30
> > etc.
> > 
> > That way we don't end up duplicating all these packages, and instead
> > can use  tests/docker/Dockerfiles/ubuntu1804.docker.  This ensures
> > that if a user needs to reproduce a build failure on their own local
> > aarch64 machine, they can run docker and get the exact same build
> > architecture.
> > 
> > It also has the benefit that we don't need to worry about how to
> > setup gitlab runners for every distro we care about. We only need to
> > do gitlab runner for the standard host distro, which spawns a pristine
> > throwaway docker env.
> > 
> > I appreciate this is a big change from what you've done in this patch
> > though, so don't consider this comment a blocker for initial merge.
> > I think we should do this as the long term strategy though. Essentially
> > for Linux builds, everything should always be container based.
> 
> Agreed. You should be able to set up a fairly minimal environment,
> which consists of Docker, gitlab-runner and not much else, using a
> long-term supported distro such as CentOS and then just schedule
> whatever container build on it. No need to provision a new machine
> every time a new Fedora release comes out, just create a container
> image for it and add it to the mix.
>

Hi Andrea,

There's nothing preventing this from happening, but limiting the
runners to this configuration, prevents a lot more from happening.

> Additionally, the gitlab-runner Docker executor provides more
> isolation than the shell executor, so running untrusted builds
> becomes a more reasonable proposition - this is how the shared
> runners on gitlab.com work - and you don't have to worry about your
> jobs cleaning up properly after themselves nearly as much.
>

I understand and agree to the the benefits of using the gitlab-runner
Docker executor... until you want to run tests on non-Docker
environments :).

Hopefully the explanation on my previous reply to Daniel will also
serve for the points you raised here.  I would have loved to have
worked on a more abstract, container only environments, but that
proved to be unrealistic.

Cheers,
- Cleber.

> -- 
> Andrea Bolognani / Red Hat / Virtualization
>