Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:

> I've introduce an extra Osstest::Toolstack which help to install extra
> package, 

I've commented on this.

> and use ballonning for Dom0, 500MB for Dom0 is definetly not
> enough.

This is for overriding Dom0MemFixed I think?

I think going to ballooning is a bit too far, I think instead the actual
size of dom0 ought to be something which can be overridden via a runvar,
so that the openstack tests can ask for a bigger one.

> The ts-devstack script does prepare a bit more the host, clone devstack,
> then run ./stack.sh, which is a bit like raisin. Once the machine ready,
> the integration test suite from OpenStack, Tempest, is started. Do you
> think those two step should be in separate test, one for devstack, and one
> for Tempest?

I don't really understand the difference between them well enough, but I
would say if they are separate things then different steps would be good
-- since if nothing else it gives a clearer indication in the test
report what had failed.

>   I have not done it, but we could have some smoke test before
> Tempest where osstest tryied to start a guest.

A test can have a dependency on a build job, but I'm not sure about
another test, but that would seem generally useful and would allow your
new test to depend on test-ARCH-ARCH-libvirt.

However I think we don't actually want to do that:

For the openstack branch we will be using known good versions of all the
other branches, so that test-ARCH-ARCH-libvirt will have happened
already (when the libvirt branch got pushed).

For the other branches then we already have loads of tests which might
all fail for the same reason which you could serialise to prevent this,
but it would make the whole flight take much longer and it isn't really
needed.

> Then later, there will be the question of which tree to track, devstack?
> nova? Or don't track any and just test with the master branch from time to
> time.

This is a complicated one, especially for things which don't fit into
the "one tree one branch" model of things...

In terms of gating (which matters to us for regression tracking even if
we don't care about the output of the gate) is it the case that if you
clone $rootthing (whatever that is) and select a given revision of that
you will always get the same thing?

Or is it like raisin where you clone $rootthing and select a revision of
that and it will clone the latest version of everything at that time?

I'm expecting the second one?

I think the important thing is we would want to be testing stuff which
has already gone through openstack testing?

Maybe we want to track particular OStack releases?

Ian.