Re: [PATCH v2] OSSTEST: introduce a raisin build test

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2015-05-07 at 10:31 +0100, Stefano Stabellini wrote:
> On Thu, 7 May 2015, Ian Campbell wrote:
> > On Wed, 2015-05-06 at 17:02 +0100, Stefano Stabellini wrote:
> > > On Wed, 6 May 2015, Ian Campbell wrote:
> > > > On Wed, 2015-05-06 at 15:43 +0100, Stefano Stabellini wrote:
> > > > [...]
> > > > > +    echo >>config ENABLED_COMPONENTS=\\"seabios ovmf xen qemu qemu_traditional libvirt\\"
> > > > [...]
> > > > > +    echo >>config XEN_URL=\\"$r{tree_xen}\\"
> > > > > +    echo >>config QEMU_URL=\\"$r{tree_qemuu}\\"
> > > > > +    echo >>config QEMU_TRADITIONAL_URL=\\"$r{tree_qemu}\\"
> > > > > +    echo >>config SEABIOS_URL=\\"$r{tree_seabios}\\"
> > > > > +    echo >>config LIBVIRT_URL=\\"$r{tree_libvirt}\\"
> > > > > +    echo >>config OVMF_URL=\\"$r{tree_ovmf}\\"
> > > > 
> > > > What will raisin do if one or more of these runvars is not set for some
> > > > reason yet the thing is listed in ENABLED_COMPONENTS?
> > > 
> > > It will fail with an error and quit
> > 
> > Imagine a future version of this test script which has been extended to
> > support some new component, something which we cannot (or don't way to)
> > test with an older version of Xen (perhaps it doesn't build, or relies
> > on some newer Xen version somehow).
> > 
> > In that case we would want osstest to instruct raisin to not build that
> > component, which we would likely do by omitting the component from the
> > runvars I think.
> > 
> > So ENABLED_COMPONENTS needs to be generated too, not just hardcoded. I
> > suppose we could do it in an ad-hoc way every time we add a new
> > component to this base set, but that seems like it would get even uglier
> > than just doing it now.
> 
> I think you have a point there. we could get rid of ENABLED_COMPONENTS
> from the raisin config file and rely on the revision_ variables: raisin
> will not try to build something with a blank revision, when
> ENABLED_COMPONENTS is not missing.

That would work too, yes.

> > The converse is also worth consideration -- what if osstest asks raisin
> > to build something it doesn't know about (imagine e.g. a bisection over
> > a raisin change to add a component).
> 
> That should return an error from raisin. Osstest ought to know that it
> cannot ask raisin to build something is not capable of.

True, I'm not sure if the osstest bisector will currently cope with the
set of trees supported by a component changing, but I suppose it would
have to learn to do so...

Ian.