Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell <ian.campbell@citrix.com>]

On Tue, 2015-05-12 at 10:20 +0100, Stefano Stabellini wrote:
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> 
> ---
> 
> Changes in v4:
> - introduce enable_raisin in mfi-common: only build raisin when building
> xen-unstable
> - start off from the default raisin config, then append osstest config
> options to it
> - do not write variable to the raisin config if the conrresponding
> runvar is not set
> - remove TREE_OVMF and TREE_SEABIOS
> 
> Changes in v3:
> - use $raisindir throughout ts-raisin-build
> - do not specify ENABLED_COMPONENTS so that empty REVISION variables can
> be used to disable building a raisin component
> 
> Changes in v2:
> - set revision_* variables in mfi-common;
> - in ts-raisin-build set the *_REVISION config options based on the
>   revision_* variables;
> - in ts-raisin-build, call store_revision appropriately;
> - divide the output in an hypervisor and a tools tarball.
> ---
>  ap-common       |    3 +
>  mfi-common      |   36 +++++++++++
>  sg-run-job      |    5 ++
>  ts-raisin-build |  186 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 230 insertions(+)
>  create mode 100755 ts-raisin-build
> 
> diff --git a/ap-common b/ap-common
> index 64749e3..f8c02a7 100644
> --- a/ap-common
> +++ b/ap-common
> @@ -47,6 +47,9 @@
>  # rumpsrc-related runvars needed only for old rumpuser-xen
>  # (ie ones which need $bodges=1 in ts-rumpuserxen-build)
>  
> +: ${TREE_RAISIN:=git://xenbits.xen.org/people/sstabellini/raisin.git}
> +: ${DEFAULT_REVISION_RAISIN:=master}

If master is what you get if you just do git clone URL then I think you
can omit this and its usage in mfi-common, build_clone will see the
empty revision_xen and accept the default from the clone.

Otherwise I think this script, together with the corresponding raisin
change does the right thing.

The next step towards having this actually used by the automated tests
would be to add cases for raisin to ap-fetch-version and
ap-fetch-version-old. Those should return the "new version of raisin to
be tested" and "the already tested version of raisin for other branches
to use" respectively.

Then you need to add some code to cr-daily-branch which calls
"determine_version REVISION_RAISIN raisin RAISIN" if REVISION_RAISIN is
not set, e.g. like it does for libvirt.

That should be enough to get xen-unstable flights doing something useful
with this new test case, I think.

You can more closely emulate the automated environment by using
cr-daily-branch instead of make-flight directly. The easiest way is to
use the helper script. e.g.

        ./standalone make-flight xen-unstable

Use -f NAME to name the flight something other than "standalone", which
can be useful if you want to generate and compare multiple flights.

Have you seen ./mg-show-flight-runvars <FLIGHT_NAME>? It shows you the
result of make-flight.

A few thoughts:

We currently have build jobs for both normal and the XSM case.
Duplicating the raisin jobs is also going to duplicate the build for
everything else it builds, which might be wasteful?

More generally this now ties everything together into one build job.
Currently while bisecting osstest can try and reuse e.g. the xen job
with a different libvirt build, which saves some time. I think this is
an unavoidable consequence of how raisin is designed to work and we'll
just have to suck it up.

The only alternative I can see would be for raisin to have an
"incremental" mode where it can be pointed at the results of a previous
raisin build and build additional components based on that. I suspect
you don't want this complexity in raisin though.

> diff --git a/mfi-common b/mfi-common
> index 16fc8c5..eebad34 100644
> --- a/mfi-common
> +++ b/mfi-common
> @@ -148,6 +148,17 @@ create_build_jobs () {
>      *) enable_ovmf=true;
>      esac
>  
> +    case "$xenbranch" in
> +    xen-3.*-testing) enable_raisin=false;;
> +    xen-4.0-testing) enable_raisin=false;;
> +    xen-4.1-testing) enable_raisin=false;;
> +    xen-4.2-testing) enable_raisin=false;;
> +    xen-4.3-testing) enable_raisin=false;;
> +    xen-4.4-testing) enable_raisin=false;;
> +    xen-4.5-testing) enable_raisin=false;;
> +    *)               enable_raisin=true;

Hr, $xenbranch==xen-unstable for a bunch of other flights too, e.g. ovmf
and seabios.

Eventually this is what we would want, but for now might it be better to
test $branch so we test exactly with the xen-unstable flight?

($branch == the actual thing under test, $xenbranch == the version of
Xen to use when testing $branch, which may differ if $branch is not a
Xen of some sort)