[PATCH v4] OSSTEST: introduce a raisin build test

[PATCH v4] OSSTEST: introduce a raisin build test

[Stefano Stabellini]

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

---

Changes in v4:
- introduce enable_raisin in mfi-common: only build raisin when building
xen-unstable
- start off from the default raisin config, then append osstest config
options to it
- do not write variable to the raisin config if the conrresponding
runvar is not set
- remove TREE_OVMF and TREE_SEABIOS

Changes in v3:
- use $raisindir throughout ts-raisin-build
- do not specify ENABLED_COMPONENTS so that empty REVISION variables can
be used to disable building a raisin component

Changes in v2:
- set revision_* variables in mfi-common;
- in ts-raisin-build set the *_REVISION config options based on the
  revision_* variables;
- in ts-raisin-build, call store_revision appropriately;
- divide the output in an hypervisor and a tools tarball.
---
 ap-common       |    3 +
 mfi-common      |   36 +++++++++++
 sg-run-job      |    5 ++
 ts-raisin-build |  186 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 230 insertions(+)
 create mode 100755 ts-raisin-build

diff --git a/ap-common b/ap-common
index 64749e3..f8c02a7 100644
--- a/ap-common
+++ b/ap-common
@@ -47,6 +47,9 @@
 # rumpsrc-related runvars needed only for old rumpuser-xen
 # (ie ones which need $bodges=1 in ts-rumpuserxen-build)
 
+: ${TREE_RAISIN:=git://xenbits.xen.org/people/sstabellini/raisin.git}
+: ${DEFAULT_REVISION_RAISIN:=master}
+
 : ${TREE_SEABIOS_UPSTREAM:=git://git.seabios.org/seabios.git}
 : ${PUSH_TREE_SEABIOS:=$XENBITS:/home/xen/git/osstest/seabios.git}
 : ${BASE_TREE_SEABIOS:=git://xenbits.xen.org/osstest/seabios.git}
diff --git a/mfi-common b/mfi-common
index 16fc8c5..eebad34 100644
--- a/mfi-common
+++ b/mfi-common
@@ -148,6 +148,17 @@ create_build_jobs () {
     *) enable_ovmf=true;
     esac
 
+    case "$xenbranch" in
+    xen-3.*-testing) enable_raisin=false;;
+    xen-4.0-testing) enable_raisin=false;;
+    xen-4.1-testing) enable_raisin=false;;
+    xen-4.2-testing) enable_raisin=false;;
+    xen-4.3-testing) enable_raisin=false;;
+    xen-4.4-testing) enable_raisin=false;;
+    xen-4.5-testing) enable_raisin=false;;
+    *)               enable_raisin=true;
+    esac
+
     eval "
         arch_runvars=\"\$ARCH_RUNVARS_$arch\"
     "
@@ -215,6 +226,31 @@ create_build_jobs () {
 
     fi
 
+    if [ "x$REVISION_RAISIN" != xdisable ] && [ "$enable_raisin" = "true" ]; then
+
+    ./cs-job-create $flight build-$arch-raisin build-raisin                  \
+                arch=$arch                                                   \
+        tree_xen=$TREE_XEN                                                   \
+                $RUNVARS $BUILD_RUNVARS $BUILD_RAISIN_RUNVARS $arch_runvars \
+                $suite_runvars                                               \
+                host_hostflags=$build_hostflags                              \
+                buildjob=${bfi}build-$arch                                   \
+                tree_raisin=$TREE_RAISIN                                     \
+                tree_qemuu=$TREE_QEMU_UPSTREAM                               \
+                tree_qemu=$TREE_QEMU                                         \
+                tree_seabios=$TREE_SEABIOS                                   \
+                tree_libvirt=$TREE_LIBVIRT                                   \
+                tree_ovmf=$TREE_OVMF                                         \
+                revision_xen=$REVISION_XEN                                   \
+                revision_qemu=$REVISION_QEMU                                 \
+                revision_qemuu=$REVISION_QEMU_UPSTREAM                       \
+                revision_seabios=$REVISION_SEABIOS                           \
+                revision_ovmf=$REVISION_OVMF                                 \
+                revision_libvirt=$REVISION_LIBVIRT                           \
+                revision_raisin=${REVISION_RAISIN:-${DEFAULT_REVISION_RAISIN}}\
+
+    fi
+
     if branch_wants_rumpkernel_tests; then
 
     case $arch in
diff --git a/sg-run-job b/sg-run-job
index eae159d..449118d 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -346,6 +346,7 @@ proc need-hosts/build {} { return BUILD }
 proc need-hosts/build-kern {} { return BUILD }
 proc need-hosts/build-libvirt {} { return BUILD }
 proc need-hosts/build-rumpuserxen {} { return BUILD }
+proc need-hosts/build-raisin {} { return BUILD }
 
 proc run-job/build {} {
     run-ts . = ts-xen-build
@@ -364,6 +365,10 @@ proc run-job/build-rumpuserxen {} {
     run-ts . = ts-xen-build + host tools
 }
 
+proc run-job/build-raisin {} {
+    run-ts . = ts-raisin-build
+}
+
 proc prepare-build-host {} {
     global jobinfo
     run-ts broken = ts-hosts-allocate + host
diff --git a/ts-raisin-build b/ts-raisin-build
new file mode 100755
index 0000000..ab84498
--- /dev/null
+++ b/ts-raisin-build
@@ -0,0 +1,186 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+selectbuildhost(\@ARGV);
+# remaining arguments are passed as targets to "make"
+builddirsprops();
+
+my $raisindir="$builddir/raisin";
+
+sub checkout () {
+    prepbuilddirs();
+    build_clone($ho, 'raisin', $builddir, 'raisin');
+
+    target_cmd_build($ho, 100, $builddir, <<END.
+        cd $raisindir
+        cp defconfig config
+    >>config
+
+    echo >>config MAKE=\\"make $makeflags\\"
+    echo >>config PREFIX=\\"/usr\\"
+    echo >>config DESTDIR=dist
+    echo >>config ENABLED_COMPONENTS=\\"seabios ovmf xen qemu qemu_traditional libvirt\\"
+END
+               (nonempty($r{tree_xen}) ? <<END : '').
+    echo >>config XEN_URL=\\"$r{tree_xen}\\"
+END
+               (nonempty($r{tree_qemuu}) ? <<END : '').
+    echo >>config QEMU_URL=\\"$r{tree_qemuu}\\"
+END
+               (nonempty($r{tree_qemu}) ? <<END : '').
+    echo >>config QEMU_TRADITIONAL_URL=\\"$r{tree_qemu}\\"
+END
+               (nonempty($r{tree_seabios}) ? <<END : '').
+    echo >>config SEABIOS_URL=\\"$r{tree_seabios}\\"
+END
+               (nonempty($r{tree_libvirt}) ? <<END : '').
+    echo >>config LIBVIRT_URL=\\"$r{tree_libvirt}\\"
+END
+               (nonempty($r{tree_ovmf}) ? <<END : '').
+    echo >>config OVMF_URL=\\"$r{tree_ovmf}\\"
+END
+
+               (nonempty($r{revision_xen}) ? <<END : '').
+    echo >>config XEN_REVISION=\\"$r{revision_xen}\\"
+END
+               (nonempty($r{revision_qemuu}) ? <<END : '').
+    echo >>config QEMU_REVISION=\\"$r{revision_qemuu}\\"
+END
+               (nonempty($r{revision_qemu}) ? <<END : '').
+    echo >>config QEMU_TRADITIONAL_REVISION=\\"$r{revision_qemu}\\"
+END
+               (nonempty($r{revision_seabios}) ? <<END : '').
+    echo >>config SEABIOS_REVISION=\\"$r{revision_seabios}\\"
+END
+               (nonempty($r{revision_libvirt}) ? <<END : '').
+    echo >>config LIBVIRT_REVISION=\\"$r{revision_libvirt}\\"
+END
+               (nonempty($r{revision_ovmf}) ? <<END : '')
+    echo >>config OVMF_REVISION=\\"$r{revision_ovmf}\\"
+END
+               );
+}
+
+sub build () {
+    target_cmd_root($ho, <<END);
+        cd $raisindir
+        ./raise -y install-builddep
+END
+#/;
+    target_cmd_build($ho, 9000, $builddir, <<END);
+        cd $raisindir
+        ./raise -n build
+END
+#/;
+}
+
+sub checkoutput () {
+    target_cmd_build($ho, 100, $builddir, <<END);
+            cd $raisindir/dist
+            ls boot/xen.gz
+            ls usr/sbin/xenstored
+            ls usr/sbin/xenconsoled
+            ls usr/lib/libxenlight.so
+            ls usr/sbin/xl
+            ls usr/lib/xen/boot/hvmloader
+            ls usr/lib/xen/bin/qemu-dm
+            ls usr/lib/xen/bin/qemu-system-i386
+            ls usr/sbin/libvirtd
+END
+#/;
+}
+
+sub collectversions () {
+    store_revision($ho, 'xen', "$raisindir/xen-dir", 1);
+    store_revision($ho, 'qemu', "$raisindir/qemu-traditional-dir", 1);
+    store_revision($ho, 'qemuu', "$raisindir/qemu-dir", 1);
+    store_revision($ho, 'seabios', "$raisindir/seabios-dir", 1);
+    store_revision($ho, 'ovmf', "$raisindir/ovmf-dir", 1);
+    store_revision($ho, 'libvirt', "$raisindir/libvirt-dir", 1);
+}
+
+sub divide () {
+    # Only move hv to xeninstall, so that we can have
+    # xenpolicy in tools tarball.
+    #
+    # The files inside boot/ after `make dist' are
+    # xen-$XEN_VERSION: Xen binary
+    # xen.gz/xen: symlink to xen-$XEN_VERSION
+    # xen-$MAJOR: symlink to xen-$XEN_VERSION
+    # xen-$MAJOR.$MINOR: symlink to xen-$XEN_VERSION
+    # xen-sym-$XEN_VERSION: Xen symbol
+    # xenpolicy-$XEN_VERSION: flask policy binary if xsm is enabled
+    #
+    # So the following snippet will leave xenpolicy* in
+    # install/boot and get packaged to tools tarball.
+    target_cmd_build($ho, 100, $builddir, <<END);
+        cd raisin
+        mkdir xendist
+        for f in *dist; do
+            mkdir -p \$f/lib
+        done
+        if test -d dist/boot; then
+            if test -f dist/boot/xen.gz || test -f dist/boot/xen; then
+                mkdir xendist/boot
+                mvfiles=`find dist/boot -name 'xen[a-z]*' -prune -o -name 'xen*' -print`
+                mv \$mvfiles xendist/boot/.
+            fi
+        fi
+END
+}
+
+sub stash () {
+    foreach my $part ('', 'xen') {
+        built_stash($ho, $builddir,
+                    "raisin/${part}dist",
+                    "${part}dist");
+    }
+    built_stash_file($ho, $builddir, "xen-syms", "raisin/xen-dir/xen/xen-syms", 1);
+    built_stash_file($ho, $builddir, "raisin-config", "raisin/config", 1);
+    built_stash_file($ho, $builddir, "seabios-config",
+             "raisin/seabios-dir/.config", 1);
+    built_compress_stashed("xen-syms");
+}
+
+
+our @probs;
+
+sub trapping ($) {
+    my ($sub) = @_;
+    my $tok= eval { $sub->(); 1; };
+    if (!$tok) { push @probs, $@; print STDERR "failure (trapped): $@\n"; }
+}
+
+checkout();
+trapping(\&build);
+trapping(\&checkoutput);
+trapping(\&collectversions);
+divide();
+stash();
+
+die "*** something failed:\n\n".(join "\n\n",@probs)."\n** something failed"
+    if @probs;
+
-- 
1.7.10.4

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Tue, 2015-05-12 at 10:20 +0100, Stefano Stabellini wrote:
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> 
> ---
> 
> Changes in v4:
> - introduce enable_raisin in mfi-common: only build raisin when building
> xen-unstable
> - start off from the default raisin config, then append osstest config
> options to it
> - do not write variable to the raisin config if the conrresponding
> runvar is not set
> - remove TREE_OVMF and TREE_SEABIOS
> 
> Changes in v3:
> - use $raisindir throughout ts-raisin-build
> - do not specify ENABLED_COMPONENTS so that empty REVISION variables can
> be used to disable building a raisin component
> 
> Changes in v2:
> - set revision_* variables in mfi-common;
> - in ts-raisin-build set the *_REVISION config options based on the
>   revision_* variables;
> - in ts-raisin-build, call store_revision appropriately;
> - divide the output in an hypervisor and a tools tarball.
> ---
>  ap-common       |    3 +
>  mfi-common      |   36 +++++++++++
>  sg-run-job      |    5 ++
>  ts-raisin-build |  186 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 230 insertions(+)
>  create mode 100755 ts-raisin-build
> 
> diff --git a/ap-common b/ap-common
> index 64749e3..f8c02a7 100644
> --- a/ap-common
> +++ b/ap-common
> @@ -47,6 +47,9 @@
>  # rumpsrc-related runvars needed only for old rumpuser-xen
>  # (ie ones which need $bodges=1 in ts-rumpuserxen-build)
>  
> +: ${TREE_RAISIN:=git://xenbits.xen.org/people/sstabellini/raisin.git}
> +: ${DEFAULT_REVISION_RAISIN:=master}

If master is what you get if you just do git clone URL then I think you
can omit this and its usage in mfi-common, build_clone will see the
empty revision_xen and accept the default from the clone.

Otherwise I think this script, together with the corresponding raisin
change does the right thing.

The next step towards having this actually used by the automated tests
would be to add cases for raisin to ap-fetch-version and
ap-fetch-version-old. Those should return the "new version of raisin to
be tested" and "the already tested version of raisin for other branches
to use" respectively.

Then you need to add some code to cr-daily-branch which calls
"determine_version REVISION_RAISIN raisin RAISIN" if REVISION_RAISIN is
not set, e.g. like it does for libvirt.

That should be enough to get xen-unstable flights doing something useful
with this new test case, I think.

You can more closely emulate the automated environment by using
cr-daily-branch instead of make-flight directly. The easiest way is to
use the helper script. e.g.

        ./standalone make-flight xen-unstable

Use -f NAME to name the flight something other than "standalone", which
can be useful if you want to generate and compare multiple flights.

Have you seen ./mg-show-flight-runvars <FLIGHT_NAME>? It shows you the
result of make-flight.

A few thoughts:

We currently have build jobs for both normal and the XSM case.
Duplicating the raisin jobs is also going to duplicate the build for
everything else it builds, which might be wasteful?

More generally this now ties everything together into one build job.
Currently while bisecting osstest can try and reuse e.g. the xen job
with a different libvirt build, which saves some time. I think this is
an unavoidable consequence of how raisin is designed to work and we'll
just have to suck it up.

The only alternative I can see would be for raisin to have an
"incremental" mode where it can be pointed at the results of a previous
raisin build and build additional components based on that. I suspect
you don't want this complexity in raisin though.

> diff --git a/mfi-common b/mfi-common
> index 16fc8c5..eebad34 100644
> --- a/mfi-common
> +++ b/mfi-common
> @@ -148,6 +148,17 @@ create_build_jobs () {
>      *) enable_ovmf=true;
>      esac
>  
> +    case "$xenbranch" in
> +    xen-3.*-testing) enable_raisin=false;;
> +    xen-4.0-testing) enable_raisin=false;;
> +    xen-4.1-testing) enable_raisin=false;;
> +    xen-4.2-testing) enable_raisin=false;;
> +    xen-4.3-testing) enable_raisin=false;;
> +    xen-4.4-testing) enable_raisin=false;;
> +    xen-4.5-testing) enable_raisin=false;;
> +    *)               enable_raisin=true;

Hr, $xenbranch==xen-unstable for a bunch of other flights too, e.g. ovmf
and seabios.

Eventually this is what we would want, but for now might it be better to
test $branch so we test exactly with the xen-unstable flight?

($branch == the actual thing under test, $xenbranch == the version of
Xen to use when testing $branch, which may differ if $branch is not a
Xen of some sort)

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH v4] OSSTEST: introduce a raisin build test"):
> More generally this now ties everything together into one build job.

I hadn't spotted that.

I think this is very undesirable.

> Currently while bisecting osstest can try and reuse e.g. the xen job
> with a different libvirt build, which saves some time. I think this is
> an unavoidable consequence of how raisin is designed to work and we'll
> just have to suck it up.

It's not just that.  It also means that failures to build different
components all get smashed into the same test step failure.  So the
osstest reports and the information in the database will lose the
knowledge of whilch component failed to build.

> The only alternative I can see would be for raisin to have an
> "incremental" mode where it can be pointed at the results of a previous
> raisin build and build additional components based on that. I suspect
> you don't want this complexity in raisin though.

If raisin doesn't want to support building individual components based
on the outputs of previous builds, then I think we need to have
osstest continue to build things the way it currently does, at least
for the main tests.

It's fine of course to have an additional test to check that raisin
works.

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Stefano Stabellini]

On Tue, 12 May 2015, Ian Campbell wrote:
> On Tue, 2015-05-12 at 10:20 +0100, Stefano Stabellini wrote:
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > 
> > ---
> > 
> > Changes in v4:
> > - introduce enable_raisin in mfi-common: only build raisin when building
> > xen-unstable
> > - start off from the default raisin config, then append osstest config
> > options to it
> > - do not write variable to the raisin config if the conrresponding
> > runvar is not set
> > - remove TREE_OVMF and TREE_SEABIOS
> > 
> > Changes in v3:
> > - use $raisindir throughout ts-raisin-build
> > - do not specify ENABLED_COMPONENTS so that empty REVISION variables can
> > be used to disable building a raisin component
> > 
> > Changes in v2:
> > - set revision_* variables in mfi-common;
> > - in ts-raisin-build set the *_REVISION config options based on the
> >   revision_* variables;
> > - in ts-raisin-build, call store_revision appropriately;
> > - divide the output in an hypervisor and a tools tarball.
> > ---
> >  ap-common       |    3 +
> >  mfi-common      |   36 +++++++++++
> >  sg-run-job      |    5 ++
> >  ts-raisin-build |  186 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 230 insertions(+)
> >  create mode 100755 ts-raisin-build
> > 
> > diff --git a/ap-common b/ap-common
> > index 64749e3..f8c02a7 100644
> > --- a/ap-common
> > +++ b/ap-common
> > @@ -47,6 +47,9 @@
> >  # rumpsrc-related runvars needed only for old rumpuser-xen
> >  # (ie ones which need $bodges=1 in ts-rumpuserxen-build)
> >  
> > +: ${TREE_RAISIN:=git://xenbits.xen.org/people/sstabellini/raisin.git}
> > +: ${DEFAULT_REVISION_RAISIN:=master}
> 
> If master is what you get if you just do git clone URL then I think you
> can omit this and its usage in mfi-common, build_clone will see the
> empty revision_xen and accept the default from the clone.

OK


> Otherwise I think this script, together with the corresponding raisin
> change does the right thing.
> 
> The next step towards having this actually used by the automated tests
> would be to add cases for raisin to ap-fetch-version and
> ap-fetch-version-old. Those should return the "new version of raisin to
> be tested" and "the already tested version of raisin for other branches
> to use" respectively.
> 
> Then you need to add some code to cr-daily-branch which calls
> "determine_version REVISION_RAISIN raisin RAISIN" if REVISION_RAISIN is
> not set, e.g. like it does for libvirt.
> 
> That should be enough to get xen-unstable flights doing something useful
> with this new test case, I think.
> 
> You can more closely emulate the automated environment by using
> cr-daily-branch instead of make-flight directly. The easiest way is to
> use the helper script. e.g.
> 
>         ./standalone make-flight xen-unstable
> 
> Use -f NAME to name the flight something other than "standalone", which
> can be useful if you want to generate and compare multiple flights.
> 
> Have you seen ./mg-show-flight-runvars <FLIGHT_NAME>? It shows you the
> result of make-flight.
> 
> A few thoughts:
> 
> We currently have build jobs for both normal and the XSM case.
> Duplicating the raisin jobs is also going to duplicate the build for
> everything else it builds, which might be wasteful?

We could disable the raisin build if [ enable_xsm = "true" ] ?


> More generally this now ties everything together into one build job.
> Currently while bisecting osstest can try and reuse e.g. the xen job
> with a different libvirt build, which saves some time. I think this is
> an unavoidable consequence of how raisin is designed to work and we'll
> just have to suck it up.

It is possible to just build xen first and libvirt later, as long as you
reuse the same raisin environment, or you install xen on the system
first.


> The only alternative I can see would be for raisin to have an
> "incremental" mode where it can be pointed at the results of a previous
> raisin build and build additional components based on that. I suspect
> you don't want this complexity in raisin though.

I don't want to support the case where raisin is asked to build a
component based on a previous raisin build at an alternative location.
But building Xen first, then going back to the same raisin directory and
building libvirt later is fine.


> > diff --git a/mfi-common b/mfi-common
> > index 16fc8c5..eebad34 100644
> > --- a/mfi-common
> > +++ b/mfi-common
> > @@ -148,6 +148,17 @@ create_build_jobs () {
> >      *) enable_ovmf=true;
> >      esac
> >  
> > +    case "$xenbranch" in
> > +    xen-3.*-testing) enable_raisin=false;;
> > +    xen-4.0-testing) enable_raisin=false;;
> > +    xen-4.1-testing) enable_raisin=false;;
> > +    xen-4.2-testing) enable_raisin=false;;
> > +    xen-4.3-testing) enable_raisin=false;;
> > +    xen-4.4-testing) enable_raisin=false;;
> > +    xen-4.5-testing) enable_raisin=false;;
> > +    *)               enable_raisin=true;
> 
> Hr, $xenbranch==xen-unstable for a bunch of other flights too, e.g. ovmf
> and seabios.
> 
> Eventually this is what we would want, but for now might it be better to
> test $branch so we test exactly with the xen-unstable flight?
> 
> ($branch == the actual thing under test, $xenbranch == the version of
> Xen to use when testing $branch, which may differ if $branch is not a
> Xen of some sort)

OK

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Tue, 2015-05-12 at 12:20 +0100, Stefano Stabellini wrote:

> > We currently have build jobs for both normal and the XSM case.
> > Duplicating the raisin jobs is also going to duplicate the build for
> > everything else it builds, which might be wasteful?
> 
> We could disable the raisin build if [ enable_xsm = "true" ] ?

That isn't necessary now. I was thinking about the future when we may
want to switch to raisin for building everything (i.e. removing the
current build jobs).

> > More generally this now ties everything together into one build job.
> > Currently while bisecting osstest can try and reuse e.g. the xen job
> > with a different libvirt build, which saves some time. I think this is
> > an unavoidable consequence of how raisin is designed to work and we'll
> > just have to suck it up.
> 
> It is possible to just build xen first and libvirt later, as long as you
> reuse the same raisin environment, or you install xen on the system
> first.

I don't think we can install Xen, since this will be a potentially
shared build host.

> > The only alternative I can see would be for raisin to have an
> > "incremental" mode where it can be pointed at the results of a previous
> > raisin build and build additional components based on that. I suspect
> > you don't want this complexity in raisin though.
> 
> I don't want to support the case where raisin is asked to build a
> component based on a previous raisin build at an alternative location.
> But building Xen first, then going back to the same raisin directory and
> building libvirt later is fine.

Would a separate clone of the same raisin version with some sort of
"dist" directory transported over be sufficient and supportable? Or are
raisin's outputs not in one place and easily transportable?

i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
corresponding build-$ARCH, unpacks them and asks libvirt to build
against that tree.

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Stefano Stabellini]

On Tue, 12 May 2015, Ian Campbell wrote:
> On Tue, 2015-05-12 at 12:20 +0100, Stefano Stabellini wrote:
> 
> > > We currently have build jobs for both normal and the XSM case.
> > > Duplicating the raisin jobs is also going to duplicate the build for
> > > everything else it builds, which might be wasteful?
> > 
> > We could disable the raisin build if [ enable_xsm = "true" ] ?
> 
> That isn't necessary now. I was thinking about the future when we may
> want to switch to raisin for building everything (i.e. removing the
> current build jobs).
> 
> > > More generally this now ties everything together into one build job.
> > > Currently while bisecting osstest can try and reuse e.g. the xen job
> > > with a different libvirt build, which saves some time. I think this is
> > > an unavoidable consequence of how raisin is designed to work and we'll
> > > just have to suck it up.
> > 
> > It is possible to just build xen first and libvirt later, as long as you
> > reuse the same raisin environment, or you install xen on the system
> > first.
> 
> I don't think we can install Xen, since this will be a potentially
> shared build host.
> 
> > > The only alternative I can see would be for raisin to have an
> > > "incremental" mode where it can be pointed at the results of a previous
> > > raisin build and build additional components based on that. I suspect
> > > you don't want this complexity in raisin though.
> > 
> > I don't want to support the case where raisin is asked to build a
> > component based on a previous raisin build at an alternative location.
> > But building Xen first, then going back to the same raisin directory and
> > building libvirt later is fine.
> 
> Would a separate clone of the same raisin version with some sort of
> "dist" directory transported over be sufficient and supportable? Or are
> raisin's outputs not in one place and easily transportable?
>
> i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> corresponding build-$ARCH, unpacks them and asks libvirt to build
> against that tree.

Moving the dist directory over should work, although I have never tested
this configuration.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
> > Would a separate clone of the same raisin version with some sort of
> > "dist" directory transported over be sufficient and supportable? Or are
> > raisin's outputs not in one place and easily transportable?
> >
> > i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> > corresponding build-$ARCH, unpacks them and asks libvirt to build
> > against that tree.
> 
> Moving the dist directory over should work, although I have never tested
> this configuration.

Would you be willing to support this as a requirement going forward?

I assume that it is not also necessary to reclone all the trees for the
preexisting components, just the new ones?

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Stefano Stabellini]

On Wed, 13 May 2015, Ian Campbell wrote:
> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
> > > Would a separate clone of the same raisin version with some sort of
> > > "dist" directory transported over be sufficient and supportable? Or are
> > > raisin's outputs not in one place and easily transportable?
> > >
> > > i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> > > corresponding build-$ARCH, unpacks them and asks libvirt to build
> > > against that tree.
> > 
> > Moving the dist directory over should work, although I have never tested
> > this configuration.
> 
> Would you be willing to support this as a requirement going forward?

Yeah, I think it is OK

> I assume that it is not also necessary to reclone all the trees for the
> preexisting components, just the new ones?

Only if the user asks for a components to be built, the corresponding
tree is cloned.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Wed, 2015-05-13 at 12:48 +0100, Stefano Stabellini wrote:
> On Wed, 13 May 2015, Ian Campbell wrote:
> > On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
> > > > Would a separate clone of the same raisin version with some sort of
> > > > "dist" directory transported over be sufficient and supportable? Or are
> > > > raisin's outputs not in one place and easily transportable?
> > > >
> > > > i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> > > > corresponding build-$ARCH, unpacks them and asks libvirt to build
> > > > against that tree.
> > > 
> > > Moving the dist directory over should work, although I have never tested
> > > this configuration.
> > 
> > Would you be willing to support this as a requirement going forward?
> 
> Yeah, I think it is OK
> 
> > I assume that it is not also necessary to reclone all the trees for the
> > preexisting components, just the new ones?
> 
> Only if the user asks for a components to be built, the corresponding
> tree is cloned.

OK, so the general pattern would be

ts-xen-build:

        Clones raisin
        
        Calls raisin with COMPONENTS="xen seabios qemu qemu-trad"
        # Essentially the current set from xen.git:Config.mk

        Builds xendist.tar and dist.tar from raisin's outputdir
        
ts-foo-build 

        Clones raisin

        Unpacks result of ts-xen-build into $WHERE (and any other
        required components, of which I expect there will be few)
        
        Calls raisin with COMPONENTS="foo"
        
        Builds dist.tar of that 
        
Where the obvious foo right now is libvirt, but eventually would include
grub2 etc. We might also eventually consider moving things out of
ts-xen-build.

I didn't say it, but in both cases there is an implicit "if
$r{use_raisin} else to the old thing fi" there.

The only problem I foresee is that if $WHERE above == raisin's outputdir
then foo's dist.tar would include all of the Xen stuff. How hard would
it be to have those split?

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[George Dunlap]

On Wed, May 13, 2015 at 12:48 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> On Wed, 13 May 2015, Ian Campbell wrote:
>> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
>> > > Would a separate clone of the same raisin version with some sort of
>> > > "dist" directory transported over be sufficient and supportable? Or are
>> > > raisin's outputs not in one place and easily transportable?
>> > >
>> > > i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
>> > > corresponding build-$ARCH, unpacks them and asks libvirt to build
>> > > against that tree.
>> >
>> > Moving the dist directory over should work, although I have never tested
>> > this configuration.
>>
>> Would you be willing to support this as a requirement going forward?
>
> Yeah, I think it is OK
>
>> I assume that it is not also necessary to reclone all the trees for the
>> preexisting components, just the new ones?
>
> Only if the user asks for a components to be built, the corresponding
> tree is cloned.

Won't the problem here be disentangling the stuff installed in dist/
(or whatever it's called) from the things we want to rebuild vs the
things we want to change?

I.e., ideally if you want to build just xen.git, you want dist/ to
contain the output of the previous build of seabios, qemut, qemuu, &c,
but *not* the output of previous xen.git builds (or, ideally, the
output of previous libvirt, pvgrub, or stubdom builds).  Just tar and
untarr'ing dist/ after a full build won't accomplish that.

Would it make sense to do some sort of "save snapshot" functionality
that would tar up the dist/ before building a particular component,
such that it could be used later?  Sort of a "stage 2*" for raisin.
:-)

 -George

* Referring to Gentoo.  Not sure the comparison is 100% accurate.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Mon, 2015-05-18 at 11:08 +0100, George Dunlap wrote:
> On Wed, May 13, 2015 at 12:48 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > On Wed, 13 May 2015, Ian Campbell wrote:
> >> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
> >> > > Would a separate clone of the same raisin version with some sort of
> >> > > "dist" directory transported over be sufficient and supportable? Or are
> >> > > raisin's outputs not in one place and easily transportable?
> >> > >
> >> > > i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> >> > > corresponding build-$ARCH, unpacks them and asks libvirt to build
> >> > > against that tree.
> >> >
> >> > Moving the dist directory over should work, although I have never tested
> >> > this configuration.
> >>
> >> Would you be willing to support this as a requirement going forward?
> >
> > Yeah, I think it is OK
> >
> >> I assume that it is not also necessary to reclone all the trees for the
> >> preexisting components, just the new ones?
> >
> > Only if the user asks for a components to be built, the corresponding
> > tree is cloned.
> 
> Won't the problem here be disentangling the stuff installed in dist/
> (or whatever it's called) from the things we want to rebuild vs the
> things we want to change?

>From the osstest PoV at least the proposal here only involves building
additional things, not rebuilding anything which came from a previous
build.

e.g. given a build of xen.git now do a build of libvirt.git using those
previously built Xen libs.

But there is still the issue of separating stuff built in Pass-A from
the stuff in Pass-B.

Raisin could presumably have a concept of two dist dirs, dist.base and
dist with the former being r/o. But that sounds to me like the sort of
thing you wouldn't want in Raisin.

Per component dist dirs is similarly surely possible but perhaps not
something raisin wants.

> I.e., ideally if you want to build just xen.git, you want dist/ to
> contain the output of the previous build of seabios, qemut, qemuu, &c,
> but *not* the output of previous xen.git builds (or, ideally, the
> output of previous libvirt, pvgrub, or stubdom builds).  Just tar and
> untarr'ing dist/ after a full build won't accomplish that.
> 
> Would it make sense to do some sort of "save snapshot" functionality
> that would tar up the dist/ before building a particular component,
> such that it could be used later?  Sort of a "stage 2*" for raisin.
> :-)
> 
>  -George
> 
> * Referring to Gentoo.  Not sure the comparison is 100% accurate.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[George Dunlap]

On 05/18/2015 11:33 AM, Ian Campbell wrote:
> On Mon, 2015-05-18 at 11:08 +0100, George Dunlap wrote:
>> On Wed, May 13, 2015 at 12:48 PM, Stefano Stabellini
>> <stefano.stabellini@eu.citrix.com> wrote:
>>> On Wed, 13 May 2015, Ian Campbell wrote:
>>>> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
>>>>>> Would a separate clone of the same raisin version with some sort of
>>>>>> "dist" directory transported over be sufficient and supportable? Or are
>>>>>> raisin's outputs not in one place and easily transportable?
>>>>>>
>>>>>> i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
>>>>>> corresponding build-$ARCH, unpacks them and asks libvirt to build
>>>>>> against that tree.
>>>>>
>>>>> Moving the dist directory over should work, although I have never tested
>>>>> this configuration.
>>>>
>>>> Would you be willing to support this as a requirement going forward?
>>>
>>> Yeah, I think it is OK
>>>
>>>> I assume that it is not also necessary to reclone all the trees for the
>>>> preexisting components, just the new ones?
>>>
>>> Only if the user asks for a components to be built, the corresponding
>>> tree is cloned.
>>
>> Won't the problem here be disentangling the stuff installed in dist/
>> (or whatever it's called) from the things we want to rebuild vs the
>> things we want to change?
> 
> From the osstest PoV at least the proposal here only involves building
> additional things, not rebuilding anything which came from a previous
> build.
> 
> e.g. given a build of xen.git now do a build of libvirt.git using those
> previously built Xen libs.

Sure; but what I'm saying is if you do xen-full-build, you'll have a
dist/ which contains:
 * qemut
 * qemuu
 * seabios
 * xen
 * libvirt
 * (&c)

But when you re-build just libvirt, what you want is a dist/ that contains:
 * qemut
 * qemuu
 * seabios
 * xen

Specifically, you want it *not* to contain anything from the previous
libvirt builds.  That's what I'm talking about.

> Per component dist dirs is similarly surely possible but perhaps not
> something raisin wants.

You could in theory have per-component "output" directories, and then a
global "input" directory which was blown away at the beginning of every
raisin build and re-constructed as needed.  That would be the sort of
equivalent of the mock-style RPM build (where the chroot represents the
global "input").

Not sure how well that would work, though.

 -George

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Mon, 2015-05-18 at 11:54 +0100, George Dunlap wrote:
> On 05/18/2015 11:33 AM, Ian Campbell wrote:
> > On Mon, 2015-05-18 at 11:08 +0100, George Dunlap wrote:
> >> On Wed, May 13, 2015 at 12:48 PM, Stefano Stabellini
> >> <stefano.stabellini@eu.citrix.com> wrote:
> >>> On Wed, 13 May 2015, Ian Campbell wrote:
> >>>> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
> >>>>>> Would a separate clone of the same raisin version with some sort of
> >>>>>> "dist" directory transported over be sufficient and supportable? Or are
> >>>>>> raisin's outputs not in one place and easily transportable?
> >>>>>>
> >>>>>> i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> >>>>>> corresponding build-$ARCH, unpacks them and asks libvirt to build
> >>>>>> against that tree.
> >>>>>
> >>>>> Moving the dist directory over should work, although I have never tested
> >>>>> this configuration.
> >>>>
> >>>> Would you be willing to support this as a requirement going forward?
> >>>
> >>> Yeah, I think it is OK
> >>>
> >>>> I assume that it is not also necessary to reclone all the trees for the
> >>>> preexisting components, just the new ones?
> >>>
> >>> Only if the user asks for a components to be built, the corresponding
> >>> tree is cloned.
> >>
> >> Won't the problem here be disentangling the stuff installed in dist/
> >> (or whatever it's called) from the things we want to rebuild vs the
> >> things we want to change?
> > 
> > From the osstest PoV at least the proposal here only involves building
> > additional things, not rebuilding anything which came from a previous
> > build.
> > 
> > e.g. given a build of xen.git now do a build of libvirt.git using those
> > previously built Xen libs.
> 
> Sure; but what I'm saying is if you do xen-full-build, you'll have a
> dist/ which contains:
>  * qemut
>  * qemuu
>  * seabios
>  * xen
>  * libvirt
>  * (&c)
> 
> But when you re-build just libvirt, what you want is a dist/ that contains:
>  * qemut
>  * qemuu
>  * seabios
>  * xen
> 
> Specifically, you want it *not* to contain anything from the previous
> libvirt builds.  That's what I'm talking about.

That's not what I was talking about ;-).

WRT the osstest usage the first build wouldn't be a full build, and in
particular it would exclude the libvirt.

I appreciate there may be reasons to care about the scenario you
presented, but right now I'm trying to figure out how we can best
integrate raisin into osstest and whether it can some how be made
suitable for building actual build artefacts for osstest to use for
further testing, as opposed to just existing as a test case for the sole
purpose of testing that raisin works.

If in solving that we also address your scenario, then great.

> > Per component dist dirs is similarly surely possible but perhaps not
> > something raisin wants.
> 
> You could in theory have per-component "output" directories, and then a
> global "input" directory which was blown away at the beginning of every
> raisin build and re-constructed as needed.  That would be the sort of
> equivalent of the mock-style RPM build (where the chroot represents the
> global "input").
> 
> Not sure how well that would work, though.

In essence everything builds into dist.$component and then at the end of
each component raisin automatically takes that and overlays whatever it
contains over some central dist.all which subsequent components actually
build against? Perhaps with a mode to seed dist.all from dist.* iff
dist.all doesn't exist.

I can see how that might work in general and I think it would solve at
least osstest's desired use case but I don't know enough about raisin
internals to know if it will actually fit in. Lets see what Stefano
says.

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[George Dunlap]

On 05/18/2015 12:21 PM, Ian Campbell wrote:
> On Mon, 2015-05-18 at 11:54 +0100, George Dunlap wrote:
>> On 05/18/2015 11:33 AM, Ian Campbell wrote:
>>> On Mon, 2015-05-18 at 11:08 +0100, George Dunlap wrote:
>>>> On Wed, May 13, 2015 at 12:48 PM, Stefano Stabellini
>>>> <stefano.stabellini@eu.citrix.com> wrote:
>>>>> On Wed, 13 May 2015, Ian Campbell wrote:
>>>>>> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
>>>>>>>> Would a separate clone of the same raisin version with some sort of
>>>>>>>> "dist" directory transported over be sufficient and supportable? Or are
>>>>>>>> raisin's outputs not in one place and easily transportable?
>>>>>>>>
>>>>>>>> i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
>>>>>>>> corresponding build-$ARCH, unpacks them and asks libvirt to build
>>>>>>>> against that tree.
>>>>>>>
>>>>>>> Moving the dist directory over should work, although I have never tested
>>>>>>> this configuration.
>>>>>>
>>>>>> Would you be willing to support this as a requirement going forward?
>>>>>
>>>>> Yeah, I think it is OK
>>>>>
>>>>>> I assume that it is not also necessary to reclone all the trees for the
>>>>>> preexisting components, just the new ones?
>>>>>
>>>>> Only if the user asks for a components to be built, the corresponding
>>>>> tree is cloned.
>>>>
>>>> Won't the problem here be disentangling the stuff installed in dist/
>>>> (or whatever it's called) from the things we want to rebuild vs the
>>>> things we want to change?
>>>
>>> From the osstest PoV at least the proposal here only involves building
>>> additional things, not rebuilding anything which came from a previous
>>> build.
>>>
>>> e.g. given a build of xen.git now do a build of libvirt.git using those
>>> previously built Xen libs.
>>
>> Sure; but what I'm saying is if you do xen-full-build, you'll have a
>> dist/ which contains:
>>  * qemut
>>  * qemuu
>>  * seabios
>>  * xen
>>  * libvirt
>>  * (&c)
>>
>> But when you re-build just libvirt, what you want is a dist/ that contains:
>>  * qemut
>>  * qemuu
>>  * seabios
>>  * xen
>>
>> Specifically, you want it *not* to contain anything from the previous
>> libvirt builds.  That's what I'm talking about.
> 
> That's not what I was talking about ;-).
> 
> WRT the osstest usage the first build wouldn't be a full build, and in
> particular it would exclude the libvirt.
> 
> I appreciate there may be reasons to care about the scenario you
> presented, but right now I'm trying to figure out how we can best
> integrate raisin into osstest and whether it can some how be made
> suitable for building actual build artefacts for osstest to use for
> further testing, as opposed to just existing as a test case for the sole
> purpose of testing that raisin works.

That's what I'm trying to solve too. :-)

So it sounds like we have different ideas about what osstest needs.  I
had assumed that since osstest separately tests the various qemu trees,
seabios, &c, that what we had envisioned for raisin was something
similar: that the "xen.git" raisin test would use pre-built qemuu,
qemut, and seabios components and only build what was in xen.git.

It sounds like you're saying just to have a "base Xen" build that would
build what currently comes out of xen.git, and then base our other
components on top of that.  In which case what you described would
probably work just fine.

>>> Per component dist dirs is similarly surely possible but perhaps not
>>> something raisin wants.
>>
>> You could in theory have per-component "output" directories, and then a
>> global "input" directory which was blown away at the beginning of every
>> raisin build and re-constructed as needed.  That would be the sort of
>> equivalent of the mock-style RPM build (where the chroot represents the
>> global "input").
>>
>> Not sure how well that would work, though.
> 
> In essence everything builds into dist.$component and then at the end of
> each component raisin automatically takes that and overlays whatever it
> contains over some central dist.all which subsequent components actually
> build against? Perhaps with a mode to seed dist.all from dist.* iff
> dist.all doesn't exist.

Yeah, the basic idea would be when you run build, you rm -rf dist.all.
Then if $dependency is in $COMPONENTS, then you clone (if necessary) &
build; if not, you copy from dist.$dependency into dist.all.  If
$dependency is neither in $COMPONENTS nor in dist.$dependency, you throw
an error.

That solves the most general case; but it sounds like you care mostly
about the very specific case of dealing with components that depend on
the current output of xen.git.  Starting simple may be fine.

 -George

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Mon, 2015-05-18 at 14:05 +0100, George Dunlap wrote:
> It sounds like you're saying just to have a "base Xen" build that would
> build what currently comes out of xen.git, and then base our other
> components on top of that.

Correct, otherwise you get one big job which can fail due to any
component, which makes reporting harder. It is nice to spot
ts-libvirt-build vs ts-xen-build failing in the reports rather than just
ts-raisin-build for everything.

More critically one big job makes the bisector less effective since it
has to rebuild things which haven't changed just because something which
shares the monolothic build job has changed. e.g. today a libvirt bisect
won't rebuild Xen if it doesn't have to.

> In which case what you described would probably work just fine.
> 
> >>> Per component dist dirs is similarly surely possible but perhaps not
> >>> something raisin wants.
> >>
> >> You could in theory have per-component "output" directories, and then a
> >> global "input" directory which was blown away at the beginning of every
> >> raisin build and re-constructed as needed.  That would be the sort of
> >> equivalent of the mock-style RPM build (where the chroot represents the
> >> global "input").
> >>
> >> Not sure how well that would work, though.
> > 
> > In essence everything builds into dist.$component and then at the end of
> > each component raisin automatically takes that and overlays whatever it
> > contains over some central dist.all which subsequent components actually
> > build against? Perhaps with a mode to seed dist.all from dist.* iff
> > dist.all doesn't exist.
> 
> Yeah, the basic idea would be when you run build, you rm -rf dist.all.
> Then if $dependency is in $COMPONENTS, then you clone (if necessary) &
> build; if not, you copy from dist.$dependency into dist.all.  If
> $dependency is neither in $COMPONENTS nor in dist.$dependency, you throw
> an error.
> 
> That solves the most general case; but it sounds like you care mostly
> about the very specific case of dealing with components that depend on
> the current output of xen.git.  Starting simple may be fine.

Currently we only have ts-*-build things which depend on the output of
ts-xen-build (in fact, we only have ts-libvirt-build).

I'm not sure if there will be others in the future, I suppose
ts-rump{qemu,xenstore,foo}-build -> ts-rumpkernel-build -> ts-xen-build
might eventually be a possibility...

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[George Dunlap]

On 05/18/2015 02:14 PM, Ian Campbell wrote:
>> That solves the most general case; but it sounds like you care mostly
>> about the very specific case of dealing with components that depend on
>> the current output of xen.git.  Starting simple may be fine.
> 
> Currently we only have ts-*-build things which depend on the output of
> ts-xen-build (in fact, we only have ts-libvirt-build).
> 
> I'm not sure if there will be others in the future, I suppose
> ts-rump{qemu,xenstore,foo}-build -> ts-rumpkernel-build -> ts-xen-build
> might eventually be a possibility...

I guess I was assuming that at some point you might have the following
builds and dependencies (not sure these are all correct):
 ts-seabios-build: [none]
 ts-qemut-build: [none]
 ts-qemuu-build: ts-seabios-build
 ts-xen-build: ts-qemut-build ts-qemuu-build
 ts-libvirt-build: ts-xen-build
 &c

I'm not arguing for this, I'm just trying to explain the problem I was
initially trying to solve. :-)

But as we don't have any tests for seabios and qemu* in isolation, I
guess it doesn't really make sense to treat them separately.

 -George

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Mon, 2015-05-18 at 14:23 +0100, George Dunlap wrote:
> On 05/18/2015 02:14 PM, Ian Campbell wrote:
> >> That solves the most general case; but it sounds like you care mostly
> >> about the very specific case of dealing with components that depend on
> >> the current output of xen.git.  Starting simple may be fine.
> > 
> > Currently we only have ts-*-build things which depend on the output of
> > ts-xen-build (in fact, we only have ts-libvirt-build).
> > 
> > I'm not sure if there will be others in the future, I suppose
> > ts-rump{qemu,xenstore,foo}-build -> ts-rumpkernel-build -> ts-xen-build
> > might eventually be a possibility...
> 
> I guess I was assuming that at some point you might have the following
> builds and dependencies (not sure these are all correct):
>  ts-seabios-build: [none]
>  ts-qemut-build: [none]
>  ts-qemuu-build: ts-seabios-build
>  ts-xen-build: ts-qemut-build ts-qemuu-build
>  ts-libvirt-build: ts-xen-build
>  &c

NB: ts-xen-build depends on ts-seabios-build directly, not via
ts-qemu?-build since the BIOS becomes part of hvmloader not qemu.

I think ts-xen-build <-> ts-qemu?-build is a little circular today in
xen.git (qemu uses libxenctrl), but building ts-xen-build first (i.e.
reversing the deps you have there) is how I would solve that (and I
expect raisin did so).

> I'm not arguing for this, I'm just trying to explain the problem I was
> initially trying to solve. :-)
> 
> But as we don't have any tests for seabios and qemu* in isolation, I
> guess it doesn't really make sense to treat them separately.

It could very well eventually make sense to split up things which used
to be part of the monolithic xen.git build into separate builds. qemuu
would be an ideal candidate, for example.

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH v4] OSSTEST: introduce a raisin build test"):
> On Mon, 2015-05-18 at 14:05 +0100, George Dunlap wrote:
> > That solves the most general case; but it sounds like you care mostly
> > about the very specific case of dealing with components that depend on
> > the current output of xen.git.  Starting simple may be fine.
> 
> Currently we only have ts-*-build things which depend on the output of
> ts-xen-build (in fact, we only have ts-libvirt-build).

That's not true.  We have:

 job                      step               script

 build-amd64              xen-build          ts-xen-build
 build-amd64-rumpuserxen  rumpuserxen-build  ts-rumpuserxen-build
 build-amd64-rumpuserxen  xen-build          ts-xen-build 

where each of the lines in my table, above, uses output from the
previous line.

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Ian Campbell]

On Mon, 2015-05-18 at 14:33 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH v4] OSSTEST: introduce a raisin build test"):
> > On Mon, 2015-05-18 at 14:05 +0100, George Dunlap wrote:
> > > That solves the most general case; but it sounds like you care mostly
> > > about the very specific case of dealing with components that depend on
> > > the current output of xen.git.  Starting simple may be fine.
> > 
> > Currently we only have ts-*-build things which depend on the output of
> > ts-xen-build (in fact, we only have ts-libvirt-build).
> 
> That's not true.  We have:
> 
>  job                      step               script
> 
>  build-amd64              xen-build          ts-xen-build
>  build-amd64-rumpuserxen  rumpuserxen-build  ts-rumpuserxen-build
>  build-amd64-rumpuserxen  xen-build          ts-xen-build 
> 
> where each of the lines in my table, above, uses output from the
> previous line.

Indeed, I even seem to have partially remembered that in the next para
(partially because I had it as in the future...)

Ian.

Re: [PATCH v4] OSSTEST: introduce a raisin build test

[Stefano Stabellini]

On Mon, 18 May 2015, George Dunlap wrote:
> On 05/18/2015 12:21 PM, Ian Campbell wrote:
> > On Mon, 2015-05-18 at 11:54 +0100, George Dunlap wrote:
> >> On 05/18/2015 11:33 AM, Ian Campbell wrote:
> >>> On Mon, 2015-05-18 at 11:08 +0100, George Dunlap wrote:
> >>>> On Wed, May 13, 2015 at 12:48 PM, Stefano Stabellini
> >>>> <stefano.stabellini@eu.citrix.com> wrote:
> >>>>> On Wed, 13 May 2015, Ian Campbell wrote:
> >>>>>> On Tue, 2015-05-12 at 12:46 +0100, Stefano Stabellini wrote:
> >>>>>>>> Would a separate clone of the same raisin version with some sort of
> >>>>>>>> "dist" directory transported over be sufficient and supportable? Or are
> >>>>>>>> raisin's outputs not in one place and easily transportable?
> >>>>>>>>
> >>>>>>>> i.e. today build-$ARCH-libvirt picks up the dist.tar.gz files from the
> >>>>>>>> corresponding build-$ARCH, unpacks them and asks libvirt to build
> >>>>>>>> against that tree.
> >>>>>>>
> >>>>>>> Moving the dist directory over should work, although I have never tested
> >>>>>>> this configuration.
> >>>>>>
> >>>>>> Would you be willing to support this as a requirement going forward?
> >>>>>
> >>>>> Yeah, I think it is OK
> >>>>>
> >>>>>> I assume that it is not also necessary to reclone all the trees for the
> >>>>>> preexisting components, just the new ones?
> >>>>>
> >>>>> Only if the user asks for a components to be built, the corresponding
> >>>>> tree is cloned.
> >>>>
> >>>> Won't the problem here be disentangling the stuff installed in dist/
> >>>> (or whatever it's called) from the things we want to rebuild vs the
> >>>> things we want to change?
> >>>
> >>> From the osstest PoV at least the proposal here only involves building
> >>> additional things, not rebuilding anything which came from a previous
> >>> build.
> >>>
> >>> e.g. given a build of xen.git now do a build of libvirt.git using those
> >>> previously built Xen libs.
> >>
> >> Sure; but what I'm saying is if you do xen-full-build, you'll have a
> >> dist/ which contains:
> >>  * qemut
> >>  * qemuu
> >>  * seabios
> >>  * xen
> >>  * libvirt
> >>  * (&c)
> >>
> >> But when you re-build just libvirt, what you want is a dist/ that contains:
> >>  * qemut
> >>  * qemuu
> >>  * seabios
> >>  * xen
> >>
> >> Specifically, you want it *not* to contain anything from the previous
> >> libvirt builds.  That's what I'm talking about.
> > 
> > That's not what I was talking about ;-).
> > 
> > WRT the osstest usage the first build wouldn't be a full build, and in
> > particular it would exclude the libvirt.
> > 
> > I appreciate there may be reasons to care about the scenario you
> > presented, but right now I'm trying to figure out how we can best
> > integrate raisin into osstest and whether it can some how be made
> > suitable for building actual build artefacts for osstest to use for
> > further testing, as opposed to just existing as a test case for the sole
> > purpose of testing that raisin works.
> 
> That's what I'm trying to solve too. :-)
> 
> So it sounds like we have different ideas about what osstest needs.  I
> had assumed that since osstest separately tests the various qemu trees,
> seabios, &c, that what we had envisioned for raisin was something
> similar: that the "xen.git" raisin test would use pre-built qemuu,
> qemut, and seabios components and only build what was in xen.git.
> 
> It sounds like you're saying just to have a "base Xen" build that would
> build what currently comes out of xen.git, and then base our other
> components on top of that.  In which case what you described would
> probably work just fine.
> 
> >>> Per component dist dirs is similarly surely possible but perhaps not
> >>> something raisin wants.
> >>
> >> You could in theory have per-component "output" directories, and then a
> >> global "input" directory which was blown away at the beginning of every
> >> raisin build and re-constructed as needed.  That would be the sort of
> >> equivalent of the mock-style RPM build (where the chroot represents the
> >> global "input").
> >>
> >> Not sure how well that would work, though.
> > 
> > In essence everything builds into dist.$component and then at the end of
> > each component raisin automatically takes that and overlays whatever it
> > contains over some central dist.all which subsequent components actually
> > build against? Perhaps with a mode to seed dist.all from dist.* iff
> > dist.all doesn't exist.
> 
> Yeah, the basic idea would be when you run build, you rm -rf dist.all.
> Then if $dependency is in $COMPONENTS, then you clone (if necessary) &
> build; if not, you copy from dist.$dependency into dist.all.  If
> $dependency is neither in $COMPONENTS nor in dist.$dependency, you throw
> an error.
> 
> That solves the most general case; but it sounds like you care mostly
> about the very specific case of dealing with components that depend on
> the current output of xen.git.  Starting simple may be fine.

Yeah, I would keep things simple and let OSSTest do its tar & scp thing
with the dist directory. Of course raisin would have to consider it a
valid usage scenario. Each new feature comes with a cost in complexity,
I don't think the net benefit would be worthwhile in this case.