* [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version
@ 2015-07-24 16:28 Ian Campbell
2015-07-24 16:29 ` Ian Jackson
2015-08-03 16:01 ` Ian Campbell
0 siblings, 2 replies; 5+ messages in thread
From: Ian Campbell @ 2015-07-24 16:28 UTC (permalink / raw)
To: ian.jackson, xen-devel; +Cc: Ian Campbell
There are several steps to this:
- Identify $prevxenbranch, that is the branch which precedes
$xenbranch.
- Create appropriate build jobs.
- Add support in ts-xen-install for overriding {xen,}buildjob on a
per-ident basis
- Add a new receipt test-pair-oneway which only migrates from
src_host to dst_host and not the reverse
- Create appropriate test jobs, overridding the default builds for
src_host.
Currently we only do this for xen* branches and using xl, but in the
future we may wish to add to the libvirt branch too.
It would be nice to try and reuse the builds from the last flight
which tested the $prevxenbranch baseline. I've not dont that here.
+build-amd64-prev arch amd64
+build-amd64-prev build_lvextend_max 50
+build-amd64-prev enable_ovmf true
+build-amd64-prev enable_xend false
+build-amd64-prev enable_xsm false
+build-amd64-prev host_hostflags share-build-wheezy-amd64,arch-amd64,suite-wheezy,purpose-build
+build-amd64-prev revision_ovmf
+build-amd64-prev revision_qemu
+build-amd64-prev revision_qemuu c4a962ec0c61aa9b860a3635c8424472e6c2cc2c
+build-amd64-prev revision_seabios
+build-amd64-prev revision_xen 666b80f239c566283cb1b3435180d99a329d0156
+build-amd64-prev tree_ovmf
+build-amd64-prev tree_qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
+build-amd64-prev tree_qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
+build-amd64-prev tree_seabios
+build-amd64-prev tree_xen git://xenbits.xen.org/xen.git
+build-i386-prev arch i386
+build-i386-prev build_lvextend_max 50
+build-i386-prev enable_ovmf true
+build-i386-prev enable_xend false
+build-i386-prev enable_xsm false
+build-i386-prev host_hostflags share-build-wheezy-i386,arch-i386,suite-wheezy,purpose-build
+build-i386-prev revision_ovmf
+build-i386-prev revision_qemu
+build-i386-prev revision_qemuu c4a962ec0c61aa9b860a3635c8424472e6c2cc2c
+build-i386-prev revision_seabios
+build-i386-prev revision_xen 666b80f239c566283cb1b3435180d99a329d0156
+build-i386-prev tree_ovmf
+build-i386-prev tree_qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
+build-i386-prev tree_qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
+build-i386-prev tree_seabios
+build-i386-prev tree_xen git://xenbits.xen.org/xen.git
+test-amd64-amd64-upgrade all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test,equiv-1
+test-amd64-amd64-upgrade arch amd64
+test-amd64-amd64-upgrade buildjob build-amd64
+test-amd64-amd64-upgrade debian_arch amd64
+test-amd64-amd64-upgrade debian_kernkind pvops
+test-amd64-amd64-upgrade kernbuildjob build-amd64-pvops
+test-amd64-amd64-upgrade kernkind pvops
+test-amd64-amd64-upgrade src_host_buildjob build-amd64-prev
+test-amd64-amd64-upgrade src_host_xenbuildjob build-amd64-prev
+test-amd64-amd64-upgrade toolstack xl
+test-amd64-amd64-upgrade xenbuildjob build-amd64
+test-amd64-i386-upgrade all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test,equiv-1
+test-amd64-i386-upgrade arch i386
+test-amd64-i386-upgrade buildjob build-i386
+test-amd64-i386-upgrade debian_arch i386
+test-amd64-i386-upgrade debian_kernkind pvops
+test-amd64-i386-upgrade kernbuildjob build-i386-pvops
+test-amd64-i386-upgrade kernkind pvops
+test-amd64-i386-upgrade src_host_buildjob build-i386-prev
+test-amd64-i386-upgrade src_host_xenbuildjob build-amd64-prev
+test-amd64-i386-upgrade toolstack xl
+test-amd64-i386-upgrade xenbuildjob build-amd64
NB the regular build jobs are, as expected, unchanged (and different to the
ones above):
build-amd64 revision_xen 9256f66c1606cd9339412bff7fbc7bd9e8beb28c
build-i386 revision_xen 9256f66c1606cd9339412bff7fbc7bd9e8beb28c
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v2:
- Don't do this for Xen 4.0 and earlier
- Update some comments and change some XXX to TODO
- Match branch against xen-*
---
cr-daily-branch | 4 ++++
cri-common | 20 ++++++++++++++++++++
make-flight | 16 ++++++++++++++++
mfi-common | 38 ++++++++++++++++++++++++++++++++++++++
sg-run-job | 14 ++++++++++++--
ts-xen-install | 3 ++-
6 files changed, 92 insertions(+), 3 deletions(-)
diff --git a/cr-daily-branch b/cr-daily-branch
index 1fcfd9d..7402d3f 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -117,6 +117,10 @@ if [ "x$REVISION_XEN" = x ]; then
REVISION_XEN="`fetch_version $xenbranch`"
export REVISION_XEN
fi
+if [ "x$REVISION_PREVXEN" = x -a "x$prevxenbranch" != "x" ]; then
+ REVISION_PREVXEN="`./ap-fetch-version-baseline $prevxenbranch`"
+ export REVISION_PREVXEN
+fi
if [ "x$REVISION_QEMU_UPSTREAM" = x ]; then
case "$xenbranch" in
xen-3.*|xen-4.[01]-*)
diff --git a/cri-common b/cri-common
index 5c58482..94696ab 100644
--- a/cri-common
+++ b/cri-common
@@ -60,6 +60,24 @@ repo_tree_rev_fetch_git () {
git rev-parse $localtag^0
}
+select_prevxenbranch () {
+ local b
+ local p
+ for b in $(./mg-list-all-branches) ; do # already sorted by version
+ case "$b" in
+ xen*)
+ if [ "x$b" = "x$xenbranch" ] ; then
+ break
+ else
+ p=$b
+ fi
+ ;;
+ *) ;;
+ esac
+ done
+ prevxenbranch=$p
+}
+
select_xenbranch () {
case "$branch" in
xen-*) tree=xen; xenbranch=$branch ;;
@@ -83,6 +101,8 @@ select_xenbranch () {
if [ x$qemuubranch = x ]; then
qemuubranch="qemu-upstream-${xenbranch#xen-}"
fi
+
+ select_prevxenbranch
}
select_branch () {
diff --git a/make-flight b/make-flight
index edca13f..7bf5bc8 100755
--- a/make-flight
+++ b/make-flight
@@ -520,6 +520,22 @@ test_matrix_do_one () {
$debian_runvars \
all_hostflags=$most_hostflags,equiv-1
+ if branch_wants_migrupgrade_tests ; then
+ # x86 only for now
+ if [ x$xenarch = xamd64 ] ; then
+ job_create_test test-$xenarch$kern-$dom0arch-migrupgrade test-pair-oneway \
+ $onetoolstack $xenarch $dom0arch \
+ !host !host_hostflags \
+ src_host_buildjob=${prevbfi}build-$dom0arch-prev \
+ src_host_xenbuildjob=${prevbfi}build-$xenarch-prev \
+ $debian_runvars \
+ all_hostflags=$most_hostflags,equiv-1
+ fi
+ # TODO it would be nice to test i386->amd64 tools migration too (for
+ # Xen >= 4.6), but currently $r{arch} is used in several places,
+ # which would need looking at.
+ fi
+
if [ x$test_pvh = xy -a $xenarch = amd64 -a $dom0arch = amd64 ]; then
for cpuvendor in amd intel; do
diff --git a/mfi-common b/mfi-common
index 23f8015..28a39fd 100644
--- a/mfi-common
+++ b/mfi-common
@@ -41,6 +41,15 @@ branch_wants_rumpkernel_tests () {
esac
}
+branch_wants_migrupgrade_tests () {
+ case "$branch" in
+ xen-3.*-testing) return 1 ;;
+ xen-4.0-testing) return 1 ;;
+ xen-*) return 0;;
+ *) return 1 ;;
+ esac
+}
+
xenbranch_xsm_variants () {
case "$xenbranch" in
xen-3.*-testing) echo "false";;
@@ -161,6 +170,14 @@ create_build_jobs () {
*) enable_ovmf=true;
esac
+ want_prevxen=n
+ if branch_wants_migrupgrade_tests ; then
+ # Only x86 for now
+ if [ x$arch = xamd64 -o x$arch = xi386 ] ; then
+ want_prevxen=y
+ fi
+ fi
+
eval "
arch_runvars=\"\$ARCH_RUNVARS_$arch\"
"
@@ -191,6 +208,27 @@ create_build_jobs () {
revision_ovmf=$REVISION_OVMF
done
+ if [ x$want_prevxen = xy ] ; then
+ if [ "x$REVISION_PREVXEN" = x ] ; then
+ echo >&2 "prevxen ?"; exit 1
+ fi
+ # TODO could find latest pass on that branch and attempt to reuse.
+ #bfiprevxen=...
+ #
+ # To avoid the need for TREE_PREVQEMU, TREE_PREVOVMF etc we
+ # only pass tree_xen and revision_xen here and therefore
+ # pickup the versions implicitly selected by the
+ # $REVISION_PREVXEN.
+ ./cs-job-create $flight build-$arch-prev build \
+ arch=$arch enable_xend=false enable_ovmf=$enable_ovmf \
+ enable_xsm=false \
+ tree_xen=$TREE_XEN \
+ $RUNVARS $BUILD_RUNVARS $BUILD_XEN_RUNVARS $arch_runvars \
+ $suite_runvars \
+ host_hostflags=$build_hostflags \
+ revision_xen=$REVISION_PREVXEN
+ fi
+
if [ $build_extraxend = "true" ] ; then
job_create_build build-$arch-xend build \
arch=$arch enable_xend=true enable_ovmf=$enable_ovmf \
diff --git a/sg-run-job b/sg-run-job
index bba52ed..385e937 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -294,13 +294,16 @@ proc run-job/test-debianhvm {} {
test-guest debianhvm
}
-proc need-hosts/test-pair {} { return {src_host dst_host} }
-proc run-job/test-pair {} {
+proc setup-test-pair {} {
run-ts . = ts-debian-install dst_host
run-ts . = ts-debian-fixup dst_host + debian
run-ts . = ts-guests-nbd-mirror + dst_host src_host + debian
per-host-ts . =(*) {ts-leak-check basis}
run-ts . = ts-guest-start + src_host + debian
+}
+proc need-hosts/test-pair {} { return {src_host dst_host} }
+proc run-job/test-pair {} {
+ setup-test-pair
run-ts . = ts-guest-migrate src_host dst_host + debian
run-ts . = ts-guest-migrate dst_host src_host + debian
run-ts . = ts-guest-stop src_host + debian
@@ -310,6 +313,13 @@ proc run-job/test-pair {} {
# run-ts . remus-failover ts-remus-check src_host dst_host + debian
}
+proc need-hosts/test-pair-oneway {} { return {src_host dst_host} }
+proc run-job/test-pair-oneway {} {
+ setup-test-pair
+ run-ts . = ts-guest-migrate src_host dst_host + debian
+ run-ts . = ts-guest-stop dst_host + debian
+}
+
proc test-guest-migr {g} {
if {[catch { run-ts . = ts-migrate-support-check + host $g }]} return
diff --git a/ts-xen-install b/ts-xen-install
index 0f53382..e9e4ada 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -75,7 +75,8 @@ sub extract () {
foreach my $part (@parts) {
target_extract_jobdistpath($ho, $part, "path_${part}dist",
- $r{"${part}buildjob"}, \%distpath);
+ $r{"$ho->{Ident}_${part}buildjob"} // $r{"${part}buildjob"},
+ \%distpath);
}
if (target_file_exists($ho, "/usr/lib64/efi/xen.efi")) {
target_cmd_root($ho,<<END);
--
2.1.4
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version
2015-07-24 16:28 [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version Ian Campbell
@ 2015-07-24 16:29 ` Ian Jackson
2015-08-03 16:01 ` Ian Campbell
1 sibling, 0 replies; 5+ messages in thread
From: Ian Jackson @ 2015-07-24 16:29 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v2] Arrange to test migration from the previous Xen version"):
> There are several steps to this:
>
> - Identify $prevxenbranch, that is the branch which precedes
> $xenbranch.
> - Create appropriate build jobs.
> - Add support in ts-xen-install for overriding {xen,}buildjob on a
> per-ident basis
> - Add a new receipt test-pair-oneway which only migrates from
> src_host to dst_host and not the reverse
> - Create appropriate test jobs, overridding the default builds for
> src_host.
>
> Currently we only do this for xen* branches and using xl, but in the
> future we may wish to add to the libvirt branch too.
>
> It would be nice to try and reuse the builds from the last flight
> which tested the $prevxenbranch baseline. I've not dont that here.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version
2015-07-24 16:28 [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version Ian Campbell
2015-07-24 16:29 ` Ian Jackson
@ 2015-08-03 16:01 ` Ian Campbell
2015-08-11 11:30 ` Ian Campbell
1 sibling, 1 reply; 5+ messages in thread
From: Ian Campbell @ 2015-08-03 16:01 UTC (permalink / raw)
To: ian.jackson, xen-devel
On Fri, 2015-07-24 at 17:28 +0100, Ian Campbell wrote:
> @@ -191,6 +208,27 @@ create_build_jobs () {
> revision_ovmf=$REVISION_OVMF
> done
>
> + if [ x$want_prevxen = xy ] ; then
> + if [ "x$REVISION_PREVXEN" = x ] ; then
> + echo >&2 "prevxen ?"; exit 1
> + fi
This breaks things with standalone mode, or any make-flight which didn't
come from cr-daily-branch.
In such cases we don't have REVISION_XEN or TREE_XEN either, we just get
the defaults.
I think we need to do something like select_prevxenbranch but to pick a
xen.git branch name rather than an osstest branch name.
Or we quietly skip this test if REVISION_PREVXEN is not set.
One to chew on I think.
Ian.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version
2015-08-03 16:01 ` Ian Campbell
@ 2015-08-11 11:30 ` Ian Campbell
2015-08-11 16:16 ` Ian Campbell
0 siblings, 1 reply; 5+ messages in thread
From: Ian Campbell @ 2015-08-11 11:30 UTC (permalink / raw)
To: ian.jackson, xen-devel
On Mon, 2015-08-03 at 17:01 +0100, Ian Campbell wrote:
> On Fri, 2015-07-24 at 17:28 +0100, Ian Campbell wrote:
> > @@ -191,6 +208,27 @@ create_build_jobs () {
> > revision_ovmf=$REVISION_OVMF
> > done
> >
> > + if [ x$want_prevxen = xy ] ; then
> > + if [ "x$REVISION_PREVXEN" = x ] ; then
> > + echo >&2 "prevxen ?"; exit 1
> > + fi
>
> This breaks things with standalone mode, or any make-flight which didn't
> come from cr-daily-branch.
>
> In such cases we don't have REVISION_XEN or TREE_XEN either, we just get
> the defaults.
>
> I think we need to do something like select_prevxenbranch but to pick a
> xen.git branch name rather than an osstest branch name.
>
> Or we quietly skip this test if REVISION_PREVXEN is not set.
>
> One to chew on I think.
At the moment I'm somewhat inclined towards omitting the build-$ARCH-prev
job in this case but still creating the associated test jobs.
In standalone mode this may still be useful (maybe your hosts are already
configured and you want to run an individual step).
In production mode the test jobs will then fail their ts-build-check step,
which correctly reflects what has happened.
I think this is the effect of the following incremental patch.
Ian.
diff --git a/mfi-common b/mfi-common
index 737db99..810e533 100644
--- a/mfi-common
+++ b/mfi-common
@@ -208,10 +208,7 @@ create_build_jobs () {
revision_ovmf=$REVISION_OVMF
done
- if [ x$want_prevxen = xy ] ; then
- if [ "x$REVISION_PREVXEN" = x ] ; then
- echo >&2 "prevxen ?"; exit 1
- fi
+ if [ x$want_prevxen = xy -a "x$REVISION_PREVXEN" != x ] ; then
# TODO could find latest pass on that branch and attempt to reuse.
#bfiprevxen=...
#
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version
2015-08-11 11:30 ` Ian Campbell
@ 2015-08-11 16:16 ` Ian Campbell
0 siblings, 0 replies; 5+ messages in thread
From: Ian Campbell @ 2015-08-11 16:16 UTC (permalink / raw)
To: ian.jackson, xen-devel
On Tue, 2015-08-11 at 12:30 +0100, Ian Campbell wrote:
> On Mon, 2015-08-03 at 17:01 +0100, Ian Campbell wrote:
> > On Fri, 2015-07-24 at 17:28 +0100, Ian Campbell wrote:
> > > @@ -191,6 +208,27 @@ create_build_jobs () {
> > > revision_ovmf=$REVISION_OVMF
> > > done
> > >
> > > + if [ x$want_prevxen = xy ] ; then
> > > + if [ "x$REVISION_PREVXEN" = x ] ; then
> > > + echo >&2 "prevxen ?"; exit 1
> > > + fi
> >
> > This breaks things with standalone mode, or any make-flight which
> > didn't
> > come from cr-daily-branch.
> >
> > In such cases we don't have REVISION_XEN or TREE_XEN either, we just
> > get
> > the defaults.
> >
> > I think we need to do something like select_prevxenbranch but to pick a
> > xen.git branch name rather than an osstest branch name.
> >
> > Or we quietly skip this test if REVISION_PREVXEN is not set.
> >
> > One to chew on I think.
>
> At the moment I'm somewhat inclined towards omitting the build-$ARCH-prev
> job in this case but still creating the associated test jobs.
We discussed this IRL and concluded that build-$ARCH already ignores the
$xenbranch when REVISION_XEN is not specified i.e it always builds whatever
clone gives you, AKA master for xen.git and that having build-$ARCH-prev do
the same was at least no worse than that under those circumstances.
As a future yakk shaving exercise something should be taught to translate
osstest's $xenbranch in to a xen.git branch name (ap-push/ap-fetch-version
has this sort of code already) at which time this can all be fixed up.
I need to rebase over Wei's recent libvirt changes and will drop the 'if [
"x$REVISION_PREVXEN" = x ]' block.
Ian.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-08-11 16:16 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-24 16:28 [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version Ian Campbell
2015-07-24 16:29 ` Ian Jackson
2015-08-03 16:01 ` Ian Campbell
2015-08-11 11:30 ` Ian Campbell
2015-08-11 16:16 ` Ian Campbell
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).