[OSSTEST PATCH 0/6] make-distros-flight runvar fixes

[OSSTEST PATCH 0/6] make-distros-flight runvar fixes

[Ian Jackson]

I hope this series will fix the Cambridge osstest instance's failure
to build anything in its distros-debian-* tests.  I'm going to push it
shortly.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[OSSTEST PATCH 1/6] mg-list-all-branches: avoid mistakenly generating `.' in the output

[Ian Jackson]

The regex in mg-list-all-branches assumes that the BRANCHES= will
either be a singleton entry separated from the following command by a
hard tab or a single quoted list of space separated entries, however
the xen-unstable-coverity line is singleton separated from the command
by a single space.

We could fix this by using a hard tab, but that ends up aligning
things in an aesthetically displeasing way, and relying on hard tabs
is fragile.

Instead, improve the parsing in mg-list-all-branches: break out a
couple of semantically (as well as syntactically) common regexp
elements out into variables, and then provide two regexps: one which
matches shell "assign default values" substitutions, and the other
which matches the ordinary shell assignments.

We use an empty pair of () in the first regexp to make sure that they
both produce the branch name list in $2.  (It would be possible to use
named capture groups but I'm not sure whether all our perls are recent
enough.)

I have verified that the actual difference in output right now is just
to remove the erroneous `.' entry.

Reported-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
 mg-list-all-branches |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/mg-list-all-branches b/mg-list-all-branches
index 87703c7..62d3ff1 100755
--- a/mg-list-all-branches
+++ b/mg-list-all-branches
@@ -7,11 +7,16 @@ use Sort::Versions;
 
 our %branches;
 
+my $branchvar_re = '(?:EXTRA_)?BRANCHES';
+my $branchchr_re = '[-.0-9a-z ]';
+
 foreach my $f (qw(cr-for-branches crontab)) {
     open C, $f or die $!;
     while (<C>) {
-        next unless m/(?:EXTRA_)?BRANCHES[:+]?='?([-.0-9a-z ]+)/;
-        $branches{$_}=1 foreach split /\s+/, $1;
+        next unless
+	    m/\$\{$branchvar_re[:+]?=()($branchchr_re+)\b/ ||
+	    m/$branchvar_re[:+]?=('?)($branchchr_re+?)\1\s/;
+        $branches{$_}=1 foreach split /\s+/, $2;
     }
     close C or die $!;
 }
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[OSSTEST PATCH 2/6] make-distros-flight: Remove useless conditional

[Ian Jackson]

debian_suite is always set in "case $branch in" at the top of
make-distros-flight.

Remove it as part of rationalising the suite variables in this area.

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 make-distros-flight |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/make-distros-flight b/make-distros-flight
index 9d04d3b..9b4684d 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -171,12 +171,8 @@ test_matrix_do_one () {
 
   for domU in $domUarches ; do
 
-    if [ -n $debian_suite ] ; then
-        gsuite=$debian_suite
-
-        test_do_one_netboot
-
-    fi
+    gsuite=$debian_suite
+    test_do_one_netboot
 
     if [ $debian_suite = "snapshot" ]; then
         gsuite=sid
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[OSSTEST PATCH 3/6] make-distros-flight: Key "snapshot" special case off $branch

[Ian Jackson]

This eliminates two places where $debian_suite is expected to contain
"snapshot" rather than a suite.

We want to change debian_suite to always contain a real suite, so that
we can fold its uses in to other suite variables.

No functional change for existing branches.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 make-distros-flight |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/make-distros-flight b/make-distros-flight
index 9b4684d..060eda3 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -66,7 +66,7 @@ test_matrix_branch_filter_callback () {
 test_do_one_netboot () {
   local diurl=
 
-  if [ x$gsuite = xsnapshot ] ; then
+  if [ $branch = distros-debian-snapshot ] ; then
       diurl="http://d-i.debian.org/daily-images/$domU/daily/netboot"
       gsuite=sid
       gver=daily
@@ -174,7 +174,7 @@ test_matrix_do_one () {
     gsuite=$debian_suite
     test_do_one_netboot
 
-    if [ $debian_suite = "snapshot" ]; then
+    if [ $branch = distros-debian-snapshot ]; then
         gsuite=sid
         for cd in current weekly ; do
           case ${domU}_${gsuite} in
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[OSSTEST PATCH 4/6] make-flight: Make debian_suite always contain a suite

[Ian Jackson]

The setting of the suite to "sid" in the snapshot case now occurs in
the case at the top of the script.

This means that gsuite always contains the actual suite name, and the
special cases which set it to sid can be removed.

No functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 make-distros-flight |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/make-distros-flight b/make-distros-flight
index 060eda3..fa5b71b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -34,6 +34,7 @@ defsuite=`getconfig DebianSuite`
 defguestsuite=`getconfig GuestDebianSuite`
 
 case $branch in
+  distros-debian-snapshot) debian_suite=sid ;;
   distros-debian-*) debian_suite=${branch#distros-debian-} ;;
   *)                echo $branch >&2; exit 1               ;;
 esac
@@ -68,7 +69,6 @@ test_do_one_netboot () {
 
   if [ $branch = distros-debian-snapshot ] ; then
       diurl="http://d-i.debian.org/daily-images/$domU/daily/netboot"
-      gsuite=sid
       gver=daily
   else
     #local mirror="http://`getconfig DebianMirrorHost`/`getconfig DebianMirrorSubpath`"
@@ -175,7 +175,6 @@ test_matrix_do_one () {
     test_do_one_netboot
 
     if [ $branch = distros-debian-snapshot ]; then
-        gsuite=sid
         for cd in current weekly ; do
           case ${domU}_${gsuite} in
             armhf_*) continue;; # No iso targets for armhf
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[OSSTEST PATCH 5/6] make-distros-flight: Always set guest_suite and defguestsuite

[Ian Jackson]

Abolish the shell variables $gsuite and $debian_suite (which were
referred to only in make-distros-flight) and set and use the variables
guest_suite and defguestsuite.

These variables are used by the machinery in mfi-common to populate
the runvars.

No functional change (as seen in standalone-generate-dump-flight-runvars,
with mg-list-all-branches edited to use crontab-cambridge).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 make-distros-flight |   21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/make-distros-flight b/make-distros-flight
index fa5b71b..8ab00b1 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -31,14 +31,15 @@ flight=`./cs-flight-create $blessing $branch`
 . ./mfi-common
 
 defsuite=`getconfig DebianSuite`
-defguestsuite=`getconfig GuestDebianSuite`
 
 case $branch in
-  distros-debian-snapshot) debian_suite=sid ;;
-  distros-debian-*) debian_suite=${branch#distros-debian-} ;;
+  distros-debian-snapshot) guest_suite=sid ;;
+  distros-debian-*) guest_suite=${branch#distros-debian-} ;;
   *)                echo $branch >&2; exit 1               ;;
 esac
 
+defguestsuite=$guest_suite
+
 job_create_build_filter_callback () {
   local job=$1; shift
 
@@ -74,11 +75,11 @@ test_do_one_netboot () {
     #local mirror="http://`getconfig DebianMirrorHost`/`getconfig DebianMirrorSubpath`"
     # XXX local mirror seems to serve up stale files.
     local mirror="http://ftp.debian.org/debian"
-    diurl="$mirror/dists/$gsuite/main/installer-$domU/current/images/netboot"
-    gver=$gsuite
+    diurl="$mirror/dists/$guest_suite/main/installer-$domU/current/images/netboot"
+    gver=$guest_suite
   fi
 
-  case ${domU}_${gsuite} in
+  case ${domU}_${guest_suite} in
     armhf_squeeze) return;; # No armhf in Squeeze
     armhf_wheezy) return;; # No armhf guest support in Wheezy
     *) ;;
@@ -96,7 +97,7 @@ test_do_one_netboot () {
       diurl="$diurl/debian-installer/arm64";;
   esac
 
-  case ${dom0arch}_${domU}_${gsuite} in
+  case ${dom0arch}_${domU}_${guest_suite} in
     arm*_arm*_*) bootloader="pygrub";; # no pvgrub for arm
 
     # Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
@@ -113,7 +114,7 @@ test_do_one_netboot () {
     test-debian-di xl $xenarch $dom0arch                        \
       kernbuildjob=${bfi}build-$dom0arch-$kernbuild             \
       debian_arch=$domU                                         \
-      debian_suite=$gsuite                                      \
+      debian_suite=$guest_suite                                      \
       debian_method=netboot                                     \
       debian_netboot_kernel="$diurl/vmlinuz"                    \
       debian_netboot_ramdisk="$diurl/initrd.gz"                 \
@@ -171,12 +172,12 @@ test_matrix_do_one () {
 
   for domU in $domUarches ; do
 
-    gsuite=$debian_suite
+    guest_suite=$guest_suite
     test_do_one_netboot
 
     if [ $branch = distros-debian-snapshot ]; then
         for cd in current weekly ; do
-          case ${domU}_${gsuite} in
+          case ${domU}_${guest_suite} in
             armhf_*) continue;; # No iso targets for armhf
             *) ;;
           esac
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[OSSTEST PATCH 6/6] mfi-common: Do not set di_version runvar to empty string

[Ian Jackson]

Specifically, do not set all_host_di_version to the shell variable
$di_version unless the latter has a nonempty value.  A set but empty
value for all_host_di_version does not default to the version for the
specific suite.  So this produces install failures.

This bug seems to have been introduced fairly recently, as fallout
from recent di_version handling changes.

diffing standalone-generate-dump-flight-runvars shows the expected
changes.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 mfi-common |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/mfi-common b/mfi-common
index ab7c5e3..ca1e0dd 100644
--- a/mfi-common
+++ b/mfi-common
@@ -414,7 +414,10 @@ test_matrix_iterate () {
            ;;
     esac
 
-    hostos_runvars="all_host_suite=$suite all_host_di_version=$di_version"
+    hostos_runvars="all_host_suite=$suite"
+    case "$di_version" in
+      ?*) hostos_runvars+=" all_host_di_version=$di_version"
+    esac
 
     for kern in ''; do
 
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel