[OSSTEST PATCH 1/8] Executive: Fix an undef warning message

[OSSTEST PATCH 1/8] Executive: Fix an undef warning message

[Ian Jackson]

$onhost can be undef too

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 Osstest/Executive.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/Executive.pm b/Osstest/Executive.pm
index 80e70070..a0d9f81e 100644
--- a/Osstest/Executive.pm
+++ b/Osstest/Executive.pm
@@ -1283,7 +1283,8 @@ END
     return sub {
         my ($job, $hostidname, $onhost, $uptoincl_testid) = @_;
 
-	my $memokey = "$job $hostidname $onhost ".($uptoincl_testid//"");
+	my $memokey = "$job $hostidname "
+	  .($onhost//"")." ".($uptoincl_testid//"");
 	my $memo = $our_memo->{$memokey};
 	return @$memo if $memo;
 
-- 
2.20.1

[OSSTEST PATCH 2/8] standalone: Use mkdir -p

[Ian Jackson]

These two mkdir calls could fail if
standalone-generate-dump-flight-runvars is run without a log
directory, because they were not concurrency-correct.

mkdir -p should fix that.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 standalone | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/standalone b/standalone
index 9553d6c9..4d1f3513 100755
--- a/standalone
+++ b/standalone
@@ -181,12 +181,8 @@ check_repos() {
 }
 
 ensure_logs() {
-    if [ ! -d "logs" ] ; then
-	mkdir "logs"
-    fi
-    if [ ! -d "logs/$flight" ] ; then
-	mkdir "logs/$flight"
-    fi
+    mkdir -p "logs"
+    mkdir -p "logs/$flight"
 }
 
 with_logging() {
-- 
2.20.1

[OSSTEST PATCH 3/8] sg-run-job: Preserve step state "fail" if set by test script

[Ian Jackson]

If the test script exits nonzero but after setting the step status to
'fail', we can leave it that way.  This is particularly relevant if
the iffail in the job spec says 'broken' or something.  After this
change, a step can decide to override that.

An alternative would be to have the step script exit zero, but of
course that would (generally) leave the job to continue running more
steps!

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 tcl/JobDB-Executive.tcl | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tcl/JobDB-Executive.tcl b/tcl/JobDB-Executive.tcl
index 4fe85696..38248823 100644
--- a/tcl/JobDB-Executive.tcl
+++ b/tcl/JobDB-Executive.tcl
@@ -325,6 +325,7 @@ proc step-set-status {flight job stepno st} {
                AND status<>'aborted'
                AND status<>'broken'
                AND status<>'starved'
+               AND status<>'fail'
         "
         set pause 0
         db-execute-array stopinfo "
-- 
2.20.1

[OSSTEST PATCH 4/8] ts-hosts-allocate-Executive: Allow to tolerate missing resources

[Ian Jackson]

Now, a job can specify that lack of a suitable host should be treated
as a plain test failure (ie, subject to the usual regression analysis)
rather than as an infrastructure or configuration problem.

This will be useful for some tests which don't work in some branches
because of lack of suitable hardware.  We want to avoid encoding our
hardware availability situation in make-flight.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 ts-hosts-allocate-Executive | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/ts-hosts-allocate-Executive b/ts-hosts-allocate-Executive
index 698437c0..58d2a389 100755
--- a/ts-hosts-allocate-Executive
+++ b/ts-hosts-allocate-Executive
@@ -704,6 +704,10 @@ sub alloc_hosts () {
 	my ($ok, $bookinglist) = attempt_allocation({
             ts_hosts_allocate_precheck => 1,
         }, 0);
+	if ($ok == $alloc_starved_r && $r{hostalloc_missing_expected}) {
+	    broken 'no suitable hosts available (as possibly expected)',
+	      'fail';
+	}
 	die $ok if $ok>1;
     }
 
-- 
2.20.1

[OSSTEST PATCH 5/8] cri-getplatforms: Give names to xenarch and suite

[Ian Jackson]

No functional change.  This will be useful in a moment.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 cri-getplatforms | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/cri-getplatforms b/cri-getplatforms
index 2b8cee0b..1f206908 100755
--- a/cri-getplatforms
+++ b/cri-getplatforms
@@ -17,9 +17,11 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 getplatforms () {
+	local xenarch=$1
+	local suite=$2
         perl -e '
                 use Osstest;
                 csreadconfig();
-                print join " ", $mhostdb->get_arch_platforms("'$blessing'", "'$1'", "'$2'") or die $!;
+                print join " ", $mhostdb->get_arch_platforms("'$blessing'", "'$xenarch'", "'$suite'") or die $!;
         '
 }
-- 
2.20.1

[OSSTEST PATCH 6/8] cri-getplatforms: Honour new MF_SIMULATE_PLATFORMS env var

[Ian Jackson]

This is to be expanded by the shell, using eval, so that it can refer
to $xenarch, $suite and $blessing.

No functional change if this variable is unset, or empty.  If it is
set to a single space, cri-getplatforms produces no output (as it does
anyway in standalone mode).

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 cri-getplatforms | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/cri-getplatforms b/cri-getplatforms
index 1f206908..9f4cae56 100755
--- a/cri-getplatforms
+++ b/cri-getplatforms
@@ -19,6 +19,10 @@
 getplatforms () {
 	local xenarch=$1
 	local suite=$2
+	if [ "x$MF_SIMULATE_PLATFORMS" != x ]; then
+		eval "echo $MF_SIMULATE_PLATFORMS"
+		return
+	fi
         perl -e '
                 use Osstest;
                 csreadconfig();
-- 
2.20.1

[OSSTEST PATCH 7/8] standalone-generate-dump-flight-runvars: Simulate cri-getplatforms

[Ian Jackson]

Set MF_SIMULATE_PLATFORMS to a suitable value if it is
not *set*.  (Distinguishing unset from set to empty.)

I have verified that this, plus the preceding commits to
cri-getplatforms, produces no change in the output of
  MF_SIMULATE_PLATFORMS='' OSSTEST_CONFIG=standalone-config-example eatmydata ./standalone-generate-dump-flight-runvars

Without the MF_SIMULATE_PLATFORMS setting it adds several new jobs to
each flight, name things like this:
  test-amd64-$arch1-xl-simplat-$arch2-$suite

The purpose of this right now is to provide a way to dry-run test the
next change.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 standalone-generate-dump-flight-runvars | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/standalone-generate-dump-flight-runvars b/standalone-generate-dump-flight-runvars
index 5c93b0af..07b9c126 100755
--- a/standalone-generate-dump-flight-runvars
+++ b/standalone-generate-dump-flight-runvars
@@ -45,6 +45,9 @@ fi
 : ${AP_FETCH_PLACEHOLDERS:=y}
 export AP_FETCH_PLACEHOLDERS
 
+: ${MF_SIMULATE_PLATFORMS='simplat-$xenarch-$suite'}
+export MF_SIMULATE_PLATFORMS
+
 export OSSTEST_HOSTSLIST_DUMMY=1
 
 if [ "x$AP_FETCH_PLACEHOLDERS" != xy ]; then
-- 
2.20.1

[OSSTEST PATCH 8/8] Tolerate lack of platform-specific hosts in old Xen branches

[Ian Jackson]

Right now we have a situation where these can't all be made to work
because because some older Xen branches are hard to make work on
current Debian stable, and we have some hardware (which we have tagged
as specific "platforms") which doesn't work with oldstable.

This seems like a general problem, so fix it this way.

Note that we still treat these failed allocations as failures, so they
are subject to regression analysis and ought not to appear willy-nilly
on existing branches.

Runvar dump shows the addition of this runvar
   hostalloc_missing_expected=1
to
   qemu-upstream-4.6-testing
   xen-4.6-testing
   ...
   qemu-upstream-4.14-testing
   xen-4.14-testing
inclusive.

Signed-off-by: Ian Jackson <iwj@xenproject.org>
---
 make-flight | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index d5a3d99a..fb38bc50 100755
--- a/make-flight
+++ b/make-flight
@@ -643,13 +643,27 @@ do_pv_debian_tests () {
   for xsm in $xsms ; do
     # Basic PV Linux test with xl
     for platform in '' `getplatforms $xenarch $defsuite` ; do
+      platform_runvars=''
 
       # xsm test is not platform specific
       if [ x$xsm = xtrue -a x$platform != x ]; then
           continue
       fi
 
-      do_pv_debian_test_one xl '' xl "$platform" enable_xsm=$xsm
+      missing_expected=false
+      if [ x$platform != x ]; then
+        case "$xenbranch" in
+          xen-3.*-testing)  missing_expected=true ;;
+          xen-4.*-testing)  missing_expected=true ;;
+          *) ;;
+        esac
+      fi
+      if $missing_expected; then
+        platform_runvars+=hostalloc_missing_expected=1
+      fi
+
+      do_pv_debian_test_one xl '' xl "$platform" enable_xsm=$xsm \
+                            $platform_runvars
 
     done
   done
-- 
2.20.1