[PATCH v2 1/2] cr-daily-branch: Begin to support other reasons for forcing a baseline.

[PATCH v2 1/2] cr-daily-branch: Begin to support other reasons for forcing a baseline.

[Ian Campbell]

By converting the current boolean $force_baseline into a keyword
indicating the reason.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cr-daily-branch | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/cr-daily-branch b/cr-daily-branch
index 34b6d2b..7e3e69e 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -47,7 +47,7 @@ determine_version () {
 	local tversionvar=$1
 	local tbranch=$2
 	local treevarwhich=$3
-	if [ "x$tbranch" = "x$branch" ] && ! $force_baseline; then
+	if [ "x$tbranch" = "x$branch" ] && [ "x$force_baseline" = x ]; then
                 if [ "x$FORCE_REVISION" != x ]; then
                         tversion="$FORCE_REVISION"
                 else
@@ -70,7 +70,7 @@ fetch_version () {
 
 treeurl=`./ap-print-url $branch`
 
-force_baseline=false
+force_baseline='' # Non-empty = indication why we are forcing baseline.
 skipidentical=true
 wantpush=$OSSTEST_PUSH
 
@@ -91,7 +91,7 @@ if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
 	if [ "x$testedflight" = x ]; then
 		wantpush=false
 		skipidentical=false
-		force_baseline=true
+		force_baseline='untested'
 		if [ "x$treeurl" != xnone: ]; then
 			treearg=--tree-$tree=$treeurl
 		fi
@@ -248,7 +248,8 @@ heading=tmp/$flight.heading-info
 : >$heading
 sgr_args+=" --info-headers --include-begin=$heading"
 
-if $force_baseline; then
+case "$force_baseline" in
+    untested)
 	subject_prefix="[$branch baseline test] "
 	cat >>$heading <<END
 "Old" tested version had not actually been tested; therefore in this
@@ -256,7 +257,10 @@ flight we test it, rather than a new candidate.  The baseline, if
 any, is the most recent actually tested revision.
 
 END
-fi
+    ;;
+    '') # Not forcing a baseline, nothing to say.
+    ;;
+esac
 
 revlog=tmp/$flight.revision-log
 
-- 
2.1.4

[PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Campbell]

Provide a new cr-daily-branch setting OSSTEST_BASELINES_ONLY which
causes it to only attempt to test the current baseline (if it is
untested) and never the tip version. Such tests will not result in any
push.

Add a cronjob to Cambridge which runs in this manner, ensuring that
there will usually be some sort of reasonably up to date baseline for
any given branch which can be used for comparisons in adhoc testing or
bisections.

This will also give us some data on the success of various branches on
the set of machines in Cambridge, which can be useful/interesting.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v2: Wording tweak.
---
 cr-daily-branch   | 13 ++++++++++++-
 crontab-cambridge |  1 +
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/cr-daily-branch b/cr-daily-branch
index 7e3e69e..ed8f8c0 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -85,7 +85,11 @@ check_tested () {
 	  "$@"
 }
 
-if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
+if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
+    force_baseline=baselines-only
+    wantpush=false
+    skipidentical=true
+elif [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
 	testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
 
 	if [ "x$testedflight" = x ]; then
@@ -258,6 +262,13 @@ any, is the most recent actually tested revision.
 
 END
     ;;
+    baselines-only)
+        #subject-prefix="[... ] "
+        cat >> $heading <<END
+This run is configured for baseline tests only.
+
+END
+    ;;
     '') # Not forcing a baseline, nothing to say.
     ;;
 esac
diff --git a/crontab-cambridge b/crontab-cambridge
index 2146311..64e4f27 100644
--- a/crontab-cambridge
+++ b/crontab-cambridge
@@ -2,5 +2,6 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
 # m            h       dom mon dow     command
 4-59/30        *       * * *           cd testing.git && BRANCHES=osstest              ./cr-for-branches branches -q "./cr-daily-branch --real"
+11-59/30	*	* * *		cd testing.git && OSSTEST_BASELINES_ONLY=y	./cr-for-branches branches -q "./cr-daily-branch --real"
 22		8	* * *		cd testing.git && BRANCHES=maintjobs		./cr-for-branches . -w ./cr-all-branch-statuses ''
 3              4       * * *           savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
-- 
2.1.4

Re: [PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Campbell]

On Mon, 2015-07-27 at 11:55 +0100, Ian Campbell wrote:
> Provide a new cr-daily-branch setting OSSTEST_BASELINES_ONLY which
> causes it to only attempt to test the current baseline (if it is
> untested)

There is a problem with this "if it is untested". This is implemented below
by setting skipidentical=true. However that is comparing ap-fetch-version
with ap-fetch-version-old which uses the main push gate results, i.e. it is
always going to see the same old and new revisions (they are both the
baseline) and skip it. Not sure how I missed this before!

I think I need to do something with check_tested to find a flight in the
database.

>  and never the tip version. Such tests will not result in any
> push.
> 
> Add a cronjob to Cambridge which runs in this manner, ensuring that
> there will usually be some sort of reasonably up to date baseline for
> any given branch which can be used for comparisons in adhoc testing or
> bisections.
> 
> This will also give us some data on the success of various branches on
> the set of machines in Cambridge, which can be useful/interesting.
> 
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
> v2: Wording tweak.
> ---
>  cr-daily-branch   | 13 ++++++++++++-
>  crontab-cambridge |  1 +
>  2 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/cr-daily-branch b/cr-daily-branch
> index 7e3e69e..ed8f8c0 100755
> --- a/cr-daily-branch
> +++ b/cr-daily-branch
> @@ -85,7 +85,11 @@ check_tested () {
>  	  "$@"
>  }
>  
> -if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
> +if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
> +    force_baseline=baselines-only
> +    wantpush=false
> +    skipidentical=true
> +elif [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
>  	testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
>  
>  	if [ "x$testedflight" = x ]; then
> @@ -258,6 +262,13 @@ any, is the most recent actually tested revision.
>  
>  END
>      ;;
> +    baselines-only)
> +        #subject-prefix="[... ] "
> +        cat >> $heading <<END
> +This run is configured for baseline tests only.
> +
> +END
> +    ;;
>      '') # Not forcing a baseline, nothing to say.
>      ;;
>  esac
> diff --git a/crontab-cambridge b/crontab-cambridge
> index 2146311..64e4f27 100644
> --- a/crontab-cambridge
> +++ b/crontab-cambridge
> @@ -2,5 +2,6 @@ 
> PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
>  MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
>  # m            h       dom mon dow     command
>  4-59/30        *       * * *           cd testing.git && 
> BRANCHES=osstest              ./cr-for-branches branches -q "./cr-daily
> -branch --real"
> +11-59/30	*	* * *		cd testing.git && 
> OSSTEST_BASELINES_ONLY=y	./cr-for-branches branches -q "./cr
> -daily-branch --real"
>  22		8	* * *		cd testing.git && 
> BRANCHES=maintjobs		./cr-for-branches . -w ./cr-all-branch
> -statuses ''
>  3              4       * * *           savelog -c28 testing.git/tmp/cr
> -for-branches.log >/dev/null

Re: [PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Campbell]

On Wed, 2015-08-12 at 15:08 +0100, Ian Campbell wrote:
> I think I need to do something with check_tested to find a flight in the
> database.

I came up with this incremental patch. I need to generate an up to date
baseline for a flight on the Cambridge infra in order to fully test all the
cases so I'm just running a linux-arm-xen flight (it doesn't change often
and only has a handful of jobs). I'll repost once that is done and I've
tested some more.

diff --git a/cr-daily-branch b/cr-daily-branch
index 4b9b525..f9edbd4 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -100,6 +100,13 @@ if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
     force_baseline=baselines-only
     wantpush=false
     skipidentical=true
+    if [ "x$treeurl" != xnone: ]; then
+	treearg=--tree-$tree=$treeurl
+    fi
+    tested_revision=`check_tested $treearg --print-revision=$tree`
+    if [ "x$tested_revision" != x ]; then
+	OLD_REVISION="$tested_revision"
+    fi
 elif [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
 	testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
 
Ian.

Re: [PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH v2 2/2] cambridge: arrange to test each new baseline"):
> diff --git a/cr-daily-branch b/cr-daily-branch
> index 4b9b525..f9edbd4 100755
> --- a/cr-daily-branch
> +++ b/cr-daily-branch
> @@ -100,6 +100,13 @@ if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
>      force_baseline=baselines-only
>      wantpush=false
>      skipidentical=true
> +    if [ "x$treeurl" != xnone: ]; then
> +	treearg=--tree-$tree=$treeurl
> +    fi
> +    tested_revision=`check_tested $treearg --print-revision=$tree`
> +    if [ "x$tested_revision" != x ]; then
> +	OLD_REVISION="$tested_revision"
> +    fi

This is strikingly similar to the code inside
   if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
near line 99.

Maybe, instead, you should nobble the
     if [ "x$OLD_REVISION" = x ]; then
bit which sets OLD_REVISION, somehow, and then the later code is
nearly right.

Ian.

Re: [PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Campbell]

On Wed, 2015-08-12 at 16:46 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [PATCH v2 2/2] cambridge: arrange 
> to test each new baseline"):
> > diff --git a/cr-daily-branch b/cr-daily-branch
> > index 4b9b525..f9edbd4 100755
> > --- a/cr-daily-branch
> > +++ b/cr-daily-branch
> > @@ -100,6 +100,13 @@ if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
> >      force_baseline=baselines-only
> >      wantpush=false
> >      skipidentical=true
> > +    if [ "x$treeurl" != xnone: ]; then
> > +	treearg=--tree-$tree=$treeurl
> > +    fi
> > +    tested_revision=`check_tested $treearg --print-revision=$tree`
> > +    if [ "x$tested_revision" != x ]; then
> > +	OLD_REVISION="$tested_revision"
> > +    fi
> 
> This is strikingly similar to the code inside
>    if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
> near line 99.
> 
> Maybe, instead, you should nobble the
>      if [ "x$OLD_REVISION" = x ]; then
> bit which sets OLD_REVISION, somehow, and then the later code is
> nearly right.

The relevant hunks (proper, not incremental) would then be something like
this (untested):

diff --git a/cr-daily-branch b/cr-daily-branch
index 06df5a0..caf5b5b 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -91,18 +91,28 @@ if [ "x$OLD_REVISION" = x ]; then
            OSSTEST_NO_BASELINE=y
            OLD_REVISION=flight-`check_tested`
            ;;
-       *) OLD_REVISION="`./ap-fetch-version-old $branch`";;
+       *)
+           if [ "x$OSSTEST_BASELINES_ONLY" != xy ] ; then
+               OLD_REVISION="`./ap-fetch-version-old $branch`"
+           fi
+           ;;
     esac
     export OLD_REVISION
 fi
 
 if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
-       testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
+       if [ "x$OSSTEST_BASELINES_ONLY" != xy ] ; then
+           testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
+       fi
 
        if [ "x$testedflight" = x ]; then
                wantpush=false
                skipidentical=false
-               force_baseline='untested'
+               if [ "x$OSSTEST_BASELINES_ONLY" = xy ] ; then
+                   force_baseline=baselines-only
+               else
+                   force_baseline='untested'
+               fi
                if [ "x$treeurl" != xnone: ]; then
                        treearg=--tree-$tree=$treeurl
                fi

Is that better?

I also considered refactoring this bit:

		if [ "x$treeurl" != xnone: ]; then
			treearg=--tree-$tree=$treeurl
		fi
		tested_revision=`check_tested $treearg --print-revision=$tree`
		if [ "x$tested_revision" != x ]; then
			OLD_REVISION="$tested_revision"
		fi

into a helper e.g. select_last_test_revision.

Ian.

Re: [PATCH v2 2/2] cambridge: arrange to test each new baseline

[Ian Jackson]

Ian Campbell writes ("Re: [Xen-devel] [PATCH v2 2/2] cambridge: arrange to test each new baseline"):
> I also considered refactoring this bit:
> 
> 		if [ "x$treeurl" != xnone: ]; then
> 			treearg=--tree-$tree=$treeurl
> 		fi
> 		tested_revision=`check_tested $treearg --print-revision=$tree`
> 		if [ "x$tested_revision" != x ]; then
> 			OLD_REVISION="$tested_revision"
> 		fi
> 
> into a helper e.g. select_last_test_revision.

That might be best as it gives it a name.

Ian.