[OSSTEST PATCH 0/9] mg-repro-flight: Provide --rebuild to make variant build jobs too

[OSSTEST PATCH 0/9] mg-repro-flight: Provide --rebuild to make variant build jobs too

[Ian Jackson]

It is annoying that mg-repro-flight cannot run a build for you too.
Fix this.

This is on xenbits in
  https://xenbits.xen.org/git-http/people/iwj/osstest.git
  xenbits.xen.org:/home/iwj/ext/osstest.git
etc. as the branch
  wip.repro-flight-builds.v1

Ian Jackson (9):
  mg-execute-flight: Save an mro in tmp/
  cs-adjust-flight: Break out copy_jobs_*
  cs-adjust-flight: Use db_prepare and honour -D -D for sql dumps
  mg-repro-setup: Improve a doc message slightly
  mg-repro-setup: Break out compute_adjusts
  mg-repro-setup: Move logging setup to later
  mg-repro-setup: Allow arguments to badusage
  mg-repro-setup: Move flight creation up before task creation
  mg-repro-flight: Provide --rebuild to make variant build jobs too

 cs-adjust-flight  | 149 ++++++++++++++++++++++++++++++++++++++++++++++++------
 mg-execute-flight |   1 +
 mg-repro-setup    | 122 +++++++++++++++++++++++++++++++++++++-------
 3 files changed, 238 insertions(+), 34 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 0/9] mg-repro-flight: Provide --rebuild to make variant build jobs too

[Ian Jackson]

It is annoying that mg-repro-flight cannot run a build for you too.
Fix this.

This is on xenbits in
  https://xenbits.xen.org/git-http/people/iwj/osstest.git
  xenbits.xen.org:/home/iwj/ext/osstest.git
etc. as the branch
  wip.repro-flight-builds.v1

Ian Jackson (9):
  mg-execute-flight: Save an mro in tmp/
  cs-adjust-flight: Break out copy_jobs_*
  cs-adjust-flight: Use db_prepare and honour -D -D for sql dumps
  mg-repro-setup: Improve a doc message slightly
  mg-repro-setup: Break out compute_adjusts
  mg-repro-setup: Move logging setup to later
  mg-repro-setup: Allow arguments to badusage
  mg-repro-setup: Move flight creation up before task creation
  mg-repro-flight: Provide --rebuild to make variant build jobs too

 cs-adjust-flight  | 149 ++++++++++++++++++++++++++++++++++++++++++++++++------
 mg-execute-flight |   1 +
 mg-repro-setup    | 122 +++++++++++++++++++++++++++++++++++++-------
 3 files changed, 238 insertions(+), 34 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 1/9] mg-execute-flight: Save an mro in tmp/

[Ian Jackson]

This may be useful for some things.  For example, it will be used in
just a moment.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-execute-flight | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mg-execute-flight b/mg-execute-flight
index b3cdf431..391f4810 100755
--- a/mg-execute-flight
+++ b/mg-execute-flight
@@ -81,6 +81,7 @@ echo >&5 "flight $flight reporting"
 
 ./sg-report-flight $basisflightopt \
 	--html-dir=$OSSTEST_HTMLPUB_DIR/$flight/ \
+	--machine-readable-output=tmp/$flight.mro \
 	$flight >tmp/$flight.report
 
 subject=`sed -n 1p <tmp/$flight.report`
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 1/9] mg-execute-flight: Save an mro in tmp/

[Ian Jackson]

This may be useful for some things.  For example, it will be used in
just a moment.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-execute-flight | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mg-execute-flight b/mg-execute-flight
index b3cdf431..391f4810 100755
--- a/mg-execute-flight
+++ b/mg-execute-flight
@@ -81,6 +81,7 @@ echo >&5 "flight $flight reporting"
 
 ./sg-report-flight $basisflightopt \
 	--html-dir=$OSSTEST_HTMLPUB_DIR/$flight/ \
+	--machine-readable-output=tmp/$flight.mro \
 	$flight >tmp/$flight.report
 
 subject=`sed -n 1p <tmp/$flight.report`
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 2/9] cs-adjust-flight: Break out copy_jobs_*

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cs-adjust-flight | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index ee1d917c..badabeff 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -194,10 +194,8 @@ sub do_rm_job ($) {
     verbose "$dstflight.$job job deleted\n" if $count;
 }
 
-sub copy_jobs ($$) {
-    my ($srcflight, $jobs) = @_;
-    prep_rm_jobs();
-    my @job_copy_qs = map { $dbh_tests->prepare($_) }
+sub copy_jobs_qs () {
+    map { $dbh_tests->prepare($_) }
         ("INSERT INTO jobs (flight, job, recipe, status)".
          "     SELECT ?, job, recipe, 'queued'".
          "       FROM jobs".
@@ -206,12 +204,22 @@ sub copy_jobs ($$) {
          "     SELECT ?, job, name, val, 'f'".
          "       FROM runvars".
          "      WHERE flight = ? AND job = ? AND NOT synth");
+}
+sub copy_jobs_do ($$) {
+    my ($qs, $srcflight, $srcjob) = @_;
+    $_->execute($dstflight, $srcflight, $job) foreach @$qs;
+    verbose "$dstflight.$job job created from $srcflight.$job\n";
+}
+
+sub copy_jobs ($$) {
+    my ($srcflight, $jobs) = @_;
+    prep_rm_jobs();
+    my @job_copy_qs = copy_jobs_qs;
     for_jobs($dstflight, $jobs, \&do_rm_job, 1);
     for_jobs($srcflight, $jobs, \&do_rm_job);
     for_jobs($srcflight, $jobs, sub {
         my ($job) = @_;
-        $_->execute($dstflight, $srcflight, $job) foreach @job_copy_qs;
-	verbose "$dstflight.$job job created from $srcflight.$job\n";
+	copy_jobs_do(\@job_copy_qs, $srcflight, $job);
     });
 }
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 2/9] cs-adjust-flight: Break out copy_jobs_*

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cs-adjust-flight | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index ee1d917c..badabeff 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -194,10 +194,8 @@ sub do_rm_job ($) {
     verbose "$dstflight.$job job deleted\n" if $count;
 }
 
-sub copy_jobs ($$) {
-    my ($srcflight, $jobs) = @_;
-    prep_rm_jobs();
-    my @job_copy_qs = map { $dbh_tests->prepare($_) }
+sub copy_jobs_qs () {
+    map { $dbh_tests->prepare($_) }
         ("INSERT INTO jobs (flight, job, recipe, status)".
          "     SELECT ?, job, recipe, 'queued'".
          "       FROM jobs".
@@ -206,12 +204,22 @@ sub copy_jobs ($$) {
          "     SELECT ?, job, name, val, 'f'".
          "       FROM runvars".
          "      WHERE flight = ? AND job = ? AND NOT synth");
+}
+sub copy_jobs_do ($$) {
+    my ($qs, $srcflight, $srcjob) = @_;
+    $_->execute($dstflight, $srcflight, $job) foreach @$qs;
+    verbose "$dstflight.$job job created from $srcflight.$job\n";
+}
+
+sub copy_jobs ($$) {
+    my ($srcflight, $jobs) = @_;
+    prep_rm_jobs();
+    my @job_copy_qs = copy_jobs_qs;
     for_jobs($dstflight, $jobs, \&do_rm_job, 1);
     for_jobs($srcflight, $jobs, \&do_rm_job);
     for_jobs($srcflight, $jobs, sub {
         my ($job) = @_;
-        $_->execute($dstflight, $srcflight, $job) foreach @job_copy_qs;
-	verbose "$dstflight.$job job created from $srcflight.$job\n";
+	copy_jobs_do(\@job_copy_qs, $srcflight, $job);
     });
 }
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 3/9] cs-adjust-flight: Use db_prepare and honour -D -D for sql dumps

[Ian Jackson]

This makes debugging it easier.  No functional change with zero or one
occurrences of -D.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cs-adjust-flight | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index badabeff..cc1684b4 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -75,6 +75,7 @@ csreadconfig();
 our $debug = 0;
 
 open VERBOSE, ">/dev/null" or die $!;
+open DEBUG, ">/dev/null" or die $!;
 our $verbose_buffer;
 
 while (@ARGV && $ARGV[0] =~ m/^\-/) {
@@ -83,6 +84,7 @@ while (@ARGV && $ARGV[0] =~ m/^\-/) {
     if (m/^-v$/) {
 	open VERBOSE, ">&STDERR" or die $!;
     } elsif (m/^-D$/) {
+        open DEBUG, ">&STDERR" if $debug;
         $debug = 1;
     } else {
         die "$_ ?";
@@ -138,9 +140,9 @@ sub for_things ($$$$$$$) {
     my ($spec, $fn, $ifnone,
         $table, $keycol, $basecond, $basecondvals) = @_;
 
-    my $thing_q = $dbh_tests->prepare
+    my $thing_q = db_prepare
         ("SELECT * FROM $table WHERE $basecond AND $keycol = ?");
-    my $things_q = $dbh_tests->prepare
+    my $things_q = db_prepare
         ("SELECT * FROM $table WHERE $basecond");
 
     my ($specfn,$exact) = notspec_exfn("FOR_THINGS $table.$keycol",$spec);
@@ -182,7 +184,7 @@ our @job_rm_qs;
 
 sub prep_rm_jobs () {
     return if @job_rm_qs;
-    @job_rm_qs = map { $dbh_tests->prepare($_) }
+    @job_rm_qs = map { db_prepare($_) }
         ("DELETE FROM runvars WHERE flight = ? AND job = ?",
          "DELETE FROM jobs WHERE flight = ? AND job = ?");
 }
@@ -195,7 +197,7 @@ sub do_rm_job ($) {
 }
 
 sub copy_jobs_qs () {
-    map { $dbh_tests->prepare($_) }
+    map { db_prepare($_) }
         ("INSERT INTO jobs (flight, job, recipe, status)".
          "     SELECT ?, job, recipe, 'queued'".
          "       FROM jobs".
@@ -205,8 +207,8 @@ sub copy_jobs_qs () {
          "       FROM runvars".
          "      WHERE flight = ? AND job = ? AND NOT synth");
 }
-sub copy_jobs_do ($$) {
-    my ($qs, $srcflight, $srcjob) = @_;
+sub copy_jobs_do ($$$) {
+    my ($qs, $srcflight, $job) = @_;
     $_->execute($dstflight, $srcflight, $job) foreach @$qs;
     verbose "$dstflight.$job job created from $srcflight.$job\n";
 }
@@ -253,9 +255,9 @@ sub change__copy_jobs {
     copy_jobs($srcflight, $jobs);
 }
 
-our $runvar_rm_q = $dbh_tests->prepare
+our $runvar_rm_q = db_prepare
     ("DELETE FROM runvars WHERE flight = ? AND job = ? AND name = ?");
-our $runvar_insert_q = $dbh_tests->prepare
+our $runvar_insert_q = db_prepare
     ("INSERT INTO runvars (flight, job, name, val, synth)".
      " VALUES (?, ?, ?, ?, 'f')");
 
@@ -430,7 +432,7 @@ sub change__job_status {
     die unless @changes >= 1;
     my $jobs = shift @changes;
 
-    my $q = $dbh_tests->prepare(<<END);
+    my $q = db_prepare(<<END);
         SELECT status
           FROM jobs
          WHERE flight = ? AND job = ?
@@ -449,12 +451,12 @@ sub change__jobs_rename {
     my $jobs = shift @changes;
     my $perlop = shift @changes;
 
-    my $jobq = $dbh_tests->prepare(<<END);
+    my $jobq = db_prepare(<<END);
     UPDATE jobs
        SET job = ?
      WHERE flight = ? AND job = ?
 END
-    my $runvarq = $dbh_tests->prepare(<<END);
+    my $runvarq = db_prepare(<<END);
     UPDATE runvars
        SET job = ?
      WHERE flight = ? AND job = ?
@@ -472,7 +474,7 @@ sub changes () {
 
     if ($dstflight =~ m/^\d+$/) {
 	my $steps_q =
-	    $dbh_tests->prepare("SELECT * FROM steps WHERE flight = ?");
+	    db_prepare("SELECT * FROM steps WHERE flight = ?");
 	$steps_q->execute($dstflight);
 	my $step = $steps_q->fetchrow_hashref();
 	if ($step) {
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 3/9] cs-adjust-flight: Use db_prepare and honour -D -D for sql dumps

[Ian Jackson]

This makes debugging it easier.  No functional change with zero or one
occurrences of -D.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cs-adjust-flight | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index badabeff..cc1684b4 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -75,6 +75,7 @@ csreadconfig();
 our $debug = 0;
 
 open VERBOSE, ">/dev/null" or die $!;
+open DEBUG, ">/dev/null" or die $!;
 our $verbose_buffer;
 
 while (@ARGV && $ARGV[0] =~ m/^\-/) {
@@ -83,6 +84,7 @@ while (@ARGV && $ARGV[0] =~ m/^\-/) {
     if (m/^-v$/) {
 	open VERBOSE, ">&STDERR" or die $!;
     } elsif (m/^-D$/) {
+        open DEBUG, ">&STDERR" if $debug;
         $debug = 1;
     } else {
         die "$_ ?";
@@ -138,9 +140,9 @@ sub for_things ($$$$$$$) {
     my ($spec, $fn, $ifnone,
         $table, $keycol, $basecond, $basecondvals) = @_;
 
-    my $thing_q = $dbh_tests->prepare
+    my $thing_q = db_prepare
         ("SELECT * FROM $table WHERE $basecond AND $keycol = ?");
-    my $things_q = $dbh_tests->prepare
+    my $things_q = db_prepare
         ("SELECT * FROM $table WHERE $basecond");
 
     my ($specfn,$exact) = notspec_exfn("FOR_THINGS $table.$keycol",$spec);
@@ -182,7 +184,7 @@ our @job_rm_qs;
 
 sub prep_rm_jobs () {
     return if @job_rm_qs;
-    @job_rm_qs = map { $dbh_tests->prepare($_) }
+    @job_rm_qs = map { db_prepare($_) }
         ("DELETE FROM runvars WHERE flight = ? AND job = ?",
          "DELETE FROM jobs WHERE flight = ? AND job = ?");
 }
@@ -195,7 +197,7 @@ sub do_rm_job ($) {
 }
 
 sub copy_jobs_qs () {
-    map { $dbh_tests->prepare($_) }
+    map { db_prepare($_) }
         ("INSERT INTO jobs (flight, job, recipe, status)".
          "     SELECT ?, job, recipe, 'queued'".
          "       FROM jobs".
@@ -205,8 +207,8 @@ sub copy_jobs_qs () {
          "       FROM runvars".
          "      WHERE flight = ? AND job = ? AND NOT synth");
 }
-sub copy_jobs_do ($$) {
-    my ($qs, $srcflight, $srcjob) = @_;
+sub copy_jobs_do ($$$) {
+    my ($qs, $srcflight, $job) = @_;
     $_->execute($dstflight, $srcflight, $job) foreach @$qs;
     verbose "$dstflight.$job job created from $srcflight.$job\n";
 }
@@ -253,9 +255,9 @@ sub change__copy_jobs {
     copy_jobs($srcflight, $jobs);
 }
 
-our $runvar_rm_q = $dbh_tests->prepare
+our $runvar_rm_q = db_prepare
     ("DELETE FROM runvars WHERE flight = ? AND job = ? AND name = ?");
-our $runvar_insert_q = $dbh_tests->prepare
+our $runvar_insert_q = db_prepare
     ("INSERT INTO runvars (flight, job, name, val, synth)".
      " VALUES (?, ?, ?, ?, 'f')");
 
@@ -430,7 +432,7 @@ sub change__job_status {
     die unless @changes >= 1;
     my $jobs = shift @changes;
 
-    my $q = $dbh_tests->prepare(<<END);
+    my $q = db_prepare(<<END);
         SELECT status
           FROM jobs
          WHERE flight = ? AND job = ?
@@ -449,12 +451,12 @@ sub change__jobs_rename {
     my $jobs = shift @changes;
     my $perlop = shift @changes;
 
-    my $jobq = $dbh_tests->prepare(<<END);
+    my $jobq = db_prepare(<<END);
     UPDATE jobs
        SET job = ?
      WHERE flight = ? AND job = ?
 END
-    my $runvarq = $dbh_tests->prepare(<<END);
+    my $runvarq = db_prepare(<<END);
     UPDATE runvars
        SET job = ?
      WHERE flight = ? AND job = ?
@@ -472,7 +474,7 @@ sub changes () {
 
     if ($dstflight =~ m/^\d+$/) {
 	my $steps_q =
-	    $dbh_tests->prepare("SELECT * FROM steps WHERE flight = ?");
+	    db_prepare("SELECT * FROM steps WHERE flight = ?");
 	$steps_q->execute($dstflight);
 	my $step = $steps_q->fetchrow_hashref();
 	if ($step) {
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 4/9] mg-repro-setup: Improve a doc message slightly

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index 6ed4d85e..c8bcad33 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -33,7 +33,7 @@ usage () { cat <<END
    (default <ident> is \`host')
 
  OPTIONs
-   -t<duration>         estimated duration (default = 28d)
+   -t<duration>         estimated overall allocation lifetime (default = 28d)
    --rogue              bypass queuing system and allocate now
    --capture            do not suppress log capture
    -r<var>=<value>      set runvar
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 4/9] mg-repro-setup: Improve a doc message slightly

[Ian Jackson]

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index 6ed4d85e..c8bcad33 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -33,7 +33,7 @@ usage () { cat <<END
    (default <ident> is \`host')
 
  OPTIONs
-   -t<duration>         estimated duration (default = 28d)
+   -t<duration>         estimated overall allocation lifetime (default = 28d)
    --rogue              bypass queuing system and allocate now
    --capture            do not suppress log capture
    -r<var>=<value>      set runvar
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 5/9] mg-repro-setup: Break out compute_adjusts

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index c8bcad33..10ab65a8 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -54,7 +54,6 @@ set -e -o posix
 
 mgexecflags=()
 adjustsets=()
-adjusts=()
 allocs=()
 logfile=tmp/mg-repro-setup.log
 duration=28d
@@ -104,14 +103,17 @@ adjrunvar () {
 	delrunvar "$1"
 	adjusts+=(runvar-set "$job" "$1" "$2")
 }
-
-for arg in "${adjustsets[@]}"; do
-	case "$arg" in
-	!*|^*)	delrunvar "${arg#?}"		;;
-	*=*)	adjrunvar "${arg%%=*}" "${arg#*=}"	;;
-	*)	bad-adjuistset-pattern			;;
-	esac
-done
+compute_adjusts () {
+	adjusts=()
+	for arg in "$@"; do
+		case "$arg" in
+		!*|^*)	delrunvar "${arg#?}"		;;
+		*=*)	adjrunvar "${arg%%=*}" "${arg#*=}"	;;
+		*)	bad-adjuistset-pattern			;;
+		esac
+	done
+}
+compute_adjusts "${adjustsets[@]}"
 
 while [ $# -ne 0 ]; do
 	arg=$1; shift
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 5/9] mg-repro-setup: Break out compute_adjusts

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index c8bcad33..10ab65a8 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -54,7 +54,6 @@ set -e -o posix
 
 mgexecflags=()
 adjustsets=()
-adjusts=()
 allocs=()
 logfile=tmp/mg-repro-setup.log
 duration=28d
@@ -104,14 +103,17 @@ adjrunvar () {
 	delrunvar "$1"
 	adjusts+=(runvar-set "$job" "$1" "$2")
 }
-
-for arg in "${adjustsets[@]}"; do
-	case "$arg" in
-	!*|^*)	delrunvar "${arg#?}"		;;
-	*=*)	adjrunvar "${arg%%=*}" "${arg#*=}"	;;
-	*)	bad-adjuistset-pattern			;;
-	esac
-done
+compute_adjusts () {
+	adjusts=()
+	for arg in "$@"; do
+		case "$arg" in
+		!*|^*)	delrunvar "${arg#?}"		;;
+		*=*)	adjrunvar "${arg%%=*}" "${arg#*=}"	;;
+		*)	bad-adjuistset-pattern			;;
+		esac
+	done
+}
+compute_adjusts "${adjustsets[@]}"
 
 while [ $# -ne 0 ]; do
 	arg=$1; shift
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 6/9] mg-repro-setup: Move logging setup to later

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index 10ab65a8..b41bf478 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -115,6 +115,13 @@ compute_adjusts () {
 }
 compute_adjusts "${adjustsets[@]}"
 
+progressf () { printf >&2 "$@"; }
+progress () { progressf "%s\n" "$1"; }
+
+progress "logging to $logfile"
+savelog "$logfile"
+exec 3>"$logfile"
+
 while [ $# -ne 0 ]; do
 	arg=$1; shift
 
@@ -160,13 +167,6 @@ while [ $# -ne 0 ]; do
 	esac
 done
 
-progressf () { printf >&2 "$@"; }
-progress () { progressf "%s\n" "$1"; }
-
-progress "logging to $logfile"
-savelog "$logfile"
-exec 3>"$logfile"
-
 OSSTEST_TASK=$(perl -e '
 	use Osstest;
 	use Osstest::Executive;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 6/9] mg-repro-setup: Move logging setup to later

[Ian Jackson]

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index 10ab65a8..b41bf478 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -115,6 +115,13 @@ compute_adjusts () {
 }
 compute_adjusts "${adjustsets[@]}"
 
+progressf () { printf >&2 "$@"; }
+progress () { progressf "%s\n" "$1"; }
+
+progress "logging to $logfile"
+savelog "$logfile"
+exec 3>"$logfile"
+
 while [ $# -ne 0 ]; do
 	arg=$1; shift
 
@@ -160,13 +167,6 @@ while [ $# -ne 0 ]; do
 	esac
 done
 
-progressf () { printf >&2 "$@"; }
-progress () { progressf "%s\n" "$1"; }
-
-progress "logging to $logfile"
-savelog "$logfile"
-exec 3>"$logfile"
-
 OSSTEST_TASK=$(perl -e '
 	use Osstest;
 	use Osstest::Executive;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 7/9] mg-repro-setup: Allow arguments to badusage

[Ian Jackson]

No functional change with existing call sites.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index b41bf478..d63e29b6 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -46,7 +46,7 @@ END
 
 }
 
-badusage () { echo >&2 "bad usage"; usage >&2; exit 126; }
+badusage () { echo >&2 "bad usage$*"; usage >&2; exit 126; }
 
 set -e -o posix
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 7/9] mg-repro-setup: Allow arguments to badusage

[Ian Jackson]

No functional change with existing call sites.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index b41bf478..d63e29b6 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -46,7 +46,7 @@ END
 
 }
 
-badusage () { echo >&2 "bad usage"; usage >&2; exit 126; }
+badusage () { echo >&2 "bad usage$*"; usage >&2; exit 126; }
 
 set -e -o posix
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 8/9] mg-repro-setup: Move flight creation up before task creation

[Ian Jackson]

No significant functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index d63e29b6..2e1d3b88 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -167,6 +167,9 @@ while [ $# -ne 0 ]; do
 	esac
 done
 
+flight=$(./cs-adjust-flight new:$blessing)
+progress "new flight is $flight"
+
 OSSTEST_TASK=$(perl -e '
 	use Osstest;
 	use Osstest::Executive;
@@ -176,9 +179,6 @@ OSSTEST_TASK=$(perl -e '
 ')
 export OSSTEST_TASK
 
-flight=$(./cs-adjust-flight new:$blessing)
-progress "new flight is $flight"
-
 if [ "${alloc_idents[*]}" ]; then
 	progress "allocating ${alloc_idents[*]} ..."
 	alloc_output=tmp/$flight.allocations
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 8/9] mg-repro-setup: Move flight creation up before task creation

[Ian Jackson]

No significant functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 mg-repro-setup | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/mg-repro-setup b/mg-repro-setup
index d63e29b6..2e1d3b88 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -167,6 +167,9 @@ while [ $# -ne 0 ]; do
 	esac
 done
 
+flight=$(./cs-adjust-flight new:$blessing)
+progress "new flight is $flight"
+
 OSSTEST_TASK=$(perl -e '
 	use Osstest;
 	use Osstest::Executive;
@@ -176,9 +179,6 @@ OSSTEST_TASK=$(perl -e '
 ')
 export OSSTEST_TASK
 
-flight=$(./cs-adjust-flight new:$blessing)
-progress "new flight is $flight"
-
 if [ "${alloc_idents[*]}" ]; then
 	progress "allocating ${alloc_idents[*]} ..."
 	alloc_output=tmp/$flight.allocations
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST PATCH 9/9] mg-repro-flight: Provide --rebuild to make variant build jobs too

[Ian Jackson]

This allows a single command to repro a particular job with a variety
of different source code.

The implementation technique is:
  - run the build job in a separate flight, so that it can run
    with a separate task which gives its host up after the build
  - do much of the heavy lifting of runvar fiddling etc. in
    a new helper routine in cs-adjust-flight

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cs-adjust-flight | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 mg-repro-setup   |  84 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 192 insertions(+), 1 deletion(-)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index cc1684b4..5be5af38 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -469,6 +469,115 @@ END
 	$runvarq->execute($newjob, $dstflight, $oldjob);
     })
 }
+
+sub change__repro_buildjobs {
+    # helper for mg-repro-setup
+    # arguments to this change are
+    #    EXAMPLE-FLIGHT EXAMPLE-JOB THING...
+    # where THING is
+    #    .BUILDJOB =BUILDJOBRUNVAR +TREE=URL#REVISION
+    # (we eat all until it doesn't start with . = +)
+    # stdout output is
+    #    list of runvar settings for repro job
+    die unless @changes >= 2;
+    my $eflight = shift @changes;
+    my $ejob = shift @changes;
+    my @refspecs;
+    my @varspecs;
+    my %treespecs;
+    while (@changes && $changes[0] =~ m/^[.=+]/) {
+	local $_ = shift @changes;
+	if (m/^[.=]/) {
+	    push @refspecs, $_;
+	} elsif (m/^\+(.*?)\=(.*)\#(.*)$/) {
+	    die $1 if $treespecs{$1};
+	    $treespecs{$1} = [$2,$3];
+	} elsif (m/^-r/) {
+	    push @varspecs, $_;
+	} else {
+	    die "bad .BUILDJOB / =BUILDJOBRUNVAR / +TREE=URL#REVISION): $_\n";
+	}
+    }
+    my $testq = db_prepare(<<END);
+SELECT name, val FROM runvars WHERE flight=? AND job=? AND name like '%job';
+END
+    my $buildq_txt = <<END;
+SELECT name FROM runvars WHERE flight=? AND job=? AND ('f'
+END
+    my @buildq_xvars;
+    foreach my $t (sort keys %treespecs) {
+	$buildq_txt .= " OR name=?";
+	push @buildq_xvars, "tree_$t";
+    }
+    $buildq_txt .= ")";
+    my $buildq = db_prepare($buildq_txt);
+
+    my %bjobs;
+
+    $testq->execute($eflight, $ejob);
+    while (my ($refvar, $bjobref) = $testq->fetchrow_array()) {
+	my ($bflight, $bjob) = flight_otherjob($eflight,$bjobref);
+	if (@refspecs) {
+	    my $y = 0;
+	    foreach my $rs (@refspecs) {
+		if ($rs eq ".$bjob" || $rs eq "=$refvar") {
+		    $y = 1;
+		    last;
+		}
+	    }
+	    next unless $y;
+	}
+	$buildq->execute($bflight,$bjob,@buildq_xvars);
+	my %got;
+	while (my ($treevar) = $buildq->fetchrow_array()) {
+	    $treevar =~ m/^tree_/ or die "$treevar ?";
+	    $got{$'} = 1;
+	    next unless $treespecs{$'};
+	    $treespecs{$'}[2]++;
+	}
+	next unless %got || @refspecs;
+
+	$bjobs{$bjob} //= { Template => $bflight };
+	$bjobs{$bjob}{Template} eq $bflight or
+	    die "inconsistent $bjob: $bjobs{$bjob}{Template} != $bflight";
+	push @{ $bjobs{$bjob}{Refs} }, $refvar;
+	$bjobs{$bjob}{Trees}{$_} = 1 foreach keys %got;
+    }
+
+    foreach my $tree (sort keys %treespecs) {
+	die "unused tree/revision adjustment $tree"
+	    .(@refspecs
+              ? " (no tree_$tree var in any of "
+              .(join ' ', map { "$bjobs{$_}{Template}.$_" }
+                sort keys %bjobs).")"
+              : '')
+            unless $treespecs{$tree}[2];
+    }
+
+    my @copy_jobs_qs = copy_jobs_qs();
+    foreach my $bjob (sort keys %bjobs) {
+	copy_jobs_do(\@copy_jobs_qs, $bjobs{$bjob}{Template}, $bjob);
+	foreach my $tree (sort keys %{ $bjobs{$bjob}{Trees} }) {
+	    runvar_set($bjob, "tree_$tree",     $treespecs{$tree}[0]);
+	    runvar_set($bjob, "revision_$tree", $treespecs{$tree}[1]);
+	}
+	foreach (@varspecs) {
+	    if (m/^-r!|^-r^/) {
+		$runvar_rm_q->execute($dstflight, $bjob, $');
+	    } elsif (m/^-r(.*?)=/) {
+		runvar_set($bjob, $1, $');
+	    } else {
+		die "bad -r $_ ?";
+	    }
+	}
+	foreach my $refvar (@{ $bjobs{$bjob}{Refs} }) {
+	    die "$bjob $refvar $& ?" if
+                "$bjob.$refvar" =~ m{[^+=_./,:0-9a-z-]}i;
+	    print "runvar-set . $refvar $dstflight.$bjob\n" or die $!;
+	}
+    }
+}
+
 sub changes () {
     debug("CHANGES...\n");
 
diff --git a/mg-repro-setup b/mg-repro-setup
index 2e1d3b88..abff2ef7 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -19,7 +19,7 @@
 
 usage () { cat <<END
 
-./mg-repro-setup [OPTION...] EXAMPLE-FLIGHT JOB TESTID [HOSTSPEC...]
+./mg-repro-setup [OPTION...] EXAMPLE-FLIGHT JOB TESTID [REBUILD|HOSTSPEC...]
 
  Creates a new flight containg a copy of JOB from EXAMPLE-FLIGHT
  adjusted to use the same builds as JOB (ie, no rebuilds),
@@ -42,6 +42,49 @@ usage () { cat <<END
    -E... -f... -P       as for mg-execute-flight
    --autoalloc-nofree   allocate hosts as for production, but keep them
                          (specify no HOSTSPECS; remember to deallocate later)
+
+ REBUILD is
+   --rebuild [-B<blessing>] [-r...]
+             [.BUILDJOB | =BUILDJOBRUNVAR...]
+             +TREE=URL#REVISION...
+       Also use a different build.  Specifically, use URL and REVISION
+       for certain builds which mention TREE and which are referenced 
+       (directly) in the job JOB in EXAMPLE-FLIGHT.  The affected builds
+       are those referenced by any of the runvars BUILDJOBRUNVAR (in
+       which case only that job reference is edited) or any whose job
+       name is any of the BUILDJOB (in which case all references to that
+       job are adjusted).  If no .BUILDJOB and no =BUILDJOBRUNVAR are
+       specified, then all jobs referred to from the example JOB
+       which mention TREE are affected.
+
+       For example,
+           --rebuild =xenbuildjob \
+                 +xen=git://xenbits.xen.org/people/alice/xen.git#fixes
+       would look for xenbuild and (say) find that it referred to
+       build-amd64; it would then copy and use the build-amd64 job
+       that EXAMPLE-FLIGHT.JOB used, adjusting \`tree_xen' and
+       \`revision_xen' as specified, and use that for \`xenbuildjob' in
+       the repro (but not for \`buildjob')
+
+       Whereas
+           --rebuild .build-amd64 \
+                 +linux=git://xenbits.xen.org/people/alice/linux.git#fixes
+       would replace all references to any job named build-amd64
+       with a new build-amd64 job.
+
+       And
+           --rebuild \
+                 +xen=git://xenbits.xen.org/people/alice/linux.git#fixes
+       would replace all builds mentioning xen, including for example
+       build-amd64 (for xenbuildjob) and build-i386 (for buildjob)
+       in an x86 32-on-64 flight.
+
+       Host allocation for the build is done "normally" (ie, the host
+       is thrown away after the build is complete), with a default
+       blessing of \`adhoc'.
+
+       --rebuild is affected by -E or -P but not general -B or -f.
+
 END
 
 }
@@ -122,10 +165,37 @@ progress "logging to $logfile"
 savelog "$logfile"
 exec 3>"$logfile"
 
+rebuilds_flight=''
+
 while [ $# -ne 0 ]; do
 	arg=$1; shift
 
 	case "$arg" in
+
+	--rebuild)
+		rebuilds_blessing=adhoc
+		rebuild_specs=()
+		while true; do
+			case "$1" in
+			[.=+]*|-r) rebuild_specs+=("$1");       shift ;;
+		        -B?*)	rebuilds_blessing=${1#-B};      shift ;;
+			-*)	badusage ': bad --rebuild option'     ;;
+                        *)      break                                 ;;
+			esac
+		done
+		if [ x$rebuilds_flight = x ]; then
+			rebuilds_flight=$(
+				./cs-adjust-flight new:$rebuilds_blessing
+			)
+		fi
+
+		adjusts+=($(
+			./cs-adjust-flight $rebuilds_flight \
+				repro-buildjobs $example_flight $job \
+                                "${rebuild_specs[@]}"
+		))
+		;;
+
 	none:)
 		# provided so we can repro a job with no hosts
 		;;
@@ -170,6 +240,18 @@ done
 flight=$(./cs-adjust-flight new:$blessing)
 progress "new flight is $flight"
 
+if [ "x$rebuilds_flight" != x ]; then
+	progress "running build(s) $rebuilds_flight"
+	./mg-execute-flight -B$rebuilds_blessing --progress-fd=2 \
+		"${mgexecflags[@]}" $rebuilds_flight
+
+	mro=tmp/$rebuilds_flight.mro
+	if ! egrep '^perfect' >/dev/null $mro; then
+		echo >&2 "build(s) failed (no 'perfect' in $mro)"
+		exit 1
+	fi
+fi
+
 OSSTEST_TASK=$(perl -e '
 	use Osstest;
 	use Osstest::Executive;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [OSSTEST PATCH 9/9] mg-repro-flight: Provide --rebuild to make variant build jobs too

[Ian Jackson]

This allows a single command to repro a particular job with a variety
of different source code.

The implementation technique is:
  - run the build job in a separate flight, so that it can run
    with a separate task which gives its host up after the build
  - do much of the heavy lifting of runvar fiddling etc. in
    a new helper routine in cs-adjust-flight

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 cs-adjust-flight | 109 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 mg-repro-setup   |  84 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 192 insertions(+), 1 deletion(-)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index cc1684b4..5be5af38 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -469,6 +469,115 @@ END
 	$runvarq->execute($newjob, $dstflight, $oldjob);
     })
 }
+
+sub change__repro_buildjobs {
+    # helper for mg-repro-setup
+    # arguments to this change are
+    #    EXAMPLE-FLIGHT EXAMPLE-JOB THING...
+    # where THING is
+    #    .BUILDJOB =BUILDJOBRUNVAR +TREE=URL#REVISION
+    # (we eat all until it doesn't start with . = +)
+    # stdout output is
+    #    list of runvar settings for repro job
+    die unless @changes >= 2;
+    my $eflight = shift @changes;
+    my $ejob = shift @changes;
+    my @refspecs;
+    my @varspecs;
+    my %treespecs;
+    while (@changes && $changes[0] =~ m/^[.=+]/) {
+	local $_ = shift @changes;
+	if (m/^[.=]/) {
+	    push @refspecs, $_;
+	} elsif (m/^\+(.*?)\=(.*)\#(.*)$/) {
+	    die $1 if $treespecs{$1};
+	    $treespecs{$1} = [$2,$3];
+	} elsif (m/^-r/) {
+	    push @varspecs, $_;
+	} else {
+	    die "bad .BUILDJOB / =BUILDJOBRUNVAR / +TREE=URL#REVISION): $_\n";
+	}
+    }
+    my $testq = db_prepare(<<END);
+SELECT name, val FROM runvars WHERE flight=? AND job=? AND name like '%job';
+END
+    my $buildq_txt = <<END;
+SELECT name FROM runvars WHERE flight=? AND job=? AND ('f'
+END
+    my @buildq_xvars;
+    foreach my $t (sort keys %treespecs) {
+	$buildq_txt .= " OR name=?";
+	push @buildq_xvars, "tree_$t";
+    }
+    $buildq_txt .= ")";
+    my $buildq = db_prepare($buildq_txt);
+
+    my %bjobs;
+
+    $testq->execute($eflight, $ejob);
+    while (my ($refvar, $bjobref) = $testq->fetchrow_array()) {
+	my ($bflight, $bjob) = flight_otherjob($eflight,$bjobref);
+	if (@refspecs) {
+	    my $y = 0;
+	    foreach my $rs (@refspecs) {
+		if ($rs eq ".$bjob" || $rs eq "=$refvar") {
+		    $y = 1;
+		    last;
+		}
+	    }
+	    next unless $y;
+	}
+	$buildq->execute($bflight,$bjob,@buildq_xvars);
+	my %got;
+	while (my ($treevar) = $buildq->fetchrow_array()) {
+	    $treevar =~ m/^tree_/ or die "$treevar ?";
+	    $got{$'} = 1;
+	    next unless $treespecs{$'};
+	    $treespecs{$'}[2]++;
+	}
+	next unless %got || @refspecs;
+
+	$bjobs{$bjob} //= { Template => $bflight };
+	$bjobs{$bjob}{Template} eq $bflight or
+	    die "inconsistent $bjob: $bjobs{$bjob}{Template} != $bflight";
+	push @{ $bjobs{$bjob}{Refs} }, $refvar;
+	$bjobs{$bjob}{Trees}{$_} = 1 foreach keys %got;
+    }
+
+    foreach my $tree (sort keys %treespecs) {
+	die "unused tree/revision adjustment $tree"
+	    .(@refspecs
+              ? " (no tree_$tree var in any of "
+              .(join ' ', map { "$bjobs{$_}{Template}.$_" }
+                sort keys %bjobs).")"
+              : '')
+            unless $treespecs{$tree}[2];
+    }
+
+    my @copy_jobs_qs = copy_jobs_qs();
+    foreach my $bjob (sort keys %bjobs) {
+	copy_jobs_do(\@copy_jobs_qs, $bjobs{$bjob}{Template}, $bjob);
+	foreach my $tree (sort keys %{ $bjobs{$bjob}{Trees} }) {
+	    runvar_set($bjob, "tree_$tree",     $treespecs{$tree}[0]);
+	    runvar_set($bjob, "revision_$tree", $treespecs{$tree}[1]);
+	}
+	foreach (@varspecs) {
+	    if (m/^-r!|^-r^/) {
+		$runvar_rm_q->execute($dstflight, $bjob, $');
+	    } elsif (m/^-r(.*?)=/) {
+		runvar_set($bjob, $1, $');
+	    } else {
+		die "bad -r $_ ?";
+	    }
+	}
+	foreach my $refvar (@{ $bjobs{$bjob}{Refs} }) {
+	    die "$bjob $refvar $& ?" if
+                "$bjob.$refvar" =~ m{[^+=_./,:0-9a-z-]}i;
+	    print "runvar-set . $refvar $dstflight.$bjob\n" or die $!;
+	}
+    }
+}
+
 sub changes () {
     debug("CHANGES...\n");
 
diff --git a/mg-repro-setup b/mg-repro-setup
index 2e1d3b88..abff2ef7 100755
--- a/mg-repro-setup
+++ b/mg-repro-setup
@@ -19,7 +19,7 @@
 
 usage () { cat <<END
 
-./mg-repro-setup [OPTION...] EXAMPLE-FLIGHT JOB TESTID [HOSTSPEC...]
+./mg-repro-setup [OPTION...] EXAMPLE-FLIGHT JOB TESTID [REBUILD|HOSTSPEC...]
 
  Creates a new flight containg a copy of JOB from EXAMPLE-FLIGHT
  adjusted to use the same builds as JOB (ie, no rebuilds),
@@ -42,6 +42,49 @@ usage () { cat <<END
    -E... -f... -P       as for mg-execute-flight
    --autoalloc-nofree   allocate hosts as for production, but keep them
                          (specify no HOSTSPECS; remember to deallocate later)
+
+ REBUILD is
+   --rebuild [-B<blessing>] [-r...]
+             [.BUILDJOB | =BUILDJOBRUNVAR...]
+             +TREE=URL#REVISION...
+       Also use a different build.  Specifically, use URL and REVISION
+       for certain builds which mention TREE and which are referenced 
+       (directly) in the job JOB in EXAMPLE-FLIGHT.  The affected builds
+       are those referenced by any of the runvars BUILDJOBRUNVAR (in
+       which case only that job reference is edited) or any whose job
+       name is any of the BUILDJOB (in which case all references to that
+       job are adjusted).  If no .BUILDJOB and no =BUILDJOBRUNVAR are
+       specified, then all jobs referred to from the example JOB
+       which mention TREE are affected.
+
+       For example,
+           --rebuild =xenbuildjob \
+                 +xen=git://xenbits.xen.org/people/alice/xen.git#fixes
+       would look for xenbuild and (say) find that it referred to
+       build-amd64; it would then copy and use the build-amd64 job
+       that EXAMPLE-FLIGHT.JOB used, adjusting \`tree_xen' and
+       \`revision_xen' as specified, and use that for \`xenbuildjob' in
+       the repro (but not for \`buildjob')
+
+       Whereas
+           --rebuild .build-amd64 \
+                 +linux=git://xenbits.xen.org/people/alice/linux.git#fixes
+       would replace all references to any job named build-amd64
+       with a new build-amd64 job.
+
+       And
+           --rebuild \
+                 +xen=git://xenbits.xen.org/people/alice/linux.git#fixes
+       would replace all builds mentioning xen, including for example
+       build-amd64 (for xenbuildjob) and build-i386 (for buildjob)
+       in an x86 32-on-64 flight.
+
+       Host allocation for the build is done "normally" (ie, the host
+       is thrown away after the build is complete), with a default
+       blessing of \`adhoc'.
+
+       --rebuild is affected by -E or -P but not general -B or -f.
+
 END
 
 }
@@ -122,10 +165,37 @@ progress "logging to $logfile"
 savelog "$logfile"
 exec 3>"$logfile"
 
+rebuilds_flight=''
+
 while [ $# -ne 0 ]; do
 	arg=$1; shift
 
 	case "$arg" in
+
+	--rebuild)
+		rebuilds_blessing=adhoc
+		rebuild_specs=()
+		while true; do
+			case "$1" in
+			[.=+]*|-r) rebuild_specs+=("$1");       shift ;;
+		        -B?*)	rebuilds_blessing=${1#-B};      shift ;;
+			-*)	badusage ': bad --rebuild option'     ;;
+                        *)      break                                 ;;
+			esac
+		done
+		if [ x$rebuilds_flight = x ]; then
+			rebuilds_flight=$(
+				./cs-adjust-flight new:$rebuilds_blessing
+			)
+		fi
+
+		adjusts+=($(
+			./cs-adjust-flight $rebuilds_flight \
+				repro-buildjobs $example_flight $job \
+                                "${rebuild_specs[@]}"
+		))
+		;;
+
 	none:)
 		# provided so we can repro a job with no hosts
 		;;
@@ -170,6 +240,18 @@ done
 flight=$(./cs-adjust-flight new:$blessing)
 progress "new flight is $flight"
 
+if [ "x$rebuilds_flight" != x ]; then
+	progress "running build(s) $rebuilds_flight"
+	./mg-execute-flight -B$rebuilds_blessing --progress-fd=2 \
+		"${mgexecflags[@]}" $rebuilds_flight
+
+	mro=tmp/$rebuilds_flight.mro
+	if ! egrep '^perfect' >/dev/null $mro; then
+		echo >&2 "build(s) failed (no 'perfect' in $mro)"
+		exit 1
+	fi
+fi
+
 OSSTEST_TASK=$(perl -e '
 	use Osstest;
 	use Osstest::Executive;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel