[OSSTEST PATCH 0/5] Email improvements

[OSSTEST PATCH 0/5] Email improvements

[Ian Jackson]

This miniature series provides documentation of the main email
reports, and some osstest-generated headers to assist with email
filtering.

Ian.

[OSSTEST PATCH 1/5] sg-report-flight: Use `bodyprint' and `bodyprintf'

[Ian Jackson]

This will allows us to divert the output so that we can (optionally)
produce email headers too.  We move the error checking into the
function, obviously.

For now, this is a simple replacement with no functional change,
except that some call sites now have error checking which previously
did not.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-report-flight |   53 ++++++++++++++++++++++++++++-------------------------
 1 file changed, 28 insertions(+), 25 deletions(-)

diff --git a/sg-report-flight b/sg-report-flight
index 1f69c08..f9f6e24 100755
--- a/sg-report-flight
+++ b/sg-report-flight
@@ -490,11 +490,14 @@ END
     };
 }
 
+sub bodyprint { print @_ or die $!; }
+sub bodyprintf { printf @_ or die $!; }
+
 sub printversions ($) {
     my ($thisthat) = @_;
     my ($sv) = $specver{$thisthat};
     foreach my $k (sort keys %$sv) {
-        printf " %-${tl}s %s\n", $k, $sv->{$k} or die $!;
+        bodyprintf " %-${tl}s %s\n", $k, $sv->{$k};
         print MRO "version $thisthat $k $sv->{$k}\n" or die $!;
     }
 }
@@ -512,25 +515,25 @@ sub print_pushgate_summary () {
 	[ $blessings[0] ], $maxflight, [ $branch ],
 	$tree, $specver{that}{$tree}, $specver{this}{$tree}
 	);
-    print "\n";
+    bodyprint "\n";
     my $now = time;
     my $pinfo = sub {
 	my ($what, $flightkey, $countkey) = @_;
 	my $f = $info->{$flightkey};
 	my $count = $info->{$countkey};
-	printf "%-20s", $what;
+	bodyprintf "%-20s", $what;
 	if ($f) {
-	    printf(" %6d  %s %4d days",
+	    bodyprintf(" %6d  %s %4d days",
 		   $f->{flight},
 		   show_abs_time($f->{started}),
 		   floor(($now - $f->{started}) / 86400));
 	} else {
-	    printf(" %34s     ","(not found)");
+	    bodyprintf(" %34s     ","(not found)");
 	}
 	if (defined $count) {
-	    printf(" %4d attempts", $count);
+	    bodyprintf(" %4d attempts", $count);
 	}
-	print "\n";
+	bodyprint "\n";
     };
     $pinfo->('Last test of basis', "Basis",'');
     $pinfo->('Failing since', "FirstAfterBasis","CountAfterBasis")
@@ -541,7 +544,7 @@ sub print_pushgate_summary () {
 
 sub printout {
     my ($r, @failures) = @_;
-    print <<END or die $!;
+    bodyprint <<END;
 $r->{Flight}: $r->{OutcomeSummary}
 
 END
@@ -549,13 +552,13 @@ END
         STDOUT->flush();
         system_checked('cat','--',$include);
     }
-    print <<END or die $!;
+    bodyprint <<END;
 flight $r->{Flight} $branch $r->{FlightInfo}{blessing} [$r->{FlightInfo}{intended}]
 $c{ReportHtmlPubBaseUrl}/$r->{Flight}/
 END
     if (defined $r->{Overall}) {
-        print "\n" or die $!;
-        print $r->{Overall} or die $!;
+        bodyprint "\n";
+        bodyprint $r->{Overall};
     }
 
     my %notsucceeds;
@@ -599,44 +602,44 @@ END
     }
     foreach my $cat (sort keys %notsucceeds) {
         $cat =~ m/^\w+ / or die;
-        print "\n$'\n" or die $!;
+        bodyprint "\n$'\n";
 	foreach (sort { $a->[0] cmp $b->[0] } @{ $notsucceeds{$cat} }) {
-	    print $_->[1], "\n" or die $!;
+	    bodyprint $_->[1], "\n";
 	}
     }
 
     if (!%{ $r->{Failures} }) {
-        print "All tests in this flight passed" or die $!;
+        bodyprint "All tests in this flight passed";
     }
-    print "\n" or die $!;
+    bodyprint "\n";
 
     if (keys %{ $specver{this} }) {
-        print "version targeted for testing:\n" or die $!;
+        bodyprint "version targeted for testing:\n";
         printversions('this');
     }
     if (keys %{ $specver{that} }) {
-        print "baseline version:\n" or die $!;
+        bodyprint "baseline version:\n";
         printversions('that');
     }
 
     print_pushgate_summary();
 
     if (@includefiles) {
-        print "\n","-"x60, "\n" or die $!;
+        bodyprint "\n","-"x60, "\n";
     }
     foreach my $include (@includefiles) {
         STDOUT->flush();
         system_checked('cat','--',$include);
-        print "-"x60, "\n" or die $!;
+        bodyprint "-"x60, "\n";
     }
 
-    print "\njobs:\n", $r->{JobsSummary}, "\n" or die $!;
+    bodyprint "\njobs:\n", $r->{JobsSummary}, "\n";
 
 #    foreach my $jt (@{ $r->{JobTexts} }) {
-#	print '-'x$cw or die $!;
-#        print "\n" or die $!;
-#        print join "\n", @$jt or die $!;
-#        print "\n" or die $!;
+#	bodyprint '-'x$cw;
+#        bodyprint "\n";
+#        bodyprint join "\n", @$jt;
+#        bodyprint "\n";
 #    }
 
     my $hostname= `hostname -f`;
@@ -644,7 +647,7 @@ END
 
     my $logs= $c{Logs};
 
-    print <<END or die $!;
+    bodyprint <<END;
 
 ------------------------------------------------------------
 sg-report-flight on $hostname
-- 
1.7.10.4

[OSSTEST PATCH 2/5] sg-report-flight: Make bodyprint[f] defer the output

[Ian Jackson]

Provide and use variables $header_text and $body_text.

Still no functional change.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-report-flight |   16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/sg-report-flight b/sg-report-flight
index f9f6e24..bfc8787 100755
--- a/sg-report-flight
+++ b/sg-report-flight
@@ -490,8 +490,11 @@ END
     };
 }
 
-sub bodyprint { print @_ or die $!; }
-sub bodyprintf { printf @_ or die $!; }
+our $header_text;
+our $body_text;
+
+sub bodyprint { $body_text .= $_ foreach @_; }
+sub bodyprintf { my $fmt = shift @_; $body_text .= sprintf $fmt, @_; }
 
 sub printversions ($) {
     my ($thisthat) = @_;
@@ -544,9 +547,12 @@ sub print_pushgate_summary () {
 
 sub printout {
     my ($r, @failures) = @_;
-    bodyprint <<END;
-$r->{Flight}: $r->{OutcomeSummary}
+    $header_text = '';
+    $body_text = '';
 
+    # Caller expects to provide `Subject: ', so this must come first.
+    $header_text .= <<END or die $!;
+$r->{Flight}: $r->{OutcomeSummary}
 END
     foreach my $include (@includebeginfiles) {
         STDOUT->flush();
@@ -656,6 +662,8 @@ images: $c{Images}
 
 $c{ReportTrailer}
 END
+
+    print $header_text, "\n", $body_text;
 }
 
 our %heisenjustified;
-- 
1.7.10.4

[OSSTEST PATCH 3/5] Emails: Provide README.email and link to it

[Ian Jackson]

Also link to README, from the email footer.

I have tested that the link to README works right now.  The link to
README.email does not yet, of course, but it will start to work when
osstest pushes this change to production.

CC: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

squash! Provide README.email
---
 README.email      |  106 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 production-config |    4 ++
 2 files changed, 110 insertions(+)
 create mode 100644 README.email

diff --git a/README.email b/README.email
new file mode 100644
index 0000000..3f95fa8
--- /dev/null
+++ b/README.email
@@ -0,0 +1,106 @@
+Email reports
+=============
+
+osstest's primary output is the email reports to the mailing list,
+along with the associated HTML scoreboards.
+
+Subject line
+------------
+
+  Subject: [xen-4.4-testing test] 58975: tolerable trouble: broken/pass - PUSHED
+            ^^^^^^^^^^^^^^^ ^^^^  ^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^	  ^^^^^^
+       	    branch          |  	  flight  overall outcome         	    |
+                            |	       			     	       push mark
+ 		  always exactly "test"
+
+
+The overall outcome is preceded by `tolerable' if the there were no
+excusable regressions, or `regressions - ' if there were.
+
+The overall outcome is then one of:
+   all pass
+   FAIL
+   ALL FAIL
+   trouble: <set of individual job outcomes>
+
+
+Main mail body listing failed tests
+-----------------------------------
+
+The main top part of the body lists test failures.  Each line lists a
+failing test step.  These are categorised by how much of problem the
+failure is.
+
+Each line looks something like this:
+
+ test-amd64-i386-xl-qemuu-win7-amd64 14 guest-localmigrate.2 fail pass in 58960
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^ ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
+     job                              |    step testid       results
+                          step number-'
+
+The results normally start with the result in this flight, and can be
+followed by various information about the status of this step in the
+history.  Here are some examples:
+
+   fail REGR. vs. 58530
+
+      This step failed just now; whereas it previously passed in the
+      baseline (in flight 58530).  This is a regression, which is
+      normally blocking.
+
+   fail like 58963
+
+      This step failed just now; but it also failed on the baseline,
+      in flight 58963.  So it is not a regression.
+
+   fail never pass
+
+      This step has never passed on this branch and is not a
+      regression.
+
+   fail pass in 58863
+
+      This step failed just now; but it passed in 58863 which was
+      another test of the same version.  Such failures are not
+      blocking.
+
+   fail baseline untested
+
+      The results processor didn't find any executions of this step in
+      tests of the baseline version.  So it would not be able to
+      detect regressions of this test.  Perhaps the test has been
+      recently introduced.
+
+   fail in 58948 pass in 58965
+   fail in 58948 like 37628
+
+      The results processor used 58948 (another flight testing the
+      just-tested version) to convince itself that some other test is
+      failing intermittently.  However, in 58948, this step failed,
+      which would be a problem.  But this step passed in 58965, or
+      failed in 37628, so it is not a blocker (cf `pass in' and like',
+      above).
+
+   broken REGR. vs. 36514
+
+      The step failed and this is probably an indication of a problem
+      with the test infrastructure.  Such failures are always regarded
+      as regressions because they may mask other problems.
+
+   blocked n/a
+
+      The test step was blocked by a previous failure.  This is not
+      regarded as a regression - instead, the underlying failure is
+      checked for regression.
+
+   queued
+   running
+   preparing
+
+      The flight was cancelled before it finished executing.  (Also
+      appears if a report is manually requested on a still-running
+      flight.)
+
+The flight numbers quoted in these step failure summaries may
+sometimes be suffixed with `-bisect' to show that the indicated flight
+was part of a bisection attempt, rather than a main test flight.
diff --git a/production-config b/production-config
index 0a528e7..052314d 100644
--- a/production-config
+++ b/production-config
@@ -68,6 +68,10 @@ ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}
 
+Explanation of these reports, and of osstest in general, is at
+    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
+    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master
+
 Test harness code can be found at
     http://xenbits.xen.org/gitweb?p=osstest.git;a=summary
 END
-- 
1.7.10.4

[OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers

[Ian Jackson]

Specifically:
 * Provide headerprint() in sg-report-flight
 * Use it to add X-Osstest-Versions headers based on %specver
 * Cause cr-daily-branch to always enable the feature
 * Document this in README.email

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 README.email     |   14 ++++++++++++++
 cr-daily-branch  |    2 +-
 sg-report-flight |   10 ++++++++++
 3 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/README.email b/README.email
index 3f95fa8..be79f88 100644
--- a/README.email
+++ b/README.email
@@ -104,3 +104,17 @@ history.  Here are some examples:
 The flight numbers quoted in these step failure summaries may
 sometimes be suffixed with `-bisect' to show that the indicated flight
 was part of a bisection attempt, rather than a main test flight.
+
+
+Email information headers
+-------------------------
+
+Test report emails sent by osstest have some extra information in the
+email headers, where applicable:
+
+X-Osstest-Versions-This:
+    osstest=cc8b79ce9586d2b0fbddbd4260e876eab1d408d4
+X-Osstest-Versions-That:
+    osstest=280a18b2b2612174e473a1c1a137ddd47c49aab9
+    ^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+     tree     revision
diff --git a/cr-daily-branch b/cr-daily-branch
index c7d1071..34b6d2b 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -246,7 +246,7 @@ $DAILY_BRANCH_POSTMAKE_HOOK
 
 heading=tmp/$flight.heading-info
 : >$heading
-sgr_args+=" --include-begin=$heading"
+sgr_args+=" --info-headers --include-begin=$heading"
 
 if $force_baseline; then
 	subject_prefix="[$branch baseline test] "
diff --git a/sg-report-flight b/sg-report-flight
index bfc8787..76d6ab7 100755
--- a/sg-report-flight
+++ b/sg-report-flight
@@ -37,6 +37,7 @@ our %specver;
 our (@blessings, @branches_also);
 our $mro;
 our $htmldir;
+our $want_info_headers;
 our ($branch, $branches_cond_q);
 our @allows;
 our (@includebeginfiles,@includefiles);
@@ -56,6 +57,8 @@ while (@ARGV && $ARGV[0] =~ m/^-/) {
         open MRO, "> $mro.new" or die "$mro.new $!";
     } elsif (m/^--html-dir=(.*)$/) {
         $htmldir= $1;
+    } elsif (m/^--info-headers$/) {
+        $want_info_headers = 1;
     } elsif (m/^--blessings?=(.*)$/) {
         push @blessings, split ',', $1;
     } elsif (m/^--branches-also?=(.*)$/) {
@@ -496,11 +499,18 @@ our $body_text;
 sub bodyprint { $body_text .= $_ foreach @_; }
 sub bodyprintf { my $fmt = shift @_; $body_text .= sprintf $fmt, @_; }
 
+sub headerprint {
+    return unless $want_info_headers;
+    $header_text .= $_ foreach @_;
+}
+
 sub printversions ($) {
     my ($thisthat) = @_;
     my ($sv) = $specver{$thisthat};
+    headerprint "X-Osstest-Versions-\u$thisthat:\n" if %$sv;
     foreach my $k (sort keys %$sv) {
         bodyprintf " %-${tl}s %s\n", $k, $sv->{$k};
+	headerprint "    $k=$sv->{$k}\n";
         print MRO "version $thisthat $k $sv->{$k}\n" or die $!;
     }
 }
-- 
1.7.10.4

[OSSTEST PATCH 5/5] Emails: Provide X-Osstest-Failures

[Ian Jackson]

Specifically:
 * $cat now contains SORTLETTER BLOCKSTATUS DESCRIPTION...
 * Rewrap the lines setting $cat
 * Add $notsucceeds{$cat}[][2] containing one line for the new header
 * Generate the new header
 * Document the new header

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 README.email     |   21 +++++++++++++++++++++
 sg-report-flight |   26 +++++++++++++++++++-------
 2 files changed, 40 insertions(+), 7 deletions(-)

diff --git a/README.email b/README.email
index be79f88..6dc992a 100644
--- a/README.email
+++ b/README.email
@@ -112,6 +112,27 @@ Email information headers
 Test report emails sent by osstest have some extra information in the
 email headers, where applicable:
 
+X-Osstest-Failures:
+    osstest:test-armhf-armhf-xl-midway:guest-start:fail:heisenbug
+    osstest:test-amd64-amd64-xl-rtds:xen-boot:fail:blocked
+    ^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^ ^^^^^^^
+    branch    job                       |      |    |
+			      step testid      |    analysis
+				               |
+				     step status
+
+The `analysis' field may be:
+
+   regression   "Tests which did not succeed and are blocking"
+   heisenbug    "Tests which are failing intermittently (not blocking)"
+   allowable    "Regressions which are regarded as allowable"
+   nonblocking  "Tests which did not succeed, but are not blocking"
+   fail         "Tests which did not succeed" (non-regression-test flights)
+
+When using email filtering, you should filter on the relevant From:
+line as well as X-Osstest-Failures, as there is nothing in
+X-Osstest-Failures identifying the instance.
+
 X-Osstest-Versions-This:
     osstest=cc8b79ce9586d2b0fbddbd4260e876eab1d408d4
 X-Osstest-Versions-That:
diff --git a/sg-report-flight b/sg-report-flight
index 76d6ab7..a5b9e6c 100755
--- a/sg-report-flight
+++ b/sg-report-flight
@@ -587,18 +587,21 @@ END
         my $heisenflightp= $failv->{Flight} != $specflight;
         print DEBUG "PRINTOUT ",Dumper($failv);
         if (!keys %{ $specver{that} }) {
-            $cat= "A Tests which did not succeed,\n".
+            $cat= "A fail ".
+"Tests which did not succeed,\n".
 "including tests which could not be run:";
         } elsif ($failv->{Blocker}) {
-            $cat= "A Tests which did not succeed and are blocking,\n".
+            $cat= "A regression ".
+"Tests which did not succeed and are blocking,\n".
 "including tests which could not be run:";
         } elsif ($failv->{Heisen}) {
-            $cat= "K Tests which are failing intermittently (not blocking):";
+            $cat= "K heisenbug ".
+"Tests which are failing intermittently (not blocking):";
         } elsif ($failv->{Allow}) {
-            $cat= "M ".
+            $cat= "M allowable ".
 "Regressions which are regarded as allowable (not blocking):";
         } else {
-            $cat= "P ".
+            $cat= "P nonblocking ".
 "Tests which did not succeed, but are not blocking:";
         }
         my $heisencat= \$heisenreported_jobstep{"$j->{job} $s->{testid}"};
@@ -614,13 +617,22 @@ END
         $text =~ s/ *$//;
         while (length($text) > $cw) { last unless $text =~ s/(.* ) /$1/; }
 
-	push @{ $notsucceeds{$cat} }, [ "$s->{status} $xstatus", $text ];
+	$cat =~ m/\w+ (\S+) / or die "$cat ?";
+	my $blockstatus = $1;
+
+	push @{ $notsucceeds{$cat} }, [
+	    "$s->{status} $xstatus",
+	    $text,
+	    "$branch:$j->{job}:$s->{testid}:$s->{status}:$blockstatus",
+	];
     }
+    headerprint "X-Osstest-Failures:\n" if %notsucceeds;
     foreach my $cat (sort keys %notsucceeds) {
-        $cat =~ m/^\w+ / or die;
+        $cat =~ m/^\w+ \S+ / or die;
         bodyprint "\n$'\n";
 	foreach (sort { $a->[0] cmp $b->[0] } @{ $notsucceeds{$cat} }) {
 	    bodyprint $_->[1], "\n";
+	    headerprint "    $_->[2]\n";
 	}
     }
 
-- 
1.7.10.4

Re: [OSSTEST PATCH 0/5] Email improvements

[Ian Campbell]

On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> This miniature series

It says something about this corner of the world when 5 patches is
considered miniature!

Re: [OSSTEST PATCH 1/5] sg-report-flight: Use `bodyprint' and `bodyprintf'

[Ian Campbell]

On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> This will allows us to divert the output so that we can (optionally)
> produce email headers too.  We move the error checking into the
> function, obviously.
> 
> For now, this is a simple replacement with no functional change,
> except that some call sites now have error checking which previously
> did not.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 2/5] sg-report-flight: Make bodyprint[f] defer the output

[Ian Campbell]

On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> Provide and use variables $header_text and $body_text.
> 
> Still no functional change.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 3/5] Emails: Provide README.email and link to it

[Ian Campbell]

On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> Also link to README, from the email footer.
> 
> I have tested that the link to README works right now.  The link to
> README.email does not yet, of course, but it will start to work when
> osstest pushes this change to production.
> 
> CC: Andrew Cooper <andrew.cooper3@citrix.com>
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> 
> squash! Provide README.email

pumpkin!

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers

[Ian Campbell]

On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> Specifically:
>  * Provide headerprint() in sg-report-flight
>  * Use it to add X-Osstest-Versions headers based on %specver
>  * Cause cr-daily-branch to always enable the feature
>  * Document this in README.email
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> ---
>  README.email     |   14 ++++++++++++++
>  cr-daily-branch  |    2 +-
>  sg-report-flight |   10 ++++++++++
>  3 files changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/README.email b/README.email
> index 3f95fa8..be79f88 100644
> --- a/README.email
> +++ b/README.email
> @@ -104,3 +104,17 @@ history.  Here are some examples:
>  The flight numbers quoted in these step failure summaries may
>  sometimes be suffixed with `-bisect' to show that the indicated flight
>  was part of a bisection attempt, rather than a main test flight.
> +
> +
> +Email information headers
> +-------------------------
> +
> +Test report emails sent by osstest have some extra information in the
> +email headers, where applicable:
> +
> +X-Osstest-Versions-This:
> +    osstest=cc8b79ce9586d2b0fbddbd4260e876eab1d408d4
> +X-Osstest-Versions-That:
> +    osstest=280a18b2b2612174e473a1c1a137ddd47c49aab9
> +    ^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +     tree     revision

Some description of what this and that mean might be useful for those in
the know? Specifically that that is the baseline.

Also, I think in real life several trees might be included?

Re: [OSSTEST PATCH 5/5] Emails: Provide X-Osstest-Failures

[Ian Campbell]

On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> Specifically:
>  * $cat now contains SORTLETTER BLOCKSTATUS DESCRIPTION...
>  * Rewrap the lines setting $cat
>  * Add $notsucceeds{$cat}[][2] containing one line for the new header
>  * Generate the new header
>  * Document the new header
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers"):
> On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> > +Email information headers
> > +-------------------------
> > +
> > +Test report emails sent by osstest have some extra information in the
> > +email headers, where applicable:
> > +
> > +X-Osstest-Versions-This:
> > +    osstest=cc8b79ce9586d2b0fbddbd4260e876eab1d408d4
> > +X-Osstest-Versions-That:
> > +    osstest=280a18b2b2612174e473a1c1a137ddd47c49aab9
> > +    ^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +     tree     revision
> 
> Some description of what this and that mean might be useful for those in
> the know? Specifically that that is the baseline.

I have added:

| `This' is the version being tested, and `That' is the baseline.

> Also, I think in real life several trees might be included?

Not with the current state of cr-daily-branch.

Ian.

Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers

[Ian Campbell]

On Tue, 2015-06-30 at 15:42 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers"):
> > On Tue, 2015-06-30 at 14:49 +0100, Ian Jackson wrote:
> > > +Email information headers
> > > +-------------------------
> > > +
> > > +Test report emails sent by osstest have some extra information in the
> > > +email headers, where applicable:
> > > +
> > > +X-Osstest-Versions-This:
> > > +    osstest=cc8b79ce9586d2b0fbddbd4260e876eab1d408d4
> > > +X-Osstest-Versions-That:
> > > +    osstest=280a18b2b2612174e473a1c1a137ddd47c49aab9
> > > +    ^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > +     tree     revision
> > 
> > Some description of what this and that mean might be useful for those in
> > the know? Specifically that that is the baseline.
> 
> I have added:
> 
> | `This' is the version being tested, and `That' is the baseline.

Acked-by: Ian Campbell <ian.campbell@citrix.com>

> > Also, I think in real life several trees might be included?
> 
> Not with the current state of cr-daily-branch.

Nor bisects?

Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers

[Ian Jackson]

Ian Campbell writes ("Re: [OSSTEST PATCH 4/5] Emails: Provide X-Osstest-Versions-This and That headers"):
> On Tue, 2015-06-30 at 15:42 +0100, Ian Jackson wrote:
> > I have added:
> > 
> > | `This' is the version being tested, and `That' is the baseline.
> 
> Acked-by: Ian Campbell <ian.campbell@citrix.com>

Thanks.

> > > Also, I think in real life several trees might be included?
> > 
> > Not with the current state of cr-daily-branch.
> 
> Nor bisects?

The bisection emails do not contain any extra header information (even
after these patches), nor are they documented yet.

Ian.