xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
* [OSSTEST PATCH 1/5] ms-flights-summary: Do gather_stats on jobs a bit later
@ 2016-07-18 14:04 Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 2/5] ms-flights-summary: Do gather_stats without GROUP BY Ian Jackson
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Ian Jackson @ 2016-07-18 14:04 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Jackson

We are going to want to invent a fake status for `preparing' jobs
which have no allocation, separate from ones which do have an
allocation.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ms-flights-summary | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ms-flights-summary b/ms-flights-summary
index ea72088..413b703 100755
--- a/ms-flights-summary
+++ b/ms-flights-summary
@@ -342,11 +342,14 @@ enumerate_flights();
 
 foreach my $f (keys %flights) {
     enumerate_jobs($flights{$f});
-    gather_stats($flights{$f});
 }
 
 gather_events();
 
+foreach my $f (keys %flights) {
+    gather_stats($flights{$f});
+}
+
 printf("<html><head><title>Flights - %s</title></head><body>\n",
        $c{DnsDomain} // '?');
 printf("<p>Report at ".fmttime(time)." based on plan at ".fmttime($plan->{Start})."</p>\n");
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 5+ messages in thread
* [OSSTEST PATCH 2/5] ms-flights-summary: Do gather_stats without GROUP BY
  2016-07-18 14:04 [OSSTEST PATCH 1/5] ms-flights-summary: Do gather_stats on jobs a bit later Ian Jackson
@ 2016-07-18 14:04 ` Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 3/5] ms-flights-summary: Invent a `prep.alloc.' pseudo job state Ian Jackson
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Ian Jackson @ 2016-07-18 14:04 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Jackson

We are going to want to look at each job's Reso separately.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ms-flights-summary | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/ms-flights-summary b/ms-flights-summary
index 413b703..98cca90 100755
--- a/ms-flights-summary
+++ b/ms-flights-summary
@@ -119,14 +119,13 @@ sub gather_stats($) {
     my ($f) = @_;
 
     my $stats= $dbh_tests->selectall_arrayref(<<END);
-        SELECT status,COUNT(*) FROM jobs
+        SELECT job,status FROM jobs
 	    WHERE flight=$f->{Nr}
-            GROUP BY status
 END
     for my $row (@{$stats}) {
-	my ($stat,$count) = @$row;
-	$f->{Stats}{lc($stat)} = $count;
-	$global_stats{lc($stat)} += $count;
+	my ($job,$stat) = @$row;
+	$f->{Stats}{lc($stat)}++;
+	$global_stats{lc($stat)}++;
     }
 
     $f->{NrJobs} = keys %{$f->{Jobs}};
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 5+ messages in thread
* [OSSTEST PATCH 3/5] ms-flights-summary: Invent a `prep.alloc.' pseudo job state
  2016-07-18 14:04 [OSSTEST PATCH 1/5] ms-flights-summary: Do gather_stats on jobs a bit later Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 2/5] ms-flights-summary: Do gather_stats without GROUP BY Ian Jackson
@ 2016-07-18 14:04 ` Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 4/5] ms-flights-summary: Treat preparing jobs outside the plan as `queued' Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 5/5] sg-execute-flight: Fix an error message Ian Jackson
  3 siblings, 0 replies; 5+ messages in thread
From: Ian Jackson @ 2016-07-18 14:04 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Jackson

This allows us to separate out `preparing' jobs into ones which are in
our data plan and ones which are not.  The ones which are not may not
have quite started to run ts-hosts-allocate, or may still be in the
planning queue and not made it into the projection.

In either case we don't have an estimated finish time for them.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ms-flights-summary | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/ms-flights-summary b/ms-flights-summary
index 98cca90..9a814ea 100755
--- a/ms-flights-summary
+++ b/ms-flights-summary
@@ -124,6 +124,9 @@ sub gather_stats($) {
 END
     for my $row (@{$stats}) {
 	my ($job,$stat) = @$row;
+        if ($stat eq 'preparing') {
+            $stat = %{ $f->{Jobs}{$job}{Reso} } ? 'prep.alloc.' : 'preparing';
+        }
 	$f->{Stats}{lc($stat)}++;
 	$global_stats{lc($stat)}++;
     }
@@ -137,11 +140,12 @@ sub sort_stats($) {
     my %so = (
 	queued => 1,
 	preparing => 2,
-	blocked => 3,
-	running => 4,
-	pass => 5,
-	fail => 6,
-	broken => 7,
+	'prep.alloc.' => 3,
+	blocked => 4,
+	running => 5,
+	pass => 6,
+	fail => 7,
+	broken => 8,
     );
     return sort { ($so{$a}//0) <=> ($so{$b}//0) } (keys %{$stats});
 }
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 5+ messages in thread
* [OSSTEST PATCH 4/5] ms-flights-summary: Treat preparing jobs outside the plan as `queued'
  2016-07-18 14:04 [OSSTEST PATCH 1/5] ms-flights-summary: Do gather_stats on jobs a bit later Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 2/5] ms-flights-summary: Do gather_stats without GROUP BY Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 3/5] ms-flights-summary: Invent a `prep.alloc.' pseudo job state Ian Jackson
@ 2016-07-18 14:04 ` Ian Jackson
  2016-07-18 14:04 ` [OSSTEST PATCH 5/5] sg-execute-flight: Fix an error message Ian Jackson
  3 siblings, 0 replies; 5+ messages in thread
From: Ian Jackson @ 2016-07-18 14:04 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Jackson

These jobs are still to run but we have no time estimate for them.

So do not add them to UnqueuedJobs; they then contribute to making the
`End of phase/flight' indication *not* bold when there are such jobs.

And bolden their figure in the table.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ms-flights-summary | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/ms-flights-summary b/ms-flights-summary
index 9a814ea..e4ce03b 100755
--- a/ms-flights-summary
+++ b/ms-flights-summary
@@ -132,7 +132,9 @@ END
     }
 
     $f->{NrJobs} = keys %{$f->{Jobs}};
-    $f->{UnqueuedJobs} = $f->{NrJobs} - ($f->{Stats}{queued}//0);
+    $f->{UnqueuedJobs} = $f->{NrJobs}
+        - ($f->{Stats}{queued}//0)
+        - ($f->{Stats}{preparing}//0);
 }
 
 sub sort_stats($) {
@@ -407,7 +409,8 @@ foreach my $f (sort keys %flights) {
     print "  <td align=right><strong>$fi->{NrJobs}</strong></td>\n";
     foreach (@summarycounts) {
 	my $s = ($fi->{Stats}{$_} || '');
-	$s = "<strong>$s</strong>" if $s && $_ eq 'queued';
+	$s = "<strong>$s</strong>"
+            if $s && ($_ eq 'queued' || $_ eq 'preparing');
 	print "  <td align=right>$s</td>";
     }
     my $expectedend = fmttime($fi->{ExpectedEnd});
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 5+ messages in thread
* [OSSTEST PATCH 5/5] sg-execute-flight: Fix an error message
  2016-07-18 14:04 [OSSTEST PATCH 1/5] ms-flights-summary: Do gather_stats on jobs a bit later Ian Jackson
                   ` (2 preceding siblings ...)
  2016-07-18 14:04 ` [OSSTEST PATCH 4/5] ms-flights-summary: Treat preparing jobs outside the plan as `queued' Ian Jackson
@ 2016-07-18 14:04 ` Ian Jackson
  3 siblings, 0 replies; 5+ messages in thread
From: Ian Jackson @ 2016-07-18 14:04 UTC (permalink / raw)
  To: xen-devel; +Cc: Ian Jackson

When pid is undefined, meaning we didn't find gotpid, print gotpid,
not pid.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 sg-execute-flight | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sg-execute-flight b/sg-execute-flight
index 4e3fcf2..fd98291 100755
--- a/sg-execute-flight
+++ b/sg-execute-flight
@@ -167,7 +167,7 @@ proc main_iteration {} {
         incr ix
     }
     if {![info exists pid]} {
-        log "unexpected child \[$pid\] $how $st"
+        log "unexpected child \[$gotpid\] $how $st"
         return
     }
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-07-18 14:04 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-07-18 14:04 [OSSTEST PATCH 1/5] ms-flights-summary: Do gather_stats on jobs a bit later Ian Jackson
2016-07-18 14:04 ` [OSSTEST PATCH 2/5] ms-flights-summary: Do gather_stats without GROUP BY Ian Jackson
2016-07-18 14:04 ` [OSSTEST PATCH 3/5] ms-flights-summary: Invent a `prep.alloc.' pseudo job state Ian Jackson
2016-07-18 14:04 ` [OSSTEST PATCH 4/5] ms-flights-summary: Treat preparing jobs outside the plan as `queued' Ian Jackson
2016-07-18 14:04 ` [OSSTEST PATCH 5/5] sg-execute-flight: Fix an error message Ian Jackson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).