All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ian Jackson <ian.jackson@eu.citrix.com>
To: <xen-devel@lists.xenproject.org>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Subject: [Xen-devel] [OSSTEST PATCH 07/13] sg-report-host-history: Store per-job query results in %$jr
Date: Fri, 8 Nov 2019 18:49:55 +0000	[thread overview]
Message-ID: <20191108185001.3319-8-ian.jackson@eu.citrix.com> (raw)
In-Reply-To: <20191108185001.3319-1-ian.jackson@eu.citrix.com>

jobquery now looks for the subquery results in %$jr, under the
cachekey, and only runs the query if it's not found.  It then stores
the value.

We are going to persist the contents of %$jr across runs, and then
this will avoid rerunning queries needlessly.

No functional change yet.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 sg-report-host-history | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/sg-report-host-history b/sg-report-host-history
index 4c40cbec..8767b25d 100755
--- a/sg-report-host-history
+++ b/sg-report-host-history
@@ -109,11 +109,21 @@ END
     print DEBUG "MINFLIGHT $minflight\n";
 }
 
+our $jqcachemisses = 0;
+our $jqtotal = 0;
+
 sub jobquery ($$$) {
     my ($q, $jr, $cachekey) = @_;
-    my ($q, $jr) = @_;
-    $q->execute($jr->{flight}, $jr->{job});
-    return $q->fetchrow_hashref();
+    $jqtotal++;
+    $cachekey = '%'.$cachekey;
+    my $cached = $jr->{$cachekey};
+    if (!$cached) {
+	$jqcachemisses++;
+	$q->execute($jr->{flight}, $jr->{job});
+	$cached = $q->fetchrow_hashref();
+	$jr->{$cachekey} = $cached;
+    }
+    return $cached;
 }
 
 our %hosts;
@@ -215,6 +225,12 @@ END
     my $inrows = $hosts{$hostname};
     print DEBUG "FOUND ", (scalar @$inrows), " ROWS for $hostname\n";
 
+    # Each entry in @$inrows is a $jr, which is a hash
+    # It has keys for the result columns in mainquery
+    # It also has keys '%<letter>' (yes, with a literal '%')
+    # which are the results of per-job queries.
+    # The contents of $jr for each job is cached across runs. (TODO)
+
     my @rows;
     foreach my $jr (@$inrows) {
 	print DEBUG "JOB $jr->{flight}.$jr->{job} ";
@@ -377,3 +393,5 @@ foreach my $host (sort keys %hosts) {
 	reporthost $host;
     });
 }
+
+print DEBUG "JQ CACHE ".($jqtotal-$jqcachemisses)." / $jqtotal\n";
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  parent reply	other threads:[~2019-11-08 18:50 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-08 18:49 [Xen-devel] [OSSTEST PATCH 00/13] Speed up and restore host history Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 01/13] sg-report-host-history: Improve debugging output Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 02/13] sg-report-host-history: New --no-install option for testing Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 03/13] sg-report-host-history: Move `computeflightsrange' after hosts Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 04/13] sg-report-host-history: Actually honour $minflight Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 05/13] sg-report-host-history: Get job status from mainquery Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 06/13] sg-report-host-history: Add $cachekey argument to jobquery Ian Jackson
2019-11-08 18:49 ` Ian Jackson [this message]
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 08/13] sg-report-host-history: Write cache entries Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 09/13] sg-report-host-history: Write cache entries for tail, too Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 10/13] sg-report-host-history: Read cache entries Ian Jackson
2019-11-08 18:49 ` [Xen-devel] [OSSTEST PATCH 11/13] sg-report-host-history: Move job runvars query later Ian Jackson
2019-11-08 18:50 ` [Xen-devel] [OSSTEST PATCH 12/13] sg-report-host-history: Cache runvar queries (power information) Ian Jackson
2019-11-08 18:50 ` [Xen-devel] [OSSTEST PATCH 13/13] Revert "sg-report-host-history: Reduce limit from 2000 to 200" Ian Jackson
2019-11-08 19:45 ` [Xen-devel] [OSSTEST PATCH 00/13] Speed up and restore host history Sander Eikelenboom
2019-11-11 11:00   ` Ian Jackson
2019-11-11 12:03     ` Sander Eikelenboom
2019-11-11 14:00       ` Ian Jackson
2019-11-20 17:54         ` Ian Jackson
2019-11-21  5:45           ` Jürgen Groß
2019-11-25 14:53             ` Ian Jackson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191108185001.3319-8-ian.jackson@eu.citrix.com \
    --to=ian.jackson@eu.citrix.com \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.