From: Ian Jackson <ian.jackson@eu.citrix.com>
To: xen-devel@lists.xenproject.org
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Subject: [OSSTEST PATCH v2 21/41] sg-report-host-history: Find flight limit by flight start date
Date: Fri, 31 Jul 2020 12:38:00 +0100 [thread overview]
Message-ID: <20200731113820.5765-22-ian.jackson@eu.citrix.com> (raw)
In-Reply-To: <20200731113820.5765-1-ian.jackson@eu.citrix.com>
By default we look for anything in (roughly) the last year.
This query is in fact quite fast because the flights table is small.
There is still the per-host limit of $limit (2000) recent runs.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
sg-report-host-history | 56 ++++++++++++++++++++----------------------
1 file changed, 27 insertions(+), 29 deletions(-)
diff --git a/sg-report-host-history b/sg-report-host-history
index 9730ae7a..a159df3e 100755
--- a/sg-report-host-history
+++ b/sg-report-host-history
@@ -29,6 +29,7 @@ use POSIX;
use Osstest::Executive qw(:DEFAULT :colours);
our $limit= 2000;
+our $timelimit= 86400 * (366 + 14);
our $flightlimit;
our $htmlout = ".";
our $read_existing=1;
@@ -45,6 +46,8 @@ while (@ARGV && $ARGV[0] =~ m/^-/) {
last if m/^--?$/;
if (m/^--(limit)\=([1-9]\d*)$/) {
$$1= $2;
+ } elsif (m/^--time-limit\=([1-9]\d*)$/) {
+ $timelimit= $1;
} elsif (m/^--flight-limit\=([1-9]\d*)$/) {
$flightlimit= $1;
} elsif (restrictflight_arg($_)) {
@@ -108,38 +111,33 @@ sub read_existing_logs ($) {
}
sub computeflightsrange () {
- if (!$flightlimit) {
- my $flagscond =
- '('.join(' OR ', map { "f.hostflag = 'blessed-$_'" } @blessings).')';
- my $nhostsq = db_prepare(<<END);
- SELECT count(*)
- FROM resources r
- WHERE restype='host'
- AND EXISTS (SELECT 1
- FROM hostflags f
- WHERE f.hostname=r.resname
- AND $flagscond)
+ if ($flightlimit) {
+ my $minflightsq = db_prepare(<<END);
+ SELECT flight
+ FROM (
+ SELECT flight
+ FROM flights
+ WHERE $restrictflight_cond
+ ORDER BY flight DESC
+ LIMIT $flightlimit
+ ) f
+ ORDER BY flight ASC
+ LIMIT 1
END
- $nhostsq->execute();
- my ($nhosts) = $nhostsq->fetchrow_array();
- print DEBUG "COUNTED $nhosts hosts\n";
- $flightlimit = $nhosts * $limit * 2;
- }
-
- my $minflightsq = db_prepare(<<END);
- SELECT flight
- FROM (
+ $minflightsq->execute();
+ ($minflight,) = $minflightsq->fetchrow_array();
+ } else {
+ my $minflightsq = db_prepare(<<END);
SELECT flight
- FROM flights
- WHERE $restrictflight_cond
- ORDER BY flight DESC
- LIMIT $flightlimit
- ) f
- ORDER BY flight ASC
- LIMIT 1
+ FROM flights
+ WHERE started >= ?
+ ORDER BY flight ASC
+ LIMIT 1
END
- $minflightsq->execute();
- ($minflight,) = $minflightsq->fetchrow_array();
+ my $now = time // die $!;
+ $minflightsq->execute($now - $timelimit);
+ ($minflight,) = $minflightsq->fetchrow_array();
+ }
$minflight //= 0;
$flightcond = "(flight > $minflight)";
--
2.20.1
next prev parent reply other threads:[~2020-07-31 11:56 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-31 11:37 [OSSTEST PATCH v2 00/41] Performance work Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 01/41] Add cperl-indent-level to .dir-locals.el Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 02/41] SQL: Use "LIKE" rather than "like", etc Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 03/41] SQL: Fix incorrect LIKE pattern syntax (literals) Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 04/41] SQL: Fix incorrect LIKE pattern syntax (program variables) Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 05/41] sg-report-flight: Add a comment re same-flight search narrowing Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 06/41] sg-report-flight: Sort failures by job name as last resort Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 07/41] schema: Provide indices for sg-report-flight Ian Jackson
2020-07-31 14:21 ` George Dunlap
2020-07-31 14:55 ` Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 08/41] sg-report-flight: Ask the db for flights of interest Ian Jackson
2020-07-31 14:17 ` George Dunlap
2020-07-31 15:43 ` Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 09/41] sg-report-flight: Use WITH to use best index use for $flightsq Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 10/41] sg-report-flight: Use WITH clause to use index for $anypassq Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 11/41] sg-report-flight: Use the job row from the intitial query Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 12/41] Executive: Use index for report__find_test Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 13/41] duration_estimator: Ignore truncated jobs unless we know the step Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 14/41] duration_estimator: Introduce some _qtxt variables Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 15/41] duration_estimator: Explicitly provide null in general host q Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 16/41] duration_estimator: Return job column in first query Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 17/41] duration_estimator: Move $uptincl_testid to separate @x_params Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 18/41] duration_estimator: Move duration query loop into database Ian Jackson
2020-07-31 11:37 ` [OSSTEST PATCH v2 19/41] Executive: Drop redundant AND clause Ian Jackson
2020-07-31 14:17 ` George Dunlap
2020-07-31 11:37 ` [OSSTEST PATCH v2 20/41] schema: Add index for quick lookup by host Ian Jackson
2020-07-31 11:38 ` Ian Jackson [this message]
2020-07-31 11:38 ` [OSSTEST PATCH v2 22/41] sg-report-host-history: Drop per-job debug etc Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 23/41] Executive: Export opendb_tests Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 24/41] sg-report-host-history: Add a debug print after sorting jobs Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 25/41] sg-report-host-history: Do the main query per host Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 26/41] sg-report-host-history: Rerganisation: Make mainquery per-host Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 27/41] sg-report-host-history: Rerganisation: Read old logs later Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 28/41] sg-report-host-history: Rerganisation: Change loops Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 29/41] sg-report-host-history: Drop a redundznt AND clause Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 30/41] sg-report-host-history: Fork Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 31/41] schema: Add index to help cs-bisection-step Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 32/41] adhoc-revtuple-generator: Fix an undef warning in a debug print Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 33/41] cs-bisection-step: Generalise qtxt_common_rev_ok Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 34/41] cs-bisection-step: Move an AND Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 35/41] cs-bisection-step: Break out qtxt_common_ok Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 36/41] cs-bisection-step: Use db_prepare a few times instead of ->do Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 37/41] cs-bisection-step: temporary table: Insert only rows we care about Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 38/41] SQL: Change LIKE E'...\\_...' to LIKE '...\_...' Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 39/41] cs-bisection-step: Add a debug print when we run dot(1) Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 40/41] cs-bisection-step: Lay out the revision tuple graph once Ian Jackson
2020-07-31 11:38 ` [OSSTEST PATCH v2 41/41] duration_estimator: Clarify recentflights query a bit Ian Jackson
2020-07-31 14:04 ` George Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200731113820.5765-22-ian.jackson@eu.citrix.com \
--to=ian.jackson@eu.citrix.com \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).