From: George Dunlap <George.Dunlap@citrix.com>
To: Ian Jackson <Ian.Jackson@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Subject: Re: [OSSTEST PATCH 04/14] sg-report-flight: Ask the db for flights of interest
Date: Wed, 22 Jul 2020 12:10:11 +0000 [thread overview]
Message-ID: <3966AFCB-7B7B-45BE-A3F1-7E04943EEEFA@citrix.com> (raw)
In-Reply-To: <20200721184205.15232-5-ian.jackson@eu.citrix.com>
> On Jul 21, 2020, at 7:41 PM, Ian Jackson <ian.jackson@eu.citrix.com> wrote:
>
> Specifically, we narrow the initial query to flights which have at
> least some job with the built_revision_foo we are looking for.
>
> This condition is strictly broader than that implemented inside the
> flight search loop, so there is no functional change.
>
> Perf: runtime of my test case now ~300s-500s.
>
> Example query before (from the Perl DBI trace):
>
> SELECT * FROM (
> SELECT flight, blessing FROM flights
> WHERE (branch='xen-unstable')
> AND EXISTS (SELECT 1
> FROM jobs
> WHERE jobs.flight = flights.flight
> AND jobs.job = ?)
>
> AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
> ORDER BY flight DESC
> LIMIT 1000
> ) AS sub
> ORDER BY blessing ASC, flight DESC
This one says:
Find the 1000 most recent flights
Where
branch is "xen-unstable”
one of its jobs is $job
And blessing is “real”
But why are we selecting ‘blessing’ from these, if we’ve specified that blessing = “real”? Isn’t that redundant?
>
> With these bind variables:
>
> "test-armhf-armhf-libvirt"
>
> After:
>
> SELECT * FROM (
> SELECT DISTINCT flight, blessing
> FROM flights
> JOIN runvars r1 USING (flight)
>
> WHERE (branch='xen-unstable')
> AND ( (TRUE AND flight <= 151903) AND (blessing='real') )
> AND EXISTS (SELECT 1
> FROM jobs
> WHERE jobs.flight = flights.flight
> AND jobs.job = ?)
>
> AND r1.name LIKE 'built_revision_%'
> AND r1.name = ?
> AND r1.val= ?
>
> ORDER BY flight DESC
> LIMIT 1000
> ) AS sub
> ORDER BY blessing ASC, flight DESC
So this says:
Find me the most 1000 recent flights
Where:
branch is “xen-unstable”
flight <= 15903
blessing is “real”
One of its jobs is $job
It has a runvar matching given $name and $val
And of course it uses the ’name LIKE ‘built_revision_%’ index.
Still don’t understand the ’TRUE AND’ and ‘AS sub’ bits, but it looks to me like it’s substantially the same query, with additional $name = $val runvar restriction.
And given that you say, "This condition is strictly broader than that implemented inside the flight search loop”, I take it that it’s again mainly to take advantage of the new index?
-George
next prev parent reply other threads:[~2020-07-22 12:10 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-21 18:41 [OSSTEST PATCH 00/14] Flight report performance improvements Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 01/14] sg-report-flight: Add a comment re same-flight search narrowing Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 02/14] sg-report-flight: Sort failures by job name as last resort Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 03/14] schema: Provide indices for sg-report-flight Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 04/14] sg-report-flight: Ask the db for flights of interest Ian Jackson
2020-07-22 12:10 ` George Dunlap [this message]
2020-07-22 14:03 ` Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 05/14] sg-report-flight: Use WITH to use best index use for $flightsq Ian Jackson
2020-07-22 12:47 ` George Dunlap
2020-07-22 14:06 ` Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 06/14] sg-report-flight: Use WITH clause to use index for $anypassq Ian Jackson
2020-07-27 16:15 ` George Dunlap
2020-07-31 10:41 ` Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 07/14] sg-report-flight: Use the job row from the intitial query Ian Jackson
2020-07-21 18:41 ` [OSSTEST PATCH 08/14] Executive: Use index for report__find_test Ian Jackson
2020-07-22 11:33 ` George Dunlap
2020-07-22 13:49 ` Ian Jackson
2020-07-21 18:42 ` [OSSTEST PATCH 09/14] duration_estimator: Ignore truncated jobs unless we know the step Ian Jackson
2020-07-21 18:42 ` [OSSTEST PATCH 10/14] duration_estimator: Introduce some _qtxt variables Ian Jackson
2020-07-21 18:42 ` [OSSTEST PATCH 11/14] duration_estimator: Explicitly provide null in general host q Ian Jackson
2020-07-21 18:42 ` [OSSTEST PATCH 12/14] duration_estimator: Return job column in first query Ian Jackson
2020-07-21 18:42 ` [OSSTEST PATCH 13/14] duration_estimator: Move $uptincl_testid to separate @x_params Ian Jackson
2020-07-21 18:42 ` [OSSTEST PATCH 14/14] duration_estimator: Move duration query loop into database Ian Jackson
2020-07-27 17:43 ` George Dunlap
2020-07-31 10:39 ` Ian Jackson
2020-07-31 10:45 ` George Dunlap
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3966AFCB-7B7B-45BE-A3F1-7E04943EEEFA@citrix.com \
--to=george.dunlap@citrix.com \
--cc=Ian.Jackson@citrix.com \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).