From: "Li, Aubrey" <aubrey.li@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>, Andi Kleen <ak@linux.intel.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>,
Christoph Lameter <cl@linux.com>, Aubrey Li <aubrey.li@intel.com>,
tglx@linutronix.de, len.brown@intel.com, rjw@rjwysocki.net,
tim.c.chen@linux.intel.com, arjan@linux.intel.com,
paulmck@linux.vnet.ibm.com, yang.zhang.wz@gmail.com,
x86@kernel.org, linux-kernel@vger.kernel.org,
daniel.lezcano@linaro.org
Subject: Re: [RFC PATCH v1 00/11] Create fast idle path for short idle periods
Date: Tue, 18 Jul 2017 11:14:57 +0800 [thread overview]
Message-ID: <34371ef8-b8bc-d2bf-93de-3fccd6beb032@linux.intel.com> (raw)
In-Reply-To: <20170717192309.ubn5muvc3u7htuaw@hirez.programming.kicks-ass.net>
On 2017/7/18 3:23, Peter Zijlstra wrote:
> On Fri, Jul 14, 2017 at 09:26:19AM -0700, Andi Kleen wrote:
>>> And as said; Daniel has been working on a better predictor -- now he's
>>> probably not used it on the network workload you're looking at, so that
>>> might be something to consider.
>>
>> Deriving a better idle predictor is a bit orthogonal to fast idle.
>
> No. If you want a different C state selected we need to fix the current
> C state selector. We're not going to tinker.
>
> And the predictor is probably the most fundamental part of the whole C
> state selection logic.
>
> Now I think the problem is that the current predictor goes for an
> average idle duration. This means that we, on average, get it wrong 50%
> of the time. For performance that's bad.
>
> If you want to improve the worst case, we need to consider a cumulative
> distribution function, and staying with the Gaussian assumption already
> present, that would mean using:
>
> 1 x - mu
> CDF(x) = - [ 1 + erf(-------------) ]
> 2 sigma sqrt(2)
>
> Where, per the normal convention mu is the average and sigma^2 the
> variance. See also:
>
> https://en.wikipedia.org/wiki/Normal_distribution
>
> We then solve CDF(x) = n% to find the x for which we get it wrong n% of
> the time (IIRC something like: 'mu - 2sigma' ends up being 5% or so).
>
> This conceptually gets us better exit latency for the cases where we got
> it wrong before, and practically pushes down the estimate which gets us
> C1 longer.
>
> Of course, this all assumes a Gaussian distribution to begin with, if we
> get bimodal (or worse) distributions we can still get it wrong. To fix
> that, we'd need to do something better than what we currently have.
>
Maybe you are talking about applying some machine learning algorithm online
to fit a multivariate normal distribution, :)
Well, back to the problem, when the scheduler picks up idle thread, it does
not look at the history, nor make the prediction. So it's possible it has
to switch back a task ASAP when it's going into idle(very common under some
workloads).
That is, (idle_entry + idle_exit) > idle. If the system has multiple
hardware idle states, then:
(idle_entry + idle_exit + HW_entry + HW_exit) > HW_sleep
So we eventually want the idle path lighter than what we currently have.
A complex predictor may have high accuracy, but the cost could be high as well.
We need a tradeoff here IMHO. I'll check Daniel's work to understand how/if
it's better than menu governor.
Thanks,
-Aubrey
next prev parent reply other threads:[~2017-07-18 3:15 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-10 1:38 [RFC PATCH v1 00/11] Create fast idle path for short idle periods Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 01/11] sched/idle: create a fast " Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 02/11] cpuidle: attach cpuidle governor statistics to the per-CPU device Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 03/11] cpuidle: introduce cpuidle governor for idle prediction Aubrey Li
2017-07-12 12:16 ` Peter Zijlstra
2017-07-10 1:38 ` [RFC PATCH v1 04/11] sched/idle: make the fast idle path for short idle periods Aubrey Li
2017-07-11 12:58 ` Paul E. McKenney
2017-07-11 16:33 ` Frederic Weisbecker
2017-07-11 18:11 ` Paul E. McKenney
2017-07-12 3:19 ` Li, Aubrey
2017-07-12 5:03 ` Paul E. McKenney
2017-07-12 5:22 ` Li, Aubrey
2017-07-12 12:19 ` Peter Zijlstra
2017-07-10 1:38 ` [RFC PATCH v1 05/11] cpuidle: update idle statistics before cpuidle governor Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 06/11] timers: keep sleep length updated as needed Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 07/11] cpuidle: make idle residency update more generic Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 08/11] cpuidle: menu: remove reduplicative implementation Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 09/11] cpuidle: menu: feed cpuidle prediction to menu governor Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 10/11] cpuidle: update cpuidle governor when needed Aubrey Li
2017-07-10 1:38 ` [RFC PATCH v1 11/11] sched/idle: Add a tuning knob to allow changing fast idle threshold Aubrey Li
2017-07-10 8:46 ` [RFC PATCH v1 00/11] Create fast idle path for short idle periods Peter Zijlstra
2017-07-10 9:29 ` Wanpeng Li
2017-07-10 13:59 ` Li, Aubrey
2017-07-10 14:46 ` Andi Kleen
2017-07-10 16:42 ` Peter Zijlstra
2017-07-10 17:27 ` Andi Kleen
2017-07-11 4:40 ` Li, Aubrey
2017-07-11 9:41 ` Peter Zijlstra
2017-07-11 16:09 ` Frederic Weisbecker
2017-07-11 16:34 ` Peter Zijlstra
2017-07-11 18:09 ` Paul E. McKenney
2017-07-12 11:54 ` Peter Zijlstra
2017-07-12 15:56 ` Paul E. McKenney
2017-07-12 17:46 ` Peter Zijlstra
2017-07-12 18:53 ` Paul E. McKenney
2017-07-12 19:00 ` Paul E. McKenney
2017-07-19 13:43 ` Frederic Weisbecker
2017-07-19 14:51 ` Paul E. McKenney
2017-07-12 12:22 ` Peter Zijlstra
2017-07-12 15:54 ` Paul E. McKenney
2017-07-12 17:17 ` Peter Zijlstra
2017-07-12 17:57 ` Peter Zijlstra
2017-07-12 18:50 ` Paul E. McKenney
2017-07-12 18:46 ` Paul E. McKenney
2017-07-13 8:34 ` Peter Zijlstra
2017-07-12 4:15 ` Li, Aubrey
2017-07-12 8:34 ` Peter Zijlstra
2017-07-12 21:32 ` Andi Kleen
2017-07-13 8:36 ` Peter Zijlstra
2017-07-13 14:48 ` Li, Aubrey
2017-07-13 14:53 ` Peter Zijlstra
2017-07-13 15:13 ` Li, Aubrey
2017-07-13 18:28 ` Peter Zijlstra
2017-07-14 3:56 ` Li, Aubrey
2017-07-14 15:38 ` Peter Zijlstra
2017-07-14 15:52 ` Arjan van de Ven
2017-07-14 15:58 ` Peter Zijlstra
2017-07-14 16:03 ` Andi Kleen
2017-07-17 9:21 ` Peter Zijlstra
2017-07-17 13:41 ` Li, Aubrey
2017-07-14 15:53 ` Andi Kleen
2017-07-14 16:06 ` Peter Zijlstra
2017-07-14 16:26 ` Andi Kleen
2017-07-17 19:23 ` Peter Zijlstra
2017-07-17 19:27 ` Arjan van de Ven
2017-07-17 19:46 ` Thomas Gleixner
2017-07-17 19:51 ` Arjan van de Ven
2017-07-17 19:59 ` Thomas Gleixner
2017-07-17 19:48 ` Arjan van de Ven
2017-07-17 19:53 ` Thomas Gleixner
2017-07-17 19:55 ` Arjan van de Ven
2017-07-18 3:23 ` Li, Aubrey
2017-07-18 18:55 ` Peter Zijlstra
2017-07-18 3:14 ` Li, Aubrey [this message]
2017-07-18 4:45 ` Andi Kleen
2017-07-18 6:43 ` Thomas Gleixner
2017-07-18 6:56 ` Li, Aubrey
2017-07-18 15:20 ` Paul E. McKenney
2017-07-18 15:29 ` Arjan van de Ven
2017-07-18 16:36 ` Peter Zijlstra
2017-07-18 16:37 ` Arjan van de Ven
2017-07-18 17:05 ` Peter Zijlstra
2017-07-19 5:44 ` Li, Aubrey
2017-07-19 14:48 ` Paul E. McKenney
2017-07-19 15:03 ` Christopher Lameter
2017-07-19 16:54 ` Paul E. McKenney
2017-07-20 1:40 ` Li, Aubrey
2017-07-20 12:50 ` Paul E. McKenney
2017-07-20 13:45 ` Arjan van de Ven
2017-07-20 14:19 ` Peter Zijlstra
2017-07-20 16:02 ` Paul E. McKenney
2017-07-18 16:45 ` Peter Zijlstra
2017-07-18 6:59 ` Andi Kleen
2017-07-18 7:19 ` Thomas Gleixner
2017-07-19 6:12 ` Li, Aubrey
2017-07-19 7:55 ` Thomas Gleixner
2017-07-20 1:56 ` Li, Aubrey
2017-07-20 8:11 ` Thomas Gleixner
2017-07-20 13:48 ` Arjan van de Ven
2017-07-18 7:24 ` Thomas Gleixner
2017-07-18 13:23 ` Peter Zijlstra
2017-07-19 13:43 ` Peter Zijlstra
2017-07-13 15:20 ` Paul E. McKenney
2017-07-14 3:47 ` Li, Aubrey
2017-07-14 4:05 ` Paul E. McKenney
2017-07-17 13:24 ` Li, Aubrey
2017-07-17 13:58 ` Peter Zijlstra
2017-07-17 14:02 ` Li, Aubrey
2017-07-12 12:14 ` Peter Zijlstra
2017-07-11 17:58 ` Christoph Lameter
2017-07-12 2:07 ` Li, Aubrey
2017-07-12 2:35 ` Li, Aubrey
2017-07-12 18:10 ` Peter Zijlstra
2017-07-11 9:02 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=34371ef8-b8bc-d2bf-93de-3fccd6beb032@linux.intel.com \
--to=aubrey.li@linux.intel.com \
--cc=ak@linux.intel.com \
--cc=arjan@linux.intel.com \
--cc=aubrey.li@intel.com \
--cc=cl@linux.com \
--cc=daniel.lezcano@linaro.org \
--cc=fweisbec@gmail.com \
--cc=len.brown@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=rjw@rjwysocki.net \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=x86@kernel.org \
--cc=yang.zhang.wz@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).