From: Vincent Guittot <vincent.guittot@linaro.org>
To: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
"mingo@redhat.com" <mingo@redhat.com>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Yuyang Du <yuyang.du@intel.com>,
mgalbraith@suse.de, linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 06/16] sched: Disable WAKE_AFFINE for asymmetric configurations
Date: Tue, 24 May 2016 17:53:27 +0200 [thread overview]
Message-ID: <CAKfTPtB7bjy7skRQ1JyWdCZ1ZAM1pgsF7f1c_GLOffte=2p4zQ@mail.gmail.com> (raw)
In-Reply-To: <20160524150204.GI27946@e105550-lin.cambridge.arm.com>
On 24 May 2016 at 17:02, Morten Rasmussen <morten.rasmussen@arm.com> wrote:
> On Tue, May 24, 2016 at 03:52:00PM +0200, Vincent Guittot wrote:
>> On 24 May 2016 at 15:36, Morten Rasmussen <morten.rasmussen@arm.com> wrote:
>> > On Tue, May 24, 2016 at 03:27:05PM +0200, Vincent Guittot wrote:
>> >> On 24 May 2016 at 15:16, Morten Rasmussen <morten.rasmussen@arm.com> wrote:
>> >> > On Tue, May 24, 2016 at 02:12:38PM +0200, Vincent Guittot wrote:
>> >> >> On 24 May 2016 at 12:29, Morten Rasmussen <morten.rasmussen@arm.com> wrote:
>> >> >> > On Tue, May 24, 2016 at 11:10:28AM +0200, Vincent Guittot wrote:
>> >> >> >> On 23 May 2016 at 12:58, Morten Rasmussen <morten.rasmussen@arm.com> wrote:
>> >> >> >> > If the system has cpu of different compute capacities (e.g. big.LITTLE)
>> >> >> >> > let affine wakeups be constrained to cpus of the same type.
>> >> >> >>
>> >> >> >> Can you explain why you don't want wake affine with cpus with
>> >> >> >> different compute capacity ?
>> >> >> >
>> >> >> > I should have made the overall idea a bit more clear. The idea is to
>> >> >> > deal with cross-capacity migrations in the find_idlest_{group, cpu}{}
>> >> >> > path so we don't have to touch select_idle_sibling().
>> >> >> > select_idle_sibling() is critical for wake-up latency, and I'm assumed
>> >> >> > that people wouldn't like adding extra overhead in there to deal with
>> >> >> > capacity and utilization.
>> >> >>
>> >> >> So this means that we will never use the quick path of
>> >> >> select_idle_sibling for cross capacity migration but always the one
>> >> >> with extra overhead?
>> >> >
>> >> > Yes. select_idle_sibling() is only used to choose among equal capacity
>> >> > cpus (capacity_orig).
>> >> >
>> >> >> Patch 9 adds more tests for enabling wake_affine path. Can't it also
>> >> >> be used for cross capacity migration ? so we can use wake_affine if
>> >> >> the task or the cpus (even with different capacity) doesn't need this
>> >> >> extra overhead
>> >> >
>> >> > The test in patch 9 is to determine whether we are happy with the
>> >> > capacity of the previous cpu, or we should go look for one with more
>> >> > capacity. I don't see how we can use select_idle_sibling() unmodified
>> >> > for sched domains containing cpus of different capacity to select an
>> >> > appropriate cpu. It is just picking an idle cpu, it might have high
>> >> > capacity or low, it wouldn't care.
>> >> >
>> >> > How would you avoid the overhead of checking capacity and utilization of
>> >> > the cpus and still pick an appropriate cpu?
>> >>
>> >> My point is that there is some wake up case where we don't care about
>> >> the capacity and utilization of cpus even for cross capacity migration
>> >> and we will never take benefit of this fast path.
>> >> You have added an extra check for setting want_affine in patch 9 which
>> >> uses capacity and utilization of cpu to disable this fast path when a
>> >> task needs more capacity than available. Can't you use this function
>> >> to disable the want_affine for cross-capacity migration situation that
>> >> cares of the capacity and need the full scan of sched_domain but keep
>> >> it enable for other cases ?
>> >
>> > It is not clear to me what the other cases are. What kind of cases do
>> > you have in mind?
>>
>> As an example, you have a task A that have to be on a big CPU because
>> of the requirement of compute capacity, that wakes up a task B that
>> can run on any cpu according to its utilization. The fast wake up path
>> is fine for task B whatever prev cpu is.
>
> In that case, we will take always take fast path (select_idle_sibling())
> for task B if wake_wide() allows it, which should be fine.
Even if want_affine is set, the wake up of task B will not use the fast path.
The affine_sd will not be set because the sched_domain, which have
both cpus, will not have the SD_WAKE_AFFINE flag according to this
patch, isn't it ?
So task B can't use the fast path whereas nothing prevent him to take
benefit of it
Am I missing something ?
>
> wake_cap() will return true as the B's prev_cpu is either a big cpu
> (first criteria) or have sufficient capacity for B (second criteria).
> Given that wake_wide() allows returns false as well and there are no
> restrictions, want_affine will be true. Depending on where wake_affine()
> sends us, we will use select_idle_sibling() to search around B's
> prev_cpu or this cpu (where task A is running).
>
> We avoid the overhead of looking for cpu capacity and utilization, but
> we have restricted the search space for select_idle_sibling(). In case
> B's prev_cpu is a little cpu, the choice whether we looks for little or
> big capacity cpus depends on the wake_affine()'s decision. So the search
> space isn't as wide as it could be.
>
> To expand the search space we would have be able to adjust the
> sched_domain level at which select_idle_sibling() is operating, so we
> can look at same-capacity cpus only in the fast path for tasks like A,
> and look at all cpus for tasks like B. It could possibly be done, if we
> dare touching select_idle_sibling() ;-) I still have to look at those
> patches PeterZ posted a while back.
>
> TLDR; The fast path should already be used for task B, but the cpu
> search space is restricted to a specific subset of cpus selected by
> wake_affine() which isn't ideal, but much less invasive in terms of code
> changes.
next prev parent reply other threads:[~2016-05-24 15:53 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-23 10:58 [PATCH 00/16] sched: Clean-ups and asymmetric cpu capacity support Morten Rasmussen
2016-05-23 10:58 ` [PATCH 01/16] sched: Fix power to capacity renaming in comment Morten Rasmussen
2016-05-23 10:58 ` [PATCH 02/16] sched/fair: Consistent use of prev_cpu in wakeup path Morten Rasmussen
2016-06-01 19:49 ` Peter Zijlstra
2016-05-23 10:58 ` [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide Morten Rasmussen
2016-05-23 11:12 ` Mike Galbraith
2016-05-23 12:00 ` Morten Rasmussen
2016-05-23 13:00 ` Mike Galbraith
2016-05-23 14:10 ` Morten Rasmussen
2016-05-23 15:42 ` Mike Galbraith
2016-05-23 23:17 ` Yuyang Du
2016-05-23 23:04 ` Yuyang Du
2016-06-01 19:57 ` Peter Zijlstra
2016-06-02 8:05 ` Peter Zijlstra
2016-06-07 12:08 ` Morten Rasmussen
2016-05-23 10:58 ` [PATCH 04/16] sched/fair: Optimize find_idlest_cpu() when there is no choice Morten Rasmussen
2016-05-24 6:29 ` Mike Galbraith
2016-05-24 8:05 ` Morten Rasmussen
2016-05-24 8:12 ` Mike Galbraith
2016-06-01 19:59 ` Peter Zijlstra
2016-06-07 14:25 ` Morten Rasmussen
2016-05-23 10:58 ` [PATCH 05/16] sched: Introduce SD_ASYM_CPUCAPACITY sched_domain topology flag Morten Rasmussen
2016-05-23 10:58 ` [PATCH 06/16] sched: Disable WAKE_AFFINE for asymmetric configurations Morten Rasmussen
2016-05-24 9:10 ` Vincent Guittot
2016-05-24 10:29 ` Morten Rasmussen
2016-05-24 12:12 ` Vincent Guittot
2016-05-24 13:16 ` Morten Rasmussen
2016-05-24 13:27 ` Vincent Guittot
2016-05-24 13:36 ` Morten Rasmussen
2016-05-24 13:52 ` Vincent Guittot
2016-05-24 15:02 ` Morten Rasmussen
2016-05-24 15:53 ` Vincent Guittot [this message]
2016-05-25 9:12 ` Morten Rasmussen
2016-05-26 6:45 ` Vincent Guittot
2016-06-07 16:50 ` Morten Rasmussen
2016-05-23 10:58 ` [PATCH 07/16] sched: Make SD_BALANCE_WAKE a topology flag Morten Rasmussen
2016-05-24 23:52 ` Yuyang Du
2016-05-25 9:27 ` Morten Rasmussen
2016-06-01 20:18 ` Peter Zijlstra
2016-06-08 8:45 ` Morten Rasmussen
2016-05-23 10:58 ` [PATCH 08/16] sched: Store maximum per-cpu capacity in root domain Morten Rasmussen
2016-05-23 10:58 ` [PATCH 09/16] sched/fair: Let asymmetric cpu configurations balance at wake-up Morten Rasmussen
2016-05-24 0:04 ` Yuyang Du
2016-05-24 8:10 ` Morten Rasmussen
2016-05-24 7:03 ` Mike Galbraith
2016-05-24 7:15 ` Mike Galbraith
2016-05-25 6:57 ` Wanpeng Li
2016-05-25 9:49 ` Morten Rasmussen
2016-05-25 10:29 ` Wanpeng Li
2016-05-25 10:54 ` Morten Rasmussen
2016-05-25 11:18 ` Wanpeng Li
2016-06-02 14:21 ` Peter Zijlstra
2016-06-08 11:29 ` Morten Rasmussen
2016-06-08 14:36 ` Peter Zijlstra
2016-05-23 10:58 ` [PATCH 10/16] sched/fair: Compute task/cpu utilization at wake-up more correctly Morten Rasmussen
2016-05-23 10:58 ` [PATCH 11/16] sched/fair: Consider spare capacity in find_idlest_group() Morten Rasmussen
2016-05-23 10:58 ` [PATCH 12/16] sched: Add per-cpu max capacity to sched_group_capacity Morten Rasmussen
2016-05-23 10:58 ` [PATCH 13/16] sched/fair: Avoid pulling tasks from non-overloaded higher capacity groups Morten Rasmussen
2016-05-23 10:58 ` [PATCH 14/16] arm: Set SD_ASYM_CPUCAPACITY for big.LITTLE platforms Morten Rasmussen
2016-05-23 10:58 ` [PATCH 15/16] arm: Set SD_BALANCE_WAKE flag for asymmetric capacity systems Morten Rasmussen
2016-05-23 10:58 ` [PATCH 16/16] arm: Update arch_scale_cpu_capacity() to reflect change to define Morten Rasmussen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAKfTPtB7bjy7skRQ1JyWdCZ1ZAM1pgsF7f1c_GLOffte=2p4zQ@mail.gmail.com' \
--to=vincent.guittot@linaro.org \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgalbraith@suse.de \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=peterz@infradead.org \
--cc=yuyang.du@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).