Re: [RFC PATCH 1/2] sched: Clean up SD_BALANCE_WAKE flags in sched domain build-up

From: Vincent Guittot <vincent.guittot@linaro.org>
To: Yuyang Du <yuyang.du@intel.com>
Cc: Mike Galbraith <umgwanakikbuti@gmail.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Benjamin Segall <bsegall@google.com>,
	Paul Turner <pjt@google.com>,
	Morten Rasmussen <morten.rasmussen@arm.com>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>
Subject: Re: [RFC PATCH 1/2] sched: Clean up SD_BALANCE_WAKE flags in sched domain build-up
Date: Wed, 1 Jun 2016 10:32:53 +0200	[thread overview]
Message-ID: <CAKfTPtBig+kRVeOoYqhoa1GRXzbjMKmR8ODgAGZr0Jcm-YZncQ@mail.gmail.com> (raw)
In-Reply-To: <20160601000105.GU18670@intel.com>

On 1 June 2016 at 02:01, Yuyang Du <yuyang.du@intel.com> wrote:
> On Wed, Jun 01, 2016 at 07:07:13AM +0200, Mike Galbraith wrote:
>> On Tue, 2016-05-31 at 09:31 +0800, Yuyang Du wrote:
>> > On Tue, May 31, 2016 at 11:21:46AM +0200, Peter Zijlstra wrote:
>> > > On Tue, May 31, 2016 at 09:11:37AM +0800, Yuyang Du wrote:
>> > > > The SD_BALANCE_WAKE is irrelevant in the contexts of these two removals,
>> > > > and in addition SD_BALANCE_WAKE is not and should not be set in any
>> > > > sched_domain flags so far.
>> > >
>> > > This Changelog doesn't make any sense...
>> >
>> > How? SD_BALANCE_WAKE is not in any sched_domain flags (sd->flags), even if
>> > it is, it is not used anywhere, no?
>>
>> If the user chooses to set SD_BALANCE_WAKE in sd->flags, it is in fact
>> used.  It's just not turned on by default due to full balance on every
>> wakeup being far too painful to do by default.
>
> Yup. Up to this point, we don't have any disagreement. And I don't think we
> have any disagreement conceptually. What the next patch really does is:
>
> (1) we don't remove SD_BALANCE_WAKE as an important sched_domain flag, on
>     the contrary, we strengthen it.
>
> (2) the semantic of SD_BALANCE_WAKE is currently represented by SD_WAKE_AFFINE,
>     we actually remove this representation.
>
> (3) regarding the semantic of SD_WAKE_AFFINE, it is really not about selecting
>     waker CPU or about the fast path. Conceptually, it is just saying the waker
>     CPU is a valid and important candidate if SD_BALANCE_WAKE, which is just so
>     obvious, so I don't think it deserves to be a separate sched_domain flag.
>
> (4) the outcome is, if SD_BALANCE_WAKE, we definitely will/should try waker CPU,
>     and if !SD_BALANCE_WAKE, we don't try waker CPU. So nothing functional is
>     changed.


AFAIU, there is 4 possible cases during wake up:
- we don't want any balance at wake so we don't have SD_BALANCE_WAKE
nor SD_WAKE_AFFINE in sched_domain->flags
- we only want wake affine balance check so we only have
SD_WAKE_AFFINE in sched_domain->flags
- we want wake_affine and full load balance at wake so we have both
SD_BALANCE_WAKE and SD_WAKE_AFFINE in sched_domain->flags
- we want  full load balance but want to skip wake affine fast path so
we only have SD_BALANCE_WAKE in sched_domain->flags

I'm not sure that we can still do only wake_affine or only full
load_balance with your changes whereas these sequences are valid ones

Vincent