[PATCH] sched/fair: fix find_idlest_group() to handle CPU affinity

[PATCH] sched/fair: fix find_idlest_group() to handle CPU affinity

[Vincent Guittot]

Because of CPU affinity, the local group can be skipped which breaks the
assumption that statistics are always collected for local group. With
uninitialized local_sgs, the comparison is meaningless and the behavior
unpredictable. This can even end up to use local pointer which is to
NULL in this case.

If the local group has been skipped because of CPU affinity, we return
the idlest group.

Fixes: 57abff067a08 ("sched/fair: Rework find_idlest_group()")
Reported-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Tested-by: John Stultz <john.stultz@linaro.org>
---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 08a233e..146b6c8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8417,6 +8417,10 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p,
 	if (!idlest)
 		return NULL;
 
+	/* The local group has been skipped because of CPU affinity */
+	if (!local)
+		return idlest;
+
 	/*
 	 * If the local group is idler than the selected idlest group
 	 * don't try and push the task.
-- 
2.7.4

Re: [PATCH] sched/fair: fix find_idlest_group() to handle CPU affinity

[Valentin Schneider]

On 04/12/2019 18:21, Vincent Guittot wrote:
> Because of CPU affinity, the local group can be skipped which breaks the
> assumption that statistics are always collected for local group. With
> uninitialized local_sgs, the comparison is meaningless and the behavior
> unpredictable. This can even end up to use local pointer which is to
> NULL in this case.
> 
> If the local group has been skipped because of CPU affinity, we return
> the idlest group.
> 

I stared at find_idlest_group() before the rework out of curiosity and
AFAICT the "never visit local group" thing was there already. However, we
would only use the load and spare capacity of that group, and the relevant
variables where initialized to ULONG_MAX and 0 respectively. This would lead
us to return 'idlest' (or 'most_spare_sg', but it's the same as 'idlest' now).

So IMO this is just restoring the previous behaviour, which is what we want
methinks.

Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

Re: [PATCH] sched/fair: fix find_idlest_group() to handle CPU affinity

[Qais Yousef]

On 12/04/19 19:21, Vincent Guittot wrote:
> Because of CPU affinity, the local group can be skipped which breaks the
> assumption that statistics are always collected for local group. With
> uninitialized local_sgs, the comparison is meaningless and the behavior
> unpredictable. This can even end up to use local pointer which is to
> NULL in this case.

For the record; I think this is safe and I can't see how the local_sgs can be
used uninitialized, but experience shows that if this happened once it's likely
to happen again when things change. So I think it'd be safer to always
initialize local_sgs to something sensible and avoid future trouble. I don't
see any cost to initializing it.

My 2p :-) The change is good for me as-is otherwise.

Cheers

--
Qais Yousef

> 
> If the local group has been skipped because of CPU affinity, we return
> the idlest group.
> 
> Fixes: 57abff067a08 ("sched/fair: Rework find_idlest_group()")
> Reported-by: John Stultz <john.stultz@linaro.org>
> Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
> Tested-by: John Stultz <john.stultz@linaro.org>
> ---
>  kernel/sched/fair.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 08a233e..146b6c8 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -8417,6 +8417,10 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p,
>  	if (!idlest)
>  		return NULL;
>  
> +	/* The local group has been skipped because of CPU affinity */
> +	if (!local)
> +		return idlest;
> +
>  	/*
>  	 * If the local group is idler than the selected idlest group
>  	 * don't try and push the task.
> -- 
> 2.7.4
> 

Re: [PATCH] sched/fair: fix find_idlest_group() to handle CPU affinity

[John Stultz]

On Wed, Dec 4, 2019 at 10:21 AM Vincent Guittot
<vincent.guittot@linaro.org> wrote:
>
> Because of CPU affinity, the local group can be skipped which breaks the
> assumption that statistics are always collected for local group. With
> uninitialized local_sgs, the comparison is meaningless and the behavior
> unpredictable. This can even end up to use local pointer which is to
> NULL in this case.
>
> If the local group has been skipped because of CPU affinity, we return
> the idlest group.
>
> Fixes: 57abff067a08 ("sched/fair: Rework find_idlest_group()")
> Reported-by: John Stultz <john.stultz@linaro.org>
> Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
> Tested-by: John Stultz <john.stultz@linaro.org>

Just wanted to follow up on this, as its seemed to have missed -rc2?

thanks
-john

[tip: sched/urgent] sched/fair: Fix find_idlest_group() to handle CPU affinity

[tip-bot2 for Vincent Guittot]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     7ed735c33104f3c6194fbc67e3a8b6e64ae84ad1
Gitweb:        https://git.kernel.org/tip/7ed735c33104f3c6194fbc67e3a8b6e64ae84ad1
Author:        Vincent Guittot <vincent.guittot@linaro.org>
AuthorDate:    Wed, 04 Dec 2019 19:21:40 +01:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Tue, 17 Dec 2019 13:32:48 +01:00

sched/fair: Fix find_idlest_group() to handle CPU affinity

Because of CPU affinity, the local group can be skipped which breaks the
assumption that statistics are always collected for local group. With
uninitialized local_sgs, the comparison is meaningless and the behavior
unpredictable. This can even end up to use local pointer which is to
NULL in this case.

If the local group has been skipped because of CPU affinity, we return
the idlest group.

Fixes: 57abff067a08 ("sched/fair: Rework find_idlest_group()")
Reported-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: John Stultz <john.stultz@linaro.org>
Cc: rostedt@goodmis.org
Cc: valentin.schneider@arm.com
Cc: mingo@redhat.com
Cc: mgorman@suse.de
Cc: juri.lelli@redhat.com
Cc: dietmar.eggemann@arm.com
Cc: bsegall@google.com
Cc: qais.yousef@arm.com
Link: https://lkml.kernel.org/r/1575483700-22153-1-git-send-email-vincent.guittot@linaro.org
---
 kernel/sched/fair.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 08a233e..146b6c8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8417,6 +8417,10 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p,
 	if (!idlest)
 		return NULL;
 
+	/* The local group has been skipped because of CPU affinity */
+	if (!local)
+		return idlest;
+
 	/*
 	 * If the local group is idler than the selected idlest group
 	 * don't try and push the task.