From: Morten Rasmussen <morten.rasmussen@arm.com>
To: Rik van Riel <riel@redhat.com>
Cc: peterz@infradead.org, mingo@redhat.com, dietmar.eggemann@arm.com,
yuyang.du@intel.com, vincent.guittot@linaro.org,
mgalbraith@suse.de, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 02/13] sched/fair: Consistent use of prev_cpu in wakeup path
Date: Thu, 23 Jun 2016 10:56:14 +0100 [thread overview]
Message-ID: <20160623095613.GA5606@e105550-lin.cambridge.arm.com> (raw)
In-Reply-To: <1466618651.15275.21.camel@redhat.com>
On Wed, Jun 22, 2016 at 02:04:11PM -0400, Rik van Riel wrote:
> On Wed, 2016-06-22 at 18:03 +0100, Morten Rasmussen wrote:
> > In commit ac66f5477239 ("sched/numa: Introduce migrate_swap()")
> > select_task_rq() got a 'cpu' argument to enable overriding of
> > prev_cpu
> > in special cases (NUMA task swapping). However, the
> > select_task_rq_fair() helper functions: wake_affine() and
> > select_idle_sibling(), still use task_cpu(p) directly to work out
> > prev_cpu which leads to inconsistencies.
> >
> > This patch passes prev_cpu (potentially overridden by NUMA code) into
> > the helper functions to ensure prev_cpu is indeed the same cpu
> > everywhere in the wakeup path.
> >
> > cc: Ingo Molnar <mingo@redhat.com>
> > cc: Peter Zijlstra <peterz@infradead.org>
> > cc: Rik van Riel <riel@redhat.com>
> >
> > Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
> > ---
> > kernel/sched/fair.c | 24 +++++++++++++-----------
> > 1 file changed, 13 insertions(+), 11 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index c6dd8bab010c..eec8e29104f9 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -656,7 +656,7 @@ static u64 sched_vslice(struct cfs_rq *cfs_rq,
> > struct sched_entity *se)
> > }
> >
> > #ifdef CONFIG_SMP
> > -static int select_idle_sibling(struct task_struct *p, int cpu);
> > +static int select_idle_sibling(struct task_struct *p, int prev_cpu,
> > int cpu);
> > static unsigned long task_h_load(struct task_struct *p);
> >
> > /*
> > @@ -1483,7 +1483,8 @@ static void task_numa_compare(struct
> > task_numa_env *env,
> > * Call select_idle_sibling to maybe find a better one.
> > */
> > if (!cur)
> > - env->dst_cpu = select_idle_sibling(env->p, env-
> > >dst_cpu);
> > + env->dst_cpu = select_idle_sibling(env->p, env-
> > >src_cpu,
> > + env->dst_cpu);
>
> It is worth remembering that "prev" will only
> ever be returned by select_idle_sibling() if
> it is part of the same NUMA node as target.
>
> That means this patch does not change behaviour
> of the NUMA balancing code, since that always
> migrates between nodes.
>
> Now lets look at try_to_wake_up(). It will pass
> p->wake_cpu as the argument for "prev_cpu", which
> again appears to be the same CPU number as that used
> by the current code.
IIUC, p->wake_cpu != task_cpu(p) if task_numa_migrate() decided to call
migrate_swap() on the task while it was sleeping intending it to swap
places with a task on a different NUMA node when it wakes up. Using
p->wake_cpu in select_idle_sibling() as "prev_cpu" when called through
try_to_wake_up()->select_task_rq() should only make a difference if the
target cpu happens to share cache with it and it is idle.
if (prev != target && cpus_share_cache(prev, target) && idle_cpu(prev))
return prev;
The selection of the target cpu for select_idle_sibling() is also
slightly affected as wake_affine() currently compares task_cpu(p) and
smp_processor_id(), and then picks p->wake_cpu or smp_processor_id()
depending on the outcome. With this patch wake_affine() uses
p->wake_cpu instead of task_cpu(p) so we actually compare the candidates
we choose between.
I think that would lead to some minor changes in behaviour in a few
corner cases, but I mainly wrote the patch as I thought it was very
confusing that we could have different "prev_cpu"s in different parts of
the select_task_rq_fair() code path.
>
> I have no objection to your patch, but must be
> overlooking something, since I cannot find a change
> in behaviour that your patch would create.
Thanks for confirming that it shouldn't change anything for NUMA load
balancing. That is what I hope for :-)
next prev parent reply other threads:[~2016-06-23 9:55 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-22 17:03 [PATCH v2 00/13] sched: Clean-ups and asymmetric cpu capacity support Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 01/13] sched: Fix power to capacity renaming in comment Morten Rasmussen
2016-08-10 18:03 ` [tip:sched/core] sched/core: " tip-bot for Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 02/13] sched/fair: Consistent use of prev_cpu in wakeup path Morten Rasmussen
2016-06-22 18:04 ` Rik van Riel
2016-06-23 9:56 ` Morten Rasmussen [this message]
2016-06-23 12:24 ` Rik van Riel
2016-08-10 18:03 ` [tip:sched/core] sched/fair: Make the use of prev_cpu consistent in the " tip-bot for Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 03/13] sched/fair: Optimize find_idlest_cpu() when there is no choice Morten Rasmussen
2016-07-13 12:20 ` Vincent Guittot
2016-08-10 18:03 ` [tip:sched/core] " tip-bot for Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 04/13] sched: Introduce SD_ASYM_CPUCAPACITY sched_domain topology flag Morten Rasmussen
2016-07-11 9:55 ` Peter Zijlstra
2016-07-11 10:42 ` Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 05/13] sched: Enable SD_BALANCE_WAKE for asymmetric capacity systems Morten Rasmussen
2016-07-11 10:04 ` Peter Zijlstra
2016-07-11 10:37 ` Morten Rasmussen
2016-07-11 11:04 ` Morten Rasmussen
2016-07-11 11:24 ` Peter Zijlstra
2016-07-12 14:26 ` Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 06/13] sched: Store maximum per-cpu capacity in root domain Morten Rasmussen
2016-07-11 10:18 ` Peter Zijlstra
2016-07-11 16:16 ` Dietmar Eggemann
2016-07-12 11:42 ` Peter Zijlstra
2016-07-13 11:18 ` Dietmar Eggemann
2016-07-13 12:40 ` Vincent Guittot
2016-07-13 13:48 ` Dietmar Eggemann
2016-07-13 16:37 ` Morten Rasmussen
2016-07-14 13:25 ` Vincent Guittot
2016-07-14 15:15 ` Morten Rasmussen
2016-07-15 11:46 ` Morten Rasmussen
2016-07-15 13:39 ` Vincent Guittot
2016-07-15 16:02 ` Morten Rasmussen
2016-07-18 12:48 ` Vincent Guittot
2016-07-18 15:11 ` Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 07/13] sched/fair: Let asymmetric cpu configurations balance at wake-up Morten Rasmussen
2016-07-11 11:13 ` Peter Zijlstra
2016-07-11 12:32 ` Morten Rasmussen
2016-07-13 12:56 ` Vincent Guittot
2016-07-13 16:14 ` Morten Rasmussen
2016-07-14 13:45 ` Vincent Guittot
2016-07-15 8:37 ` Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 08/13] sched/fair: Compute task/cpu utilization at wake-up more correctly Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 09/13] sched/fair: Consider spare capacity in find_idlest_group() Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 10/13] sched: Add per-cpu max capacity to sched_group_capacity Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 11/13] sched/fair: Avoid pulling tasks from non-overloaded higher capacity groups Morten Rasmussen
2016-06-23 21:20 ` Sai Gurrappadi
2016-06-30 7:49 ` Morten Rasmussen
2016-07-14 16:39 ` Sai Gurrappadi
2016-07-15 8:39 ` Morten Rasmussen
2016-07-12 12:59 ` Peter Zijlstra
2016-07-12 14:34 ` Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 12/13] arm: Set SD_ASYM_CPUCAPACITY for big.LITTLE platforms Morten Rasmussen
2016-06-22 17:03 ` [PATCH v2 13/13] arm: Update arch_scale_cpu_capacity() to reflect change to define Morten Rasmussen
2016-06-28 10:20 ` [PATCH v2 00/13] sched: Clean-ups and asymmetric cpu capacity support Koan-Sin Tan
2016-06-30 7:53 ` Morten Rasmussen
2016-07-08 7:35 ` KEITA KOBAYASHI
2016-07-08 8:18 ` Morten Rasmussen
2016-07-11 8:33 ` Morten Rasmussen
2016-07-11 12:44 ` Vincent Guittot
2016-07-12 13:25 ` Peter Zijlstra
2016-07-12 14:39 ` Morten Rasmussen
2016-07-13 12:06 ` Vincent Guittot
2016-07-13 15:54 ` Morten Rasmussen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160623095613.GA5606@e105550-lin.cambridge.arm.com \
--to=morten.rasmussen@arm.com \
--cc=dietmar.eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgalbraith@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=vincent.guittot@linaro.org \
--cc=yuyang.du@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).