From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Vincent Guittot <vincent.guittot@linaro.org>,
"peterz@infradead.org" <peterz@infradead.org>,
"mingo@kernel.org" <mingo@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"preeti@linux.vnet.ibm.com" <preeti@linux.vnet.ibm.com>,
Morten Rasmussen <Morten.Rasmussen@arm.com>,
"kamalesh@linux.vnet.ibm.com" <kamalesh@linux.vnet.ibm.com>
Cc: "riel@redhat.com" <riel@redhat.com>,
"efault@gmx.de" <efault@gmx.de>,
"nicolas.pitre@linaro.org" <nicolas.pitre@linaro.org>,
"linaro-kernel@lists.linaro.org" <linaro-kernel@lists.linaro.org>
Subject: Re: [PATCH v10 11/11] sched: move cfs task on a CPU with higher capacity
Date: Thu, 26 Mar 2015 14:19:35 +0000 [thread overview]
Message-ID: <55141577.8040902@arm.com> (raw)
In-Reply-To: <1425052454-25797-12-git-send-email-vincent.guittot@linaro.org>
On 27/02/15 15:54, Vincent Guittot wrote:
> When a CPU is used to handle a lot of IRQs or some RT tasks, the remaining
> capacity for CFS tasks can be significantly reduced. Once we detect such
> situation by comparing cpu_capacity_orig and cpu_capacity, we trig an idle
> load balance to check if it's worth moving its tasks on an idle CPU.
> It's worth trying to move the task before the CPU is fully utilized to
> minimize the preemption by irq or RT tasks.
>
> Once the idle load_balance has selected the busiest CPU, it will look for an
> active load balance for only two cases :
> - there is only 1 task on the busiest CPU.
> - we haven't been able to move a task of the busiest rq.
>
> A CPU with a reduced capacity is included in the 1st case, and it's worth to
> actively migrate its task if the idle CPU has got more available capacity for
> CFS tasks. This test has been added in need_active_balance.
>
> As a sidenote, this will not generate more spurious ilb because we already
> trig an ilb if there is more than 1 busy cpu. If this cpu is the only one that
> has a task, we will trig the ilb once for migrating the task.
>
> The nohz_kick_needed function has been cleaned up a bit while adding the new
> test
>
> env.src_cpu and env.src_rq must be set unconditionnally because they are used
> in need_active_balance which is called even if busiest->nr_running equals 1
>
> Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
> ---
> kernel/sched/fair.c | 69 ++++++++++++++++++++++++++++++++++++-----------------
> 1 file changed, 47 insertions(+), 22 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 7420d21..e70c315 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -6855,6 +6855,19 @@ static int need_active_balance(struct lb_env *env)
> return 1;
> }
>
> + /*
> + * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.
> + * It's worth migrating the task if the src_cpu's capacity is reduced
> + * because of other sched_class or IRQs if more capacity stays
> + * available on dst_cpu.
> + */
> + if ((env->idle != CPU_NOT_IDLE) &&
> + (env->src_rq->cfs.h_nr_running == 1)) {
> + if ((check_cpu_capacity(env->src_rq, sd)) &&
> + (capacity_of(env->src_cpu)*sd->imbalance_pct < capacity_of(env->dst_cpu)*100))
> + return 1;
> + }
> +
> return unlikely(sd->nr_balance_failed > sd->cache_nice_tries+2);
> }
>
> @@ -6954,6 +6967,9 @@ static int load_balance(int this_cpu, struct rq *this_rq,
>
> schedstat_add(sd, lb_imbalance[idle], env.imbalance);
>
> + env.src_cpu = busiest->cpu;
Isn't this 'env.src_cpu = busiest->cpu;' or 'env.src_cpu =
cpu_of(busiest);' already needed due to the existing ASYM_PACKING check
in need_active_balance() 'if ( ... && env->src_cpu > env->dst_cpu)' for
CPU_NEWLY_IDLE? Otherwise like you said, in these 'busiest->nr_running
equals 1' instances, env->src_cpu is un-initialized.
> + env.src_rq = busiest;
> +
> ld_moved = 0;
> if (busiest->nr_running > 1) {
> /*
> @@ -6963,8 +6979,6 @@ static int load_balance(int this_cpu, struct rq *this_rq,
> * correctly treated as an imbalance.
> */
> env.flags |= LBF_ALL_PINNED;
> - env.src_cpu = busiest->cpu;
> - env.src_rq = busiest;
> env.loop_max = min(sysctl_sched_nr_migrate, busiest->nr_running);
>
> more_balance:
[...]
next prev parent reply other threads:[~2015-03-26 14:19 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-02-27 15:54 [PATCH v10 00/11] sched: consolidation of CPU capacity and usage Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 01/11] sched: add utilization_avg_contrib Vincent Guittot
2015-03-27 11:40 ` [tip:sched/core] sched: Add sched_avg::utilization_avg_contrib tip-bot for Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 02/11] sched: Track group sched_entity usage contributions Vincent Guittot
2015-03-27 11:40 ` [tip:sched/core] " tip-bot for Morten Rasmussen
2015-02-27 15:54 ` [PATCH v10 03/11] sched: remove frequency scaling from cpu_capacity Vincent Guittot
2015-03-27 11:40 ` [tip:sched/core] sched: Remove " tip-bot for Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 04/11] sched: Make sched entity usage tracking scale-invariant Vincent Guittot
2015-03-03 12:51 ` Dietmar Eggemann
2015-03-04 7:54 ` Vincent Guittot
2015-03-04 7:46 ` Vincent Guittot
2015-03-27 11:40 ` [tip:sched/core] " tip-bot for Morten Rasmussen
2015-03-23 13:19 ` [PATCH v10 04/11] " Peter Zijlstra
2015-03-24 10:00 ` Vincent Guittot
2015-03-25 17:33 ` Peter Zijlstra
2015-03-25 18:08 ` Vincent Guittot
2015-03-26 17:38 ` Morten Rasmussen
2015-03-26 17:40 ` Morten Rasmussen
2015-03-26 17:46 ` [PATCH 1/2] sched: Change arch_scale_*() functions to scale input factor Morten Rasmussen
2015-03-26 17:46 ` [PATCH 2/2] sched: Make sched entity usage tracking scale-invariant Morten Rasmussen
2015-03-26 17:47 ` [PATCH v10 04/11] " Peter Zijlstra
2015-03-26 17:51 ` Morten Rasmussen
2015-03-27 8:17 ` Vincent Guittot
2015-03-27 9:05 ` Vincent Guittot
2015-04-02 16:53 ` Morten Rasmussen
2015-04-02 17:32 ` Peter Zijlstra
2015-04-07 13:31 ` Morten Rasmussen
2015-03-27 11:43 ` [tip:sched/core] sched: Optimize freq invariant accounting tip-bot for Peter Zijlstra
2015-02-27 15:54 ` [PATCH v10 05/11] sched: make scale_rt invariant with frequency Vincent Guittot
2015-03-27 11:41 ` [tip:sched/core] sched: Make " tip-bot for Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 06/11] sched: add per rq cpu_capacity_orig Vincent Guittot
2015-03-27 11:41 ` [tip:sched/core] sched: Add struct rq::cpu_capacity_orig tip-bot for Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 07/11] sched: get CPU's usage statistic Vincent Guittot
2015-03-03 12:47 ` Dietmar Eggemann
2015-03-04 7:53 ` Vincent Guittot
2015-03-04 7:48 ` Vincent Guittot
2015-03-27 11:41 ` [tip:sched/core] sched: Calculate CPU' s usage statistic and put it into struct sg_lb_stats::group_usage tip-bot for Vincent Guittot
2015-03-27 15:12 ` [PATCH v10 07/11] sched: get CPU's usage statistic Xunlei Pang
2015-03-27 15:37 ` Vincent Guittot
2015-04-01 3:22 ` Xunlei Pang
2015-02-27 15:54 ` [PATCH v10 08/11] sched: replace capacity_factor by usage Vincent Guittot
2015-03-27 11:42 ` [tip:sched/core] sched: Replace " tip-bot for Vincent Guittot
2015-03-27 14:52 ` [PATCH v10 08/11] sched: replace " Xunlei Pang
2015-03-27 15:59 ` Vincent Guittot
2015-04-01 3:37 ` Xunlei Pang
2015-04-01 9:06 ` Vincent Guittot
2015-04-01 14:54 ` Xunlei Pang
2015-04-01 15:57 ` Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 09/11] sched; remove unused capacity_orig from Vincent Guittot
2015-03-03 10:18 ` Morten Rasmussen
2015-03-03 10:33 ` Vincent Guittot
2015-03-03 10:35 ` [PATCH v10 09/11] sched; remove unused capacity_orig Vincent Guittot
2015-03-27 11:42 ` [tip:sched/core] sched: Remove unused struct sched_group_capacity ::capacity_orig tip-bot for Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 10/11] sched: add SD_PREFER_SIBLING for SMT level Vincent Guittot
2015-03-02 11:52 ` Srikar Dronamraju
2015-03-03 8:38 ` Vincent Guittot
2015-03-23 9:11 ` Peter Zijlstra
2015-03-23 9:59 ` Preeti U Murthy
2015-03-26 10:55 ` Peter Zijlstra
2015-03-26 12:03 ` Preeti U Murthy
2015-03-27 11:42 ` [tip:sched/core] sched: Add " tip-bot for Vincent Guittot
2015-02-27 15:54 ` [PATCH v10 11/11] sched: move cfs task on a CPU with higher capacity Vincent Guittot
2015-03-26 14:19 ` Dietmar Eggemann [this message]
2015-03-26 15:43 ` Vincent Guittot
2015-03-27 11:42 ` [tip:sched/core] sched: Move CFS tasks to CPUs " tip-bot for Vincent Guittot
2015-03-11 10:10 ` [PATCH v10 00/11] sched: consolidation of CPU capacity and usage Vincent Guittot
2015-04-02 1:47 ` Wanpeng Li
2015-04-02 7:30 ` Vincent Guittot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55141577.8040902@arm.com \
--to=dietmar.eggemann@arm.com \
--cc=Morten.Rasmussen@arm.com \
--cc=efault@gmx.de \
--cc=kamalesh@linux.vnet.ibm.com \
--cc=linaro-kernel@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=nicolas.pitre@linaro.org \
--cc=peterz@infradead.org \
--cc=preeti@linux.vnet.ibm.com \
--cc=riel@redhat.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).