From: Vincent Guittot <vincent.guittot@linaro.org> To: Peter Zijlstra <peterz@infradead.org> Cc: Ingo Molnar <mingo@kernel.org>, linux-kernel <linux-kernel@vger.kernel.org>, Russell King - ARM Linux <linux@arm.linux.org.uk>, LAK <linux-arm-kernel@lists.infradead.org>, Preeti U Murthy <preeti@linux.vnet.ibm.com>, Morten Rasmussen <Morten.Rasmussen@arm.com>, Mike Galbraith <efault@gmx.de>, Nicolas Pitre <nicolas.pitre@linaro.org>, "linaro-kernel@lists.linaro.org" <linaro-kernel@lists.linaro.org>, Daniel Lezcano <daniel.lezcano@linaro.org> Subject: Re: [PATCH v2 10/11] sched: move cfs task on a CPU with higher capacity Date: Tue, 3 Jun 2014 14:31:38 +0200 [thread overview] Message-ID: <CAKfTPtBRkWY1hTBqd9cQKetFbWEVvJVo-wKUz9MN-ZfZLq4TuA@mail.gmail.com> (raw) In-Reply-To: <20140603111532.GW11096@twins.programming.kicks-ass.net> On 3 June 2014 13:15, Peter Zijlstra <peterz@infradead.org> wrote: > On Mon, Jun 02, 2014 at 07:06:44PM +0200, Vincent Guittot wrote: >> > Could you detail those conditions? FWIW those make excellent Changelog >> > material. >> >> I have looked back into my tests and traces: >> >> In a 1st test, the capacity of the CPU was still above half default >> value (power=538) unlike what i remembered. So it's some what "normal" >> to keep the task on CPU0 which also handles IRQ because sg_capacity >> still returns 1. > > OK, so I suspect that once we move to utilization based capacity stuff > we'll do the migration IF the task indeed requires more cpu than can be > provided by the reduced, one, right? The current version of the patchset only checks if the capacity of a CPU has significantly reduced that we should look for another CPU. But we effectively could also add compare the remaining capacity with the task load > >> In a 2nd test,the main task runs (most of the time) on CPU0 whereas >> the max power of the latter is only 623 and the cpu_power goes below >> 512 (power=330) during the use case. So the sg_capacity of CPU0 is >> null but the main task still stays on CPU0. >> The use case (scp transfer) is made of a long running task (ssh) and a >> periodic short task (scp). ssh runs on CPU0 and scp runs each 6ms on >> CPU1. The newly idle load balance on CPU1 doesn't pull the long >> running task although sg_capacity is null because of >> sd->nr_balance_failed is never incremented and load_balance doesn't >> trig an active load_balance. When an idle balance occurs in the middle >> of the newly idle balance, the ssh long task migrates on CPU1 but as >> soon as it sleeps and wakes up, it goes back on CPU0 because of the >> wake affine which migrates it back on CPU0 (issue solved by patch 09). > > OK, so there's two problems here, right? > 1) we don't migrate away from cpu0 > 2) if we do, we get pulled back. > > And patch 9 solves 2, so maybe enhance its changelog to mention this > slightly more explicit. > > Which leaves us with 1.. interesting problem. I'm just not sure > endlessly kicking a low capacity cpu is the right fix for that. What prevent us to migrate the task directly is the fact that nr_balance_failed is not incremented for newly idle and it's the only condition for active migration (except asym feature) We could add a additional test in need_active_balance for newly_idle load balance. Something like: if ((sd->flags & SD_SHARE_PKG_RESOURCES) && (senv->rc_rq->cpu_power_orig * 100) > (env->src_rq->group_power * env->sd->imbalance_pct)) return 1; > >
WARNING: multiple messages have this Message-ID (diff)
From: vincent.guittot@linaro.org (Vincent Guittot) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 10/11] sched: move cfs task on a CPU with higher capacity Date: Tue, 3 Jun 2014 14:31:38 +0200 [thread overview] Message-ID: <CAKfTPtBRkWY1hTBqd9cQKetFbWEVvJVo-wKUz9MN-ZfZLq4TuA@mail.gmail.com> (raw) In-Reply-To: <20140603111532.GW11096@twins.programming.kicks-ass.net> On 3 June 2014 13:15, Peter Zijlstra <peterz@infradead.org> wrote: > On Mon, Jun 02, 2014 at 07:06:44PM +0200, Vincent Guittot wrote: >> > Could you detail those conditions? FWIW those make excellent Changelog >> > material. >> >> I have looked back into my tests and traces: >> >> In a 1st test, the capacity of the CPU was still above half default >> value (power=538) unlike what i remembered. So it's some what "normal" >> to keep the task on CPU0 which also handles IRQ because sg_capacity >> still returns 1. > > OK, so I suspect that once we move to utilization based capacity stuff > we'll do the migration IF the task indeed requires more cpu than can be > provided by the reduced, one, right? The current version of the patchset only checks if the capacity of a CPU has significantly reduced that we should look for another CPU. But we effectively could also add compare the remaining capacity with the task load > >> In a 2nd test,the main task runs (most of the time) on CPU0 whereas >> the max power of the latter is only 623 and the cpu_power goes below >> 512 (power=330) during the use case. So the sg_capacity of CPU0 is >> null but the main task still stays on CPU0. >> The use case (scp transfer) is made of a long running task (ssh) and a >> periodic short task (scp). ssh runs on CPU0 and scp runs each 6ms on >> CPU1. The newly idle load balance on CPU1 doesn't pull the long >> running task although sg_capacity is null because of >> sd->nr_balance_failed is never incremented and load_balance doesn't >> trig an active load_balance. When an idle balance occurs in the middle >> of the newly idle balance, the ssh long task migrates on CPU1 but as >> soon as it sleeps and wakes up, it goes back on CPU0 because of the >> wake affine which migrates it back on CPU0 (issue solved by patch 09). > > OK, so there's two problems here, right? > 1) we don't migrate away from cpu0 > 2) if we do, we get pulled back. > > And patch 9 solves 2, so maybe enhance its changelog to mention this > slightly more explicit. > > Which leaves us with 1.. interesting problem. I'm just not sure > endlessly kicking a low capacity cpu is the right fix for that. What prevent us to migrate the task directly is the fact that nr_balance_failed is not incremented for newly idle and it's the only condition for active migration (except asym feature) We could add a additional test in need_active_balance for newly_idle load balance. Something like: if ((sd->flags & SD_SHARE_PKG_RESOURCES) && (senv->rc_rq->cpu_power_orig * 100) > (env->src_rq->group_power * env->sd->imbalance_pct)) return 1; > >
next prev parent reply other threads:[~2014-06-03 12:32 UTC|newest] Thread overview: 216+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-05-23 15:52 [PATCH v2 00/11] sched: consolidation of cpu_power Vincent Guittot 2014-05-23 15:52 ` Vincent Guittot 2014-05-23 15:52 ` [PATCH v2 01/11] sched: fix imbalance flag reset Vincent Guittot 2014-05-23 15:52 ` Vincent Guittot 2014-05-25 10:33 ` Preeti U Murthy 2014-05-25 10:33 ` Preeti U Murthy 2014-05-26 7:49 ` Vincent Guittot 2014-05-26 7:49 ` Vincent Guittot 2014-05-26 9:16 ` Preeti U Murthy 2014-05-26 9:16 ` Preeti U Murthy 2014-05-26 10:14 ` Vincent Guittot 2014-05-26 10:14 ` Vincent Guittot 2014-05-23 15:52 ` [PATCH v2 02/11] sched: remove a wake_affine condition Vincent Guittot 2014-05-23 15:52 ` Vincent Guittot 2014-05-27 12:48 ` Peter Zijlstra 2014-05-27 12:48 ` Peter Zijlstra 2014-05-27 15:19 ` Vincent Guittot 2014-05-27 15:19 ` Vincent Guittot 2014-05-27 15:39 ` Peter Zijlstra 2014-05-27 15:39 ` Peter Zijlstra 2014-05-27 16:14 ` Vincent Guittot 2014-05-27 16:14 ` Vincent Guittot 2014-05-28 6:49 ` Vincent Guittot 2014-05-28 6:49 ` Vincent Guittot 2014-05-28 15:09 ` Dietmar Eggemann 2014-05-28 15:09 ` Dietmar Eggemann 2014-05-28 15:25 ` Vincent Guittot 2014-05-28 15:25 ` Vincent Guittot 2014-05-27 13:43 ` Peter Zijlstra 2014-05-27 13:43 ` Peter Zijlstra 2014-05-27 13:45 ` Peter Zijlstra 2014-05-27 13:45 ` Peter Zijlstra 2014-05-27 15:20 ` Vincent Guittot 2014-05-27 15:20 ` Vincent Guittot 2014-05-23 15:52 ` [PATCH v2 03/11] sched: fix avg_load computation Vincent Guittot 2014-05-23 15:52 ` Vincent Guittot 2014-05-27 13:48 ` Peter Zijlstra 2014-05-27 13:48 ` Peter Zijlstra 2014-05-23 15:52 ` [PATCH v2 04/11] sched: Allow all archs to set the power_orig Vincent Guittot 2014-05-23 15:52 ` Vincent Guittot 2014-05-30 14:04 ` Dietmar Eggemann 2014-05-30 14:04 ` Dietmar Eggemann 2014-05-30 14:46 ` Peter Zijlstra 2014-05-30 14:46 ` Peter Zijlstra 2014-05-30 20:50 ` Vincent Guittot 2014-05-30 20:50 ` Vincent Guittot 2014-06-04 9:42 ` Dietmar Eggemann 2014-06-04 9:42 ` Dietmar Eggemann 2014-06-04 11:15 ` Vincent Guittot 2014-06-04 11:15 ` Vincent Guittot 2014-06-05 8:59 ` Dietmar Eggemann 2014-06-05 8:59 ` Dietmar Eggemann 2014-06-16 9:01 ` Vincent Guittot 2014-06-16 9:01 ` Vincent Guittot 2014-06-03 13:22 ` Morten Rasmussen 2014-06-03 13:22 ` Morten Rasmussen 2014-06-03 14:02 ` Vincent Guittot 2014-06-03 14:02 ` Vincent Guittot 2014-06-04 11:17 ` Morten Rasmussen 2014-06-04 11:17 ` Morten Rasmussen 2014-06-06 7:01 ` Vincent Guittot 2014-06-06 7:01 ` Vincent Guittot 2014-05-23 15:52 ` [PATCH v2 05/11] ARM: topology: use new cpu_power interface Vincent Guittot 2014-05-23 15:52 ` Vincent Guittot 2014-05-25 13:22 ` Preeti U Murthy 2014-05-25 13:22 ` Preeti U Murthy 2014-05-26 8:25 ` Vincent Guittot 2014-05-26 8:25 ` Vincent Guittot 2014-05-26 9:19 ` Preeti U Murthy 2014-05-26 9:19 ` Preeti U Murthy 2014-05-23 15:53 ` [PATCH v2 06/11] sched: add per rq cpu_power_orig Vincent Guittot 2014-05-23 15:53 ` Vincent Guittot 2014-05-23 15:53 ` [PATCH v2 07/11] Revert "sched: Put rq's sched_avg under CONFIG_FAIR_GROUP_SCHED" Vincent Guittot 2014-05-23 15:53 ` Vincent Guittot 2014-05-23 15:53 ` [PATCH v2 08/11] sched: get CPU's activity statistic Vincent Guittot 2014-05-23 15:53 ` Vincent Guittot 2014-05-27 17:32 ` Peter Zijlstra 2014-05-27 17:32 ` Peter Zijlstra 2014-05-28 7:01 ` Vincent Guittot 2014-05-28 7:01 ` Vincent Guittot 2014-05-28 12:10 ` Morten Rasmussen 2014-05-28 12:10 ` Morten Rasmussen 2014-05-28 13:15 ` Vincent Guittot 2014-05-28 13:15 ` Vincent Guittot 2014-05-28 15:47 ` Morten Rasmussen 2014-05-28 15:47 ` Morten Rasmussen 2014-05-28 16:39 ` Vincent Guittot 2014-05-28 16:39 ` Vincent Guittot 2014-06-03 12:03 ` Morten Rasmussen 2014-06-03 12:03 ` Morten Rasmussen 2014-06-03 15:59 ` Peter Zijlstra 2014-06-03 15:59 ` Peter Zijlstra 2014-06-03 17:41 ` Morten Rasmussen 2014-06-03 17:41 ` Morten Rasmussen 2014-06-03 15:50 ` Peter Zijlstra 2014-06-03 15:50 ` Peter Zijlstra 2014-06-03 17:20 ` Morten Rasmussen 2014-06-03 17:20 ` Morten Rasmussen 2014-06-04 7:47 ` Vincent Guittot 2014-06-04 7:47 ` Vincent Guittot 2014-06-04 8:08 ` Peter Zijlstra 2014-06-04 8:08 ` Peter Zijlstra 2014-06-04 8:55 ` Morten Rasmussen 2014-06-04 8:55 ` Morten Rasmussen 2014-06-04 9:23 ` Peter Zijlstra 2014-06-04 9:23 ` Peter Zijlstra 2014-06-04 9:35 ` Vincent Guittot 2014-06-04 9:35 ` Vincent Guittot 2014-06-04 10:25 ` Morten Rasmussen 2014-06-04 10:25 ` Morten Rasmussen 2014-06-04 9:44 ` Morten Rasmussen 2014-06-04 9:44 ` Morten Rasmussen 2014-06-04 9:32 ` Vincent Guittot 2014-06-04 9:32 ` Vincent Guittot 2014-06-04 10:00 ` Morten Rasmussen 2014-06-04 10:00 ` Morten Rasmussen 2014-06-04 10:17 ` Peter Zijlstra 2014-06-04 10:17 ` Peter Zijlstra 2014-06-04 10:36 ` Morten Rasmussen 2014-06-04 10:36 ` Morten Rasmussen 2014-06-04 10:55 ` Peter Zijlstra 2014-06-04 10:55 ` Peter Zijlstra 2014-06-04 11:07 ` Vincent Guittot 2014-06-04 11:07 ` Vincent Guittot 2014-06-04 11:23 ` Morten Rasmussen 2014-06-04 11:23 ` Morten Rasmussen 2014-06-04 11:52 ` Vincent Guittot 2014-06-04 11:52 ` Vincent Guittot 2014-06-04 13:09 ` Morten Rasmussen 2014-06-04 13:09 ` Morten Rasmussen 2014-06-04 13:23 ` Morten Rasmussen 2014-06-04 13:23 ` Morten Rasmussen 2014-05-28 15:17 ` Peter Zijlstra 2014-05-28 15:17 ` Peter Zijlstra 2014-06-03 15:40 ` Peter Zijlstra 2014-06-03 15:40 ` Peter Zijlstra 2014-06-03 17:16 ` Morten Rasmussen 2014-06-03 17:16 ` Morten Rasmussen 2014-06-03 17:37 ` Peter Zijlstra 2014-06-03 17:37 ` Peter Zijlstra 2014-06-03 17:39 ` Peter Zijlstra 2014-06-03 17:39 ` Peter Zijlstra 2014-06-03 23:11 ` Yuyang Du 2014-06-03 23:11 ` Yuyang Du 2014-05-30 9:50 ` Dietmar Eggemann 2014-05-30 9:50 ` Dietmar Eggemann 2014-05-30 19:20 ` Vincent Guittot 2014-05-30 19:20 ` Vincent Guittot 2014-06-01 11:33 ` Dietmar Eggemann 2014-06-01 11:33 ` Dietmar Eggemann 2014-06-02 14:07 ` Vincent Guittot 2014-06-02 14:07 ` Vincent Guittot 2014-05-23 15:53 ` [PATCH v2 09/11] sched: test the cpu's capacity in wake affine Vincent Guittot 2014-05-23 15:53 ` Vincent Guittot 2014-05-28 10:58 ` Peter Zijlstra 2014-05-28 10:58 ` Peter Zijlstra 2014-05-28 11:15 ` Vincent Guittot 2014-05-28 11:15 ` Vincent Guittot 2014-11-24 0:34 ` Wanpeng Li 2014-11-24 0:34 ` Wanpeng Li 2014-11-24 13:23 ` Vincent Guittot 2014-11-24 13:23 ` Vincent Guittot 2014-05-23 15:53 ` [PATCH v2 10/11] sched: move cfs task on a CPU with higher capacity Vincent Guittot 2014-05-23 15:53 ` Vincent Guittot 2014-05-29 9:50 ` Peter Zijlstra 2014-05-29 9:50 ` Peter Zijlstra 2014-05-29 19:37 ` Vincent Guittot 2014-05-29 19:37 ` Vincent Guittot 2014-05-30 6:29 ` Peter Zijlstra 2014-05-30 6:29 ` Peter Zijlstra 2014-05-30 20:05 ` Vincent Guittot 2014-05-30 20:05 ` Vincent Guittot 2014-06-02 17:06 ` Vincent Guittot 2014-06-02 17:06 ` Vincent Guittot 2014-06-03 11:15 ` Peter Zijlstra 2014-06-03 11:15 ` Peter Zijlstra 2014-06-03 12:31 ` Vincent Guittot [this message] 2014-06-03 12:31 ` Vincent Guittot 2014-05-29 14:04 ` Peter Zijlstra 2014-05-29 14:04 ` Peter Zijlstra 2014-05-29 19:44 ` Vincent Guittot 2014-05-29 19:44 ` Vincent Guittot 2014-05-30 13:26 ` Dietmar Eggemann 2014-05-30 13:26 ` Dietmar Eggemann 2014-05-30 19:24 ` Vincent Guittot 2014-05-30 19:24 ` Vincent Guittot 2014-05-30 19:45 ` Nicolas Pitre 2014-05-30 19:45 ` Nicolas Pitre 2014-05-30 20:07 ` Vincent Guittot 2014-05-30 20:07 ` Vincent Guittot 2014-05-23 15:53 ` [PATCH v2 11/11] sched: replace capacity by activity Vincent Guittot 2014-05-23 15:53 ` Vincent Guittot 2014-05-29 13:55 ` Peter Zijlstra 2014-05-29 13:55 ` Peter Zijlstra 2014-05-29 19:51 ` Vincent Guittot 2014-05-29 19:51 ` Vincent Guittot 2014-06-02 6:21 ` Preeti U Murthy 2014-06-02 6:21 ` Preeti U Murthy 2014-06-03 9:50 ` Vincent Guittot 2014-06-03 9:50 ` Vincent Guittot 2014-05-29 14:02 ` Peter Zijlstra 2014-05-29 14:02 ` Peter Zijlstra 2014-05-29 19:56 ` Vincent Guittot 2014-05-29 19:56 ` Vincent Guittot 2014-05-30 6:34 ` Peter Zijlstra 2014-05-30 6:34 ` Peter Zijlstra 2014-05-30 19:13 ` Vincent Guittot 2014-05-30 19:13 ` Vincent Guittot 2014-05-26 9:44 ` [PATCH v2 00/11] sched: consolidation of cpu_power Preeti U Murthy 2014-05-26 9:44 ` Preeti U Murthy 2014-05-26 10:04 ` Vincent Guittot 2014-05-26 10:04 ` Vincent Guittot 2014-05-26 15:54 ` Vincent Guittot 2014-05-26 15:54 ` Vincent Guittot 2014-05-27 5:47 ` Preeti U Murthy 2014-05-27 5:47 ` Preeti U Murthy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CAKfTPtBRkWY1hTBqd9cQKetFbWEVvJVo-wKUz9MN-ZfZLq4TuA@mail.gmail.com \ --to=vincent.guittot@linaro.org \ --cc=Morten.Rasmussen@arm.com \ --cc=daniel.lezcano@linaro.org \ --cc=efault@gmx.de \ --cc=linaro-kernel@lists.linaro.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux@arm.linux.org.uk \ --cc=mingo@kernel.org \ --cc=nicolas.pitre@linaro.org \ --cc=peterz@infradead.org \ --cc=preeti@linux.vnet.ibm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.