From: Valentin Schneider <valentin.schneider@arm.com>
To: Steve Sistare <steven.sistare@oracle.com>,
mingo@redhat.com, peterz@infradead.org
Cc: subhra.mazumdar@oracle.com, dhaval.giani@oracle.com,
daniel.m.jordan@oracle.com, pavel.tatashin@microsoft.com,
matt@codeblueprint.co.uk, umgwanakikbuti@gmail.com,
riel@redhat.com, jbacik@fb.com, juri.lelli@redhat.com,
vincent.guittot@linaro.org, quentin.perret@arm.com,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4 00/10] steal tasks to improve CPU utilization
Date: Fri, 7 Dec 2018 20:30:03 +0000 [thread overview]
Message-ID: <e9ba0a93-4799-c230-8dcb-6580f6c00635@arm.com> (raw)
In-Reply-To: <1544131696-2888-1-git-send-email-steven.sistare@oracle.com>
Hi Steve,
On 06/12/2018 21:28, Steve Sistare wrote:
> When a CPU has no more CFS tasks to run, and idle_balance() fails to
> find a task, then attempt to steal a task from an overloaded CPU in the
> same LLC. Maintain and use a bitmap of overloaded CPUs to efficiently
> identify candidates. To minimize search time, steal the first migratable
> task that is found when the bitmap is traversed. For fairness, search
> for migratable tasks on an overloaded CPU in order of next to run.
>
> This simple stealing yields a higher CPU utilization than idle_balance()
> alone, because the search is cheap, so it may be called every time the CPU
> is about to go idle. idle_balance() does more work because it searches
> widely for the busiest queue, so to limit its CPU consumption, it declines
> to search if the system is too busy. Simple stealing does not offload the
> globally busiest queue, but it is much better than running nothing at all.
>
> The bitmap of overloaded CPUs is a new type of sparse bitmap, designed to
> reduce cache contention vs the usual bitmap when many threads concurrently
> set, clear, and visit elements.
>
> Patch 1 defines the sparsemask type and its operations.
>
> Patches 2, 3, and 4 implement the bitmap of overloaded CPUs.
>
> Patches 5 and 6 refactor existing code for a cleaner merge of later
> patches.
>
> Patches 7 and 8 implement task stealing using the overloaded CPUs bitmap.
>
> Patch 9 disables stealing on systems with more than 2 NUMA nodes for the
> time being because of performance regressions that are not due to stealing
> per-se. See the patch description for details.
>
> Patch 10 adds schedstats for comparing the new behavior to the old, and
> provided as a convenience for developers only, not for integration.
>
[...]
I've run my usual tests ([1]) on my HiKey960 with
- Just stealing (only misfit tests)
- Stealing rebased on top of EAS (misfit + EAS tests), and with stealing
gated by:
----->8-----
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 17ab4db..8b5172f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7152,7 +7152,8 @@ done: __maybe_unused;
rq_idle_stamp_update(rq);
new_tasks = idle_balance(rq, rf);
- if (new_tasks == 0)
+ if (new_tasks == 0 &&
+ (!static_key_unlikely(&sched_energy_present) || READ_ONCE(rq->rd->overutilized))
new_tasks = try_steal(rq, rf);
if (new_tasks)
-----8<-----
It all looks good from my end - if things were to go wrong on big.LITTLE
platforms it'd be here. It might be a convoluted way of using this tag,
but you can have my
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
as a "it doesn't break my stuff" seal.
As far as the patches go, with my last comments in mind it looks good to me
so you can also have:
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
for patches [2-8]. I haven't delved on the sparsemask details. As for patch
9, you might want to run other benchmarks (Peter suggested specjbb) to see
if it is truly need.
[1]: https://github.com/ARM-software/lisa/tree/next/lisa/tests/kernel/scheduler
next prev parent reply other threads:[~2018-12-07 20:30 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-06 21:28 [PATCH v4 00/10] steal tasks to improve CPU utilization Steve Sistare
2018-12-06 21:28 ` [PATCH v4 01/10] sched: Provide sparsemask, a reduced contention bitmap Steve Sistare
2019-01-31 19:18 ` Tim Chen
2018-12-06 21:28 ` [PATCH v4 02/10] sched/topology: Provide hooks to allocate data shared per LLC Steve Sistare
2018-12-06 21:28 ` [PATCH v4 03/10] sched/topology: Provide cfs_overload_cpus bitmap Steve Sistare
2018-12-07 20:20 ` Valentin Schneider
2018-12-07 22:35 ` Steven Sistare
2018-12-08 18:33 ` Valentin Schneider
2018-12-06 21:28 ` [PATCH v4 04/10] sched/fair: Dynamically update cfs_overload_cpus Steve Sistare
2018-12-07 20:20 ` Valentin Schneider
2018-12-07 22:35 ` Steven Sistare
2018-12-08 18:47 ` Valentin Schneider
2018-12-06 21:28 ` [PATCH v4 05/10] sched/fair: Hoist idle_stamp up from idle_balance Steve Sistare
2018-12-06 21:28 ` [PATCH v4 06/10] sched/fair: Generalize the detach_task interface Steve Sistare
2018-12-06 21:28 ` [PATCH v4 07/10] sched/fair: Provide can_migrate_task_llc Steve Sistare
2018-12-06 21:28 ` [PATCH v4 08/10] sched/fair: Steal work from an overloaded CPU when CPU goes idle Steve Sistare
2018-12-07 20:21 ` Valentin Schneider
2018-12-07 22:36 ` Steven Sistare
2018-12-08 18:39 ` Valentin Schneider
2018-12-06 21:28 ` [PATCH v4 09/10] sched/fair: disable stealing if too many NUMA nodes Steve Sistare
2018-12-07 11:43 ` Valentin Schneider
2018-12-07 13:37 ` Steven Sistare
2018-12-06 21:28 ` [PATCH v4 10/10] sched/fair: Provide idle search schedstats Steve Sistare
2018-12-07 11:56 ` Valentin Schneider
2018-12-07 13:45 ` Steven Sistare
2018-12-24 12:25 ` Rick Lindsley
2019-01-14 17:04 ` Steven Sistare
2018-12-07 20:30 ` Valentin Schneider [this message]
2018-12-07 22:36 ` [PATCH v4 00/10] steal tasks to improve CPU utilization Steven Sistare
2019-02-01 15:07 ` Valentin Schneider
2018-12-10 16:10 ` Vincent Guittot
2018-12-10 16:29 ` Steven Sistare
2018-12-10 16:33 ` Vincent Guittot
2018-12-10 17:08 ` Vincent Guittot
2018-12-10 17:20 ` Steven Sistare
2018-12-10 17:06 ` Valentin Schneider
2019-01-04 13:44 ` Shijith Thotton
2019-01-14 16:55 ` Steven Sistare
2019-01-31 17:16 ` Dhaval Giani
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e9ba0a93-4799-c230-8dcb-6580f6c00635@arm.com \
--to=valentin.schneider@arm.com \
--cc=daniel.m.jordan@oracle.com \
--cc=dhaval.giani@oracle.com \
--cc=jbacik@fb.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=matt@codeblueprint.co.uk \
--cc=mingo@redhat.com \
--cc=pavel.tatashin@microsoft.com \
--cc=peterz@infradead.org \
--cc=quentin.perret@arm.com \
--cc=riel@redhat.com \
--cc=steven.sistare@oracle.com \
--cc=subhra.mazumdar@oracle.com \
--cc=umgwanakikbuti@gmail.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).