From: Valentin Schneider <valentin.schneider@arm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@kernel.org, peterz@infradead.org,
vincent.guittot@linaro.org, morten.rasmussen@arm.com,
Dietmar.Eggemann@arm.com
Subject: [PATCH 0/5] sched/fair: NOHZ cleanups and misfit improvement
Date: Thu, 17 Jan 2019 15:34:06 +0000 [thread overview]
Message-ID: <20190117153411.2390-1-valentin.schneider@arm.com> (raw)
In
commit 5fbdfae5221a ("sched/fair: Kick nohz balance if rq->misfit_task_load")
was added a trigger for nohz kicks, which is required to offload misfit tasks
from LITTLE to big CPUs. However, those kicks could be issued a lot more
frequently than what is strictly needed.
This patch-set brings some unrelated nohz code cleanups, and tunes down
unneeded nohz kicks.
- Patches [1-3] do some cleaning up of the current nohz code
- Patches [4-5] tweak the nohz kick conditions for asymmetric systems
* Testing
** kick_ilb() hits
This causes a large reduction in calls to kick_ilb() (and thus subsequent
rescheduling interrupts & useless nohz balance calls) in most scenarios.
The "best case" one is running NR_BIG_CPUS big tasks, which I tested with
4 50% periodic tasks running for 5 seconds on my HiKey960 (4x4 big.LITTLE):
| CPU | hits (baseline) | hits (patchset) |
|-----+-----------------+-----------------|
| 0 | 31 | 41 |
| 1 | 21 | 3 |
| 2 | 35 | 2 |
| 3 | 9 | 4 |
|-----+-----------------+-----------------|
| 4 | 170 | 4 |
| 5 | 573 | 4 |
| 6 | 544 | 4 |
| 7 | 579 | 4 |
Something a bit less idealistic with NR_CPUS-1 big tasks still shows some
improvements (7 100% tasks running for 5 seconds on my HiKey960):
| CPU | hits (baseline) | hits (patchset) |
|-----+-----------------+-----------------|
| 0 | 14 | 122 |
| 1 | 47 | 162 |
| 2 | 11 | 156 |
| 3 | 9 | 3 |
|-----+-----------------+-----------------|
| 4 | 53 | 6 |
| 5 | 276 | 13 |
| 6 | 312 | 7 |
| 7 | 250 | 11 |
I was surprised to see such an increase in calls to kick_ilb() from LITTLE
CPUs ([0-3]), but after a bit of investigation it turns out that the big
CPUs would always run nohz_balancer_kick() a jiffy before the LITTLEs, so
the LITTLEs would always bail out because nohz.next_balance had just been
updated before they called nohz_balancer_kick(). IOW,
time_before(now, nohz.next_balance)
would always be true on CPUs [0-3] during my workload. Quieting the kicks
issued by the big CPUs allowed the LITTLEs to execute nohz_balancer_kick()
past that condition, explaining the higher number of kicks issued from LITTLE
CPUs.
** misfit behaviour
For good measure I also ran the usual misfit tests [1] which showed no
particular change.
* Notes
Since patch 05 introduces yet another label in kick_nohz_balancer(),
I benchmarked splitting its "slowpath" (all of the rcu derefs) into
its own function and didn't find any real overhead. If folks don't
hate the idea I can add it to a v2 or post it separately.
[1]: https://github.com/ARM-software/lisa/blob/next/lisa/tests/kernel/scheduler/misfit.py
Valentin Schneider (5):
sched/fair: Use for_each_cpu_and for asym-packing nohz kicks
sched/fair: Explain LLC nohz kick condition
sched/fair: Prune nohz_balancer_kick() comment block
sched/fair: Tune down misfit nohz kicks
sched/fair: Skip LLC nohz logic for asymmetric systems
kernel/sched/fair.c | 62 ++++++++++++++++++++++++++++++++-------------
1 file changed, 44 insertions(+), 18 deletions(-)
--
2.20.1
next reply other threads:[~2019-01-17 15:37 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-17 15:34 Valentin Schneider [this message]
2019-01-17 15:34 ` [PATCH 1/5] sched/fair: Use for_each_cpu_and for asym-packing nohz kicks Valentin Schneider
2019-02-11 10:50 ` [tip:sched/core] sched/fair: Simplify nohz_balancer_kick() tip-bot for Valentin Schneider
2019-01-17 15:34 ` [PATCH 2/5] sched/fair: Explain LLC nohz kick condition Valentin Schneider
2019-02-11 10:51 ` [tip:sched/core] " tip-bot for Valentin Schneider
2019-01-17 15:34 ` [PATCH 3/5] sched/fair: Prune nohz_balancer_kick() comment block Valentin Schneider
2019-02-11 10:51 ` [tip:sched/core] sched/fair: Prune, fix and simplify the " tip-bot for Valentin Schneider
2019-01-17 15:34 ` [PATCH 4/5] sched/fair: Tune down misfit nohz kicks Valentin Schneider
2019-02-06 16:04 ` Peter Zijlstra
2019-02-06 17:25 ` Valentin Schneider
2019-02-07 9:57 ` Peter Zijlstra
2019-01-17 15:34 ` [PATCH 5/5] sched/fair: Skip LLC nohz logic for asymmetric systems Valentin Schneider
2019-02-06 16:14 ` Peter Zijlstra
2019-02-06 17:26 ` Valentin Schneider
2019-02-07 9:56 ` Peter Zijlstra
2019-02-07 11:31 ` Valentin Schneider
2019-02-06 14:33 ` [PATCH 0/5] sched/fair: NOHZ cleanups and misfit improvement Valentin Schneider
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190117153411.2390-1-valentin.schneider@arm.com \
--to=valentin.schneider@arm.com \
--cc=Dietmar.Eggemann@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=morten.rasmussen@arm.com \
--cc=peterz@infradead.org \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.