From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
To: linux-kernel@vger.kernel.org
Cc: "Joel Fernandes (Google)" <joel@joelfernandes.org>,
Paul McKenney <paulmck@kernel.org>,
Frederic Weisbecker <fweisbec@gmail.com>,
Dietmar Eggeman <dietmar.eggemann@arm.com>,
Qais Yousef <qais.yousef@arm.com>,
Ben Segall <bsegall@google.com>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>, Mel Gorman <mgorman@suse.de>,
Peter Zijlstra <peterz@infradead.org>,
Steven Rostedt <rostedt@goodmis.org>,
Vincent Guittot <vincent.guittot@linaro.org>
Subject: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ
Date: Fri, 22 Jan 2021 10:46:00 -0500 [thread overview]
Message-ID: <20210122154600.1722680-1-joel@joelfernandes.org> (raw)
On an octacore ARM64 device running ChromeOS Linux kernel v5.4, I found
that there are a lot of calls to update_blocked_averages(). This causes
the schedule loop to slow down to taking upto 500 micro seconds at
times (due to newidle load balance). I have also seen this manifest in
the periodic balancer.
Closer look shows that the problem is caused by the following
ingredients:
1. If the system has a lot of inactive CGroups (thanks Dietmar for
suggesting to inspect /proc/sched_debug for this), this can make
__update_blocked_fair() take a long time.
2. The device has a lot of CPUs in a cluster which causes schedutil in a
shared frequency domain configuration to be slower than usual. (the load
average updates also try to update the frequency in schedutil).
3. The CPU is running at a low frequency causing the scheduler/schedutil
code paths to take longer than when running at a high CPU frequency.
The fix is simply rate limit the calls to update_blocked_averages to 20
times per second. It appears that updating the blocked average less
often is sufficient. Currently I see about 200 calls per second
sometimes, which seems overkill.
schbench shows a clear improvement with the change:
Without patch:
~/schbench -t 2 -m 2 -r 5
Latency percentiles (usec) runtime 5 (s) (212 total samples)
50.0th: 210 (106 samples)
75.0th: 619 (53 samples)
90.0th: 665 (32 samples)
95.0th: 703 (11 samples)
*99.0th: 12656 (8 samples)
99.5th: 12784 (1 samples)
99.9th: 13424 (1 samples)
min=15, max=13424
With patch:
~/schbench -t 2 -m 2 -r 5
Latency percentiles (usec) runtime 5 (s) (214 total samples)
50.0th: 188 (108 samples)
75.0th: 238 (53 samples)
90.0th: 623 (32 samples)
95.0th: 657 (12 samples)
*99.0th: 717 (7 samples)
99.5th: 725 (2 samples)
99.9th: 725 (0 samples)
Cc: Paul McKenney <paulmck@kernel.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Suggested-by: Dietmar Eggeman <dietmar.eggemann@arm.com>
Co-developed-by: Qais Yousef <qais.yousef@arm.com>
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 04a3ce20da67..fe2dc0024db5 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8381,7 +8381,7 @@ static bool update_nohz_stats(struct rq *rq, bool force)
if (!cpumask_test_cpu(cpu, nohz.idle_cpus_mask))
return false;
- if (!force && !time_after(jiffies, rq->last_blocked_load_update_tick))
+ if (!force && !time_after(jiffies, rq->last_blocked_load_update_tick + (HZ/20)))
return true;
update_blocked_averages(cpu);
--
2.30.0.280.ga3ce27912f-goog
next reply other threads:[~2021-01-22 17:41 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-22 15:46 Joel Fernandes (Google) [this message]
2021-01-22 16:56 ` [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ Vincent Guittot
2021-01-22 18:39 ` Qais Yousef
2021-01-22 19:14 ` Joel Fernandes
2021-01-25 13:23 ` Vincent Guittot
2021-01-26 16:36 ` Qais Yousef
2021-01-22 19:10 ` Joel Fernandes
2021-01-25 10:44 ` Dietmar Eggemann
2021-01-25 17:30 ` Vincent Guittot
2021-01-25 17:53 ` Dietmar Eggemann
2021-01-25 14:42 ` Vincent Guittot
2021-01-27 18:43 ` Joel Fernandes
2021-01-28 13:57 ` Vincent Guittot
2021-01-28 15:09 ` Joel Fernandes
2021-01-28 16:57 ` Qais Yousef
[not found] ` <CAKfTPtBvwm9vZb5C=2oTF6N-Ht6Rvip4Lv18yi7O3G8e-_ZWdg@mail.gmail.com>
2021-01-29 17:27 ` Vincent Guittot
2021-02-03 11:54 ` Dietmar Eggemann
2021-02-03 13:12 ` Vincent Guittot
2021-02-04 9:47 ` Dietmar Eggemann
2021-02-03 17:09 ` Qais Yousef
2021-02-03 17:35 ` Vincent Guittot
2021-02-04 10:45 ` Qais Yousef
2021-02-03 19:56 ` Joel Fernandes
2021-03-23 21:37 ` Tim Chen
2021-03-24 13:44 ` Vincent Guittot
2021-03-24 16:05 ` Tim Chen
2021-04-07 14:02 ` Vincent Guittot
2021-04-07 17:19 ` Tim Chen
2021-04-08 14:51 ` Vincent Guittot
2021-04-08 23:05 ` Tim Chen
2021-04-09 15:26 ` Vincent Guittot
2021-04-09 17:59 ` Tim Chen
2021-05-10 21:59 ` Tim Chen
2021-05-11 15:25 ` Vincent Guittot
2021-05-11 17:25 ` Tim Chen
2021-05-11 17:56 ` Vincent Guittot
2021-05-12 13:59 ` Qais Yousef
2021-05-13 18:45 ` Tim Chen
2021-05-17 16:14 ` Qais Yousef
2021-06-11 20:00 ` Tim Chen
2021-06-18 10:28 ` Vincent Guittot
2021-06-18 16:14 ` Tim Chen
2021-06-25 8:50 ` Vincent Guittot
2021-02-01 15:13 ` Joel Fernandes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210122154600.1722680-1-joel@joelfernandes.org \
--to=joel@joelfernandes.org \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=fweisbec@gmail.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=qais.yousef@arm.com \
--cc=rostedt@goodmis.org \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).