From: "tip-bot2 for Josh Don" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Josh Don <joshdon@google.com>,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: sched/core] sched: reduce sched slice for SCHED_IDLE entities
Date: Thu, 09 Sep 2021 11:18:27 -0000 [thread overview]
Message-ID: <163118630759.25758.9246067278003147655.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20210820010403.946838-4-joshdon@google.com>
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 7e2ce158699bb7b6a489c7c1d89c0dde2d4ceef5
Gitweb: https://git.kernel.org/tip/7e2ce158699bb7b6a489c7c1d89c0dde2d4ceef5
Author: Josh Don <joshdon@google.com>
AuthorDate: Thu, 19 Aug 2021 18:04:02 -07:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Thu, 09 Sep 2021 11:27:31 +02:00
sched: reduce sched slice for SCHED_IDLE entities
Use a small, non-scaled min granularity for SCHED_IDLE entities, when
competing with normal entities. This reduces the latency of getting
a normal entity back on cpu, at the expense of increased context
switch frequency of SCHED_IDLE entities.
The benefit of this change is to reduce the round-robin latency for
normal entities when competing with a SCHED_IDLE entity.
Example: on a machine with HZ=1000, spawned two threads, one of which is
SCHED_IDLE, and affined to one cpu. Without this patch, the SCHED_IDLE
thread runs for 4ms then waits for 1.4s. With this patch, it runs for
1ms and waits 340ms (as it round-robins with the other thread).
Signed-off-by: Josh Don <joshdon@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://lore.kernel.org/r/20210820010403.946838-4-joshdon@google.com
---
kernel/sched/debug.c | 2 ++
kernel/sched/fair.c | 29 ++++++++++++++++++++++++-----
kernel/sched/sched.h | 1 +
3 files changed, 27 insertions(+), 5 deletions(-)
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 3353857..317ef56 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -305,6 +305,7 @@ static __init int sched_init_debug(void)
debugfs_create_u32("latency_ns", 0644, debugfs_sched, &sysctl_sched_latency);
debugfs_create_u32("min_granularity_ns", 0644, debugfs_sched, &sysctl_sched_min_granularity);
+ debugfs_create_u32("idle_min_granularity_ns", 0644, debugfs_sched, &sysctl_sched_idle_min_granularity);
debugfs_create_u32("wakeup_granularity_ns", 0644, debugfs_sched, &sysctl_sched_wakeup_granularity);
debugfs_create_u32("latency_warn_ms", 0644, debugfs_sched, &sysctl_resched_latency_warn_ms);
@@ -806,6 +807,7 @@ static void sched_debug_header(struct seq_file *m)
SEQ_printf(m, " .%-40s: %Ld.%06ld\n", #x, SPLIT_NS(x))
PN(sysctl_sched_latency);
PN(sysctl_sched_min_granularity);
+ PN(sysctl_sched_idle_min_granularity);
PN(sysctl_sched_wakeup_granularity);
P(sysctl_sched_child_runs_first);
P(sysctl_sched_features);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d7c0b9d..7330a77 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -60,6 +60,14 @@ unsigned int sysctl_sched_min_granularity = 750000ULL;
static unsigned int normalized_sysctl_sched_min_granularity = 750000ULL;
/*
+ * Minimal preemption granularity for CPU-bound SCHED_IDLE tasks.
+ * Applies only when SCHED_IDLE tasks compete with normal tasks.
+ *
+ * (default: 0.75 msec)
+ */
+unsigned int sysctl_sched_idle_min_granularity = 750000ULL;
+
+/*
* This value is kept at sysctl_sched_latency/sysctl_sched_min_granularity
*/
static unsigned int sched_nr_latency = 8;
@@ -665,6 +673,8 @@ static u64 __sched_period(unsigned long nr_running)
return sysctl_sched_latency;
}
+static bool sched_idle_cfs_rq(struct cfs_rq *cfs_rq);
+
/*
* We calculate the wall-time slice from the period by taking a part
* proportional to the weight.
@@ -674,6 +684,8 @@ static u64 __sched_period(unsigned long nr_running)
static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)
{
unsigned int nr_running = cfs_rq->nr_running;
+ struct sched_entity *init_se = se;
+ unsigned int min_gran;
u64 slice;
if (sched_feat(ALT_PERIOD))
@@ -684,12 +696,13 @@ static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)
for_each_sched_entity(se) {
struct load_weight *load;
struct load_weight lw;
+ struct cfs_rq *qcfs_rq;
- cfs_rq = cfs_rq_of(se);
- load = &cfs_rq->load;
+ qcfs_rq = cfs_rq_of(se);
+ load = &qcfs_rq->load;
if (unlikely(!se->on_rq)) {
- lw = cfs_rq->load;
+ lw = qcfs_rq->load;
update_load_add(&lw, se->load.weight);
load = &lw;
@@ -697,8 +710,14 @@ static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)
slice = __calc_delta(slice, se->load.weight, load);
}
- if (sched_feat(BASE_SLICE))
- slice = max(slice, (u64)sysctl_sched_min_granularity);
+ if (sched_feat(BASE_SLICE)) {
+ if (se_is_idle(init_se) && !sched_idle_cfs_rq(cfs_rq))
+ min_gran = sysctl_sched_idle_min_granularity;
+ else
+ min_gran = sysctl_sched_min_granularity;
+
+ slice = max_t(u64, slice, min_gran);
+ }
return slice;
}
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 413298d..6b2d8b7 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2400,6 +2400,7 @@ extern const_debug unsigned int sysctl_sched_migration_cost;
#ifdef CONFIG_SCHED_DEBUG
extern unsigned int sysctl_sched_latency;
extern unsigned int sysctl_sched_min_granularity;
+extern unsigned int sysctl_sched_idle_min_granularity;
extern unsigned int sysctl_sched_wakeup_granularity;
extern int sysctl_resched_latency_warn_ms;
extern int sysctl_resched_latency_warn_once;
next prev parent reply other threads:[~2021-09-09 11:18 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-20 1:03 [PATCH v3 0/4] SCHED_IDLE extensions Josh Don
2021-08-20 1:04 ` [PATCH v3 2/4] sched: account number of SCHED_IDLE entities on each cfs_rq Josh Don
2021-08-24 7:57 ` Vincent Guittot
2021-09-09 11:18 ` [tip: sched/core] sched: Account " tip-bot2 for Josh Don
2021-10-05 14:12 ` tip-bot2 for Josh Don
2021-08-20 1:04 ` [PATCH v3 3/4] sched: reduce sched slice for SCHED_IDLE entities Josh Don
2021-08-23 10:08 ` Vincent Guittot
2021-08-23 17:40 ` Josh Don
2021-08-24 7:56 ` Vincent Guittot
2021-08-24 10:24 ` Jiang Biao
2021-08-24 17:04 ` Josh Don
2021-08-25 2:43 ` Jiang Biao
2021-09-09 11:18 ` tip-bot2 for Josh Don [this message]
2021-10-05 14:12 ` [tip: sched/core] " tip-bot2 for Josh Don
2021-08-20 1:04 ` [PATCH v3 4/4] sched: adjust sleeper credit " Josh Don
2021-08-23 10:09 ` Vincent Guittot
2021-08-24 8:16 ` Jiang Biao
2021-08-24 17:12 ` Josh Don
2021-09-09 11:18 ` [tip: sched/core] " tip-bot2 for Josh Don
2021-10-05 14:12 ` tip-bot2 for Josh Don
[not found] ` <20210906124702.Q6G0oOWwFOmQSl_jmRms3XQgfz4ROzfE71r3SNgWSf0@z>
2021-09-06 12:47 ` [PATCH v3 1/4] sched: cgroup SCHED_IDLE support alexs
2021-08-20 8:39 ` Tao Zhou
2021-08-23 17:29 ` Josh Don
2021-08-23 22:45 ` Tao Zhou
2021-09-02 1:22 ` Daniel Jordan
2021-09-03 1:21 ` Josh Don
2021-09-08 20:38 ` Daniel Jordan
2021-09-08 18:36 ` Josh Don
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=163118630759.25758.9246067278003147655.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=joshdon@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=vincent.guittot@linaro.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).