From: Zhang Qiao <zhangqiao22@huawei.com>
To: <linux-kernel@vger.kernel.org>
Cc: <mingo@redhat.com>, <peterz@infradead.org>,
<juri.lelli@redhat.com>, <vincent.guittot@linaro.org>,
<dietmar.eggemann@arm.com>, <rostedt@goodmis.org>,
<bsegall@google.com>, <mgorman@suse.de>, <bristot@redhat.com>,
<vschneid@redhat.com>, <rkagan@amazon.de>,
<zhangqiao22@huawei.com>
Subject: [PATCH v2] sched/fair: sanitize vruntime of entity being migrated
Date: Mon, 6 Mar 2023 21:24:18 +0800 [thread overview]
Message-ID: <20230306132418.50389-1-zhangqiao22@huawei.com> (raw)
Commit 829c1651e9c4 ("sched/fair: sanitize vruntime of
entity being placed") fix an overflowing bug, but ignore
a case that se->exec_start is reset after a migration.
For fixing this case, we reset the vruntime of a long
sleeping task in migrate_task_rq_fair().
Fixes: 829c1651e9c4 ("sched/fair: sanitize vruntime of entity being placed")
Suggested-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Zhang Qiao <zhangqiao22@huawei.com>
---
v1 -> v2:
- fix some typos and update comments
- reformat the patch
---
kernel/sched/fair.c | 76 ++++++++++++++++++++++++++++++++-------------
1 file changed, 55 insertions(+), 21 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 7a1b1f855b96..74c9918ffe76 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4648,11 +4648,45 @@ static void check_spread(struct cfs_rq *cfs_rq, struct sched_entity *se)
#endif
}
+static inline bool entity_is_long_sleep(struct sched_entity *se)
+{
+ struct cfs_rq *cfs_rq;
+ u64 sleep_time;
+
+ if (se->exec_start == 0)
+ return false;
+
+ cfs_rq = cfs_rq_of(se);
+ sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start;
+ if ((s64)sleep_time > 60LL * NSEC_PER_SEC)
+ return true;
+
+ return false;
+}
+
+static inline u64 sched_sleeper_credit(struct sched_entity *se)
+{
+ unsigned long thresh;
+
+ if (se_is_idle(se))
+ thresh = sysctl_sched_min_granularity;
+ else
+ thresh = sysctl_sched_latency;
+
+ /*
+ * Halve their sleep time's effect, to allow
+ * for a gentler effect of sleepers:
+ */
+ if (sched_feat(GENTLE_FAIR_SLEEPERS))
+ thresh >>= 1;
+
+ return thresh;
+}
+
static void
place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
{
u64 vruntime = cfs_rq->min_vruntime;
- u64 sleep_time;
/*
* The 'current' period is already promised to the current tasks,
@@ -4664,23 +4698,8 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
vruntime += sched_vslice(cfs_rq, se);
/* sleeps up to a single latency don't count. */
- if (!initial) {
- unsigned long thresh;
-
- if (se_is_idle(se))
- thresh = sysctl_sched_min_granularity;
- else
- thresh = sysctl_sched_latency;
-
- /*
- * Halve their sleep time's effect, to allow
- * for a gentler effect of sleepers:
- */
- if (sched_feat(GENTLE_FAIR_SLEEPERS))
- thresh >>= 1;
-
- vruntime -= thresh;
- }
+ if (!initial)
+ vruntime -= sched_sleeper_credit(se);
/*
* Pull vruntime of the entity being placed to the base level of
@@ -4689,8 +4708,7 @@ place_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int initial)
* the base as it may be too far off and the comparison may get
* inversed due to s64 overflow.
*/
- sleep_time = rq_clock_task(rq_of(cfs_rq)) - se->exec_start;
- if ((s64)sleep_time > 60LL * NSEC_PER_SEC)
+ if (entity_is_long_sleep(se))
se->vruntime = vruntime;
else
se->vruntime = max_vruntime(se->vruntime, vruntime);
@@ -7635,7 +7653,23 @@ static void migrate_task_rq_fair(struct task_struct *p, int new_cpu)
if (READ_ONCE(p->__state) == TASK_WAKING) {
struct cfs_rq *cfs_rq = cfs_rq_of(se);
- se->vruntime -= u64_u32_load(cfs_rq->min_vruntime);
+ /*
+ * We determine whether a task sleeps for long by checking
+ * se->exec_start, and if it is, we sanitize its vruntime at
+ * place_entity(). However, after a migration, this detection
+ * method fails due to se->exec_start being reset.
+ *
+ * For fixing this case, we add the same check here. For a task
+ * which has slept for a long time, its vruntime should be reset
+ * to cfs_rq->min_vruntime with a sleep credit. Because waking
+ * task's vruntime will be added to cfs_rq->min_vruntime when
+ * enqueue, we only need to reset the se->vruntime of waking task
+ * to a credit here.
+ */
+ if (entity_is_long_sleep(se))
+ se->vruntime = -sched_sleeper_credit(se);
+ else
+ se->vruntime -= u64_u32_load(cfs_rq->min_vruntime);
}
if (!task_on_rq_migrating(p)) {
--
2.17.1
next reply other threads:[~2023-03-06 12:57 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-06 13:24 Zhang Qiao [this message]
2023-03-06 13:53 ` [PATCH v2] sched/fair: sanitize vruntime of entity being migrated Vincent Guittot
2023-03-07 10:26 ` Vincent Guittot
2023-03-07 11:05 ` Zhang Qiao
2023-03-07 13:41 ` Zhang Qiao
2023-03-08 8:01 ` Vincent Guittot
2023-03-08 12:55 ` Vincent Guittot
2023-03-09 8:37 ` Zhang Qiao
2023-03-09 9:09 ` Dietmar Eggemann
2023-03-09 9:30 ` Zhang Qiao
2023-03-09 10:48 ` Vincent Guittot
2023-03-09 14:23 ` Zhang Qiao
2023-03-07 2:16 ` kernel test robot
2023-03-07 12:45 ` Dietmar Eggemann
2023-03-07 14:06 ` Zhang Qiao
2023-03-09 9:43 ` Zhang Qiao
2023-03-08 14:33 ` Chen Yu
2023-03-09 13:05 ` Peter Zijlstra
2023-03-09 13:34 ` Vincent Guittot
2023-03-09 14:28 ` Peter Zijlstra
2023-03-09 14:36 ` Peter Zijlstra
2023-03-09 15:14 ` Vincent Guittot
2023-03-10 14:29 ` Vincent Guittot
2023-03-11 9:57 ` Zhang Qiao
2023-03-13 14:23 ` Vincent Guittot
2023-03-14 11:03 ` Zhang Qiao
2023-03-14 13:26 ` Vincent Guittot
2023-03-14 13:38 ` Zhang Qiao
2023-03-14 13:39 ` Vincent Guittot
2023-03-14 15:32 ` Vincent Guittot
2023-03-15 9:16 ` Zhang Qiao
2023-03-15 15:30 ` Vincent Guittot
2023-03-13 9:06 ` Dietmar Eggemann
2023-03-13 18:17 ` Dietmar Eggemann
2023-03-14 7:41 ` Vincent Guittot
2023-03-14 12:07 ` Peter Zijlstra
2023-03-14 13:24 ` Vincent Guittot
2023-03-14 17:16 ` Peter Zijlstra
2023-03-15 7:18 ` Vincent Guittot
2023-03-15 8:42 ` Vincent Guittot
2023-03-15 10:15 ` Dietmar Eggemann
2023-03-15 10:21 ` Vincent Guittot
2023-03-15 13:35 ` Dietmar Eggemann
2023-03-15 15:32 ` Vincent Guittot
2023-03-14 13:29 ` Dietmar Eggemann
2023-03-14 13:37 ` Dietmar Eggemann
2023-03-17 16:08 Vincent Guittot
2023-03-18 7:45 ` Zhang Qiao
2023-03-20 12:29 ` Dietmar Eggemann
2023-03-20 13:26 ` Vincent Guittot
2023-03-21 10:02 ` Peter Zijlstra
2023-03-21 10:29 ` Dietmar Eggemann
2023-03-21 10:49 ` Peter Zijlstra
2023-03-21 11:12 ` Vincent Guittot
2023-03-21 11:13 ` Dietmar Eggemann
2023-03-21 12:26 ` Peter Zijlstra
2023-03-21 12:28 ` Peter Zijlstra
2023-03-21 12:38 ` Vincent Guittot
2023-03-24 4:05 ` Chen Yu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230306132418.50389-1-zhangqiao22@huawei.com \
--to=zhangqiao22@huawei.com \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rkagan@amazon.de \
--cc=rostedt@goodmis.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.