From: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp> To: LKML <linux-kernel@vger.kernel.org>, cgroups <cgroups@vger.kernel.org> Cc: Ingo Molnar <mingo@elte.hu>, Peter Zijlstra <a.p.zijlstra@chello.nl> Subject: [PATCH 3/3] sched: fix cgroup movement of waking process Date: Tue, 13 Dec 2011 15:59:33 +0900 [thread overview] Message-ID: <20111213155933.36cd824a.nishimura@mxp.nes.nec.co.jp> (raw) In-Reply-To: <20111213155710.5b453415.nishimura@mxp.nes.nec.co.jp> There is a small race between try_to_wake_up() and sched_move_task(), which is trying to move the process being woken up. try_to_wake_up() on CPU0 sched_move_task() on CPU1 --------------------------------+--------------------------------- raw_spin_lock_irqsave(p->pi_lock) task_waking_fair() ->p.se.vruntime -= cfs_rq->min_vruntime ttwu_queue() ->send reschedule IPI to another CPU1 raw_spin_unlock_irqsave(p->pi_lock) task_rq_lock() -> tring to aquire both p->pi_lock and rq->lock with IRQ disabled task_move_group_fair() ->p.se.vruntime -= (old)cfs_rq->min_vruntime ->p.se.vruntime += (new)cfs_rq->min_vruntime task_rq_unlock() (via IPI) sched_ttwu_pending() raw_spin_lock(rq->lock) ttwu_do_activate() ... enqueue_entity() child.se->vruntime += cfs_rq->min_vruntime raw_spin_unlock(rq->lock) As a result, vruntime of the process becomes far bigger than min_vruntime, if (new)cfs_rq->min_vruntime >> (old)cfs_rq->min_vruntime. This patch fixes this problem by just ignoring such process in task_move_group_fair(), because the vruntime has already been normalized in task_waking_fair(). Signed-off-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp> --- kernel/sched_fair.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched_fair.c b/kernel/sched_fair.c index bdaa4ab..3feb3a2 100644 --- a/kernel/sched_fair.c +++ b/kernel/sched_fair.c @@ -4925,10 +4925,10 @@ static void task_move_group_fair(struct task_struct *p, int on_rq) * to another cgroup's rq. This does somewhat interfere with the * fair sleeper stuff for the first placement, but who cares. */ - if (!on_rq && p->state != TASK_RUNNING) + if (!on_rq && p->state != TASK_RUNNING && p->state != TASK_WAKING) p->se.vruntime -= cfs_rq_of(&p->se)->min_vruntime; set_task_rq(p, task_cpu(p)); - if (!on_rq && p->state != TASK_RUNNING) + if (!on_rq && p->state != TASK_RUNNING && p->state != TASK_WAKING) p->se.vruntime += cfs_rq_of(&p->se)->min_vruntime; } #endif -- 1.7.1
WARNING: multiple messages have this Message-ID (diff)
From: Daisuke Nishimura <nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org> To: LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> Cc: Ingo Molnar <mingo-X9Un+BFzKDI@public.gmane.org>, Peter Zijlstra <a.p.zijlstra-/NLkJaSkS4VmR6Xm/wNWPw@public.gmane.org> Subject: [PATCH 3/3] sched: fix cgroup movement of waking process Date: Tue, 13 Dec 2011 15:59:33 +0900 [thread overview] Message-ID: <20111213155933.36cd824a.nishimura@mxp.nes.nec.co.jp> (raw) In-Reply-To: <20111213155710.5b453415.nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org> There is a small race between try_to_wake_up() and sched_move_task(), which is trying to move the process being woken up. try_to_wake_up() on CPU0 sched_move_task() on CPU1 --------------------------------+--------------------------------- raw_spin_lock_irqsave(p->pi_lock) task_waking_fair() ->p.se.vruntime -= cfs_rq->min_vruntime ttwu_queue() ->send reschedule IPI to another CPU1 raw_spin_unlock_irqsave(p->pi_lock) task_rq_lock() -> tring to aquire both p->pi_lock and rq->lock with IRQ disabled task_move_group_fair() ->p.se.vruntime -= (old)cfs_rq->min_vruntime ->p.se.vruntime += (new)cfs_rq->min_vruntime task_rq_unlock() (via IPI) sched_ttwu_pending() raw_spin_lock(rq->lock) ttwu_do_activate() ... enqueue_entity() child.se->vruntime += cfs_rq->min_vruntime raw_spin_unlock(rq->lock) As a result, vruntime of the process becomes far bigger than min_vruntime, if (new)cfs_rq->min_vruntime >> (old)cfs_rq->min_vruntime. This patch fixes this problem by just ignoring such process in task_move_group_fair(), because the vruntime has already been normalized in task_waking_fair(). Signed-off-by: Daisuke Nishimura <nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org> --- kernel/sched_fair.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched_fair.c b/kernel/sched_fair.c index bdaa4ab..3feb3a2 100644 --- a/kernel/sched_fair.c +++ b/kernel/sched_fair.c @@ -4925,10 +4925,10 @@ static void task_move_group_fair(struct task_struct *p, int on_rq) * to another cgroup's rq. This does somewhat interfere with the * fair sleeper stuff for the first placement, but who cares. */ - if (!on_rq && p->state != TASK_RUNNING) + if (!on_rq && p->state != TASK_RUNNING && p->state != TASK_WAKING) p->se.vruntime -= cfs_rq_of(&p->se)->min_vruntime; set_task_rq(p, task_cpu(p)); - if (!on_rq && p->state != TASK_RUNNING) + if (!on_rq && p->state != TASK_RUNNING && p->state != TASK_WAKING) p->se.vruntime += cfs_rq_of(&p->se)->min_vruntime; } #endif -- 1.7.1 -- To unsubscribe from this list: send the line "unsubscribe cgroups" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-12-13 7:01 UTC|newest] Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top 2011-12-13 6:57 [PATCH 0/3] sched: some fixes for vruntime calculation related to cgroup movement Daisuke Nishimura 2011-12-13 6:57 ` Daisuke Nishimura 2011-12-13 6:57 ` [PATCH 1/3] sched: fix cgroup movement of newly created process Daisuke Nishimura 2011-12-13 6:57 ` Daisuke Nishimura 2011-12-13 12:01 ` Paul Turner 2011-12-13 12:01 ` Paul Turner 2011-12-14 1:04 ` Daisuke Nishimura 2011-12-14 1:04 ` Daisuke Nishimura 2011-12-13 12:41 ` Peter Zijlstra 2011-12-13 12:41 ` Peter Zijlstra 2011-12-14 1:05 ` Daisuke Nishimura 2011-12-14 1:05 ` Daisuke Nishimura 2011-12-13 6:58 ` [PATCH 2/3] sched: fix cgroup movement of forking process Daisuke Nishimura 2011-12-13 6:58 ` Daisuke Nishimura 2011-12-13 12:22 ` Paul Turner 2011-12-13 12:22 ` Paul Turner 2011-12-13 6:59 ` Daisuke Nishimura [this message] 2011-12-13 6:59 ` [PATCH 3/3] sched: fix cgroup movement of waking process Daisuke Nishimura 2011-12-13 12:24 ` Paul Turner 2011-12-13 12:24 ` Paul Turner 2011-12-15 5:35 ` [PATCH -tip 0/3] sched: some fixes for vruntime calculation related to cgroup movement(v2) Daisuke Nishimura 2011-12-15 5:36 ` [PATCH 1/3] sched: fix cgroup movement of newly created process Daisuke Nishimura 2011-12-15 5:36 ` Daisuke Nishimura 2011-12-21 11:45 ` [tip:sched/core] sched: Fix " tip-bot for Daisuke Nishimura 2011-12-15 5:36 ` [PATCH 2/3] sched: fix cgroup movement of forking process Daisuke Nishimura 2011-12-15 5:36 ` Daisuke Nishimura 2011-12-21 11:44 ` [tip:sched/core] sched: Fix " tip-bot for Daisuke Nishimura 2011-12-21 17:26 ` Tejun Heo 2011-12-21 17:37 ` Tejun Heo 2011-12-22 1:54 ` Frederic Weisbecker 2011-12-22 2:01 ` Tejun Heo 2011-12-15 5:37 ` [PATCH 3/3] sched: fix cgroup movement of waking process Daisuke Nishimura 2011-12-21 11:45 ` [tip:sched/core] sched: Fix " tip-bot for Daisuke Nishimura 2011-12-19 2:55 ` [PATCH -tip 0/3] sched: some fixes for vruntime calculation related to cgroup movement(v2) Daisuke Nishimura 2011-12-19 2:55 ` Daisuke Nishimura
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20111213155933.36cd824a.nishimura@mxp.nes.nec.co.jp \ --to=nishimura@mxp.nes.nec.co.jp \ --cc=a.p.zijlstra@chello.nl \ --cc=cgroups@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@elte.hu \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.