All of lore.kernel.org
 help / color / mirror / Atom feed
From: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
To: LKML <linux-kernel@vger.kernel.org>, cgroups <cgroups@vger.kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Paul Turner <pjt@google.com>,
	Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Subject: [PATCH 1/3] sched: fix cgroup movement of newly created process
Date: Thu, 15 Dec 2011 14:36:07 +0900	[thread overview]
Message-ID: <20111215143607.2ee12c5d.nishimura@mxp.nes.nec.co.jp> (raw)
In-Reply-To: <20111215143506.335e443e.nishimura@mxp.nes.nec.co.jp>

There is a small race between do_fork() and sched_move_task(), which is
trying to move the child.

            do_fork()                 sched_move_task()
--------------------------------+---------------------------------
  copy_process()
    sched_fork()
      task_fork_fair()
        -> vruntime of the child is initialized
           based on that of the parent.
  -> we can see the child in "tasks" file now.
                                    task_rq_lock()
                                    task_move_group_fair()
                                      -> child.se.vruntime
                                           -= (old)cfs_rq->min_vruntime
                                           += (new)cfs_rq->min_vruntime
                                    task_rq_unlock()
  wake_up_new_task()
    ...
    enqueue_entity()
      child.se.vruntime += cfs_rq->min_vruntime

As a result, vruntime of the child becomes far bigger than min_vruntime,
if (new)cfs_rq->min_vruntime >> (old)cfs_rq->min_vruntime.

This patch fixes this problem by just ignoring such process in
task_move_group_fair(), because the vruntime has already been normalized in
task_fork_fair().

Signed-off-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
---
 kernel/sched/fair.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index a4d2b7a..43cadd0 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5336,6 +5336,19 @@ static void task_move_group_fair(struct task_struct *p, int on_rq)
 	 * to another cgroup's rq. This does somewhat interfere with the
 	 * fair sleeper stuff for the first placement, but who cares.
 	 */
+	/*
+	 * When !on_rq, vruntime of the task has usually NOT been normalized.
+	 * But there are some cases where it has already been normalized:
+	 *
+	 * - Moving a forked child which is waiting for being woken up by
+	 *   wake_up_new_task().
+	 *
+	 * To prevent boost or penalty in the new cfs_rq caused by delta
+	 * min_vruntime between the two cfs_rqs, we skip vruntime adjustment.
+	 */
+	if (!on_rq && !p->se.sum_exec_runtime)
+		on_rq = 1;
+
 	if (!on_rq)
 		p->se.vruntime -= cfs_rq_of(&p->se)->min_vruntime;
 	set_task_rq(p, task_cpu(p));
-- 
1.7.1


WARNING: multiple messages have this Message-ID (diff)
From: Daisuke Nishimura <nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
To: LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	cgroups <cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Cc: Ingo Molnar <mingo-X9Un+BFzKDI@public.gmane.org>,
	Peter Zijlstra
	<a.p.zijlstra-/NLkJaSkS4VmR6Xm/wNWPw@public.gmane.org>,
	Paul Turner <pjt-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Daisuke Nishimura
	<nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
Subject: [PATCH 1/3] sched: fix cgroup movement of newly created process
Date: Thu, 15 Dec 2011 14:36:07 +0900	[thread overview]
Message-ID: <20111215143607.2ee12c5d.nishimura@mxp.nes.nec.co.jp> (raw)
In-Reply-To: <20111215143506.335e443e.nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>

There is a small race between do_fork() and sched_move_task(), which is
trying to move the child.

            do_fork()                 sched_move_task()
--------------------------------+---------------------------------
  copy_process()
    sched_fork()
      task_fork_fair()
        -> vruntime of the child is initialized
           based on that of the parent.
  -> we can see the child in "tasks" file now.
                                    task_rq_lock()
                                    task_move_group_fair()
                                      -> child.se.vruntime
                                           -= (old)cfs_rq->min_vruntime
                                           += (new)cfs_rq->min_vruntime
                                    task_rq_unlock()
  wake_up_new_task()
    ...
    enqueue_entity()
      child.se.vruntime += cfs_rq->min_vruntime

As a result, vruntime of the child becomes far bigger than min_vruntime,
if (new)cfs_rq->min_vruntime >> (old)cfs_rq->min_vruntime.

This patch fixes this problem by just ignoring such process in
task_move_group_fair(), because the vruntime has already been normalized in
task_fork_fair().

Signed-off-by: Daisuke Nishimura <nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
---
 kernel/sched/fair.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index a4d2b7a..43cadd0 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -5336,6 +5336,19 @@ static void task_move_group_fair(struct task_struct *p, int on_rq)
 	 * to another cgroup's rq. This does somewhat interfere with the
 	 * fair sleeper stuff for the first placement, but who cares.
 	 */
+	/*
+	 * When !on_rq, vruntime of the task has usually NOT been normalized.
+	 * But there are some cases where it has already been normalized:
+	 *
+	 * - Moving a forked child which is waiting for being woken up by
+	 *   wake_up_new_task().
+	 *
+	 * To prevent boost or penalty in the new cfs_rq caused by delta
+	 * min_vruntime between the two cfs_rqs, we skip vruntime adjustment.
+	 */
+	if (!on_rq && !p->se.sum_exec_runtime)
+		on_rq = 1;
+
 	if (!on_rq)
 		p->se.vruntime -= cfs_rq_of(&p->se)->min_vruntime;
 	set_task_rq(p, task_cpu(p));
-- 
1.7.1

--
To unsubscribe from this list: send the line "unsubscribe cgroups" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply	other threads:[~2011-12-15  5:43 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-12-13  6:57 [PATCH 0/3] sched: some fixes for vruntime calculation related to cgroup movement Daisuke Nishimura
2011-12-13  6:57 ` Daisuke Nishimura
2011-12-13  6:57 ` [PATCH 1/3] sched: fix cgroup movement of newly created process Daisuke Nishimura
2011-12-13  6:57   ` Daisuke Nishimura
2011-12-13 12:01   ` Paul Turner
2011-12-13 12:01     ` Paul Turner
2011-12-14  1:04     ` Daisuke Nishimura
2011-12-14  1:04       ` Daisuke Nishimura
2011-12-13 12:41   ` Peter Zijlstra
2011-12-13 12:41     ` Peter Zijlstra
2011-12-14  1:05     ` Daisuke Nishimura
2011-12-14  1:05       ` Daisuke Nishimura
2011-12-13  6:58 ` [PATCH 2/3] sched: fix cgroup movement of forking process Daisuke Nishimura
2011-12-13  6:58   ` Daisuke Nishimura
2011-12-13 12:22   ` Paul Turner
2011-12-13 12:22     ` Paul Turner
2011-12-13  6:59 ` [PATCH 3/3] sched: fix cgroup movement of waking process Daisuke Nishimura
2011-12-13  6:59   ` Daisuke Nishimura
2011-12-13 12:24   ` Paul Turner
2011-12-13 12:24     ` Paul Turner
2011-12-15  5:35 ` [PATCH -tip 0/3] sched: some fixes for vruntime calculation related to cgroup movement(v2) Daisuke Nishimura
2011-12-15  5:36   ` Daisuke Nishimura [this message]
2011-12-15  5:36     ` [PATCH 1/3] sched: fix cgroup movement of newly created process Daisuke Nishimura
2011-12-21 11:45     ` [tip:sched/core] sched: Fix " tip-bot for Daisuke Nishimura
2011-12-15  5:36   ` [PATCH 2/3] sched: fix cgroup movement of forking process Daisuke Nishimura
2011-12-15  5:36     ` Daisuke Nishimura
2011-12-21 11:44     ` [tip:sched/core] sched: Fix " tip-bot for Daisuke Nishimura
2011-12-21 17:26       ` Tejun Heo
2011-12-21 17:37         ` Tejun Heo
2011-12-22  1:54           ` Frederic Weisbecker
2011-12-22  2:01             ` Tejun Heo
2011-12-15  5:37   ` [PATCH 3/3] sched: fix cgroup movement of waking process Daisuke Nishimura
2011-12-21 11:45     ` [tip:sched/core] sched: Fix " tip-bot for Daisuke Nishimura
2011-12-19  2:55   ` [PATCH -tip 0/3] sched: some fixes for vruntime calculation related to cgroup movement(v2) Daisuke Nishimura
2011-12-19  2:55     ` Daisuke Nishimura

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20111215143607.2ee12c5d.nishimura@mxp.nes.nec.co.jp \
    --to=nishimura@mxp.nes.nec.co.jp \
    --cc=a.p.zijlstra@chello.nl \
    --cc=cgroups@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=pjt@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.