All of lore.kernel.org
 help / color / mirror / Atom feed
From: YT Chang <yt.chang@mediatek.com>
To: YT Chang <yt.chang@mediatek.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Matthias Brugger <matthias.bgg@gmail.com>
Cc: <wsd_upstream@mediatek.com>, <linux-kernel@vger.kernel.org>,
	<linux-arm-kernel@lists.infradead.org>,
	<linux-mediatek@lists.infradead.org>
Subject: [PATCH 1/1] sched: cfs_rq h_load might not update due to irq disable
Date: Thu, 21 Nov 2019 16:30:09 +0800	[thread overview]
Message-ID: <1574325009-10846-1-git-send-email-yt.chang@mediatek.com> (raw)

Syndrome:

Two CPUs might do idle balance in the same time.
One CPU does idle balance and pulls some tasks.
However before pick next task, ALL task are pulled back to other CPU.
That results in infinite loop in both CPUs.

=========================================
code flow:

in pick_next_task_fair()

again:

if nr_running == 0
	goto idle
pick next task
	return

idle:
	idle_balance
       /* pull some tasks from other CPU,
        * However other CPU are also do idle balance,
	* and pull back these task */

	go to again

=========================================
The result to pull ALL tasks back when the task_h_load
is incorrect and too low.

static unsigned long task_h_load(struct task_struct *p)
{
        struct cfs_rq *cfs_rq = task_cfs_rq(p);

	update_cfs_rq_h_load(cfs_rq);
	return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
			cfs_rq->runnable_load_avg + 1);
}

The cfs_rq->h_load is incorrect and might too small.
The original idea of cfs_rq::last_h_load_update will not
update cfs_rq::h_load more than once a jiffies.
When the Two CPUs pull each other in the pick_next_task_fair,
the irq disabled and result in jiffie not update.
(Other CPUs wait for runqueue lock locked by the two CPUs.
So, ALL CPUs are irq disabled.)

Solution:
cfs_rq h_load might not update due to irq disable
use sched_clock instead jiffies

Signed-off-by: YT Chang <yt.chang@mediatek.com>
---
 kernel/sched/fair.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 83ab35e..231c53f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7578,9 +7578,11 @@ static void update_cfs_rq_h_load(struct cfs_rq *cfs_rq)
 {
 	struct rq *rq = rq_of(cfs_rq);
 	struct sched_entity *se = cfs_rq->tg->se[cpu_of(rq)];
-	unsigned long now = jiffies;
+	u64 now = sched_clock_cpu(cpu_of(rq));
 	unsigned long load;
 
+	now = now * HZ >> 30;
+
 	if (cfs_rq->last_h_load_update == now)
 		return;
 
-- 
1.9.1

WARNING: multiple messages have this Message-ID (diff)
From: YT Chang <yt.chang@mediatek.com>
To: YT Chang <yt.chang@mediatek.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Matthias Brugger <matthias.bgg@gmail.com>
Cc: linux-arm-kernel@lists.infradead.org,
	linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org,
	wsd_upstream@mediatek.com
Subject: [PATCH 1/1] sched: cfs_rq h_load might not update due to irq disable
Date: Thu, 21 Nov 2019 16:30:09 +0800	[thread overview]
Message-ID: <1574325009-10846-1-git-send-email-yt.chang@mediatek.com> (raw)

Syndrome:

Two CPUs might do idle balance in the same time.
One CPU does idle balance and pulls some tasks.
However before pick next task, ALL task are pulled back to other CPU.
That results in infinite loop in both CPUs.

=========================================
code flow:

in pick_next_task_fair()

again:

if nr_running == 0
	goto idle
pick next task
	return

idle:
	idle_balance
       /* pull some tasks from other CPU,
        * However other CPU are also do idle balance,
	* and pull back these task */

	go to again

=========================================
The result to pull ALL tasks back when the task_h_load
is incorrect and too low.

static unsigned long task_h_load(struct task_struct *p)
{
        struct cfs_rq *cfs_rq = task_cfs_rq(p);

	update_cfs_rq_h_load(cfs_rq);
	return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
			cfs_rq->runnable_load_avg + 1);
}

The cfs_rq->h_load is incorrect and might too small.
The original idea of cfs_rq::last_h_load_update will not
update cfs_rq::h_load more than once a jiffies.
When the Two CPUs pull each other in the pick_next_task_fair,
the irq disabled and result in jiffie not update.
(Other CPUs wait for runqueue lock locked by the two CPUs.
So, ALL CPUs are irq disabled.)

Solution:
cfs_rq h_load might not update due to irq disable
use sched_clock instead jiffies

Signed-off-by: YT Chang <yt.chang@mediatek.com>
---
 kernel/sched/fair.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 83ab35e..231c53f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7578,9 +7578,11 @@ static void update_cfs_rq_h_load(struct cfs_rq *cfs_rq)
 {
 	struct rq *rq = rq_of(cfs_rq);
 	struct sched_entity *se = cfs_rq->tg->se[cpu_of(rq)];
-	unsigned long now = jiffies;
+	u64 now = sched_clock_cpu(cpu_of(rq));
 	unsigned long load;
 
+	now = now * HZ >> 30;
+
 	if (cfs_rq->last_h_load_update == now)
 		return;
 
-- 
1.9.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

WARNING: multiple messages have this Message-ID (diff)
From: YT Chang <yt.chang@mediatek.com>
To: YT Chang <yt.chang@mediatek.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Matthias Brugger <matthias.bgg@gmail.com>
Cc: linux-arm-kernel@lists.infradead.org,
	linux-mediatek@lists.infradead.org, linux-kernel@vger.kernel.org,
	wsd_upstream@mediatek.com
Subject: [PATCH 1/1] sched: cfs_rq h_load might not update due to irq disable
Date: Thu, 21 Nov 2019 16:30:09 +0800	[thread overview]
Message-ID: <1574325009-10846-1-git-send-email-yt.chang@mediatek.com> (raw)

Syndrome:

Two CPUs might do idle balance in the same time.
One CPU does idle balance and pulls some tasks.
However before pick next task, ALL task are pulled back to other CPU.
That results in infinite loop in both CPUs.

=========================================
code flow:

in pick_next_task_fair()

again:

if nr_running == 0
	goto idle
pick next task
	return

idle:
	idle_balance
       /* pull some tasks from other CPU,
        * However other CPU are also do idle balance,
	* and pull back these task */

	go to again

=========================================
The result to pull ALL tasks back when the task_h_load
is incorrect and too low.

static unsigned long task_h_load(struct task_struct *p)
{
        struct cfs_rq *cfs_rq = task_cfs_rq(p);

	update_cfs_rq_h_load(cfs_rq);
	return div64_ul(p->se.avg.load_avg_contrib * cfs_rq->h_load,
			cfs_rq->runnable_load_avg + 1);
}

The cfs_rq->h_load is incorrect and might too small.
The original idea of cfs_rq::last_h_load_update will not
update cfs_rq::h_load more than once a jiffies.
When the Two CPUs pull each other in the pick_next_task_fair,
the irq disabled and result in jiffie not update.
(Other CPUs wait for runqueue lock locked by the two CPUs.
So, ALL CPUs are irq disabled.)

Solution:
cfs_rq h_load might not update due to irq disable
use sched_clock instead jiffies

Signed-off-by: YT Chang <yt.chang@mediatek.com>
---
 kernel/sched/fair.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 83ab35e..231c53f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7578,9 +7578,11 @@ static void update_cfs_rq_h_load(struct cfs_rq *cfs_rq)
 {
 	struct rq *rq = rq_of(cfs_rq);
 	struct sched_entity *se = cfs_rq->tg->se[cpu_of(rq)];
-	unsigned long now = jiffies;
+	u64 now = sched_clock_cpu(cpu_of(rq));
 	unsigned long load;
 
+	now = now * HZ >> 30;
+
 	if (cfs_rq->last_h_load_update == now)
 		return;
 
-- 
1.9.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

             reply	other threads:[~2019-11-21  8:30 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-21  8:30 YT Chang [this message]
2019-11-21  8:30 ` [PATCH 1/1] sched: cfs_rq h_load might not update due to irq disable YT Chang
2019-11-21  8:30 ` YT Chang
2019-11-21 12:38 ` Peter Zijlstra
2019-11-21 12:38   ` Peter Zijlstra
2019-11-21 12:38   ` Peter Zijlstra
2019-11-26  3:00   ` Kathleen Chang
2019-11-26  3:00     ` Kathleen Chang
2019-11-26  3:00     ` Kathleen Chang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1574325009-10846-1-git-send-email-yt.chang@mediatek.com \
    --to=yt.chang@mediatek.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mediatek@lists.infradead.org \
    --cc=matthias.bgg@gmail.com \
    --cc=peterz@infradead.org \
    --cc=wsd_upstream@mediatek.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.