LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Vincent Guittot <vincent.guittot@linaro.org>
To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com,
	dietmar.eggemann@arm.com, rostedt@goodmis.org,
	bsegall@google.com, mgorman@suse.de,
	linux-kernel@vger.kernel.org
Cc: valentin.schneider@arm.com, pauld@redhat.com, hdanton@sina.com,
	Vincent Guittot <vincent.guittot@linaro.org>
Subject: [PATCH 3/4 v2] sched/fair: minimize concurrent LBs between domain level
Date: Mon, 21 Sep 2020 09:24:23 +0200
Message-ID: <20200921072424.14813-4-vincent.guittot@linaro.org> (raw)
In-Reply-To: <20200921072424.14813-1-vincent.guittot@linaro.org>

sched domains tend to trigger simultaneously the load balance loop but
the larger domains often need more time to collect statistics. This
slowness makes the larger domain trying to detach tasks from a rq whereas
tasks already migrated somewhere else at a sub-domain level. This is not
a real problem for idle LB because the period of smaller domains will
increase with its CPUs being busy and this will let time for higher ones
to pulled tasks. But this becomes a problem when all CPUs are already busy
because all domains stay synced when they trigger their LB.

A simple way to minimize simultaneous LB of all domains is to decrement the
the busy interval by 1 jiffies. Because of the busy_factor, the interval of
larger domain will not be a multiple of smaller ones anymore.

Reviewed-by: Phil Auld <pauld@redhat.com>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
---
 kernel/sched/fair.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d8320dc9d014..458702062d3b 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9785,6 +9785,15 @@ get_sd_balance_interval(struct sched_domain *sd, int cpu_busy)
 
 	/* scale ms to jiffies */
 	interval = msecs_to_jiffies(interval);
+
+	/*
+	 * Reduce likelihood of busy balancing at higher domains racing with
+	 * balancing at lower domains by preventing their balancing periods
+	 * from being multiples of each other.
+	 */
+	if (cpu_busy)
+		interval -= 1;
+
 	interval = clamp(interval, 1UL, max_load_balance_interval);
 
 	return interval;
-- 
2.17.1


  parent reply index

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-21  7:24 [PATCH 0/4 v2] sched/fair: Improve fairness between cfs tasks Vincent Guittot
2020-09-21  7:24 ` [PATCH 1/4 v2] sched/fair: relax constraint on task's load during load balance Vincent Guittot
2020-09-23 14:43   ` Valentin Schneider
2020-09-29  7:56   ` [tip: sched/core] sched/fair: Relax " tip-bot2 for Vincent Guittot
2020-09-21  7:24 ` [PATCH 2/4 v2] sched/fair: reduce minimal imbalance threshold Vincent Guittot
2020-09-23 14:43   ` Valentin Schneider
2020-09-29  7:56   ` [tip: sched/core] sched/fair: Reduce " tip-bot2 for Vincent Guittot
2020-09-21  7:24 ` Vincent Guittot [this message]
2020-09-23 14:43   ` [PATCH 3/4 v2] sched/fair: minimize concurrent LBs between domain level Valentin Schneider
2020-09-29  7:56   ` [tip: sched/core] sched/fair: Minimize " tip-bot2 for Vincent Guittot
2020-09-21  7:24 ` [PATCH 4/4 v2] sched/fair: reduce busy load balance interval Vincent Guittot
2020-09-23 14:43   ` Valentin Schneider
2020-09-29  7:56   ` [tip: sched/core] sched/fair: Reduce " tip-bot2 for Vincent Guittot
2020-09-23 15:47 ` [PATCH 0/4 v2] sched/fair: Improve fairness between cfs tasks Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200921072424.14813-4-vincent.guittot@linaro.org \
    --to=vincent.guittot@linaro.org \
    --cc=bsegall@google.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=hdanton@sina.com \
    --cc=juri.lelli@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=pauld@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=valentin.schneider@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git