LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Viresh Kumar <viresh.kumar@linaro.org>
To: Ingo Molnar <mingo@redhat.com>, Peter Zijlstra <peterz@infradead.org>
Cc: Viresh Kumar <viresh.kumar@linaro.org>,
	linux-kernel@vger.kernel.org,
	Vincent Guittot <vincent.guittot@linaro.org>,
	tkjos@google.com, Daniel Lezcano <daniel.lezcano@linaro.org>,
	quentin.perret@linaro.org, chris.redpath@arm.com,
	steven.sistare@oracle.com, subhra.mazumdar@oracle.com,
	songliubraving@fb.com
Subject: [PATCH V3 0/2] sched/fair: Fallback to sched-idle CPU in absence of idle CPUs
Date: Wed, 26 Jun 2019 10:36:28 +0530
Message-ID: <cover.1561523542.git.viresh.kumar@linaro.org> (raw)

Hi,

We try to find an idle CPU to run the next task, but in case we don't
find an idle CPU it is better to pick a CPU which will run the task the
soonest, for performance reason.

A CPU which isn't idle but has only SCHED_IDLE activity queued on it
should be a good target based on this criteria as any normal fair task
will most likely preempt the currently running SCHED_IDLE task
immediately. In fact, choosing a SCHED_IDLE CPU over a fully idle one
shall give better results as it should be able to run the task sooner
than an idle CPU (which requires to be woken up from an idle state).

This patchset updates both fast and slow paths with this optimization.

Testing is done with the help of rt-app currently and here are the
details:

- Tested on Octacore Hikey platform (all CPUs change frequency
  together).

- rt-app json [1] creates few tasks and we monitor the scheduling
  latency for them by looking at "wu_lat" field (usec).

- The histograms are created using
  https://github.com/adkein/textogram: textogram -a 0 -z 1000 -n 10

- the stats are accumulated using: https://github.com/nferraz/st

- NOTE: The % values shown don't add up, just look at total numbers
  instead


Test 1: Create 8 CFS tasks (no SCHED_IDLE tasks) without this patchset:

      0 - 100  : ##################################################   72% (3688)
    100 - 200  : ################                                     24% (1253)
    200 - 300  : ##                                                    2% (149)
    300 - 400  :                                                       0% (22)
    400 - 500  :                                                       0% (1)
    500 - 600  :                                                       0% (3)
    600 - 700  :                                                       0% (1)
    700 - 800  :                                                       0% (1)
    800 - 900  :
    900 - 1000 :                                                       0% (1)
         >1000 : 0% (17)

   N       min     max     sum     mean    stddev
   5136    0       2452    535985  104.358 104.585


Test 2: Create 8 CFS tasks and 5 SCHED_IDLE tasks:

        A. Without sched-idle patchset:

      0 - 100  : ##################################################   88% (3102)
    100 - 200  : ##                                                    4% (148)
    200 - 300  :                                                       1% (41)
    300 - 400  :                                                       0% (27)
    400 - 500  :                                                       0% (33)
    500 - 600  :                                                       0% (32)
    600 - 700  :                                                       1% (36)
    700 - 800  :                                                       0% (27)
    800 - 900  :                                                       0% (19)
    900 - 1000 :                                                       0% (26)
         >1000 : 34% (1218)

   N       min     max     sum             mean    stddev
   4710    0       67664   5.25956e+06     1116.68 2315.09


        B. With sched-idle patchset:

      0 - 100  : ##################################################   99% (5042)
    100 - 200  :                                                       0% (8)
    200 - 300  :
    300 - 400  :
    400 - 500  :                                                       0% (2)
    500 - 600  :                                                       0% (1)
    600 - 700  :
    700 - 800  :                                                       0% (1)
    800 - 900  :                                                       0% (1)
    900 - 1000 :
         >1000 : 0% (40)

   N       min     max     sum     mean    stddev
   5095    0       7773    523170  102.683 475.482


The mean latency dropped to 10% and the stddev to around 25% with this
patchset.

@Song: Can you please see if the slowpath changes bring any further
improvements in your test case ?

V2->V3:
- Removed a pointless branch from 1/2 (PeterZ).
- Removed the RFC tags as patches are getting in better shape now.
- Updated the slow path as well, earlier versions only supported fast
  paths.
- Rebased over latest tip/master, fixed rebase conflicts.
- Improved commit logs.

--
viresh

[1] https://pastebin.com/TMHGGBxD

Viresh Kumar (2):
  sched: Start tracking SCHED_IDLE tasks count in cfs_rq
  sched/fair: Fallback to sched-idle CPU if idle CPU isn't found

 kernel/sched/fair.c  | 57 ++++++++++++++++++++++++++++++++++----------
 kernel/sched/sched.h |  2 ++
 2 files changed, 47 insertions(+), 12 deletions(-)

-- 
2.21.0.rc0.269.g1a574e7a288b


             reply index

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-26  5:06 Viresh Kumar [this message]
2019-06-26  5:06 ` [PATCH V3 1/2] sched: Start tracking SCHED_IDLE tasks count in cfs_rq Viresh Kumar
2019-07-25 16:15   ` [tip:sched/core] sched/fair: " tip-bot for Viresh Kumar
2019-06-26  5:06 ` [PATCH V3 2/2] sched/fair: Fallback to sched-idle CPU if idle CPU isn't found Viresh Kumar
2019-06-29  1:16   ` Subhra Mazumdar
2019-07-01  8:03     ` Viresh Kumar
2019-07-01 22:08       ` Subhra Mazumdar
2019-07-02  8:35         ` Peter Zijlstra
2019-07-02 16:32           ` Subhra Mazumdar
2019-07-25 16:16   ` [tip:sched/core] sched/fair: Fall back " tip-bot for Viresh Kumar
2019-07-01 13:43 ` [PATCH V3 0/2] sched/fair: Fallback to sched-idle CPU in absence of idle CPUs Peter Zijlstra
2019-07-03  9:13   ` Viresh Kumar
2019-12-09  3:50 ` Wanpeng Li
2019-12-10  6:33   ` Viresh Kumar
2019-12-10 11:15     ` Wanpeng Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1561523542.git.viresh.kumar@linaro.org \
    --to=viresh.kumar@linaro.org \
    --cc=chris.redpath@arm.com \
    --cc=daniel.lezcano@linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=quentin.perret@linaro.org \
    --cc=songliubraving@fb.com \
    --cc=steven.sistare@oracle.com \
    --cc=subhra.mazumdar@oracle.com \
    --cc=tkjos@google.com \
    --cc=vincent.guittot@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git