All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v7 00/11] track CPU utilization
@ 2018-06-28 15:45 Vincent Guittot
  2018-06-28 15:45 ` [PATCH 01/11] sched/pelt: Move pelt related code in a dedicated file Vincent Guittot
                   ` (11 more replies)
  0 siblings, 12 replies; 44+ messages in thread
From: Vincent Guittot @ 2018-06-28 15:45 UTC (permalink / raw)
  To: peterz, mingo, linux-kernel
  Cc: rjw, juri.lelli, dietmar.eggemann, Morten.Rasmussen,
	viresh.kumar, valentin.schneider, patrick.bellasi, joel,
	daniel.lezcano, quentin.perret, luca.abeni, claudio,
	Vincent Guittot

This patchset initially tracked only the utilization of RT rq. During
OSPM summit, it has been discussed the opportunity to extend it in order
to get an estimate of the utilization of the CPU.

- Patches 1 move pelt code in a dedicated file and remove some blank lines
  
- Patches 2-3 add utilization tracking for rt_rq.

When both cfs and rt tasks compete to run on a CPU, we can see some frequency
drops with schedutil governor. In such case, the cfs_rq's utilization doesn't
reflect anymore the utilization of cfs tasks but only the remaining part that
is not used by rt tasks. We should monitor the stolen utilization and take
it into account when selecting OPP. This patchset doesn't change the OPP
selection policy for RT tasks but only for CFS tasks

A rt-app use case which creates an always running cfs thread and a rt threads
that wakes up periodically with both threads pinned on same CPU, show lot of 
frequency switches of the CPU whereas the CPU never goes idles during the 
test. I can share the json file that I used for the test if someone is
interested in.

For a 15 seconds long test on a hikey 6220 (octo core cortex A53 platfrom),
the cpufreq statistics outputs (stats are reset just before the test) : 
$ cat /sys/devices/system/cpu/cpufreq/policy0/stats/total_trans
without patchset : 1230
with patchset : 14

If we replace the cfs thread of rt-app by a sysbench cpu test, we can see
performance improvements:

- Without patchset :
Test execution summary:
    total time:                          15.0009s
    total number of events:              4903
    total time taken by event execution: 14.9972
    per-request statistics:
         min:                                  1.23ms
         avg:                                  3.06ms
         max:                                 13.16ms
         approx.  95 percentile:              12.73ms

Threads fairness:
    events (avg/stddev):           4903.0000/0.00
    execution time (avg/stddev):   14.9972/0.00

- With patchset:
Test execution summary:
    total time:                          15.0014s
    total number of events:              7694
    total time taken by event execution: 14.9979
    per-request statistics:
         min:                                  1.23ms
         avg:                                  1.95ms
         max:                                 10.49ms
         approx.  95 percentile:              10.39ms

Threads fairness:
    events (avg/stddev):           7694.0000/0.00
    execution time (avg/stddev):   14.9979/0.00

The performance improvement is 56% for this use case.

- Patches 4-5 add utilization tracking for dl_rq in order to solve similar
  problem as with rt_rq. Nevertheless, we keep using dl bandwidth as default
  level of requirement for dl tasks. The dl utilization is used to check that
  the CPU is not overloaded which is not always reflected when using dl
  bandwidth

- Patches 6-7 add utilization tracking for interrupt and use it select OPP
  A test with iperf on hikey 6220 gives: 
    w/o patchset	    w/ patchset
    Tx 276 Mbits/sec        304 Mbits/sec +10%
    Rx 299 Mbits/sec        328 Mbits/sec +09%
    
    8 iterations of iperf -c server_address -r -t 5
    stdev is lower than 1%
    Only WFI idle state is enable (shallowest arm idle state)

- Patch 8 merges sugov_aggregate_util and sugov_get_util as proposed by Peter

- Patches 9 uses rt, dl and interrupt utilization in the scale_rt_capacity()
  and remove  the use of sched_rt_avg_update.

- Patches 10 removes the unused sched_avg_update code

- Patch 11 removes the unused sched_time_avg_ms

Change since v6:
- add more comments load tracking metrics 
- merge sugov_aggregate_util and sugov_get_util

Change since v4:
- add support of periodic update of blocked utilization
- rebase on lastest tip/sched/core

Change since v3:
- add support of periodic update of blocked utilization
- rebase on lastest tip/sched/core

Change since v2:
- move pelt code into a dedicated pelt.c file
- rebase on load tracking changes

Change since v1:
- Only a rebase. I have addressed the comments on previous version in
  patch 1/2


Vincent Guittot (11):
  sched/pelt: Move pelt related code in a dedicated file
  sched/rt: add rt_rq utilization tracking
  cpufreq/schedutil: use rt utilization tracking
  sched/dl: add dl_rq utilization tracking
  cpufreq/schedutil: use dl utilization tracking
  sched/irq: add irq utilization tracking
  cpufreq/schedutil: take into account interrupt
  sched: schedutil: remove sugov_aggregate_util()
  sched: use pelt for scale_rt_capacity()
  sched: remove rt_avg code
  proc/sched: remove unused sched_time_avg_ms

 include/linux/sched/sysctl.h     |   1 -
 kernel/sched/Makefile            |   2 +-
 kernel/sched/core.c              |  38 +---
 kernel/sched/cpufreq_schedutil.c |  65 ++++---
 kernel/sched/deadline.c          |   8 +-
 kernel/sched/fair.c              | 403 +++++----------------------------------
 kernel/sched/pelt.c              | 399 ++++++++++++++++++++++++++++++++++++++
 kernel/sched/pelt.h              |  72 +++++++
 kernel/sched/rt.c                |  15 +-
 kernel/sched/sched.h             |  68 +++++--
 kernel/sysctl.c                  |   8 -
 11 files changed, 632 insertions(+), 447 deletions(-)
 create mode 100644 kernel/sched/pelt.c
 create mode 100644 kernel/sched/pelt.h

-- 
2.7.4


^ permalink raw reply	[flat|nested] 44+ messages in thread

end of thread, other threads:[~2018-07-31  8:21 UTC | newest]

Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-28 15:45 [PATCH v7 00/11] track CPU utilization Vincent Guittot
2018-06-28 15:45 ` [PATCH 01/11] sched/pelt: Move pelt related code in a dedicated file Vincent Guittot
2018-07-15 23:26   ` [tip:sched/core] sched/pelt: Move PELT " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 02/11] sched/rt: add rt_rq utilization tracking Vincent Guittot
2018-07-15 23:27   ` [tip:sched/core] sched/rt: Add " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 03/11] cpufreq/schedutil: use rt " Vincent Guittot
2018-07-06  5:56   ` Viresh Kumar
2018-07-15 23:27   ` [tip:sched/core] cpufreq/schedutil: Use RT " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 04/11] sched/dl: add dl_rq " Vincent Guittot
2018-07-15 23:28   ` [tip:sched/core] sched/dl: Add " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 05/11] cpufreq/schedutil: use dl " Vincent Guittot
2018-07-06  5:59   ` Viresh Kumar
2018-07-15 23:28   ` [tip:sched/core] cpufreq/schedutil: Use DL " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 06/11] sched/irq: add irq " Vincent Guittot
2018-07-15 23:29   ` [tip:sched/core] sched/irq: Add IRQ " tip-bot for Vincent Guittot
2018-07-26  3:09   ` [PATCH 06/11] sched/irq: add irq " Wanpeng Li
2018-07-30 16:43     ` Vincent Guittot
2018-07-31  3:32       ` Wanpeng Li
2018-07-31  8:21         ` Vincent Guittot
2018-06-28 15:45 ` [PATCH 07/11] cpufreq/schedutil: take into account interrupt Vincent Guittot
2018-07-06  6:00   ` Viresh Kumar
2018-07-06  9:14     ` Peter Zijlstra
2018-07-06  9:21       ` Vincent Guittot
2018-07-15 23:29   ` [tip:sched/core] cpufreq/schedutil: Take time spent in interrupts into account tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 08/11] sched: schedutil: remove sugov_aggregate_util() Vincent Guittot
2018-07-06  6:02   ` Viresh Kumar
2018-07-15 23:30   ` [tip:sched/core] sched/cpufreq: Remove sugov_aggregate_util() tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 09/11] sched: use pelt for scale_rt_capacity() Vincent Guittot
2018-07-15 22:15   ` Ingo Molnar
2018-07-15 22:46     ` Joe Perches
2018-07-16 11:24     ` Vincent Guittot
2018-07-16 11:39       ` Ingo Molnar
2018-07-15 23:32   ` [tip:sched/core] sched/core: Use PELT " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 10/11] sched: remove rt_avg code Vincent Guittot
2018-07-15 23:33   ` [tip:sched/core] sched/core: Remove the " tip-bot for Vincent Guittot
2018-06-28 15:45 ` [PATCH 11/11] proc/sched: remove unused sched_time_avg_ms Vincent Guittot
2018-06-28 15:51   ` Luis R. Rodriguez
2018-06-29  5:49     ` Vincent Guittot
2018-07-15 23:33   ` [tip:sched/core] sched/sysctl: Remove unused sched_time_avg_ms sysctl tip-bot for Vincent Guittot
2018-07-05 12:36 ` [PATCH v7 00/11] track CPU utilization Peter Zijlstra
2018-07-05 13:32   ` Vincent Guittot
2018-07-06  6:05   ` Viresh Kumar
2018-07-06  9:18     ` Peter Zijlstra
2018-07-15 23:34   ` [tip:sched/core] sched/cpufreq: Clarify sugov_get_util() tip-bot for Peter Zijlstra

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.