From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Juri Lelli <juri.lelli@redhat.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>,
Steven Rostedt <rostedt@goodmis.org>,
Luca Abeni <luca.abeni@santannapisa.it>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Wei Wang <wvw@google.com>, Quentin Perret <qperret@google.com>,
Alessio Balsini <balsini@google.com>,
Pavan Kondeti <pkondeti@codeaurora.org>,
Patrick Bellasi <patrick.bellasi@matbug.net>,
Morten Rasmussen <morten.rasmussen@arm.com>,
Valentin Schneider <valentin.schneider@arm.com>,
Qais Yousef <qais.yousef@arm.com>,
linux-kernel@vger.kernel.org
Subject: [PATCH 0/4] Capacity awareness for SCHED_DEADLINE
Date: Wed, 8 Apr 2020 11:50:08 +0200 [thread overview]
Message-ID: <20200408095012.3819-1-dietmar.eggemann@arm.com> (raw)
The SCHED_DEADLINE (DL) admission control does not work correctly on
heterogeneous (asymmetric CPU capacity) systems such as Arm big.LITTLE
or DynamIQ.
Let's fix this by explicitly considering CPU capacity in DL admission
control and task migration.
The DL sched class now attempts to avoid missing task deadlines due to
smaller CPU (CPU capacity < 1024) not being capable enough to finish a
task in time. It does so by trying to place a task so that its CPU
capacity scaled deadline is not smaller than its runtime.
Changes RFC [1] -> v1:
Only use static values for CPU bandwidth (sched_dl_entity::dl_runtime,
::dl_deadline) and CPU capacity (arch_scale_cpu_capacity()) to fix DL
admission control.
Dynamic values for CPU bandwidth (sched_dl_entity::runtime, ::deadline)
and CPU capacity (capacity_of()) are considered to be more related to
energy trade-off calculations which could be later introduced using the
Energy Model.
Since the design of the DL and RT sched classes are very similar, the
implementation follows the overall design of RT capacity awareness
(commit 804d402fb6f6 ("sched/rt: Make RT capacity-aware")).
Per-patch changes:
(1) Store CPU capacity sum in the root domain during
build_sched_domains() [patch 1/4]
(2) Adjust to RT capacity awareness design [patch 3/4]
(3) Remove CPU capacity aware placement in switched_to_dl()
(dl_migrate callback) [RFC patch 3/6]
Balance callbacks (push, pull) run only in schedule_tail()
__schedule(), rt_mutex_setprio() or __sched_setscheduler().
DL throttling leads to a call to __dequeue_task_dl() which is not a
full task dequeue. The task is still enqueued and only removed from
the rq.
So a queue_balance_callback() call in update_curr_dl()->
__dequeue_task_dl() will not be followed by a balance_callback()
call in one of the 4 functions mentioned above.
(4) Remove 'dynamic CPU bandwidth' consideration and only support
'static CPU bandwidth' (ratio between sched_dl_entity::dl_runtime
and ::dl_deadline) [RFC patch 4/6]
(5) Remove modification to migration logic which tried to schedule
small tasks on LITTLE CPUs [RFC patch 6/6]
[1] https://lore.kernel.org/r/20190506044836.2914-1-luca.abeni@santannapisa.it
The following rt-app testcase tailored to Arm64 Hikey960:
root@h960:~# cat /sys/devices/system/cpu/cpu*/cpu_capacity
462
462
462
462
1024
1024
1024
1024
shows the expected behavior.
According to the following condition in dl_task_fits_capacity()
cap_scale(dl_deadline, arch_scale_cpu_capacity(cpu)) >= dl_runtime
thread0-[0-3] are placed on a big CPUs whereas thread1-[0-3] run on a
LITTLE CPU respectively.
...
"tasks" : {
"thread0" : {
"policy" : "SCHED_DEADLINE",
"instance" : 4,
"timer" : { "ref" : "unique0", "period" : 16000, "mode" : "absolute" },
"run" : 10000,
"dl-runtime" : 11000,
"dl-period" : 16000,
"dl-deadline" : 16000
},
"thread1" : {
"policy" : "SCHED_DEADLINE",
"instance" : 4,
"delay" : 1000,
"timer" : { "ref" : "unique1", "period" : 16000, "mode" : "absolute" },
"run" : 5500,
"dl-runtime" : 6500
"dl-period" : 16000,
"dl-deadline" : 16000
}
...
Tests were run with Performance CPUfreq governor so that the Schedutil
CPUfreq governor DL threads (sugov:[0,4]), necessary on a
slow-switching platform like Hikey960, do not interfere with the
rt-app test tasks. Using Schedutil would require to lower the number of
tasks to 3 instances each.
Dietmar Eggemann (1):
sched/topology: Store root domain CPU capacity sum
Luca Abeni (3):
sched/deadline: Improve admission control for asymmetric CPU
capacities
sched/deadline: Make DL capacity-aware
sched/deadline: Implement fallback mechanism for !fit case
kernel/sched/cpudeadline.c | 23 +++++++++++++++++++++
kernel/sched/deadline.c | 41 +++++++++++++++++++++++---------------
kernel/sched/sched.h | 33 ++++++++++++++++++++++++++++--
kernel/sched/topology.c | 14 +++++++++----
4 files changed, 89 insertions(+), 22 deletions(-)
--
2.17.1
next reply other threads:[~2020-04-08 9:50 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-08 9:50 Dietmar Eggemann [this message]
2020-04-08 9:50 ` [PATCH 1/4] sched/topology: Store root domain CPU capacity sum Dietmar Eggemann
2020-04-08 12:29 ` Vincent Guittot
2020-04-08 16:30 ` Dietmar Eggemann
2020-04-08 17:03 ` Vincent Guittot
2020-04-09 13:50 ` Dietmar Eggemann
2020-04-09 14:13 ` Vincent Guittot
2020-04-14 9:20 ` Dietmar Eggemann
2020-04-14 12:45 ` Quentin Perret
2020-04-14 15:27 ` Dietmar Eggemann
2020-04-14 15:43 ` Vincent Guittot
2020-04-08 9:50 ` [PATCH 2/4] sched/deadline: Improve admission control for asymmetric CPU capacities Dietmar Eggemann
2020-04-08 10:42 ` Valentin Schneider
2020-04-08 12:26 ` Dietmar Eggemann
2020-04-08 13:30 ` luca abeni
2020-04-08 14:23 ` Qais Yousef
2020-04-08 15:01 ` Valentin Schneider
2020-04-09 17:29 ` Dietmar Eggemann
2020-04-14 11:40 ` Qais Yousef
2020-04-14 14:29 ` Valentin Schneider
2020-04-14 15:41 ` Qais Yousef
2020-04-14 14:28 ` Valentin Schneider
2020-04-17 12:19 ` Juri Lelli
2020-04-17 14:55 ` Dietmar Eggemann
2020-04-17 15:08 ` Juri Lelli
2020-04-17 15:47 ` Juri Lelli
2020-04-08 9:50 ` [PATCH 3/4] sched/deadline: Make DL capacity-aware Dietmar Eggemann
2020-04-10 12:52 ` Juri Lelli
2020-04-15 9:39 ` Dietmar Eggemann
2020-04-15 13:20 ` Juri Lelli
2020-04-15 16:42 ` luca abeni
2020-04-16 13:19 ` Juri Lelli
2020-04-08 9:50 ` [PATCH 4/4] sched/deadline: Implement fallback mechanism for !fit case Dietmar Eggemann
2020-04-09 10:25 ` Qais Yousef
2020-04-09 13:00 ` luca abeni
2020-04-09 14:55 ` Qais Yousef
2020-04-09 18:43 ` Dietmar Eggemann
2020-04-14 11:29 ` Qais Yousef
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200408095012.3819-1-dietmar.eggemann@arm.com \
--to=dietmar.eggemann@arm.com \
--cc=balsini@google.com \
--cc=bristot@redhat.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luca.abeni@santannapisa.it \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=patrick.bellasi@matbug.net \
--cc=peterz@infradead.org \
--cc=pkondeti@codeaurora.org \
--cc=qais.yousef@arm.com \
--cc=qperret@google.com \
--cc=rostedt@goodmis.org \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
--cc=wvw@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.