From: Yafang Shao <laoar.shao@gmail.com>
To: peterz@infradead.org, mingo@redhat.com, mgorman@suse.de,
juri.lelli@redhat.com, vincent.guittot@linaro.org,
dietmar.eggemann@arm.com, rostedt@goodmis.org,
bsegall@google.com, bristot@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org,
achaiken@aurora.tech, Yafang Shao <laoar.shao@gmail.com>
Subject: [PATCH v4 0/8] sched: support schedstats for RT sched class
Date: Sun, 5 Sep 2021 14:35:39 +0000 [thread overview]
Message-ID: <20210905143547.4668-1-laoar.shao@gmail.com> (raw)
Why do we need schedstats ?
==========================
schedstats is a useful feature to do thread-level latency analysis. Our
usecase as follows,
Userspace Code Scope Profiler
{
user_func_abc(); <---- uprobe_scope_begin() get start schedstats
...
user_func_xyz(); <---- uprobe_scope_end() get end schedstats
}
Then with the result of (end - begin) we can get below latency details
in a specific user scope,
scope_latency = Wait + Sleep + Blocked [1] + Run (stime + utime)
If there's no schedstats we have to trace the heavy sched::sched_switch
and do a lot more stuff.
[1]. With patch #4 and don't include sum_block_runtime in sum_sleep_runtime
Support schedstats for RT sched class
=====================================
If we want to use the schedstats facility to trace other sched classes, we
should make it independent of fair sched class. The struct sched_statistics
is the schedular statistics of a task_struct or a task_group. So we can
move it into struct task_struct and struct task_group to achieve the goal.
After the patch, schestats are orgnized as follows,
struct task_struct {
...
struct sched_entity se;
struct sched_rt_entity rt;
struct sched_dl_entity dl;
...
struct sched_statistics stats;
...
};
Regarding the task group, schedstats is only supported for fair group
sched, and a new struct sched_entity_stats is introduced, suggested by
Peter -
struct sched_entity_stats {
struct sched_entity se;
struct sched_statistics stats;
} __no_randomize_layout;
Then with the se in a task_group, we can easily get the stats.
The sched_statistics members may be frequently modified when schedstats is
enabled, in order to avoid impacting on random data which may in the same
cacheline with them, the struct sched_statistics is defined as cacheline
aligned.
As this patch changes the core struct of scheduler, so I verified the
performance it may impact on the scheduler with 'perf bench sched
pipe', suggested by Mel. Below is the result, in which all the values
are in usecs/op.
Before After
kernel.sched_schedstats=0 5.2~5.4 5.2~5.4
kernel.sched_schedstats=1 5.3~5.5 5.3~5.5
[These data is a little difference with the earlier version, that is
because my old test machine is destroyed so I have to use a new
different test machine.]
Almost no impact on the sched performance.
The user can get the schedstats information in the same way in fair sched
class. For example,
fair RT
/proc/[pid]/sched /proc/[pid]/sched
schedstats is not supported for RT group.
The sched:sched_stat_{wait, sleep, iowait, blocked} tracepoints can
be used to trace RT tasks as well.
Support schedstats for any other sched classes
==============================================
After this patchset, it is very easy to extend the schedstats to any
other sched classes. The deadline sched class is also supported in this
patchset.
Changes Since v3:
Various code improvement per Peter,
- don't support schedstats for rt group
- introduce struct sched_entity_stats for fair group
- change the position of 'struct sched_statistics stats'
- fixes indent issue
- change the output format in /proc/[pid]/sched
- add the usecase of schedstats
- support schedstats for deadline task
- and other suggestions
Changes Since v2:
- Fixes the output format in /proc/[pid]/sched
- Rebase it on the latest code
- Redo the performance test
Changes since v1:
- Fix the build failure reported by kernel test robot.
- Add the performance data with 'perf bench sched pipe', suggested by
Mel.
- Make the struct sched_statistics cacheline aligned.
- Introduce task block time in schedstats
Changes since RFC:
- improvement of schedstats helpers, per Mel.
- make struct schedstats independent of fair sched class
Yafang Shao (8):
sched, fair: use __schedstat_set() in set_next_entity()
sched: make struct sched_statistics independent of fair sched class
sched: make schedstats helpers independent of fair sched class
sched: introduce task block time in schedstats
sched, rt: support sched_stat_runtime tracepoint for RT sched class
sched, rt: support schedstats for RT sched class
sched, dl: support sched_stat_runtime tracepoint for deadline sched
class
sched, dl: support schedstats for deadline sched class
include/linux/sched.h | 8 +-
kernel/sched/core.c | 25 +++---
kernel/sched/deadline.c | 99 +++++++++++++++++++++-
kernel/sched/debug.c | 97 +++++++++++----------
kernel/sched/fair.c | 177 +++++++++++----------------------------
kernel/sched/rt.c | 130 +++++++++++++++++++++++++++-
kernel/sched/stats.c | 104 +++++++++++++++++++++++
kernel/sched/stats.h | 49 +++++++++++
kernel/sched/stop_task.c | 4 +-
9 files changed, 500 insertions(+), 193 deletions(-)
--
2.18.2
next reply other threads:[~2021-09-05 14:36 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-05 14:35 Yafang Shao [this message]
2021-09-05 14:35 ` [PATCH v4 1/8] sched, fair: use __schedstat_set() in set_next_entity() Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched/fair: Use " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 2/8] sched: make struct sched_statistics independent of fair sched class Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched: Make " tip-bot2 for Yafang Shao
2021-09-21 6:17 ` Dan Carpenter
2021-09-21 7:17 ` Peter Zijlstra
2021-09-21 7:40 ` Dan Carpenter
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 3/8] sched: make schedstats helpers " Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched: Make " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 4/8] sched: introduce task block time in schedstats Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched: Introduce " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 5/8] sched, rt: support sched_stat_runtime tracepoint for RT sched class Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched/rt: Support " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 6/8] sched, rt: support schedstats " Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched/rt: Support " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 7/8] sched, dl: support sched_stat_runtime tracepoint for deadline " Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched/dl: Support " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-05 14:35 ` [PATCH v4 8/8] sched, dl: support schedstats " Yafang Shao
2021-09-16 11:59 ` [tip: sched/core] sched/dl: Support " tip-bot2 for Yafang Shao
2021-10-05 14:12 ` tip-bot2 for Yafang Shao
2021-09-10 17:00 ` [PATCH v4 0/8] sched: support schedstats for RT " Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210905143547.4668-1-laoar.shao@gmail.com \
--to=laoar.shao@gmail.com \
--cc=achaiken@aurora.tech \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).