From: Qais Yousef <qais.yousef@arm.com>
To: Steven Rostedt <rostedt@goodmis.org>,
Vincent Guittot <vincent.guittot@linaro.org>
Cc: Mel Gorman <mgorman@suse.de>,
Patrick Bellasi <patrick.bellasi@matbug.net>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Randy Dunlap <rdunlap@infradead.org>,
Jonathan Corbet <corbet@lwn.net>,
Juri Lelli <juri.lelli@redhat.com>,
Ben Segall <bsegall@google.com>,
Luis Chamberlain <mcgrof@kernel.org>,
Kees Cook <keescook@chromium.org>,
Iurii Zaikin <yzaikin@google.com>,
Quentin Perret <qperret@google.com>,
Valentin Schneider <valentin.schneider@arm.com>,
Pavan Kondeti <pkondeti@codeaurora.org>,
linux-doc@vger.kernel.org,
linux-kernel <linux-kernel@vger.kernel.org>,
linux-fs <linux-fsdevel@vger.kernel.org>
Subject: Re: [PATCH 1/2] sched/uclamp: Add a new sysctl to control RT default boost value
Date: Thu, 11 Jun 2020 11:13:49 +0100 [thread overview]
Message-ID: <20200611101349.v3utkqrcegthhahr@e107158-lin.cambridge.arm.com> (raw)
In-Reply-To: <20200608104424.10781990@gandalf.local.home>
On 06/08/20 10:44, Steven Rostedt wrote:
> On Mon, 8 Jun 2020 13:31:03 +0100
> Qais Yousef <qais.yousef@arm.com> wrote:
>
> > I admit I don't know how much of these numbers is ftrace overhead. When trying
>
> Note, if you want to get a better idea of how long a function runs, put it
> into set_ftrace_filter, and then trace it. That way you remove the overhead
> of the function graph tracer when its nesting within a function.
Thanks for the tip!
With CONFIG_FAIR_GROUP_SCHED I see (uclamp disabled)
sched-pipe-602 [001] 73.755392: funcgraph_entry: 2.080 us | activate_task();
sched-pipe-602 [001] 73.755399: funcgraph_entry: 2.000 us | deactivate_task();
sched-pipe-601 [001] 73.755407: funcgraph_entry: 2.220 us | activate_task();
sched-pipe-601 [001] 73.755414: funcgraph_entry: 2.020 us | deactivate_task();
sched-pipe-602 [001] 73.755422: funcgraph_entry: 2.160 us | activate_task();
sched-pipe-602 [001] 73.755429: funcgraph_entry: 1.920 us | deactivate_task();
sched-pipe-601 [001] 73.755437: funcgraph_entry: 2.260 us | activate_task();
sched-pipe-601 [001] 73.755444: funcgraph_entry: 2.080 us | deactivate_task();
sched-pipe-602 [001] 73.755452: funcgraph_entry: 2.160 us | activate_task();
sched-pipe-602 [001] 73.755459: funcgraph_entry: 2.080 us | deactivate_task();
sched-pipe-601 [001] 73.755468: funcgraph_entry: 2.200 us | activate_task();
sched-pipe-601 [001] 73.755521: funcgraph_entry: 3.160 us | activate_task();
update_cfs_group() overhead
sched-pipe-622 [001] 156.790478: funcgraph_entry: 0.820 us | update_cfs_group();
sched-pipe-622 [001] 156.790483: funcgraph_entry: 0.840 us | update_cfs_group();
sched-pipe-622 [001] 156.790485: funcgraph_entry: 0.820 us | update_cfs_group();
sched-pipe-622 [001] 156.790487: funcgraph_entry: 0.820 us | update_cfs_group();
sched-pipe-622 [001] 156.790488: funcgraph_entry: 0.800 us | update_cfs_group();
sched-pipe-622 [001] 156.790508: funcgraph_entry: 1.040 us | update_cfs_group();
sched-pipe-622 [001] 156.790510: funcgraph_entry: 0.920 us | update_cfs_group();
sched-pipe-622 [001] 156.790511: funcgraph_entry: 1.040 us | update_cfs_group();
sched-pipe-622 [001] 156.790513: funcgraph_entry: 0.840 us | update_cfs_group();
sched-pipe-623 [001] 156.790540: funcgraph_entry: 1.160 us | update_cfs_group();
sched-pipe-623 [001] 156.790543: funcgraph_entry: 1.020 us | update_cfs_group();
sched-pipe-623 [001] 156.790544: funcgraph_entry: 0.880 us | update_cfs_group();
sched-pipe-623 [001] 156.790546: funcgraph_entry: 0.840 us | update_cfs_group();
sched-pipe-621 [001] 156.790905: funcgraph_entry: 1.780 us | update_cfs_group();
sched-pipe-621 [001] 156.790908: funcgraph_entry: 1.060 us | update_cfs_group();
sched-pipe-621 [001] 156.790910: funcgraph_entry: 0.880 us | update_cfs_group();
sched-pipe-621 [001] 156.790912: funcgraph_entry: 0.880 us | update_cfs_group();
sched-pipe-621 [001] 156.790916: funcgraph_entry: 0.800 us | update_cfs_group();
sched-pipe-621 [001] 156.790917: funcgraph_entry: 0.820 us | update_cfs_group();
sched-pipe-621 [001] 156.790919: funcgraph_entry: 0.840 us | update_cfs_group();
sched-pipe-621 [001] 156.790921: funcgraph_entry: 0.880 us | update_cfs_group();
sched-pipe-621 [001] 156.790932: funcgraph_entry: 0.960 us | update_cfs_group();
sched-pipe-621 [001] 156.790934: funcgraph_entry: 0.960 us | update_cfs_group();
sched-pipe-621 [001] 156.790936: funcgraph_entry: 1.080 us | update_cfs_group();
sched-pipe-621 [001] 156.790937: funcgraph_entry: 0.840 us | update_cfs_group();
Without CONFIG_FAIR_GROUP_SCHED and without CONFIG_UCLAMP_TASK
sched-pipe-604 [001] 76.386078: funcgraph_entry: 1.380 us | activate_task();
sched-pipe-604 [001] 76.386084: funcgraph_entry: 1.360 us | deactivate_task();
sched-pipe-605 [001] 76.386091: funcgraph_entry: 1.400 us | activate_task();
sched-pipe-605 [001] 76.386096: funcgraph_entry: 1.260 us | deactivate_task();
sched-pipe-604 [001] 76.386104: funcgraph_entry: 1.500 us | activate_task();
sched-pipe-604 [001] 76.386109: funcgraph_entry: 1.280 us | deactivate_task();
sched-pipe-605 [001] 76.386117: funcgraph_entry: 1.380 us | activate_task();
sched-pipe-605 [001] 76.386122: funcgraph_entry: 1.300 us | deactivate_task();
sched-pipe-604 [001] 76.386130: funcgraph_entry: 1.380 us | activate_task();
sched-pipe-604 [001] 76.386135: funcgraph_entry: 1.260 us | deactivate_task();
sched-pipe-605 [001] 76.386142: funcgraph_entry: 1.400 us | activate_task();
sched-pipe-605 [001] 76.386148: funcgraph_entry: 1.340 us | deactivate_task();
So approximately 800ns are added by update_cfs_group() for enqueue and dequeue.
This overhead affects 2 tasks in the tests, so the total effect on the
generated usecs/ops
2 * enqueue_overhead + 2 * dequeue overhead = 4 * ~800ns = 3.2 us
Which explains the 3us drop I see when fair group config is enabled.
Applying similar analysis to uclamp
With uclamp enabled
sched-pipe-610 [001] 173.429431: funcgraph_entry: 1.580 us | activate_task();
sched-pipe-610 [001] 173.429437: funcgraph_entry: 1.440 us | deactivate_task();
sched-pipe-609 [001] 173.429444: funcgraph_entry: 1.580 us | activate_task();
sched-pipe-609 [001] 173.429450: funcgraph_entry: 1.440 us | deactivate_task();
sched-pipe-610 [001] 173.429458: funcgraph_entry: 1.700 us | activate_task();
sched-pipe-610 [001] 173.429464: funcgraph_entry: 1.460 us | deactivate_task();
sched-pipe-609 [001] 173.429471: funcgraph_entry: 1.540 us | activate_task();
sched-pipe-609 [001] 173.429477: funcgraph_entry: 1.460 us | deactivate_task();
sched-pipe-610 [001] 173.429485: funcgraph_entry: 1.560 us | activate_task();
sched-pipe-610 [001] 173.429491: funcgraph_entry: 1.500 us | deactivate_task();
sched-pipe-609 [001] 173.429498: funcgraph_entry: 1.600 us | activate_task();
sched-pipe-609 [001] 173.429504: funcgraph_entry: 1.460 us | deactivate_task();
Which adds approximately 200ns at enqueue and dequeue.
2 * enqueue_overhead + 2 * dequeue overhead = 4 * ~200ns = 0.8 us
Which would explain the ~1us drop I've seen with uclamp when running sched
bench. Apologies for the very course averaging of the numbers from my side.
As a reminder the results I reported before:
*** uclamp disabled/fair group enabled ***
# Executed 50000 pipe operations between two threads
Total time: 0.958 [sec]
19.177100 usecs/op
52145 ops/sec
*** uclamp disabled/fair group disabled ***
# Executed 50000 pipe operations between two threads
Total time: 0.808 [sec]
16.176200 usecs/op
61819 ops/sec
*** uclamp enabled/fair group disabled ***
# Executed 50000 pipe operations between two threads
Total time: 0.856 [sec]
17.125740 usecs/op
58391 ops/sec
Based on my observation with code shuffling it seems a lot of this 200ns comes
from terrible I$ performance on the particular platform I am testing on.
When I run on x86 machine, if I interpreted perf annotation correctly I see D$
misses on accessing rq->uclamp_rq.bucket[] and p->uclamp[]. But I'll share this
result on a separate email in-reply to Mel.
Thanks
--
Qais Yousef
next prev parent reply other threads:[~2020-06-11 10:14 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-11 15:40 [PATCH 1/2] sched/uclamp: Add a new sysctl to control RT default boost value Qais Yousef
2020-05-11 15:40 ` [PATCH 2/2] Documentation/sysctl: Document uclamp sysctl knobs Qais Yousef
2020-05-11 17:18 ` [PATCH 1/2] sched/uclamp: Add a new sysctl to control RT default boost value Qais Yousef
2020-05-12 2:10 ` Pavan Kondeti
2020-05-12 11:46 ` Qais Yousef
2020-05-15 11:08 ` Patrick Bellasi
2020-05-18 8:31 ` Dietmar Eggemann
2020-05-18 16:49 ` Qais Yousef
2020-05-28 13:23 ` Peter Zijlstra
2020-05-28 15:58 ` Qais Yousef
2020-05-28 16:11 ` Peter Zijlstra
2020-05-28 16:51 ` Qais Yousef
2020-05-28 18:29 ` Peter Zijlstra
2020-05-28 19:08 ` Patrick Bellasi
2020-05-28 19:20 ` Dietmar Eggemann
2020-05-29 9:11 ` Qais Yousef
2020-05-29 10:21 ` Mel Gorman
2020-05-29 15:11 ` Qais Yousef
2020-05-29 16:02 ` Mel Gorman
2020-05-29 16:05 ` Qais Yousef
2020-05-29 10:08 ` Mel Gorman
2020-05-29 16:04 ` Qais Yousef
2020-05-29 16:57 ` Mel Gorman
2020-06-02 16:46 ` Dietmar Eggemann
2020-06-03 8:29 ` Patrick Bellasi
2020-06-03 10:10 ` Mel Gorman
2020-06-03 14:59 ` Vincent Guittot
2020-06-03 16:52 ` Qais Yousef
2020-06-04 12:14 ` Vincent Guittot
2020-06-05 10:45 ` Qais Yousef
2020-06-09 15:29 ` Vincent Guittot
2020-06-08 12:31 ` Qais Yousef
2020-06-08 13:06 ` Valentin Schneider
2020-06-08 14:44 ` Steven Rostedt
2020-06-11 10:13 ` Qais Yousef [this message]
2020-06-09 17:10 ` Vincent Guittot
2020-06-11 10:24 ` Qais Yousef
2020-06-11 12:01 ` Vincent Guittot
2020-06-23 15:44 ` Qais Yousef
2020-06-24 8:45 ` Vincent Guittot
2020-06-05 7:55 ` Patrick Bellasi
2020-06-05 11:32 ` Qais Yousef
2020-06-05 13:27 ` Patrick Bellasi
2020-06-03 9:40 ` Mel Gorman
2020-06-03 12:41 ` Qais Yousef
2020-06-04 13:40 ` Mel Gorman
2020-06-05 10:58 ` Qais Yousef
2020-06-11 10:58 ` Qais Yousef
2020-06-16 11:08 ` Qais Yousef
2020-06-16 13:56 ` Lukasz Luba
-- strict thread matches above, loose matches on Subject: below --
2020-04-03 12:30 Qais Yousef
2020-04-14 18:21 ` Patrick Bellasi
2020-04-15 7:46 ` Patrick Bellasi
2020-04-20 15:04 ` Qais Yousef
2020-04-20 8:24 ` Dietmar Eggemann
2020-04-20 15:19 ` Qais Yousef
2020-04-21 0:52 ` Steven Rostedt
2020-04-21 11:16 ` Dietmar Eggemann
2020-04-21 11:23 ` Qais Yousef
2020-04-20 14:50 ` Qais Yousef
2020-04-15 10:11 ` Quentin Perret
2020-04-20 15:08 ` Qais Yousef
2020-04-20 8:29 ` Dietmar Eggemann
2020-04-20 15:13 ` Qais Yousef
2020-04-21 11:18 ` Dietmar Eggemann
2020-04-21 11:27 ` Qais Yousef
2020-04-22 10:59 ` Dietmar Eggemann
2020-04-22 13:13 ` Qais Yousef
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200611101349.v3utkqrcegthhahr@e107158-lin.cambridge.arm.com \
--to=qais.yousef@arm.com \
--cc=bsegall@google.com \
--cc=corbet@lwn.net \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=keescook@chromium.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mcgrof@kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=patrick.bellasi@matbug.net \
--cc=peterz@infradead.org \
--cc=pkondeti@codeaurora.org \
--cc=qperret@google.com \
--cc=rdunlap@infradead.org \
--cc=rostedt@goodmis.org \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
--cc=yzaikin@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).