From: "Ning, Hongyu" <hongyu.ning@linux.intel.com>
To: Peter Zijlstra <peterz@infradead.org>,
joel@joelfernandes.org, chris.hyser@oracle.com,
joshdon@google.com, mingo@kernel.org, vincent.guittot@linaro.org,
valentin.schneider@arm.com, mgorman@suse.de
Cc: linux-kernel@vger.kernel.org, tglx@linutronix.de, "Li,
Aubrey" <aubrey.li@linux.intel.com>,
Tim Chen <tim.c.chen@linux.intel.com>
Subject: Re: [PATCH 00/19] sched: Core Scheduling
Date: Fri, 30 Apr 2021 14:47:00 +0800 [thread overview]
Message-ID: <a49ea23a-998e-2282-4c93-5c6c94f2c28d@linux.intel.com> (raw)
In-Reply-To: <20210422120459.447350175@infradead.org>
On 2021/4/22 20:04, Peter Zijlstra wrote:
> Hai,
>
> This is an agressive fold of all the core-scheduling work so far. I've stripped
> a whole bunch of tags along the way (hopefully not too many, please yell if you
> feel I made a mistake), including tested-by. Please retest.
>
> Changes since the last partial post is dropping all the cgroup stuff and
> PR_SCHED_CORE_CLEAR as well as that exec() behaviour in order to later resolve
> the cgroup issue.
>
> Since we're really rather late for the coming merge window, my plan was to
> merge the lot right after the merge window.
>
> Again, please test.
>
> These patches should shortly be available in my queue.git.
>
> ---
> b/kernel/sched/core_sched.c | 229 ++++++
> b/tools/testing/selftests/sched/.gitignore | 1
> b/tools/testing/selftests/sched/Makefile | 14
> b/tools/testing/selftests/sched/config | 1
> b/tools/testing/selftests/sched/cs_prctl_test.c | 338 +++++++++
> include/linux/sched.h | 19
> include/uapi/linux/prctl.h | 8
> kernel/Kconfig.preempt | 6
> kernel/fork.c | 4
> kernel/sched/Makefile | 1
> kernel/sched/core.c | 858 ++++++++++++++++++++++--
> kernel/sched/cpuacct.c | 12
> kernel/sched/deadline.c | 38 -
> kernel/sched/debug.c | 4
> kernel/sched/fair.c | 276 +++++--
> kernel/sched/idle.c | 13
> kernel/sched/pelt.h | 2
> kernel/sched/rt.c | 31
> kernel/sched/sched.h | 393 ++++++++--
> kernel/sched/stop_task.c | 14
> kernel/sched/topology.c | 4
> kernel/sys.c | 5
> tools/include/uapi/linux/prctl.h | 8
> 23 files changed, 2057 insertions(+), 222 deletions(-)
>
Adding sysbench/uperf/wis performance results for reference:
- kernel under test:
-- above patchset of core-scheduling + local fix for softlockup issue: https://lore.kernel.org/lkml/5c289c5a-a120-a1d0-ca89-2724a1445fe8@linux.intel.com/
-- coresched_v10 kernel source: https://github.com/digitalocean/linux-coresched/commits/coresched/v10-v5.10.y
- workloads:
-- A. sysbench cpu (192 threads) + sysbench cpu (192 threads)
-- B. sysbench cpu (192 threads) + sysbench mysql (192 threads)
-- C. uperf netperf.xml (192 threads over TCP or UDP protocol separately)
-- D. will-it-scale context_switch via pipe (192 threads)
- test machine setup:
CPU(s): 192
On-line CPU(s) list: 0-191
Thread(s) per core: 2
Core(s) per socket: 48
Socket(s): 2
NUMA node(s): 4
- performance change key info:
--workload B: coresched (cs_on), sysbench mysql performance drop around 20% vs coresched_v10
--workload C, coresched (cs_on), uperf performance increased almost double vs coresched_v10
--workload C, default (cs_off), uperf performance drop over 20% vs coresched_v10, same issue seen on v5.12-rc8 base (w/o coresched patchset)
--workload D, coresched (cs_on), wis performance increased almost double vs coresched_v10
- performance info of workloads, normalized based on coresched_v10 results
--workload A:
Note:
* no performance change compared to coresched_v10
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| | ** | coresched_peterz_aubrey_fix_base_v5.12-rc8 | coresched_peterz_aubrey_fix_base_v5.12-rc8 | *** | coresched_v10_base_v5.10.11 | coresched_v10_base_v5.10.11 |
+=======================================+======+==============================================+================================================+=======+===============================+=================================+
| workload | ** | sysbench cpu * 192 | sysbench cpu * 192 | *** | sysbench cpu * 192 | sysbench cpu * 192 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| prctl/cgroup | ** | prctl on workload cpu_0 | prctl on workload cpu_1 | *** | cg_sysbench_cpu_0 | cg_sysbench_cpu_1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| record_item | ** | Tput_avg (events/s) | Tput_avg (events/s) | *** | Tput_avg (events/s) | Tput_avg (events/s) |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| coresched normalized vs coresched_v10 | ** | 0.99 | 1.01 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| default normalized vs coresched_v10 | ** | 1.03 | 0.98 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| smtoff normalized vs coresched_v10 | ** | 1.01 | 0.99 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
--workload B:
Note:
* coresched (cs_on), sysbench mysql performance drop around 20% vs coresched_v10
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| | ** | coresched_peterz_aubrey_fix_base_v5.12-rc8 | coresched_peterz_aubrey_fix_base_v5.12-rc8 | *** | coresched_v10_base_v5.10.11 | coresched_v10_base_v5.10.11 |
+=======================================+======+==============================================+================================================+=======+===============================+=================================+
| workload | ** | sysbench cpu * 192 | sysbench mysql * 192 | *** | sysbench cpu * 192 | sysbench mysql * 192 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| prctl/cgroup | ** | prctl on workload cpu_0 | prctl on workload mysql_0 | *** | cg_sysbench_cpu_0 | cg_sysbench_mysql_0 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| record_item | ** | Tput_avg (events/s) | Tput_avg (events/s) | *** | Tput_avg (events/s) | Tput_avg (events/s) |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| coresched normalized vs coresched_v10 | ** | 1.03 | 0.77 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| default normalized vs coresched_v10 | ** | 1.02 | 0.9 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| smtoff normalized vs coresched_v10 | ** | 0.94 | 1.14 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
--workload C:
Note:
* coresched (cs_on), uperf performance increased almost double vs coresched_v10
* default (cs_off), uperf performance drop over 20% vs coresched_v10, same issue seen on v5.12-rc8 base (w/o coresched patchset)
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| | ** | coresched_peterz_aubrey_fix_base_v5.12-rc8 | coresched_peterz_aubrey_fix_base_v5.12-rc8 | *** | coresched_v10_base_v5.10.11 | coresched_v10_base_v5.10.11 |
+=======================================+======+==============================================+================================================+=======+===============================+=================================+
| workload | ** | uperf netperf TCP * 192 | uperf netperf UDP * 192 | *** | uperf netperf TCP * 192 | uperf netperf UDP * 192 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| prctl/cgroup | ** | prctl on workload uperf | prctl on workload uperf | *** | cg_uperf | cg_uperf |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| record_item | ** | Tput_avg (Gb/s) | Tput_avg (Gb/s) | *** | Tput_avg (Gb/s) | Tput_avg (Gb/s) |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| coresched normalized vs coresched_v10 | ** | 1.87 | 1.99 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| default normalized vs coresched_v10 | ** | 0.78 | 0.74 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
| smtoff normalized vs coresched_v10 | ** | 0.87 | 0.95 | *** | 1 | 1 |
+---------------------------------------+------+----------------------------------------------+------------------------------------------------+-------+-------------------------------+---------------------------------+
--workload D:
Note:
* coresched (cs_on), wis performance increased almost double vs coresched_v10
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
| | ** | coresched_peterz_aubrey_fix_base_v5.12-rc8 | *** | coresched_v10_base_v5.10.11 |
+=======================================+======+==============================================+=======+===============================+
| workload | ** | will-it-scale * 192 | *** | will-it-scale * 192 |
| | | (pipe based context_switch) | | (pipe based context_switch) |
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
| prctl/cgroup | ** | prctl on workload wis | *** | cg_wis |
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
| record_item | ** | threads_avg | *** | threads_avg |
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
| coresched normalized vs coresched_v10 | ** | 1.98 | *** | 1 |
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
| default normalized vs coresched_v10 | ** | 1.13 | *** | 1 |
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
| smtoff normalized vs coresched_v10 | ** | 1.32 | *** | 1 |
+---------------------------------------+------+----------------------------------------------+-------+-------------------------------+
-- notes on record_item:
* coresched normalized vs coresched_v10: smton, cs enabled, test result normalized by result of coresched_v10 under same config
* default normalized vs coresched_v10: smton, cs disabled, test result normalized by result of coresched_v10 under same config
* smtoff normalized vs coresched_v10: smtoff, test result normalized by result of coresched_v10 under same config
Hongyu
next prev parent reply other threads:[~2021-04-30 6:47 UTC|newest]
Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-22 12:04 [PATCH 00/19] sched: Core Scheduling Peter Zijlstra
2021-04-22 12:05 ` [PATCH 01/19] sched/fair: Add a few assertions Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-05-13 8:56 ` Ning, Hongyu
2021-04-22 12:05 ` [PATCH 02/19] sched: Provide raw_spin_rq_*lock*() helpers Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 03/19] sched: Wrap rq::lock access Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 04/19] sched: Prepare for Core-wide rq->lock Peter Zijlstra
2021-04-24 1:22 ` Josh Don
2021-04-26 8:31 ` Peter Zijlstra
2021-04-26 22:21 ` Josh Don
2021-04-27 17:10 ` Don Hiatt
2021-04-27 23:35 ` Josh Don
2021-04-28 1:03 ` Aubrey Li
2021-04-28 6:05 ` Aubrey Li
2021-04-28 10:57 ` Aubrey Li
2021-04-28 16:41 ` Don Hiatt
2021-04-29 20:48 ` Josh Don
2021-04-29 21:09 ` Don Hiatt
2021-04-29 23:22 ` Josh Don
2021-04-30 16:18 ` Don Hiatt
2021-04-30 8:26 ` Aubrey Li
2021-04-28 16:04 ` Don Hiatt
2021-04-27 23:30 ` Josh Don
2021-04-28 9:13 ` Peter Zijlstra
2021-04-28 10:35 ` Aubrey Li
2021-04-28 11:03 ` Peter Zijlstra
2021-04-28 14:18 ` Paul E. McKenney
2021-04-29 20:11 ` Josh Don
2021-05-03 19:17 ` Peter Zijlstra
2021-04-28 7:13 ` Peter Zijlstra
2021-04-28 6:02 ` Aubrey Li
2021-04-29 8:03 ` Aubrey Li
2021-04-29 20:39 ` Josh Don
2021-04-30 8:20 ` Aubrey Li
2021-04-30 8:48 ` Josh Don
2021-04-30 14:15 ` Aubrey Li
2021-05-04 7:38 ` Peter Zijlstra
2021-05-05 16:20 ` Don Hiatt
2021-05-06 10:25 ` Peter Zijlstra
2021-05-07 9:50 ` [PATCH v2 " Peter Zijlstra
2021-05-08 8:07 ` Aubrey Li
2021-05-12 9:07 ` Peter Zijlstra
2021-04-22 12:05 ` [PATCH 05/19] sched: " Peter Zijlstra
2021-05-07 9:50 ` [PATCH v2 " Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 06/19] sched: Optimize rq_lockp() usage Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 07/19] sched: Allow sched_core_put() from atomic context Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 08/19] sched: Introduce sched_class::pick_task() Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 09/19] sched: Basic tracking of matching tasks Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 10/19] sched: Add core wide task selection and scheduling Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 11/19] sched/fair: Fix forced idle sibling starvation corner case Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Vineeth Pillai
2021-04-22 12:05 ` [PATCH 12/19] sched: Fix priority inversion of cookied task with sibling Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Joel Fernandes (Google)
2021-04-22 12:05 ` [PATCH 13/19] sched/fair: Snapshot the min_vruntime of CPUs on force idle Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Joel Fernandes (Google)
2021-04-22 12:05 ` [PATCH 14/19] sched: Trivial forced-newidle balancer Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 15/19] sched: Migration changes for core scheduling Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Aubrey Li
2021-04-22 12:05 ` [PATCH 16/19] sched: Trivial core scheduling cookie management Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 17/19] sched: Inherit task cookie on fork() Peter Zijlstra
2021-05-10 16:06 ` Joel Fernandes
2021-05-10 16:22 ` Chris Hyser
2021-05-10 20:47 ` Joel Fernandes
2021-05-10 21:38 ` Chris Hyser
2021-05-12 9:05 ` Peter Zijlstra
2021-05-12 20:20 ` Josh Don
2021-05-12 21:07 ` Don Hiatt
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 18/19] sched: prctl() core-scheduling interface Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Chris Hyser
2021-06-14 23:36 ` [PATCH 18/19] " Josh Don
2021-06-15 11:31 ` Joel Fernandes
2021-08-05 16:53 ` Eugene Syromiatnikov
2021-08-05 17:00 ` Peter Zijlstra
2021-08-17 15:15 ` Eugene Syromiatnikov
2021-08-17 15:52 ` Peter Zijlstra
2021-08-17 23:17 ` Eugene Syromiatnikov
2021-08-19 11:09 ` [PATCH] sched: Fix Core-wide rq->lock for uninitialized CPUs Peter Zijlstra
2021-08-19 15:50 ` Tao Zhou
2021-08-19 16:19 ` Eugene Syromiatnikov
2021-08-20 0:18 ` Josh Don
2021-08-20 10:02 ` Peter Zijlstra
2021-08-23 9:07 ` [tip: sched/urgent] " tip-bot2 for Peter Zijlstra
2021-04-22 12:05 ` [PATCH 19/19] kselftest: Add test for core sched prctl interface Peter Zijlstra
2021-05-12 10:28 ` [tip: sched/core] " tip-bot2 for Chris Hyser
2021-04-22 16:43 ` [PATCH 00/19] sched: Core Scheduling Don Hiatt
2021-04-22 17:29 ` Peter Zijlstra
2021-04-30 6:47 ` Ning, Hongyu [this message]
2021-05-06 10:29 ` Peter Zijlstra
2021-05-06 12:53 ` Ning, Hongyu
2021-05-07 18:02 ` Joel Fernandes
2021-05-10 16:16 ` Vincent Guittot
2021-05-11 7:00 ` Vincent Guittot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a49ea23a-998e-2282-4c93-5c6c94f2c28d@linux.intel.com \
--to=hongyu.ning@linux.intel.com \
--cc=aubrey.li@linux.intel.com \
--cc=chris.hyser@oracle.com \
--cc=joel@joelfernandes.org \
--cc=joshdon@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).