From: Parth Shah <parth@linux.ibm.com>
To: Subhra Mazumdar <subhra.mazumdar@oracle.com>,
linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, mingo@redhat.com, tglx@linutronix.de,
steven.sistare@oracle.com, dhaval.giani@oracle.com,
daniel.lezcano@linaro.org, vincent.guittot@linaro.org,
viresh.kumar@linaro.org, tim.c.chen@linux.intel.com,
mgorman@techsingularity.net
Subject: Re: [PATCH v3 5/7] sched: SIS_CORE to disable idle core search
Date: Mon, 1 Jul 2019 15:27:54 +0530 [thread overview]
Message-ID: <be91602a-0243-e094-8c8f-ceed314d10ce@linux.ibm.com> (raw)
In-Reply-To: <59ab08d5-8b7c-00b9-230b-7c0b307a675f@oracle.com>
On 6/29/19 3:59 AM, Subhra Mazumdar wrote:
>
> On 6/28/19 12:01 PM, Parth Shah wrote:
>>
>> On 6/27/19 6:59 AM, subhra mazumdar wrote:
>>> Use SIS_CORE to disable idle core search. For some workloads
>>> select_idle_core becomes a scalability bottleneck, removing it improves
>>> throughput. Also there are workloads where disabling it can hurt latency,
>>> so need to have an option.
>>>
>>> Signed-off-by: subhra mazumdar <subhra.mazumdar@oracle.com>
>>> ---
>>> kernel/sched/fair.c | 8 +++++---
>>> 1 file changed, 5 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>>> index c1ca88e..6a74808 100644
>>> --- a/kernel/sched/fair.c
>>> +++ b/kernel/sched/fair.c
>>> @@ -6280,9 +6280,11 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>>> if (!sd)
>>> return target;
>>>
>>> - i = select_idle_core(p, sd, target);
>>> - if ((unsigned)i < nr_cpumask_bits)
>>> - return i;
>>> + if (sched_feat(SIS_CORE)) {
>>> + i = select_idle_core(p, sd, target);
>>> + if ((unsigned)i < nr_cpumask_bits)
>>> + return i;
>>> + }
>> This can have significant performance loss if disabled. The select_idle_core spreads
>> workloads quickly across the cores, hence disabling this leaves much of the work to
>> be offloaded to load balancer to move task across the cores. Latency sensitive
>> and long running multi-threaded workload should see the regression under this conditions.
> Yes in case of SPARC SMT8 I did notice that (see cover letter). That's why
> it is a feature that is ON by default, but can be turned OFF for specific
> workloads on x86 SMT2 that can benefit from it.
>> Also, systems like POWER9 has sd_llc as a pair of core only. So it
>> won't benefit from the limits and hence also hiding your code in select_idle_cpu
>> behind static keys will be much preferred.
> If it doesn't hurt then I don't see the point.
>
So these is the result from POWER9 system with your patches:
System configuration: 2 Socket, 44 cores, 176 CPUs
Experiment setup:
===========
=> Setup 1:
- 44 tasks doing just while(1), this is to make select_idle_core return -1 most times
- perf bench sched messaging -g 1 -l 1000000
+-----------+--------+--------------+--------+
| Baseline | stddev | Patch | stddev |
+-----------+--------+--------------+--------+
| 135 | 3.21 | 158(-17.03%) | 4.69 |
+-----------+--------+--------------+--------+
=> Setup 2:
- schbench -m44 -t 1
+=======+==========+=========+=========+==========+
| %ile | Baseline | stddev | patch | stddev |
+=======+==========+=========+=========+==========+
| 50 | 10 | 3.49 | 10 | 2.29 |
+-------+----------+---------+---------+----------+
| 95 | 467 | 4.47 | 469 | 0.81 |
+-------+----------+---------+---------+----------+
| 99 | 571 | 21.32 | 584 | 18.69 |
+-------+----------+---------+---------+----------+
| 99.5 | 629 | 30.05 | 641 | 20.95 |
+-------+----------+---------+---------+----------+
| 99.9 | 780 | 40.38 | 773 | 44.2 |
+-------+----------+---------+---------+----------+
I guess it doesn't make much difference in schbench results but hackbench (perf bench)
seems to have an observable regression.
Best,
Parth
next prev parent reply other threads:[~2019-07-01 9:58 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-27 1:29 [RESEND PATCH v3 0/7] Improve scheduler scalability for fast path subhra mazumdar
2019-06-27 1:29 ` [PATCH v3 1/7] sched: limit cpu search in select_idle_cpu subhra mazumdar
2019-06-28 18:47 ` Parth Shah
2019-06-28 22:21 ` Subhra Mazumdar
2019-06-27 1:29 ` [PATCH v3 2/7] sched: introduce per-cpu var next_cpu to track search limit subhra mazumdar
2019-06-27 1:29 ` [PATCH v3 3/7] sched: rotate the cpu search window for better spread subhra mazumdar
2019-06-28 11:54 ` Srikar Dronamraju
2019-06-28 22:34 ` Subhra Mazumdar
2019-06-28 18:36 ` Parth Shah
2019-06-28 22:14 ` Subhra Mazumdar
2019-06-27 1:29 ` [PATCH v3 4/7] sched: add sched feature to disable idle core search subhra mazumdar
2019-06-27 1:29 ` [PATCH v3 5/7] sched: SIS_CORE " subhra mazumdar
2019-06-28 19:01 ` Parth Shah
2019-06-28 22:29 ` Subhra Mazumdar
2019-07-01 9:57 ` Parth Shah [this message]
2019-07-01 20:37 ` Subhra Mazumdar
2019-07-04 12:34 ` Parth Shah
2019-07-14 1:16 ` Subhra Mazumdar
2019-06-27 1:29 ` [PATCH v3 6/7] x86/smpboot: introduce per-cpu variable for HT siblings subhra mazumdar
2019-06-27 6:51 ` Thomas Gleixner
2019-06-27 6:54 ` Thomas Gleixner
2019-06-28 1:06 ` Subhra Mazumdar
2019-06-28 1:02 ` Subhra Mazumdar
2019-06-27 1:29 ` [PATCH v3 7/7] sched: use per-cpu variable cpumask_weight_sibling subhra mazumdar
2019-07-01 9:02 ` [RESEND PATCH v3 0/7] Improve scheduler scalability for fast path Peter Zijlstra
2019-07-01 13:55 ` Patrick Bellasi
2019-07-01 14:04 ` Peter Zijlstra
2019-07-08 22:32 ` Tim Chen
2019-07-01 14:06 ` Peter Zijlstra
2019-07-02 0:01 ` Subhra Mazumdar
2019-07-02 8:54 ` Patrick Bellasi
2019-07-03 3:52 ` Subhra Mazumdar
2019-07-04 11:35 ` Parth Shah
-- strict thread matches above, loose matches on Subject: below --
2019-06-09 1:49 [PATCH " subhra mazumdar
2019-06-09 1:49 ` [PATCH v3 5/7] sched: SIS_CORE to disable idle core search subhra mazumdar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=be91602a-0243-e094-8c8f-ceed314d10ce@linux.ibm.com \
--to=parth@linux.ibm.com \
--cc=daniel.lezcano@linaro.org \
--cc=dhaval.giani@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=steven.sistare@oracle.com \
--cc=subhra.mazumdar@oracle.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=vincent.guittot@linaro.org \
--cc=viresh.kumar@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).