From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
To: Ingo Molnar <mingo@kernel.org>, Gleb Natapov <gleb@redhat.com>,
Marcelo Tosatti <mtosatti@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Avi Kivity <avi.kivity@gmail.com>,
"H. Peter Anvin" <hpa@zytor.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Rik van Riel <riel@redhat.com>,
Srikar <srikar@linux.vnet.ibm.com>,
"Nikunj A. Dadhania" <nikunj@linux.vnet.ibm.com>,
KVM <kvm@vger.kernel.org>, Jiannan Ouyang <ouyang@cs.pitt.edu>,
Chegu Vinod <chegu_vinod@hp.com>,
"Andrew M. Theurer" <habanero@linux.vnet.ibm.com>,
LKML <linux-kernel@vger.kernel.org>,
Srivatsa Vaddagiri <srivatsa.vaddagiri@gmail.com>,
Andrew Jones <drjones@redhat.com>
Subject: Re: [PATCH V3 RESEND RFC 1/2] sched: Bail out of yield_to when source and target runqueue has one task
Date: Sun, 27 Jan 2013 22:28:37 +0530 [thread overview]
Message-ID: <51055CBD.3050904@linux.vnet.ibm.com> (raw)
In-Reply-To: <20130125184911.GE31022@gmail.com>
On 01/26/2013 12:19 AM, Ingo Molnar wrote:
>
> * Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com> wrote:
>
>> On 01/25/2013 04:17 PM, Ingo Molnar wrote:
>>>
>>> * Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com> wrote:
>>>
>>>> * Ingo Molnar <mingo@kernel.org> [2013-01-24 11:32:13]:
>>>>
>>>>>
>>>>> * Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com> wrote:
>>>>>
>>>>>> From: Peter Zijlstra <peterz@infradead.org>
>>>>>>
>>>>>> In case of undercomitted scenarios, especially in large guests
>>>>>> yield_to overhead is significantly high. when run queue length of
>>>>>> source and target is one, take an opportunity to bail out and return
>>>>>> -ESRCH. This return condition can be further exploited to quickly come
>>>>>> out of PLE handler.
>>>>>>
>>>>>> (History: Raghavendra initially worked on break out of kvm ple handler upon
>>>>>> seeing source runqueue length = 1, but it had to export rq length).
>>>>>> Peter came up with the elegant idea of return -ESRCH in scheduler core.
>>>>>>
>>>>>> Signed-off-by: Peter Zijlstra <peterz@infradead.org>
>>>>>> Raghavendra, Checking the rq length of target vcpu condition added.(thanks Avi)
>>>>>> Reviewed-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>>>>>> Signed-off-by: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
>>>>>> Acked-by: Andrew Jones <drjones@redhat.com>
>>>>>> Tested-by: Chegu Vinod <chegu_vinod@hp.com>
>>>>>> ---
>>>>>>
>>>>>> kernel/sched/core.c | 25 +++++++++++++++++++------
>>>>>> 1 file changed, 19 insertions(+), 6 deletions(-)
>>>>>>
>>>>>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>>>>>> index 2d8927f..fc219a5 100644
>>>>>> --- a/kernel/sched/core.c
>>>>>> +++ b/kernel/sched/core.c
>>>>>> @@ -4289,7 +4289,10 @@ EXPORT_SYMBOL(yield);
>>>>>> * It's the caller's job to ensure that the target task struct
>>>>>> * can't go away on us before we can do any checks.
>>>>>> *
>>>>>> - * Returns true if we indeed boosted the target task.
>>>>>> + * Returns:
>>>>>> + * true (>0) if we indeed boosted the target task.
>>>>>> + * false (0) if we failed to boost the target.
>>>>>> + * -ESRCH if there's no task to yield to.
>>>>>> */
>>>>>> bool __sched yield_to(struct task_struct *p, bool preempt)
>>>>>> {
>>>>>> @@ -4303,6 +4306,15 @@ bool __sched yield_to(struct task_struct *p, bool preempt)
>>>>>>
>>>>>> again:
>>>>>> p_rq = task_rq(p);
>>>>>> + /*
>>>>>> + * If we're the only runnable task on the rq and target rq also
>>>>>> + * has only one task, there's absolutely no point in yielding.
>>>>>> + */
>>>>>> + if (rq->nr_running == 1 && p_rq->nr_running == 1) {
>>>>>> + yielded = -ESRCH;
>>>>>> + goto out_irq;
>>>>>> + }
>>>>>
>>>>> Looks good to me in principle.
>>>>>
>>>>> Would be nice to get more consistent benchmark numbers. Once
>>>>> those are unambiguously showing that this is a win:
>>>>>
>>>>> Acked-by: Ingo Molnar <mingo@kernel.org>
>>>>>
>>>>
>>>> I ran the test with kernbench and sysbench again on 32 core mx3850
>>>> machine with 32 vcpu guests. Results shows definite improvements.
>>>>
>>>> ebizzy and dbench show similar improvement for 1x overcommit
>>>> (note that stdev for 1x in dbench is lesser improvemet is now seen at
>>>> only 20%)
>>>>
>>>> [ all the experiments are taken out of 8 run averages ].
>>>>
>>>> The patches benefit large guest undercommit scenarios, so I believe
>>>> with large guest performance improvemnt is even significant. [ Chegu
>>>> Vinod results show performance near to no ple cases ]. Unfortunately I
>>>> do not have a machine to test larger guest (>32).
>>>>
>>>> Ingo, Please let me know if this is okay to you.
>>>>
>>>> base kernel = 3.8.0-rc4
>>>>
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> kernbench (time in sec lower is better)
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> base stdev patched stdev %improve
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> 1x 46.6028 1.8672 42.4494 1.1390 8.91234
>>>> 2x 99.9074 9.1859 90.4050 2.6131 9.51121
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> sysbench (time in sec lower is better)
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> base stdev patched stdev %improve
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> 1x 18.7402 0.3764 17.7431 0.3589 5.32065
>>>> 2x 13.2238 0.1935 13.0096 0.3152 1.61981
>>>> +-----------+-----------+-----------+------------+-----------+
>>>>
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> ebizzy (records/sec higher is better)
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> base stdev patched stdev %improve
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> 1x 2421.9000 19.1801 5883.1000 112.7243 142.91259
>>>> +-----------+-----------+-----------+------------+-----------+
>>>>
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> dbench (throughput MB/sec higher is better)
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> base stdev patched stdev %improve
>>>> +-----------+-----------+-----------+------------+-----------+
>>>> 1x 11675.9900 857.4154 14103.5000 215.8425 20.79061
>>>> +-----------+-----------+-----------+------------+-----------+
>>>
>>> The numbers look pretty convincing, thanks. The workloads were
>>> CPU bound most of the time, right?
>>
>> Yes. CPU bound most of the time. I also used tmpfs to reduce
>> io overhead (for dbbench).
>
> Ok, cool.
>
> Which tree will this be upstreamed through - the KVM tree? I'd
> suggest the KVM tree because KVM will be the one exposed to the
> effects of this change.
Thanks Ingo.
Marcelo, Could you please take this into kvm tree.. ?
next prev parent reply other threads:[~2013-01-27 17:01 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-01-22 7:38 [PATCH V3 RESEND RFC 0/2] kvm: Improving undercommit scenarios Raghavendra K T
2013-01-22 7:39 ` [PATCH V3 RESEND RFC 1/2] sched: Bail out of yield_to when source and target runqueue has one task Raghavendra K T
2013-01-24 10:32 ` Ingo Molnar
2013-01-25 10:40 ` Raghavendra K T
2013-01-25 10:47 ` Ingo Molnar
2013-01-25 15:54 ` Raghavendra K T
2013-01-25 18:49 ` Ingo Molnar
2013-01-27 16:58 ` Raghavendra K T [this message]
2013-01-25 11:05 ` Andrew Jones
2013-01-25 15:58 ` Raghavendra K T
2013-01-22 7:39 ` [PATCH V3 RESEND RFC 2/2] kvm: Handle yield_to failure return code for potential undercommit case Raghavendra K T
2013-01-23 13:57 ` [PATCH V3 RESEND RFC 0/2] kvm: Improving undercommit scenarios Andrew Jones
2013-01-24 8:27 ` Raghavendra K T
2013-01-29 14:05 ` Gleb Natapov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51055CBD.3050904@linux.vnet.ibm.com \
--to=raghavendra.kt@linux.vnet.ibm.com \
--cc=avi.kivity@gmail.com \
--cc=chegu_vinod@hp.com \
--cc=drjones@redhat.com \
--cc=gleb@redhat.com \
--cc=habanero@linux.vnet.ibm.com \
--cc=hpa@zytor.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mingo@redhat.com \
--cc=mtosatti@redhat.com \
--cc=nikunj@linux.vnet.ibm.com \
--cc=ouyang@cs.pitt.edu \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=srikar@linux.vnet.ibm.com \
--cc=srivatsa.vaddagiri@gmail.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.