From: Wanpeng Li <kernellwp@gmail.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: LKML <linux-kernel@vger.kernel.org>, kvm <kvm@vger.kernel.org>,
"Radim Krčmář" <rkrcmar@redhat.com>,
"Sean Christopherson" <sean.j.christopherson@intel.com>,
"Vitaly Kuznetsov" <vkuznets@redhat.com>,
"Wanpeng Li" <wanpengli@tencent.com>,
"Jim Mattson" <jmattson@google.com>,
"Joerg Roedel" <joro@8bytes.org>
Subject: Re: [PATCH 1/2] KVM: X86: Single target IPI fastpath
Date: Tue, 12 Nov 2019 09:33:49 +0800 [thread overview]
Message-ID: <CANRm+CzK_h2E9XWFipkNpAALLCBcM2vrUkdBpumwmT9AP09hfA@mail.gmail.com> (raw)
In-Reply-To: <4418c734-68e1-edaf-c939-f24d041acf2e@redhat.com>
On Tue, 12 Nov 2019 at 05:59, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 09/11/19 08:05, Wanpeng Li wrote:
> > From: Wanpeng Li <wanpengli@tencent.com>
> >
> > This patch tries to optimize x2apic physical destination mode, fixed delivery
> > mode single target IPI by delivering IPI to receiver immediately after sender
> > writes ICR vmexit to avoid various checks when possible.
> >
> > Testing on Xeon Skylake server:
> >
> > The virtual IPI latency from sender send to receiver receive reduces more than
> > 330+ cpu cycles.
> >
> > Running hackbench(reschedule ipi) in the guest, the avg handle time of MSR_WRITE
> > caused vmexit reduces more than 1000+ cpu cycles:
> >
> > Before patch:
> >
> > VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
> > MSR_WRITE 5417390 90.01% 16.31% 0.69us 159.60us 1.08us
> >
> > After patch:
> >
> > VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
> > MSR_WRITE 6726109 90.73% 62.18% 0.48us 191.27us 0.58us
>
> Do you have retpolines enabled? The bulk of the speedup might come just
> from the indirect jump.
Adding 'mitigations=off' to the host grub parameter:
Before patch:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
MSR_WRITE 2681713 92.98% 77.52% 0.38us 18.54us
0.73us ( +- 0.02% )
After patch:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
MSR_WRITE 2953447 92.48% 62.47% 0.30us 59.09us
0.40us ( +- 0.02% )
Actually, this is not the first attempt to add shortcut for MSR writes
which performance sensitive, the other effort is tscdeadline timer
from Isaku Yamahata, https://patchwork.kernel.org/cover/10541035/ ,
ICR and TSCDEADLINE MSR writes cause the main MSR write vmexits in our
product observation, multicast IPIs are not as common as unicast IPI
like RESCHEDULE_VECTOR and CALL_FUNCTION_SINGLE_VECTOR etc. As far as
I know, something similar to this patch has already been deployed in
some cloud companies private kvm fork.
Wanpeng
next prev parent reply other threads:[~2019-11-12 1:34 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-09 7:05 [PATCH 1/2] KVM: X86: Single target IPI fastpath Wanpeng Li
2019-11-09 7:05 ` [PATCH 2/2] KVM: LAPIC: micro-optimize fixed mode ipi delivery Wanpeng Li
2019-11-11 21:59 ` Paolo Bonzini
2019-11-12 1:34 ` Wanpeng Li
2019-11-10 1:59 ` [PATCH 1/2] KVM: X86: Single target IPI fastpath kbuild test robot
2019-11-10 1:59 ` kbuild test robot
2019-11-10 1:59 ` [PATCH] KVM: X86: fix semicolon.cocci warnings kbuild test robot
2019-11-10 1:59 ` kbuild test robot
2019-11-11 13:06 ` [PATCH 1/2] KVM: X86: Single target IPI fastpath Vitaly Kuznetsov
2019-11-12 1:18 ` Wanpeng Li
2019-11-11 21:59 ` Paolo Bonzini
2019-11-12 1:33 ` Wanpeng Li [this message]
2019-11-13 6:05 ` Wanpeng Li
2019-11-14 3:12 ` Wanpeng Li
2019-11-14 11:58 ` Paolo Bonzini
2019-11-14 15:22 ` Sean Christopherson
2019-11-14 15:44 ` Paolo Bonzini
2019-11-14 16:34 ` Sean Christopherson
2019-11-15 5:56 ` Wanpeng Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CANRm+CzK_h2E9XWFipkNpAALLCBcM2vrUkdBpumwmT9AP09hfA@mail.gmail.com \
--to=kernellwp@gmail.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=sean.j.christopherson@intel.com \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.