From: Wanpeng Li <kernellwp@gmail.com>
To: Zeng Guang <guang.zeng@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
Sean Christopherson <seanjc@google.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>,
Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
kvm <kvm@vger.kernel.org>,
Dave Hansen <dave.hansen@linux.intel.com>,
Tony Luck <tony.luck@intel.com>,
Kan Liang <kan.liang@linux.intel.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
"H. Peter Anvin" <hpa@zytor.com>,
Kim Phillips <kim.phillips@amd.com>,
Jarkko Sakkinen <jarkko@kernel.org>,
Jethro Beekman <jethro@fortanix.com>,
Kai Huang <kai.huang@intel.com>,
"the arch/x86 maintainers" <x86@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Robert Hu <robert.hu@intel.com>, Gao Chao <chao.gao@intel.com>
Subject: Re: [PATCH 0/5] IPI virtualization support for VM
Date: Fri, 16 Jul 2021 17:25:08 +0800 [thread overview]
Message-ID: <CANRm+Cy=ncU-H7duei5q+CG+pm-kXvG8N8CiUQavQ3OEpDj9eg@mail.gmail.com> (raw)
In-Reply-To: <20210716064808.14757-1-guang.zeng@intel.com>
On Fri, 16 Jul 2021 at 15:14, Zeng Guang <guang.zeng@intel.com> wrote:
>
> Current IPI process in guest VM will virtualize the writing to interrupt
> command register(ICR) of the local APIC which will cause VM-exit anyway
> on source vCPU. Frequent VM-exit could induce much overhead accumulated
> if running IPI intensive task.
>
> IPI virtualization as a new VT-x feature targets to eliminate VM-exits
> when issuing IPI on source vCPU. It introduces a new VM-execution
> control - "IPI virtualization"(bit4) in the tertiary processor-based
> VM-exection controls and a new data structure - "PID-pointer table
> address" and "Last PID-pointer index" referenced by the VMCS. When "IPI
> virtualization" is enabled, processor emulateds following kind of writes
> to APIC registers that would send IPIs, moreover without causing VM-exits.
> - Memory-mapped ICR writes
> - MSR-mapped ICR writes
> - SENDUIPI execution
>
> This patch series implement IPI virtualization support in KVM.
>
> Patches 1-3 add tertiary processor-based VM-execution support
> framework.
>
> Patch 4 implement interrupt dispatch support in x2APIC mode with
> APIC-write VM exit. In previous platform, no CPU would produce
> APIC-write VM exit with exit qulification 300H when the "virtual x2APIC
> mode" VM-execution control was 1.
>
> Patch 5 implement IPI virtualization related function including
> feature enabling through tertiary processor-based VM-execution in
> various scenario of VMCS configuration, PID table setup in vCPU creation
> and vCPU block consideration.
>
> Document for IPI virtualization is now available at the latest "Intel
> Architecture Instruction Set Extensions Programming Reference".
>
> Document Link:
> https://software.intel.com/content/www/us/en/develop/download/intel-architecture-instruction-set-extensions-programming-reference.html
>
> We did experiment to measure average time sending IPI from source vCPU
> to the target vCPU completing the IPI handling by kvm unittest w/ and
> w/o IPI virtualization. When IPI virtualizatin enabled, it will reduce
> 22.21% and 15.98% cycles comsuming in xAPIC mode and x2APIC mode
> respectly.
>
> KMV unittest:vmexit/ipi, 2 vCPU, AP runs without halt to ensure no VM
> exit impact on target vCPU.
>
> Cycles of IPI
> xAPIC mode x2APIC mode
> test w/o IPIv w/ IPIv w/o IPIv w/ IPIv
> 1 6106 4816 4265 3768
> 2 6244 4656 4404 3546
> 3 6165 4658 4233 3474
> 4 5992 4710 4363 3430
> 5 6083 4741 4215 3551
> 6 6238 4904 4304 3547
> 7 6164 4617 4263 3709
> 8 5984 4763 4518 3779
> 9 5931 4712 4645 3667
> 10 5955 4530 4332 3724
> 11 5897 4673 4283 3569
> 12 6140 4794 4178 3598
> 13 6183 4728 4363 3628
> 14 5991 4994 4509 3842
> 15 5866 4665 4520 3739
> 16 6032 4654 4229 3701
> 17 6050 4653 4185 3726
> 18 6004 4792 4319 3746
> 19 5961 4626 4196 3392
> 20 6194 4576 4433 3760
>
> Average cycles 6059 4713.1 4337.85 3644.8
> %Reduction -22.21% -15.98%
Commit a9ab13ff6e (KVM: X86: Improve latency for single target IPI
fastpath) mentioned that the whole ipi fastpath feature reduces the
latency from 4238 to 3293 around 22.3% on SKX server, why your IPIv
hardware acceleration is worse than software emulation? In addition,
please post the IPI microbenchmark score w/ and w/o the
patchset.(https://lore.kernel.org/kvm/20171219085010.4081-1-ynorov@caviumnetworks.com),
I found that the hardware acceleration is not always outstanding.
https://lore.kernel.org/kvm/CANRm+Cx597FNRUCyVz1D=B6Vs2GX3Sw57X7Muk+yMpi_hb+v1w@mail.gmail.com
Wanpeng
next prev parent reply other threads:[~2021-07-16 9:25 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-16 6:48 [PATCH 0/5] IPI virtualization support for VM Zeng Guang
2021-07-16 6:48 ` [PATCH 1/6] x86/feat_ctl: Add new VMX feature, Tertiary VM-Execution control Zeng Guang
2021-07-28 23:44 ` Sean Christopherson
2021-07-29 15:56 ` Paolo Bonzini
2021-08-02 8:22 ` Zeng Guang
2021-08-02 16:20 ` Sean Christopherson
2021-07-16 6:48 ` [PATCH 2/6] KVM: VMX: Extend BUILD_CONTROLS_SHADOW macro to support 64-bit variation Zeng Guang
2021-07-16 6:48 ` [PATCH 3/6] KVM: VMX: Detect Tertiary VM-Execution control when setup VMCS config Zeng Guang
2021-07-29 0:03 ` Sean Christopherson
2021-08-02 6:59 ` Zeng Guang
2021-07-16 6:48 ` [PATCH 4/6] KVM: VMX: dump_vmcs() reports tertiary_exec_control field as well Zeng Guang
2021-07-16 6:48 ` [PATCH 5/6] KVM: x86: Support interrupt dispatch in x2APIC mode with APIC-write VM exit Zeng Guang
2021-07-16 6:48 ` [PATCH 6/6] KVM: VMX: enable IPI virtualization Zeng Guang
2021-07-16 9:52 ` Paolo Bonzini
2021-07-17 3:55 ` Zeng Guang
2021-07-18 20:32 ` Paolo Bonzini
2021-07-19 12:38 ` Zeng Guang
2021-07-19 13:58 ` Paolo Bonzini
2021-07-20 1:07 ` Zeng Guang
2021-07-19 13:16 ` Zeng Guang
2021-07-16 9:25 ` Wanpeng Li [this message]
2021-07-17 1:46 ` [PATCH 0/5] IPI virtualization support for VM Zeng Guang
2021-07-19 7:26 ` Zeng Guang
2021-07-19 7:37 ` Wanpeng Li
2021-07-23 6:15 ` Zeng Guang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CANRm+Cy=ncU-H7duei5q+CG+pm-kXvG8N8CiUQavQ3OEpDj9eg@mail.gmail.com' \
--to=kernellwp@gmail.com \
--cc=bp@alien8.de \
--cc=chao.gao@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=guang.zeng@intel.com \
--cc=hpa@zytor.com \
--cc=jarkko@kernel.org \
--cc=jethro@fortanix.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kai.huang@intel.com \
--cc=kan.liang@linux.intel.com \
--cc=kim.phillips@amd.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=robert.hu@intel.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.