From: "Tian, Kevin" <kevin.tian@intel.com>
To: "Gao, Chao" <chao.gao@intel.com>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"Kohler, Jon" <jon@nutanix.com>
Cc: "Christopherson,, Sean" <seanjc@google.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
"x86@kernel.org" <x86@kernel.org>,
"H. Peter Anvin" <hpa@zytor.com>
Subject: RE: [RFC v2] KVM: x86/vmx: Suppress posted interrupt notification when CPU is in host
Date: Wed, 28 Sep 2022 08:29:48 +0000 [thread overview]
Message-ID: <BL1PR11MB52712631CBBED273F5E4E1828C549@BL1PR11MB5271.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20220923085806.384344-1-chao.gao@intel.com>
> From: Gao, Chao <chao.gao@intel.com>
> Sent: Friday, September 23, 2022 4:58 PM
>
> PIN (Posted interrupt notification) is useless to host as KVM always syncs
> pending guest interrupts in PID to guest's vAPIC before each VM entry. In
> fact, Sending PINs to a CPU running in host will lead to additional
> overhead due to interrupt handling.
>
> Currently, software path, vmx_deliver_posted_interrupt(), is optimized to
> issue PINs only if target vCPU is in IN_GUEST_MODE. But hardware paths
> (VT-d and Intel IPI virtualization) aren't optimized.
>
> Set PID.SN right after VM exits and clear it before VM entry to minimize
> the chance of hardware issuing PINs to a CPU when it's in host.
>
> Also honour PID.SN bit in vmx_deliver_posted_interrupt().
>
> Opportunistically clean up vmx_vcpu_pi_put(); when a vCPU is preempted,
> it is pointless to update PID.NV to wakeup vector since notification is
> anyway suppressed. And since PID.SN should be already set for running
> vCPUs, so, don't set it again for preempted vCPUs.
>
> When IPI virtualization is enabled, this patch increases "perf bench" [*]
> by 6.56%, and PIN count in 1 second drops from tens of thousands to
> hundreds. But cpuid loop test shows this patch causes 1.58% overhead in
> VM-exit round-trip latency.
>
> [*] test cmd: perf bench sched pipe -T. Note that we change the source
> code to pin two threads to two different vCPUs so that it can reproduce
> stable results.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> ---
> RFC: I am not sure whether the benefits outweighs the extra VM-exit cost.
>
> Changes in v2 (addressed comments from Kevin):
> - measure/estimate the impact to non-IPC-intensive cases
> - don't tie PID.SN to vcpu->mode. Instead, clear PID.SN
> right before VM-entry and set it after VM-exit.
One correction here. My comment in v1 [1] was actually close to Sean's
suggestion, i.e. opposite to above description:
--
I wonder whether it makes more sense to have 'sn' closely sync-ed
with vcpu->mode, e.g. having a kvm_x86_set_vcpu_mode() ops
to translate vcpu->mode into vmx/svm specific hardware bits like
'sn' here. Then call it in common place when vcpu->mode is changed.
--
[1] https://lore.kernel.org/lkml/BN9PR11MB52766B74ADFBAEC0AA205E298CB39@BN9PR11MB5276.namprd11.prod.outlook.com/
next prev parent reply other threads:[~2022-09-28 8:30 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-23 8:58 [RFC v2] KVM: x86/vmx: Suppress posted interrupt notification when CPU is in host Chao Gao
2022-09-26 16:19 ` Sean Christopherson
2022-09-27 6:32 ` Chao Gao
2022-09-27 21:43 ` Sean Christopherson
2022-09-28 11:26 ` Chao Gao
2022-09-28 8:29 ` Tian, Kevin [this message]
2022-09-28 10:56 ` Chao Gao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BL1PR11MB52712631CBBED273F5E4E1828C549@BL1PR11MB5271.namprd11.prod.outlook.com \
--to=kevin.tian@intel.com \
--cc=bp@alien8.de \
--cc=chao.gao@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=jon@nutanix.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).