[PATCH] x86/hyper-v: implement EOI assist

[PATCH] x86/hyper-v: implement EOI assist

[Vitaly Kuznetsov]

Hyper-V TLFS suggests an optimization to avoid imminent VMExit on EOI:
"The OS performs an EOI by atomically writing zero to the EOI Assist field
of the virtual VP assist page and checking whether the "No EOI required"
field was previously zero. If it was, the OS must write to the
HV_X64_APIC_EOI MSR thereby triggering an intercept into the hypervisor."

Implement the optimization in Linux.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 arch/x86/hyperv/hv_apic.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
index 8eb6fbee8e13..5c056b8aebef 100644
--- a/arch/x86/hyperv/hv_apic.c
+++ b/arch/x86/hyperv/hv_apic.c
@@ -86,6 +86,11 @@ static void hv_apic_write(u32 reg, u32 val)
 
 static void hv_apic_eoi_write(u32 reg, u32 val)
 {
+	struct hv_vp_assist_page *hvp = hv_vp_assist_page[smp_processor_id()];
+
+	if (hvp && (xchg(&hvp->apic_assist, 0) & 0x1))
+		return;
+
 	wrmsr(HV_X64_MSR_EOI, val, 0);
 }
 
-- 
2.20.1

Re: [PATCH] x86/hyper-v: implement EOI assist

[Vitaly Kuznetsov]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:

> Hyper-V TLFS suggests an optimization to avoid imminent VMExit on EOI:
> "The OS performs an EOI by atomically writing zero to the EOI Assist field
> of the virtual VP assist page and checking whether the "No EOI required"
> field was previously zero. If it was, the OS must write to the
> HV_X64_APIC_EOI MSR thereby triggering an intercept into the hypervisor."
>
> Implement the optimization in Linux.
>

Simon, Long,

did you get a chance to run some tests with this?

-- 
Vitaly

RE: [PATCH] x86/hyper-v: implement EOI assist

[Long Li]

>>>Subject: Re: [PATCH] x86/hyper-v: implement EOI assist
>>>
>>>Vitaly Kuznetsov <vkuznets@redhat.com> writes:
>>>
>>>> Hyper-V TLFS suggests an optimization to avoid imminent VMExit on EOI:
>>>> "The OS performs an EOI by atomically writing zero to the EOI Assist
>>>> field of the virtual VP assist page and checking whether the "No EOI
>>>required"
>>>> field was previously zero. If it was, the OS must write to the
>>>> HV_X64_APIC_EOI MSR thereby triggering an intercept into the
>>>hypervisor."
>>>>
>>>> Implement the optimization in Linux.
>>>>
>>>
>>>Simon, Long,
>>>
>>>did you get a chance to run some tests with this?

I have ran some tests on Azure L80s_v2.

With 10 NVMe disks on raid0 and formatted to EXT4, I'm getting 2.6m max IOPS with the patch, compared to 2.55m IOPS before.

The VM has been running stable. Thank you!

Tested-by: Long Li <longli@microsoft.com>

>>>
>>>--
>>>Vitaly

[tip:x86/platform] x86/hyper-v: Implement EOI assist

[tip-bot for Vitaly Kuznetsov]

Commit-ID:  ba696429d290690db967e5f49463df4b2c1314a4
Gitweb:     https://git.kernel.org/tip/ba696429d290690db967e5f49463df4b2c1314a4
Author:     Vitaly Kuznetsov <vkuznets@redhat.com>
AuthorDate: Wed, 3 Apr 2019 19:03:09 +0200
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 16 Apr 2019 12:28:23 +0200

x86/hyper-v: Implement EOI assist

Hyper-V TLFS suggests an optimization to avoid imminent VMExit on EOI:
"The OS performs an EOI by atomically writing zero to the EOI Assist field
of the virtual VP assist page and checking whether the "No EOI required"
field was previously zero. If it was, the OS must write to the
HV_X64_APIC_EOI MSR thereby triggering an intercept into the hypervisor."

Implement the optimization in Linux.

Tested-by: Long Li <longli@microsoft.com>
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael Kelley (EOSG) <Michael.H.Kelley@microsoft.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sasha Levin <sashal@kernel.org>
Cc: Simon Xiao <sixiao@microsoft.com>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-hyperv@vger.kernel.org
Link: http://lkml.kernel.org/r/20190403170309.4107-1-vkuznets@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/hyperv/hv_apic.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c
index 8eb6fbee8e13..5c056b8aebef 100644
--- a/arch/x86/hyperv/hv_apic.c
+++ b/arch/x86/hyperv/hv_apic.c
@@ -86,6 +86,11 @@ static void hv_apic_write(u32 reg, u32 val)
 
 static void hv_apic_eoi_write(u32 reg, u32 val)
 {
+	struct hv_vp_assist_page *hvp = hv_vp_assist_page[smp_processor_id()];
+
+	if (hvp && (xchg(&hvp->apic_assist, 0) & 0x1))
+		return;
+
 	wrmsr(HV_X64_MSR_EOI, val, 0);
 }