RE: [PATCH v2] x86/hyperv: make vapic support x2apic mode

From: Michael Kelley <mikelley@microsoft.com>
To: Roman Kagan <rkagan@virtuozzo.com>, vkuznets <vkuznets@redhat.com>
Cc: "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	Tianyu Lan <Tianyu.Lan@microsoft.com>,
	Joerg Roedel <jroedel@suse.de>, KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	Sasha Levin <sashal@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"x86@kernel.org" <x86@kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v2] x86/hyperv: make vapic support x2apic mode
Date: Fri, 4 Oct 2019 03:01:51 +0000	[thread overview]
Message-ID: <CY4PR21MB0136269170E69EA8F02A89E9D79E0@CY4PR21MB0136.namprd21.prod.outlook.com> (raw)
In-Reply-To: <20191003125236.GA2424@rkaganb.sw.ru>

From: Roman Kagan <rkagan@virtuozzo.com> Sent: Thursday, October 3, 2019 5:53 AM
> >
> > AFAIU you're trying to mirror native_x2apic_icr_write() here but this is
> > different from what hv_apic_icr_write() does
> > (SET_APIC_DEST_FIELD(id)).
> 
> Right.  In xapic mode the ICR2 aka the high 4 bytes of ICR is programmed
> with the destination id in the highest byte; in x2apic mode the whole
> ICR2 is set to the 32bit destination id.
> 
> > Is it actually correct? (I think you've tested this and it is but)
> 
> As I wrote in the commit log, I haven't tested it in the sense that I
> ran a Linux guest in a Hyper-V VM exposing x2apic to the guest, because
> I didn't manage to configure it to do so.  OTOH I did run a Windows
> guest in QEMU/KVM with hv_apic and x2apic enabled and saw it write
> destination ids unshifted to the ICR2 part of ICR, so I assume it's
> correct.
> 
> > Michael, could you please shed some light here?
> 
> Would be appreciated, indeed.
> 

The newest version of Hyper-V provides an x2apic in a guest VM when the
number of vCPUs in the VM is > 240.  This version of Hyper-V is beginning
to be deployed in Azure to enable the M416v2 VM size, but the functionality
is not yet available for the on-premises version of Hyper-V.  However, I can
test this configuration internally with the above patch -- give me a few days.

An additional complication is that when running on Intel processors that offer
vAPIC functionality, the Hyper-V "hints" value does *not* recommend using the
MSR-based APIC accesses.  In this case, memory-mapped access to the x2apic
registers is faster than the synthetic MSRs.  I've already looked at a VM that has
the x2apic, and indeed that is the case, so the above code wouldn't run
anyway.  But I can temporarily code around that for testing purposes and see
if everything works.

Michael