RE: [PATCH 2/4] arm64: hyperv: Add support for Hyper-V as a hypervisor

[Michael Kelley <mikelley@microsoft.com>]

From: Marc Zyngier <marc.zyngier@arm.com> Sent: Thursday, December 13, 2018 3:23 AM

> >> As Will said, this isn't a viable option. Please follow SMCCC 1.1.
> >
> > I'll have to start a conversation with the Hyper-V team about this.
> > I don't know why they chose to use HVC #1 or this register scheme
> > for output values.  It may be tough to change at this point because
> > there are Windows guests on Hyper-V for ARM64 that are already
> > using this approach.
> 
> I appreciate you already have stuff in the wild, but there is definitely
> a case to be made for supporting architecturally specified mechanisms in
> a hypervisor, and SMCCC is definitely part of it (I'm certainly curious
> of how you support the Spectre mitigation otherwise).
>

The Hyper-V guys I need to discuss this with are not back from the
holidays until January 7th.  I'll follow up on this thread once I've
had that conversation.

> >>> +static int hv_cpu_init(unsigned int cpu)
> >>> +{
> >>> +	u64 msr_vp_index;
> >>> +
> >>> +	hv_get_vp_index(msr_vp_index);
> >>> +
> >>> +	hv_vp_index[smp_processor_id()] = msr_vp_index;
> >>> +
> >>> +	if (msr_vp_index > hv_max_vp_index)
> >>> +		hv_max_vp_index = msr_vp_index;
> >>> +
> >>> +	return 0;
> >>> +}
> >>
> >> Is that some new way to describe a CPU topology? If so, why isn't that
> >> exposed via the ACPI tables that the kernel already parses?
> >
> > Hyper-V's hypercall interface uses vCPU identifiers that are not
> > guaranteed to be consecutive integers or to match what ACPI shows.
> > No topology information is implied -- it's just unique identifiers.  The
> > hv_vp_index array provides easy mapping from Linux's consecutive
> > integer IDs for CPUs when needed to construct hypercall arguments.
> 
> That's extremely odd. The hypervisor obviously knows which vCPU is doing
> a hypercall, and if referencing another vCPU, the virtualized MPIDR_EL1
> value should be used. I don't think deviating from the architecture is a
> good idea (but I appreciate this is none of your doing). Following the
> architecture would allow this code to directly use the cpu_logical_map
> infrastructure we already have.

I see what you are getting at.  However, some Hyper-V hypercalls allow
specifying arbitrary sets of vCPUs.  These hypercalls are used to define
target processors in the virtual PCI code (which I have not yet brought over
to ARM64) and in enlightenments for IPIs and TLB flushes (used by
Windows guests and Linux guests on x86, but not yet brought over to Linux
ARM64, if they ever will be).  These hypercalls take bitmaps as arguments,
similar to a Linux cpumask, as defined in Sections 7.8.7.3 thru 7.8.7.5 in the
Hyper-V TLFS.  So Hyper-V defines its own VP index that is akin to the index 
into the cpu_logical_map, though it may not be the same mapping.  My earlier
comments may have been misleading -- the Hyper-V VP index is an integer
ranging from 0 thru (# vCPUs - 1).

With these requirements, Hyper-V defining its own VP index seems like
a reasonable thing to do.  And since Hyper-V provides the same hypercall
interfaces for both x86 and ARM64 implementations, and for Windows
guests, there's not much choice but to use the Hyper-V VP index as specified.

Michael