[PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64

[PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64

[Michael Kelley]

This series enables Linux guests running on Hyper-V on ARM64
hardware. New ARM64-specific code in arch/arm64/hyperv initializes
Hyper-V, including its interrupts and hypercall mechanism.
Existing architecture independent drivers for Hyper-V's VMbus and
synthetic devices just work when built for ARM64. Hyper-V code is
built and included in the image and modules only if CONFIG_HYPERV
is enabled.

The seven patches are organized as follows:

1) Update include/linux/arm-smccc.h to provide an HVC wrapper
   variant that returns results in other than X0 thru X3

2) Add definitions and functions for making Hyper-V hypercalls
   and getting/setting virtual processor registers provided by
   Hyper-V

3) Add architecture specific definitions needed by the
   architecture independent Hyper-V clocksource driver in
   drivers/clocksource/hyperv_timer.c. Update the clocksource
   driver to be initialized on ARM64.

4) Add functions needed by the arch independent VMbus driver
   for reporting a panic to Hyper-V and as stubs for the kexec
   and crash handlers.

5) Add Hyper-V initialization code and utility functions that
   report Hyper-v status.

6) Export screen_info so it may be used by the Hyper-V frame buffer
   driver built as a module. It is already exported for x86,
   powerpc, and alpha architectures.

7) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64.

Hyper-V on ARM64 runs with a 4 Kbyte page size, but allows guests
with 4K/16K/64K page size. Linux guests with this ARM64 enablement
code work with all three supported ARM64 page sizes.

The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has
x86/x64-specific code and is not being built for ARM64. Fixing
this driver to enable vPCI devices on ARM64 will be done later.

In a few cases, terminology from the x86/x64 world has been carried
over into the ARM64 code ("MSR", "TSC").  Hyper-V still uses the
x86/x64 terminology and has not replaced it with something more
generic, so the code uses the Hyper-V terminology.  This will be
fixed when Hyper-V updates the usage in the TLFS.

This patch set is based on the hyperv-next branch of the code tree
https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/

Changes in v9:
* Added Patch 1 to enable making an SMCCC compliant hypercall
  that returns results in other than registers X0 thru X3, per
  version 1.2 and later of the SMCCC spec.
* Using the ability to return results in registers X6 and X7,
  converted hv_get_vpreg_128() to use a "fast" hypercall that
  passes inputs and outputs in registers, and in doing so eliminated
  a lot of memory allocation complexity.
* Cleaned up some extra blank lines and use of spaces in aligning
  local variables. [Sunil Muthuswamy]
* Based on discussion about future directions, reverted the
  population of hv_vp_index array to use a cpuhp state instead
  of a hypercall, which is like it was in v7 and earlier.

Changes in v8:
* Removed a lot of code based on refactoring the boundary between
  arch independent and arch dependent code for Hyper-V, per comments
  from Arnd Bergmann. The removed code was either duplicated on
  the x86 side, or has been folded into architecture independent
  code as not really being architecture dependent.
* Added config dependency on !CONFIG_CPU_BIG_ENDIAN [Arnd Bergmann]
* Reworked the approach to Hyper-V initialization. The functionality
  is the same, but is now structured like the Xen code with an early
  init function called in setup_arch() and an early initcall to
  finish the initialization. [Arnd Bergmann]

Changes in v7:
* Separately upstreamed split of hyperv-tlfs.h into arch dependent
  and independent versions.  In this patch set, update the ARM64
  hyperv-tlfs.h to include architecture independent definitions.
  This approach eliminates a lot of lines of otherwise duplicated
  code on the ARM64 side.
* Break ARM64 mshyperv.h into smaller pieces. Have an initial
  baseline, and add code along with patches for a particular
  functional area. [Marc Zyngier]
* In mshyperv.h, use static inline functions instead of #defines
  where possible. [Arnd Bergmann]
* Use VMbus INTID obtained from ACPI DSDT instead of hardcoding.
  The STIMER INTID is still hardcoded because it is needed
  before Linux has initialized the ACPI subsystem, so it can't
  be obtained from the DSDT.  Wedging it into the GTDT seems
  dubious, so was not done. [Marc Zyngier]
* Update Hyper-V page size allocation functions to use
  alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd
  Bergmann]
* Various other minor changes based on feedback and to rebase
  to latest linux-next [Marc Zyngier and Arnd Bergmann]

Changes in v6:
* Use SMCCC hypercall interface instead of direct invocation
  of HVC instruction and the Hyper-V hypercall interface
  [Marc Zyngier]
* Reimplemented functions to alloc/free Hyper-V size pages
  using kmalloc/kfree since kmalloc now guarantees alignment of
  power of 2 size allocations [Marc Zyngier]
* Export screen_info in arm64 architecture so it can be used
  by the Hyper-V buffer driver built as a module
* Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c
  to better reflect its content
* Fixed the bit position of certain feature flags presented by
  Hyper-V to the guest.  The bit positions on ARM64 don't match
  the position on x86 like originally thought.
* Minor fixups to rebase to 5.6-rc5 linux-next

Changes in v5:
* Minor fixups to rebase to 5.4-rc1 linux-next

Changes in v4:
* Moved clock-related code into an architecture independent
  Hyper-V clocksource driver that is already upstream. Clock
  related code is removed from this patch set except for the
  ARM64 specific interrupt handler. [Marc Zyngier]
* Separately upstreamed the split of mshyperv.h into arch independent
  and arch dependent portions. The arch independent portion has been
  removed from this patch set.
* Divided patch #2 of the series into multiple smaller patches
  [Marc Zyngier]
* Changed a dozen or so smaller things based on feedback
  [Marc Zyngier, Will Deacon]
* Added functions to alloc/free Hyper-V size pages for use by
  drivers for Hyper-V synthetic devices when updated to not assume
  guest page size and Hyper-v page size are the same

Changes in v3:
* Added initialization of hv_vp_index array like was recently
  added on x86 branch [KY Srinivasan]
* Changed Hyper-V ARM64 register symbols to be all uppercase 
  instead of mixed case [KY Srinivasan]
* Separated mshyperv.h into two files, one architecture
  independent and one architecture dependent. After this code
  is upstream, will make changes to the x86 code to use the
  architecture independent file and remove duplication. And
  once we have a multi-architecture Hyper-V TLFS, will do a
  separate patch to split hyperv-tlfs.h in the same way.
  [KY Srinivasan]
* Minor tweaks to rebase to latest linux-next code

Changes in v2:
* Removed patch to implement slow_virt_to_phys() on ARM64.
  Use of slow_virt_to_phys() in arch independent Hyper-V
  drivers has been eliminated by commit 6ba34171bcbd
  ("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()")
* Minor tweaks to rebase to latest linux-next code
*** BLURB HERE ***

Michael Kelley (7):
  smccc: Add HVC call variant with result registers other than 0 thru 3
  arm64: hyperv: Add Hyper-V hypercall and register access utilities
  arm64: hyperv: Add Hyper-V clocksource/clockevent support
  arm64: hyperv: Add kexec and panic handlers
  arm64: hyperv: Initialize hypervisor on boot
  arm64: efi: Export screen_info
  Drivers: hv: Enable Hyper-V code to be built on ARM64

 MAINTAINERS                          |   3 +
 arch/arm64/Kbuild                    |   1 +
 arch/arm64/hyperv/Makefile           |   2 +
 arch/arm64/hyperv/hv_core.c          | 178 +++++++++++++++++++++++++++++++++++
 arch/arm64/hyperv/mshyperv.c         | 173 ++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/hyperv-tlfs.h |  69 ++++++++++++++
 arch/arm64/include/asm/mshyperv.h    |  72 ++++++++++++++
 arch/arm64/kernel/efi.c              |   1 +
 arch/arm64/kernel/setup.c            |   4 +
 drivers/clocksource/hyperv_timer.c   |  14 +++
 drivers/hv/Kconfig                   |   3 +-
 include/linux/arm-smccc.h            |  29 ++++--
 12 files changed, 542 insertions(+), 7 deletions(-)
 create mode 100644 arch/arm64/hyperv/Makefile
 create mode 100644 arch/arm64/hyperv/hv_core.c
 create mode 100644 arch/arm64/hyperv/mshyperv.c
 create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
 create mode 100644 arch/arm64/include/asm/mshyperv.h

-- 
1.8.3.1

[PATCH v9 1/7] smccc: Add HVC call variant with result registers other than 0 thru 3

[Michael Kelley]

Hypercalls to Hyper-V on ARM64 may return results in registers other
than X0 thru X3, as permitted by the SMCCC spec version 1.2 and later.
Accommodate this by adding a variant of arm_smccc_1_1_hvc that allows
the caller to specify which 3 registers are returned in addition to X0.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
---
There are several ways to support returning results from registers
other than X0 thru X3, and Hyper-V usage should be compatible with
whatever the maintainers prefer.  What's implemented in this patch
may be the most flexible, but it has the downside of not being a
true function interface in that args 0 thru 2 must be fixed strings,
and not general "C" expressions.

Other alternatives include:
* Create a variant that hard codes to return X5 thru X7, though
  in the future there may be Hyper-V hypercalls that need a
  different hard-coded variant.
* Return all of X0 thru X7 in a larger result structure. That
  approach may execute more memory stores, but performance is unlikely
  to be an issue for the Hyper-V hypercalls that would use it.
  However, it's possible in the future that Hyper-V results might
  be beyond X7, as allowed by the SMCCC v1.3 spec.
* The macro __arm_smccc_1_1() could be cloned in Hyper-V specific
  code and modified to meet Hyper-V specific needs, but this seems
  undesirable since Hyper-V is operating within the v1.2 spec.

In any of these cases, the call might be renamed from "_1_1_" to
"_1_2_" to reflect conformance to the later spec version.


 include/linux/arm-smccc.h | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
index f860645..acda958 100644
--- a/include/linux/arm-smccc.h
+++ b/include/linux/arm-smccc.h
@@ -300,12 +300,12 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
  * entitled to optimise the whole sequence away. "volatile" is what
  * makes it stick.
  */
-#define __arm_smccc_1_1(inst, ...)					\
+#define __arm_smccc_1_1(inst, reg1, reg2, reg3, ...)			\
 	do {								\
 		register unsigned long r0 asm("r0");			\
-		register unsigned long r1 asm("r1");			\
-		register unsigned long r2 asm("r2");			\
-		register unsigned long r3 asm("r3"); 			\
+		register unsigned long r1 asm(reg1);			\
+		register unsigned long r2 asm(reg2);			\
+		register unsigned long r3 asm(reg3);			\
 		__declare_args(__count_args(__VA_ARGS__), __VA_ARGS__);	\
 		asm volatile(inst "\n" :				\
 			     "=r" (r0), "=r" (r1), "=r" (r2), "=r" (r3)	\
@@ -328,7 +328,8 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
  * to the SMC instruction. The return values are updated with the content
  * from register 0 to 3 on return from the SMC instruction if not NULL.
  */
-#define arm_smccc_1_1_smc(...)	__arm_smccc_1_1(SMCCC_SMC_INST, __VA_ARGS__)
+#define arm_smccc_1_1_smc(...)\
+	__arm_smccc_1_1(SMCCC_SMC_INST, "r1", "r2", "r3", __VA_ARGS__)
 
 /*
  * arm_smccc_1_1_hvc() - make an SMCCC v1.1 compliant HVC call
@@ -344,7 +345,23 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
  * to the HVC instruction. The return values are updated with the content
  * from register 0 to 3 on return from the HVC instruction if not NULL.
  */
-#define arm_smccc_1_1_hvc(...)	__arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__)
+#define arm_smccc_1_1_hvc(...) \
+	__arm_smccc_1_1(SMCCC_HVC_INST, "r1", "r2", "r3", __VA_ARGS__)
+
+/*
+ * arm_smccc_1_1_hvc_reg() - make an SMCCC v1.1 compliant HVC call
+ * specifying output registers
+ *
+ * This is a variant of arm_smccc_1_1_hvc() that allows specifying
+ * three registers from which result values will be returned in
+ * addition to r0.
+ *
+ * @a0-a2: register specifications for 3 return registers (e.g., "r5")
+ * @a3-a10: arguments passed in registers 0 to 7
+ * @res: result values from register 0 and the three registers specified
+ * in a0-a2.
+ */
+#define arm_smccc_1_1_hvc_reg(...) __arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__)
 
 /*
  * Like arm_smccc_1_1* but always returns SMCCC_RET_NOT_SUPPORTED.
-- 
1.8.3.1

[PATCH v9 2/7] arm64: hyperv: Add Hyper-V hypercall and register access utilities

[Michael Kelley]

hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level
Functional Spec (TLFS), and #includes the architecture-independent
part of hyperv-tlfs.h in include/asm-generic.  The published TLFS
is distinctly oriented to x86/x64, so the ARM64-specific
hyperv-tlfs.h includes information for ARM64 that is not yet formally
published. The TLFS is available here:

  docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs

mshyperv.h defines Linux-specific structures and routines for
interacting with Hyper-V on ARM64, and #includes the architecture-
independent part of mshyperv.h in include/asm-generic.

Use these definitions to provide utility functions to make
Hyper-V hypercalls and to get and set Hyper-V provided
registers associated with a virtual processor.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
---
 MAINTAINERS                          |   3 +
 arch/arm64/Kbuild                    |   1 +
 arch/arm64/hyperv/Makefile           |   2 +
 arch/arm64/hyperv/hv_core.c          | 126 +++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/hyperv-tlfs.h |  69 +++++++++++++++++++
 arch/arm64/include/asm/mshyperv.h    |  54 +++++++++++++++
 6 files changed, 255 insertions(+)
 create mode 100644 arch/arm64/hyperv/Makefile
 create mode 100644 arch/arm64/hyperv/hv_core.c
 create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
 create mode 100644 arch/arm64/include/asm/mshyperv.h

diff --git a/MAINTAINERS b/MAINTAINERS
index bfc1b86..7a47a90 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8232,6 +8232,9 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
 F:	Documentation/ABI/stable/sysfs-bus-vmbus
 F:	Documentation/ABI/testing/debugfs-hyperv
 F:	Documentation/networking/device_drivers/ethernet/microsoft/netvsc.rst
+F:	arch/arm64/hyperv
+F:	arch/arm64/include/asm/hyperv-tlfs.h
+F:	arch/arm64/include/asm/mshyperv.h
 F:	arch/x86/hyperv
 F:	arch/x86/include/asm/hyperv-tlfs.h
 F:	arch/x86/include/asm/mshyperv.h
diff --git a/arch/arm64/Kbuild b/arch/arm64/Kbuild
index d646582..7a37608 100644
--- a/arch/arm64/Kbuild
+++ b/arch/arm64/Kbuild
@@ -3,4 +3,5 @@ obj-y			+= kernel/ mm/
 obj-$(CONFIG_NET)	+= net/
 obj-$(CONFIG_KVM)	+= kvm/
 obj-$(CONFIG_XEN)	+= xen/
+obj-$(subst m,y,$(CONFIG_HYPERV))	+= hyperv/
 obj-$(CONFIG_CRYPTO)	+= crypto/
diff --git a/arch/arm64/hyperv/Makefile b/arch/arm64/hyperv/Makefile
new file mode 100644
index 0000000..1697d30
--- /dev/null
+++ b/arch/arm64/hyperv/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-y		:= hv_core.o
diff --git a/arch/arm64/hyperv/hv_core.c b/arch/arm64/hyperv/hv_core.c
new file mode 100644
index 0000000..d5a8a6e
--- /dev/null
+++ b/arch/arm64/hyperv/hv_core.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Low level utility routines for interacting with Hyper-V.
+ *
+ * Copyright (C) 2021, Microsoft, Inc.
+ *
+ * Author : Michael Kelley <mikelley@microsoft.com>
+ */
+
+#include <linux/types.h>
+#include <linux/export.h>
+#include <linux/mm.h>
+#include <linux/hyperv.h>
+#include <linux/arm-smccc.h>
+#include <linux/module.h>
+#include <asm-generic/bug.h>
+#include <asm/hyperv-tlfs.h>
+#include <asm/mshyperv.h>
+
+/*
+ * hv_do_hypercall- Invoke the specified hypercall
+ */
+u64 hv_do_hypercall(u64 control, void *input, void *output)
+{
+	struct arm_smccc_res	res;
+	u64			input_address;
+	u64			output_address;
+
+	input_address = input ? virt_to_phys(input) : 0;
+	output_address = output ? virt_to_phys(output) : 0;
+
+	arm_smccc_1_1_hvc(HV_FUNC_ID, control,
+			  input_address, output_address, &res);
+	return res.a0;
+}
+EXPORT_SYMBOL_GPL(hv_do_hypercall);
+
+/*
+ * hv_do_fast_hypercall8 -- Invoke the specified hypercall
+ * with arguments in registers instead of physical memory.
+ * Avoids the overhead of virt_to_phys for simple hypercalls.
+ */
+
+u64 hv_do_fast_hypercall8(u16 code, u64 input)
+{
+	struct arm_smccc_res	res;
+	u64			control;
+
+	control = (u64)code | HV_HYPERCALL_FAST_BIT;
+
+	arm_smccc_1_1_hvc(HV_FUNC_ID, control, input, &res);
+	return res.a0;
+}
+EXPORT_SYMBOL_GPL(hv_do_fast_hypercall8);
+
+/*
+ * Set a single VP register to a 64-bit value.
+ */
+void hv_set_vpreg(u32 msr, u64 value)
+{
+	struct arm_smccc_res res;
+
+	arm_smccc_1_1_hvc(
+		HV_FUNC_ID,
+		HVCALL_SET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT |
+			HV_HYPERCALL_REP_COMP_1,
+		HV_PARTITION_ID_SELF,
+		HV_VP_INDEX_SELF,
+		msr,
+		0,
+		value,
+		0,
+		&res);
+
+	/*
+	 * Something is fundamentally broken in the hypervisor if
+	 * setting a VP register fails. There's really no way to
+	 * continue as a guest VM, so panic.
+	 */
+	BUG_ON((res.a0 & HV_HYPERCALL_RESULT_MASK) != HV_STATUS_SUCCESS);
+}
+EXPORT_SYMBOL_GPL(hv_set_vpreg);
+
+/*
+ * Get the value of a single VP register.  One version
+ * returns just 64 bits and another returns the full 128 bits.
+ * The two versions are separate to avoid complicating the
+ * calling sequence for the more frequently used 64 bit version.
+ */
+
+void hv_get_vpreg_128(u32 msr, struct hv_get_vp_registers_output *result)
+{
+	struct arm_smccc_res res;
+
+	arm_smccc_1_1_hvc_reg("r5", "r6", "r7",
+		HV_FUNC_ID,
+		HVCALL_GET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT |
+			HV_HYPERCALL_REP_COMP_1,
+		HV_PARTITION_ID_SELF,
+		HV_VP_INDEX_SELF,
+		msr,
+		0,
+		&res);
+
+	/*
+	 * Something is fundamentally broken in the hypervisor if
+	 * getting a VP register fails. There's really no way to
+	 * continue as a guest VM, so panic.
+	 */
+	BUG_ON((res.a0 & HV_HYPERCALL_RESULT_MASK) != HV_STATUS_SUCCESS);
+
+	result->as64.low = res.a2;  /* r6 */
+	result->as64.high = res.a3; /* r7 */
+}
+EXPORT_SYMBOL_GPL(hv_get_vpreg_128);
+
+u64 hv_get_vpreg(u32 msr)
+{
+	struct hv_get_vp_registers_output output;
+
+	hv_get_vpreg_128(msr, &output);
+
+	return output.as64.low;
+}
+EXPORT_SYMBOL_GPL(hv_get_vpreg);
diff --git a/arch/arm64/include/asm/hyperv-tlfs.h b/arch/arm64/include/asm/hyperv-tlfs.h
new file mode 100644
index 0000000..4d964a7
--- /dev/null
+++ b/arch/arm64/include/asm/hyperv-tlfs.h
@@ -0,0 +1,69 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * This file contains definitions from the Hyper-V Hypervisor Top-Level
+ * Functional Specification (TLFS):
+ * https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs
+ *
+ * Copyright (C) 2021, Microsoft, Inc.
+ *
+ * Author : Michael Kelley <mikelley@microsoft.com>
+ */
+
+#ifndef _ASM_HYPERV_TLFS_H
+#define _ASM_HYPERV_TLFS_H
+
+#include <linux/types.h>
+
+/*
+ * All data structures defined in the TLFS that are shared between Hyper-V
+ * and a guest VM use Little Endian byte ordering.  This matches the default
+ * byte ordering of Linux running on ARM64, so no special handling is required.
+ */
+
+/*
+ * These Hyper-V registers provide information equivalent to the CPUID
+ * instruction on x86/x64.
+ */
+#define HV_REGISTER_HYPERVISOR_VERSION		0x00000100 /*CPUID 0x40000002 */
+#define HV_REGISTER_FEATURES			0x00000200 /*CPUID 0x40000003 */
+#define HV_REGISTER_ENLIGHTENMENTS		0x00000201 /*CPUID 0x40000004 */
+
+/*
+ * Group C Features. See the asm-generic version of hyperv-tlfs.h
+ * for a description of Feature Groups.
+ */
+
+/* Crash MSRs available */
+#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE	BIT(8)
+
+/* STIMER direct mode is available */
+#define HV_STIMER_DIRECT_MODE_AVAILABLE		BIT(13)
+
+/*
+ * Synthetic register definitions equivalent to MSRs on x86/x64
+ */
+#define HV_REGISTER_CRASH_P0		0x00000210
+#define HV_REGISTER_CRASH_P1		0x00000211
+#define HV_REGISTER_CRASH_P2		0x00000212
+#define HV_REGISTER_CRASH_P3		0x00000213
+#define HV_REGISTER_CRASH_P4		0x00000214
+#define HV_REGISTER_CRASH_CTL		0x00000215
+
+#define HV_REGISTER_GUEST_OSID		0x00090002
+#define HV_REGISTER_VP_INDEX		0x00090003
+#define HV_REGISTER_TIME_REF_COUNT	0x00090004
+#define HV_REGISTER_REFERENCE_TSC	0x00090017
+
+#define HV_REGISTER_SINT0		0x000A0000
+#define HV_REGISTER_SCONTROL		0x000A0010
+#define HV_REGISTER_SIEFP		0x000A0012
+#define HV_REGISTER_SIMP		0x000A0013
+#define HV_REGISTER_EOM			0x000A0014
+
+#define HV_REGISTER_STIMER0_CONFIG	0x000B0000
+#define HV_REGISTER_STIMER0_COUNT	0x000B0001
+
+#include <asm-generic/hyperv-tlfs.h>
+
+#endif
diff --git a/arch/arm64/include/asm/mshyperv.h b/arch/arm64/include/asm/mshyperv.h
new file mode 100644
index 0000000..c448704
--- /dev/null
+++ b/arch/arm64/include/asm/mshyperv.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Linux-specific definitions for managing interactions with Microsoft's
+ * Hyper-V hypervisor. The definitions in this file are specific to
+ * the ARM64 architecture.  See include/asm-generic/mshyperv.h for
+ * definitions are that architecture independent.
+ *
+ * Definitions that are specified in the Hyper-V Top Level Functional
+ * Spec (TLFS) should not go in this file, but should instead go in
+ * hyperv-tlfs.h.
+ *
+ * Copyright (C) 2021, Microsoft, Inc.
+ *
+ * Author : Michael Kelley <mikelley@microsoft.com>
+ */
+
+#ifndef _ASM_MSHYPERV_H
+#define _ASM_MSHYPERV_H
+
+#include <linux/types.h>
+#include <linux/arm-smccc.h>
+#include <asm/hyperv-tlfs.h>
+
+/*
+ * Declare calls to get and set Hyper-V VP register values on ARM64, which
+ * requires a hypercall.
+ */
+
+extern void hv_set_vpreg(u32 reg, u64 value);
+extern u64 hv_get_vpreg(u32 reg);
+extern void hv_get_vpreg_128(u32 reg, struct hv_get_vp_registers_output *result);
+
+static inline void hv_set_register(unsigned int reg, u64 value)
+{
+	hv_set_vpreg(reg, value);
+}
+
+static inline u64 hv_get_register(unsigned int reg)
+{
+	return hv_get_vpreg(reg);
+}
+
+/* SMCCC hypercall parameters */
+#define HV_SMCCC_FUNC_NUMBER	1
+#define HV_FUNC_ID	ARM_SMCCC_CALL_VAL(			\
+				ARM_SMCCC_STD_CALL,		\
+				ARM_SMCCC_SMC_64,		\
+				ARM_SMCCC_OWNER_VENDOR_HYP,	\
+				HV_SMCCC_FUNC_NUMBER)
+
+#include <asm-generic/mshyperv.h>
+
+#endif
-- 
1.8.3.1

[PATCH v9 3/7] arm64: hyperv: Add Hyper-V clocksource/clockevent support

[Michael Kelley]

Add architecture specific definitions and functions needed
by the architecture independent Hyper-V clocksource driver.
Update the Hyper-V clocksource driver to be initialized
on ARM64.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
---
 arch/arm64/include/asm/mshyperv.h  | 12 ++++++++++++
 drivers/clocksource/hyperv_timer.c | 14 ++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/arch/arm64/include/asm/mshyperv.h b/arch/arm64/include/asm/mshyperv.h
index c448704..b17299c 100644
--- a/arch/arm64/include/asm/mshyperv.h
+++ b/arch/arm64/include/asm/mshyperv.h
@@ -21,6 +21,7 @@
 #include <linux/types.h>
 #include <linux/arm-smccc.h>
 #include <asm/hyperv-tlfs.h>
+#include <clocksource/arm_arch_timer.h>
 
 /*
  * Declare calls to get and set Hyper-V VP register values on ARM64, which
@@ -41,6 +42,17 @@ static inline u64 hv_get_register(unsigned int reg)
 	return hv_get_vpreg(reg);
 }
 
+/* Define the interrupt ID used by STIMER0 Direct Mode interrupts. This
+ * value can't come from ACPI tables because it is needed before the
+ * Linux ACPI subsystem is initialized.
+ */
+#define HYPERV_STIMER0_VECTOR	31
+
+static inline u64 hv_get_raw_timer(void)
+{
+	return arch_timer_read_counter();
+}
+
 /* SMCCC hypercall parameters */
 #define HV_SMCCC_FUNC_NUMBER	1
 #define HV_FUNC_ID	ARM_SMCCC_CALL_VAL(			\
diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
index ba04cb3..044efb1 100644
--- a/drivers/clocksource/hyperv_timer.c
+++ b/drivers/clocksource/hyperv_timer.c
@@ -473,3 +473,17 @@ void __init hv_init_clocksource(void)
 	hv_setup_sched_clock(read_hv_sched_clock_msr);
 }
 EXPORT_SYMBOL_GPL(hv_init_clocksource);
+
+/* Initialize everything on ARM64 */
+static int __init hyperv_timer_init(struct acpi_table_header *table)
+{
+	if (!hv_is_hyperv_initialized())
+		return -EINVAL;
+
+	hv_init_clocksource();
+	if (hv_stimer_alloc(true))
+		return -EINVAL;
+
+	return 0;
+}
+TIMER_ACPI_DECLARE(hyperv, ACPI_SIG_GTDT, hyperv_timer_init);
-- 
1.8.3.1

[PATCH v9 4/7] arm64: hyperv: Add kexec and panic handlers

[Michael Kelley]

Add function to inform Hyper-V about a guest panic.

Also add functions to set up and remove kexec and panic
handlers, which are currently unused on ARM64 but are
called from architecture independent code in the VMbus
driver.

This code is built only when CONFIG_HYPERV is enabled.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
---
 arch/arm64/hyperv/Makefile   |  2 +-
 arch/arm64/hyperv/hv_core.c  | 52 ++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/hyperv/mshyperv.c | 54 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 107 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/hyperv/mshyperv.c

diff --git a/arch/arm64/hyperv/Makefile b/arch/arm64/hyperv/Makefile
index 1697d30..87c31c0 100644
--- a/arch/arm64/hyperv/Makefile
+++ b/arch/arm64/hyperv/Makefile
@@ -1,2 +1,2 @@
 # SPDX-License-Identifier: GPL-2.0
-obj-y		:= hv_core.o
+obj-y		:= hv_core.o mshyperv.o
diff --git a/arch/arm64/hyperv/hv_core.c b/arch/arm64/hyperv/hv_core.c
index d5a8a6e..fdcd69f 100644
--- a/arch/arm64/hyperv/hv_core.c
+++ b/arch/arm64/hyperv/hv_core.c
@@ -124,3 +124,55 @@ u64 hv_get_vpreg(u32 msr)
 	return output.as64.low;
 }
 EXPORT_SYMBOL_GPL(hv_get_vpreg);
+
+/*
+ * hyperv_report_panic - report a panic to Hyper-V.  This function uses
+ * the older version of the Hyper-V interface that admittedly doesn't
+ * pass enough information to be useful beyond just recording the
+ * occurrence of a panic. The parallel hv_kmsg_dump() uses the
+ * new interface that allows reporting 4 Kbytes of data, which is much
+ * more useful. Hyper-V on ARM64 always supports the newer interface, but
+ * we retain support for the older version because the sysadmin is allowed
+ * to disable the newer version via sysctl in case of information security
+ * concerns about the more verbose version.
+ */
+void hyperv_report_panic(struct pt_regs *regs, long err, bool in_die)
+{
+	static bool	panic_reported;
+	u64		guest_id;
+
+	/* Don't report a panic to Hyper-V if we're not going to panic */
+	if (in_die && !panic_on_oops)
+		return;
+
+	/*
+	 * We prefer to report panic on 'die' chain as we have proper
+	 * registers to report, but if we miss it (e.g. on BUG()) we need
+	 * to report it on 'panic'.
+	 *
+	 * Calling code in the 'die' and 'panic' paths ensures that only
+	 * one CPU is running this code, so no atomicity is needed.
+	 */
+	if (panic_reported)
+		return;
+	panic_reported = true;
+
+	guest_id = hv_get_vpreg(HV_REGISTER_GUEST_OSID);
+
+	/*
+	 * Hyper-V provides the ability to store only 5 values.
+	 * Pick the passed in error value, the guest_id, and the PC.
+	 * The first two general registers are added arbitrarily.
+	 */
+	hv_set_vpreg(HV_REGISTER_CRASH_P0, err);
+	hv_set_vpreg(HV_REGISTER_CRASH_P1, guest_id);
+	hv_set_vpreg(HV_REGISTER_CRASH_P2, regs->pc);
+	hv_set_vpreg(HV_REGISTER_CRASH_P3, regs->regs[0]);
+	hv_set_vpreg(HV_REGISTER_CRASH_P4, regs->regs[1]);
+
+	/*
+	 * Let Hyper-V know there is crash data available
+	 */
+	hv_set_vpreg(HV_REGISTER_CRASH_CTL, HV_CRASH_CTL_CRASH_NOTIFY);
+}
+EXPORT_SYMBOL_GPL(hyperv_report_panic);
diff --git a/arch/arm64/hyperv/mshyperv.c b/arch/arm64/hyperv/mshyperv.c
new file mode 100644
index 0000000..d202b4c
--- /dev/null
+++ b/arch/arm64/hyperv/mshyperv.c
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Core routines for interacting with Microsoft's Hyper-V hypervisor.
+ * Includes hypervisor initialization, and handling of crashes and
+ * kexecs through a set of static "handler" variables set by the
+ * architecture independent VMbus driver.
+ *
+ * Copyright (C) 2021, Microsoft, Inc.
+ *
+ * Author : Michael Kelley <mikelley@microsoft.com>
+ */
+
+#include <linux/types.h>
+#include <linux/export.h>
+#include <linux/ptrace.h>
+
+/*
+ * The VMbus handler functions are no-ops on ARM64 because
+ * VMbus interrupts are handled as percpu IRQs.
+ */
+void hv_setup_vmbus_handler(void (*handler)(void))
+{
+}
+EXPORT_SYMBOL_GPL(hv_setup_vmbus_handler);
+
+void hv_remove_vmbus_handler(void)
+{
+}
+EXPORT_SYMBOL_GPL(hv_remove_vmbus_handler);
+
+/*
+ * The kexec and crash handler functions are
+ * currently no-ops on ARM64.
+ */
+void hv_setup_kexec_handler(void (*handler)(void))
+{
+}
+EXPORT_SYMBOL_GPL(hv_setup_kexec_handler);
+
+void hv_remove_kexec_handler(void)
+{
+}
+EXPORT_SYMBOL_GPL(hv_remove_kexec_handler);
+
+void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs))
+{
+}
+EXPORT_SYMBOL_GPL(hv_setup_crash_handler);
+
+void hv_remove_crash_handler(void)
+{
+}
+EXPORT_SYMBOL_GPL(hv_remove_crash_handler);
-- 
1.8.3.1

[PATCH v9 5/7] arm64: hyperv: Initialize hypervisor on boot

[Michael Kelley]

Add ARM64-specific code to initialize the Hyper-V
hypervisor when booting as a guest VM. Provide functions
and data structures indicating hypervisor status that
are needed by VMbus driver.

This code is built only when CONFIG_HYPERV is enabled.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
---
 arch/arm64/hyperv/mshyperv.c      | 119 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/mshyperv.h |   6 ++
 arch/arm64/kernel/setup.c         |   4 ++
 3 files changed, 129 insertions(+)

diff --git a/arch/arm64/hyperv/mshyperv.c b/arch/arm64/hyperv/mshyperv.c
index d202b4c..125e83e 100644
--- a/arch/arm64/hyperv/mshyperv.c
+++ b/arch/arm64/hyperv/mshyperv.c
@@ -14,6 +14,125 @@
 #include <linux/types.h>
 #include <linux/export.h>
 #include <linux/ptrace.h>
+#include <linux/errno.h>
+#include <linux/acpi.h>
+#include <linux/version.h>
+#include <linux/cpuhotplug.h>
+#include <linux/slab.h>
+#include <linux/cpumask.h>
+#include <asm/mshyperv.h>
+
+static bool		hyperv_initialized;
+struct ms_hyperv_info	ms_hyperv __ro_after_init;
+EXPORT_SYMBOL_GPL(ms_hyperv);
+
+u32	*hv_vp_index;
+EXPORT_SYMBOL_GPL(hv_vp_index);
+
+u32	hv_max_vp_index;
+EXPORT_SYMBOL_GPL(hv_max_vp_index);
+
+static int hv_cpu_init(unsigned int cpu)
+{
+	hv_vp_index[cpu] = hv_get_vpreg(HV_REGISTER_VP_INDEX);
+	return 0;
+}
+
+void __init hyperv_early_init(void)
+{
+	struct hv_get_vp_registers_output	result;
+	u32	a, b, c, d;
+	u64	guest_id;
+
+	/*
+	 * If we're in a VM on Hyper-V, the ACPI hypervisor_id field will
+	 * have the string "MsHyperV".
+	 */
+	if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8))
+		return;
+
+	/* Setup the guest ID */
+	guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0);
+	hv_set_vpreg(HV_REGISTER_GUEST_OSID, guest_id);
+
+	/* Get the features and hints from Hyper-V */
+	hv_get_vpreg_128(HV_REGISTER_FEATURES, &result);
+	ms_hyperv.features = result.as32.a;
+	ms_hyperv.misc_features = result.as32.c;
+
+	hv_get_vpreg_128(HV_REGISTER_ENLIGHTENMENTS, &result);
+	ms_hyperv.hints = result.as32.a;
+
+	pr_info("Hyper-V: Features 0x%x, hints 0x%x, misc 0x%x\n",
+		ms_hyperv.features, ms_hyperv.hints, ms_hyperv.misc_features);
+
+	/*
+	 * If Hyper-V has crash notifications, set crash_kexec_post_notifiers
+	 * so that we will report the panic to Hyper-V before running kdump.
+	 */
+	if (ms_hyperv.misc_features & HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE)
+		crash_kexec_post_notifiers = true;
+
+	/* Get information about the Hyper-V host version */
+	hv_get_vpreg_128(HV_REGISTER_HYPERVISOR_VERSION, &result);
+	a = result.as32.a;
+	b = result.as32.b;
+	c = result.as32.c;
+	d = result.as32.d;
+	pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n",
+		b >> 16, b & 0xFFFF, a,	d & 0xFFFFFF, c, d >> 24);
+
+	hyperv_initialized = true;
+}
+
+static int __init hyperv_init(void)
+{
+	int	i;
+
+	/* Allocate and initialize percpu VP index array */
+	hv_max_vp_index = num_possible_cpus();
+	hv_vp_index = kmalloc_array(hv_max_vp_index, sizeof(*hv_vp_index),
+				    GFP_KERNEL);
+	if (!hv_vp_index) {
+		hv_max_vp_index = 0;
+		return -ENOMEM;
+	}
+
+	for (i = 0; i < hv_max_vp_index; i++)
+		hv_vp_index[i] = VP_INVAL;
+
+	if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "arm64/hyperv_init:online",
+					hv_cpu_init, NULL) < 0) {
+		hv_max_vp_index = 0;
+		kfree(hv_vp_index);
+		hv_vp_index = NULL;
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+early_initcall(hyperv_init);
+
+/* This routine is called before kexec/kdump. It does required cleanup. */
+void hyperv_cleanup(void)
+{
+	hv_set_vpreg(HV_REGISTER_GUEST_OSID, 0);
+
+}
+EXPORT_SYMBOL_GPL(hyperv_cleanup);
+
+bool hv_is_hyperv_initialized(void)
+{
+	return hyperv_initialized;
+}
+EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized);
+
+bool hv_is_hibernation_supported(void)
+{
+	return false;
+}
+EXPORT_SYMBOL_GPL(hv_is_hibernation_supported);
 
 /*
  * The VMbus handler functions are no-ops on ARM64 because
diff --git a/arch/arm64/include/asm/mshyperv.h b/arch/arm64/include/asm/mshyperv.h
index b17299c..86ca5c5 100644
--- a/arch/arm64/include/asm/mshyperv.h
+++ b/arch/arm64/include/asm/mshyperv.h
@@ -23,6 +23,12 @@
 #include <asm/hyperv-tlfs.h>
 #include <clocksource/arm_arch_timer.h>
 
+#if IS_ENABLED(CONFIG_HYPERV)
+void __init hyperv_early_init(void);
+#else
+static inline void hyperv_early_init(void) {};
+#endif
+
 /*
  * Declare calls to get and set Hyper-V VP register values on ARM64, which
  * requires a hypercall.
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index c18aacd..1df8285 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -49,6 +49,7 @@
 #include <asm/traps.h>
 #include <asm/efi.h>
 #include <asm/xen/hypervisor.h>
+#include <asm/mshyperv.h>
 #include <asm/mmu_context.h>
 
 static int num_standard_resources;
@@ -340,6 +341,9 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
 	if (acpi_disabled)
 		unflatten_device_tree();
 
+	/* Do after acpi_boot_table_init() so local FADT is available */
+	hyperv_early_init();
+
 	bootmem_init();
 
 	kasan_init();
-- 
1.8.3.1

[PATCH v9 6/7] arm64: efi: Export screen_info

[Michael Kelley]

The Hyper-V frame buffer driver may be built as a module, and
it needs access to screen_info. So export screen_info.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/arm64/kernel/efi.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index fa02efb..e1be6c4 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -55,6 +55,7 @@ static __init pteval_t create_mapping_protection(efi_memory_desc_t *md)
 
 /* we will fill this structure from the stub, so don't put it in .bss */
 struct screen_info screen_info __section(".data");
+EXPORT_SYMBOL(screen_info);
 
 int __init efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md)
 {
-- 
1.8.3.1

[PATCH v9 7/7] Drivers: hv: Enable Hyper-V code to be built on ARM64

[Michael Kelley]

Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on
ARM64, causing the Hyper-V specific code to be built.

Signed-off-by: Michael Kelley <mikelley@microsoft.com>
Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
---
 drivers/hv/Kconfig | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig
index 79e5356..d492682 100644
--- a/drivers/hv/Kconfig
+++ b/drivers/hv/Kconfig
@@ -4,7 +4,8 @@ menu "Microsoft Hyper-V guest support"
 
 config HYPERV
 	tristate "Microsoft Hyper-V client drivers"
-	depends on X86 && ACPI && X86_LOCAL_APIC && HYPERVISOR_GUEST
+	depends on ACPI && ((X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) \
+		|| (ARM64 && !CPU_BIG_ENDIAN))
 	select PARAVIRT
 	select X86_HV_CALLBACK_VECTOR
 	help
-- 
1.8.3.1

RE: [PATCH v9 5/7] arm64: hyperv: Initialize hypervisor on boot

[Sunil Muthuswamy]

>  static int num_standard_resources;
> @@ -340,6 +341,9 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
>  	if (acpi_disabled)
>  		unflatten_device_tree();
> 
> +	/* Do after acpi_boot_table_init() so local FADT is available */
> +	hyperv_early_init();
> +
>  	bootmem_init();
> 
>  	kasan_init();
> --
> 1.8.3.1

Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com>

RE: [PATCH v9 2/7] arm64: hyperv: Add Hyper-V hypercall and register access utilities

[Sunil Muthuswamy]

> +static inline u64 hv_get_register(unsigned int reg)
> +{
> +	return hv_get_vpreg(reg);
> +}
> +
> +/* SMCCC hypercall parameters */
> +#define HV_SMCCC_FUNC_NUMBER	1
> +#define HV_FUNC_ID	ARM_SMCCC_CALL_VAL(			\
> +				ARM_SMCCC_STD_CALL,		\
> +				ARM_SMCCC_SMC_64,		\
> +				ARM_SMCCC_OWNER_VENDOR_HYP,	\
> +				HV_SMCCC_FUNC_NUMBER)
> +
> +#include <asm-generic/mshyperv.h>
> +
> +#endif
> --
> 1.8.3.1

Reviewed-by: Sunil Muthuswamy <sunilmut@microsoft.com>

RE: [PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64

[Michael Kelley]

From: Michael Kelley <mikelley@microsoft.com> Sent: Monday, March 8, 2021 11:57 AM
> 
> This series enables Linux guests running on Hyper-V on ARM64
> hardware. New ARM64-specific code in arch/arm64/hyperv initializes
> Hyper-V, including its interrupts and hypercall mechanism.
> Existing architecture independent drivers for Hyper-V's VMbus and
> synthetic devices just work when built for ARM64. Hyper-V code is
> built and included in the image and modules only if CONFIG_HYPERV
> is enabled.

ARM64 maintainers --

What are the prospects for getting your review and Ack on this patch set?
We're wanting to get the Hyper-V support on ARM64 finally accepted upstream.
Previous comments should be addressed in this revision, with perhaps a
remaining discussion point around the alternate SMCCC hypercall interface
in Patch 1 that makes use of changes in v1.2 of the SMCCC spec.  There are
several viable approaches that I've noted in the patch, depending on
your preferences.

Michael

> 
> The seven patches are organized as follows:
> 
> 1) Update include/linux/arm-smccc.h to provide an HVC wrapper
>    variant that returns results in other than X0 thru X3
> 
> 2) Add definitions and functions for making Hyper-V hypercalls
>    and getting/setting virtual processor registers provided by
>    Hyper-V
> 
> 3) Add architecture specific definitions needed by the
>    architecture independent Hyper-V clocksource driver in
>    drivers/clocksource/hyperv_timer.c. Update the clocksource
>    driver to be initialized on ARM64.
> 
> 4) Add functions needed by the arch independent VMbus driver
>    for reporting a panic to Hyper-V and as stubs for the kexec
>    and crash handlers.
> 
> 5) Add Hyper-V initialization code and utility functions that
>    report Hyper-v status.
> 
> 6) Export screen_info so it may be used by the Hyper-V frame buffer
>    driver built as a module. It is already exported for x86,
>    powerpc, and alpha architectures.
> 
> 7) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64.
> 
> Hyper-V on ARM64 runs with a 4 Kbyte page size, but allows guests
> with 4K/16K/64K page size. Linux guests with this ARM64 enablement
> code work with all three supported ARM64 page sizes.
> 
> The Hyper-V vPCI driver at drivers/pci/host/pci-hyperv.c has
> x86/x64-specific code and is not being built for ARM64. Fixing
> this driver to enable vPCI devices on ARM64 will be done later.
> 
> In a few cases, terminology from the x86/x64 world has been carried
> over into the ARM64 code ("MSR", "TSC").  Hyper-V still uses the
> x86/x64 terminology and has not replaced it with something more
> generic, so the code uses the Hyper-V terminology.  This will be
> fixed when Hyper-V updates the usage in the TLFS.
> 
> This patch set is based on the hyperv-next branch of the code tree
> https://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git/ 
> 
> Changes in v9:
> * Added Patch 1 to enable making an SMCCC compliant hypercall
>   that returns results in other than registers X0 thru X3, per
>   version 1.2 and later of the SMCCC spec.
> * Using the ability to return results in registers X6 and X7,
>   converted hv_get_vpreg_128() to use a "fast" hypercall that
>   passes inputs and outputs in registers, and in doing so eliminated
>   a lot of memory allocation complexity.
> * Cleaned up some extra blank lines and use of spaces in aligning
>   local variables. [Sunil Muthuswamy]
> * Based on discussion about future directions, reverted the
>   population of hv_vp_index array to use a cpuhp state instead
>   of a hypercall, which is like it was in v7 and earlier.
> 
> Changes in v8:
> * Removed a lot of code based on refactoring the boundary between
>   arch independent and arch dependent code for Hyper-V, per comments
>   from Arnd Bergmann. The removed code was either duplicated on
>   the x86 side, or has been folded into architecture independent
>   code as not really being architecture dependent.
> * Added config dependency on !CONFIG_CPU_BIG_ENDIAN [Arnd Bergmann]
> * Reworked the approach to Hyper-V initialization. The functionality
>   is the same, but is now structured like the Xen code with an early
>   init function called in setup_arch() and an early initcall to
>   finish the initialization. [Arnd Bergmann]
> 
> Changes in v7:
> * Separately upstreamed split of hyperv-tlfs.h into arch dependent
>   and independent versions.  In this patch set, update the ARM64
>   hyperv-tlfs.h to include architecture independent definitions.
>   This approach eliminates a lot of lines of otherwise duplicated
>   code on the ARM64 side.
> * Break ARM64 mshyperv.h into smaller pieces. Have an initial
>   baseline, and add code along with patches for a particular
>   functional area. [Marc Zyngier]
> * In mshyperv.h, use static inline functions instead of #defines
>   where possible. [Arnd Bergmann]
> * Use VMbus INTID obtained from ACPI DSDT instead of hardcoding.
>   The STIMER INTID is still hardcoded because it is needed
>   before Linux has initialized the ACPI subsystem, so it can't
>   be obtained from the DSDT.  Wedging it into the GTDT seems
>   dubious, so was not done. [Marc Zyngier]
> * Update Hyper-V page size allocation functions to use
>   alloc_page() if PAGE_SIZE == HV_HYP_PAGE_SIZE [Arnd
>   Bergmann]
> * Various other minor changes based on feedback and to rebase
>   to latest linux-next [Marc Zyngier and Arnd Bergmann]
> 
> Changes in v6:
> * Use SMCCC hypercall interface instead of direct invocation
>   of HVC instruction and the Hyper-V hypercall interface
>   [Marc Zyngier]
> * Reimplemented functions to alloc/free Hyper-V size pages
>   using kmalloc/kfree since kmalloc now guarantees alignment of
>   power of 2 size allocations [Marc Zyngier]
> * Export screen_info in arm64 architecture so it can be used
>   by the Hyper-V buffer driver built as a module
> * Renamed source file arch/arm64/hyperv/hv_init.c to hv_core.c
>   to better reflect its content
> * Fixed the bit position of certain feature flags presented by
>   Hyper-V to the guest.  The bit positions on ARM64 don't match
>   the position on x86 like originally thought.
> * Minor fixups to rebase to 5.6-rc5 linux-next
> 
> Changes in v5:
> * Minor fixups to rebase to 5.4-rc1 linux-next
> 
> Changes in v4:
> * Moved clock-related code into an architecture independent
>   Hyper-V clocksource driver that is already upstream. Clock
>   related code is removed from this patch set except for the
>   ARM64 specific interrupt handler. [Marc Zyngier]
> * Separately upstreamed the split of mshyperv.h into arch independent
>   and arch dependent portions. The arch independent portion has been
>   removed from this patch set.
> * Divided patch #2 of the series into multiple smaller patches
>   [Marc Zyngier]
> * Changed a dozen or so smaller things based on feedback
>   [Marc Zyngier, Will Deacon]
> * Added functions to alloc/free Hyper-V size pages for use by
>   drivers for Hyper-V synthetic devices when updated to not assume
>   guest page size and Hyper-v page size are the same
> 
> Changes in v3:
> * Added initialization of hv_vp_index array like was recently
>   added on x86 branch [KY Srinivasan]
> * Changed Hyper-V ARM64 register symbols to be all uppercase
>   instead of mixed case [KY Srinivasan]
> * Separated mshyperv.h into two files, one architecture
>   independent and one architecture dependent. After this code
>   is upstream, will make changes to the x86 code to use the
>   architecture independent file and remove duplication. And
>   once we have a multi-architecture Hyper-V TLFS, will do a
>   separate patch to split hyperv-tlfs.h in the same way.
>   [KY Srinivasan]
> * Minor tweaks to rebase to latest linux-next code
> 
> Changes in v2:
> * Removed patch to implement slow_virt_to_phys() on ARM64.
>   Use of slow_virt_to_phys() in arch independent Hyper-V
>   drivers has been eliminated by commit 6ba34171bcbd
>   ("Drivers: hv: vmbus: Remove use of slow_virt_to_phys()")
> * Minor tweaks to rebase to latest linux-next code
> *** BLURB HERE ***
> 
> Michael Kelley (7):
>   smccc: Add HVC call variant with result registers other than 0 thru 3
>   arm64: hyperv: Add Hyper-V hypercall and register access utilities
>   arm64: hyperv: Add Hyper-V clocksource/clockevent support
>   arm64: hyperv: Add kexec and panic handlers
>   arm64: hyperv: Initialize hypervisor on boot
>   arm64: efi: Export screen_info
>   Drivers: hv: Enable Hyper-V code to be built on ARM64
> 
>  MAINTAINERS                          |   3 +
>  arch/arm64/Kbuild                    |   1 +
>  arch/arm64/hyperv/Makefile           |   2 +
>  arch/arm64/hyperv/hv_core.c          | 178 +++++++++++++++++++++++++++++++++++
>  arch/arm64/hyperv/mshyperv.c         | 173 ++++++++++++++++++++++++++++++++++
>  arch/arm64/include/asm/hyperv-tlfs.h |  69 ++++++++++++++
>  arch/arm64/include/asm/mshyperv.h    |  72 ++++++++++++++
>  arch/arm64/kernel/efi.c              |   1 +
>  arch/arm64/kernel/setup.c            |   4 +
>  drivers/clocksource/hyperv_timer.c   |  14 +++
>  drivers/hv/Kconfig                   |   3 +-
>  include/linux/arm-smccc.h            |  29 ++++--
>  12 files changed, 542 insertions(+), 7 deletions(-)
>  create mode 100644 arch/arm64/hyperv/Makefile
>  create mode 100644 arch/arm64/hyperv/hv_core.c
>  create mode 100644 arch/arm64/hyperv/mshyperv.c
>  create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h
>  create mode 100644 arch/arm64/include/asm/mshyperv.h
> 
> --
> 1.8.3.1

Re: [PATCH v9 1/7] smccc: Add HVC call variant with result registers other than 0 thru 3

[Mark Rutland]

Hi Michael,

On Mon, Mar 08, 2021 at 11:57:13AM -0800, Michael Kelley wrote:
> Hypercalls to Hyper-V on ARM64 may return results in registers other
> than X0 thru X3, as permitted by the SMCCC spec version 1.2 and later.
> Accommodate this by adding a variant of arm_smccc_1_1_hvc that allows
> the caller to specify which 3 registers are returned in addition to X0.
> 
> Signed-off-by: Michael Kelley <mikelley@microsoft.com>
> ---
> There are several ways to support returning results from registers
> other than X0 thru X3, and Hyper-V usage should be compatible with
> whatever the maintainers prefer.  What's implemented in this patch
> may be the most flexible, but it has the downside of not being a
> true function interface in that args 0 thru 2 must be fixed strings,
> and not general "C" expressions.

For the benefit of others here, SMCCCv1.2 allows:

* SMC64/HVC64 to use all of x1-x17 for both parameters and return values
* SMC32/HVC32 to use all of r1-r7 for both parameters and return values

The rationale for this was to make it possible to pass a large number of
arguments in one call without the hypervisor/firmware needing to access
the memory of the caller.

My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
functions which read all the permitted argument registers from a struct,
and write all the permitted result registers to a struct, leaving it to
callers to set those up and decompose them.

That way we only have to write one implementation that all callers can
use, which'll be far easier to maintain. I suspect that in general the
cost of temporarily bouncing the values through memory will be dominated
by whatever the hypervisor/firmware is going to do, and if it's not we
can optimize that away in future.

> Other alternatives include:
> * Create a variant that hard codes to return X5 thru X7, though
>   in the future there may be Hyper-V hypercalls that need a
>   different hard-coded variant.
> * Return all of X0 thru X7 in a larger result structure. That
>   approach may execute more memory stores, but performance is unlikely
>   to be an issue for the Hyper-V hypercalls that would use it.
>   However, it's possible in the future that Hyper-V results might
>   be beyond X7, as allowed by the SMCCC v1.3 spec.

As above, something of this sort would be my preferred approach.

Thanks,
Mark.

> * The macro __arm_smccc_1_1() could be cloned in Hyper-V specific
>   code and modified to meet Hyper-V specific needs, but this seems
>   undesirable since Hyper-V is operating within the v1.2 spec.
> 
> In any of these cases, the call might be renamed from "_1_1_" to
> "_1_2_" to reflect conformance to the later spec version.
> 
> 
>  include/linux/arm-smccc.h | 29 +++++++++++++++++++++++------
>  1 file changed, 23 insertions(+), 6 deletions(-)
> 
> diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
> index f860645..acda958 100644
> --- a/include/linux/arm-smccc.h
> +++ b/include/linux/arm-smccc.h
> @@ -300,12 +300,12 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
>   * entitled to optimise the whole sequence away. "volatile" is what
>   * makes it stick.
>   */
> -#define __arm_smccc_1_1(inst, ...)					\
> +#define __arm_smccc_1_1(inst, reg1, reg2, reg3, ...)			\
>  	do {								\
>  		register unsigned long r0 asm("r0");			\
> -		register unsigned long r1 asm("r1");			\
> -		register unsigned long r2 asm("r2");			\
> -		register unsigned long r3 asm("r3"); 			\
> +		register unsigned long r1 asm(reg1);			\
> +		register unsigned long r2 asm(reg2);			\
> +		register unsigned long r3 asm(reg3);			\
>  		__declare_args(__count_args(__VA_ARGS__), __VA_ARGS__);	\
>  		asm volatile(inst "\n" :				\
>  			     "=r" (r0), "=r" (r1), "=r" (r2), "=r" (r3)	\
> @@ -328,7 +328,8 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
>   * to the SMC instruction. The return values are updated with the content
>   * from register 0 to 3 on return from the SMC instruction if not NULL.
>   */
> -#define arm_smccc_1_1_smc(...)	__arm_smccc_1_1(SMCCC_SMC_INST, __VA_ARGS__)
> +#define arm_smccc_1_1_smc(...)\
> +	__arm_smccc_1_1(SMCCC_SMC_INST, "r1", "r2", "r3", __VA_ARGS__)
>  
>  /*
>   * arm_smccc_1_1_hvc() - make an SMCCC v1.1 compliant HVC call
> @@ -344,7 +345,23 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
>   * to the HVC instruction. The return values are updated with the content
>   * from register 0 to 3 on return from the HVC instruction if not NULL.
>   */
> -#define arm_smccc_1_1_hvc(...)	__arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__)
> +#define arm_smccc_1_1_hvc(...) \
> +	__arm_smccc_1_1(SMCCC_HVC_INST, "r1", "r2", "r3", __VA_ARGS__)
> +
> +/*
> + * arm_smccc_1_1_hvc_reg() - make an SMCCC v1.1 compliant HVC call
> + * specifying output registers
> + *
> + * This is a variant of arm_smccc_1_1_hvc() that allows specifying
> + * three registers from which result values will be returned in
> + * addition to r0.
> + *
> + * @a0-a2: register specifications for 3 return registers (e.g., "r5")
> + * @a3-a10: arguments passed in registers 0 to 7
> + * @res: result values from register 0 and the three registers specified
> + * in a0-a2.
> + */
> +#define arm_smccc_1_1_hvc_reg(...) __arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__)
>  
>  /*
>   * Like arm_smccc_1_1* but always returns SMCCC_RET_NOT_SUPPORTED.
> -- 
> 1.8.3.1
> 

RE: [PATCH v9 1/7] smccc: Add HVC call variant with result registers other than 0 thru 3

[Michael Kelley]

From: Mark Rutland <mark.rutland@arm.com> Sent: Wednesday, March 24, 2021 9:55 AM
> 
> Hi Michael,
> 
> On Mon, Mar 08, 2021 at 11:57:13AM -0800, Michael Kelley wrote:
> > Hypercalls to Hyper-V on ARM64 may return results in registers other
> > than X0 thru X3, as permitted by the SMCCC spec version 1.2 and later.
> > Accommodate this by adding a variant of arm_smccc_1_1_hvc that allows
> > the caller to specify which 3 registers are returned in addition to X0.
> >
> > Signed-off-by: Michael Kelley <mikelley@microsoft.com>
> > ---
> > There are several ways to support returning results from registers
> > other than X0 thru X3, and Hyper-V usage should be compatible with
> > whatever the maintainers prefer.  What's implemented in this patch
> > may be the most flexible, but it has the downside of not being a
> > true function interface in that args 0 thru 2 must be fixed strings,
> > and not general "C" expressions.
> 
> For the benefit of others here, SMCCCv1.2 allows:
> 
> * SMC64/HVC64 to use all of x1-x17 for both parameters and return values
> * SMC32/HVC32 to use all of r1-r7 for both parameters and return values
> 
> The rationale for this was to make it possible to pass a large number of
> arguments in one call without the hypervisor/firmware needing to access
> the memory of the caller.
> 
> My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
> functions which read all the permitted argument registers from a struct,
> and write all the permitted result registers to a struct, leaving it to
> callers to set those up and decompose them.
> 
> That way we only have to write one implementation that all callers can
> use, which'll be far easier to maintain. I suspect that in general the
> cost of temporarily bouncing the values through memory will be dominated
> by whatever the hypervisor/firmware is going to do, and if it's not we
> can optimize that away in future.
> 

Thanks for the feedback, and I'm working on implementing this approach.
But I've hit a snag in that gcc limits the "asm" statement to 30 arguments,
which gives us 15 registers as parameters and 15 registers as return
values, instead of the 18 each allowed by SMCCC v1.2.  I will continue
with the 15 register limit for now, unless someone knows a way to exceed
that.  The alternative would be to go to pure assembly language.

I'll post a standalone RFC patch when I have something that works.  My
C pre-processor wizardry is limited, so others will probably know some
tricks that can improve on my first cut.

Michael

Re: [PATCH v9 1/7] smccc: Add HVC call variant with result registers other than 0 thru 3

[Mark Rutland]

On Thu, Mar 25, 2021 at 04:55:51AM +0000, Michael Kelley wrote:
> From: Mark Rutland <mark.rutland@arm.com> Sent: Wednesday, March 24, 2021 9:55 AM
> > For the benefit of others here, SMCCCv1.2 allows:
> > 
> > * SMC64/HVC64 to use all of x1-x17 for both parameters and return values
> > * SMC32/HVC32 to use all of r1-r7 for both parameters and return values
> > 
> > The rationale for this was to make it possible to pass a large number of
> > arguments in one call without the hypervisor/firmware needing to access
> > the memory of the caller.
> > 
> > My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
> > functions which read all the permitted argument registers from a struct,
> > and write all the permitted result registers to a struct, leaving it to
> > callers to set those up and decompose them.
> > 
> > That way we only have to write one implementation that all callers can
> > use, which'll be far easier to maintain. I suspect that in general the
> > cost of temporarily bouncing the values through memory will be dominated
> > by whatever the hypervisor/firmware is going to do, and if it's not we
> > can optimize that away in future.
> 
> Thanks for the feedback, and I'm working on implementing this approach.
> But I've hit a snag in that gcc limits the "asm" statement to 30 arguments,
> which gives us 15 registers as parameters and 15 registers as return
> values, instead of the 18 each allowed by SMCCC v1.2.  I will continue
> with the 15 register limit for now, unless someone knows a way to exceed
> that.  The alternative would be to go to pure assembly language.

I realise in retrospect this is not clear, but when I said "assembly
functions" I had meant raw assembly functions rather than inline
assembly.

We already have __arm_smccc_smc and __arm_smccc_hvc assembly functions
in arch/{arm,arm64}/kernel/smccc-call.S, and I'd expected we'd add the
full fat SMCCCv1.2 variants there.

Thanks,
Mark.

RE: [PATCH v9 1/7] smccc: Add HVC call variant with result registers other than 0 thru 3

[Michael Kelley]

From: Mark Rutland <mark.rutland@arm.com> Sent: Thursday, March 25, 2021 2:56 AM
> 
> On Thu, Mar 25, 2021 at 04:55:51AM +0000, Michael Kelley wrote:
> > From: Mark Rutland <mark.rutland@arm.com> Sent: Wednesday, March 24, 2021 9:55 AM
> > > For the benefit of others here, SMCCCv1.2 allows:
> > >
> > > * SMC64/HVC64 to use all of x1-x17 for both parameters and return values
> > > * SMC32/HVC32 to use all of r1-r7 for both parameters and return values
> > >
> > > The rationale for this was to make it possible to pass a large number of
> > > arguments in one call without the hypervisor/firmware needing to access
> > > the memory of the caller.
> > >
> > > My preference would be to add arm_smccc_1_2_{hvc,smc}() assembly
> > > functions which read all the permitted argument registers from a struct,
> > > and write all the permitted result registers to a struct, leaving it to
> > > callers to set those up and decompose them.
> > >
> > > That way we only have to write one implementation that all callers can
> > > use, which'll be far easier to maintain. I suspect that in general the
> > > cost of temporarily bouncing the values through memory will be dominated
> > > by whatever the hypervisor/firmware is going to do, and if it's not we
> > > can optimize that away in future.
> >
> > Thanks for the feedback, and I'm working on implementing this approach.
> > But I've hit a snag in that gcc limits the "asm" statement to 30 arguments,
> > which gives us 15 registers as parameters and 15 registers as return
> > values, instead of the 18 each allowed by SMCCC v1.2.  I will continue
> > with the 15 register limit for now, unless someone knows a way to exceed
> > that.  The alternative would be to go to pure assembly language.
> 
> I realise in retrospect this is not clear, but when I said "assembly
> functions" I had meant raw assembly functions rather than inline
> assembly.
> 
> We already have __arm_smccc_smc and __arm_smccc_hvc assembly functions
> in arch/{arm,arm64}/kernel/smccc-call.S, and I'd expected we'd add the
> full fat SMCCCv1.2 variants there.
> 

FWIW, here's an inline assembly version that I have working.  On the plus
side, gcc does a decent job of optimizing.  It doesn't store to memory any
result registers that aren't consumed by the caller.  On the downside, it's
limited to 15 args and 15 results as noted previously.  So it doesn't meet
your goal of fully implementing the v1.2 spec.  Also, all 15 input arguments
must be initialized or gcc complains about using uninitialized values.

This version should handle both the 32-bit and 64-bit worlds, though I've
only tested in the 64-bit world.

I've made the input and output structures be arrays rather than listing
each register as a separate field.  Either approach should work, and I'm not
sure what the tradeoffs are.

But if building on what Sudeep Holla has already done with raw assembly
is preferred, I'm OK with that as well.

Michael

diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h
index acda958..e98cf07 100644
--- a/include/linux/arm-smccc.h
+++ b/include/linux/arm-smccc.h
@@ -408,5 +408,112 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1,
 		method;							\
 	})
 
+
+#ifdef CONFIG_ARM64
+#define SMCCC_1_2_REG_COUNT	18
+
+#define ARM64_ADDITIONAL_RESULT_DECLARATIONS				\
+		register unsigned long r8 asm("r8");			\
+		register unsigned long r9 asm("r9");			\
+		register unsigned long r10 asm("r10");			\
+		register unsigned long r11 asm("r11");			\
+		register unsigned long r12 asm("r12");			\
+		register unsigned long r13 asm("r13");			\
+		register unsigned long r14 asm("r14");
+
+#define ARM64_ADDITIONAL_ARG_DECLARATIONS				\
+		register unsigned long arg8 asm("r8") = __a->reg[8];	\
+		register unsigned long arg9 asm("r9") = __a->reg[9];	\
+		register unsigned long arg10 asm("r10") = __a->reg[10];	\
+		register unsigned long arg11 asm("r11") = __a->reg[11];	\
+		register unsigned long arg12 asm("r12") = __a->reg[12];	\
+		register unsigned long arg13 asm("r13") = __a->reg[13];	\
+		register unsigned long arg14 asm("r14") = __a->reg[14];
+
+#define ARM64_ADDITIONAL_OUTPUT_REGS					\
+			, "=r" (r8), "=r" (r9), "=r" (r10), "=r" (r11),	\
+			"=r" (r12), "=r" (r13), "=r" (r14)
+
+#define ARM64_ADDITIONAL_INPUT_REGS					\
+			, "r" (arg8), "r" (arg9), "r" (arg10),		\
+			"r" (arg11), "r" (arg12), "r" (arg13),		\
+			"r" (arg14)
+
+#define ARM64_ADDITIONAL_RESULTS					\
+			__r->reg[8] = r8;				\
+			__r->reg[9] = r9;				\
+			__r->reg[10] = r10;				\
+			__r->reg[11] = r11;				\
+			__r->reg[12] = r12;				\
+			__r->reg[13] = r13;				\
+			__r->reg[14] = r14;
+
+#else /* CONFIG_ARM64 */
+
+#define SMCCC_1_2_REG_COUNT	8
+#define ARM64_ADDITIONAL_RESULT_DECLARATIONS
+#define ARM64_ADDITIONAL_ARG_DECLARATIONS
+#define ARM64_ADDITIONAL_OUTPUT_REGS
+#define ARM64_ADDITIONAL_INPUT_REGS
+#define ARM64_ADDITIONAL_RESULTS
+
+#endif /* CONFIG_ARM64 */
+
+struct arm_smccc_1_2_regs {
+	unsigned long reg[SMCCC_1_2_REG_COUNT];
+};
+
+
+#define __arm_smccc_1_2(inst, args, res)				\
+	do {								\
+		struct arm_smccc_1_2_regs * __a = args;			\
+		struct arm_smccc_1_2_regs * __r = res;			\
+		register unsigned long r0 asm("r0");			\
+		register unsigned long r1 asm("r1");			\
+		register unsigned long r2 asm("r2");			\
+		register unsigned long r3 asm("r3");			\
+		register unsigned long r4 asm("r4");			\
+		register unsigned long r5 asm("r5");			\
+		register unsigned long r6 asm("r6");			\
+		register unsigned long r7 asm("r7");			\
+		ARM64_ADDITIONAL_RESULT_DECLARATIONS			\
+		register unsigned long arg0 asm("r0") = (u32)(__a->reg[0]); \
+		register unsigned long arg1 asm("r1") = __a->reg[1];	\
+		register unsigned long arg2 asm("r2") = __a->reg[2];	\
+		register unsigned long arg3 asm("r3") = __a->reg[3];	\
+		register unsigned long arg4 asm("r4") = __a->reg[4];	\
+		register unsigned long arg5 asm("r5") = __a->reg[5];	\
+		register unsigned long arg6 asm("r6") = __a->reg[6];	\
+		register unsigned long arg7 asm("r7") = __a->reg[7];	\
+		ARM64_ADDITIONAL_ARG_DECLARATIONS			\
+		asm volatile(inst "\n" :				\
+			"=r" (r0), "=r" (r1), "=r" (r2), "=r" (r3),	\
+			"=r" (r4), "=r" (r5), "=r" (r6), "=r" (r7)	\
+			ARM64_ADDITIONAL_OUTPUT_REGS :			\
+			"r" (arg0), "r" (arg1), "r" (arg2),		\
+			"r" (arg3), "r" (arg4), "r" (arg5),		\
+			"r" (arg6), "r" (arg7)				\
+			ARM64_ADDITIONAL_INPUT_REGS :			\
+			"memory");					\
+		if(__r) {						\
+			__r->reg[0] = r0;				\
+			__r->reg[1] = r1;				\
+			__r->reg[2] = r2;				\
+			__r->reg[3] = r3;				\
+			__r->reg[4] = r4;				\
+			__r->reg[5] = r5;				\
+			__r->reg[6] = r6;				\
+			__r->reg[7] = r7;				\
+			ARM64_ADDITIONAL_RESULTS			\
+		}							\
+	} while (0)
+
+#define arm_smccc_1_2_smc(args, res)					\
+	__arm_smccc_1_2(SMCCC_SMC_INST, args, res)
+
+#define arm_smccc_1_2_hvc(args, res)					\
+	__arm_smccc_1_2(SMCCC_HVC_INST, args, res)
+
+
 #endif /*__ASSEMBLY__*/
 #endif /*__LINUX_ARM_SMCCC_H*/

RE: [PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64

[Michael Kelley]

From: Michael Kelley <mikelley@microsoft.com>
> Sent: Wednesday, March 24, 2021 8:55 AM
> To: will@kernel.org; catalin.marinas@arm.com; Mark Rutland <Mark.Rutland@arm.com>;
> lorenzo.pieralisi@arm.com; sudeep.holla@arm.com
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; linux-
> hyperv@vger.kernel.org; linux-efi@vger.kernel.org; arnd@arndb.de; wei.liu@kernel.org;
> ardb@kernel.org; daniel.lezcano@linaro.org; KY Srinivasan <kys@microsoft.com>
> Subject: RE: [PATCH v9 0/7] Enable Linux guests on Hyper-V on ARM64
> 
> From: Michael Kelley <mikelley@microsoft.com> Sent: Monday, March 8, 2021 11:57 AM
> >
> > This series enables Linux guests running on Hyper-V on ARM64
> > hardware. New ARM64-specific code in arch/arm64/hyperv initializes
> > Hyper-V, including its interrupts and hypercall mechanism.
> > Existing architecture independent drivers for Hyper-V's VMbus and
> > synthetic devices just work when built for ARM64. Hyper-V code is
> > built and included in the image and modules only if CONFIG_HYPERV
> > is enabled.
> 
> ARM64 maintainers --
> 
> What are the prospects for getting your review and Ack on this patch set?
> We're wanting to get the Hyper-V support on ARM64 finally accepted upstream.
> Previous comments should be addressed in this revision, with perhaps a
> remaining discussion point around the alternate SMCCC hypercall interface
> in Patch 1 that makes use of changes in v1.2 of the SMCCC spec.  There are
> several viable approaches that I've noted in the patch, depending on
> your preferences.
> 
> Michael

Thanks, Mark, for jumping in on the SMCCC hypercall interface.  But I'm still
looking for feedback or ACKs on the other patches in the series.  There's only
one place in Patch 2 of the series that needs the SMCCC v1.2 interface, and I'd
like to be able to respond to any remaining issues with the other patches
while the SMCCC details are finished up.

Michael