This patch series enables Linux guests running on Hyper-V on ARM64 hardware. New ARM64-specific code in arch/arm64/hyperv initializes Hyper-V, including its synthetic clocks and hypercall mechanism. Existing architecture independent drivers for Hyper-V's VMbus and synthetic devices just work when built for ARM64. Hyper-V code is built and included in the image and modules only if CONFIG_HYPERV is enabled. The five patches are organized as follows: 1) Implement slow_virt_to_phys() function for ARM64. This function parallels the same function for x86/x64, and is used by the Hyper-V synthetic network driver. 2) Add include files that define the Hyper-V interface as described in the Hyper-V Top Level Functional Spec (TLFS), plus additional definitions specific to Linux running on Hyper-V. 3) Add core Hyper-V support on ARM64, including hypercalls, synthetic clock initialization, and interrupt handlers. 4) Update the existing VMbus driver to generalize interrupt management across x86/x64 and ARM64. 5) Make CONFIG_HYPERV selectable on ARM64 in addition to x86/x64. Some areas of Linux guests on Hyper-V on ARM64 are a work- in-progress, primarily due to work still being done in Hyper-V: * Hyper-V on ARM64 currently runs with a 4 Kbyte page size, and only supports guests with a 4 Kbyte page size. Because Hyper-V uses shared pages to communicate between the guest and the hypervisor, there are open design decisions on the page size to use when the guest is using 16K/64K pages. Once those issues are resolved and Hyper-V fully supports 16K/64K guest pages, changes may be needed in the Linux drivers for Hyper-V synthetic devices. * Hyper-V on ARM64 does not currently support mapping PCI devices into the guest address space. The Hyper-V PCI driver at drivers/pci/host/pci-hyperv.c has x86/x64-specific code and is not being built for ARM64. In a few cases, terminology from the x86/x64 world has been carried over into the ARM64 code ("MSR", "TSC"). Hyper-V still uses the x86/x64 terminology and has not replaced it with something more generic, so the code uses the Hyper-V terminology. This will be fixed when Hyper-V updates the usage in the TLFS. Michael Kelley (5): arm64: mm: Add slow_virt_to_phys() arm64: hyperv: Add core Hyper-V include files arm64: hyperv: Add support for Hyper-V as a hypervisor Drivers: hv: vmbus: Add hooks for per-CPU IRQ Drivers: hv: Enable CONFIG_HYPERV on ARM64 MAINTAINERS | 3 + arch/arm64/Makefile | 1 + arch/arm64/hyperv/Makefile | 2 + arch/arm64/hyperv/hv_hvc.S | 54 +++++ arch/arm64/hyperv/hv_init.c | 411 +++++++++++++++++++++++++++++++++++ arch/arm64/hyperv/mshyperv.c | 178 +++++++++++++++ arch/arm64/include/asm/hyperv-tlfs.h | 353 ++++++++++++++++++++++++++++++ arch/arm64/include/asm/memory.h | 6 + arch/arm64/include/asm/mshyperv.h | 291 +++++++++++++++++++++++++ arch/arm64/mm/pageattr.c | 37 ++++ arch/x86/include/asm/mshyperv.h | 4 + drivers/hv/Kconfig | 3 +- drivers/hv/hv.c | 2 + 13 files changed, 1344 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/hyperv/Makefile create mode 100644 arch/arm64/hyperv/hv_hvc.S create mode 100644 arch/arm64/hyperv/hv_init.c create mode 100644 arch/arm64/hyperv/mshyperv.c create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h create mode 100644 arch/arm64/include/asm/mshyperv.h -- 1.8.3.1
Add slow_virt_to_phys() function for ARM64 that parallels the same function on x86/x64. This is needed by the architecture independent Hyper-V VMbus driver at drivers/hv/channel.c. The implementation directly translates the virtual address using the ARM64 'at' instruction. Signed-off-by: Michael Kelley <mikelley@microsoft.com> Reviewed-by: James Morris <james.morris@microsoft.com> --- arch/arm64/include/asm/memory.h | 6 ++++++ arch/arm64/mm/pageattr.c | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 49d9921..3f68dcf 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -265,6 +265,12 @@ static inline void *phys_to_virt(phys_addr_t x) } /* + * For memory where the underlying physical pages may not + * be contiguous. + */ +extern phys_addr_t slow_virt_to_phys(void *vaddr); + +/* * Drivers should NOT use these either. */ #define __pa(x) __virt_to_phys((unsigned long)(x)) diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index a563593..8a42cac 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -19,6 +19,8 @@ #include <asm/pgtable.h> #include <asm/set_memory.h> #include <asm/tlbflush.h> +#include <asm/sysreg.h> +#include <asm/barrier.h> struct page_change_data { pgprot_t set_mask; @@ -185,3 +187,38 @@ bool kernel_page_present(struct page *page) } #endif /* CONFIG_HIBERNATION */ #endif /* CONFIG_DEBUG_PAGEALLOC */ + +/* + * For virtual addresses where the underlyine physical memory may not be + * contiguous and the normal virt_to_phys gives the wrong result. This + * function does an actual translation using the 'at' instruction. + */ +phys_addr_t slow_virt_to_phys(void *virt_addr) +{ + phys_addr_t result; + unsigned long input = (unsigned long)virt_addr; + char touch; + int i; + + /* Try up to 3 times (an arbitrary number) */ + for (i = 0; i < 3; i++) { + /* Do the translation and check that it worked */ + asm volatile("at s1e1r, %0" : : "r" (input)); + isb(); + result = read_sysreg(par_el1); + if (likely(!(result & 0x1))) + return (result & GENMASK_ULL(51, 12)) | + (input & GENMASK_ULL(11, 0)); + /* + * Something failed. Read the page to fault in anything + * that isn't resident, then try again. "Anything" + * could include the page itself or hypervisor page tables. + */ + touch = READ_ONCE(*(char *)input); + dmb(sy); + } + + /* Let the caller sort it out. */ + return -1; +} +EXPORT_SYMBOL_GPL(slow_virt_to_phys); -- 1.8.3.1
hyperv-tlfs.h defines Hyper-V interfaces from the Hyper-V Top Level Functional Spec (TLFS). The TLFS is distinctly oriented to x86/x64, and Hyper-V has not separated out the architecture-dependent parts into x86/x64 vs. ARM64. So hyperv-tlfs.h includes information for ARM64 that is not yet formally published. The TLFS is available here: docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs mshyperv.h defines Linux-specific structures and routines for interacting with Hyper-V on ARM64. Signed-off-by: Michael Kelley <mikelley@microsoft.com> Reviewed-by: James Morris <james.morris@microsoft.com> --- MAINTAINERS | 2 + arch/arm64/include/asm/hyperv-tlfs.h | 353 +++++++++++++++++++++++++++++++++++ arch/arm64/include/asm/mshyperv.h | 291 +++++++++++++++++++++++++++++ 3 files changed, 646 insertions(+) create mode 100644 arch/arm64/include/asm/hyperv-tlfs.h create mode 100644 arch/arm64/include/asm/mshyperv.h diff --git a/MAINTAINERS b/MAINTAINERS index 9d5eeff..9c06145 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6617,6 +6617,8 @@ F: arch/x86/include/asm/trace/hyperv.h F: arch/x86/include/asm/hyperv-tlfs.h F: arch/x86/kernel/cpu/mshyperv.c F: arch/x86/hyperv +F: arch/arm64/include/asm/hyperv-tlfs.h +F: arch/arm64/include/asm/mshyperv.h F: drivers/hid/hid-hyperv.c F: drivers/hv/ F: drivers/input/serio/hyperv-keyboard.c diff --git a/arch/arm64/include/asm/hyperv-tlfs.h b/arch/arm64/include/asm/hyperv-tlfs.h new file mode 100644 index 0000000..3f88306 --- /dev/null +++ b/arch/arm64/include/asm/hyperv-tlfs.h @@ -0,0 +1,353 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * This file contains definitions from the Hyper-V Hypervisor Top-Level + * Functional Specification (TLFS): + * https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs + * + * Copyright (C) 2018, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or + * NON INFRINGEMENT. See the GNU General Public License for more + * details. + */ + +#ifndef _ASM_ARM64_HYPERV_H +#define _ASM_ARM64_HYPERV_H + +#include <linux/types.h> + +/* + * These Hyper-V registers provide information equivalent to the CPUID + * instruction on x86/x64. + */ +#define HvRegisterHypervisorVersion 0x00000100 /*CPUID 0x40000002 */ +#define HvRegisterPrivilegesAndFeaturesInfo 0x00000200 /*CPUID 0x40000003 */ +#define HvRegisterFeaturesInfo 0x00000201 /*CPUID 0x40000004 */ +#define HvRegisterImplementationLimitsInfo 0x00000202 /*CPUID 0x40000005 */ +#define HvARM64RegisterInterfaceVersion 0x00090006 /*CPUID 0x40000001 */ + +/* + * Feature identification. HvRegisterPrivilegesAndFeaturesInfo returns a + * 128-bit value with flags indicating which features are available to the + * partition based upon the current partition privileges. The 128-bit + * value is broken up with different portions stored in different 32-bit + * fields in the ms_hyperv structure. + */ + +/* Partition Reference Counter available*/ +#define HV_MSR_TIME_REF_COUNT_AVAILABLE (1 << 1) + +/* + * Synthetic Timers available + */ +#define HV_MSR_SYNTIMER_AVAILABLE (1 << 3) + +/* Frequency MSRs available */ +#define HV_FEATURE_FREQUENCY_MSRS_AVAILABLE (1 << 8) + +/* Reference TSC available */ +#define HV_MSR_REFERENCE_TSC_AVAILABLE (1 << 9) + +/* Crash MSR available */ +#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE (1 << 10) + + +/* + * This group of flags is in the high order 64-bits of the returned + * 128-bit value. + */ + +/* STIMER direct mode is available */ +#define HV_STIMER_DIRECT_MODE_AVAILABLE (1 << 19) + +/* + * Implementation recommendations in register + * HvRegisterFeaturesInfo. Indicates which behaviors the hypervisor + * recommends the OS implement for optimal performance. + */ + +/* + * Recommend not using Auto EOI + */ +#define HV_DEPRECATING_AEOI_RECOMMENDED (1 << 9) + +/* + * Temporary #defines for compatibility with architecture + * independent Hyper-V drivers. Remove these once x86-isms + * have been removed from arch independent drivers. + */ +#define HV_X64_MSR_SYNTIMER_AVAILABLE \ + HV_MSR_SYNTIMER_AVAILABLE +#define HV_X64_STIMER_DIRECT_MODE_AVAILABLE \ + HV_STIMER_DIRECT_MODE_AVAILABLE +#define HV_X64_DEPRECATING_AEOI_RECOMMENDED \ + HV_DEPRECATING_AEOI_RECOMMENDED +#define HV_X64_MSR_STIMER0_COUNT 0 +#define HV_X64_MSR_STIMER0_CONFIG 0 +#define HV_X64_MSR_SINT0 0 + +/* + * Synthetic register definitions equivalent to MSRs on x86/x64 + */ +#define HvRegisterCrashP0 0x00000210 +#define HvRegisterCrashP1 0x00000211 +#define HvRegisterCrashP2 0x00000212 +#define HvRegisterCrashP3 0x00000213 +#define HvRegisterCrashP4 0x00000214 +#define HvRegisterCrashCtl 0x00000215 + +#define HvRegisterGuestOsId 0x00090002 +#define HvRegisterVpIndex 0x00090003 +#define HvRegisterTimeRefCount 0x00090004 +#define HvRegisterReferenceTsc 0x00090017 + +#define HvRegisterSint0 0x000A0000 +#define HvRegisterSint1 0x000A0001 +#define HvRegisterSint2 0x000A0002 +#define HvRegisterSint3 0x000A0003 +#define HvRegisterSint4 0x000A0004 +#define HvRegisterSint5 0x000A0005 +#define HvRegisterSint6 0x000A0006 +#define HvRegisterSint7 0x000A0007 +#define HvRegisterSint8 0x000A0008 +#define HvRegisterSint9 0x000A0009 +#define HvRegisterSint10 0x000A000A +#define HvRegisterSint11 0x000A000B +#define HvRegisterSint12 0x000A000C +#define HvRegisterSint13 0x000A000D +#define HvRegisterSint14 0x000A000E +#define HvRegisterSint15 0x000A000F +#define HvRegisterScontrol 0x000A0010 +#define HvRegisterSversion 0x000A0011 +#define HvRegisterSifp 0x000A0012 +#define HvRegisterSipp 0x000A0013 +#define HvRegisterEom 0x000A0014 +#define HvRegisterSirbp 0x000A0015 + +#define HvRegisterStimer0Config 0x000B0000 +#define HvRegisterStimer0Count 0x000B0001 +#define HvRegisterStimer1Config 0x000B0002 +#define HvRegisterStimer1Count 0x000B0003 +#define HvRegisterStimer2Config 0x000B0004 +#define HvRegisterStimer2Count 0x000B0005 +#define HvRegisterStimer3Config 0x000B0006 +#define HvRegisterStimer3Count 0x000B0007 + +/* + * Crash notification flag used in the + * CrashCtl register. + */ +#define HV_CRASH_CTL_CRASH_NOTIFY (1ULL << 63) + +/* + * The guest OS needs to register the guest ID with the hypervisor. + * The guest ID is a 64 bit entity and the structure of this ID is + * specified in the Hyper-V specification: + * + * msdn.microsoft.com/en-us/library/windows/hardware/ff542653%28v=vs.85%29.aspx + * + * While the current guideline does not specify how Linux guest ID(s) + * need to be generated, our plan is to publish the guidelines for + * Linux and other guest operating systems that currently are hosted + * on Hyper-V. The implementation here conforms to this yet + * unpublished guidelines. + * + * + * Bit(s) + * 63 - Indicates if the OS is Open Source or not; 1 is Open Source + * 62:56 - Os Type; Linux is 0x100 + * 55:48 - Distro specific identification + * 47:16 - Linux kernel version number + * 15:0 - Distro specific identification + * + * + */ +#define HV_LINUX_VENDOR_ID 0x8100 + +/* Declare the various hypercall operations. */ +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE 0x0002 +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST 0x0003 +#define HVCALL_NOTIFY_LONG_SPIN_WAIT 0x0008 +#define HVCALL_SEND_IPI 0x000b +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX 0x0013 +#define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX 0x0014 +#define HVCALL_SEND_IPI_EX 0x0015 +#define HVCALL_GET_VP_REGISTERS 0x0050 +#define HVCALL_SET_VP_REGISTERS 0x0051 +#define HVCALL_POST_MESSAGE 0x005c +#define HVCALL_SIGNAL_EVENT 0x005d +#define HVCALL_RETARGET_INTERRUPT 0x007e +#define HVCALL_START_VIRTUAL_PROCESSOR 0x0099 +#define HVCALL_GET_VP_INDEX_FROM_APICID 0x009a + +/* Declare standard hypercall field values. */ +#define HV_PARTITION_ID_SELF ((u64)-1) +#define HV_VP_INDEX_SELF ((u32)-2) + +#define HV_HYPERCALL_FAST_BIT BIT(16) +#define HV_HYPERCALL_REP_COUNT_1 BIT_ULL(32) +#define HV_HYPERCALL_RESULT_MASK GENMASK_ULL(15, 0) + +/* Define the hypercall status result */ + +union hv_hypercall_status { + u64 as_uint64; + struct { + u16 status; + u16 reserved; + u16 reps_completed; /* Low 12 bits */ + u16 reserved2; + }; +}; + +/* hypercall status code */ +#define HV_STATUS_SUCCESS 0 +#define HV_STATUS_INVALID_HYPERCALL_CODE 2 +#define HV_STATUS_INVALID_HYPERCALL_INPUT 3 +#define HV_STATUS_INVALID_ALIGNMENT 4 +#define HV_STATUS_INSUFFICIENT_MEMORY 11 +#define HV_STATUS_INVALID_CONNECTION_ID 18 +#define HV_STATUS_INSUFFICIENT_BUFFERS 19 + +/* Define output layout for Get VP Register hypercall */ +struct hv_get_vp_register_output { + u64 registervaluelow; + u64 registervaluehigh; +}; + +#define HV_FLUSH_ALL_PROCESSORS BIT(0) +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES BIT(1) +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY BIT(2) +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT BIT(3) + +enum HV_GENERIC_SET_FORMAT { + HV_GENERIC_SET_SPARSE_4K, + HV_GENERIC_SET_ALL, +}; + +/* + * The Hyper-V TimeRefCount register and the TSC + * page provide a guest VM clock with 100ns tick rate + */ +#define HV_CLOCK_HZ (NSEC_PER_SEC/100) + +/* + * The fields in this structure are set by Hyper-V and read + * by the Linux guest. They should be accessed with READ_ONCE() + * so the compiler doesn't optimize in a way that will cause + * problems. + */ +struct ms_hyperv_tsc_page { + u32 tsc_sequence; + u32 reserved1; + u64 tsc_scale; + s64 tsc_offset; + u64 reserved2[509]; +}; + +/* Define the number of synthetic interrupt sources. */ +#define HV_SYNIC_SINT_COUNT (16) +/* Define the expected SynIC version. */ +#define HV_SYNIC_VERSION_1 (0x1) + +#define HV_SYNIC_CONTROL_ENABLE (1ULL << 0) +#define HV_SYNIC_SIMP_ENABLE (1ULL << 0) +#define HV_SYNIC_SIEFP_ENABLE (1ULL << 0) +#define HV_SYNIC_SINT_MASKED (1ULL << 16) +#define HV_SYNIC_SINT_AUTO_EOI (1ULL << 17) +#define HV_SYNIC_SINT_VECTOR_MASK (0xFF) + +#define HV_SYNIC_STIMER_COUNT (4) + +/* Define synthetic interrupt controller message constants. */ +#define HV_MESSAGE_SIZE (256) +#define HV_MESSAGE_PAYLOAD_BYTE_COUNT (240) +#define HV_MESSAGE_PAYLOAD_QWORD_COUNT (30) + +/* Define hypervisor message types. */ +enum hv_message_type { + HVMSG_NONE = 0x00000000, + + /* Memory access messages. */ + HVMSG_UNMAPPED_GPA = 0x80000000, + HVMSG_GPA_INTERCEPT = 0x80000001, + + /* Timer notification messages. */ + HVMSG_TIMER_EXPIRED = 0x80000010, + + /* Error messages. */ + HVMSG_INVALID_VP_REGISTER_VALUE = 0x80000020, + HVMSG_UNRECOVERABLE_EXCEPTION = 0x80000021, + HVMSG_UNSUPPORTED_FEATURE = 0x80000022, + + /* Trace buffer complete messages. */ + HVMSG_EVENTLOG_BUFFERCOMPLETE = 0x80000040, +}; + +/* Define synthetic interrupt controller message flags. */ +union hv_message_flags { + __u8 asu8; + struct { + __u8 msg_pending:1; + __u8 reserved:7; + }; +}; + +/* Define port identifier type. */ +union hv_port_id { + __u32 asu32; + struct { + __u32 id:24; + __u32 reserved:8; + } u; +}; + +/* Define synthetic interrupt controller message header. */ +struct hv_message_header { + __u32 message_type; + __u8 payload_size; + union hv_message_flags message_flags; + __u8 reserved[2]; + union { + __u64 sender; + union hv_port_id port; + }; +}; + +/* Define synthetic interrupt controller message format. */ +struct hv_message { + struct hv_message_header header; + union { + __u64 payload[HV_MESSAGE_PAYLOAD_QWORD_COUNT]; + } u; +}; + +/* Define the synthetic interrupt message page layout. */ +struct hv_message_page { + struct hv_message sint_message[HV_SYNIC_SINT_COUNT]; +}; + +/* Define timer message payload structure. */ +struct hv_timer_message_payload { + __u32 timer_index; + __u32 reserved; + __u64 expiration_time; /* When the timer expired */ + __u64 delivery_time; /* When the message was delivered */ +}; + +#define HV_STIMER_ENABLE (1ULL << 0) +#define HV_STIMER_PERIODIC (1ULL << 1) +#define HV_STIMER_LAZY (1ULL << 2) +#define HV_STIMER_AUTOENABLE (1ULL << 3) +#define HV_STIMER_SINT(config) (__u8)(((config) >> 16) & 0x0F) + +#endif diff --git a/arch/arm64/include/asm/mshyperv.h b/arch/arm64/include/asm/mshyperv.h new file mode 100644 index 0000000..1428d13 --- /dev/null +++ b/arch/arm64/include/asm/mshyperv.h @@ -0,0 +1,291 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Linux-specific definitions for managing interactions with Microsoft's + * Hyper-V hypervisor. Definitions that are specified in the Hyper-V + * Top Level Functional Spec (TLFS) should not go in this file, but + * should instead go in hyperv-tlfs.h. + * + * Copyright (C) 2018, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or + * NON INFRINGEMENT. See the GNU General Public License for more + * details. + */ + +#ifndef _ASM_ARM64_MSHYPERV_H +#define _ASM_ARM64_MSHYPERV_H + +#include <linux/types.h> +#include <linux/interrupt.h> +#include <linux/clocksource.h> +#include <linux/irq.h> +#include <linux/irqdesc.h> +#include <asm/hyperv-tlfs.h> + +/* + * Hyper-V always runs with a page size of 4096. These definitions + * are used when communicating with Hyper-V using guest physical + * pages and guest physical page addresses, since the guest page + * size may not be 4096 on ARM64. + */ +#define HV_HYP_PAGE_SIZE 4096 +#define HV_HYP_PAGE_SHIFT 12 +#define HV_HYP_PAGE_MASK (~(HV_HYP_PAGE_SIZE - 1)) + + +struct ms_hyperv_info { + u32 features; + u32 misc_features; + u32 hints; + u32 max_vp_index; + u32 max_lp_index; +}; +extern struct ms_hyperv_info ms_hyperv; + +/* + * Define the IRQ numbers/vectors used by Hyper-V VMbus interrupts + * and by STIMER0 Direct Mode interrupts. Hyper-V should be supplying + * these values through ACPI, but there are no other interrupting + * devices in a Hyper-V VM on ARM64, so it's OK to hard code for now. + * The "CALLBACK_VECTOR" terminology is a left-over from the x86/x64 + * world that is used in architecture independent Hyper-V code. + */ +#define HYPERVISOR_CALLBACK_VECTOR 16 +#define HV_STIMER0_IRQNR 17 + +extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr); +extern u64 hv_do_fast_hypercall8(u16 control, u64 input8); + +extern u64 hv_do_hvc(u64 control, ...); +extern u64 hv_do_hvc_fast_get(u64 control, u64 input1, u64 input2, u64 input3, + struct hv_get_vp_register_output *output); + +/* + * Declare calls to get and set Hyper-V VP register values on ARM64, which + * requires a hypercall. + */ +extern void hv_set_vpreg(u32 reg, u64 value); +extern u64 hv_get_vpreg(u32 reg); +extern void hv_get_vpreg_128(u32 reg, struct hv_get_vp_register_output *result); + +/* + * The guest OS needs to register the guest ID with the hypervisor. + * The guest ID is a 64 bit entity and the structure of this ID is + * specified in the Hyper-V specification: + * + * msdn.microsoft.com/en-us/library/windows/hardware/ff542653%28v=vs.85%29.aspx + * + * While the current guideline does not specify how Linux guest ID(s) + * need to be generated, our plan is to publish the guidelines for + * Linux and other guest operating systems that currently are hosted + * on Hyper-V. The implementation here conforms to this yet + * unpublished guidelines. + * + * + * Bit(s) + * 63 - Indicates if the OS is Open Source or not; 1 is Open Source + * 62:56 - Os Type; Linux is 0x100 + * 55:48 - Distro specific identification + * 47:16 - Linux kernel version number + * 15:0 - Distro specific identification + * + * Generate the guest ID based on the guideline described above. + */ + +static inline __u64 generate_guest_id(__u64 d_info1, __u64 kernel_version, + __u64 d_info2) +{ + __u64 guest_id = 0; + + guest_id = (((__u64)HV_LINUX_VENDOR_ID) << 48); + guest_id |= (d_info1 << 48); + guest_id |= (kernel_version << 16); + guest_id |= d_info2; + + return guest_id; +} + + +/* Free the message slot and signal end-of-message if required */ +static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type) +{ + /* + * On crash we're reading some other CPU's message page and we need + * to be careful: this other CPU may already had cleared the header + * and the host may already had delivered some other message there. + * In case we blindly write msg->header.message_type we're going + * to lose it. We can still lose a message of the same type but + * we count on the fact that there can only be one + * CHANNELMSG_UNLOAD_RESPONSE and we don't care about other messages + * on crash. + */ + if (cmpxchg(&msg->header.message_type, old_msg_type, + HVMSG_NONE) != old_msg_type) + return; + + /* + * Make sure the write to MessageType (ie set to + * HVMSG_NONE) happens before we read the + * MessagePending and EOMing. Otherwise, the EOMing + * will not deliver any more messages since there is + * no empty slot + */ + mb(); + + if (msg->header.message_flags.msg_pending) { + /* + * This will cause message queue rescan to + * possibly deliver another msg from the + * hypervisor + */ + hv_set_vpreg(HvRegisterEom, 0); + } +} + +/* + * Use the Hyper-V provided stimer0 as the timer that is made + * available to the architecture independent Hyper-V drivers. + */ +#define hv_init_timer(timer, tick) \ + hv_set_vpreg(HvRegisterStimer0Count + (2*timer), tick) +#define hv_init_timer_config(timer, val) \ + hv_set_vpreg(HvRegisterStimer0Config + (2*timer), val) +#define hv_get_current_tick(tick) \ + (tick = hv_get_vpreg(HvRegisterTimeRefCount)) + +#define hv_get_simp(val) (val = hv_get_vpreg(HvRegisterSipp)) +#define hv_set_simp(val) hv_set_vpreg(HvRegisterSipp, val) + +#define hv_get_siefp(val) (val = hv_get_vpreg(HvRegisterSifp)) +#define hv_set_siefp(val) hv_set_vpreg(HvRegisterSifp, val) + +#define hv_get_synic_state(val) (val = hv_get_vpreg(HvRegisterScontrol)) +#define hv_set_synic_state(val) hv_set_vpreg(HvRegisterScontrol, val) + +#define hv_get_vp_index(index) (index = hv_get_vpreg(HvRegisterVpIndex)) + +/* + * Hyper-V SINT registers are numbered sequentially, so we can just + * add the SINT number to the register number of SINT0 + */ +#define hv_get_synint_state(sint_num, val) \ + (val = hv_get_vpreg(HvRegisterSint0 + sint_num)) +#define hv_set_synint_state(sint_num, val) \ + hv_set_vpreg(HvRegisterSint0 + sint_num, val) + +void hv_setup_vmbus_irq(void (*handler)(void)); +void hv_remove_vmbus_irq(void); +void hv_enable_vmbus_irq(void); +void hv_disable_vmbus_irq(void); + +void hv_setup_kexec_handler(void (*handler)(void)); +void hv_remove_kexec_handler(void); +void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs)); +void hv_remove_crash_handler(void); + +#if IS_ENABLED(CONFIG_HYPERV) +extern struct clocksource *hyperv_cs; + +/* + * Hypervisor's notion of virtual processor ID is different from + * Linux' notion of CPU ID. This information can only be retrieved + * in the context of the calling CPU. Setup a map for easy access + * to this information. + */ +extern u32 *hv_vp_index; +extern u32 hv_max_vp_index; + +/** + * hv_cpu_number_to_vp_number() - Map CPU to VP. + * @cpu_number: CPU number in Linux terms + * + * This function returns the mapping between the Linux processor + * number and the hypervisor's virtual processor number, useful + * in making hypercalls and such that talk about specific + * processors. + * + * Return: Virtual processor number in Hyper-V terms + */ +static inline int hv_cpu_number_to_vp_number(int cpu_number) +{ + return hv_vp_index[cpu_number]; +} + +void hyperv_report_panic(struct pt_regs *regs, long err); +bool hv_is_hyperv_initialized(void); +void hyperv_cleanup(void); +#else /* CONFIG_HYPERV */ +static inline bool hv_is_hyperv_initialized(void) { return false; } +static inline void hyperv_cleanup(void) {} +#endif /* CONFIG_HYPERV */ + +#if IS_ENABLED(CONFIG_HYPERV) +#define hv_enable_stimer0_percpu_irq(irq) enable_percpu_irq(irq, 0) +#define hv_disable_stimer0_percpu_irq(irq) disable_percpu_irq(irq) +extern int hv_setup_stimer0_irq(int *irq, int *vector, void (*handler)(void)); +extern void hv_remove_stimer0_irq(int irq); +#endif + +extern struct ms_hyperv_tsc_page *hv_get_tsc_page(void); +static inline u64 hv_read_tsc_page_tsc(const struct ms_hyperv_tsc_page *tsc_pg, + u64 *cur_tsc) +{ + u64 scale, offset; + u32 sequence; + + /* + * The protocol for reading Hyper-V TSC page is specified in Hypervisor + * Top-Level Functional Specification. To get the reference time we + * must do the following: + * - READ ReferenceTscSequence + * A special '0' value indicates the time source is unreliable and we + * need to use something else. + * - ReferenceTime = + * ((CNTVCT_EL0) * ReferenceTscScale) >> 64) + ReferenceTscOffset + * - READ ReferenceTscSequence again. In case its value has changed + * since our first reading we need to discard ReferenceTime and repeat + * the whole sequence as the hypervisor was updating the page in + * between. + */ + do { + sequence = READ_ONCE(tsc_pg->tsc_sequence); + /* + * Make sure we read sequence before we read other values from + * TSC page. + */ + smp_rmb(); + + scale = READ_ONCE(tsc_pg->tsc_scale); + offset = READ_ONCE(tsc_pg->tsc_offset); + isb(); + *cur_tsc = read_sysreg(cntvct_el0); + isb(); + + /* + * Make sure we read sequence after we read all other values + * from TSC page. + */ + smp_rmb(); + + } while (READ_ONCE(tsc_pg->tsc_sequence) != sequence); + + return mul_u64_u64_shr(*cur_tsc, scale, 64) + offset; +} + +static inline u64 hv_read_tsc_page(const struct ms_hyperv_tsc_page *tsc_pg) +{ + u64 cur_tsc; + + return hv_read_tsc_page_tsc(tsc_pg, &cur_tsc); +} + +#endif -- 1.8.3.1
Add ARM64-specific code to enable Hyper-V. This code includes: * Detecting Hyper-V and initializing the guest/Hyper-V interface * Setting up Hyper-V's synthetic clocks * Making hypercalls using the HVC instruction * Setting up VMbus and stimer0 interrupts * Setting up kexec and crash handlers This code is architecture dependent code and is mostly driven by architecture independent code in the VMbus driver in drivers/hv/hv.c and drivers/hv/vmbus_drv.c. This code is built only when CONFIG_HYPERV is enabled. Signed-off-by: Michael Kelley <mikelley@microsoft.com> Reviewed-by: James Morris <james.morris@microsoft.com> --- MAINTAINERS | 1 + arch/arm64/Makefile | 1 + arch/arm64/hyperv/Makefile | 2 + arch/arm64/hyperv/hv_hvc.S | 54 ++++++ arch/arm64/hyperv/hv_init.c | 411 +++++++++++++++++++++++++++++++++++++++++++ arch/arm64/hyperv/mshyperv.c | 178 +++++++++++++++++++ 6 files changed, 647 insertions(+) create mode 100644 arch/arm64/hyperv/Makefile create mode 100644 arch/arm64/hyperv/hv_hvc.S create mode 100644 arch/arm64/hyperv/hv_init.c create mode 100644 arch/arm64/hyperv/mshyperv.c diff --git a/MAINTAINERS b/MAINTAINERS index 9c06145..946021e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6619,6 +6619,7 @@ F: arch/x86/kernel/cpu/mshyperv.c F: arch/x86/hyperv F: arch/arm64/include/asm/hyperv-tlfs.h F: arch/arm64/include/asm/mshyperv.h +F: arch/arm64/hyperv F: drivers/hid/hid-hyperv.c F: drivers/hv/ F: drivers/input/serio/hyperv-keyboard.c diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 4527226..4b384a2 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -105,6 +105,7 @@ core-y += arch/arm64/kernel/ arch/arm64/mm/ core-$(CONFIG_NET) += arch/arm64/net/ core-$(CONFIG_KVM) += arch/arm64/kvm/ core-$(CONFIG_XEN) += arch/arm64/xen/ +core-$(CONFIG_HYPERV) += arch/arm64/hyperv/ core-$(CONFIG_CRYPTO) += arch/arm64/crypto/ libs-y := arch/arm64/lib/ $(libs-y) core-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a diff --git a/arch/arm64/hyperv/Makefile b/arch/arm64/hyperv/Makefile new file mode 100644 index 0000000..988eda5 --- /dev/null +++ b/arch/arm64/hyperv/Makefile @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-y := hv_init.o hv_hvc.o mshyperv.o diff --git a/arch/arm64/hyperv/hv_hvc.S b/arch/arm64/hyperv/hv_hvc.S new file mode 100644 index 0000000..8263696 --- /dev/null +++ b/arch/arm64/hyperv/hv_hvc.S @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Microsoft Hyper-V hypervisor invocation routines + * + * Copyright (C) 2018, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or + * NON INFRINGEMENT. See the GNU General Public License for more + * details. + */ + +#include <linux/linkage.h> + + .text +/* + * Do the HVC instruction. For Hyper-V the argument is always 1. + * x0 contains the hypercall control value, while additional registers + * vary depending on the hypercall, and whether the hypercall arguments + * are in memory or in registers (a "fast" hypercall per the Hyper-V + * TLFS). When the arguments are in memory x1 is the guest physical + * address of the input arguments, and x2 is the guest physical + * address of the output arguments. When the arguments are in + * registers, the register values depends on the hypercall. Note + * that this version cannot return any values in registers. + */ +ENTRY(hv_do_hvc) + hvc #1 + ret +ENDPROC(hv_do_hvc) + +/* + * This variant of HVC invocation is for hv_get_vpreg and + * hv_get_vpreg_128. The input parameters are passed in registers + * along with a pointer in x4 to where the output result should + * be stored. The output is returned in x15 and x16. x18 is used as + * scratch space to avoid buildng a stack frame, as Hyper-V does + * not preserve registers x0-x17. + */ +ENTRY(hv_do_hvc_fast_get) + mov x18, x4 + hvc #1 + str x15,[x18] + str x16,[x18,#8] + ret +ENDPROC(hv_do_hvc_fast_get) diff --git a/arch/arm64/hyperv/hv_init.c b/arch/arm64/hyperv/hv_init.c new file mode 100644 index 0000000..dcab9e3 --- /dev/null +++ b/arch/arm64/hyperv/hv_init.c @@ -0,0 +1,411 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Initialization of the interface with Microsoft's Hyper-V hypervisor, + * and various low level utility routines for interacting with Hyper-V. + * + * Copyright (C) 2018, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or + * NON INFRINGEMENT. See the GNU General Public License for more + * details. + */ + + +#include <linux/types.h> +#include <linux/version.h> +#include <linux/export.h> +#include <linux/vmalloc.h> +#include <linux/mm.h> +#include <linux/clocksource.h> +#include <linux/sched_clock.h> +#include <linux/acpi.h> +#include <linux/module.h> +#include <linux/hyperv.h> +#include <linux/slab.h> +#include <linux/cpuhotplug.h> +#include <linux/psci.h> +#include <asm-generic/bug.h> +#include <asm/hypervisor.h> +#include <asm/hyperv-tlfs.h> +#include <asm/mshyperv.h> +#include <asm/sysreg.h> +#include <clocksource/arm_arch_timer.h> + +static bool hyperv_initialized; +struct ms_hyperv_info ms_hyperv; +EXPORT_SYMBOL_GPL(ms_hyperv); + +static struct ms_hyperv_tsc_page *tsc_pg; + +struct ms_hyperv_tsc_page *hv_get_tsc_page(void) +{ + return tsc_pg; +} +EXPORT_SYMBOL_GPL(hv_get_tsc_page); + +static u64 read_hv_sched_clock_tsc(void) +{ + u64 current_tick = hv_read_tsc_page(tsc_pg); + + if (current_tick == U64_MAX) + current_tick = hv_get_vpreg(HvRegisterTimeRefCount); + + return current_tick; +} + +static u64 read_hv_clock_tsc(struct clocksource *arg) +{ + u64 current_tick = hv_read_tsc_page(tsc_pg); + + if (current_tick == U64_MAX) + current_tick = hv_get_vpreg(HvRegisterTimeRefCount); + + return current_tick; +} + +static struct clocksource hyperv_cs_tsc = { + .name = "hyperv_clocksource_tsc_page", + .rating = 400, + .read = read_hv_clock_tsc, + .mask = CLOCKSOURCE_MASK(64), + .flags = CLOCK_SOURCE_IS_CONTINUOUS, +}; + +static u64 read_hv_sched_clock_msr(void) +{ + return hv_get_vpreg(HvRegisterTimeRefCount); +} + +static u64 read_hv_clock_msr(struct clocksource *arg) +{ + return hv_get_vpreg(HvRegisterTimeRefCount); +} + +static struct clocksource hyperv_cs_msr = { + .name = "hyperv_clocksource_msr", + .rating = 400, + .read = read_hv_clock_msr, + .mask = CLOCKSOURCE_MASK(64), + .flags = CLOCK_SOURCE_IS_CONTINUOUS, +}; + +struct clocksource *hyperv_cs; +EXPORT_SYMBOL_GPL(hyperv_cs); + +u32 *hv_vp_index; +EXPORT_SYMBOL_GPL(hv_vp_index); + +u32 hv_max_vp_index; + +static int hv_cpu_init(unsigned int cpu) +{ + u64 msr_vp_index; + + hv_get_vp_index(msr_vp_index); + + hv_vp_index[smp_processor_id()] = msr_vp_index; + + if (msr_vp_index > hv_max_vp_index) + hv_max_vp_index = msr_vp_index; + + return 0; +} + +/* + * This function is invoked via the ACPI clocksource probe mechanism. We + * don't actually use any values from the ACPI GTDT table, but we set up + * the Hyper-V synthetic clocksource and do other initialization for + * interacting with Hyper-V the first time. Using early_initcall to invoke + * this function is too late because interrupts are already enabled at that + * point, and sched_clock_register must run before interrupts are enabled. + * + * 1. Setup the guest ID. + * 2. Get features and hints info from Hyper-V + * 3. Setup per-cpu VP indices. + * 4. Register Hyper-V specific clocksource. + * 5. Register the scheduler clock. + */ + +static int __init hyperv_init(struct acpi_table_header *table) +{ + struct hv_get_vp_register_output result; + u32 a, b, c, d; + u64 guest_id; + + /* + * If we're in a VM on Hyper-V, the ACPI hypervisor_id field will + * have the string "MsHyperV". + */ + if (strncmp((char *)&acpi_gbl_FADT.hypervisor_id, "MsHyperV", 8)) + return 1; + + /* Setup the guest ID */ + guest_id = generate_guest_id(0, LINUX_VERSION_CODE, 0); + hv_set_vpreg(HvRegisterGuestOsId, guest_id); + + /* Get the features and hints from Hyper-V */ + hv_get_vpreg_128(HvRegisterPrivilegesAndFeaturesInfo, &result); + ms_hyperv.features = lower_32_bits(result.registervaluelow); + ms_hyperv.misc_features = upper_32_bits(result.registervaluehigh); + + hv_get_vpreg_128(HvRegisterFeaturesInfo, &result); + ms_hyperv.hints = lower_32_bits(result.registervaluelow); + + pr_info("Hyper-V: Features 0x%x, hints 0x%x\n", + ms_hyperv.features, ms_hyperv.hints); + + /* + * Direct mode is the only option for STIMERs provided Hyper-V + * on ARM64, so Hyper-V doesn't actually set the flag. But add the + * flag so the architecture independent code in drivers/hv/hv.c + * will correctly use that mode. + */ + ms_hyperv.misc_features |= HV_STIMER_DIRECT_MODE_AVAILABLE; + + /* + * Hyper-V on ARM64 doesn't support AutoEOI. Add the hint + * that tells architecture independent code not to use this + * feature. + */ + ms_hyperv.hints |= HV_DEPRECATING_AEOI_RECOMMENDED; + + /* Get information about the Hyper-V host version */ + hv_get_vpreg_128(HvRegisterHypervisorVersion, &result); + a = lower_32_bits(result.registervaluelow); + b = upper_32_bits(result.registervaluelow); + c = lower_32_bits(result.registervaluehigh); + d = upper_32_bits(result.registervaluehigh); + pr_info("Hyper-V: Host Build %d.%d.%d.%d-%d-%d\n", + b >> 16, b & 0xFFFF, a, d & 0xFFFFFF, c, d >> 24); + + /* Allocate percpu VP index */ + hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index), + GFP_KERNEL); + if (!hv_vp_index) + return 1; + + if (cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "arm64/hyperv_init:online", + hv_cpu_init, NULL) < 0) + goto free_vp_index; + + /* + * Try to set up what Hyper-V calls the "TSC reference page", which + * uses the ARM Generic Timer virtual counter with some scaling + * information to provide a fast and stable guest VM clocksource. + * If the TSC reference page can't be set up, fall back to reading + * the guest clock provided by Hyper-V's synthetic reference time + * register. + */ + if (ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE) { + + u64 tsc_msr; + phys_addr_t phys_addr; + + tsc_pg = __vmalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL, PAGE_KERNEL); + if (tsc_pg) { + phys_addr = page_to_phys(vmalloc_to_page(tsc_pg)); + tsc_msr = hv_get_vpreg(HvRegisterReferenceTsc); + tsc_msr &= GENMASK_ULL(11, 0); + tsc_msr = tsc_msr | 0x1 | (u64)phys_addr; + hv_set_vpreg(HvRegisterReferenceTsc, tsc_msr); + hyperv_cs = &hyperv_cs_tsc; + sched_clock_register(read_hv_sched_clock_tsc, + 64, HV_CLOCK_HZ); + } + } + + if (!hyperv_cs && + (ms_hyperv.features & HV_MSR_TIME_REF_COUNT_AVAILABLE)) { + hyperv_cs = &hyperv_cs_msr; + sched_clock_register(read_hv_sched_clock_msr, + 64, HV_CLOCK_HZ); + } + + if (hyperv_cs) { + hyperv_cs->archdata.vdso_direct = false; + clocksource_register_hz(hyperv_cs, HV_CLOCK_HZ); + } + + hyperv_initialized = true; + return 0; + +free_vp_index: + kfree(hv_vp_index); + hv_vp_index = NULL; + return 1; +} +TIMER_ACPI_DECLARE(hyperv, ACPI_SIG_GTDT, hyperv_init); + +/* + * This routine is called before kexec/kdump, it does the required cleanup. + */ +void hyperv_cleanup(void) +{ + /* Reset our OS id */ + hv_set_vpreg(HvRegisterGuestOsId, 0); + +} +EXPORT_SYMBOL_GPL(hyperv_cleanup); + +/* + * hv_do_hypercall- Invoke the specified hypercall + */ +u64 hv_do_hypercall(u64 control, void *input, void *output) +{ + u64 input_address; + u64 output_address; + + input_address = input ? virt_to_phys(input) : 0; + output_address = output ? virt_to_phys(output) : 0; + return hv_do_hvc(control, input_address, output_address); +} +EXPORT_SYMBOL_GPL(hv_do_hypercall); + +/* + * hv_do_fast_hypercall8 -- Invoke the specified hypercall + * with arguments in registers instead of physical memory. + * Avoids the overhead of virt_to_phys for simple hypercalls. + */ + +u64 hv_do_fast_hypercall8(u16 code, u64 input) +{ + u64 control; + + control = (u64)code | HV_HYPERCALL_FAST_BIT; + return hv_do_hvc(control, input); +} +EXPORT_SYMBOL_GPL(hv_do_fast_hypercall8); + + +/* + * Set a single VP register to a 64-bit value. + */ +void hv_set_vpreg(u32 msr, u64 value) +{ + union hv_hypercall_status status; + + status.as_uint64 = hv_do_hvc( + HVCALL_SET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | + HV_HYPERCALL_REP_COUNT_1, + HV_PARTITION_ID_SELF, + HV_VP_INDEX_SELF, + msr, + 0, + value, + 0); + + /* + * Something is fundamentally broken in the hypervisor if + * setting a VP register fails. There's really no way to + * continue as a guest VM, so panic. + */ + BUG_ON(status.status != HV_STATUS_SUCCESS); +} +EXPORT_SYMBOL_GPL(hv_set_vpreg); + + +/* + * Get the value of a single VP register, and only the low order 64 bits. + */ +u64 hv_get_vpreg(u32 msr) +{ + union hv_hypercall_status status; + struct hv_get_vp_register_output output; + + status.as_uint64 = hv_do_hvc_fast_get( + HVCALL_GET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | + HV_HYPERCALL_REP_COUNT_1, + HV_PARTITION_ID_SELF, + HV_VP_INDEX_SELF, + msr, + &output); + + /* + * Something is fundamentally broken in the hypervisor if + * getting a VP register fails. There's really no way to + * continue as a guest VM, so panic. + */ + BUG_ON(status.status != HV_STATUS_SUCCESS); + + return output.registervaluelow; +} +EXPORT_SYMBOL_GPL(hv_get_vpreg); + +/* + * Get the value of a single VP register that is 128 bits in size. This is a + * separate call in order to avoid complicating the calling sequence for + * the much more frequently used 64-bit version. + */ +void hv_get_vpreg_128(u32 msr, struct hv_get_vp_register_output *result) +{ + union hv_hypercall_status status; + + status.as_uint64 = hv_do_hvc_fast_get( + HVCALL_GET_VP_REGISTERS | HV_HYPERCALL_FAST_BIT | + HV_HYPERCALL_REP_COUNT_1, + HV_PARTITION_ID_SELF, + HV_VP_INDEX_SELF, + msr, + result); + + /* + * Something is fundamentally broken in the hypervisor if + * getting a VP register fails. There's really no way to + * continue as a guest VM, so panic. + */ + BUG_ON(status.status != HV_STATUS_SUCCESS); + + return; + +} +EXPORT_SYMBOL_GPL(hv_get_vpreg_128); + +void hyperv_report_panic(struct pt_regs *regs, long err) +{ + static bool panic_reported; + u64 guest_id; + + /* + * We prefer to report panic on 'die' chain as we have proper + * registers to report, but if we miss it (e.g. on BUG()) we need + * to report it on 'panic'. + */ + if (panic_reported) + return; + panic_reported = true; + + guest_id = hv_get_vpreg(HvRegisterGuestOsId); + + /* + * Hyper-V provides the ability to store only 5 values. + * Pick the passed in error value, the guest_id, and the PC. + * The first two general registers are added arbitrarily. + */ + hv_set_vpreg(HvRegisterCrashP0, err); + hv_set_vpreg(HvRegisterCrashP1, guest_id); + hv_set_vpreg(HvRegisterCrashP2, regs->pc); + hv_set_vpreg(HvRegisterCrashP3, regs->regs[0]); + hv_set_vpreg(HvRegisterCrashP4, regs->regs[1]); + + /* + * Let Hyper-V know there is crash data available + */ + hv_set_vpreg(HvRegisterCrashCtl, HV_CRASH_CTL_CRASH_NOTIFY); +} +EXPORT_SYMBOL_GPL(hyperv_report_panic); + + +bool hv_is_hyperv_initialized(void) +{ + return hyperv_initialized; +} +EXPORT_SYMBOL_GPL(hv_is_hyperv_initialized); diff --git a/arch/arm64/hyperv/mshyperv.c b/arch/arm64/hyperv/mshyperv.c new file mode 100644 index 0000000..3ef0555 --- /dev/null +++ b/arch/arm64/hyperv/mshyperv.c @@ -0,0 +1,178 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Core routines for interacting with Microsoft's Hyper-V hypervisor, + * including setting up VMbus and STIMER interrupts, and handling + * crashes and kexecs. These interactions are through a set of + * static "handler" variables set by the architecture independent + * VMbus and STIMER drivers. This design is used to meet x86/x64 + * requirements for avoiding direct linkages and allowing the VMbus + * and STIMER drivers to be unloaded and reloaded. + * + * Copyright (C) 2018, Microsoft, Inc. + * + * Author : Michael Kelley <mikelley@microsoft.com> + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or + * NON INFRINGEMENT. See the GNU General Public License for more + * details. + */ + +#include <linux/types.h> +#include <linux/export.h> +#include <linux/interrupt.h> +#include <linux/kexec.h> +#include <linux/acpi.h> +#include <linux/ptrace.h> +#include <asm/hyperv-tlfs.h> +#include <asm/mshyperv.h> + +static void (*vmbus_handler)(void); +static void (*hv_stimer0_handler)(void); +static void (*hv_kexec_handler)(void); +static void (*hv_crash_handler)(struct pt_regs *regs); + +static int vmbus_irq; +static long __percpu *vmbus_evt; +static long __percpu *stimer0_evt; + +irqreturn_t hyperv_vector_handler(int irq, void *dev_id) +{ + if (vmbus_handler) + vmbus_handler(); + return IRQ_HANDLED; +} + +/* Must be done just once */ +void hv_setup_vmbus_irq(void (*handler)(void)) +{ + int result; + + vmbus_handler = handler; + vmbus_irq = acpi_register_gsi(NULL, HYPERVISOR_CALLBACK_VECTOR, + ACPI_LEVEL_SENSITIVE, ACPI_ACTIVE_HIGH); + if (vmbus_irq <= 0) { + pr_err("Can't register Hyper-V VMBus GSI. Error %d", + vmbus_irq); + vmbus_irq = 0; + return; + } + vmbus_evt = alloc_percpu(long); + result = request_percpu_irq(vmbus_irq, hyperv_vector_handler, + "Hyper-V VMbus", vmbus_evt); + if (result) { + pr_err("Can't request Hyper-V VMBus IRQ %d. Error %d", + vmbus_irq, result); + free_percpu(vmbus_evt); + acpi_unregister_gsi(vmbus_irq); + vmbus_irq = 0; + } +} +EXPORT_SYMBOL_GPL(hv_setup_vmbus_irq); + +/* Must be done just once */ +void hv_remove_vmbus_irq(void) +{ + if (vmbus_irq) { + free_percpu_irq(vmbus_irq, vmbus_evt); + free_percpu(vmbus_evt); + acpi_unregister_gsi(vmbus_irq); + } +} +EXPORT_SYMBOL_GPL(hv_remove_vmbus_irq); + +/* Must be done by each CPU */ +void hv_enable_vmbus_irq(void) +{ + enable_percpu_irq(vmbus_irq, 0); +} +EXPORT_SYMBOL_GPL(hv_enable_vmbus_irq); + +/* Must be done by each CPU */ +void hv_disable_vmbus_irq(void) +{ + disable_percpu_irq(vmbus_irq); +} +EXPORT_SYMBOL_GPL(hv_disable_vmbus_irq); + +/* Routines to do per-architecture handling of STIMER0 when in Direct Mode */ + +static irqreturn_t hv_stimer0_vector_handler(int irq, void *dev_id) +{ + if (hv_stimer0_handler) + hv_stimer0_handler(); + return IRQ_HANDLED; +} + +int hv_setup_stimer0_irq(int *irq, int *vector, void (*handler)(void)) +{ + int localirq; + int result; + + localirq = acpi_register_gsi(NULL, HV_STIMER0_IRQNR, + ACPI_LEVEL_SENSITIVE, ACPI_ACTIVE_HIGH); + if (localirq <= 0) { + pr_err("Can't register Hyper-V stimer0 GSI. Error %d", + localirq); + *irq = 0; + return -1; + } + stimer0_evt = alloc_percpu(long); + result = request_percpu_irq(localirq, hv_stimer0_vector_handler, + "Hyper-V stimer0", stimer0_evt); + if (result) { + pr_err("Can't request Hyper-V stimer0 IRQ %d. Error %d", + localirq, result); + free_percpu(stimer0_evt); + acpi_unregister_gsi(localirq); + *irq = 0; + return -1; + } + + hv_stimer0_handler = handler; + *vector = HV_STIMER0_IRQNR; + *irq = localirq; + return 0; +} +EXPORT_SYMBOL_GPL(hv_setup_stimer0_irq); + +void hv_remove_stimer0_irq(int irq) +{ + hv_stimer0_handler = NULL; + if (irq) { + free_percpu_irq(irq, stimer0_evt); + free_percpu(stimer0_evt); + acpi_unregister_gsi(irq); + } +} +EXPORT_SYMBOL_GPL(hv_remove_stimer0_irq); + +void hv_setup_kexec_handler(void (*handler)(void)) +{ + hv_kexec_handler = handler; +} +EXPORT_SYMBOL_GPL(hv_setup_kexec_handler); + +void hv_remove_kexec_handler(void) +{ + hv_kexec_handler = NULL; +} +EXPORT_SYMBOL_GPL(hv_remove_kexec_handler); + +void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs)) +{ + hv_crash_handler = handler; +} +EXPORT_SYMBOL_GPL(hv_setup_crash_handler); + +void hv_remove_crash_handler(void) +{ + hv_crash_handler = NULL; +} +EXPORT_SYMBOL_GPL(hv_remove_crash_handler); -- 1.8.3.1
Add hooks to enable/disable a per-CPU IRQ for VMbus. These hooks are in the architecture independent setup and shutdown paths for Hyper-V, and are needed by Linux guests on Hyper-V on ARM64. The x86/x64 implementation is null because VMbus interrupts on x86/x64 don't use an IRQ. Signed-off-by: Michael Kelley <mikelley@microsoft.com> Reviewed-by: James Morris <james.morris@microsoft.com> --- arch/x86/include/asm/mshyperv.h | 4 ++++ drivers/hv/hv.c | 2 ++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 3cd1431..27d0e83 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -101,6 +101,10 @@ static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type) void hv_setup_vmbus_irq(void (*handler)(void)); void hv_remove_vmbus_irq(void); +/* On x86/x64, there isn't a real IRQ to be enabled/disable */ +static inline void hv_enable_vmbus_irq(void) {} +static inline void hv_disable_vmbus_irq(void) {} + void hv_setup_kexec_handler(void (*handler)(void)); void hv_remove_kexec_handler(void); void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs)); diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index 658dc76..15f835a 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -298,6 +298,7 @@ int hv_synic_init(unsigned int cpu) hv_set_siefp(siefp.as_uint64); /* Setup the shared SINT. */ + hv_enable_vmbus_irq(); hv_get_synint_state(HV_X64_MSR_SINT0 + VMBUS_MESSAGE_SINT, shared_sint.as_uint64); @@ -431,6 +432,7 @@ int hv_synic_cleanup(unsigned int cpu) hv_get_synic_state(sctrl.as_uint64); sctrl.enable = 0; hv_set_synic_state(sctrl.as_uint64); + hv_disable_vmbus_irq(); return 0; } -- 1.8.3.1
Update drivers/hv/Kconfig so CONFIG_HYPERV can be selected on ARM64, causing the Hyper-V specific code to be built. Signed-off-by: Michael Kelley <mikelley@microsoft.com> Reviewed-by: James Morris <james.morris@microsoft.com> --- drivers/hv/Kconfig | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig index 97954f5..c3e11a2 100644 --- a/drivers/hv/Kconfig +++ b/drivers/hv/Kconfig @@ -4,7 +4,8 @@ menu "Microsoft Hyper-V guest support" config HYPERV tristate "Microsoft Hyper-V client drivers" - depends on X86 && ACPI && PCI && X86_LOCAL_APIC && HYPERVISOR_GUEST + depends on ACPI && PCI && \ + ((X86 && X86_LOCAL_APIC && HYPERVISOR_GUEST) || ARM64) select PARAVIRT help Select this option to run Linux as a Hyper-V client operating -- 1.8.3.1
Hi,
On Tue, Jun 19, 2018 at 02:23:11PM -0700, Michael Kelley wrote:
> Add slow_virt_to_phys() function for ARM64 that parallels the same
> function on x86/x64. This is needed by the architecture independent
> Hyper-V VMbus driver at drivers/hv/channel.c. The implementation
> directly translates the virtual address using the ARM64 'at'
> instruction.
>
> Signed-off-by: Michael Kelley <mikelley@microsoft.com>
> Reviewed-by: James Morris <james.morris@microsoft.com>
> ---
> arch/arm64/include/asm/memory.h | 6 ++++++
> arch/arm64/mm/pageattr.c | 37 +++++++++++++++++++++++++++++++++++++
> 2 files changed, 43 insertions(+)
>
> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
> index 49d9921..3f68dcf 100644
> --- a/arch/arm64/include/asm/memory.h
> +++ b/arch/arm64/include/asm/memory.h
> @@ -265,6 +265,12 @@ static inline void *phys_to_virt(phys_addr_t x)
> }
>
> /*
> + * For memory where the underlying physical pages may not
> + * be contiguous.
> + */
> +extern phys_addr_t slow_virt_to_phys(void *vaddr);
> +
> +/*
> * Drivers should NOT use these either.
> */
> #define __pa(x) __virt_to_phys((unsigned long)(x))
> diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c
> index a563593..8a42cac 100644
> --- a/arch/arm64/mm/pageattr.c
> +++ b/arch/arm64/mm/pageattr.c
> @@ -19,6 +19,8 @@
> #include <asm/pgtable.h>
> #include <asm/set_memory.h>
> #include <asm/tlbflush.h>
> +#include <asm/sysreg.h>
> +#include <asm/barrier.h>
>
> struct page_change_data {
> pgprot_t set_mask;
> @@ -185,3 +187,38 @@ bool kernel_page_present(struct page *page)
> }
> #endif /* CONFIG_HIBERNATION */
> #endif /* CONFIG_DEBUG_PAGEALLOC */
> +
> +/*
> + * For virtual addresses where the underlyine physical memory may not be
> + * contiguous and the normal virt_to_phys gives the wrong result. This
> + * function does an actual translation using the 'at' instruction.
> + */
> +phys_addr_t slow_virt_to_phys(void *virt_addr)
> +{
> + phys_addr_t result;
> + unsigned long input = (unsigned long)virt_addr;
> + char touch;
> + int i;
> +
> + /* Try up to 3 times (an arbitrary number) */
> + for (i = 0; i < 3; i++) {
> + /* Do the translation and check that it worked */
> + asm volatile("at s1e1r, %0" : : "r" (input));
> + isb();
> + result = read_sysreg(par_el1);
> + if (likely(!(result & 0x1)))
> + return (result & GENMASK_ULL(51, 12)) |
> + (input & GENMASK_ULL(11, 0));
> + /*
> + * Something failed. Read the page to fault in anything
> + * that isn't resident, then try again. "Anything"
> + * could include the page itself or hypervisor page tables.
> + */
> + touch = READ_ONCE(*(char *)input);
> + dmb(sy);
> + }
> +
> + /* Let the caller sort it out. */
> + return -1;
AFAICT, callers of slow_virt_to_phys() don't check the value, so this doesn't
seem safe, and I'm concerned by the possibility of spurious failures given we
only retry a fixed number of times.
I think it would be better to walk the page tables, as x86 does, as that's
guaranteed to complete within a fixed bound (i.e. after we read all the levls
of table).
Thanks,
Mark.