Linux-HyperV Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor
@ 2020-09-14 11:27 Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 01/18] asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT Wei Liu
                   ` (17 more replies)
  0 siblings, 18 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu, x86,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Samuel Ortiz

Hi all

Here we propose this patch series to make Linux run as the root partition [0]
on Microsoft Hypervisor [1].

Microsoft wants to create a complete virtualization stack with Linux and
Microsoft Hypervisor. There will be a subsequent patch series to provide a
device node (/dev/mshv) such that userspace programs can create and run virtual
machines. We've also ported Cloud Hypervisor [3] over and have been able to
boot a Linux guest with Virtio devices since late July.

Being an RFC sereis, this implements only the absolutely necessary
components to get things running.  I will break down this series a bit.

A large portion of this series consists of patches that augment hyperv-tlfs.h.
They should be rather uncontroversial and can be applied right away.

A few key things other than the changes to hyperv-tlfs.h:

1. Linux needs to setup existing Hyper-V facilities differently.
2. Linux needs to make a few hypercalls to bring up APs.
3. Interrupts are remapped by IOMMU, which is controlled by the hypervisor.
   Linux needs to make hypercalls to map and unmap interrupts. This is
   done by introducing a new MSI irqdomain and new irqchips.

#3 is perhaps the thing that we feel least confident about. We drew inspiration
from the Xen code in Linux. We are of course open to criticism and sugguestions
on how to make it better / acceptable to upstream.

We're aware of tglx's series to change some of the MSI code, so we may need to
change some of the code after that series is upstreamed. But it wouldn't hurt
to throw this out as soon as possible for feedback.

Comments and suggestions are welcome.

Thanks,
Wei.

[0] Just think of it like Xen's Dom0.
[1] Hyper-V is more well-known, but it really refers to the whole stack
    including the hypervisor and other components that run in Windows kernel
    and userspace.
[3] https://github.com/cloud-hypervisor/

Cc: x86@kernel.org
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Sunil Muthuswamy <sunilmut@microsoft.com>
Cc: Nuno Das Neves <nudasnev@microsoft.com>
Cc: Vineeth Pillai <viremana@linux.microsoft.com>
Cc: Samuel Ortiz <sameo@linux.intel.com>

Wei Liu (18):
  asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to
    HV_CPU_MANAGEMENT
  x86/hyperv: detect if Linux is the root partition
  Drivers: hv: vmbus: skip VMBus initialization if Linux is root
  iommu/hyperv: don't setup IRQ remapping when running as root
  clocksource/hyperv: use MSR-based access if running as root
  x86/hyperv: allocate output arg pages if required
  x86/hyperv: extract partition ID from Microsoft Hypervisor if
    necessary
  x86/hyperv: handling hypercall page setup for root
  x86/hyperv: provide a bunch of helper functions
  x86/hyperv: implement and use hv_smp_prepare_cpus
  asm-generic/hyperv: update hv_msi_entry
  asm-generic/hyperv: update hv_interrupt_entry
  asm-generic/hyperv: introduce hv_device_id and auxiliary structures
  asm-generic/hyperv: import data structures for mapping device
    interrupts
  x86/apic/msi: export pci_msi_get_hwirq
  x86/hyperv: implement MSI domain for root partition
  x86/ioapic: export a few functions and data structures via io_apic.h
  x86/hyperv: handle IO-APIC when running as root

 arch/x86/hyperv/Makefile            |   4 +-
 arch/x86/hyperv/hv_init.c           | 126 +++++-
 arch/x86/hyperv/hv_proc.c           | 209 ++++++++++
 arch/x86/hyperv/irqdomain.c         | 580 ++++++++++++++++++++++++++++
 arch/x86/include/asm/hyperv-tlfs.h  |  23 ++
 arch/x86/include/asm/io_apic.h      |  20 +
 arch/x86/include/asm/mshyperv.h     |  13 +-
 arch/x86/include/asm/msi.h          |   3 +
 arch/x86/kernel/apic/io_apic.c      |  28 +-
 arch/x86/kernel/apic/msi.c          |   3 +-
 arch/x86/kernel/cpu/mshyperv.c      |  43 +++
 drivers/clocksource/hyperv_timer.c  |   3 +
 drivers/hv/vmbus_drv.c              |   3 +
 drivers/iommu/hyperv-iommu.c        |   3 +-
 drivers/pci/controller/pci-hyperv.c |   2 +-
 include/asm-generic/hyperv-tlfs.h   | 243 +++++++++++-
 16 files changed, 1268 insertions(+), 38 deletions(-)
 create mode 100644 arch/x86/hyperv/hv_proc.c
 create mode 100644 arch/x86/hyperv/irqdomain.c

-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 01/18] asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 02/18] x86/hyperv: detect if Linux is the root partition Wei Liu
                   ` (16 subsequent siblings)
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Arnd Bergmann, open list:GENERIC INCLUDE/ASM HEADER FILES

This makes the name match Hyper-V TLFS.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 include/asm-generic/hyperv-tlfs.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index e73a11850055..e6903589a82a 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -88,7 +88,7 @@
 #define HV_CONNECT_PORT				BIT(7)
 #define HV_ACCESS_STATS				BIT(8)
 #define HV_DEBUGGING				BIT(11)
-#define HV_CPU_POWER_MANAGEMENT			BIT(12)
+#define HV_CPU_MANAGEMENT			BIT(12)
 
 
 /*
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 02/18] x86/hyperv: detect if Linux is the root partition
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 01/18] asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 03/18] Drivers: hv: vmbus: skip VMBus initialization if Linux is root Wei Liu
                   ` (15 subsequent siblings)
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

For now we can use the privilege flag to check. Stash the value to be
used later.

Put in a bunch of defines for future use when we want to have more
fine-grained detection.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/hv_init.c          |  4 ++++
 arch/x86/include/asm/hyperv-tlfs.h | 10 ++++++++++
 arch/x86/include/asm/mshyperv.h    |  2 ++
 arch/x86/kernel/cpu/mshyperv.c     | 16 ++++++++++++++++
 4 files changed, 32 insertions(+)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 6035df1b49e1..cac8e4c56261 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -26,6 +26,10 @@
 #include <linux/syscore_ops.h>
 #include <clocksource/hyperv_timer.h>
 
+/* Is Linux running as the root partition? */
+bool hv_root_partition;
+EXPORT_SYMBOL_GPL(hv_root_partition);
+
 void *hv_hypercall_pg;
 EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 
diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index 7a4d2062385c..20d628c1ed50 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -21,6 +21,7 @@
 #define HYPERV_CPUID_FEATURES			0x40000003
 #define HYPERV_CPUID_ENLIGHTMENT_INFO		0x40000004
 #define HYPERV_CPUID_IMPLEMENT_LIMITS		0x40000005
+#define HYPERV_CPUID_CPU_MANAGEMENT_FEATURES	0x40000007
 #define HYPERV_CPUID_NESTED_FEATURES		0x4000000A
 
 #define HYPERV_HYPERVISOR_PRESENT_BIT		0x80000000
@@ -136,6 +137,15 @@
 /* Recommend using enlightened VMCS */
 #define HV_X64_ENLIGHTENED_VMCS_RECOMMENDED		BIT(14)
 
+/*
+ * CPU management features identification.
+ * These are HYPERV_CPUID_CPU_MANAGEMENT_FEATURES.EAX bits.
+ */
+#define HV_X64_START_LOGICAL_PROCESSOR			BIT(0)
+#define HV_X64_CREATE_ROOT_VIRTUAL_PROCESSOR		BIT(1)
+#define HV_X64_PERFORMANCE_COUNTER_SYNC			BIT(2)
+#define HV_X64_RESERVED_IDENTITY_BIT			BIT(31)
+
 /*
  * Virtual processor will never share a physical core with another virtual
  * processor, except for virtual processors that are reported as sibling SMT
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 60b944dd2df1..2a2cc81beac6 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -224,6 +224,8 @@ int hyperv_fill_flush_guest_mapping_list(
 		struct hv_guest_mapping_flush_list *flush,
 		u64 start_gfn, u64 end_gfn);
 
+extern bool hv_root_partition;
+
 #ifdef CONFIG_X86_64
 void hv_apic_init(void);
 void __init hv_init_spinlocks(void);
diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index af94f05a5c66..1bf57d310f78 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -232,6 +232,22 @@ static void __init ms_hyperv_init_platform(void)
 	pr_debug("Hyper-V: max %u virtual processors, %u logical processors\n",
 		 ms_hyperv.max_vp_index, ms_hyperv.max_lp_index);
 
+	/*
+	 * Check CPU management privilege.
+	 *
+	 * To mirror what Windows does we should extract CPU management
+	 * features and use the ReservedIdentityBit to detect if Linux is the
+	 * root partition. But that requires negotiating CPU management
+	 * interface (a process to be finalized).
+	 *
+	 * For now, use the privilege flag as the indicator for running as
+	 * root.
+	 */
+	if (cpuid_ebx(HYPERV_CPUID_FEATURES) & HV_CPU_MANAGEMENT) {
+		hv_root_partition = true;
+		pr_info("Hyper-V: running as root partition\n");
+	}
+
 	/*
 	 * Extract host information.
 	 */
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 03/18] Drivers: hv: vmbus: skip VMBus initialization if Linux is root
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 01/18] asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 02/18] x86/hyperv: detect if Linux is the root partition Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-14 11:27 ` [PATCH RFC v1 04/18] iommu/hyperv: don't setup IRQ remapping when running as root Wei Liu
                   ` (14 subsequent siblings)
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger

There is no VMBus and the other infrastructures initialized in
hv_acpi_init when Linux is running as the root partition.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 drivers/hv/vmbus_drv.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index d69f4efa3719..da4a5660f915 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -2586,6 +2586,9 @@ static int __init hv_acpi_init(void)
 	if (!hv_is_hyperv_initialized())
 		return -ENODEV;
 
+	if (hv_root_partition)
+		return -ENODEV;
+
 	init_completion(&probe_event);
 
 	/*
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 04/18] iommu/hyperv: don't setup IRQ remapping when running as root
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (2 preceding siblings ...)
  2020-09-14 11:27 ` [PATCH RFC v1 03/18] Drivers: hv: vmbus: skip VMBus initialization if Linux is root Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-18  9:12   ` Joerg Roedel
  2020-09-14 11:27 ` [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if " Wei Liu
                   ` (13 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Joerg Roedel,
	open list:IOMMU DRIVERS

The IOMMU code needs more work. We're sure for now the IRQ remapping
hooks are not applicable when Linux is the root.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 drivers/iommu/hyperv-iommu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/hyperv-iommu.c b/drivers/iommu/hyperv-iommu.c
index 8919c1c70b68..4da684ab292c 100644
--- a/drivers/iommu/hyperv-iommu.c
+++ b/drivers/iommu/hyperv-iommu.c
@@ -20,6 +20,7 @@
 #include <asm/io_apic.h>
 #include <asm/irq_remapping.h>
 #include <asm/hypervisor.h>
+#include <asm/mshyperv.h>
 
 #include "irq_remapping.h"
 
@@ -143,7 +144,7 @@ static int __init hyperv_prepare_irq_remapping(void)
 	int i;
 
 	if (!hypervisor_is_type(X86_HYPER_MS_HYPERV) ||
-	    !x2apic_supported())
+	    !x2apic_supported() || hv_root_partition)
 		return -ENODEV;
 
 	fn = irq_domain_alloc_named_id_fwnode("HYPERV-IR", 0);
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if running as root
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (3 preceding siblings ...)
  2020-09-14 11:27 ` [PATCH RFC v1 04/18] iommu/hyperv: don't setup IRQ remapping when running as root Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-15 10:10   ` Vitaly Kuznetsov
  2020-09-14 11:27 ` [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required Wei Liu
                   ` (12 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Daniel Lezcano, Thomas Gleixner

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 drivers/clocksource/hyperv_timer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
index 09aa44cb8a91..fe96082ce85e 100644
--- a/drivers/clocksource/hyperv_timer.c
+++ b/drivers/clocksource/hyperv_timer.c
@@ -426,6 +426,9 @@ static bool __init hv_init_tsc_clocksource(void)
 	if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE))
 		return false;
 
+	if (hv_root_partition)
+		return false;
+
 	hv_read_reference_counter = read_hv_clock_tsc;
 	phys_addr = virt_to_phys(hv_get_tsc_page());
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (4 preceding siblings ...)
  2020-09-14 11:27 ` [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if " Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-15 10:16   ` Vitaly Kuznetsov
  2020-09-14 11:27 ` [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary Wei Liu
                   ` (11 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

When Linux runs as the root partition, it will need to make hypercalls
which return data from the hypervisor.

Allocate pages for storing results when Linux runs as the root
partition.

Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/hv_init.c       | 45 +++++++++++++++++++++++++++++----
 arch/x86/include/asm/mshyperv.h |  1 +
 2 files changed, 41 insertions(+), 5 deletions(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index cac8e4c56261..ebba4be4185d 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -45,6 +45,9 @@ EXPORT_SYMBOL_GPL(hv_vp_assist_page);
 void  __percpu **hyperv_pcpu_input_arg;
 EXPORT_SYMBOL_GPL(hyperv_pcpu_input_arg);
 
+void  __percpu **hyperv_pcpu_output_arg;
+EXPORT_SYMBOL_GPL(hyperv_pcpu_output_arg);
+
 u32 hv_max_vp_index;
 EXPORT_SYMBOL_GPL(hv_max_vp_index);
 
@@ -75,14 +78,29 @@ static int hv_cpu_init(unsigned int cpu)
 	u64 msr_vp_index;
 	struct hv_vp_assist_page **hvp = &hv_vp_assist_page[smp_processor_id()];
 	void **input_arg;
-	struct page *pg;
+	struct page *input_pg;
 
 	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
 	/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */
-	pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
-	if (unlikely(!pg))
+	input_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
+	if (unlikely(!input_pg))
 		return -ENOMEM;
-	*input_arg = page_address(pg);
+	*input_arg = page_address(input_pg);
+
+	if (hv_root_partition) {
+		struct page *output_pg;
+		void **output_arg;
+
+		output_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
+		if (unlikely(!output_pg)) {
+			free_page((unsigned long)*input_arg);
+			*input_arg = NULL;
+			return -ENOMEM;
+		}
+
+		output_arg = (void **)this_cpu_ptr(hyperv_pcpu_output_arg);
+		*output_arg = page_address(output_pg);
+	}
 
 	hv_get_vp_index(msr_vp_index);
 
@@ -209,14 +227,25 @@ static int hv_cpu_die(unsigned int cpu)
 	unsigned int new_cpu;
 	unsigned long flags;
 	void **input_arg;
-	void *input_pg = NULL;
+	void *input_pg = NULL, *output_pg = NULL;
 
 	local_irq_save(flags);
 	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
 	input_pg = *input_arg;
 	*input_arg = NULL;
+
+	if (hv_root_partition) {
+		void **output_arg;
+
+		output_arg = (void **)this_cpu_ptr(hyperv_pcpu_output_arg);
+		output_pg = *output_arg;
+		*output_arg = NULL;
+	}
+
 	local_irq_restore(flags);
+
 	free_page((unsigned long)input_pg);
+	free_page((unsigned long)output_pg);
 
 	if (hv_vp_assist_page && hv_vp_assist_page[cpu])
 		wrmsrl(HV_X64_MSR_VP_ASSIST_PAGE, 0);
@@ -350,6 +379,12 @@ void __init hyperv_init(void)
 
 	BUG_ON(hyperv_pcpu_input_arg == NULL);
 
+	/* Allocate the per-CPU state for output arg for root */
+	if (hv_root_partition) {
+		hyperv_pcpu_output_arg = alloc_percpu(void  *);
+		BUG_ON(hyperv_pcpu_output_arg == NULL);
+	}
+
 	/* Allocate percpu VP index */
 	hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
 				    GFP_KERNEL);
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 2a2cc81beac6..f5c62140f28d 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -63,6 +63,7 @@ static inline void hv_disable_stimer0_percpu_irq(int irq) {}
 #if IS_ENABLED(CONFIG_HYPERV)
 extern void *hv_hypercall_pg;
 extern void  __percpu  **hyperv_pcpu_input_arg;
+extern void  __percpu  **hyperv_pcpu_output_arg;
 
 static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
 {
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (5 preceding siblings ...)
  2020-09-14 11:27 ` [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-15 10:27   ` Vitaly Kuznetsov
  2020-09-14 11:27 ` [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root Wei Liu
                   ` (10 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

We will need the partition ID for executing some hypercalls later.

Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/hv_init.c         | 26 ++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h   |  2 ++
 include/asm-generic/hyperv-tlfs.h |  6 ++++++
 3 files changed, 34 insertions(+)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index ebba4be4185d..0eec1ed32023 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -30,6 +30,9 @@
 bool hv_root_partition;
 EXPORT_SYMBOL_GPL(hv_root_partition);
 
+u64 hv_current_partition_id;
+EXPORT_SYMBOL_GPL(hv_current_partition_id);
+
 void *hv_hypercall_pg;
 EXPORT_SYMBOL_GPL(hv_hypercall_pg);
 
@@ -345,6 +348,26 @@ static struct syscore_ops hv_syscore_ops = {
 	.resume		= hv_resume,
 };
 
+void __init hv_get_partition_id(void)
+{
+	struct hv_get_partition_id *output_page;
+	int status;
+	unsigned long flags;
+
+	local_irq_save(flags);
+	output_page = *this_cpu_ptr(hyperv_pcpu_output_arg);
+	status = hv_do_hypercall(HVCALL_GET_PARTITION_ID, NULL, output_page) &
+		HV_HYPERCALL_RESULT_MASK;
+	if (status != HV_STATUS_SUCCESS)
+		pr_err("Failed to get partition ID: %d\n", status);
+	else
+		hv_current_partition_id = output_page->partition_id;
+	local_irq_restore(flags);
+
+	/* No point in proceeding if this failed */
+	BUG_ON(status != HV_STATUS_SUCCESS);
+}
+
 /*
  * This function is to be invoked early in the boot sequence after the
  * hypervisor has been detected.
@@ -440,6 +463,9 @@ void __init hyperv_init(void)
 
 	register_syscore_ops(&hv_syscore_ops);
 
+	if (hv_root_partition)
+		hv_get_partition_id();
+
 	return;
 
 remove_cpuhp_state:
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index f5c62140f28d..4039302e0ae9 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -65,6 +65,8 @@ extern void *hv_hypercall_pg;
 extern void  __percpu  **hyperv_pcpu_input_arg;
 extern void  __percpu  **hyperv_pcpu_output_arg;
 
+extern u64 hv_current_partition_id;
+
 static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
 {
 	u64 input_address = input ? virt_to_phys(input) : 0;
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index e6903589a82a..87b1a79b19eb 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -141,6 +141,7 @@ struct ms_hyperv_tsc_page {
 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX	0x0013
 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX	0x0014
 #define HVCALL_SEND_IPI_EX			0x0015
+#define HVCALL_GET_PARTITION_ID			0x0046
 #define HVCALL_GET_VP_REGISTERS			0x0050
 #define HVCALL_SET_VP_REGISTERS			0x0051
 #define HVCALL_POST_MESSAGE			0x005c
@@ -407,6 +408,11 @@ struct hv_tlb_flush_ex {
 	u64 gva_list[];
 } __packed;
 
+/* HvGetPartitionId hypercall (output only) */
+struct hv_get_partition_id {
+	u64 partition_id;
+} __packed;
+
 /* HvRetargetDeviceInterrupt hypercall */
 union hv_msi_entry {
 	u64 as_uint64;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (6 preceding siblings ...)
  2020-09-14 11:27 ` [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary Wei Liu
@ 2020-09-14 11:27 ` Wei Liu
  2020-09-15 10:32   ` Vitaly Kuznetsov
  2020-09-14 11:59 ` [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions Wei Liu
                   ` (9 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:27 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

When Linux is running as the root partition, the hypercall page will
have already been setup by Hyper-V. Copy the content over to the
allocated page.

The suspend, resume and cleanup paths remain untouched because they are
not supported in this setup yet.

Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 0eec1ed32023..26233aebc86c 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -25,6 +25,7 @@
 #include <linux/cpuhotplug.h>
 #include <linux/syscore_ops.h>
 #include <clocksource/hyperv_timer.h>
+#include <linux/highmem.h>
 
 /* Is Linux running as the root partition? */
 bool hv_root_partition;
@@ -448,8 +449,29 @@ void __init hyperv_init(void)
 
 	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 	hypercall_msr.enable = 1;
-	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
-	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+
+	if (hv_root_partition) {
+		struct page *pg;
+		void *src, *dst;
+
+		/*
+		 * Order is important here. We must enable the hypercall page
+		 * so it is populated with code, then copy the code to an
+		 * executable page.
+		 */
+		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+
+		pg = vmalloc_to_page(hv_hypercall_pg);
+		dst = kmap(pg);
+		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
+				MEMREMAP_WB);
+		memcpy(dst, src, PAGE_SIZE);
+		memunmap(src);
+		kunmap(pg);
+	} else {
+		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
+		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+	}
 
 	/*
 	 * Ignore any errors in setting up stimer clockevents
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (7 preceding siblings ...)
  2020-09-14 11:27 ` [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-15 11:00   ` Vitaly Kuznetsov
  2020-09-14 11:59 ` [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus Wei Liu
                   ` (8 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

They are used to deposit pages into Microsoft Hypervisor and bring up
logical and virtual processors.

Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/Makefile          |   2 +-
 arch/x86/hyperv/hv_proc.c         | 209 ++++++++++++++++++++++++++++++
 arch/x86/include/asm/mshyperv.h   |   4 +
 include/asm-generic/hyperv-tlfs.h |  56 ++++++++
 4 files changed, 270 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/hyperv/hv_proc.c

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 89b1f74d3225..565358020921 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-y			:= hv_init.o mmu.o nested.o
-obj-$(CONFIG_X86_64)	+= hv_apic.o
+obj-$(CONFIG_X86_64)	+= hv_apic.o hv_proc.o
 
 ifdef CONFIG_X86_64
 obj-$(CONFIG_PARAVIRT_SPINLOCKS)	+= hv_spinlock.o
diff --git a/arch/x86/hyperv/hv_proc.c b/arch/x86/hyperv/hv_proc.c
new file mode 100644
index 000000000000..847c72465d0e
--- /dev/null
+++ b/arch/x86/hyperv/hv_proc.c
@@ -0,0 +1,209 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/types.h>
+#include <linux/version.h>
+#include <linux/vmalloc.h>
+#include <linux/mm.h>
+#include <linux/clockchips.h>
+#include <linux/acpi.h>
+#include <linux/hyperv.h>
+#include <linux/slab.h>
+#include <linux/cpuhotplug.h>
+#include <asm/hypervisor.h>
+#include <asm/mshyperv.h>
+#include <asm/apic.h>
+
+#include <asm/trace/hyperv.h>
+
+#define HV_DEPOSIT_MAX_ORDER (8)
+#define HV_DEPOSIT_MAX (1 << HV_DEPOSIT_MAX_ORDER)
+
+#define MAX(a, b) ((a) > (b) ? (a) : (b))
+#define MIN(a, b) ((a) < (b) ? (a) : (b))
+
+/*
+ * Deposits exact number of pages
+ * Must be called with interrupts enabled
+ * Max 256 pages
+ */
+int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages)
+{
+	struct page **pages;
+	int *counts;
+	int num_allocations;
+	int i, j, page_count;
+	int order;
+	int desired_order;
+	int status;
+	int ret;
+	u64 base_pfn;
+	struct hv_deposit_memory *input_page;
+	unsigned long flags;
+
+	if (num_pages > HV_DEPOSIT_MAX)
+		return -EINVAL;
+	if (!num_pages)
+		return 0;
+
+	ret = -ENOMEM;
+
+	/* One buffer for page pointers and counts */
+	pages = page_address(alloc_page(GFP_KERNEL));
+	if (!pages)
+		goto free_buf;
+	counts = (int *)&pages[256];
+
+	/* Allocate all the pages before disabling interrupts */
+	num_allocations = 0;
+	i = 0;
+	order = HV_DEPOSIT_MAX_ORDER;
+
+	while (num_pages) {
+		/* Find highest order we can actually allocate */
+		desired_order = 31 - __builtin_clz(num_pages);
+		order = MIN(desired_order, order);
+		do {
+			pages[i] = alloc_pages_node(node, GFP_KERNEL, order);
+			if (!pages[i]) {
+				if (!order) {
+					goto err_free_allocations;
+				}
+				--order;
+			}
+		} while (!pages[i]);
+
+		split_page(pages[i], order);
+		counts[i] = 1 << order;
+		num_pages -= counts[i];
+		i++;
+		num_allocations++;
+	}
+
+	local_irq_save(flags);
+
+	input_page = *this_cpu_ptr(hyperv_pcpu_input_arg);
+
+	input_page->partition_id = partition_id;
+
+	/* Populate gpa_page_list - these will fit on the input page */
+	for (i = 0, page_count = 0; i < num_allocations; ++i) {
+		base_pfn = page_to_pfn(pages[i]);
+		for (j = 0; j < counts[i]; ++j, ++page_count)
+			input_page->gpa_page_list[page_count] = base_pfn + j;
+	}
+	status = hv_do_rep_hypercall(HVCALL_DEPOSIT_MEMORY,
+				     page_count, 0, input_page,
+				     NULL) & HV_HYPERCALL_RESULT_MASK;
+	local_irq_restore(flags);
+
+	if (status != HV_STATUS_SUCCESS) {
+		pr_err("Failed to deposit pages: %d\n", status);
+		ret = status;
+		goto err_free_allocations;
+	}
+
+	ret = 0;
+	goto free_buf;
+
+err_free_allocations:
+	for (i = 0; i < num_allocations; ++i) {
+		base_pfn = page_to_pfn(pages[i]);
+		for (j = 0; j < counts[i]; ++j)
+			__free_page(pfn_to_page(base_pfn + j));
+	}
+
+free_buf:
+	free_page((unsigned long)pages);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hv_call_deposit_pages);
+
+int hv_call_add_logical_proc(int node, u32 lp_index, u32 apic_id)
+{
+	struct hv_add_logical_processor_in *input;
+	struct hv_add_logical_processor_out *output;
+	int status;
+	unsigned long flags;
+	int ret = 0;
+
+	do {
+		local_irq_save(flags);
+
+		input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+		/* We don't do anything with the output right now */
+		output = *this_cpu_ptr(hyperv_pcpu_output_arg);
+
+		input->lp_index = lp_index;
+		input->apic_id = apic_id;
+		input->flags = 0;
+		input->proximity_domain_info.domain_id = node_to_pxm(node);
+		input->proximity_domain_info.flags.reserved = 0;
+		input->proximity_domain_info.flags.proximity_info_valid = 1;
+		input->proximity_domain_info.flags.proximity_preferred = 1;
+		status = hv_do_hypercall(HVCALL_ADD_LOGICAL_PROCESSOR,
+					 input, output);
+		local_irq_restore(flags);
+
+		if (status != HV_STATUS_INSUFFICIENT_MEMORY) {
+			if (status != HV_STATUS_SUCCESS) {
+				pr_err("%s: cpu %u apic ID %u, %d\n", __func__,
+				       lp_index, apic_id, status);
+				ret = status;
+			}
+			break;
+		}
+		ret = hv_call_deposit_pages(node, hv_current_partition_id, 1);
+
+	} while (!ret);
+
+	return ret;
+}
+
+int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags)
+{
+	struct hv_create_vp *input;
+	int status;
+	unsigned long irq_flags;
+	int ret = 0;
+
+	/* Root VPs don't seem to need pages deposited */
+	if (partition_id != hv_current_partition_id) {
+		ret = hv_call_deposit_pages(node, partition_id, 90);
+		if (ret)
+			return ret;
+	}
+
+	do {
+		local_irq_save(irq_flags);
+
+		input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+
+		input->partition_id = partition_id;
+		input->vp_index = vp_index;
+		input->flags = flags;
+		if (node != NUMA_NO_NODE) {
+			input->proximity_domain_info.domain_id = node_to_pxm(node);
+			input->proximity_domain_info.flags.reserved = 0;
+			input->proximity_domain_info.flags.proximity_info_valid = 1;
+			input->proximity_domain_info.flags.proximity_preferred = 1;
+		} else {
+			input->proximity_domain_info.as_uint64 = 0;
+		}
+		status = hv_do_hypercall(HVCALL_CREATE_VP, input, NULL);
+		local_irq_restore(irq_flags);
+
+		if (status != HV_STATUS_INSUFFICIENT_MEMORY) {
+			if (status != HV_STATUS_SUCCESS) {
+				pr_err("%s: vcpu %u, lp %u, %d\n", __func__,
+				       vp_index, flags, status);
+				ret = status;
+			}
+			break;
+		}
+		ret = hv_call_deposit_pages(node, partition_id, 1);
+
+	} while (!ret);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(hv_call_create_vp);
+
diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 4039302e0ae9..60afc3e417d0 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -67,6 +67,10 @@ extern void  __percpu  **hyperv_pcpu_output_arg;
 
 extern u64 hv_current_partition_id;
 
+int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
+int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
+int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags);
+
 static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
 {
 	u64 input_address = input ? virt_to_phys(input) : 0;
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index 87b1a79b19eb..2b05bed712c0 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -142,6 +142,8 @@ struct ms_hyperv_tsc_page {
 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX	0x0014
 #define HVCALL_SEND_IPI_EX			0x0015
 #define HVCALL_GET_PARTITION_ID			0x0046
+#define HVCALL_DEPOSIT_MEMORY			0x0048
+#define HVCALL_CREATE_VP			0x004e
 #define HVCALL_GET_VP_REGISTERS			0x0050
 #define HVCALL_SET_VP_REGISTERS			0x0051
 #define HVCALL_POST_MESSAGE			0x005c
@@ -149,6 +151,7 @@ struct ms_hyperv_tsc_page {
 #define HVCALL_POST_DEBUG_DATA			0x0069
 #define HVCALL_RETRIEVE_DEBUG_DATA		0x006a
 #define HVCALL_RESET_DEBUG_SESSION		0x006b
+#define HVCALL_ADD_LOGICAL_PROCESSOR		0x0076
 #define HVCALL_RETARGET_INTERRUPT		0x007e
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
@@ -413,6 +416,59 @@ struct hv_get_partition_id {
 	u64 partition_id;
 } __packed;
 
+/* HvDepositMemory hypercall */
+struct hv_deposit_memory {
+	u64 partition_id;
+	u64 gpa_page_list[];
+};
+
+
+struct hv_proximity_domain_flags {
+	u32 proximity_preferred : 1;
+	u32 reserved : 30;
+	u32 proximity_info_valid : 1;
+};
+
+/* Not a union in windows but useful for zeroing */
+union hv_proximity_domain_info {
+	struct {
+		u32 domain_id;
+		struct hv_proximity_domain_flags flags;
+	};
+	u64 as_uint64;
+};
+
+struct hv_lp_startup_status {
+	u64 hv_status;
+	u64 substatus1;
+	u64 substatus2;
+	u64 substatus3;
+	u64 substatus4;
+	u64 substatus5;
+	u64 substatus6;
+};
+
+/* HvAddLogicalProcessor hypercalls */
+struct hv_add_logical_processor_in {
+	u32 lp_index;
+	u32 apic_id;
+	union hv_proximity_domain_info proximity_domain_info;
+	u64 flags;
+};
+
+struct hv_add_logical_processor_out {
+	struct hv_lp_startup_status startup_status;
+};
+
+/* HvCreateVp hypercall */
+struct hv_create_vp {
+	u64 partition_id;
+	u32 vp_index;
+	u32 padding;
+	union hv_proximity_domain_info proximity_domain_info;
+	u64 flags;
+};
+
 /* HvRetargetDeviceInterrupt hypercall */
 union hv_msi_entry {
 	u64 as_uint64;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (8 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-15 11:14   ` Vitaly Kuznetsov
  2020-09-14 11:59 ` [PATCH RFC v1 11/18] asm-generic/hyperv: update hv_msi_entry Wei Liu
                   ` (7 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Microsoft Hypervisor requires the root partition to make a few
hypercalls to setup application processors before they can be used.

Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
CPU hotplug and unplug is not yet supported in this setup, so those
paths remain untouched.
---
 arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
index 1bf57d310f78..7522cae02759 100644
--- a/arch/x86/kernel/cpu/mshyperv.c
+++ b/arch/x86/kernel/cpu/mshyperv.c
@@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
 	hv_init_spinlocks();
 #endif
 }
+
+static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
+{
+#if defined(CONFIG_X86_64)
+	int i;
+	int vp_index = 1;
+	int ret;
+
+	native_smp_prepare_cpus(max_cpus);
+
+	for_each_present_cpu(i) {
+		if (i == 0)
+			continue;
+		ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
+		BUG_ON(ret);
+	}
+
+	for_each_present_cpu(i) {
+		if (i == 0)
+			continue;
+		ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);
+		BUG_ON(ret);
+	}
+#endif
+}
 #endif
 
 static void __init ms_hyperv_init_platform(void)
@@ -359,6 +384,8 @@ static void __init ms_hyperv_init_platform(void)
 
 # ifdef CONFIG_SMP
 	smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu;
+	if (hv_root_partition)
+		smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
 # endif
 
 	/*
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 11/18] asm-generic/hyperv: update hv_msi_entry
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (9 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-14 11:59 ` [PATCH RFC v1 12/18] asm-generic/hyperv: update hv_interrupt_entry Wei Liu
                   ` (6 subsequent siblings)
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

We will soon need to access fields inside the MSI address and MSI data
fields. Introduce hv_msi_address_register and hv_msi_data_register.

Fix up one user of hv_msi_entry in mshyperv.h.

No functional change expected.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/include/asm/mshyperv.h   |  4 ++--
 include/asm-generic/hyperv-tlfs.h | 28 ++++++++++++++++++++++++++--
 2 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index 60afc3e417d0..a4d46ca5a0b1 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -244,8 +244,8 @@ static inline void hv_apic_init(void) {}
 static inline void hv_set_msi_entry_from_desc(union hv_msi_entry *msi_entry,
 					      struct msi_desc *msi_desc)
 {
-	msi_entry->address = msi_desc->msg.address_lo;
-	msi_entry->data = msi_desc->msg.data;
+	msi_entry->address.as_uint32 = msi_desc->msg.address_lo;
+	msi_entry->data.as_uint32 = msi_desc->msg.data;
 }
 
 #else /* CONFIG_HYPERV */
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index 2b05bed712c0..e7e80a27777b 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -469,12 +469,36 @@ struct hv_create_vp {
 	u64 flags;
 };
 
+union hv_msi_address_register {
+	u32 as_uint32;
+	struct {
+		u32 reserved1:2;
+		u32 destination_mode:1;
+		u32 redirection_hint:1;
+		u32 reserved2:8;
+		u32 destination_id:8;
+		u32 msi_base:12;
+	};
+} __packed;
+
+union hv_msi_data_register {
+	u32 as_uint32;
+	struct {
+		u32 vector:8;
+		u32 delivery_mode:3;
+		u32 reserved1:3;
+		u32 level_assert:1;
+		u32 trigger_mode:1;
+		u32 reserved2:16;
+	};
+} __packed;
+
 /* HvRetargetDeviceInterrupt hypercall */
 union hv_msi_entry {
 	u64 as_uint64;
 	struct {
-		u32 address;
-		u32 data;
+		union hv_msi_address_register address;
+		union hv_msi_data_register data;
 	} __packed;
 };
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 12/18] asm-generic/hyperv: update hv_interrupt_entry
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (10 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 11/18] asm-generic/hyperv: update hv_msi_entry Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-10-01 14:33   ` Rob Herring
  2020-09-14 11:59 ` [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures Wei Liu
                   ` (5 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Lorenzo Pieralisi, Rob Herring, Bjorn Helgaas, Arnd Bergmann,
	open list:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS,
	open list:GENERIC INCLUDE/ASM HEADER FILES

We will soon use the same structure to handle IO-APIC interrupts as
well. Introduce an enum to identify the source and a data structure for
IO-APIC RTE.

While at it, update pci-hyperv.c to use the enum.

No functional change.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 drivers/pci/controller/pci-hyperv.c |  2 +-
 include/asm-generic/hyperv-tlfs.h   | 36 +++++++++++++++++++++++++++--
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index bf40ff09c99d..b38faa04234b 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -1215,7 +1215,7 @@ static void hv_irq_unmask(struct irq_data *data)
 	params = &hbus->retarget_msi_interrupt_params;
 	memset(params, 0, sizeof(*params));
 	params->partition_id = HV_PARTITION_ID_SELF;
-	params->int_entry.source = 1; /* MSI(-X) */
+	params->int_entry.source = HV_INTERRUPT_SOURCE_MSI;
 	hv_set_msi_entry_from_desc(&params->int_entry.msi_entry, msi_desc);
 	params->device_id = (hbus->hdev->dev_instance.b[5] << 24) |
 			   (hbus->hdev->dev_instance.b[4] << 16) |
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index e7e80a27777b..83945ada5a50 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -469,6 +469,11 @@ struct hv_create_vp {
 	u64 flags;
 };
 
+enum hv_interrupt_source {
+	HV_INTERRUPT_SOURCE_MSI = 1, /* MSI and MSI-X */
+	HV_INTERRUPT_SOURCE_IOAPIC,
+};
+
 union hv_msi_address_register {
 	u32 as_uint32;
 	struct {
@@ -502,10 +507,37 @@ union hv_msi_entry {
 	} __packed;
 };
 
+union hv_ioapic_rte {
+	u64 as_uint64;
+
+	struct {
+		u32 vector:8;
+		u32 delivery_mode:3;
+		u32 destination_mode:1;
+		u32 delivery_status:1;
+		u32 interrupt_polarity:1;
+		u32 remote_irr:1;
+		u32 trigger_mode:1;
+		u32 interrupt_mask:1;
+		u32 reserved1:15;
+
+		u32 reserved2:24;
+		u32 destination_id:8;
+	};
+
+	struct {
+		u32 low_uint32;
+		u32 high_uint32;
+	};
+} __packed;
+
 struct hv_interrupt_entry {
-	u32 source;			/* 1 for MSI(-X) */
+	u32 source;
 	u32 reserved1;
-	union hv_msi_entry msi_entry;
+	union {
+		union hv_msi_entry msi_entry;
+		union hv_ioapic_rte ioapic_rte;
+	};
 } __packed;
 
 /*
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (11 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 12/18] asm-generic/hyperv: update hv_interrupt_entry Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-15 11:16   ` Vitaly Kuznetsov
  2020-09-14 11:59 ` [PATCH RFC v1 14/18] asm-generic/hyperv: import data structures for mapping device interrupts Wei Liu
                   ` (4 subsequent siblings)
  17 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Arnd Bergmann, open list:GENERIC INCLUDE/ASM HEADER FILES

We will need to identify the device we want Microsoft Hypervisor to
manipulate.  Introduce the data structures for that purpose.

They will be used in a later patch.

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 include/asm-generic/hyperv-tlfs.h | 79 +++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index 83945ada5a50..faf892ce152d 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -612,4 +612,83 @@ struct hv_set_vp_registers_input {
 	} element[];
 } __packed;
 
+enum hv_device_type {
+	HV_DEVICE_TYPE_LOGICAL = 0,
+	HV_DEVICE_TYPE_PCI = 1,
+	HV_DEVICE_TYPE_IOAPIC = 2,
+	HV_DEVICE_TYPE_ACPI = 3,
+};
+
+typedef u16 hv_pci_rid;
+typedef u16 hv_pci_segment;
+typedef u64 hv_logical_device_id;
+union hv_pci_bdf {
+	u16 as_uint16;
+
+	struct {
+		u8 function:3;
+		u8 device:5;
+		u8 bus;
+	};
+} __packed;
+
+union hv_pci_bus_range {
+	u16 as_uint16;
+
+	struct {
+		u8 subordinate_bus;
+		u8 secondary_bus;
+	};
+} __packed;
+
+union hv_device_id {
+	u64 as_uint64;
+
+	struct {
+		u64 :62;
+		u64 device_type:2;
+	};
+
+	// HV_DEVICE_TYPE_LOGICAL
+	struct {
+		u64 id:62;
+		u64 device_type:2;
+	} logical;
+
+	// HV_DEVICE_TYPE_PCI
+	struct {
+		union {
+			hv_pci_rid rid;
+			union hv_pci_bdf bdf;
+		};
+
+		hv_pci_segment segment;
+		union hv_pci_bus_range shadow_bus_range;
+
+		u16 phantom_function_bits:2;
+		u16 source_shadow:1;
+
+		u16 rsvdz0:11;
+		u16 device_type:2;
+	} pci;
+
+	// HV_DEVICE_TYPE_IOAPIC
+	struct {
+		u8 ioapic_id;
+		u8 rsvdz0;
+		u16 rsvdz1;
+		u16 rsvdz2;
+
+		u16 rsvdz3:14;
+		u16 device_type:2;
+	} ioapic;
+
+	// HV_DEVICE_TYPE_ACPI
+	struct {
+		u32 input_mapping_base;
+		u32 input_mapping_count:30;
+		u32 device_type:2;
+	} acpi;
+} __packed;
+
 #endif
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 14/18] asm-generic/hyperv: import data structures for mapping device interrupts
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (12 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-14 11:59 ` [PATCH RFC v1 15/18] x86/apic/msi: export pci_msi_get_hwirq Wei Liu
                   ` (3 subsequent siblings)
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/include/asm/hyperv-tlfs.h | 13 +++++++++++
 include/asm-generic/hyperv-tlfs.h  | 36 ++++++++++++++++++++++++++++++
 2 files changed, 49 insertions(+)

diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index 20d628c1ed50..33210fa9899f 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -559,6 +559,19 @@ struct hv_partition_assist_pg {
 	u32 tlb_lock_count;
 };
 
+enum hv_interrupt_type {
+	HV_X64_INTERRUPT_TYPE_FIXED             = 0x0000,
+	HV_X64_INTERRUPT_TYPE_LOWESTPRIORITY    = 0x0001,
+	HV_X64_INTERRUPT_TYPE_SMI               = 0x0002,
+	HV_X64_INTERRUPT_TYPE_REMOTEREAD        = 0x0003,
+	HV_X64_INTERRUPT_TYPE_NMI               = 0x0004,
+	HV_X64_INTERRUPT_TYPE_INIT              = 0x0005,
+	HV_X64_INTERRUPT_TYPE_SIPI              = 0x0006,
+	HV_X64_INTERRUPT_TYPE_EXTINT            = 0x0007,
+	HV_X64_INTERRUPT_TYPE_LOCALINT0         = 0x0008,
+	HV_X64_INTERRUPT_TYPE_LOCALINT1         = 0x0009,
+	HV_X64_INTERRUPT_TYPE_MAXIMUM           = 0x000A,
+};
 
 #include <asm-generic/hyperv-tlfs.h>
 
diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
index faf892ce152d..410e2dd09a84 100644
--- a/include/asm-generic/hyperv-tlfs.h
+++ b/include/asm-generic/hyperv-tlfs.h
@@ -152,6 +152,8 @@ struct ms_hyperv_tsc_page {
 #define HVCALL_RETRIEVE_DEBUG_DATA		0x006a
 #define HVCALL_RESET_DEBUG_SESSION		0x006b
 #define HVCALL_ADD_LOGICAL_PROCESSOR		0x0076
+#define HVCALL_MAP_DEVICE_INTERRUPT		0x007c
+#define HVCALL_UNMAP_DEVICE_INTERRUPT		0x007d
 #define HVCALL_RETARGET_INTERRUPT		0x007e
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
@@ -691,4 +693,38 @@ union hv_device_id {
 	} acpi;
 } __packed;
 
+enum hv_interrupt_trigger_mode {
+	HV_INTERRUPT_TRIGGER_MODE_EDGE = 0,
+	HV_INTERRUPT_TRIGGER_MODE_LEVEL = 1,
+};
+
+struct hv_device_interrupt_descriptor {
+	u32 interrupt_type;
+	u32 trigger_mode;
+	u32 vector_count;
+	u32 reserved;
+	struct hv_device_interrupt_target target;
+} __packed;
+
+struct hv_input_map_device_interrupt {
+	u64 partition_id;
+	u64 device_id;
+	u64 flags;
+	struct hv_interrupt_entry logical_interrupt_entry;
+	struct hv_device_interrupt_descriptor interrupt_descriptor;
+} __packed;
+
+struct hv_output_map_device_interrupt {
+	struct hv_interrupt_entry interrupt_entry;
+} __packed;
+
+struct hv_input_unmap_device_interrupt {
+	u64 partition_id;
+	u64 device_id;
+	struct hv_interrupt_entry interrupt_entry;
+} __packed;
+
+#define HV_SOURCE_SHADOW_NONE               0x0
+#define HV_SOURCE_SHADOW_BRIDGE_BUS_RANGE   0x1
+
 #endif
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 15/18] x86/apic/msi: export pci_msi_get_hwirq
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (13 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 14/18] asm-generic/hyperv: import data structures for mapping device interrupts Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-14 11:59 ` [PATCH RFC v1 16/18] x86/hyperv: implement MSI domain for root partition Wei Liu
                   ` (2 subsequent siblings)
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Bjorn Helgaas, Andy Lutomirski

Linux will implement an MSI domain for MSI/MSI-X when running as root on
Microsoft Hypervisor. It will be using this function to reduce code
duplication.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/include/asm/msi.h | 3 +++
 arch/x86/kernel/apic/msi.c | 3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/msi.h b/arch/x86/include/asm/msi.h
index 25ddd0916bb2..5c0e102c03ec 100644
--- a/arch/x86/include/asm/msi.h
+++ b/arch/x86/include/asm/msi.h
@@ -11,4 +11,7 @@ int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec,
 
 void pci_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc);
 
+struct msi_domain_info;
+irq_hw_number_t pci_msi_get_hwirq(struct msi_domain_info *info, msi_alloc_info_t *arg);
+
 #endif /* _ASM_X86_MSI_H */
diff --git a/arch/x86/kernel/apic/msi.c b/arch/x86/kernel/apic/msi.c
index c2b2911feeef..dc9693a73933 100644
--- a/arch/x86/kernel/apic/msi.c
+++ b/arch/x86/kernel/apic/msi.c
@@ -205,11 +205,12 @@ void native_teardown_msi_irq(unsigned int irq)
 	irq_domain_free_irqs(irq, 1);
 }
 
-static irq_hw_number_t pci_msi_get_hwirq(struct msi_domain_info *info,
+irq_hw_number_t pci_msi_get_hwirq(struct msi_domain_info *info,
 					 msi_alloc_info_t *arg)
 {
 	return arg->msi_hwirq;
 }
+EXPORT_SYMBOL_GPL(pci_msi_get_hwirq);
 
 int pci_msi_prepare(struct irq_domain *domain, struct device *dev, int nvec,
 		    msi_alloc_info_t *arg)
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 16/18] x86/hyperv: implement MSI domain for root partition
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (14 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 15/18] x86/apic/msi: export pci_msi_get_hwirq Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-14 11:59 ` [PATCH RFC v1 17/18] x86/ioapic: export a few functions and data structures via io_apic.h Wei Liu
  2020-09-14 11:59 ` [PATCH RFC v1 18/18] x86/hyperv: handle IO-APIC when running as root Wei Liu
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

When Linux runs as the root partition on Microsoft Hypervisor, its
interrupts are remapped.  Linux will need to explicitly map and unmap
interrupts for hardware.

Implement an MSI domain to issue the correct hypercalls. And initialize
this irqdomain in the PCI init hook.

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/Makefile    |   2 +-
 arch/x86/hyperv/hv_init.c   |  14 ++
 arch/x86/hyperv/irqdomain.c | 355 ++++++++++++++++++++++++++++++++++++
 3 files changed, 370 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/hyperv/irqdomain.c

diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
index 565358020921..48e2c51464e8 100644
--- a/arch/x86/hyperv/Makefile
+++ b/arch/x86/hyperv/Makefile
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
-obj-y			:= hv_init.o mmu.o nested.o
+obj-y			:= hv_init.o mmu.o nested.o irqdomain.o
 obj-$(CONFIG_X86_64)	+= hv_apic.o hv_proc.o
 
 ifdef CONFIG_X86_64
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index 26233aebc86c..d26d9573ceab 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -277,10 +277,24 @@ static int hv_cpu_die(unsigned int cpu)
 	return 0;
 }
 
+int hv_setup_msi_domain_irqs(struct pci_dev *dev, int nvec, int type);
+void hv_teardown_msi_irq(unsigned int irq);
+void hv_teardown_msi_irqs(struct pci_dev *dev);
+int hv_init_msi_domain(void);
+
 static int __init hv_pci_init(void)
 {
 	int gen2vm = efi_enabled(EFI_BOOT);
 
+	if (hv_root_partition) {
+		if (hv_init_msi_domain() < 0)
+			panic("Failed to allocate MSI domain\n");
+
+		x86_msi.setup_msi_irqs = hv_setup_msi_domain_irqs;
+		x86_msi.teardown_msi_irq = hv_teardown_msi_irq;
+		x86_msi.teardown_msi_irqs = hv_teardown_msi_irqs;
+	}
+
 	/*
 	 * For Generation-2 VM, we exit from pci_arch_init() by returning 0.
 	 * The purpose is to suppress the harmless warning:
diff --git a/arch/x86/hyperv/irqdomain.c b/arch/x86/hyperv/irqdomain.c
new file mode 100644
index 000000000000..6ffe32d9cde5
--- /dev/null
+++ b/arch/x86/hyperv/irqdomain.c
@@ -0,0 +1,355 @@
+// SPDX-License-Identifier: GPL-2.0
+//
+// Irqdomain for Linux to run as the root partition on Microsoft Hypervisor.
+//
+// Authors:
+//   Sunil Muthuswamy <sunilmut@microsoft.com>
+//   Wei Liu <wei.liu@kernel.org>
+
+#include <linux/pci.h>
+#include <linux/irq.h>
+#include <asm/mshyperv.h>
+
+struct rid_data {
+	struct pci_dev *bridge;
+	u32 rid;
+};
+
+static int get_rid_cb(struct pci_dev *pdev, u16 alias, void *data)
+{
+	struct rid_data *rd = data;
+	u8 bus = PCI_BUS_NUM(rd->rid);
+
+	if (pdev->bus->number != bus || PCI_BUS_NUM(alias) != bus) {
+		rd->bridge = pdev;
+		rd->rid = alias;
+	}
+
+	return 0;
+}
+
+static union hv_device_id hv_build_pci_dev_id(struct pci_dev *dev)
+{
+	union hv_device_id dev_id;
+	struct rid_data data = {
+		.bridge = NULL,
+		.rid = PCI_DEVID(dev->bus->number, dev->devfn)
+	};
+
+	pci_for_each_dma_alias(dev, get_rid_cb, &data);
+
+	dev_id.as_uint64 = 0;
+	dev_id.device_type = HV_DEVICE_TYPE_PCI;
+	dev_id.pci.segment = pci_domain_nr(dev->bus);
+
+	dev_id.pci.bdf.bus = PCI_BUS_NUM(data.rid);
+	dev_id.pci.bdf.device = PCI_SLOT(data.rid);
+	dev_id.pci.bdf.function = PCI_FUNC(data.rid);
+	dev_id.pci.source_shadow = HV_SOURCE_SHADOW_NONE;
+
+	if (data.bridge) {
+		int pos;
+
+		/*
+		 * Microsoft Hypervisor requires a bus range when the bridge is
+		 * running in PCI-X mode.
+		 *
+		 * To distinguish conventional vs PCI-X bridge, we can check
+		 * the bridge's PCI-X Secondary Status Register, Secondary Bus
+		 * Mode and Frequency bits. See PCI Express to PCI/PCI-X Bridge
+		 * Specification Revision 1.0 5.2.2.1.3.
+		 *
+		 * Value zero means it is in conventional mode, otherwise it is
+		 * in PCI-X mode.
+		 */
+
+		pos = pci_find_capability(data.bridge, PCI_CAP_ID_PCIX);
+		if (pos) {
+			u16 status;
+
+			pci_read_config_word(data.bridge, pos +
+					PCI_X_BRIDGE_SSTATUS, &status);
+
+			if (status & PCI_X_SSTATUS_FREQ) {
+				/* Non-zero, PCI-X mode */
+				u8 sec_bus, sub_bus;
+
+				dev_id.pci.source_shadow = HV_SOURCE_SHADOW_BRIDGE_BUS_RANGE;
+
+				pci_read_config_byte(data.bridge, PCI_SECONDARY_BUS, &sec_bus);
+				dev_id.pci.shadow_bus_range.secondary_bus = sec_bus;
+				pci_read_config_byte(data.bridge, PCI_SUBORDINATE_BUS, &sub_bus);
+				dev_id.pci.shadow_bus_range.subordinate_bus = sub_bus;
+			}
+		}
+	}
+
+	return dev_id;
+}
+
+static int hv_map_msi_interrupt(struct pci_dev *dev, int vcpu, int vector,
+				struct hv_interrupt_entry *entry)
+{
+	struct hv_input_map_device_interrupt *input;
+	struct hv_output_map_device_interrupt *output;
+	struct hv_device_interrupt_descriptor *intr_desc;
+	unsigned long flags;
+	int status;
+
+	local_irq_save(flags);
+
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);
+
+	intr_desc = &input->interrupt_descriptor;
+	memset(input, 0, sizeof(*input));
+	input->partition_id = hv_current_partition_id;
+	input->device_id = hv_build_pci_dev_id(dev).as_uint64;
+	intr_desc->interrupt_type = HV_X64_INTERRUPT_TYPE_FIXED;
+	intr_desc->trigger_mode = HV_INTERRUPT_TRIGGER_MODE_EDGE;
+	intr_desc->vector_count = 1;
+	intr_desc->target.vector = vector;
+	__set_bit(vcpu, (unsigned long*)&intr_desc->target.vp_mask);
+
+	status = hv_do_rep_hypercall(HVCALL_MAP_DEVICE_INTERRUPT, 0, 0, input, output) &
+			 HV_HYPERCALL_RESULT_MASK;
+	local_irq_restore(flags);
+
+	if (status != HV_STATUS_SUCCESS) {
+		pr_err("%s: hypercall failed, status %d\n", __func__, status);
+		return status;
+	}
+
+	*entry = output->interrupt_entry;
+
+	return status;
+}
+
+static inline void entry_to_msi_msg(struct hv_interrupt_entry *entry, struct msi_msg *msg)
+{
+	/* High address is always 0 */
+	msg->address_hi = 0;
+	msg->address_lo = entry->msi_entry.address.as_uint32;
+	msg->data = entry->msi_entry.data.as_uint32;
+}
+
+static int hv_unmap_msi_interrupt(struct pci_dev *dev, struct hv_interrupt_entry *old_entry);
+static void hv_irq_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
+{
+	struct msi_desc *msidesc;
+	struct pci_dev *dev;
+	struct hv_interrupt_entry out_entry, *stored_entry;
+	struct irq_cfg *cfg = irqd_cfg(data);
+	struct cpumask *affinity;
+	int cpu, vcpu;
+	int status;
+
+	msidesc = irq_data_get_msi_desc(data);
+	dev = msi_desc_to_pci_dev(msidesc);
+
+	if (!cfg) {
+		pr_debug("%s: cfg is NULL", __func__);
+		return;
+	}
+
+	affinity = irq_data_get_effective_affinity_mask(data);
+	cpu = cpumask_first_and(affinity, cpu_online_mask);
+	vcpu = hv_cpu_number_to_vp_number(cpu);
+
+	if (data->chip_data) {
+		/*
+		 * This interrupt is already mapped. Let's unmap first.
+		 *
+		 * We don't use retarget interrupt hypercalls here because
+		 * Microsoft Hypervisor doens't allow root to change the vector
+		 * or specify VPs outside of the set that is initially used
+		 * during mapping.
+		 */
+		stored_entry = data->chip_data;
+		data->chip_data = NULL;
+
+		status = hv_unmap_msi_interrupt(dev, stored_entry);
+
+		kfree(stored_entry);
+
+		if (status != HV_STATUS_SUCCESS) {
+			pr_debug("%s: failed to unmap, status %d", __func__, status);
+			return;
+		}
+	}
+
+	stored_entry = kzalloc(sizeof(*stored_entry), GFP_ATOMIC);
+	if (!stored_entry) {
+		pr_debug("%s: failed to allocate chip data\n", __func__);
+		return;
+	}
+
+	status = hv_map_msi_interrupt(dev, vcpu, cfg->vector, &out_entry);
+	if (status != HV_STATUS_SUCCESS) {
+		kfree(stored_entry);
+		return;
+	}
+
+	*stored_entry = out_entry;
+	data->chip_data = stored_entry;
+	entry_to_msi_msg(&out_entry, msg);
+
+	return;
+}
+
+/*
+ * IRQ Chip for MSI PCI/PCI-X/PCI-Express Devices,
+ * which implement the MSI or MSI-X Capability Structure.
+ */
+static struct irq_chip hv_pci_msi_controller = {
+	.name			= "HV-PCI-MSI",
+	.irq_unmask		= pci_msi_unmask_irq,
+	.irq_mask		= pci_msi_mask_irq,
+	.irq_ack		= irq_chip_ack_parent,
+	.irq_retrigger		= irq_chip_retrigger_hierarchy,
+	.irq_compose_msi_msg	= hv_irq_compose_msi_msg,
+	.irq_set_affinity	= msi_domain_set_affinity,
+	.flags			= IRQCHIP_SKIP_SET_WAKE,
+};
+
+static struct msi_domain_ops pci_msi_domain_ops = {
+	.get_hwirq	= pci_msi_get_hwirq,
+	.msi_prepare	= pci_msi_prepare,
+	.set_desc	= pci_msi_set_desc,
+};
+
+static struct msi_domain_info hv_pci_msi_domain_info = {
+	.flags		= MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
+			  MSI_FLAG_PCI_MSIX,
+	.ops		= &pci_msi_domain_ops,
+	.chip		= &hv_pci_msi_controller,
+	.handler	= handle_edge_irq,
+	.handler_name	= "edge",
+};
+
+static struct irq_domain *hv_msi_domain;
+
+int hv_init_msi_domain(void)
+{
+	struct fwnode_handle *fn;
+
+	fn = irq_domain_alloc_named_fwnode("HV-PCI-MSI");
+	if (!fn) {
+		pr_debug("irq_domain_alloc_named_fwnode failed\n");
+		return -1;
+	}
+
+	hv_msi_domain = pci_msi_create_irq_domain(fn, &hv_pci_msi_domain_info,
+					  x86_vector_domain);
+	irq_domain_free_fwnode(fn);
+	if (!hv_msi_domain) {
+		pr_warn("Failed to initialize irqdomain for MSI/MSI-X.\n");
+		return -1;
+	}
+
+	return 0;
+}
+
+int hv_setup_msi_domain_irqs(struct pci_dev *dev, int nvec, int type)
+{
+	int ret;
+
+	if (type == PCI_CAP_ID_MSI && nvec > 1) {
+		pr_debug("%s: Multi-message PCI MSI not supported", __func__);
+		return 1;
+	}
+
+	ret = msi_domain_alloc_irqs(hv_msi_domain, &dev->dev, nvec);
+	if (ret)
+		pr_debug("%s: msi_domain_alloc_irqs returned: %d\n", __func__, ret);
+
+	return ret;
+}
+
+static int hv_unmap_interrupt(u64 id, struct hv_interrupt_entry *old_entry)
+{
+	unsigned long flags;
+	struct hv_input_unmap_device_interrupt *input;
+	struct hv_interrupt_entry *intr_entry;
+	int status;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+
+	memset(input, 0, sizeof(*input));
+	intr_entry = &input->interrupt_entry;
+	input->partition_id = hv_current_partition_id;
+	input->device_id = id;
+	*intr_entry = *old_entry;
+
+	status = hv_do_rep_hypercall(HVCALL_UNMAP_DEVICE_INTERRUPT, 0, 0, input, NULL) &
+			 HV_HYPERCALL_RESULT_MASK;
+	local_irq_restore(flags);
+
+	return status;
+}
+
+static int hv_unmap_msi_interrupt(struct pci_dev *dev, struct hv_interrupt_entry *old_entry)
+{
+	return hv_unmap_interrupt(hv_build_pci_dev_id(dev).as_uint64, old_entry)
+		& HV_HYPERCALL_RESULT_MASK;
+}
+
+static void hv_teardown_msi_irq_common(struct pci_dev *dev, struct msi_desc *msidesc, int irq)
+{
+	int status;
+	struct hv_interrupt_entry old_entry;
+	struct irq_desc *desc;
+	struct irq_data *data;
+	struct msi_msg msg;
+
+	desc = irq_to_desc(irq);
+	if (!desc) {
+		pr_debug("%s: no irq desc\n", __func__);
+		return;
+	}
+
+	data = &desc->irq_data;
+	if (!data) {
+		pr_debug("%s: no irq data\n", __func__);
+		return;
+	}
+
+	if (!data->chip_data) {
+		pr_debug("%s: no chip data\n!", __func__);
+		return;
+	}
+
+	old_entry = *(struct hv_interrupt_entry *)data->chip_data;
+	entry_to_msi_msg(&old_entry, &msg);
+
+	kfree(data->chip_data);
+	data->chip_data = NULL;
+
+	status = hv_unmap_msi_interrupt(dev, &old_entry);
+
+	if (status != HV_STATUS_SUCCESS) {
+		pr_err("%s: hypercall failed, status %d\n", __func__, status);
+		return;
+	}
+}
+
+void hv_teardown_msi_irq(unsigned int irq)
+{
+	irq_domain_free_irqs(irq, 1);
+}
+
+void hv_teardown_msi_irqs(struct pci_dev *dev)
+{
+	int i;
+	struct msi_desc *entry;
+
+	for_each_pci_msi_entry(entry, dev) {
+		if (entry->irq) {
+			for (i = 0; i < entry->nvec_used; i++) {
+				hv_teardown_msi_irq_common(dev, entry, entry->irq + i);
+				hv_teardown_msi_irq(entry->irq + i);
+			}
+		}
+	}
+}
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 17/18] x86/ioapic: export a few functions and data structures via io_apic.h
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (15 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 16/18] x86/hyperv: implement MSI domain for root partition Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  2020-09-14 11:59 ` [PATCH RFC v1 18/18] x86/hyperv: handle IO-APIC when running as root Wei Liu
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Marc Zyngier, YueHaibing

We are about to implement an irqchip for IO-APIC when Linux runs as root
on Microsoft Hypervisor. At the same time we would like to reuse
existing code as much as possible.

Move mp_chip_data to io_apic.h and make a few helper functions
non-static.

No functional change.

Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/include/asm/io_apic.h | 20 ++++++++++++++++++++
 arch/x86/kernel/apic/io_apic.c | 28 +++++++++-------------------
 2 files changed, 29 insertions(+), 19 deletions(-)

diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index fd20a2334885..9277530116fa 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -153,6 +153,15 @@ extern unsigned long io_apic_irqs;
 #define io_apic_assign_pci_irqs \
 	(mp_irq_entries && !skip_ioapic_setup && io_apic_irqs)
 
+struct mp_chip_data {
+	struct list_head irq_2_pin;
+	struct IO_APIC_route_entry entry;
+	int trigger;
+	int polarity;
+	u32 count;
+	bool isa_irq;
+};
+
 struct irq_cfg;
 extern void ioapic_insert_resources(void);
 extern int arch_early_ioapic_init(void);
@@ -196,6 +205,17 @@ extern void clear_IO_APIC(void);
 extern void restore_boot_irq_mode(void);
 extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
 extern void print_IO_APICs(void);
+
+extern struct IO_APIC_route_entry ioapic_read_entry(int apic, int pin);
+extern void ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e);
+extern void mask_ioapic_irq(struct irq_data *irq_data);
+extern void unmask_ioapic_irq(struct irq_data *irq_data);
+extern int ioapic_set_affinity(struct irq_data *irq_data, const struct cpumask *mask, bool force);
+extern struct irq_domain *mp_ioapic_irqdomain(int ioapic);
+enum irqchip_irq_state;
+extern int ioapic_irq_get_chip_state(struct irq_data *irqd,
+				enum irqchip_irq_state which,
+				bool *state);
 #else  /* !CONFIG_X86_IO_APIC */
 
 #define IO_APIC_IRQ(x)		0
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 81ffcfbfaef2..43d151f9894b 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -88,15 +88,6 @@ struct irq_pin_list {
 	int apic, pin;
 };
 
-struct mp_chip_data {
-	struct list_head irq_2_pin;
-	struct IO_APIC_route_entry entry;
-	int trigger;
-	int polarity;
-	u32 count;
-	bool isa_irq;
-};
-
 struct mp_ioapic_gsi {
 	u32 gsi_base;
 	u32 gsi_end;
@@ -154,7 +145,7 @@ static inline bool mp_is_legacy_irq(int irq)
 	return irq >= 0 && irq < nr_legacy_irqs();
 }
 
-static inline struct irq_domain *mp_ioapic_irqdomain(int ioapic)
+struct irq_domain *mp_ioapic_irqdomain(int ioapic)
 {
 	return ioapics[ioapic].irqdomain;
 }
@@ -301,7 +292,7 @@ static struct IO_APIC_route_entry __ioapic_read_entry(int apic, int pin)
 	return eu.entry;
 }
 
-static struct IO_APIC_route_entry ioapic_read_entry(int apic, int pin)
+struct IO_APIC_route_entry ioapic_read_entry(int apic, int pin)
 {
 	union entry_union eu;
 	unsigned long flags;
@@ -328,7 +319,7 @@ static void __ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e
 	io_apic_write(apic, 0x10 + 2*pin, eu.w1);
 }
 
-static void ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e)
+void ioapic_write_entry(int apic, int pin, struct IO_APIC_route_entry e)
 {
 	unsigned long flags;
 
@@ -453,7 +444,7 @@ static void io_apic_sync(struct irq_pin_list *entry)
 	readl(&io_apic->data);
 }
 
-static void mask_ioapic_irq(struct irq_data *irq_data)
+void mask_ioapic_irq(struct irq_data *irq_data)
 {
 	struct mp_chip_data *data = irq_data->chip_data;
 	unsigned long flags;
@@ -468,7 +459,7 @@ static void __unmask_ioapic(struct mp_chip_data *data)
 	io_apic_modify_irq(data, ~IO_APIC_REDIR_MASKED, 0, NULL);
 }
 
-static void unmask_ioapic_irq(struct irq_data *irq_data)
+void unmask_ioapic_irq(struct irq_data *irq_data)
 {
 	struct mp_chip_data *data = irq_data->chip_data;
 	unsigned long flags;
@@ -1868,8 +1859,7 @@ static void ioapic_configure_entry(struct irq_data *irqd)
 		__ioapic_write_entry(entry->apic, entry->pin, mpd->entry);
 }
 
-static int ioapic_set_affinity(struct irq_data *irq_data,
-			       const struct cpumask *mask, bool force)
+int ioapic_set_affinity(struct irq_data *irq_data, const struct cpumask *mask, bool force)
 {
 	struct irq_data *parent = irq_data->parent_data;
 	unsigned long flags;
@@ -1898,9 +1888,9 @@ static int ioapic_set_affinity(struct irq_data *irq_data,
  *
  * Verify that the corresponding Remote-IRR bits are clear.
  */
-static int ioapic_irq_get_chip_state(struct irq_data *irqd,
-				   enum irqchip_irq_state which,
-				   bool *state)
+int ioapic_irq_get_chip_state(struct irq_data *irqd,
+				enum irqchip_irq_state which,
+				bool *state)
 {
 	struct mp_chip_data *mcd = irqd->chip_data;
 	struct IO_APIC_route_entry rentry;
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* [PATCH RFC v1 18/18] x86/hyperv: handle IO-APIC when running as root
  2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
                   ` (16 preceding siblings ...)
  2020-09-14 11:59 ` [PATCH RFC v1 17/18] x86/ioapic: export a few functions and data structures via io_apic.h Wei Liu
@ 2020-09-14 11:59 ` Wei Liu
  17 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-14 11:59 UTC (permalink / raw)
  To: Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Just like MSI/MSI-X, IO-APIC interrupts are remapped by Microsoft
Hypervisor when Linux runs as the root partition. Implement an IRQ chip
to handle mapping and unmapping of IO-APIC interrupts.

Use custom functions for mapping and unmapping ACPI GSIs. They will
issue Microsoft Hypervisor specific hypercalls on top of the native
routines.

Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
---
 arch/x86/hyperv/hv_init.c   |  11 ++
 arch/x86/hyperv/irqdomain.c | 225 ++++++++++++++++++++++++++++++++++++
 2 files changed, 236 insertions(+)

diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index d26d9573ceab..0d6a6c0f288f 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -282,6 +282,12 @@ void hv_teardown_msi_irq(unsigned int irq);
 void hv_teardown_msi_irqs(struct pci_dev *dev);
 int hv_init_msi_domain(void);
 
+int hv_acpi_register_gsi(struct device *dev, u32 gsi, int trigger, int polarity);
+void hv_acpi_unregister_gsi(u32 gsi);
+
+extern int (*native_acpi_register_gsi)(struct device *dev, u32 gsi, int trigger, int polarity);
+extern void (*native_acpi_unregister_gsi)(u32 gsi);
+
 static int __init hv_pci_init(void)
 {
 	int gen2vm = efi_enabled(EFI_BOOT);
@@ -293,6 +299,11 @@ static int __init hv_pci_init(void)
 		x86_msi.setup_msi_irqs = hv_setup_msi_domain_irqs;
 		x86_msi.teardown_msi_irq = hv_teardown_msi_irq;
 		x86_msi.teardown_msi_irqs = hv_teardown_msi_irqs;
+
+		native_acpi_register_gsi = __acpi_register_gsi;
+		native_acpi_unregister_gsi = __acpi_unregister_gsi;
+		__acpi_register_gsi = hv_acpi_register_gsi;
+		__acpi_unregister_gsi = hv_acpi_unregister_gsi;
 	}
 
 	/*
diff --git a/arch/x86/hyperv/irqdomain.c b/arch/x86/hyperv/irqdomain.c
index 6ffe32d9cde5..2ce3dbd80c5d 100644
--- a/arch/x86/hyperv/irqdomain.c
+++ b/arch/x86/hyperv/irqdomain.c
@@ -353,3 +353,228 @@ void hv_teardown_msi_irqs(struct pci_dev *dev)
 		}
 	}
 }
+
+/* Copied from io_apic.c */
+union entry_union {
+	struct { u32 w1, w2; };
+	struct IO_APIC_route_entry entry;
+};
+
+static int hv_unmap_ioapic_interrupt(int gsi)
+{
+	union hv_device_id device_id;
+	int ioapic, ioapic_id;
+	u8 ioapic_pin;
+	struct IO_APIC_route_entry ire;
+	union entry_union eu;
+	struct hv_interrupt_entry entry;
+
+	ioapic = mp_find_ioapic(gsi);
+	ioapic_pin = mp_find_ioapic_pin(ioapic, gsi);
+	ioapic_id = mpc_ioapic_id(ioapic);
+	ire = ioapic_read_entry(ioapic, ioapic_pin);
+
+	eu.entry = ire;
+
+	/*
+	 * Polarity may have been set by us, but Hyper-V expects the exact same
+	 * entry. See the mapping routine.
+	 */
+	eu.entry.polarity = 0;
+
+	memset(&entry, 0, sizeof(entry));
+	entry.source = HV_INTERRUPT_SOURCE_IOAPIC;
+	entry.ioapic_rte.low_uint32 = eu.w1;
+	entry.ioapic_rte.high_uint32 = eu.w2;
+
+	device_id.as_uint64 = 0;
+	device_id.device_type = HV_DEVICE_TYPE_IOAPIC;
+	device_id.ioapic.ioapic_id = (u8)ioapic_id;
+
+	return hv_unmap_interrupt(device_id.as_uint64, &entry) & HV_HYPERCALL_RESULT_MASK;
+}
+
+static int hv_map_ioapic_interrupt(int ioapic_id, int trigger, int vcpu, int vector,
+		struct hv_interrupt_entry *out_entry)
+{
+	unsigned long flags;
+	struct hv_input_map_device_interrupt *input;
+	struct hv_output_map_device_interrupt *output;
+	union hv_device_id device_id;
+	struct hv_device_interrupt_descriptor *intr_desc;
+	int status;
+
+	device_id.as_uint64 = 0;
+	device_id.device_type = HV_DEVICE_TYPE_IOAPIC;
+	device_id.ioapic.ioapic_id = (u8)ioapic_id;
+
+	local_irq_save(flags);
+	input = *this_cpu_ptr(hyperv_pcpu_input_arg);
+	output = *this_cpu_ptr(hyperv_pcpu_output_arg);
+	memset(input, 0, sizeof(*input));
+	intr_desc = &input->interrupt_descriptor;
+	input->partition_id = hv_current_partition_id;
+	input->device_id = device_id.as_uint64;
+	intr_desc->interrupt_type = HV_X64_INTERRUPT_TYPE_FIXED;
+	intr_desc->target.vector = vector;
+	intr_desc->vector_count = 1;
+
+	if (trigger)
+		intr_desc->trigger_mode = HV_INTERRUPT_TRIGGER_MODE_LEVEL;
+	else
+		intr_desc->trigger_mode = HV_INTERRUPT_TRIGGER_MODE_EDGE;
+
+	__set_bit(vcpu, (unsigned long *)&intr_desc->target.vp_mask);
+
+	status = hv_do_rep_hypercall(HVCALL_MAP_DEVICE_INTERRUPT, 0, 0, input, output) &
+			 HV_HYPERCALL_RESULT_MASK;
+	local_irq_restore(flags);
+
+	*out_entry = output->interrupt_entry;
+
+	return status;
+}
+
+static unsigned int hv_ioapic_startup_irq(struct irq_data *data)
+{
+	int status;
+	struct IO_APIC_route_entry ire;
+	u32 vector;
+	struct irq_cfg *cfg;
+	int ioapic;
+	u8 ioapic_pin;
+	int ioapic_id;
+	int gsi;
+	union entry_union eu;
+	struct cpumask *affinity;
+	int cpu, vcpu;
+	struct hv_interrupt_entry entry;
+	struct mp_chip_data *mp_data = data->chip_data;
+
+	gsi = data->irq;
+	cfg = irqd_cfg(data);
+	affinity = irq_data_get_effective_affinity_mask(data);
+	cpu = cpumask_first_and(affinity, cpu_online_mask);
+	vcpu = hv_cpu_number_to_vp_number(cpu);
+
+	vector = cfg->vector;
+
+	ioapic = mp_find_ioapic(gsi);
+	ioapic_pin = mp_find_ioapic_pin(ioapic, gsi);
+	ioapic_id = mpc_ioapic_id(ioapic);
+	ire = ioapic_read_entry(ioapic, ioapic_pin);
+
+	/*
+	 * Always try unmapping. We do not have visibility into which whether
+	 * an IO-APIC has been mapped or not. We can't use chip_data because it
+	 * already points to mp_data.
+	 *
+	 * We don't use retarget interrupt hypercalls here because Hyper-V
+	 * doens't allow root to change the vector or specify VPs outside of
+	 * the set that is initially used during mapping.
+	 */
+	status = hv_unmap_ioapic_interrupt(gsi);
+
+	if (!(status == HV_STATUS_SUCCESS || status == HV_STATUS_INVALID_PARAMETER)) {
+		pr_debug("%s: unexpected unmap status %d\n", __func__, status);
+		return -1;
+	}
+
+	status = hv_map_ioapic_interrupt(ioapic_id, ire.trigger, vcpu, vector, &entry);
+
+	if (status != HV_STATUS_SUCCESS) {
+		pr_err("%s: map hypercall failed, status %d\n", __func__, status);
+		return -1;
+	}
+
+	/* Update the entry in mp_chip_data. It is used in other places. */
+	mp_data->entry = *(struct IO_APIC_route_entry *)&entry.ioapic_rte;
+
+	/* Sync polarity -- Hyper-V's returned polarity is always 0... */
+	mp_data->entry.polarity = ire.polarity;
+
+	eu.w1 = entry.ioapic_rte.low_uint32;
+	eu.w2 = entry.ioapic_rte.high_uint32;
+	ioapic_write_entry(ioapic, ioapic_pin, eu.entry);
+
+	return 0;
+}
+
+static void hv_ioapic_mask_irq(struct irq_data *data)
+{
+	mask_ioapic_irq(data);
+}
+
+static void hv_ioapic_unmask_irq(struct irq_data *data)
+{
+	unmask_ioapic_irq(data);
+}
+
+static int hv_ioapic_set_affinity(struct irq_data *data,
+			       const struct cpumask *mask, bool force)
+{
+	/*
+	 * We only update the affinity mask here. Programming the hardware is
+	 * done in irq_startup.
+	 */
+	return ioapic_set_affinity(data, mask, force);
+}
+
+void hv_ioapic_ack_level(struct irq_data *irq_data)
+{
+	/*
+	 * Per email exchange with Hyper-V team, all is needed is write to
+	 * LAPIC's EOI register. They don't support directed EOI to IO-APIC.
+	 * Hyper-V handles it for us.
+	 */
+	apic_ack_irq(irq_data);
+}
+
+struct irq_chip hv_ioapic_chip __read_mostly = {
+	.name			= "HV-IO-APIC",
+	.irq_startup		= hv_ioapic_startup_irq,
+	.irq_mask		= hv_ioapic_mask_irq,
+	.irq_unmask		= hv_ioapic_unmask_irq,
+	.irq_ack		= irq_chip_ack_parent,
+	.irq_eoi		= hv_ioapic_ack_level,
+	.irq_set_affinity	= hv_ioapic_set_affinity,
+	.irq_retrigger		= irq_chip_retrigger_hierarchy,
+	.irq_get_irqchip_state	= ioapic_irq_get_chip_state,
+	.flags			= IRQCHIP_SKIP_SET_WAKE,
+};
+
+
+int (*native_acpi_register_gsi)(struct device *dev, u32 gsi, int trigger, int polarity);
+void (*native_acpi_unregister_gsi)(u32 gsi);
+
+int hv_acpi_register_gsi(struct device *dev, u32 gsi, int trigger, int polarity)
+{
+	int irq = gsi;
+
+#ifdef CONFIG_X86_IO_APIC
+	irq = native_acpi_register_gsi(dev, gsi, trigger, polarity);
+	if (irq < 0) {
+		pr_err("native_acpi_register_gsi failed %d\n", irq);
+		return irq;
+	}
+
+	if (trigger) {
+		irq_set_status_flags(irq, IRQ_LEVEL);
+		irq_set_chip_and_handler_name(irq, &hv_ioapic_chip,
+			handle_fasteoi_irq, "ioapic-fasteoi");
+	} else {
+		irq_clear_status_flags(irq, IRQ_LEVEL);
+		irq_set_chip_and_handler_name(irq, &hv_ioapic_chip,
+			handle_edge_irq, "ioapic-edge");
+	}
+#endif
+	return irq;
+}
+
+void hv_acpi_unregister_gsi(u32 gsi)
+{
+#ifdef CONFIG_X86_IO_APIC
+	(void)hv_unmap_ioapic_interrupt(gsi);
+	native_acpi_unregister_gsi(gsi);
+#endif
+}
-- 
2.20.1


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if running as root
  2020-09-14 11:27 ` [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if " Wei Liu
@ 2020-09-15 10:10   ` Vitaly Kuznetsov
  2020-09-15 10:32     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 10:10 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Daniel Lezcano, Thomas Gleixner

Wei Liu <wei.liu@kernel.org> writes:

> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  drivers/clocksource/hyperv_timer.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
> index 09aa44cb8a91..fe96082ce85e 100644
> --- a/drivers/clocksource/hyperv_timer.c
> +++ b/drivers/clocksource/hyperv_timer.c
> @@ -426,6 +426,9 @@ static bool __init hv_init_tsc_clocksource(void)
>  	if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE))
>  		return false;
>  
> +	if (hv_root_partition)
> +		return false;
> +

Out of pure curiosity,

TSC page clocksource seems to be available to the root partition (as
HV_MSR_REFERENCE_TSC_AVAILABLE is set), why don't we use it? (I
understand that with TSC scaling support in modern CPUs even migration
is a no-issue and we can use raw TSC but this all seems to be
independent from root/non-root partition question).

>  	hv_read_reference_counter = read_hv_clock_tsc;
>  	phys_addr = virt_to_phys(hv_get_tsc_page());

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required
  2020-09-14 11:27 ` [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required Wei Liu
@ 2020-09-15 10:16   ` Vitaly Kuznetsov
  2020-09-15 12:43     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 10:16 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Wei Liu <wei.liu@kernel.org> writes:

> When Linux runs as the root partition, it will need to make hypercalls
> which return data from the hypervisor.
>
> Allocate pages for storing results when Linux runs as the root
> partition.
>
> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  arch/x86/hyperv/hv_init.c       | 45 +++++++++++++++++++++++++++++----
>  arch/x86/include/asm/mshyperv.h |  1 +
>  2 files changed, 41 insertions(+), 5 deletions(-)
>
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index cac8e4c56261..ebba4be4185d 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -45,6 +45,9 @@ EXPORT_SYMBOL_GPL(hv_vp_assist_page);
>  void  __percpu **hyperv_pcpu_input_arg;
>  EXPORT_SYMBOL_GPL(hyperv_pcpu_input_arg);
>  
> +void  __percpu **hyperv_pcpu_output_arg;
> +EXPORT_SYMBOL_GPL(hyperv_pcpu_output_arg);
> +
>  u32 hv_max_vp_index;
>  EXPORT_SYMBOL_GPL(hv_max_vp_index);
>  
> @@ -75,14 +78,29 @@ static int hv_cpu_init(unsigned int cpu)
>  	u64 msr_vp_index;
>  	struct hv_vp_assist_page **hvp = &hv_vp_assist_page[smp_processor_id()];
>  	void **input_arg;
> -	struct page *pg;
> +	struct page *input_pg;
>  
>  	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
>  	/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */
> -	pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
> -	if (unlikely(!pg))
> +	input_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
> +	if (unlikely(!input_pg))
>  		return -ENOMEM;
> -	*input_arg = page_address(pg);
> +	*input_arg = page_address(input_pg);
> +
> +	if (hv_root_partition) {
> +		struct page *output_pg;
> +		void **output_arg;
> +
> +		output_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC :
>  	GFP_KERNEL);

To simplify the code, can we just rename 'input_arg' to 'hypercall_args'
and do alloc_pages(rqs_disabled() ? GFP_ATOMIC : GFP_KERNEL, 1) to
allocate two pages above?

> +		if (unlikely(!output_pg)) {
> +			free_page((unsigned long)*input_arg);
> +			*input_arg = NULL;
> +			return -ENOMEM;
> +		}
> +
> +		output_arg = (void **)this_cpu_ptr(hyperv_pcpu_output_arg);
> +		*output_arg = page_address(output_pg);
> +	}
>  
>  	hv_get_vp_index(msr_vp_index);
>  
> @@ -209,14 +227,25 @@ static int hv_cpu_die(unsigned int cpu)
>  	unsigned int new_cpu;
>  	unsigned long flags;
>  	void **input_arg;
> -	void *input_pg = NULL;
> +	void *input_pg = NULL, *output_pg = NULL;
>  
>  	local_irq_save(flags);
>  	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
>  	input_pg = *input_arg;
>  	*input_arg = NULL;
> +
> +	if (hv_root_partition) {
> +		void **output_arg;
> +
> +		output_arg = (void **)this_cpu_ptr(hyperv_pcpu_output_arg);
> +		output_pg = *output_arg;
> +		*output_arg = NULL;
> +	}
> +
>  	local_irq_restore(flags);
> +
>  	free_page((unsigned long)input_pg);
> +	free_page((unsigned long)output_pg);
>  
>  	if (hv_vp_assist_page && hv_vp_assist_page[cpu])
>  		wrmsrl(HV_X64_MSR_VP_ASSIST_PAGE, 0);
> @@ -350,6 +379,12 @@ void __init hyperv_init(void)
>  
>  	BUG_ON(hyperv_pcpu_input_arg == NULL);
>  
> +	/* Allocate the per-CPU state for output arg for root */
> +	if (hv_root_partition) {
> +		hyperv_pcpu_output_arg = alloc_percpu(void  *);
					redundant space ^^^^^

> +		BUG_ON(hyperv_pcpu_output_arg == NULL);
> +	}
> +
>  	/* Allocate percpu VP index */
>  	hv_vp_index = kmalloc_array(num_possible_cpus(), sizeof(*hv_vp_index),
>  				    GFP_KERNEL);
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
> index 2a2cc81beac6..f5c62140f28d 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -63,6 +63,7 @@ static inline void hv_disable_stimer0_percpu_irq(int irq) {}
>  #if IS_ENABLED(CONFIG_HYPERV)
>  extern void *hv_hypercall_pg;
>  extern void  __percpu  **hyperv_pcpu_input_arg;
> +extern void  __percpu  **hyperv_pcpu_output_arg;
>  
>  static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
>  {

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary
  2020-09-14 11:27 ` [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary Wei Liu
@ 2020-09-15 10:27   ` Vitaly Kuznetsov
  2020-09-16 16:32     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 10:27 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

Wei Liu <wei.liu@kernel.org> writes:

> We will need the partition ID for executing some hypercalls later.
>
> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  arch/x86/hyperv/hv_init.c         | 26 ++++++++++++++++++++++++++
>  arch/x86/include/asm/mshyperv.h   |  2 ++
>  include/asm-generic/hyperv-tlfs.h |  6 ++++++
>  3 files changed, 34 insertions(+)
>
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index ebba4be4185d..0eec1ed32023 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -30,6 +30,9 @@
>  bool hv_root_partition;
>  EXPORT_SYMBOL_GPL(hv_root_partition);
>  
> +u64 hv_current_partition_id;
> +EXPORT_SYMBOL_GPL(hv_current_partition_id);
> +
>  void *hv_hypercall_pg;
>  EXPORT_SYMBOL_GPL(hv_hypercall_pg);
>  
> @@ -345,6 +348,26 @@ static struct syscore_ops hv_syscore_ops = {
>  	.resume		= hv_resume,
>  };
>  
> +void __init hv_get_partition_id(void)
> +{
> +	struct hv_get_partition_id *output_page;
> +	int status;
> +	unsigned long flags;
> +
> +	local_irq_save(flags);
> +	output_page = *this_cpu_ptr(hyperv_pcpu_output_arg);
> +	status = hv_do_hypercall(HVCALL_GET_PARTITION_ID, NULL, output_page) &
> +		HV_HYPERCALL_RESULT_MASK;

Nit: in this case status is 'u16', we can define it as such (instead of
signed int).

> +	if (status != HV_STATUS_SUCCESS)
> +		pr_err("Failed to get partition ID: %d\n", status);
> +	else
> +		hv_current_partition_id = output_page->partition_id;
> +	local_irq_restore(flags);
> +
> +	/* No point in proceeding if this failed */
> +	BUG_ON(status != HV_STATUS_SUCCESS);
> +}
> +
>  /*
>   * This function is to be invoked early in the boot sequence after the
>   * hypervisor has been detected.
> @@ -440,6 +463,9 @@ void __init hyperv_init(void)
>  
>  	register_syscore_ops(&hv_syscore_ops);
>  
> +	if (hv_root_partition)
> +		hv_get_partition_id();

According to TLFS, partition ID is available when AccessPartitionId
privilege is granted. I'd suggest we check that instead of
hv_root_partition (and we can set hv_current_partition_id to something
like U64_MAX so we know it wasn't acuired). So the BUG_ON condition will
move here:

        hv_get_partition_id();
        BUG_ON(hv_root_partition && hv_current_partition_id == U64_MAX);

> +
>  	return;
>  
>  remove_cpuhp_state:
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
> index f5c62140f28d..4039302e0ae9 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -65,6 +65,8 @@ extern void *hv_hypercall_pg;
>  extern void  __percpu  **hyperv_pcpu_input_arg;
>  extern void  __percpu  **hyperv_pcpu_output_arg;
>  
> +extern u64 hv_current_partition_id;
> +
>  static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
>  {
>  	u64 input_address = input ? virt_to_phys(input) : 0;
> diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
> index e6903589a82a..87b1a79b19eb 100644
> --- a/include/asm-generic/hyperv-tlfs.h
> +++ b/include/asm-generic/hyperv-tlfs.h
> @@ -141,6 +141,7 @@ struct ms_hyperv_tsc_page {
>  #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX	0x0013
>  #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX	0x0014
>  #define HVCALL_SEND_IPI_EX			0x0015
> +#define HVCALL_GET_PARTITION_ID			0x0046
>  #define HVCALL_GET_VP_REGISTERS			0x0050
>  #define HVCALL_SET_VP_REGISTERS			0x0051
>  #define HVCALL_POST_MESSAGE			0x005c
> @@ -407,6 +408,11 @@ struct hv_tlb_flush_ex {
>  	u64 gva_list[];
>  } __packed;
>  
> +/* HvGetPartitionId hypercall (output only) */
> +struct hv_get_partition_id {
> +	u64 partition_id;
> +} __packed;
> +
>  /* HvRetargetDeviceInterrupt hypercall */
>  union hv_msi_entry {
>  	u64 as_uint64;

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if running as root
  2020-09-15 10:10   ` Vitaly Kuznetsov
@ 2020-09-15 10:32     ` Wei Liu
  0 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-15 10:32 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, K. Y. Srinivasan,
	Haiyang Zhang, Stephen Hemminger, Daniel Lezcano,
	Thomas Gleixner

On Tue, Sep 15, 2020 at 12:10:04PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > ---
> >  drivers/clocksource/hyperv_timer.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
> > index 09aa44cb8a91..fe96082ce85e 100644
> > --- a/drivers/clocksource/hyperv_timer.c
> > +++ b/drivers/clocksource/hyperv_timer.c
> > @@ -426,6 +426,9 @@ static bool __init hv_init_tsc_clocksource(void)
> >  	if (!(ms_hyperv.features & HV_MSR_REFERENCE_TSC_AVAILABLE))
> >  		return false;
> >  
> > +	if (hv_root_partition)
> > +		return false;
> > +
> 
> Out of pure curiosity,
> 
> TSC page clocksource seems to be available to the root partition (as
> HV_MSR_REFERENCE_TSC_AVAILABLE is set), why don't we use it? (I
> understand that with TSC scaling support in modern CPUs even migration
> is a no-issue and we can use raw TSC but this all seems to be
> independent from root/non-root partition question).
> 

Yes. It is available to the root partition. An earlier version of this
patch did use that.  With some major restructuring of the code between
4.19 and 5.6 (splitting out the clocksource to and getting initialized
earlier), the old code wouldn't work anymore. 

The catch is that the page will have been set up by the hypervisor, so
in the function we will need to take a different path.  And making that
work again on 5.8 requires a bit more code churn.

Like I said in the cover letter, this is the bare minimum to get things
going. I try not distract people too much with less important stuff at
this stage, but making the reference TSC work again is definitely
something we want to look into in the future.

Wei.

> >  	hv_read_reference_counter = read_hv_clock_tsc;
> >  	phys_addr = virt_to_phys(hv_get_tsc_page());
> 
> -- 
> Vitaly
> 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-14 11:27 ` [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root Wei Liu
@ 2020-09-15 10:32   ` Vitaly Kuznetsov
  2020-09-15 10:37     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 10:32 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Wei Liu <wei.liu@kernel.org> writes:

> When Linux is running as the root partition, the hypercall page will
> have already been setup by Hyper-V. Copy the content over to the
> allocated page.

And we can't setup a new hypercall page by writing something different
to HV_X64_MSR_HYPERCALL, right?

>
> The suspend, resume and cleanup paths remain untouched because they are
> not supported in this setup yet.
>
> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
> Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
>  1 file changed, 24 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> index 0eec1ed32023..26233aebc86c 100644
> --- a/arch/x86/hyperv/hv_init.c
> +++ b/arch/x86/hyperv/hv_init.c
> @@ -25,6 +25,7 @@
>  #include <linux/cpuhotplug.h>
>  #include <linux/syscore_ops.h>
>  #include <clocksource/hyperv_timer.h>
> +#include <linux/highmem.h>
>  
>  /* Is Linux running as the root partition? */
>  bool hv_root_partition;
> @@ -448,8 +449,29 @@ void __init hyperv_init(void)
>  
>  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>  	hypercall_msr.enable = 1;
> -	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> -	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> +
> +	if (hv_root_partition) {
> +		struct page *pg;
> +		void *src, *dst;
> +
> +		/*
> +		 * Order is important here. We must enable the hypercall page
> +		 * so it is populated with code, then copy the code to an
> +		 * executable page.
> +		 */
> +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> +
> +		pg = vmalloc_to_page(hv_hypercall_pg);
> +		dst = kmap(pg);
> +		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
> +				MEMREMAP_WB);

memremap() can fail...

> +		memcpy(dst, src, PAGE_SIZE);
> +		memunmap(src);
> +		kunmap(pg);
> +	} else {
> +		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> +	}

Why can't we do wrmsrl() for both cases here?

>  
>  	/*
>  	 * Ignore any errors in setting up stimer clockevents

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-15 10:32   ` Vitaly Kuznetsov
@ 2020-09-15 10:37     ` Wei Liu
  2020-09-15 11:02       ` Vitaly Kuznetsov
  2020-09-16 21:34       ` [EXTERNAL] " Sunil Muthuswamy
  0 siblings, 2 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-15 10:37 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > When Linux is running as the root partition, the hypercall page will
> > have already been setup by Hyper-V. Copy the content over to the
> > allocated page.
> 
> And we can't setup a new hypercall page by writing something different
> to HV_X64_MSR_HYPERCALL, right?
> 

My understanding is that we can't, but Sunil can maybe correct me.

> >
> > The suspend, resume and cleanup paths remain untouched because they are
> > not supported in this setup yet.
> >
> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > ---
> >  arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
> >  1 file changed, 24 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> > index 0eec1ed32023..26233aebc86c 100644
> > --- a/arch/x86/hyperv/hv_init.c
> > +++ b/arch/x86/hyperv/hv_init.c
> > @@ -25,6 +25,7 @@
> >  #include <linux/cpuhotplug.h>
> >  #include <linux/syscore_ops.h>
> >  #include <clocksource/hyperv_timer.h>
> > +#include <linux/highmem.h>
> >  
> >  /* Is Linux running as the root partition? */
> >  bool hv_root_partition;
> > @@ -448,8 +449,29 @@ void __init hyperv_init(void)
> >  
> >  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >  	hypercall_msr.enable = 1;
> > -	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> > -	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> > +
> > +	if (hv_root_partition) {
> > +		struct page *pg;
> > +		void *src, *dst;
> > +
> > +		/*
> > +		 * Order is important here. We must enable the hypercall page
> > +		 * so it is populated with code, then copy the code to an
> > +		 * executable page.
> > +		 */
> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> > +
> > +		pg = vmalloc_to_page(hv_hypercall_pg);
> > +		dst = kmap(pg);
> > +		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
> > +				MEMREMAP_WB);
> 
> memremap() can fail...

And we don't care here, if it fails, we would rather it panic or oops.

I was relying on the fact that copying from / to a NULL pointer will
cause the kernel to crash. But of course it wouldn't hurt to explicitly
panic here.

> 
> > +		memcpy(dst, src, PAGE_SIZE);
> > +		memunmap(src);
> > +		kunmap(pg);
> > +	} else {
> > +		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> > +	}
> 
> Why can't we do wrmsrl() for both cases here?
> 

Because the hypercall page has already been set up when Linux is the
root.

I could've tried writing to the MSR again, but because the behaviour
here is not documented and subject to change so I didn't bother trying.

Wei.

> >  
> >  	/*
> >  	 * Ignore any errors in setting up stimer clockevents
> 
> -- 
> Vitaly
> 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions
  2020-09-14 11:59 ` [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions Wei Liu
@ 2020-09-15 11:00   ` Vitaly Kuznetsov
  2020-10-27 13:10     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 11:00 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

Wei Liu <wei.liu@kernel.org> writes:

> They are used to deposit pages into Microsoft Hypervisor and bring up
> logical and virtual processors.
>
> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
> Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  arch/x86/hyperv/Makefile          |   2 +-
>  arch/x86/hyperv/hv_proc.c         | 209 ++++++++++++++++++++++++++++++
>  arch/x86/include/asm/mshyperv.h   |   4 +
>  include/asm-generic/hyperv-tlfs.h |  56 ++++++++
>  4 files changed, 270 insertions(+), 1 deletion(-)
>  create mode 100644 arch/x86/hyperv/hv_proc.c
>
> diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
> index 89b1f74d3225..565358020921 100644
> --- a/arch/x86/hyperv/Makefile
> +++ b/arch/x86/hyperv/Makefile
> @@ -1,6 +1,6 @@
>  # SPDX-License-Identifier: GPL-2.0-only
>  obj-y			:= hv_init.o mmu.o nested.o
> -obj-$(CONFIG_X86_64)	+= hv_apic.o
> +obj-$(CONFIG_X86_64)	+= hv_apic.o hv_proc.o
>  
>  ifdef CONFIG_X86_64
>  obj-$(CONFIG_PARAVIRT_SPINLOCKS)	+= hv_spinlock.o
> diff --git a/arch/x86/hyperv/hv_proc.c b/arch/x86/hyperv/hv_proc.c
> new file mode 100644
> index 000000000000..847c72465d0e
> --- /dev/null
> +++ b/arch/x86/hyperv/hv_proc.c
> @@ -0,0 +1,209 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/types.h>
> +#include <linux/version.h>
> +#include <linux/vmalloc.h>
> +#include <linux/mm.h>
> +#include <linux/clockchips.h>
> +#include <linux/acpi.h>
> +#include <linux/hyperv.h>
> +#include <linux/slab.h>
> +#include <linux/cpuhotplug.h>
> +#include <asm/hypervisor.h>
> +#include <asm/mshyperv.h>
> +#include <asm/apic.h>
> +
> +#include <asm/trace/hyperv.h>
> +
> +#define HV_DEPOSIT_MAX_ORDER (8)
> +#define HV_DEPOSIT_MAX (1 << HV_DEPOSIT_MAX_ORDER)
> +
> +#define MAX(a, b) ((a) > (b) ? (a) : (b))
> +#define MIN(a, b) ((a) < (b) ? (a) : (b))

Nit: include/linux/kernel.h defines min() and max() macros with type
checking.

> +
> +/*
> + * Deposits exact number of pages
> + * Must be called with interrupts enabled
> + * Max 256 pages
> + */
> +int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages)
> +{
> +	struct page **pages;
> +	int *counts;
> +	int num_allocations;
> +	int i, j, page_count;
> +	int order;
> +	int desired_order;
> +	int status;
> +	int ret;
> +	u64 base_pfn;
> +	struct hv_deposit_memory *input_page;
> +	unsigned long flags;
> +
> +	if (num_pages > HV_DEPOSIT_MAX)
> +		return -EINVAL;
> +	if (!num_pages)
> +		return 0;
> +
> +	ret = -ENOMEM;
> +
> +	/* One buffer for page pointers and counts */
> +	pages = page_address(alloc_page(GFP_KERNEL));
> +	if (!pages)
> +		goto free_buf;

There is nothing to free, just do 'return -ENOMEM' here;

> +	counts = (int *)&pages[256];
> +

Oh this is weird. So 'pages' is an array of 512 'struct page *' items
and we use its second half (pages[256]) for an array of signed(!)
integers(!). Can we use a locally defined struct or something better for
that?

> +	/* Allocate all the pages before disabling interrupts */
> +	num_allocations = 0;
> +	i = 0;
> +	order = HV_DEPOSIT_MAX_ORDER;
> +
> +	while (num_pages) {
> +		/* Find highest order we can actually allocate */
> +		desired_order = 31 - __builtin_clz(num_pages);
> +		order = MIN(desired_order, order);
> +		do {
> +			pages[i] = alloc_pages_node(node, GFP_KERNEL, order);
> +			if (!pages[i]) {
> +				if (!order) {
> +					goto err_free_allocations;
> +				}
> +				--order;
> +			}
> +		} while (!pages[i]);
> +
> +		split_page(pages[i], order);
> +		counts[i] = 1 << order;
> +		num_pages -= counts[i];
> +		i++;

So here we believe we will never overrun the 2048 bytes we 'allocated'
for 'counts' above. While 'if (num_pages > HV_DEPOSIT_MAX)' presumably
guarantees that, this is not really obvious.

> +		num_allocations++;
> +	}
> +
> +	local_irq_save(flags);
> +
> +	input_page = *this_cpu_ptr(hyperv_pcpu_input_arg);
> +
> +	input_page->partition_id = partition_id;
> +
> +	/* Populate gpa_page_list - these will fit on the input page */
> +	for (i = 0, page_count = 0; i < num_allocations; ++i) {
> +		base_pfn = page_to_pfn(pages[i]);
> +		for (j = 0; j < counts[i]; ++j, ++page_count)
> +			input_page->gpa_page_list[page_count] = base_pfn + j;
> +	}
> +	status = hv_do_rep_hypercall(HVCALL_DEPOSIT_MEMORY,
> +				     page_count, 0, input_page,
> +				     NULL) & HV_HYPERCALL_RESULT_MASK;
> +	local_irq_restore(flags);
> +
> +	if (status != HV_STATUS_SUCCESS) {

Nit: same like in one ov the previous patches, status can be 'u16'.

> +		pr_err("Failed to deposit pages: %d\n", status);
> +		ret = status;
> +		goto err_free_allocations;
> +	}
> +
> +	ret = 0;
> +	goto free_buf;
> +
> +err_free_allocations:
> +	for (i = 0; i < num_allocations; ++i) {
> +		base_pfn = page_to_pfn(pages[i]);
> +		for (j = 0; j < counts[i]; ++j)
> +			__free_page(pfn_to_page(base_pfn + j));
> +	}
> +
> +free_buf:
> +	free_page((unsigned long)pages);
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hv_call_deposit_pages);
> +
> +int hv_call_add_logical_proc(int node, u32 lp_index, u32 apic_id)
> +{
> +	struct hv_add_logical_processor_in *input;
> +	struct hv_add_logical_processor_out *output;
> +	int status;
> +	unsigned long flags;
> +	int ret = 0;
> +
> +	do {
> +		local_irq_save(flags);
> +
> +		input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> +		/* We don't do anything with the output right now */
> +		output = *this_cpu_ptr(hyperv_pcpu_output_arg);
> +
> +		input->lp_index = lp_index;
> +		input->apic_id = apic_id;
> +		input->flags = 0;
> +		input->proximity_domain_info.domain_id = node_to_pxm(node);
> +		input->proximity_domain_info.flags.reserved = 0;
> +		input->proximity_domain_info.flags.proximity_info_valid = 1;
> +		input->proximity_domain_info.flags.proximity_preferred = 1;
> +		status = hv_do_hypercall(HVCALL_ADD_LOGICAL_PROCESSOR,
> +					 input, output);
> +		local_irq_restore(flags);
> +
> +		if (status != HV_STATUS_INSUFFICIENT_MEMORY) {
> +			if (status != HV_STATUS_SUCCESS) {
> +				pr_err("%s: cpu %u apic ID %u, %d\n", __func__,
> +				       lp_index, apic_id, status);
> +				ret = status;
> +			}
> +			break;

So if status == HV_STATUS_SUCCESS we break and avoid
hv_call_deposit_pages() below?

> +		}
> +		ret = hv_call_deposit_pages(node, hv_current_partition_id, 1);
> +
> +	} while (!ret);

And if hv_call_deposit_pages() returns '0' we keep doing something? Sorry
but I'm probably missing something important in the 'depositing'
process, could you please add a comment explaining what's going on here?

> +
> +	return ret;
> +}
> +
> +int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags)
> +{
> +	struct hv_create_vp *input;
> +	int status;
> +	unsigned long irq_flags;
> +	int ret = 0;
> +
> +	/* Root VPs don't seem to need pages deposited */
> +	if (partition_id != hv_current_partition_id) {
> +		ret = hv_call_deposit_pages(node, partition_id, 90);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	do {
> +		local_irq_save(irq_flags);
> +
> +		input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> +
> +		input->partition_id = partition_id;
> +		input->vp_index = vp_index;
> +		input->flags = flags;
> +		if (node != NUMA_NO_NODE) {
> +			input->proximity_domain_info.domain_id = node_to_pxm(node);
> +			input->proximity_domain_info.flags.reserved = 0;
> +			input->proximity_domain_info.flags.proximity_info_valid = 1;
> +			input->proximity_domain_info.flags.proximity_preferred = 1;
> +		} else {
> +			input->proximity_domain_info.as_uint64 = 0;
> +		}
> +		status = hv_do_hypercall(HVCALL_CREATE_VP, input, NULL);
> +		local_irq_restore(irq_flags);
> +
> +		if (status != HV_STATUS_INSUFFICIENT_MEMORY) {
> +			if (status != HV_STATUS_SUCCESS) {
> +				pr_err("%s: vcpu %u, lp %u, %d\n", __func__,
> +				       vp_index, flags, status);
> +				ret = status;
> +			}
> +			break;
> +		}
> +		ret = hv_call_deposit_pages(node, partition_id, 1);
> +
> +	} while (!ret);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(hv_call_create_vp);
> +
> diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
> index 4039302e0ae9..60afc3e417d0 100644
> --- a/arch/x86/include/asm/mshyperv.h
> +++ b/arch/x86/include/asm/mshyperv.h
> @@ -67,6 +67,10 @@ extern void  __percpu  **hyperv_pcpu_output_arg;
>  
>  extern u64 hv_current_partition_id;
>  
> +int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages);
> +int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id);
> +int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags);
> +
>  static inline u64 hv_do_hypercall(u64 control, void *input, void *output)
>  {
>  	u64 input_address = input ? virt_to_phys(input) : 0;
> diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
> index 87b1a79b19eb..2b05bed712c0 100644
> --- a/include/asm-generic/hyperv-tlfs.h
> +++ b/include/asm-generic/hyperv-tlfs.h
> @@ -142,6 +142,8 @@ struct ms_hyperv_tsc_page {
>  #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX	0x0014
>  #define HVCALL_SEND_IPI_EX			0x0015
>  #define HVCALL_GET_PARTITION_ID			0x0046
> +#define HVCALL_DEPOSIT_MEMORY			0x0048
> +#define HVCALL_CREATE_VP			0x004e
>  #define HVCALL_GET_VP_REGISTERS			0x0050
>  #define HVCALL_SET_VP_REGISTERS			0x0051
>  #define HVCALL_POST_MESSAGE			0x005c
> @@ -149,6 +151,7 @@ struct ms_hyperv_tsc_page {
>  #define HVCALL_POST_DEBUG_DATA			0x0069
>  #define HVCALL_RETRIEVE_DEBUG_DATA		0x006a
>  #define HVCALL_RESET_DEBUG_SESSION		0x006b
> +#define HVCALL_ADD_LOGICAL_PROCESSOR		0x0076
>  #define HVCALL_RETARGET_INTERRUPT		0x007e
>  #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
>  #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
> @@ -413,6 +416,59 @@ struct hv_get_partition_id {
>  	u64 partition_id;
>  } __packed;
>  
> +/* HvDepositMemory hypercall */
> +struct hv_deposit_memory {
> +	u64 partition_id;
> +	u64 gpa_page_list[];
> +};

Other structures above have '__packed' and I remember there were
different opinions if it is needed or not (for properly padded
structures). I'd suggest we stay consitent and keep adding it unless we
decide to get rid of them (but you've added it to the newly introduced
hv_get_partition_id above).
> +
> +
> +struct hv_proximity_domain_flags {
> +	u32 proximity_preferred : 1;
> +	u32 reserved : 30;
> +	u32 proximity_info_valid : 1;
> +};
> +
> +/* Not a union in windows but useful for zeroing */
> +union hv_proximity_domain_info {
> +	struct {
> +		u32 domain_id;
> +		struct hv_proximity_domain_flags flags;
> +	};
> +	u64 as_uint64;
> +};
> +
> +struct hv_lp_startup_status {
> +	u64 hv_status;
> +	u64 substatus1;
> +	u64 substatus2;
> +	u64 substatus3;
> +	u64 substatus4;
> +	u64 substatus5;
> +	u64 substatus6;
> +};
> +
> +/* HvAddLogicalProcessor hypercalls */

s/hypercalls/hypercall/

> +struct hv_add_logical_processor_in {
> +	u32 lp_index;
> +	u32 apic_id;
> +	union hv_proximity_domain_info proximity_domain_info;
> +	u64 flags;
> +};
> +
> +struct hv_add_logical_processor_out {
> +	struct hv_lp_startup_status startup_status;
> +};
> +
> +/* HvCreateVp hypercall */
> +struct hv_create_vp {
> +	u64 partition_id;
> +	u32 vp_index;
> +	u32 padding;
> +	union hv_proximity_domain_info proximity_domain_info;
> +	u64 flags;
> +};
> +
>  /* HvRetargetDeviceInterrupt hypercall */
>  union hv_msi_entry {
>  	u64 as_uint64;

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-15 10:37     ` Wei Liu
@ 2020-09-15 11:02       ` Vitaly Kuznetsov
  2020-09-15 11:16         ` Wei Liu
  2020-09-16 21:34       ` [EXTERNAL] " Sunil Muthuswamy
  1 sibling, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 11:02 UTC (permalink / raw)
  To: Wei Liu
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Wei Liu <wei.liu@kernel.org> writes:

> On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
>> Wei Liu <wei.liu@kernel.org> writes:
>> 
>> > When Linux is running as the root partition, the hypercall page will
>> > have already been setup by Hyper-V. Copy the content over to the
>> > allocated page.
>> 
>> And we can't setup a new hypercall page by writing something different
>> to HV_X64_MSR_HYPERCALL, right?
>> 
>
> My understanding is that we can't, but Sunil can maybe correct me.
>
>> >
>> > The suspend, resume and cleanup paths remain untouched because they are
>> > not supported in this setup yet.
>> >
>> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
>> > Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
>> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
>> > Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
>> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
>> > ---
>> >  arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
>> >  1 file changed, 24 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
>> > index 0eec1ed32023..26233aebc86c 100644
>> > --- a/arch/x86/hyperv/hv_init.c
>> > +++ b/arch/x86/hyperv/hv_init.c
>> > @@ -25,6 +25,7 @@
>> >  #include <linux/cpuhotplug.h>
>> >  #include <linux/syscore_ops.h>
>> >  #include <clocksource/hyperv_timer.h>
>> > +#include <linux/highmem.h>
>> >  
>> >  /* Is Linux running as the root partition? */
>> >  bool hv_root_partition;
>> > @@ -448,8 +449,29 @@ void __init hyperv_init(void)
>> >  
>> >  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> >  	hypercall_msr.enable = 1;
>> > -	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
>> > -	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > +
>> > +	if (hv_root_partition) {
>> > +		struct page *pg;
>> > +		void *src, *dst;
>> > +
>> > +		/*
>> > +		 * Order is important here. We must enable the hypercall page
>> > +		 * so it is populated with code, then copy the code to an
>> > +		 * executable page.
>> > +		 */
>> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > +
>> > +		pg = vmalloc_to_page(hv_hypercall_pg);
>> > +		dst = kmap(pg);
>> > +		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
>> > +				MEMREMAP_WB);
>> 
>> memremap() can fail...
>
> And we don't care here, if it fails, we would rather it panic or oops.
>
> I was relying on the fact that copying from / to a NULL pointer will
> cause the kernel to crash. But of course it wouldn't hurt to explicitly
> panic here.
>
>> 
>> > +		memcpy(dst, src, PAGE_SIZE);
>> > +		memunmap(src);
>> > +		kunmap(pg);
>> > +	} else {
>> > +		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
>> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> > +	}
>> 
>> Why can't we do wrmsrl() for both cases here?
>> 
>
> Because the hypercall page has already been set up when Linux is the
> root.

But you already do wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64)
in 'if (hv_root_partition)' case above, that's why I asked.

>
> I could've tried writing to the MSR again, but because the behaviour
> here is not documented and subject to change so I didn't bother trying.
>
> Wei.
>
>> >  
>> >  	/*
>> >  	 * Ignore any errors in setting up stimer clockevents
>> 
>> -- 
>> Vitaly
>> 
>

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus
  2020-09-14 11:59 ` [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus Wei Liu
@ 2020-09-15 11:14   ` Vitaly Kuznetsov
  2020-10-27 13:47     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 11:14 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	Lillian Grassin-Drake, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Wei Liu <wei.liu@kernel.org> writes:

> Microsoft Hypervisor requires the root partition to make a few
> hypercalls to setup application processors before they can be used.
>
> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
> CPU hotplug and unplug is not yet supported in this setup, so those
> paths remain untouched.
> ---
>  arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
>
> diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> index 1bf57d310f78..7522cae02759 100644
> --- a/arch/x86/kernel/cpu/mshyperv.c
> +++ b/arch/x86/kernel/cpu/mshyperv.c
> @@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
>  	hv_init_spinlocks();
>  #endif
>  }
> +
> +static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
> +{
> +#if defined(CONFIG_X86_64)

I think it makes little sense to try to make Linux work as Hyper-V root
partition when !CONFIG_X86_64. If we still care about Hyper-V enablement
for !CONFIG_X86_64 we can probably introduce something like
CONFIG_HYPERV_ROOT and enable it automatically, e.g.

config HYPERV_ROOT
        def_bool HYPERV && X86_64

and use it instead.

> +	int i;
> +	int vp_index = 1;
> +	int ret;
> +
> +	native_smp_prepare_cpus(max_cpus);
> +
> +	for_each_present_cpu(i) {
> +		if (i == 0)
> +			continue;
> +		ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
> +		BUG_ON(ret);
> +	}
> +
> +	for_each_present_cpu(i) {
> +		if (i == 0)
> +			continue;
> +		ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);

So vp_index variable is needed here to make sure there are no gaps? (or
we could've just used 'i')?

> +		BUG_ON(ret);
> +	}
> +#endif
> +}
>  #endif
>  
>  static void __init ms_hyperv_init_platform(void)
> @@ -359,6 +384,8 @@ static void __init ms_hyperv_init_platform(void)
>  
>  # ifdef CONFIG_SMP
>  	smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu;
> +	if (hv_root_partition)
> +		smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
>  # endif
>  
>  	/*

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-15 11:02       ` Vitaly Kuznetsov
@ 2020-09-15 11:16         ` Wei Liu
  2020-09-15 11:23           ` Vitaly Kuznetsov
  0 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-15 11:16 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Sep 15, 2020 at 01:02:08PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
> >> Wei Liu <wei.liu@kernel.org> writes:
> >> 
> >> > When Linux is running as the root partition, the hypercall page will
> >> > have already been setup by Hyper-V. Copy the content over to the
> >> > allocated page.
> >> 
> >> And we can't setup a new hypercall page by writing something different
> >> to HV_X64_MSR_HYPERCALL, right?
> >> 
> >
> > My understanding is that we can't, but Sunil can maybe correct me.
> >
> >> >
> >> > The suspend, resume and cleanup paths remain untouched because they are
> >> > not supported in this setup yet.
> >> >
> >> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> >> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> >> > Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
> >> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> >> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> >> > Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
> >> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> >> > ---
> >> >  arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
> >> >  1 file changed, 24 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> >> > index 0eec1ed32023..26233aebc86c 100644
> >> > --- a/arch/x86/hyperv/hv_init.c
> >> > +++ b/arch/x86/hyperv/hv_init.c
> >> > @@ -25,6 +25,7 @@
> >> >  #include <linux/cpuhotplug.h>
> >> >  #include <linux/syscore_ops.h>
> >> >  #include <clocksource/hyperv_timer.h>
> >> > +#include <linux/highmem.h>
> >> >  
> >> >  /* Is Linux running as the root partition? */
> >> >  bool hv_root_partition;
> >> > @@ -448,8 +449,29 @@ void __init hyperv_init(void)
> >> >  
> >> >  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> >  	hypercall_msr.enable = 1;
> >> > -	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> >> > -	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> > +
> >> > +	if (hv_root_partition) {
> >> > +		struct page *pg;
> >> > +		void *src, *dst;
> >> > +
> >> > +		/*
> >> > +		 * Order is important here. We must enable the hypercall page
> >> > +		 * so it is populated with code, then copy the code to an
> >> > +		 * executable page.
> >> > +		 */
> >> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> > +
> >> > +		pg = vmalloc_to_page(hv_hypercall_pg);
> >> > +		dst = kmap(pg);
> >> > +		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
> >> > +				MEMREMAP_WB);
> >> 
> >> memremap() can fail...
> >
> > And we don't care here, if it fails, we would rather it panic or oops.
> >
> > I was relying on the fact that copying from / to a NULL pointer will
> > cause the kernel to crash. But of course it wouldn't hurt to explicitly
> > panic here.
> >
> >> 
> >> > +		memcpy(dst, src, PAGE_SIZE);
> >> > +		memunmap(src);
> >> > +		kunmap(pg);
> >> > +	} else {
> >> > +		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> >> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> > +	}
> >> 
> >> Why can't we do wrmsrl() for both cases here?
> >> 
> >
> > Because the hypercall page has already been set up when Linux is the
> > root.
> 
> But you already do wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64)
> in 'if (hv_root_partition)' case above, that's why I asked.
> 

You mean extracting wrmsrl to this point?  The ordering matters. See the
comment in the root branch -- we have to enable the page before copying
the content.

What can be done is:

   if (!root) {
       /* some stuff */
   }

   wrmsrl(...)

   if (root) {
       /* some stuff */
   }

This is not looking any better than the existing code.

Wei.

> >
> > I could've tried writing to the MSR again, but because the behaviour
> > here is not documented and subject to change so I didn't bother trying.
> >
> > Wei.
> >
> >> >  
> >> >  	/*
> >> >  	 * Ignore any errors in setting up stimer clockevents
> >> 
> >> -- 
> >> Vitaly
> >> 
> >
> 
> -- 
> Vitaly
> 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures
  2020-09-14 11:59 ` [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures Wei Liu
@ 2020-09-15 11:16   ` Vitaly Kuznetsov
  2020-09-15 11:59     ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 11:16 UTC (permalink / raw)
  To: Wei Liu, Linux on Hyper-V List
  Cc: virtualization, Linux Kernel List, Michael Kelley,
	Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves, Wei Liu,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Arnd Bergmann, open list:GENERIC INCLUDE/ASM HEADER FILES

Wei Liu <wei.liu@kernel.org> writes:

> We will need to identify the device we want Microsoft Hypervisor to
> manipulate.  Introduce the data structures for that purpose.
>
> They will be used in a later patch.
>
> Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  include/asm-generic/hyperv-tlfs.h | 79 +++++++++++++++++++++++++++++++
>  1 file changed, 79 insertions(+)
>
> diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
> index 83945ada5a50..faf892ce152d 100644
> --- a/include/asm-generic/hyperv-tlfs.h
> +++ b/include/asm-generic/hyperv-tlfs.h
> @@ -612,4 +612,83 @@ struct hv_set_vp_registers_input {
>  	} element[];
>  } __packed;
>  
> +enum hv_device_type {
> +	HV_DEVICE_TYPE_LOGICAL = 0,
> +	HV_DEVICE_TYPE_PCI = 1,
> +	HV_DEVICE_TYPE_IOAPIC = 2,
> +	HV_DEVICE_TYPE_ACPI = 3,
> +};
> +
> +typedef u16 hv_pci_rid;
> +typedef u16 hv_pci_segment;
> +typedef u64 hv_logical_device_id;
> +union hv_pci_bdf {
> +	u16 as_uint16;
> +
> +	struct {
> +		u8 function:3;
> +		u8 device:5;
> +		u8 bus;
> +	};
> +} __packed;
> +
> +union hv_pci_bus_range {
> +	u16 as_uint16;
> +
> +	struct {
> +		u8 subordinate_bus;
> +		u8 secondary_bus;
> +	};
> +} __packed;
> +
> +union hv_device_id {
> +	u64 as_uint64;
> +
> +	struct {
> +		u64 :62;
> +		u64 device_type:2;
> +	};
> +
> +	// HV_DEVICE_TYPE_LOGICAL

Nit: please no '//' comments.

> +	struct {
> +		u64 id:62;
> +		u64 device_type:2;
> +	} logical;
> +
> +	// HV_DEVICE_TYPE_PCI
> +	struct {
> +		union {
> +			hv_pci_rid rid;
> +			union hv_pci_bdf bdf;
> +		};
> +
> +		hv_pci_segment segment;
> +		union hv_pci_bus_range shadow_bus_range;
> +
> +		u16 phantom_function_bits:2;
> +		u16 source_shadow:1;
> +
> +		u16 rsvdz0:11;
> +		u16 device_type:2;
> +	} pci;
> +
> +	// HV_DEVICE_TYPE_IOAPIC
> +	struct {
> +		u8 ioapic_id;
> +		u8 rsvdz0;
> +		u16 rsvdz1;
> +		u16 rsvdz2;
> +
> +		u16 rsvdz3:14;
> +		u16 device_type:2;
> +	} ioapic;
> +
> +	// HV_DEVICE_TYPE_ACPI
> +	struct {
> +		u32 input_mapping_base;
> +		u32 input_mapping_count:30;
> +		u32 device_type:2;
> +	} acpi;
> +} __packed;
> +
>  #endif

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-15 11:16         ` Wei Liu
@ 2020-09-15 11:23           ` Vitaly Kuznetsov
  2020-09-15 11:27             ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-15 11:23 UTC (permalink / raw)
  To: Wei Liu
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Wei Liu <wei.liu@kernel.org> writes:

> On Tue, Sep 15, 2020 at 01:02:08PM +0200, Vitaly Kuznetsov wrote:
>> Wei Liu <wei.liu@kernel.org> writes:
>> 
>> > On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
>> >> Wei Liu <wei.liu@kernel.org> writes:
>> >> 
>> >> > When Linux is running as the root partition, the hypercall page will
>> >> > have already been setup by Hyper-V. Copy the content over to the
>> >> > allocated page.
>> >> 
>> >> And we can't setup a new hypercall page by writing something different
>> >> to HV_X64_MSR_HYPERCALL, right?
>> >> 
>> >
>> > My understanding is that we can't, but Sunil can maybe correct me.
>> >
>> >> >
>> >> > The suspend, resume and cleanup paths remain untouched because they are
>> >> > not supported in this setup yet.
>> >> >
>> >> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> >> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
>> >> > Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
>> >> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> >> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
>> >> > Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
>> >> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
>> >> > ---
>> >> >  arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
>> >> >  1 file changed, 24 insertions(+), 2 deletions(-)
>> >> >
>> >> > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
>> >> > index 0eec1ed32023..26233aebc86c 100644
>> >> > --- a/arch/x86/hyperv/hv_init.c
>> >> > +++ b/arch/x86/hyperv/hv_init.c
>> >> > @@ -25,6 +25,7 @@
>> >> >  #include <linux/cpuhotplug.h>
>> >> >  #include <linux/syscore_ops.h>
>> >> >  #include <clocksource/hyperv_timer.h>
>> >> > +#include <linux/highmem.h>
>> >> >  
>> >> >  /* Is Linux running as the root partition? */
>> >> >  bool hv_root_partition;
>> >> > @@ -448,8 +449,29 @@ void __init hyperv_init(void)
>> >> >  
>> >> >  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> >> >  	hypercall_msr.enable = 1;
>> >> > -	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
>> >> > -	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> >> > +
>> >> > +	if (hv_root_partition) {
>> >> > +		struct page *pg;
>> >> > +		void *src, *dst;
>> >> > +
>> >> > +		/*
>> >> > +		 * Order is important here. We must enable the hypercall page
>> >> > +		 * so it is populated with code, then copy the code to an
>> >> > +		 * executable page.
>> >> > +		 */
>> >> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> >> > +
>> >> > +		pg = vmalloc_to_page(hv_hypercall_pg);
>> >> > +		dst = kmap(pg);
>> >> > +		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
>> >> > +				MEMREMAP_WB);
>> >> 
>> >> memremap() can fail...
>> >
>> > And we don't care here, if it fails, we would rather it panic or oops.
>> >
>> > I was relying on the fact that copying from / to a NULL pointer will
>> > cause the kernel to crash. But of course it wouldn't hurt to explicitly
>> > panic here.
>> >
>> >> 
>> >> > +		memcpy(dst, src, PAGE_SIZE);
>> >> > +		memunmap(src);
>> >> > +		kunmap(pg);
>> >> > +	} else {
>> >> > +		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
>> >> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>> >> > +	}
>> >> 
>> >> Why can't we do wrmsrl() for both cases here?
>> >> 
>> >
>> > Because the hypercall page has already been set up when Linux is the
>> > root.
>> 
>> But you already do wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64)
>> in 'if (hv_root_partition)' case above, that's why I asked.
>> 
>
> You mean extracting wrmsrl to this point?  The ordering matters. See the
> comment in the root branch -- we have to enable the page before copying
> the content.
>
> What can be done is:
>
>    if (!root) {
>        /* some stuff */
>    }
>
>    wrmsrl(...)
>
>    if (root) {
>        /* some stuff */
>    }
>
> This is not looking any better than the existing code.
>

Oh, I missed the comment indeed. So Hypervisor already picked a page for
us, however, it didn't enable it and it's not populated? How can we be
sure that we didn't use it for something else already? Maybe we can
still give a different known-to-be-empty page?

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-15 11:23           ` Vitaly Kuznetsov
@ 2020-09-15 11:27             ` Wei Liu
  0 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-15 11:27 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Sep 15, 2020 at 01:23:50PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > On Tue, Sep 15, 2020 at 01:02:08PM +0200, Vitaly Kuznetsov wrote:
> >> Wei Liu <wei.liu@kernel.org> writes:
> >> 
> >> > On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
> >> >> Wei Liu <wei.liu@kernel.org> writes:
> >> >> 
> >> >> > When Linux is running as the root partition, the hypercall page will
> >> >> > have already been setup by Hyper-V. Copy the content over to the
> >> >> > allocated page.
> >> >> 
> >> >> And we can't setup a new hypercall page by writing something different
> >> >> to HV_X64_MSR_HYPERCALL, right?
> >> >> 
> >> >
> >> > My understanding is that we can't, but Sunil can maybe correct me.
> >> >
> >> >> >
> >> >> > The suspend, resume and cleanup paths remain untouched because they are
> >> >> > not supported in this setup yet.
> >> >> >
> >> >> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> >> >> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> >> >> > Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
> >> >> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> >> >> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> >> >> > Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
> >> >> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> >> >> > ---
> >> >> >  arch/x86/hyperv/hv_init.c | 26 ++++++++++++++++++++++++--
> >> >> >  1 file changed, 24 insertions(+), 2 deletions(-)
> >> >> >
> >> >> > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> >> >> > index 0eec1ed32023..26233aebc86c 100644
> >> >> > --- a/arch/x86/hyperv/hv_init.c
> >> >> > +++ b/arch/x86/hyperv/hv_init.c
> >> >> > @@ -25,6 +25,7 @@
> >> >> >  #include <linux/cpuhotplug.h>
> >> >> >  #include <linux/syscore_ops.h>
> >> >> >  #include <clocksource/hyperv_timer.h>
> >> >> > +#include <linux/highmem.h>
> >> >> >  
> >> >> >  /* Is Linux running as the root partition? */
> >> >> >  bool hv_root_partition;
> >> >> > @@ -448,8 +449,29 @@ void __init hyperv_init(void)
> >> >> >  
> >> >> >  	rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> >> >  	hypercall_msr.enable = 1;
> >> >> > -	hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> >> >> > -	wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> >> > +
> >> >> > +	if (hv_root_partition) {
> >> >> > +		struct page *pg;
> >> >> > +		void *src, *dst;
> >> >> > +
> >> >> > +		/*
> >> >> > +		 * Order is important here. We must enable the hypercall page
> >> >> > +		 * so it is populated with code, then copy the code to an
> >> >> > +		 * executable page.
> >> >> > +		 */
> >> >> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> >> > +
> >> >> > +		pg = vmalloc_to_page(hv_hypercall_pg);
> >> >> > +		dst = kmap(pg);
> >> >> > +		src = memremap(hypercall_msr.guest_physical_address << PAGE_SHIFT, PAGE_SIZE,
> >> >> > +				MEMREMAP_WB);
> >> >> 
> >> >> memremap() can fail...
> >> >
> >> > And we don't care here, if it fails, we would rather it panic or oops.
> >> >
> >> > I was relying on the fact that copying from / to a NULL pointer will
> >> > cause the kernel to crash. But of course it wouldn't hurt to explicitly
> >> > panic here.
> >> >
> >> >> 
> >> >> > +		memcpy(dst, src, PAGE_SIZE);
> >> >> > +		memunmap(src);
> >> >> > +		kunmap(pg);
> >> >> > +	} else {
> >> >> > +		hypercall_msr.guest_physical_address = vmalloc_to_pfn(hv_hypercall_pg);
> >> >> > +		wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >> >> > +	}
> >> >> 
> >> >> Why can't we do wrmsrl() for both cases here?
> >> >> 
> >> >
> >> > Because the hypercall page has already been set up when Linux is the
> >> > root.
> >> 
> >> But you already do wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64)
> >> in 'if (hv_root_partition)' case above, that's why I asked.
> >> 
> >
> > You mean extracting wrmsrl to this point?  The ordering matters. See the
> > comment in the root branch -- we have to enable the page before copying
> > the content.
> >
> > What can be done is:
> >
> >    if (!root) {
> >        /* some stuff */
> >    }
> >
> >    wrmsrl(...)
> >
> >    if (root) {
> >        /* some stuff */
> >    }
> >
> > This is not looking any better than the existing code.
> >
> 
> Oh, I missed the comment indeed. So Hypervisor already picked a page for
> us, however, it didn't enable it and it's not populated?

Seems to be the case.

> How can we be
> sure that we didn't use it for something else already?

It is a page deposited to the root partition and it is not going to be
used elsewhere, nor it is visible from the root. This is my
understanding. I will let Sunil correct me if I'm wrong.

> Maybe we can
> still give a different known-to-be-empty page?
> 

That's the thing I said I didn't bother trying earlier. Something to
check when I have some spare cycles.

Wei.

> -- 
> Vitaly
> 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures
  2020-09-15 11:16   ` Vitaly Kuznetsov
@ 2020-09-15 11:59     ` Wei Liu
  0 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-09-15 11:59 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, K. Y. Srinivasan,
	Haiyang Zhang, Stephen Hemminger, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

On Tue, Sep 15, 2020 at 01:16:59PM +0200, Vitaly Kuznetsov wrote:
[...]
> > +union hv_device_id {
> > +	u64 as_uint64;
> > +
> > +	struct {
> > +		u64 :62;
> > +		u64 device_type:2;
> > +	};
> > +
> > +	// HV_DEVICE_TYPE_LOGICAL
> 
> Nit: please no '//' comments.
> 

Fixed. Thanks.

Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required
  2020-09-15 10:16   ` Vitaly Kuznetsov
@ 2020-09-15 12:43     ` Wei Liu
  2020-09-16 15:42       ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-15 12:43 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Sep 15, 2020 at 12:16:58PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > When Linux runs as the root partition, it will need to make hypercalls
> > which return data from the hypervisor.
> >
> > Allocate pages for storing results when Linux runs as the root
> > partition.
> >
> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > ---
> >  arch/x86/hyperv/hv_init.c       | 45 +++++++++++++++++++++++++++++----
> >  arch/x86/include/asm/mshyperv.h |  1 +
> >  2 files changed, 41 insertions(+), 5 deletions(-)
> >
> > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> > index cac8e4c56261..ebba4be4185d 100644
> > --- a/arch/x86/hyperv/hv_init.c
> > +++ b/arch/x86/hyperv/hv_init.c
> > @@ -45,6 +45,9 @@ EXPORT_SYMBOL_GPL(hv_vp_assist_page);
> >  void  __percpu **hyperv_pcpu_input_arg;
> >  EXPORT_SYMBOL_GPL(hyperv_pcpu_input_arg);
> >  
> > +void  __percpu **hyperv_pcpu_output_arg;
> > +EXPORT_SYMBOL_GPL(hyperv_pcpu_output_arg);
> > +
> >  u32 hv_max_vp_index;
> >  EXPORT_SYMBOL_GPL(hv_max_vp_index);
> >  
> > @@ -75,14 +78,29 @@ static int hv_cpu_init(unsigned int cpu)
> >  	u64 msr_vp_index;
> >  	struct hv_vp_assist_page **hvp = &hv_vp_assist_page[smp_processor_id()];
> >  	void **input_arg;
> > -	struct page *pg;
> > +	struct page *input_pg;
> >  
> >  	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
> >  	/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */
> > -	pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
> > -	if (unlikely(!pg))
> > +	input_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
> > +	if (unlikely(!input_pg))
> >  		return -ENOMEM;
> > -	*input_arg = page_address(pg);
> > +	*input_arg = page_address(input_pg);
> > +
> > +	if (hv_root_partition) {
> > +		struct page *output_pg;
> > +		void **output_arg;
> > +
> > +		output_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC :
> >  	GFP_KERNEL);
> 
> To simplify the code, can we just rename 'input_arg' to 'hypercall_args'
> and do alloc_pages(rqs_disabled() ? GFP_ATOMIC : GFP_KERNEL, 1) to
> allocate two pages above?

It should be doable, but I need to code it up and test it to be 100%
sure.

> 
> >  
[...]
> > +	/* Allocate the per-CPU state for output arg for root */
> > +	if (hv_root_partition) {
> > +		hyperv_pcpu_output_arg = alloc_percpu(void  *);
> 					redundant space ^^^^^

Fixed. Thanks. This is in fact copied from the input_arg code, so there
is a similar issue there to be fixed. But that's going to wait till
another day.

Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required
  2020-09-15 12:43     ` Wei Liu
@ 2020-09-16 15:42       ` Wei Liu
  2020-09-16 16:10         ` Vitaly Kuznetsov
  0 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-16 15:42 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Sep 15, 2020 at 12:43:18PM +0000, Wei Liu wrote:
> On Tue, Sep 15, 2020 at 12:16:58PM +0200, Vitaly Kuznetsov wrote:
> > Wei Liu <wei.liu@kernel.org> writes:
> > 
> > > When Linux runs as the root partition, it will need to make hypercalls
> > > which return data from the hypervisor.
> > >
> > > Allocate pages for storing results when Linux runs as the root
> > > partition.
> > >
> > > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > > ---
> > >  arch/x86/hyperv/hv_init.c       | 45 +++++++++++++++++++++++++++++----
> > >  arch/x86/include/asm/mshyperv.h |  1 +
> > >  2 files changed, 41 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> > > index cac8e4c56261..ebba4be4185d 100644
> > > --- a/arch/x86/hyperv/hv_init.c
> > > +++ b/arch/x86/hyperv/hv_init.c
> > > @@ -45,6 +45,9 @@ EXPORT_SYMBOL_GPL(hv_vp_assist_page);
> > >  void  __percpu **hyperv_pcpu_input_arg;
> > >  EXPORT_SYMBOL_GPL(hyperv_pcpu_input_arg);
> > >  
> > > +void  __percpu **hyperv_pcpu_output_arg;
> > > +EXPORT_SYMBOL_GPL(hyperv_pcpu_output_arg);
> > > +
> > >  u32 hv_max_vp_index;
> > >  EXPORT_SYMBOL_GPL(hv_max_vp_index);
> > >  
> > > @@ -75,14 +78,29 @@ static int hv_cpu_init(unsigned int cpu)
> > >  	u64 msr_vp_index;
> > >  	struct hv_vp_assist_page **hvp = &hv_vp_assist_page[smp_processor_id()];
> > >  	void **input_arg;
> > > -	struct page *pg;
> > > +	struct page *input_pg;
> > >  
> > >  	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
> > >  	/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */
> > > -	pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
> > > -	if (unlikely(!pg))
> > > +	input_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
> > > +	if (unlikely(!input_pg))
> > >  		return -ENOMEM;
> > > -	*input_arg = page_address(pg);
> > > +	*input_arg = page_address(input_pg);
> > > +
> > > +	if (hv_root_partition) {
> > > +		struct page *output_pg;
> > > +		void **output_arg;
> > > +
> > > +		output_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC :
> > >  	GFP_KERNEL);
> > 
> > To simplify the code, can we just rename 'input_arg' to 'hypercall_args'
> > and do alloc_pages(rqs_disabled() ? GFP_ATOMIC : GFP_KERNEL, 1) to
> > allocate two pages above?
> 
> It should be doable, but I need to code it up and test it to be 100%
> sure.
> 

I switch to alloc_pages to allocate an order of 2 page if necessary, but
I keep input_arg and output_arg separate because I want to avoid code
churn in other places.

Wei.

> > 
> > >  
> [...]
> > > +	/* Allocate the per-CPU state for output arg for root */
> > > +	if (hv_root_partition) {
> > > +		hyperv_pcpu_output_arg = alloc_percpu(void  *);
> > 					redundant space ^^^^^
> 
> Fixed. Thanks. This is in fact copied from the input_arg code, so there
> is a similar issue there to be fixed. But that's going to wait till
> another day.
> 
> Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required
  2020-09-16 15:42       ` Wei Liu
@ 2020-09-16 16:10         ` Vitaly Kuznetsov
  0 siblings, 0 replies; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-16 16:10 UTC (permalink / raw)
  To: Wei Liu
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Wei Liu <wei.liu@kernel.org> writes:

> On Tue, Sep 15, 2020 at 12:43:18PM +0000, Wei Liu wrote:
>> On Tue, Sep 15, 2020 at 12:16:58PM +0200, Vitaly Kuznetsov wrote:
>> > Wei Liu <wei.liu@kernel.org> writes:
>> > 
>> > > When Linux runs as the root partition, it will need to make hypercalls
>> > > which return data from the hypervisor.
>> > >
>> > > Allocate pages for storing results when Linux runs as the root
>> > > partition.
>> > >
>> > > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> > > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> > > Signed-off-by: Wei Liu <wei.liu@kernel.org>
>> > > ---
>> > >  arch/x86/hyperv/hv_init.c       | 45 +++++++++++++++++++++++++++++----
>> > >  arch/x86/include/asm/mshyperv.h |  1 +
>> > >  2 files changed, 41 insertions(+), 5 deletions(-)
>> > >
>> > > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
>> > > index cac8e4c56261..ebba4be4185d 100644
>> > > --- a/arch/x86/hyperv/hv_init.c
>> > > +++ b/arch/x86/hyperv/hv_init.c
>> > > @@ -45,6 +45,9 @@ EXPORT_SYMBOL_GPL(hv_vp_assist_page);
>> > >  void  __percpu **hyperv_pcpu_input_arg;
>> > >  EXPORT_SYMBOL_GPL(hyperv_pcpu_input_arg);
>> > >  
>> > > +void  __percpu **hyperv_pcpu_output_arg;
>> > > +EXPORT_SYMBOL_GPL(hyperv_pcpu_output_arg);
>> > > +
>> > >  u32 hv_max_vp_index;
>> > >  EXPORT_SYMBOL_GPL(hv_max_vp_index);
>> > >  
>> > > @@ -75,14 +78,29 @@ static int hv_cpu_init(unsigned int cpu)
>> > >  	u64 msr_vp_index;
>> > >  	struct hv_vp_assist_page **hvp = &hv_vp_assist_page[smp_processor_id()];
>> > >  	void **input_arg;
>> > > -	struct page *pg;
>> > > +	struct page *input_pg;
>> > >  
>> > >  	input_arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg);
>> > >  	/* hv_cpu_init() can be called with IRQs disabled from hv_resume() */
>> > > -	pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
>> > > -	if (unlikely(!pg))
>> > > +	input_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL);
>> > > +	if (unlikely(!input_pg))
>> > >  		return -ENOMEM;
>> > > -	*input_arg = page_address(pg);
>> > > +	*input_arg = page_address(input_pg);
>> > > +
>> > > +	if (hv_root_partition) {
>> > > +		struct page *output_pg;
>> > > +		void **output_arg;
>> > > +
>> > > +		output_pg = alloc_page(irqs_disabled() ? GFP_ATOMIC :
>> > >  	GFP_KERNEL);
>> > 
>> > To simplify the code, can we just rename 'input_arg' to 'hypercall_args'
>> > and do alloc_pages(rqs_disabled() ? GFP_ATOMIC : GFP_KERNEL, 1) to
>> > allocate two pages above?
>> 
>> It should be doable, but I need to code it up and test it to be 100%
>> sure.
>> 
>
> I switch to alloc_pages to allocate an order of 2 page if necessary, but
> I keep input_arg and output_arg separate because I want to avoid code
> churn in other places.
>

My idea was that we're free to choose how to use these pages, e.g. with
two pages allocated we can now do a hypercall which takes two pages of
input and produces no output other than the return value. This doesn't
contradict your suggestion to keep input_arg/output_arg as these are
just pointers to the continuous space, we can still do the trick.

I don't feel strong about this and you probably have more patches in
your stash, no need for massive re-work I believe. 

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary
  2020-09-15 10:27   ` Vitaly Kuznetsov
@ 2020-09-16 16:32     ` Wei Liu
  2020-10-27 12:19       ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-09-16 16:32 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

On Tue, Sep 15, 2020 at 12:27:16PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
[...]
> >  
> > +void __init hv_get_partition_id(void)
> > +{
> > +	struct hv_get_partition_id *output_page;
> > +	int status;
> > +	unsigned long flags;
> > +
> > +	local_irq_save(flags);
> > +	output_page = *this_cpu_ptr(hyperv_pcpu_output_arg);
> > +	status = hv_do_hypercall(HVCALL_GET_PARTITION_ID, NULL, output_page) &
> > +		HV_HYPERCALL_RESULT_MASK;
> 
> Nit: in this case status is 'u16', we can define it as such (instead of
> signed int).

Fixed.

> 
> > +	if (status != HV_STATUS_SUCCESS)
> > +		pr_err("Failed to get partition ID: %d\n", status);
> > +	else
> > +		hv_current_partition_id = output_page->partition_id;
> > +	local_irq_restore(flags);
> > +
> > +	/* No point in proceeding if this failed */
> > +	BUG_ON(status != HV_STATUS_SUCCESS);
> > +}
> > +
> >  /*
> >   * This function is to be invoked early in the boot sequence after the
> >   * hypervisor has been detected.
> > @@ -440,6 +463,9 @@ void __init hyperv_init(void)
> >  
> >  	register_syscore_ops(&hv_syscore_ops);
> >  
> > +	if (hv_root_partition)
> > +		hv_get_partition_id();
> 
> According to TLFS, partition ID is available when AccessPartitionId
> privilege is granted. I'd suggest we check that instead of
> hv_root_partition (and we can set hv_current_partition_id to something
> like U64_MAX so we know it wasn't acuired). So the BUG_ON condition will
> move here:
> 
>         hv_get_partition_id();
>         BUG_ON(hv_root_partition && hv_current_partition_id == U64_MAX);
> 

Good point. I will reorganize this a bit.

Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

* RE: [EXTERNAL] Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-15 10:37     ` Wei Liu
  2020-09-15 11:02       ` Vitaly Kuznetsov
@ 2020-09-16 21:34       ` Sunil Muthuswamy
  2020-09-17 11:06         ` Vitaly Kuznetsov
  1 sibling, 1 reply; 45+ messages in thread
From: Sunil Muthuswamy @ 2020-09-16 21:34 UTC (permalink / raw)
  To: Wei Liu, vkuznets
  Cc: Linux on Hyper-V List, virtualization, Linux Kernel List,
	Michael Kelley, Vineeth Pillai, Nuno Das Neves,
	Lillian Grassin-Drake, KY Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

> 
> On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
> > Wei Liu <wei.liu@kernel.org> writes:
> >
> > > When Linux is running as the root partition, the hypercall page will
> > > have already been setup by Hyper-V. Copy the content over to the
> > > allocated page.
> >
> > And we can't setup a new hypercall page by writing something different
> > to HV_X64_MSR_HYPERCALL, right?
> >
> 
> My understanding is that we can't, but Sunil can maybe correct me.

That is correct. For root partition, the hypervisor has already allocated the
hypercall page. The root is required to query the page, map it in its address
space and wrmsr to enable it. It cannot change the location of the page. For
guest, it can allocate and assign the hypercall page. This is covered a bit in the
hypervisor TLFS (section 3.13 in TLFS v6), for the guest side. The root side is 
not covered there, yet.
 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* RE: [EXTERNAL] Re: [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root
  2020-09-16 21:34       ` [EXTERNAL] " Sunil Muthuswamy
@ 2020-09-17 11:06         ` Vitaly Kuznetsov
  0 siblings, 0 replies; 45+ messages in thread
From: Vitaly Kuznetsov @ 2020-09-17 11:06 UTC (permalink / raw)
  To: Sunil Muthuswamy, Wei Liu
  Cc: Linux on Hyper-V List, virtualization, Linux Kernel List,
	Michael Kelley, Vineeth Pillai, Nuno Das Neves,
	Lillian Grassin-Drake, KY Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

Sunil Muthuswamy <sunilmut@microsoft.com> writes:

>> 
>> On Tue, Sep 15, 2020 at 12:32:29PM +0200, Vitaly Kuznetsov wrote:
>> > Wei Liu <wei.liu@kernel.org> writes:
>> >
>> > > When Linux is running as the root partition, the hypercall page will
>> > > have already been setup by Hyper-V. Copy the content over to the
>> > > allocated page.
>> >
>> > And we can't setup a new hypercall page by writing something different
>> > to HV_X64_MSR_HYPERCALL, right?
>> >
>> 
>> My understanding is that we can't, but Sunil can maybe correct me.
>
> That is correct. For root partition, the hypervisor has already allocated the
> hypercall page. The root is required to query the page, map it in its address
> space and wrmsr to enable it. It cannot change the location of the page. For
> guest, it can allocate and assign the hypercall page. This is covered a bit in the
> hypervisor TLFS (section 3.13 in TLFS v6), for the guest side. The root side is 
> not covered there, yet.

Ok, so it is guaranteed that root partition doesn't have this page in
its address space yet, otherwise it could've been used for something
else (in case it's just normal memory from its PoV).

Please add a comment about this as it is not really obvious.

Thanks,

-- 
Vitaly


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 04/18] iommu/hyperv: don't setup IRQ remapping when running as root
  2020-09-14 11:27 ` [PATCH RFC v1 04/18] iommu/hyperv: don't setup IRQ remapping when running as root Wei Liu
@ 2020-09-18  9:12   ` Joerg Roedel
  0 siblings, 0 replies; 45+ messages in thread
From: Joerg Roedel @ 2020-09-18  9:12 UTC (permalink / raw)
  To: Wei Liu
  Cc: Linux on Hyper-V List, virtualization, Linux Kernel List,
	Michael Kelley, Vineeth Pillai, Sunil Muthuswamy, Nuno Das Neves,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	open list:IOMMU DRIVERS

On Mon, Sep 14, 2020 at 11:27:48AM +0000, Wei Liu wrote:
> The IOMMU code needs more work. We're sure for now the IRQ remapping
> hooks are not applicable when Linux is the root.
> 
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  drivers/iommu/hyperv-iommu.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Acked-by: Joerg Roedel <jroedel@suse.de>


^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 12/18] asm-generic/hyperv: update hv_interrupt_entry
  2020-09-14 11:59 ` [PATCH RFC v1 12/18] asm-generic/hyperv: update hv_interrupt_entry Wei Liu
@ 2020-10-01 14:33   ` Rob Herring
  0 siblings, 0 replies; 45+ messages in thread
From: Rob Herring @ 2020-10-01 14:33 UTC (permalink / raw)
  To: Wei Liu
  Cc: Stephen Hemminger, Sunil Muthuswamy, Lorenzo Pieralisi,
	Arnd Bergmann, virtualization, Bjorn Helgaas, Haiyang Zhang,
	linux-arch, linux-pci, Michael Kelley, Linux on Hyper-V List,
	K. Y. Srinivasan, Nuno Das Neves, Linux Kernel List,
	Vineeth Pillai

On Mon, 14 Sep 2020 11:59:21 +0000, Wei Liu wrote:
> We will soon use the same structure to handle IO-APIC interrupts as
> well. Introduce an enum to identify the source and a data structure for
> IO-APIC RTE.
> 
> While at it, update pci-hyperv.c to use the enum.
> 
> No functional change.
> 
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> ---
>  drivers/pci/controller/pci-hyperv.c |  2 +-
>  include/asm-generic/hyperv-tlfs.h   | 36 +++++++++++++++++++++++++++--
>  2 files changed, 35 insertions(+), 3 deletions(-)
> 

Acked-by: Rob Herring <robh@kernel.org>

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary
  2020-09-16 16:32     ` Wei Liu
@ 2020-10-27 12:19       ` Wei Liu
  0 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-10-27 12:19 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

On Wed, Sep 16, 2020 at 04:32:43PM +0000, Wei Liu wrote:
> On Tue, Sep 15, 2020 at 12:27:16PM +0200, Vitaly Kuznetsov wrote:
> > Wei Liu <wei.liu@kernel.org> writes:
> [...]
> > >  
> > > +void __init hv_get_partition_id(void)
> > > +{
> > > +	struct hv_get_partition_id *output_page;
> > > +	int status;
> > > +	unsigned long flags;
> > > +
> > > +	local_irq_save(flags);
> > > +	output_page = *this_cpu_ptr(hyperv_pcpu_output_arg);
> > > +	status = hv_do_hypercall(HVCALL_GET_PARTITION_ID, NULL, output_page) &
> > > +		HV_HYPERCALL_RESULT_MASK;
> > 
> > Nit: in this case status is 'u16', we can define it as such (instead of
> > signed int).
> 
> Fixed.
> 
> > 
> > > +	if (status != HV_STATUS_SUCCESS)
> > > +		pr_err("Failed to get partition ID: %d\n", status);
> > > +	else
> > > +		hv_current_partition_id = output_page->partition_id;
> > > +	local_irq_restore(flags);
> > > +
> > > +	/* No point in proceeding if this failed */
> > > +	BUG_ON(status != HV_STATUS_SUCCESS);
> > > +}
> > > +
> > >  /*
> > >   * This function is to be invoked early in the boot sequence after the
> > >   * hypervisor has been detected.
> > > @@ -440,6 +463,9 @@ void __init hyperv_init(void)
> > >  
> > >  	register_syscore_ops(&hv_syscore_ops);
> > >  
> > > +	if (hv_root_partition)
> > > +		hv_get_partition_id();
> > 
> > According to TLFS, partition ID is available when AccessPartitionId
> > privilege is granted. I'd suggest we check that instead of
> > hv_root_partition (and we can set hv_current_partition_id to something
> > like U64_MAX so we know it wasn't acuired). So the BUG_ON condition will
> > move here:
> > 
> >         hv_get_partition_id();
> >         BUG_ON(hv_root_partition && hv_current_partition_id == U64_MAX);
> > 
> 
> Good point. I will reorganize this a bit.

Actually, our current code never stashed the feature mask that contains
that privilege anywhere.  Getting access to it will require a few more
extra patches -- I would really like to rename those fields (features,
misc_features) inside ms_hyperv to something more appropriate.

We will gate it wit hv_root_partition anyway, since we a child VM may
not have the privilege, making an unconditional BUG_ON fatal.

All in all, the current code is not too bad. I intend to keep the
current structure for my RFC v2. I will see if I can find some time to
rework the feature mask extraction code and get that upstreamed first.

Wei.

> 
> Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions
  2020-09-15 11:00   ` Vitaly Kuznetsov
@ 2020-10-27 13:10     ` Wei Liu
  0 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-10-27 13:10 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin, Arnd Bergmann,
	open list:GENERIC INCLUDE/ASM HEADER FILES

On Tue, Sep 15, 2020 at 01:00:55PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > They are used to deposit pages into Microsoft Hypervisor and bring up
> > logical and virtual processors.
> >
> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Signed-off-by: Nuno Das Neves <nudasnev@microsoft.com>
> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Co-Developed-by: Nuno Das Neves <nudasnev@microsoft.com>
> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > ---
> >  arch/x86/hyperv/Makefile          |   2 +-
> >  arch/x86/hyperv/hv_proc.c         | 209 ++++++++++++++++++++++++++++++
> >  arch/x86/include/asm/mshyperv.h   |   4 +
> >  include/asm-generic/hyperv-tlfs.h |  56 ++++++++
> >  4 files changed, 270 insertions(+), 1 deletion(-)
> >  create mode 100644 arch/x86/hyperv/hv_proc.c
> >
> > diff --git a/arch/x86/hyperv/Makefile b/arch/x86/hyperv/Makefile
> > index 89b1f74d3225..565358020921 100644
> > --- a/arch/x86/hyperv/Makefile
> > +++ b/arch/x86/hyperv/Makefile
> > @@ -1,6 +1,6 @@
> >  # SPDX-License-Identifier: GPL-2.0-only
> >  obj-y			:= hv_init.o mmu.o nested.o
> > -obj-$(CONFIG_X86_64)	+= hv_apic.o
> > +obj-$(CONFIG_X86_64)	+= hv_apic.o hv_proc.o
> >  
> >  ifdef CONFIG_X86_64
> >  obj-$(CONFIG_PARAVIRT_SPINLOCKS)	+= hv_spinlock.o
> > diff --git a/arch/x86/hyperv/hv_proc.c b/arch/x86/hyperv/hv_proc.c
> > new file mode 100644
> > index 000000000000..847c72465d0e
> > --- /dev/null
> > +++ b/arch/x86/hyperv/hv_proc.c
> > @@ -0,0 +1,209 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +#include <linux/types.h>
> > +#include <linux/version.h>
> > +#include <linux/vmalloc.h>
> > +#include <linux/mm.h>
> > +#include <linux/clockchips.h>
> > +#include <linux/acpi.h>
> > +#include <linux/hyperv.h>
> > +#include <linux/slab.h>
> > +#include <linux/cpuhotplug.h>
> > +#include <asm/hypervisor.h>
> > +#include <asm/mshyperv.h>
> > +#include <asm/apic.h>
> > +
> > +#include <asm/trace/hyperv.h>
> > +
> > +#define HV_DEPOSIT_MAX_ORDER (8)
> > +#define HV_DEPOSIT_MAX (1 << HV_DEPOSIT_MAX_ORDER)
> > +
> > +#define MAX(a, b) ((a) > (b) ? (a) : (b))
> > +#define MIN(a, b) ((a) < (b) ? (a) : (b))
> 
> Nit: include/linux/kernel.h defines min() and max() macros with type
> checking.

Fixed.

> 
> > +
> > +/*
> > + * Deposits exact number of pages
> > + * Must be called with interrupts enabled
> > + * Max 256 pages
> > + */
> > +int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages)
> > +{
> > +	struct page **pages;
> > +	int *counts;
> > +	int num_allocations;
> > +	int i, j, page_count;
> > +	int order;
> > +	int desired_order;
> > +	int status;
> > +	int ret;
> > +	u64 base_pfn;
> > +	struct hv_deposit_memory *input_page;
> > +	unsigned long flags;
> > +
> > +	if (num_pages > HV_DEPOSIT_MAX)
> > +		return -EINVAL;
> > +	if (!num_pages)
> > +		return 0;
> > +
> > +	ret = -ENOMEM;
> > +
> > +	/* One buffer for page pointers and counts */
> > +	pages = page_address(alloc_page(GFP_KERNEL));
> > +	if (!pages)
> > +		goto free_buf;
> 
> There is nothing to free, just do 'return -ENOMEM' here;
> 
> > +	counts = (int *)&pages[256];
> > +
> 
> Oh this is weird. So 'pages' is an array of 512 'struct page *' items
> and we use its second half (pages[256]) for an array of signed(!)
> integers(!). Can we use a locally defined struct or something better for
> that?
> 

This can be changed. I will make the counts array have its own buffer.

> > +	/* Allocate all the pages before disabling interrupts */
> > +	num_allocations = 0;
> > +	i = 0;
> > +	order = HV_DEPOSIT_MAX_ORDER;
> > +
> > +	while (num_pages) {
> > +		/* Find highest order we can actually allocate */
> > +		desired_order = 31 - __builtin_clz(num_pages);
> > +		order = MIN(desired_order, order);
> > +		do {
> > +			pages[i] = alloc_pages_node(node, GFP_KERNEL, order);
> > +			if (!pages[i]) {
> > +				if (!order) {
> > +					goto err_free_allocations;
> > +				}
> > +				--order;
> > +			}
> > +		} while (!pages[i]);
> > +
> > +		split_page(pages[i], order);
> > +		counts[i] = 1 << order;
> > +		num_pages -= counts[i];
> > +		i++;
> 
> So here we believe we will never overrun the 2048 bytes we 'allocated'
> for 'counts' above. While 'if (num_pages > HV_DEPOSIT_MAX)' presumably
> guarantees that, this is not really obvious.
> 

This is moot since counts is going to have its own buffer allocated with
kcalloc(HV_DEPOSIT_MAX, sizeof(int), ...).

> > +		num_allocations++;
> > +	}
> > +
> > +	local_irq_save(flags);
> > +
> > +	input_page = *this_cpu_ptr(hyperv_pcpu_input_arg);
> > +
> > +	input_page->partition_id = partition_id;
> > +
> > +	/* Populate gpa_page_list - these will fit on the input page */
> > +	for (i = 0, page_count = 0; i < num_allocations; ++i) {
> > +		base_pfn = page_to_pfn(pages[i]);
> > +		for (j = 0; j < counts[i]; ++j, ++page_count)
> > +			input_page->gpa_page_list[page_count] = base_pfn + j;
> > +	}
> > +	status = hv_do_rep_hypercall(HVCALL_DEPOSIT_MEMORY,
> > +				     page_count, 0, input_page,
> > +				     NULL) & HV_HYPERCALL_RESULT_MASK;
> > +	local_irq_restore(flags);
> > +
> > +	if (status != HV_STATUS_SUCCESS) {
> 
> Nit: same like in one ov the previous patches, status can be 'u16'.
> 

Fixed.

> > +		pr_err("Failed to deposit pages: %d\n", status);
> > +		ret = status;
> > +		goto err_free_allocations;
> > +	}
> > +
> > +	ret = 0;
> > +	goto free_buf;
> > +
> > +err_free_allocations:
> > +	for (i = 0; i < num_allocations; ++i) {
> > +		base_pfn = page_to_pfn(pages[i]);
> > +		for (j = 0; j < counts[i]; ++j)
> > +			__free_page(pfn_to_page(base_pfn + j));
> > +	}
> > +
> > +free_buf:
> > +	free_page((unsigned long)pages);
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(hv_call_deposit_pages);
> > +
> > +int hv_call_add_logical_proc(int node, u32 lp_index, u32 apic_id)
> > +{
> > +	struct hv_add_logical_processor_in *input;
> > +	struct hv_add_logical_processor_out *output;
> > +	int status;
> > +	unsigned long flags;
> > +	int ret = 0;
> > +
> > +	do {
> > +		local_irq_save(flags);
> > +
> > +		input = *this_cpu_ptr(hyperv_pcpu_input_arg);
> > +		/* We don't do anything with the output right now */
> > +		output = *this_cpu_ptr(hyperv_pcpu_output_arg);
> > +
> > +		input->lp_index = lp_index;
> > +		input->apic_id = apic_id;
> > +		input->flags = 0;
> > +		input->proximity_domain_info.domain_id = node_to_pxm(node);
> > +		input->proximity_domain_info.flags.reserved = 0;
> > +		input->proximity_domain_info.flags.proximity_info_valid = 1;
> > +		input->proximity_domain_info.flags.proximity_preferred = 1;
> > +		status = hv_do_hypercall(HVCALL_ADD_LOGICAL_PROCESSOR,
> > +					 input, output);
> > +		local_irq_restore(flags);
> > +
> > +		if (status != HV_STATUS_INSUFFICIENT_MEMORY) {
> > +			if (status != HV_STATUS_SUCCESS) {
> > +				pr_err("%s: cpu %u apic ID %u, %d\n", __func__,
> > +				       lp_index, apic_id, status);
> > +				ret = status;
> > +			}
> > +			break;
> 
> So if status == HV_STATUS_SUCCESS we break and avoid
> hv_call_deposit_pages() below?
> 

Yes, that means adding the logical processor has succeeded. There is
nothing more to do.

> > +		}
> > +		ret = hv_call_deposit_pages(node, hv_current_partition_id, 1);
> > +
> > +	} while (!ret);
> 
> And if hv_call_deposit_pages() returns '0' we keep doing something? Sorry
> but I'm probably missing something important in the 'depositing'
> process, could you please add a comment explaining what's going on here?
> 

We only get here because 1) there isn't sufficient memory in the last
iteration, 2) we've succeeded in adding a bit more memory. In this case
we will want to retry adding the logical processor.

I will add a comment before the loop.

> > +
> > +	return ret;
> > +}
> > +
[...]
> > diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
> > index 87b1a79b19eb..2b05bed712c0 100644
> > --- a/include/asm-generic/hyperv-tlfs.h
> > +++ b/include/asm-generic/hyperv-tlfs.h
> > @@ -142,6 +142,8 @@ struct ms_hyperv_tsc_page {
> >  #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX	0x0014
> >  #define HVCALL_SEND_IPI_EX			0x0015
> >  #define HVCALL_GET_PARTITION_ID			0x0046
> > +#define HVCALL_DEPOSIT_MEMORY			0x0048
> > +#define HVCALL_CREATE_VP			0x004e
> >  #define HVCALL_GET_VP_REGISTERS			0x0050
> >  #define HVCALL_SET_VP_REGISTERS			0x0051
> >  #define HVCALL_POST_MESSAGE			0x005c
> > @@ -149,6 +151,7 @@ struct ms_hyperv_tsc_page {
> >  #define HVCALL_POST_DEBUG_DATA			0x0069
> >  #define HVCALL_RETRIEVE_DEBUG_DATA		0x006a
> >  #define HVCALL_RESET_DEBUG_SESSION		0x006b
> > +#define HVCALL_ADD_LOGICAL_PROCESSOR		0x0076
> >  #define HVCALL_RETARGET_INTERRUPT		0x007e
> >  #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
> >  #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0
> > @@ -413,6 +416,59 @@ struct hv_get_partition_id {
> >  	u64 partition_id;
> >  } __packed;
> >  
> > +/* HvDepositMemory hypercall */
> > +struct hv_deposit_memory {
> > +	u64 partition_id;
> > +	u64 gpa_page_list[];
> > +};
> 
> Other structures above have '__packed' and I remember there were
> different opinions if it is needed or not (for properly padded
> structures). I'd suggest we stay consitent and keep adding it unless we
> decide to get rid of them (but you've added it to the newly introduced
> hv_get_partition_id above).

Fixed.

> > +
> > +
> > +struct hv_proximity_domain_flags {
> > +	u32 proximity_preferred : 1;
> > +	u32 reserved : 30;
> > +	u32 proximity_info_valid : 1;
> > +};
> > +
> > +/* Not a union in windows but useful for zeroing */
> > +union hv_proximity_domain_info {
> > +	struct {
> > +		u32 domain_id;
> > +		struct hv_proximity_domain_flags flags;
> > +	};
> > +	u64 as_uint64;
> > +};
> > +
> > +struct hv_lp_startup_status {
> > +	u64 hv_status;
> > +	u64 substatus1;
> > +	u64 substatus2;
> > +	u64 substatus3;
> > +	u64 substatus4;
> > +	u64 substatus5;
> > +	u64 substatus6;
> > +};
> > +
> > +/* HvAddLogicalProcessor hypercalls */
> 
> s/hypercalls/hypercall/

Fixed.

Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus
  2020-09-15 11:14   ` Vitaly Kuznetsov
@ 2020-10-27 13:47     ` Wei Liu
  2020-10-27 13:56       ` Wei Liu
  0 siblings, 1 reply; 45+ messages in thread
From: Wei Liu @ 2020-10-27 13:47 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Sep 15, 2020 at 01:14:16PM +0200, Vitaly Kuznetsov wrote:
> Wei Liu <wei.liu@kernel.org> writes:
> 
> > Microsoft Hypervisor requires the root partition to make a few
> > hypercalls to setup application processors before they can be used.
> >
> > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > ---
> > CPU hotplug and unplug is not yet supported in this setup, so those
> > paths remain untouched.
> > ---
> >  arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
> >  1 file changed, 27 insertions(+)
> >
> > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > index 1bf57d310f78..7522cae02759 100644
> > --- a/arch/x86/kernel/cpu/mshyperv.c
> > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > @@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
> >  	hv_init_spinlocks();
> >  #endif
> >  }
> > +
> > +static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
> > +{
> > +#if defined(CONFIG_X86_64)
> 
> I think it makes little sense to try to make Linux work as Hyper-V root
> partition when !CONFIG_X86_64. If we still care about Hyper-V enablement
> for !CONFIG_X86_64 we can probably introduce something like
> CONFIG_HYPERV_ROOT and enable it automatically, e.g.
> 
> config HYPERV_ROOT
>         def_bool HYPERV && X86_64
> 
> and use it instead.
> 

We have a patch for such a config option in the /dev/mshv patch set. But
that's not yet included here so I will keep this as-is.

> > +	int i;
> > +	int vp_index = 1;
> > +	int ret;
> > +
> > +	native_smp_prepare_cpus(max_cpus);
> > +
> > +	for_each_present_cpu(i) {
> > +		if (i == 0)
> > +			continue;
> > +		ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
> > +		BUG_ON(ret);
> > +	}
> > +
> > +	for_each_present_cpu(i) {
> > +		if (i == 0)
> > +			continue;
> > +		ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);
> 
> So vp_index variable is needed here to make sure there are no gaps? (or
> we could've just used 'i')?

Not sure. I didn't write the original code in this function. The last
argument (i) is the logical processor index.

I don't see a reason why vp_index and lp_index can't be the same. I will
try dropping vp_index. If that works then great; if not, I will keep the
code as-is.

Sunil, if you have more insight, please chime in.

Wei.

> 
> > +		BUG_ON(ret);
> > +	}
> > +#endif
> > +}
> >  #endif
> >  
> >  static void __init ms_hyperv_init_platform(void)
> > @@ -359,6 +384,8 @@ static void __init ms_hyperv_init_platform(void)
> >  
> >  # ifdef CONFIG_SMP
> >  	smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu;
> > +	if (hv_root_partition)
> > +		smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus;
> >  # endif
> >  
> >  	/*
> 
> -- 
> Vitaly
> 

^ permalink raw reply	[flat|nested] 45+ messages in thread

* Re: [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus
  2020-10-27 13:47     ` Wei Liu
@ 2020-10-27 13:56       ` Wei Liu
  0 siblings, 0 replies; 45+ messages in thread
From: Wei Liu @ 2020-10-27 13:56 UTC (permalink / raw)
  To: Vitaly Kuznetsov
  Cc: Wei Liu, Linux on Hyper-V List, virtualization,
	Linux Kernel List, Michael Kelley, Vineeth Pillai,
	Sunil Muthuswamy, Nuno Das Neves, Lillian Grassin-Drake,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Thomas Gleixner, Ingo Molnar, Borislav Petkov,
	maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT),
	H. Peter Anvin

On Tue, Oct 27, 2020 at 01:47:57PM +0000, Wei Liu wrote:
> On Tue, Sep 15, 2020 at 01:14:16PM +0200, Vitaly Kuznetsov wrote:
> > Wei Liu <wei.liu@kernel.org> writes:
> > 
> > > Microsoft Hypervisor requires the root partition to make a few
> > > hypercalls to setup application processors before they can be used.
> > >
> > > Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > > Signed-off-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > > Co-Developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> > > Co-Developed-by: Sunil Muthuswamy <sunilmut@microsoft.com>
> > > Signed-off-by: Wei Liu <wei.liu@kernel.org>
> > > ---
> > > CPU hotplug and unplug is not yet supported in this setup, so those
> > > paths remain untouched.
> > > ---
> > >  arch/x86/kernel/cpu/mshyperv.c | 27 +++++++++++++++++++++++++++
> > >  1 file changed, 27 insertions(+)
> > >
> > > diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c
> > > index 1bf57d310f78..7522cae02759 100644
> > > --- a/arch/x86/kernel/cpu/mshyperv.c
> > > +++ b/arch/x86/kernel/cpu/mshyperv.c
> > > @@ -203,6 +203,31 @@ static void __init hv_smp_prepare_boot_cpu(void)
> > >  	hv_init_spinlocks();
> > >  #endif
> > >  }
> > > +
> > > +static void __init hv_smp_prepare_cpus(unsigned int max_cpus)
> > > +{
> > > +#if defined(CONFIG_X86_64)
> > 
> > I think it makes little sense to try to make Linux work as Hyper-V root
> > partition when !CONFIG_X86_64. If we still care about Hyper-V enablement
> > for !CONFIG_X86_64 we can probably introduce something like
> > CONFIG_HYPERV_ROOT and enable it automatically, e.g.
> > 
> > config HYPERV_ROOT
> >         def_bool HYPERV && X86_64
> > 
> > and use it instead.
> > 
> 
> We have a patch for such a config option in the /dev/mshv patch set. But
> that's not yet included here so I will keep this as-is.
> 
> > > +	int i;
> > > +	int vp_index = 1;
> > > +	int ret;
> > > +
> > > +	native_smp_prepare_cpus(max_cpus);
> > > +
> > > +	for_each_present_cpu(i) {
> > > +		if (i == 0)
> > > +			continue;
> > > +		ret = hv_call_add_logical_proc(numa_cpu_node(i), i, cpu_physical_id(i));
> > > +		BUG_ON(ret);
> > > +	}
> > > +
> > > +	for_each_present_cpu(i) {
> > > +		if (i == 0)
> > > +			continue;
> > > +		ret = hv_call_create_vp(numa_cpu_node(i), hv_current_partition_id, vp_index++, i);
> > 
> > So vp_index variable is needed here to make sure there are no gaps? (or
> > we could've just used 'i')?
> 
> Not sure. I didn't write the original code in this function. The last
> argument (i) is the logical processor index.
> 
> I don't see a reason why vp_index and lp_index can't be the same. I will
> try dropping vp_index. If that works then great; if not, I will keep the
> code as-is.
> 
> Sunil, if you have more insight, please chime in.
> 

A quick test shows that replacing vp_index with i works just fine.

Wei.

^ permalink raw reply	[flat|nested] 45+ messages in thread

end of thread, back to index

Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-14 11:27 [PATCH RFC v1 00/18] Introducing Linux root partition support for Microsoft Hypervisor Wei Liu
2020-09-14 11:27 ` [PATCH RFC v1 01/18] asm-generic/hyperv: change HV_CPU_POWER_MANAGEMENT to HV_CPU_MANAGEMENT Wei Liu
2020-09-14 11:27 ` [PATCH RFC v1 02/18] x86/hyperv: detect if Linux is the root partition Wei Liu
2020-09-14 11:27 ` [PATCH RFC v1 03/18] Drivers: hv: vmbus: skip VMBus initialization if Linux is root Wei Liu
2020-09-14 11:27 ` [PATCH RFC v1 04/18] iommu/hyperv: don't setup IRQ remapping when running as root Wei Liu
2020-09-18  9:12   ` Joerg Roedel
2020-09-14 11:27 ` [PATCH RFC v1 05/18] clocksource/hyperv: use MSR-based access if " Wei Liu
2020-09-15 10:10   ` Vitaly Kuznetsov
2020-09-15 10:32     ` Wei Liu
2020-09-14 11:27 ` [PATCH RFC v1 06/18] x86/hyperv: allocate output arg pages if required Wei Liu
2020-09-15 10:16   ` Vitaly Kuznetsov
2020-09-15 12:43     ` Wei Liu
2020-09-16 15:42       ` Wei Liu
2020-09-16 16:10         ` Vitaly Kuznetsov
2020-09-14 11:27 ` [PATCH RFC v1 07/18] x86/hyperv: extract partition ID from Microsoft Hypervisor if necessary Wei Liu
2020-09-15 10:27   ` Vitaly Kuznetsov
2020-09-16 16:32     ` Wei Liu
2020-10-27 12:19       ` Wei Liu
2020-09-14 11:27 ` [PATCH RFC v1 08/18] x86/hyperv: handling hypercall page setup for root Wei Liu
2020-09-15 10:32   ` Vitaly Kuznetsov
2020-09-15 10:37     ` Wei Liu
2020-09-15 11:02       ` Vitaly Kuznetsov
2020-09-15 11:16         ` Wei Liu
2020-09-15 11:23           ` Vitaly Kuznetsov
2020-09-15 11:27             ` Wei Liu
2020-09-16 21:34       ` [EXTERNAL] " Sunil Muthuswamy
2020-09-17 11:06         ` Vitaly Kuznetsov
2020-09-14 11:59 ` [PATCH RFC v1 09/18] x86/hyperv: provide a bunch of helper functions Wei Liu
2020-09-15 11:00   ` Vitaly Kuznetsov
2020-10-27 13:10     ` Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 10/18] x86/hyperv: implement and use hv_smp_prepare_cpus Wei Liu
2020-09-15 11:14   ` Vitaly Kuznetsov
2020-10-27 13:47     ` Wei Liu
2020-10-27 13:56       ` Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 11/18] asm-generic/hyperv: update hv_msi_entry Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 12/18] asm-generic/hyperv: update hv_interrupt_entry Wei Liu
2020-10-01 14:33   ` Rob Herring
2020-09-14 11:59 ` [PATCH RFC v1 13/18] asm-generic/hyperv: introduce hv_device_id and auxiliary structures Wei Liu
2020-09-15 11:16   ` Vitaly Kuznetsov
2020-09-15 11:59     ` Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 14/18] asm-generic/hyperv: import data structures for mapping device interrupts Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 15/18] x86/apic/msi: export pci_msi_get_hwirq Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 16/18] x86/hyperv: implement MSI domain for root partition Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 17/18] x86/ioapic: export a few functions and data structures via io_apic.h Wei Liu
2020-09-14 11:59 ` [PATCH RFC v1 18/18] x86/hyperv: handle IO-APIC when running as root Wei Liu

Linux-HyperV Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-hyperv/0 linux-hyperv/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-hyperv linux-hyperv/ https://lore.kernel.org/linux-hyperv \
		linux-hyperv@vger.kernel.org
	public-inbox-index linux-hyperv

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-hyperv


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git