[Xen-devel] [PATCH v2 0/8] More Hyper-V infrastructure

[Xen-devel] [PATCH v2 0/8] More Hyper-V infrastructure

[Wei Liu]

This patch sereis implements several important functionalities to run
Xen on top of Hyper-V.

See individual patches for more details.

I can only test them lightly and look at disassembly output to check
correctness at this stage. At the very least Xen on Hyper-V boots up
okay, so it is not breaking anything.

Wei.


Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Wei Liu <wl@xen.org>
Cc: Roger Pau Monné <roger.pau@citrix.com>
Cc: Michael Kelley <mikelley@microsoft.com>
Cc: Paul Durrant <paul@xen.org>

Wei Liu (8):
  x86: include xen/lib.h in guest/pvh-boot.h
  x86/hyperv: detect absolutely necessary MSRs
  x86: rename guest/hypercall.h to guest/xen-hypercall.h
  x86/hyperv: setup hypercall page
  x86/hyperv: provide Hyper-V hypercall functions
  x86/hyperv: provide percpu hypercall input page
  x86/hyperv: retrieve vp_index from Hyper-V
  x86/hyperv: setup VP assist page

 xen/arch/x86/guest/hyperv/Makefile            |   1 +
 xen/arch/x86/guest/hyperv/hypercall_page.S    |  21 ++++
 xen/arch/x86/guest/hyperv/hyperv.c            | 104 +++++++++++++++++-
 xen/include/asm-x86/guest.h                   |   2 +-
 xen/include/asm-x86/guest/hyperv-hypercall.h  |  95 ++++++++++++++++
 xen/include/asm-x86/guest/hyperv.h            |  10 ++
 xen/include/asm-x86/guest/pvh-boot.h          |   2 +
 .../guest/{hypercall.h => xen-hypercall.h}    |   2 +-
 8 files changed, 232 insertions(+), 5 deletions(-)
 create mode 100644 xen/arch/x86/guest/hyperv/hypercall_page.S
 create mode 100644 xen/include/asm-x86/guest/hyperv-hypercall.h
 rename xen/include/asm-x86/guest/{hypercall.h => xen-hypercall.h} (99%)

-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 1/8] x86: include xen/lib.h in guest/pvh-boot.h

[Wei Liu]

It needs ASSERT_UNREACHABLE.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
 xen/include/asm-x86/guest/pvh-boot.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/include/asm-x86/guest/pvh-boot.h b/xen/include/asm-x86/guest/pvh-boot.h
index b8a76c4eed..48ffd1a0b1 100644
--- a/xen/include/asm-x86/guest/pvh-boot.h
+++ b/xen/include/asm-x86/guest/pvh-boot.h
@@ -30,6 +30,8 @@ void pvh_print_info(void);
 
 #else
 
+#include <xen/lib.h>
+
 #define pvh_boot 0
 
 static inline void pvh_init(multiboot_info_t **mbi, module_t **mod)
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 2/8] x86/hyperv: detect absolutely necessary MSRs

[Wei Liu]

If they are not available, disable Hyper-V related features.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
v2:
1. Fix comment style
---
 xen/arch/x86/guest/hyperv/hyperv.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 2e70b4aa82..8d38313d7a 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -32,6 +32,8 @@ static const struct hypervisor_ops ops = {
 const struct hypervisor_ops *__init hyperv_probe(void)
 {
     uint32_t eax, ebx, ecx, edx;
+    uint64_t required_msrs = HV_X64_MSR_HYPERCALL_AVAILABLE |
+        HV_X64_MSR_VP_INDEX_AVAILABLE;
 
     cpuid(0x40000000, &eax, &ebx, &ecx, &edx);
     if ( !((ebx == 0x7263694d) &&  /* "Micr" */
@@ -57,6 +59,16 @@ const struct hypervisor_ops *__init hyperv_probe(void)
     ms_hyperv.max_vp_index = eax;
     ms_hyperv.max_lp_index = ebx;
 
+    if ( (ms_hyperv.features & required_msrs) != required_msrs )
+    {
+        /*
+         * Oops, required MSRs are not available. Treat this as
+         * "Hyper-V is not available".
+         */
+        memset(&ms_hyperv, 0, sizeof(ms_hyperv));
+        return NULL;
+    }
+
     return &ops;
 }
 
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 3/8] x86: rename guest/hypercall.h to guest/xen-hypercall.h

[Wei Liu]

We will provide a header file for Hyper-V hypercalls.

No functional change.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
Reviewed-by: Paul Durrant <paul@xen.org>
---
 xen/include/asm-x86/guest.h                                | 2 +-
 xen/include/asm-x86/guest/{hypercall.h => xen-hypercall.h} | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
 rename xen/include/asm-x86/guest/{hypercall.h => xen-hypercall.h} (99%)

diff --git a/xen/include/asm-x86/guest.h b/xen/include/asm-x86/guest.h
index 94448606d4..7f09c643d4 100644
--- a/xen/include/asm-x86/guest.h
+++ b/xen/include/asm-x86/guest.h
@@ -19,11 +19,11 @@
 #ifndef __X86_GUEST_H__
 #define __X86_GUEST_H__
 
-#include <asm/guest/hypercall.h>
 #include <asm/guest/hyperv.h>
 #include <asm/guest/hypervisor.h>
 #include <asm/guest/pvh-boot.h>
 #include <asm/guest/xen.h>
+#include <asm/guest/xen-hypercall.h>
 #include <asm/pv/shim.h>
 
 #endif /* __X86_GUEST_H__ */
diff --git a/xen/include/asm-x86/guest/hypercall.h b/xen/include/asm-x86/guest/xen-hypercall.h
similarity index 99%
rename from xen/include/asm-x86/guest/hypercall.h
rename to xen/include/asm-x86/guest/xen-hypercall.h
index d0d2f5022d..681d3aecd7 100644
--- a/xen/include/asm-x86/guest/hypercall.h
+++ b/xen/include/asm-x86/guest/xen-hypercall.h
@@ -1,5 +1,5 @@
 /******************************************************************************
- * asm-x86/guest/hypercall.h
+ * asm-x86/guest/xen-hypercall.h
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms and conditions of the GNU General Public
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 4/8] x86/hyperv: setup hypercall page

[Wei Liu]

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
v2:
1. Fix issue discovered by Michael
2. Use a statically allocated page as hypercall page
---
 xen/arch/x86/guest/hyperv/Makefile         |  1 +
 xen/arch/x86/guest/hyperv/hypercall_page.S | 21 +++++++++++++++++
 xen/arch/x86/guest/hyperv/hyperv.c         | 27 +++++++++++++++++++---
 3 files changed, 46 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/x86/guest/hyperv/hypercall_page.S

diff --git a/xen/arch/x86/guest/hyperv/Makefile b/xen/arch/x86/guest/hyperv/Makefile
index 68170109a9..1a8887d2f4 100644
--- a/xen/arch/x86/guest/hyperv/Makefile
+++ b/xen/arch/x86/guest/hyperv/Makefile
@@ -1 +1,2 @@
+obj-y += hypercall_page.o
 obj-y += hyperv.o
diff --git a/xen/arch/x86/guest/hyperv/hypercall_page.S b/xen/arch/x86/guest/hyperv/hypercall_page.S
new file mode 100644
index 0000000000..6d6ab913be
--- /dev/null
+++ b/xen/arch/x86/guest/hyperv/hypercall_page.S
@@ -0,0 +1,21 @@
+#include <asm/asm_defns.h>
+#include <asm/page.h>
+
+        .section ".text.page_aligned", "ax", @progbits
+        .p2align PAGE_SHIFT
+GLOBAL(hv_hypercall_page)
+        /* Return -1 for "not yet ready" state */
+        mov -1, %rax
+        ret
+1:
+        /* Fill the rest with `ret` */
+        .fill PAGE_SIZE - (1b - hv_hypercall_page), 1, 0xc3
+        .type hv_hypercall_page, STT_OBJECT
+        .size hv_hypercall_page, PAGE_SIZE
+
+/*
+ * Local variables:
+ * tab-width: 8
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 8d38313d7a..381be2a68c 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -19,16 +19,16 @@
  * Copyright (c) 2019 Microsoft.
  */
 #include <xen/init.h>
+#include <xen/domain_page.h>
 
 #include <asm/guest.h>
 #include <asm/guest/hyperv-tlfs.h>
 
 struct ms_hyperv_info __read_mostly ms_hyperv;
 
-static const struct hypervisor_ops ops = {
-    .name = "Hyper-V",
-};
+extern char hv_hypercall_page[];
 
+static const struct hypervisor_ops ops;
 const struct hypervisor_ops *__init hyperv_probe(void)
 {
     uint32_t eax, ebx, ecx, edx;
@@ -72,6 +72,27 @@ const struct hypervisor_ops *__init hyperv_probe(void)
     return &ops;
 }
 
+static void __init setup_hypercall_page(void)
+{
+    union hv_x64_msr_hypercall_contents hypercall_msr;
+
+    rdmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+    hypercall_msr.enable = 1;
+    hypercall_msr.guest_physical_address =
+        __pa(hv_hypercall_page) >> HV_HYP_PAGE_SHIFT;
+    wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
+}
+
+static void __init setup(void)
+{
+    setup_hypercall_page();
+}
+
+static const struct hypervisor_ops ops = {
+    .name = "Hyper-V",
+    .setup = setup,
+};
+
 /*
  * Local variables:
  * mode: C
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 5/8] x86/hyperv: provide Hyper-V hypercall functions

[Wei Liu]

These functions will be used later to make hypercalls to Hyper-V.

I couldn't find reference in TLFS that Hyper-V clobbers flags and
r9-r11, but Linux's commit message says it does. Err on the safe side.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
v2:
1. Use direct call
---
 xen/include/asm-x86/guest/hyperv-hypercall.h | 95 ++++++++++++++++++++
 1 file changed, 95 insertions(+)
 create mode 100644 xen/include/asm-x86/guest/hyperv-hypercall.h

diff --git a/xen/include/asm-x86/guest/hyperv-hypercall.h b/xen/include/asm-x86/guest/hyperv-hypercall.h
new file mode 100644
index 0000000000..928d85ae7e
--- /dev/null
+++ b/xen/include/asm-x86/guest/hyperv-hypercall.h
@@ -0,0 +1,95 @@
+/******************************************************************************
+ * asm-x86/guest/hyperv-hypercall.h
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms and conditions of the GNU General Public
+ * License, version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ * Copyright (c) 2019 Microsoft.
+ */
+
+#ifndef __X86_HYPERV_HYPERCALL_H__
+#define __X86_HYPERV_HYPERCALL_H__
+
+#include <xen/types.h>
+
+#include <asm/asm_defns.h>
+#include <asm/guest/hyperv-tlfs.h>
+#include <asm/page.h>
+
+static inline uint64_t hv_do_hypercall(uint64_t control, paddr_t input, paddr_t output)
+{
+    uint64_t status;
+
+    asm volatile ("mov %[output], %%r8\n"
+                  "call hv_hypercall_page"
+                  : "=a" (status), "+c" (control),
+                    "+d" (input) ASM_CALL_CONSTRAINT
+                  : [output] "rm" (output)
+                  : "cc", "memory", "r8", "r9", "r10", "r11");
+
+    return status;
+}
+
+static inline uint64_t hv_do_fast_hypercall(uint16_t code,
+                                            uint64_t input1, uint64_t input2)
+{
+    uint64_t status;
+    uint64_t control = (uint64_t)code | HV_HYPERCALL_FAST_BIT;
+
+    asm volatile ("mov %[input2], %%r8\n"
+                  "call hv_hypercall_page"
+                  : "=a" (status), "+c" (control),
+                    "+d" (input1) ASM_CALL_CONSTRAINT
+                  : [input2] "rm" (input2)
+                  : "cc", "r8", "r9", "r10", "r11");
+
+    return status;
+}
+
+static inline uint64_t hv_do_rep_hypercall(uint16_t code, uint16_t rep_count,
+                                           uint16_t varhead_size,
+                                           paddr_t input, paddr_t output)
+{
+    uint64_t control = code;
+    uint64_t status;
+    uint16_t rep_comp;
+
+    control |= (uint64_t)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET;
+    control |= (uint64_t)rep_count << HV_HYPERCALL_REP_COMP_OFFSET;
+
+    do {
+        status = hv_do_hypercall(control, input, output);
+        if ( (status & HV_HYPERCALL_RESULT_MASK) != HV_STATUS_SUCCESS )
+            break;
+
+        rep_comp = (status & HV_HYPERCALL_REP_COMP_MASK) >>
+            HV_HYPERCALL_REP_COMP_OFFSET;
+
+        control &= ~HV_HYPERCALL_REP_START_MASK;
+        control |= (uint64_t)rep_comp << HV_HYPERCALL_REP_COMP_OFFSET;
+
+    } while ( rep_comp < rep_count );
+
+    return status;
+}
+
+#endif /* __X86_HYPERV_HYPERCALL_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 6/8] x86/hyperv: provide percpu hypercall input page

[Wei Liu]

Hyper-V's input / output argument must be 8 bytes aligned an not cross
page boundary. The easiest way to satisfy those requirements is to use
percpu page.

For the foreseeable future we only need to provide input for TLB
and APIC hypercalls, so skip setting up an output page.

The page tracking structure is not bound to hypercall because it is a
common pattern for Xen to write guest physical address to Hyper-V while
at the same time accessing the page via a pointer.

We will also need to provide an ap_setup hook for secondary cpus to
setup its own input page.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
Reviewed-by: Paul Durrant <paul@xen.org>
---
 xen/arch/x86/guest/hyperv/hyperv.c | 26 ++++++++++++++++++++++++++
 xen/include/asm-x86/guest/hyperv.h |  8 ++++++++
 2 files changed, 34 insertions(+)

diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 381be2a68c..03027bd453 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -27,6 +27,7 @@
 struct ms_hyperv_info __read_mostly ms_hyperv;
 
 extern char hv_hypercall_page[];
+DEFINE_PER_CPU_READ_MOSTLY(struct hyperv_pcpu_page, hv_pcpu_input_arg);
 
 static const struct hypervisor_ops ops;
 const struct hypervisor_ops *__init hyperv_probe(void)
@@ -83,14 +84,39 @@ static void __init setup_hypercall_page(void)
     wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
 }
 
+static void setup_hypercall_pcpu_arg(void)
+{
+    struct page_info *pg;
+    void *mapping;
+    unsigned int cpu = smp_processor_id();
+
+    pg = alloc_domheap_page(NULL, 0);
+    if ( !pg )
+        panic("Failed to setup hypercall input page for %u\n", cpu);
+
+    mapping = __map_domain_page_global(pg);
+    if ( !mapping )
+        panic("Failed to map hypercall input page for %u\n", cpu);
+
+    this_cpu(hv_pcpu_input_arg).maddr = page_to_maddr(pg);
+    this_cpu(hv_pcpu_input_arg).mapping = mapping;
+}
+
 static void __init setup(void)
 {
     setup_hypercall_page();
+    setup_hypercall_pcpu_arg();
+}
+
+static void ap_setup(void)
+{
+    setup_hypercall_pcpu_arg();
 }
 
 static const struct hypervisor_ops ops = {
     .name = "Hyper-V",
     .setup = setup,
+    .ap_setup = ap_setup,
 };
 
 /*
diff --git a/xen/include/asm-x86/guest/hyperv.h b/xen/include/asm-x86/guest/hyperv.h
index c7a7f32bd5..83f297468f 100644
--- a/xen/include/asm-x86/guest/hyperv.h
+++ b/xen/include/asm-x86/guest/hyperv.h
@@ -51,6 +51,8 @@ static inline uint64_t hv_scale_tsc(uint64_t tsc, uint64_t scale,
 
 #ifdef CONFIG_HYPERV_GUEST
 
+#include <xen/percpu.h>
+
 #include <asm/guest/hypervisor.h>
 
 struct ms_hyperv_info {
@@ -63,6 +65,12 @@ struct ms_hyperv_info {
 };
 extern struct ms_hyperv_info ms_hyperv;
 
+struct hyperv_pcpu_page {
+    paddr_t maddr;
+    void *mapping;
+};
+DECLARE_PER_CPU(struct hyperv_pcpu_page, hv_pcpu_input_arg);
+
 const struct hypervisor_ops *hyperv_probe(void);
 
 #else
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 7/8] x86/hyperv: retrieve vp_index from Hyper-V

[Wei Liu]

This will be useful when invoking hypercall that targets specific
vcpu(s).

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
v2:
1. Fold into setup_pcpu_arg function
---
 xen/arch/x86/guest/hyperv/hyperv.c | 5 +++++
 xen/include/asm-x86/guest/hyperv.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 03027bd453..5c5aed46cb 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -28,6 +28,7 @@ struct ms_hyperv_info __read_mostly ms_hyperv;
 
 extern char hv_hypercall_page[];
 DEFINE_PER_CPU_READ_MOSTLY(struct hyperv_pcpu_page, hv_pcpu_input_arg);
+DEFINE_PER_CPU_READ_MOSTLY(unsigned int, hv_vp_index);
 
 static const struct hypervisor_ops ops;
 const struct hypervisor_ops *__init hyperv_probe(void)
@@ -89,6 +90,7 @@ static void setup_hypercall_pcpu_arg(void)
     struct page_info *pg;
     void *mapping;
     unsigned int cpu = smp_processor_id();
+    uint64_t vp_index_msr;
 
     pg = alloc_domheap_page(NULL, 0);
     if ( !pg )
@@ -100,6 +102,9 @@ static void setup_hypercall_pcpu_arg(void)
 
     this_cpu(hv_pcpu_input_arg).maddr = page_to_maddr(pg);
     this_cpu(hv_pcpu_input_arg).mapping = mapping;
+
+    rdmsrl(HV_X64_MSR_VP_INDEX, vp_index_msr);
+    this_cpu(hv_vp_index) = vp_index_msr;
 }
 
 static void __init setup(void)
diff --git a/xen/include/asm-x86/guest/hyperv.h b/xen/include/asm-x86/guest/hyperv.h
index 83f297468f..4b635829f3 100644
--- a/xen/include/asm-x86/guest/hyperv.h
+++ b/xen/include/asm-x86/guest/hyperv.h
@@ -70,6 +70,7 @@ struct hyperv_pcpu_page {
     void *mapping;
 };
 DECLARE_PER_CPU(struct hyperv_pcpu_page, hv_pcpu_input_arg);
+DECLARE_PER_CPU(unsigned int, hv_vp_index);
 
 const struct hypervisor_ops *hyperv_probe(void);
 
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 8/8] x86/hyperv: setup VP assist page

[Wei Liu]

VP assist page is rather important as we need to toggle some bits in
for efficient nested virtualisation.

Preemptively split out set_vp_assist page which will be used in the resume
path.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
---
v2:
1. Use HV_HYP_PAGE_SHIFT instead
---
 xen/arch/x86/guest/hyperv/hyperv.c | 34 ++++++++++++++++++++++++++++++
 xen/include/asm-x86/guest/hyperv.h |  1 +
 2 files changed, 35 insertions(+)

diff --git a/xen/arch/x86/guest/hyperv/hyperv.c b/xen/arch/x86/guest/hyperv/hyperv.c
index 5c5aed46cb..cf6ad13e48 100644
--- a/xen/arch/x86/guest/hyperv/hyperv.c
+++ b/xen/arch/x86/guest/hyperv/hyperv.c
@@ -29,6 +29,7 @@ struct ms_hyperv_info __read_mostly ms_hyperv;
 extern char hv_hypercall_page[];
 DEFINE_PER_CPU_READ_MOSTLY(struct hyperv_pcpu_page, hv_pcpu_input_arg);
 DEFINE_PER_CPU_READ_MOSTLY(unsigned int, hv_vp_index);
+DEFINE_PER_CPU_READ_MOSTLY(struct hyperv_pcpu_page, hv_vp_assist);
 
 static const struct hypervisor_ops ops;
 const struct hypervisor_ops *__init hyperv_probe(void)
@@ -107,15 +108,48 @@ static void setup_hypercall_pcpu_arg(void)
     this_cpu(hv_vp_index) = vp_index_msr;
 }
 
+static void set_vp_assist(void)
+{
+    uint64_t val = paddr_to_pfn(this_cpu(hv_vp_assist).maddr);
+
+    val = (val << HV_HYP_PAGE_SHIFT) | HV_X64_MSR_VP_ASSIST_PAGE_ENABLE;
+
+    wrmsrl(HV_X64_MSR_VP_ASSIST_PAGE, val);
+}
+
+static void setup_vp_assist(void)
+{
+    struct page_info *pg;
+    void *mapping;
+    unsigned int cpu = smp_processor_id();
+
+    pg = alloc_domheap_page(NULL, 0);
+    if ( !pg )
+        panic("Failed to allocate vp_assist page for %u\n", cpu);
+
+    mapping = __map_domain_page_global(pg);
+    if ( !mapping )
+        panic("Failed to map vp_assist page for %u\n", cpu);
+
+    clear_page(mapping);
+
+    this_cpu(hv_vp_assist).maddr = page_to_maddr(pg);
+    this_cpu(hv_vp_assist).mapping = mapping;
+
+    set_vp_assist();
+}
+
 static void __init setup(void)
 {
     setup_hypercall_page();
     setup_hypercall_pcpu_arg();
+    setup_vp_assist();
 }
 
 static void ap_setup(void)
 {
     setup_hypercall_pcpu_arg();
+    setup_vp_assist();
 }
 
 static const struct hypervisor_ops ops = {
diff --git a/xen/include/asm-x86/guest/hyperv.h b/xen/include/asm-x86/guest/hyperv.h
index 4b635829f3..917f4e02c2 100644
--- a/xen/include/asm-x86/guest/hyperv.h
+++ b/xen/include/asm-x86/guest/hyperv.h
@@ -71,6 +71,7 @@ struct hyperv_pcpu_page {
 };
 DECLARE_PER_CPU(struct hyperv_pcpu_page, hv_pcpu_input_arg);
 DECLARE_PER_CPU(unsigned int, hv_vp_index);
+DECLARE_PER_CPU(struct hyperv_pcpu_page, hv_vp_assist);
 
 const struct hypervisor_ops *hyperv_probe(void);
 
-- 
2.20.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 6/8] x86/hyperv: provide percpu hypercall input page

[Andrew Cooper]

On 03/01/2020 16:08, Wei Liu wrote:
> @@ -83,14 +84,39 @@ static void __init setup_hypercall_page(void)
>      wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>  }
>  
> +static void setup_hypercall_pcpu_arg(void)
> +{
> +    struct page_info *pg;
> +    void *mapping;
> +    unsigned int cpu = smp_processor_id();
> +
> +    pg = alloc_domheap_page(NULL, 0);
> +    if ( !pg )
> +        panic("Failed to setup hypercall input page for %u\n", cpu);
> +
> +    mapping = __map_domain_page_global(pg);
> +    if ( !mapping )
> +        panic("Failed to map hypercall input page for %u\n", cpu);

Sorry I didn't spot this before, but an always-mapped domheap page is
just alloc_xenheap_page() (give or take NUMA positioning above the 5T
boundary, which isn't used here).

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 6/8] x86/hyperv: provide percpu hypercall input page

[Wei Liu]

On Fri, Jan 03, 2020 at 04:30:49PM +0000, Andrew Cooper wrote:
> On 03/01/2020 16:08, Wei Liu wrote:
> > @@ -83,14 +84,39 @@ static void __init setup_hypercall_page(void)
> >      wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >  }
> >  
> > +static void setup_hypercall_pcpu_arg(void)
> > +{
> > +    struct page_info *pg;
> > +    void *mapping;
> > +    unsigned int cpu = smp_processor_id();
> > +
> > +    pg = alloc_domheap_page(NULL, 0);
> > +    if ( !pg )
> > +        panic("Failed to setup hypercall input page for %u\n", cpu);
> > +
> > +    mapping = __map_domain_page_global(pg);
> > +    if ( !mapping )
> > +        panic("Failed to map hypercall input page for %u\n", cpu);
> 
> Sorry I didn't spot this before, but an always-mapped domheap page is
> just alloc_xenheap_page() (give or take NUMA positioning above the 5T
> boundary, which isn't used here).

I had the (wrong) impression that using domheap was preferred.

I'm fine with switching to xenheap, of course.

Wei.



> 
> ~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 6/8] x86/hyperv: provide percpu hypercall input page

[Andrew Cooper]

On 03/01/2020 16:55, Wei Liu wrote:
> On Fri, Jan 03, 2020 at 04:30:49PM +0000, Andrew Cooper wrote:
>> On 03/01/2020 16:08, Wei Liu wrote:
>>> @@ -83,14 +84,39 @@ static void __init setup_hypercall_page(void)
>>>      wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
>>>  }
>>>  
>>> +static void setup_hypercall_pcpu_arg(void)
>>> +{
>>> +    struct page_info *pg;
>>> +    void *mapping;
>>> +    unsigned int cpu = smp_processor_id();
>>> +
>>> +    pg = alloc_domheap_page(NULL, 0);
>>> +    if ( !pg )
>>> +        panic("Failed to setup hypercall input page for %u\n", cpu);
>>> +
>>> +    mapping = __map_domain_page_global(pg);
>>> +    if ( !mapping )
>>> +        panic("Failed to map hypercall input page for %u\n", cpu);
>> Sorry I didn't spot this before, but an always-mapped domheap page is
>> just alloc_xenheap_page() (give or take NUMA positioning above the 5T
>> boundary, which isn't used here).
> I had the (wrong) impression that using domheap was preferred.
>
> I'm fine with switching to xenheap, of course.

This is a frame which Xen needs to have a mapping to in perpetuity, to
make hypercalls.

Most examples in code are a regular domheap frame which, after some
guest action, requires mapping in Xen for a period of time, or frames
which we want to have specific NUMA properties, and may be beyond the
end of the directmap.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 6/8] x86/hyperv: provide percpu hypercall input page

[Wei Liu]

On Fri, Jan 03, 2020 at 04:57:11PM +0000, Andrew Cooper wrote:
> On 03/01/2020 16:55, Wei Liu wrote:
> > On Fri, Jan 03, 2020 at 04:30:49PM +0000, Andrew Cooper wrote:
> >> On 03/01/2020 16:08, Wei Liu wrote:
> >>> @@ -83,14 +84,39 @@ static void __init setup_hypercall_page(void)
> >>>      wrmsrl(HV_X64_MSR_HYPERCALL, hypercall_msr.as_uint64);
> >>>  }
> >>>  
> >>> +static void setup_hypercall_pcpu_arg(void)
> >>> +{
> >>> +    struct page_info *pg;
> >>> +    void *mapping;
> >>> +    unsigned int cpu = smp_processor_id();
> >>> +
> >>> +    pg = alloc_domheap_page(NULL, 0);
> >>> +    if ( !pg )
> >>> +        panic("Failed to setup hypercall input page for %u\n", cpu);
> >>> +
> >>> +    mapping = __map_domain_page_global(pg);
> >>> +    if ( !mapping )
> >>> +        panic("Failed to map hypercall input page for %u\n", cpu);
> >> Sorry I didn't spot this before, but an always-mapped domheap page is
> >> just alloc_xenheap_page() (give or take NUMA positioning above the 5T
> >> boundary, which isn't used here).
> > I had the (wrong) impression that using domheap was preferred.
> >
> > I'm fine with switching to xenheap, of course.
> 
> This is a frame which Xen needs to have a mapping to in perpetuity, to
> make hypercalls.
> 
> Most examples in code are a regular domheap frame which, after some
> guest action, requires mapping in Xen for a period of time, or frames
> which we want to have specific NUMA properties, and may be beyond the
> end of the directmap.

Alright.

If we use xenheap here I can drop the tracking structure.

I will change that in v3.

Wei.

> 
> ~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 8/8] x86/hyperv: setup VP assist page

[Wei Liu]

On Fri, Jan 03, 2020 at 04:08:25PM +0000, Wei Liu wrote:
> 
> Preemptively split out set_vp_assist page which will be used in the resume
> path.

After going through TLFS's section on reenlightenment, I don't think
this is necessary.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel