[PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[Igor Druzhinin]

TLFS 7.8.1 stipulates that "a virtual processor index must be less than
the maximum number of virtual processors per partition" that "can be obtained
through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing
the Microsoft Hypervisor Interface" defines that starting from Windows Server
2012, which allowed more than 64 CPUs to be brought up, this leaf can now
contain a value -1 basically assuming the hypervisor has no restriction while
0 (that we currently expose) means the default restriction is still present.

Along with the previous changes exposing ExProcessorMasks this allows a recent
Windows VM with Viridian extension enabled to have more than 64 vCPUs without
going into BSOD in some cases.

Since we didn't expose the leaf before and to keep CPUID data consistent for
incoming streams from previous Xen versions - let's keep it behind an option.

Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
---
Changes in v2:
- expose the option in libxl
---
 docs/man/xl.cfg.5.pod.in             |  9 ++++++++-
 tools/include/libxl.h                |  6 ++++++
 tools/libs/light/libxl_types.idl     |  1 +
 tools/libs/light/libxl_x86.c         |  4 ++++
 xen/arch/x86/hvm/viridian/viridian.c | 23 +++++++++++++++++++++++
 xen/include/public/hvm/params.h      |  7 ++++++-
 6 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index c8e017f..3467eae 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2260,11 +2260,18 @@ mask. Hence this enlightenment must be specified for guests with more
 than 64 vCPUs if B<hcall_remote_tlb_flush> and/or B<hcall_ipi> are also
 specified.
 
+=item B<no_vp_limit>
+
+This group when set indicates to a guest that the hypervisor does not
+explicitly have any limits on the number of Virtual processors a guest
+is allowed to bring up. It is strongly recommended to keep this enabled
+for guests with more than 64 vCPUs.
+
 =item B<defaults>
 
 This is a special value that enables the default set of groups, which
 is currently the B<base>, B<freq>, B<time_ref_count>, B<apic_assist>,
-B<crash_ctl> and B<stimer> groups.
+B<crash_ctl>, B<stimer> and B<no_vp_limit> groups.
 
 =item B<all>
 
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 3433c95..be1e288 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -452,6 +452,12 @@
 #define LIBXL_HAVE_VIRIDIAN_EX_PROCESSOR_MASKS 1
 
 /*
+ * LIBXL_HAVE_VIRIDIAN_NO_VP_LIMIT indicates that the 'no_vp_limit' value
+ * is present in the viridian enlightenment enumeration.
+ */
+#define LIBXL_HAVE_VIRIDIAN_NO_VP_LIMIT 1
+
+/*
  * LIBXL_HAVE_DEVICE_PCI_LIST_FREE indicates that the
  * libxl_device_pci_list_free() function is defined.
  */
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 0532473..8502b29 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -239,6 +239,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [
     (8, "stimer"),
     (9, "hcall_ipi"),
     (10, "ex_processor_masks"),
+    (11, "no_vp_limit"),
     ])
 
 libxl_hdtype = Enumeration("hdtype", [
diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c
index 86d2729..5c4c194 100644
--- a/tools/libs/light/libxl_x86.c
+++ b/tools/libs/light/libxl_x86.c
@@ -309,6 +309,7 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,
         libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_TIME_REF_COUNT);
         libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_APIC_ASSIST);
         libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CRASH_CTL);
+        libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_NO_VP_LIMIT);
     }
 
     libxl_for_each_set_bit(v, info->u.hvm.viridian_enable) {
@@ -369,6 +370,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,
     if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_EX_PROCESSOR_MASKS))
         mask |= HVMPV_ex_processor_masks;
 
+    if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_NO_VP_LIMIT))
+        mask |= HVMPV_no_vp_limit;
+
     if (mask != 0 &&
         xc_hvm_param_set(CTX->xch,
                          domid,
diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
index ed97804..ae1ea86 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -209,6 +209,29 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
         res->b = viridian_spinlock_retry_count;
         break;
 
+    case 5:
+        /*
+         * From "Requirements for Implementing the Microsoft Hypervisor
+         *  Interface":
+         *
+         * "On Windows operating systems versions through Windows Server
+         * 2008 R2, reporting the HV#1 hypervisor interface limits
+         * the Windows virtual machine to a maximum of 64 VPs, regardless of
+         * what is reported via CPUID.40000005.EAX.
+         *
+         * Starting with Windows Server 2012 and Windows 8, if
+         * CPUID.40000005.EAX containsa value of -1, Windows assumes that
+         * the hypervisor imposes no specific limit to the number of VPs.
+         * In this case, Windows Server 2012 guest VMs may use more than 64
+         * VPs, up to the maximum supported number of processors applicable
+         * to the specific Windows version being used."
+         *
+         * For compatibility we hide it behind an option.
+         */
+        if ( viridian_feature_mask(d) & HVMPV_no_vp_limit )
+            res->a = -1;
+        break;
+
     case 6:
         /* Detected and in use hardware features. */
         if ( cpu_has_vmx_virtualize_apic_accesses )
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index 3b0a0f4..805f4ca 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -168,6 +168,10 @@
 #define _HVMPV_ex_processor_masks 10
 #define HVMPV_ex_processor_masks (1 << _HVMPV_ex_processor_masks)
 
+/* Allow more than 64 VPs */
+#define _HVMPV_no_vp_limit 11
+#define HVMPV_no_vp_limit (1 << _HVMPV_no_vp_limit)
+
 #define HVMPV_feature_mask \
         (HVMPV_base_freq | \
          HVMPV_no_freq | \
@@ -179,7 +183,8 @@
          HVMPV_synic | \
          HVMPV_stimer | \
          HVMPV_hcall_ipi | \
-         HVMPV_ex_processor_masks)
+         HVMPV_ex_processor_masks | \
+         HVMPV_no_vp_limit)
 
 #endif
 
-- 
2.7.4

[PATCH v2 2/2] viridian: allow vCPU hotplug for Windows VMs

[Igor Druzhinin]

If Viridian extensions are enabled, Windows wouldn't currently allow
a hotplugged vCPU to be brought up dynamically. We need to expose a special
bit to let the guest know we allow it. Hide it behind an option to stay
on the safe side regarding compatibility with existing guests but
nevertheless set the option on by default.

Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
---
Changes on v2:
- hide the bit under an option and expose it in libxl
---
 docs/man/xl.cfg.5.pod.in             | 7 ++++++-
 tools/include/libxl.h                | 6 ++++++
 tools/libs/light/libxl_types.idl     | 1 +
 tools/libs/light/libxl_x86.c         | 4 ++++
 xen/arch/x86/hvm/viridian/viridian.c | 5 ++++-
 xen/include/public/hvm/params.h      | 7 ++++++-
 6 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 3467eae..7cdb859 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2267,11 +2267,16 @@ explicitly have any limits on the number of Virtual processors a guest
 is allowed to bring up. It is strongly recommended to keep this enabled
 for guests with more than 64 vCPUs.
 
+=item B<cpu_hotplug>
+
+This set enables dynamic changes to Virtual processor states in Windows
+guests effectively allowing vCPU hotplug.
+
 =item B<defaults>
 
 This is a special value that enables the default set of groups, which
 is currently the B<base>, B<freq>, B<time_ref_count>, B<apic_assist>,
-B<crash_ctl>, B<stimer> and B<no_vp_limit> groups.
+B<crash_ctl>, B<stimer>, B<no_vp_limit> and B<cpu_hotplug> groups.
 
 =item B<all>
 
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index be1e288..7c7c541 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -458,6 +458,12 @@
 #define LIBXL_HAVE_VIRIDIAN_NO_VP_LIMIT 1
 
 /*
+ * LIBXL_HAVE_VIRIDIAN_CPU_HOTPLUG indicates that the 'cpu_hotplug' value
+ * is present in the viridian enlightenment enumeration.
+ */
+#define LIBXL_HAVE_VIRIDIAN_CPU_HOTPLUG 1
+
+/*
  * LIBXL_HAVE_DEVICE_PCI_LIST_FREE indicates that the
  * libxl_device_pci_list_free() function is defined.
  */
diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
index 8502b29..00a8e68 100644
--- a/tools/libs/light/libxl_types.idl
+++ b/tools/libs/light/libxl_types.idl
@@ -240,6 +240,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [
     (9, "hcall_ipi"),
     (10, "ex_processor_masks"),
     (11, "no_vp_limit"),
+    (12, "cpu_hotplug"),
     ])
 
 libxl_hdtype = Enumeration("hdtype", [
diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c
index 5c4c194..91a9fc7 100644
--- a/tools/libs/light/libxl_x86.c
+++ b/tools/libs/light/libxl_x86.c
@@ -310,6 +310,7 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,
         libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_APIC_ASSIST);
         libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CRASH_CTL);
         libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_NO_VP_LIMIT);
+        libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CPU_HOTPLUG);
     }
 
     libxl_for_each_set_bit(v, info->u.hvm.viridian_enable) {
@@ -373,6 +374,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,
     if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_NO_VP_LIMIT))
         mask |= HVMPV_no_vp_limit;
 
+    if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CPU_HOTPLUG))
+        mask |= HVMPV_cpu_hotplug;
+
     if (mask != 0 &&
         xc_hvm_param_set(CTX->xch,
                          domid,
diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
index ae1ea86..b906f7b 100644
--- a/xen/arch/x86/hvm/viridian/viridian.c
+++ b/xen/arch/x86/hvm/viridian/viridian.c
@@ -76,6 +76,7 @@ typedef union _HV_CRASH_CTL_REG_CONTENTS
 } HV_CRASH_CTL_REG_CONTENTS;
 
 /* Viridian CPUID leaf 3, Hypervisor Feature Indication */
+#define CPUID3D_CPU_DYNAMIC_PARTITIONING (1 << 3)
 #define CPUID3D_CRASH_MSRS (1 << 10)
 #define CPUID3D_SINT_POLLING (1 << 17)
 
@@ -179,8 +180,10 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
         res->a = u.lo;
         res->b = u.hi;
 
+        if ( viridian_feature_mask(d) & HVMPV_cpu_hotplug )
+           res->d = CPUID3D_CPU_DYNAMIC_PARTITIONING;
         if ( viridian_feature_mask(d) & HVMPV_crash_ctl )
-            res->d = CPUID3D_CRASH_MSRS;
+            res->d |= CPUID3D_CRASH_MSRS;
         if ( viridian_feature_mask(d) & HVMPV_synic )
             res->d |= CPUID3D_SINT_POLLING;
 
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index 805f4ca..c9d6e70 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -172,6 +172,10 @@
 #define _HVMPV_no_vp_limit 11
 #define HVMPV_no_vp_limit (1 << _HVMPV_no_vp_limit)
 
+/* Enable vCPU hotplug */
+#define _HVMPV_cpu_hotplug 12
+#define HVMPV_cpu_hotplug (1 << _HVMPV_cpu_hotplug)
+
 #define HVMPV_feature_mask \
         (HVMPV_base_freq | \
          HVMPV_no_freq | \
@@ -184,7 +188,8 @@
          HVMPV_stimer | \
          HVMPV_hcall_ipi | \
          HVMPV_ex_processor_masks | \
-         HVMPV_no_vp_limit)
+         HVMPV_no_vp_limit | \
+         HVMPV_cpu_hotplug)
 
 #endif
 
-- 
2.7.4

Re: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[Igor Druzhinin]

On 12/01/2021 04:17, Igor Druzhinin wrote:
> TLFS 7.8.1 stipulates that "a virtual processor index must be less than
> the maximum number of virtual processors per partition" that "can be obtained
> through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing
> the Microsoft Hypervisor Interface" defines that starting from Windows Server
> 2012, which allowed more than 64 CPUs to be brought up, this leaf can now
> contain a value -1 basically assuming the hypervisor has no restriction while
> 0 (that we currently expose) means the default restriction is still present.
> 
> Along with the previous changes exposing ExProcessorMasks this allows a recent
> Windows VM with Viridian extension enabled to have more than 64 vCPUs without
> going into BSOD in some cases.
> 
> Since we didn't expose the leaf before and to keep CPUID data consistent for
> incoming streams from previous Xen versions - let's keep it behind an option.
> 
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
> ---

ping? Paul?

Igor

RE: [PATCH v2 2/2] viridian: allow vCPU hotplug for Windows VMs

[Paul Durrant]

> -----Original Message-----
> From: Igor Druzhinin <igor.druzhinin@citrix.com>
> Sent: 12 January 2021 04:17
> To: xen-devel@lists.xenproject.org
> Cc: iwj@xenproject.org; wl@xen.org; andrew.cooper3@citrix.com; george.dunlap@citrix.com;
> jbeulich@suse.com; julien@xen.org; sstabellini@kernel.org; anthony.perard@citrix.com; paul@xen.org;
> roger.pau@citrix.com; Igor Druzhinin <igor.druzhinin@citrix.com>
> Subject: [PATCH v2 2/2] viridian: allow vCPU hotplug for Windows VMs
> 
> If Viridian extensions are enabled, Windows wouldn't currently allow
> a hotplugged vCPU to be brought up dynamically. We need to expose a special
> bit to let the guest know we allow it. Hide it behind an option to stay
> on the safe side regarding compatibility with existing guests but
> nevertheless set the option on by default.
> 
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>

LGTM

Reviewed-by: Paul Durrant <paul@xen.org>

> ---
> Changes on v2:
> - hide the bit under an option and expose it in libxl
> ---
>  docs/man/xl.cfg.5.pod.in             | 7 ++++++-
>  tools/include/libxl.h                | 6 ++++++
>  tools/libs/light/libxl_types.idl     | 1 +
>  tools/libs/light/libxl_x86.c         | 4 ++++
>  xen/arch/x86/hvm/viridian/viridian.c | 5 ++++-
>  xen/include/public/hvm/params.h      | 7 ++++++-
>  6 files changed, 27 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
> index 3467eae..7cdb859 100644
> --- a/docs/man/xl.cfg.5.pod.in
> +++ b/docs/man/xl.cfg.5.pod.in
> @@ -2267,11 +2267,16 @@ explicitly have any limits on the number of Virtual processors a guest
>  is allowed to bring up. It is strongly recommended to keep this enabled
>  for guests with more than 64 vCPUs.
> 
> +=item B<cpu_hotplug>
> +
> +This set enables dynamic changes to Virtual processor states in Windows
> +guests effectively allowing vCPU hotplug.
> +
>  =item B<defaults>
> 
>  This is a special value that enables the default set of groups, which
>  is currently the B<base>, B<freq>, B<time_ref_count>, B<apic_assist>,
> -B<crash_ctl>, B<stimer> and B<no_vp_limit> groups.
> +B<crash_ctl>, B<stimer>, B<no_vp_limit> and B<cpu_hotplug> groups.
> 
>  =item B<all>
> 
> diff --git a/tools/include/libxl.h b/tools/include/libxl.h
> index be1e288..7c7c541 100644
> --- a/tools/include/libxl.h
> +++ b/tools/include/libxl.h
> @@ -458,6 +458,12 @@
>  #define LIBXL_HAVE_VIRIDIAN_NO_VP_LIMIT 1
> 
>  /*
> + * LIBXL_HAVE_VIRIDIAN_CPU_HOTPLUG indicates that the 'cpu_hotplug' value
> + * is present in the viridian enlightenment enumeration.
> + */
> +#define LIBXL_HAVE_VIRIDIAN_CPU_HOTPLUG 1
> +
> +/*
>   * LIBXL_HAVE_DEVICE_PCI_LIST_FREE indicates that the
>   * libxl_device_pci_list_free() function is defined.
>   */
> diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl
> index 8502b29..00a8e68 100644
> --- a/tools/libs/light/libxl_types.idl
> +++ b/tools/libs/light/libxl_types.idl
> @@ -240,6 +240,7 @@ libxl_viridian_enlightenment = Enumeration("viridian_enlightenment", [
>      (9, "hcall_ipi"),
>      (10, "ex_processor_masks"),
>      (11, "no_vp_limit"),
> +    (12, "cpu_hotplug"),
>      ])
> 
>  libxl_hdtype = Enumeration("hdtype", [
> diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c
> index 5c4c194..91a9fc7 100644
> --- a/tools/libs/light/libxl_x86.c
> +++ b/tools/libs/light/libxl_x86.c
> @@ -310,6 +310,7 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,
>          libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_APIC_ASSIST);
>          libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CRASH_CTL);
>          libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_NO_VP_LIMIT);
> +        libxl_bitmap_set(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CPU_HOTPLUG);
>      }
> 
>      libxl_for_each_set_bit(v, info->u.hvm.viridian_enable) {
> @@ -373,6 +374,9 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid,
>      if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_NO_VP_LIMIT))
>          mask |= HVMPV_no_vp_limit;
> 
> +    if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_CPU_HOTPLUG))
> +        mask |= HVMPV_cpu_hotplug;
> +
>      if (mask != 0 &&
>          xc_hvm_param_set(CTX->xch,
>                           domid,
> diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c
> index ae1ea86..b906f7b 100644
> --- a/xen/arch/x86/hvm/viridian/viridian.c
> +++ b/xen/arch/x86/hvm/viridian/viridian.c
> @@ -76,6 +76,7 @@ typedef union _HV_CRASH_CTL_REG_CONTENTS
>  } HV_CRASH_CTL_REG_CONTENTS;
> 
>  /* Viridian CPUID leaf 3, Hypervisor Feature Indication */
> +#define CPUID3D_CPU_DYNAMIC_PARTITIONING (1 << 3)
>  #define CPUID3D_CRASH_MSRS (1 << 10)
>  #define CPUID3D_SINT_POLLING (1 << 17)
> 
> @@ -179,8 +180,10 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf,
>          res->a = u.lo;
>          res->b = u.hi;
> 
> +        if ( viridian_feature_mask(d) & HVMPV_cpu_hotplug )
> +           res->d = CPUID3D_CPU_DYNAMIC_PARTITIONING;
>          if ( viridian_feature_mask(d) & HVMPV_crash_ctl )
> -            res->d = CPUID3D_CRASH_MSRS;
> +            res->d |= CPUID3D_CRASH_MSRS;
>          if ( viridian_feature_mask(d) & HVMPV_synic )
>              res->d |= CPUID3D_SINT_POLLING;
> 
> diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
> index 805f4ca..c9d6e70 100644
> --- a/xen/include/public/hvm/params.h
> +++ b/xen/include/public/hvm/params.h
> @@ -172,6 +172,10 @@
>  #define _HVMPV_no_vp_limit 11
>  #define HVMPV_no_vp_limit (1 << _HVMPV_no_vp_limit)
> 
> +/* Enable vCPU hotplug */
> +#define _HVMPV_cpu_hotplug 12
> +#define HVMPV_cpu_hotplug (1 << _HVMPV_cpu_hotplug)
> +
>  #define HVMPV_feature_mask \
>          (HVMPV_base_freq | \
>           HVMPV_no_freq | \
> @@ -184,7 +188,8 @@
>           HVMPV_stimer | \
>           HVMPV_hcall_ipi | \
>           HVMPV_ex_processor_masks | \
> -         HVMPV_no_vp_limit)
> +         HVMPV_no_vp_limit | \
> +         HVMPV_cpu_hotplug)
> 
>  #endif
> 
> --
> 2.7.4

Ping: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[Jan Beulich]

On 25.01.2021 16:37, Igor Druzhinin wrote:
> On 12/01/2021 04:17, Igor Druzhinin wrote:
>> TLFS 7.8.1 stipulates that "a virtual processor index must be less than
>> the maximum number of virtual processors per partition" that "can be obtained
>> through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing
>> the Microsoft Hypervisor Interface" defines that starting from Windows Server
>> 2012, which allowed more than 64 CPUs to be brought up, this leaf can now
>> contain a value -1 basically assuming the hypervisor has no restriction while
>> 0 (that we currently expose) means the default restriction is still present.
>>
>> Along with the previous changes exposing ExProcessorMasks this allows a recent
>> Windows VM with Viridian extension enabled to have more than 64 vCPUs without
>> going into BSOD in some cases.
>>
>> Since we didn't expose the leaf before and to keep CPUID data consistent for
>> incoming streams from previous Xen versions - let's keep it behind an option.
>>
>> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
>> ---
> 
> ping? Paul?

Paul - I saw a reply by you on patch 2, but not on this one.

libxl maintainers - both patches would want to be acked by you.

Jan

RE: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[Paul Durrant]

> -----Original Message-----
> From: Igor Druzhinin <igor.druzhinin@citrix.com>
> Sent: 12 January 2021 04:17
> To: xen-devel@lists.xenproject.org
> Cc: iwj@xenproject.org; wl@xen.org; andrew.cooper3@citrix.com; george.dunlap@citrix.com;
> jbeulich@suse.com; julien@xen.org; sstabellini@kernel.org; anthony.perard@citrix.com; paul@xen.org;
> roger.pau@citrix.com; Igor Druzhinin <igor.druzhinin@citrix.com>
> Subject: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition
> 
> TLFS 7.8.1 stipulates that "a virtual processor index must be less than
> the maximum number of virtual processors per partition" that "can be obtained
> through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing
> the Microsoft Hypervisor Interface" defines that starting from Windows Server
> 2012, which allowed more than 64 CPUs to be brought up, this leaf can now
> contain a value -1 basically assuming the hypervisor has no restriction while
> 0 (that we currently expose) means the default restriction is still present.
> 
> Along with the previous changes exposing ExProcessorMasks this allows a recent
> Windows VM with Viridian extension enabled to have more than 64 vCPUs without
> going into BSOD in some cases.
> 
> Since we didn't expose the leaf before and to keep CPUID data consistent for
> incoming streams from previous Xen versions - let's keep it behind an option.
> 
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>

Reviewed-by: Paul Durrant <paul@xen.org>

RE: Ping: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[Paul Durrant]

> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> Sent: 29 January 2021 10:31
> To: paul@xen.org; iwj@xenproject.org; wl@xen.org; anthony.perard@citrix.com
> Cc: andrew.cooper3@citrix.com; george.dunlap@citrix.com; julien@xen.org; sstabellini@kernel.org;
> roger.pau@citrix.com; xen-devel@lists.xenproject.org; Igor Druzhinin <igor.druzhinin@citrix.com>
> Subject: Ping: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition
> 
> On 25.01.2021 16:37, Igor Druzhinin wrote:
> > On 12/01/2021 04:17, Igor Druzhinin wrote:
> >> TLFS 7.8.1 stipulates that "a virtual processor index must be less than
> >> the maximum number of virtual processors per partition" that "can be obtained
> >> through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing
> >> the Microsoft Hypervisor Interface" defines that starting from Windows Server
> >> 2012, which allowed more than 64 CPUs to be brought up, this leaf can now
> >> contain a value -1 basically assuming the hypervisor has no restriction while
> >> 0 (that we currently expose) means the default restriction is still present.
> >>
> >> Along with the previous changes exposing ExProcessorMasks this allows a recent
> >> Windows VM with Viridian extension enabled to have more than 64 vCPUs without
> >> going into BSOD in some cases.
> >>
> >> Since we didn't expose the leaf before and to keep CPUID data consistent for
> >> incoming streams from previous Xen versions - let's keep it behind an option.
> >>
> >> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
> >> ---
> >
> > ping? Paul?
> 
> Paul - I saw a reply by you on patch 2, but not on this one.
> 

Oh sorry, yes. Done now.

  Paul

> libxl maintainers - both patches would want to be acked by you.
> 
> Jan

Re: [PATCH v2 1/2] viridian: remove implicit limit of 64 VPs per partition

[Anthony PERARD]

On Tue, Jan 12, 2021 at 04:17:27AM +0000, Igor Druzhinin wrote:
> TLFS 7.8.1 stipulates that "a virtual processor index must be less than
> the maximum number of virtual processors per partition" that "can be obtained
> through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing
> the Microsoft Hypervisor Interface" defines that starting from Windows Server
> 2012, which allowed more than 64 CPUs to be brought up, this leaf can now
> contain a value -1 basically assuming the hypervisor has no restriction while
> 0 (that we currently expose) means the default restriction is still present.
> 
> Along with the previous changes exposing ExProcessorMasks this allows a recent
> Windows VM with Viridian extension enabled to have more than 64 vCPUs without
> going into BSOD in some cases.
> 
> Since we didn't expose the leaf before and to keep CPUID data consistent for
> incoming streams from previous Xen versions - let's keep it behind an option.
> 
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v2 2/2] viridian: allow vCPU hotplug for Windows VMs

[Anthony PERARD]

On Tue, Jan 12, 2021 at 04:17:28AM +0000, Igor Druzhinin wrote:
> If Viridian extensions are enabled, Windows wouldn't currently allow
> a hotplugged vCPU to be brought up dynamically. We need to expose a special
> bit to let the guest know we allow it. Hide it behind an option to stay
> on the safe side regarding compatibility with existing guests but
> nevertheless set the option on by default.
> 
> Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD