xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
@ 2016-03-10  5:27 Juergen Gross
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
                   ` (3 more replies)
  0 siblings, 4 replies; 27+ messages in thread
From: Juergen Gross @ 2016-03-10  5:27 UTC (permalink / raw)
  To: xen-devel
  Cc: Juergen Gross, Wei Liu, Dario Faggioli, Ian Jackson, Stefano Stabellini

Some hardware (e.g. Dell studio 1555 laptops) require SMIs to be
called on physical cpu 0 only. Linux drivers like dcdbas or i8k try
to achieve this by pinning the running thread to cpu 0, but in Dom0
this is not enough: the vcpu must be pinned to physical cpu 0 via
Xen, too.

This patch series enhances xen tools to cope with new reactions of
the hypervisor due to the new feature.

Changes in V4:
- removed patches 1-3 (hypervisor patches) as already accepted
- patch 1 (was 4): minor code modifications as suggested by Dario Faggioli
- patch 3 (was 6): change "force" variable to bool as suggested by 
  Dario Faggioli
- patch 3 (was 6): add #define to libxl indicating availability of
  libxl_set_vcpuaffinity_force() as requested by Wei Liu
- patch 3 (was 6): add force option to xl man page as requested by 
  Dario Faggioli

Changes in V3:
- removed old patch 1 as already accepted
- new patch 1 to correct error in cpupool_unassign_cpu_helper()
- patch 2: adjust coding style as requested by Jan Beulich
- patch 2: rename pin_temp to pin_override as requested by Jan Beulich and
  suggested by David Vrabel
- patch 2: change return type of do_pin_temp() to int as requested by
  Jan Beulich
- patch 2: swap error checks in do_sched_op() as requested by Jan Beulich
- patch 2: adjust comment in xen/include/public/sched.h as requested by
  Jan Beulich
- new patch 3 to add possibility in hypervisor to undo pin override via tools
- patch 4 (was 3): adjust coding style as requested by Wei Liu
- new patch 5 to  print message how to recover from xl cpupool-cpu-remove
  errors
- new patch 6 to undo pin override via xl vcpu-pin (force option)

Changes in V2:
- add patch 1 to silence messages on suspend/resume
- add patch 3 to handle EBUSY case when removing cpu from cpupool
- limit operation to hardware domain as suggested by Jan Beulich
- some style issues corrected as requested by Jan Beulich
- use fixed width types in interface as requested by Jan Beulich
- add compat layer checking as requested by Jan Beulich


Juergen Gross (3):
  libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  libxl: print message how to recover from xl cpupool-cpu-remove errors
  libxl: add force option for xl vcpu-pin

 docs/man/xl.pod.1         |  7 ++++++-
 tools/libxc/xc_cpupool.c  | 20 +++++++++++++++++++-
 tools/libxl/libxl.c       | 31 +++++++++++++++++++++++++------
 tools/libxl/libxl.h       | 10 ++++++++++
 tools/libxl/xl_cmdimpl.c  | 32 +++++++++++++++++++++++++++-----
 tools/libxl/xl_cmdtable.c |  3 ++-
 6 files changed, 89 insertions(+), 14 deletions(-)

Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
-- 
2.6.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-03-10  5:27 [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
@ 2016-03-10  5:27 ` Juergen Gross
  2016-03-10 10:26   ` Dario Faggioli
                     ` (3 more replies)
  2016-03-10  5:27 ` [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors Juergen Gross
                   ` (2 subsequent siblings)
  3 siblings, 4 replies; 27+ messages in thread
From: Juergen Gross @ 2016-03-10  5:27 UTC (permalink / raw)
  To: xen-devel; +Cc: Juergen Gross, Wei Liu, Ian Jackson, Stefano Stabellini

The hypervisor might return EBUSY when trying to remove a cpu from a
cpupool when a domain running in this cpupool has pinned a vcpu
temporarily. Do some retries in this case, perhaps the situation
cleans up.

Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V3: adjust coding style as requested by Wei Liu

V4: minor code modifications as suggested by Dario Faggioli
---
 tools/libxc/xc_cpupool.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/xc_cpupool.c b/tools/libxc/xc_cpupool.c
index c42273e..261b9c9 100644
--- a/tools/libxc/xc_cpupool.c
+++ b/tools/libxc/xc_cpupool.c
@@ -20,6 +20,7 @@
  */
 
 #include <stdarg.h>
+#include <unistd.h>
 #include "xc_private.h"
 
 static int do_sysctl_save(xc_interface *xch, struct xen_sysctl *sysctl)
@@ -137,17 +138,34 @@ int xc_cpupool_addcpu(xc_interface *xch,
     return do_sysctl_save(xch, &sysctl);
 }
 
+/*
+ * The hypervisor might return EBUSY when trying to remove a cpu from a
+ * cpupool when a domain running in this cpupool has pinned a vcpu
+ * temporarily. Do some retries in this case, perhaps the situation
+ * cleans up.
+ */
+#define NUM_RMCPU_BUSY_RETRIES 5
+
 int xc_cpupool_removecpu(xc_interface *xch,
                          uint32_t poolid,
                          int cpu)
 {
+    unsigned retries;
+    int err;
     DECLARE_SYSCTL;
 
     sysctl.cmd = XEN_SYSCTL_cpupool_op;
     sysctl.u.cpupool_op.op = XEN_SYSCTL_CPUPOOL_OP_RMCPU;
     sysctl.u.cpupool_op.cpupool_id = poolid;
     sysctl.u.cpupool_op.cpu = (cpu < 0) ? XEN_SYSCTL_CPUPOOL_PAR_ANY : cpu;
-    return do_sysctl_save(xch, &sysctl);
+    for ( retries = 0; retries < NUM_RMCPU_BUSY_RETRIES; retries++ ) {
+        err = do_sysctl_save(xch, &sysctl);
+        if ( err < 0 && errno == EBUSY )
+            sleep(1);
+        else
+            break;
+    }
+    return err;
 }
 
 int xc_cpupool_movedomain(xc_interface *xch,
-- 
2.6.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors
  2016-03-10  5:27 [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
@ 2016-03-10  5:27 ` Juergen Gross
  2016-04-14 16:06   ` Ian Jackson
  2016-03-10  5:27 ` [PATCH v4 3/3] libxl: add force option for xl vcpu-pin Juergen Gross
  2016-03-22  7:36 ` [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
  3 siblings, 1 reply; 27+ messages in thread
From: Juergen Gross @ 2016-03-10  5:27 UTC (permalink / raw)
  To: xen-devel; +Cc: Juergen Gross, Wei Liu, Ian Jackson, Stefano Stabellini

An error occurring when calling "xl cpupool-cpu-remove" might leave
the system in a state where a cpu is neither completely free nor in
a cpupool. This can easily be repaired by adding the cpu via
"xl cpupool-cpu-add" to the cpupool where it was removed from before.
Print a message telling this the user in case of an error.

Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxl/xl_cmdimpl.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 990d3c9..411473d 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -7716,8 +7716,10 @@ int main_cpupoolcpuremove(int argc, char **argv)
         goto out;
     }
 
-    if (libxl_cpupool_cpuremove_cpumap(ctx, poolid, &cpumap))
-        fprintf(stderr, "some cpus may not have been removed from %s\n", pool);
+    if (libxl_cpupool_cpuremove_cpumap(ctx, poolid, &cpumap)) {
+        fprintf(stderr, "Some cpus may have not or only partially been removed from '%s'.\n", pool);
+        fprintf(stderr, "If a cpu can't be added to another cpupool, add it to '%s' again and retry.\n", pool);
+    }
 
     rc = EXIT_SUCCESS;
 
-- 
2.6.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* [PATCH v4 3/3] libxl: add force option for xl vcpu-pin
  2016-03-10  5:27 [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
  2016-03-10  5:27 ` [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors Juergen Gross
@ 2016-03-10  5:27 ` Juergen Gross
  2016-03-10 10:25   ` Dario Faggioli
                     ` (2 more replies)
  2016-03-22  7:36 ` [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
  3 siblings, 3 replies; 27+ messages in thread
From: Juergen Gross @ 2016-03-10  5:27 UTC (permalink / raw)
  To: xen-devel; +Cc: Juergen Gross, Wei Liu, Ian Jackson, Stefano Stabellini

In order to be able to undo a vcpu pin override in case of a kernel
driver error add a flag "-f" to the "xl vcpu-pin" command forcing the
hypervisor to undo the override.

Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
---
V4: - change "force" variable to bool as suggested by Dario Faggioli
    - add #define to libxl indicating availability of
      libxl_set_vcpuaffinity_force() as requested by Wei Liu
    - add force option to xl man page as requested by Dario Faggioli
---
 docs/man/xl.pod.1         |  7 ++++++-
 tools/libxl/libxl.c       | 31 +++++++++++++++++++++++++------
 tools/libxl/libxl.h       | 10 ++++++++++
 tools/libxl/xl_cmdimpl.c  | 26 +++++++++++++++++++++++---
 tools/libxl/xl_cmdtable.c |  3 ++-
 5 files changed, 66 insertions(+), 11 deletions(-)

diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index 4279c7c..aedcaea 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -689,7 +689,7 @@ after B<vcpu-set>, go to B<SEE ALSO> section for information.
 Lists VCPU information for a specific domain.  If no domain is
 specified, VCPU information for all domains will be provided.
 
-=item B<vcpu-pin> I<domain-id> I<vcpu> I<cpus hard> I<cpus soft>
+=item B<vcpu-pin> [I<-f|--force>] I<domain-id> I<vcpu> I<cpus hard> I<cpus soft>
 
 Set hard and soft affinity for a I<vcpu> of <domain-id>. Normally VCPUs
 can float between available CPUs whenever Xen deems a different run state
@@ -716,6 +716,11 @@ leaving its hard affinity untouched. On the othe hand:
 will set both hard and soft affinity, the former to pCPUs 3 and 4, the
 latter to pCPUs 6,7,8, and 9.
 
+Specifying I<-f> or I<--force> will remove a temporary pinning done by the
+operating system (normally this should be done by the operating system).
+In case a temporary pinning is active for a vcpu the affinity of this vcpu
+can't be changed without this option.
+
 =item B<vm-list>
 
 Prints information about guests. This list excludes information about
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 4cdc169..53f0100 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -5301,18 +5301,20 @@ err:
     return NULL;
 }
 
-int libxl_set_vcpuaffinity(libxl_ctx *ctx, uint32_t domid, uint32_t vcpuid,
-                           const libxl_bitmap *cpumap_hard,
-                           const libxl_bitmap *cpumap_soft)
+static int libxl__set_vcpuaffinity(libxl_ctx *ctx, uint32_t domid,
+                                   uint32_t vcpuid,
+                                   const libxl_bitmap *cpumap_hard,
+                                   const libxl_bitmap *cpumap_soft,
+                                   unsigned flags)
 {
     GC_INIT(ctx);
     libxl_bitmap hard, soft;
-    int rc, flags = 0;
+    int rc;
 
     libxl_bitmap_init(&hard);
     libxl_bitmap_init(&soft);
 
-    if (!cpumap_hard && !cpumap_soft) {
+    if (!cpumap_hard && !cpumap_soft && !flags) {
         rc = ERROR_INVAL;
         goto out;
     }
@@ -5327,7 +5329,7 @@ int libxl_set_vcpuaffinity(libxl_ctx *ctx, uint32_t domid, uint32_t vcpuid,
             goto out;
 
         libxl__bitmap_copy_best_effort(gc, &hard, cpumap_hard);
-        flags = XEN_VCPUAFFINITY_HARD;
+        flags |= XEN_VCPUAFFINITY_HARD;
     }
     if (cpumap_soft) {
         rc = libxl_cpu_bitmap_alloc(ctx, &soft, 0);
@@ -5378,6 +5380,23 @@ int libxl_set_vcpuaffinity(libxl_ctx *ctx, uint32_t domid, uint32_t vcpuid,
     return rc;
 }
 
+int libxl_set_vcpuaffinity(libxl_ctx *ctx, uint32_t domid, uint32_t vcpuid,
+                           const libxl_bitmap *cpumap_hard,
+                           const libxl_bitmap *cpumap_soft)
+{
+    return libxl__set_vcpuaffinity(ctx, domid, vcpuid, cpumap_hard,
+                                   cpumap_soft, 0);
+}
+
+int libxl_set_vcpuaffinity_force(libxl_ctx *ctx, uint32_t domid,
+                                 uint32_t vcpuid,
+                                 const libxl_bitmap *cpumap_hard,
+                                 const libxl_bitmap *cpumap_soft)
+{
+    return libxl__set_vcpuaffinity(ctx, domid, vcpuid, cpumap_hard,
+                                   cpumap_soft, XEN_VCPUAFFINITY_FORCE);
+}
+
 int libxl_set_vcpuaffinity_all(libxl_ctx *ctx, uint32_t domid,
                                unsigned int max_vcpus,
                                const libxl_bitmap *cpumap_hard,
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index f9e3ef5..d587e0c 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -148,6 +148,12 @@
 #define LIBXL_HAVE_VCPUINFO_SOFT_AFFINITY 1
 
 /*
+ * LIBXL_HAVE_SET_VCPUAFFINITY_FORCE indicates that the
+ * libxl_set_vcpuaffinity_force() library call is available.
+ */
+#define LIBXL_HAVE_SET_VCPUAFFINITY_FORCE 1
+
+/*
  * LIBXL_HAVE_DEVICE_DISK_DIRECT_IO_SAFE indicates that a
  * 'direct_io_safe' field (of boolean type) is present in
  * libxl_device_disk.
@@ -1715,6 +1721,10 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo);
 int libxl_set_vcpuaffinity(libxl_ctx *ctx, uint32_t domid, uint32_t vcpuid,
                            const libxl_bitmap *cpumap_hard,
                            const libxl_bitmap *cpumap_soft);
+int libxl_set_vcpuaffinity_force(libxl_ctx *ctx, uint32_t domid,
+                                 uint32_t vcpuid,
+                                 const libxl_bitmap *cpumap_hard,
+                                 const libxl_bitmap *cpumap_soft);
 int libxl_set_vcpuaffinity_all(libxl_ctx *ctx, uint32_t domid,
                                unsigned int max_vcpus,
                                const libxl_bitmap *cpumap_hard,
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 411473d..51fa487 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -5344,6 +5344,10 @@ int main_vcpulist(int argc, char **argv)
 
 int main_vcpupin(int argc, char **argv)
 {
+    static struct option opts[] = {
+        {"force", 0, 0, 'f'},
+        COMMON_LONG_OPTS
+    };
     libxl_vcpuinfo *vcpuinfo;
     libxl_bitmap cpumap_hard, cpumap_soft;;
     libxl_bitmap *soft = &cpumap_soft, *hard = &cpumap_hard;
@@ -5356,12 +5360,17 @@ int main_vcpupin(int argc, char **argv)
     const char *vcpu, *hard_str, *soft_str;
     char *endptr;
     int opt, nb_cpu, nb_vcpu, rc = EXIT_FAILURE;
+    bool force = false;
 
     libxl_bitmap_init(&cpumap_hard);
     libxl_bitmap_init(&cpumap_soft);
 
-    SWITCH_FOREACH_OPT(opt, "", NULL, "vcpu-pin", 3) {
-        /* No options */
+    SWITCH_FOREACH_OPT(opt, "f", opts, "vcpu-pin", 3) {
+    case 'f':
+        force = true;
+        break;
+    default:
+        break;
     }
 
     domid = find_domain(argv[optind]);
@@ -5376,6 +5385,10 @@ int main_vcpupin(int argc, char **argv)
             fprintf(stderr, "Error: Invalid argument %s as VCPU.\n", vcpu);
             goto out;
         }
+        if (force) {
+            fprintf(stderr, "Error: --force and 'all' as VCPU not allowed.\n");
+            goto out;
+        }
         vcpuid = -1;
     }
 
@@ -5437,7 +5450,14 @@ int main_vcpupin(int argc, char **argv)
         goto out;
     }
 
-    if (vcpuid != -1) {
+    if (force) {
+        if (libxl_set_vcpuaffinity_force(ctx, domid, vcpuid, hard, soft)) {
+            fprintf(stderr, "Could not set affinity for vcpu `%ld'.\n",
+                    vcpuid);
+            goto out;
+        }
+    }
+    else if (vcpuid != -1) {
         if (libxl_set_vcpuaffinity(ctx, domid, vcpuid, hard, soft)) {
             fprintf(stderr, "Could not set affinity for vcpu `%ld'.\n",
                     vcpuid);
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index fdc1ac6..7cc0401 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -222,7 +222,8 @@ struct cmd_spec cmd_table[] = {
     { "vcpu-pin",
       &main_vcpupin, 1, 1,
       "Set which CPUs a VCPU can use",
-      "<Domain> <VCPU|all> <Hard affinity|-|all> <Soft affinity|-|all>",
+      "[option] <Domain> <VCPU|all> <Hard affinity|-|all> <Soft affinity|-|all>",
+      "-f, --force        undo an override pinning done by the kernel",
     },
     { "vcpu-set",
       &main_vcpuset, 0, 1,
-- 
2.6.2


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 3/3] libxl: add force option for xl vcpu-pin
  2016-03-10  5:27 ` [PATCH v4 3/3] libxl: add force option for xl vcpu-pin Juergen Gross
@ 2016-03-10 10:25   ` Dario Faggioli
  2016-03-10 17:17   ` Wei Liu
  2016-04-14 16:10   ` Ian Jackson
  2 siblings, 0 replies; 27+ messages in thread
From: Dario Faggioli @ 2016-03-10 10:25 UTC (permalink / raw)
  To: Juergen Gross, xen-devel; +Cc: Ian Jackson, Wei Liu, Stefano Stabellini


[-- Attachment #1.1: Type: text/plain, Size: 788 bytes --]

On Thu, 2016-03-10 at 06:27 +0100, Juergen Gross wrote:
> In order to be able to undo a vcpu pin override in case of a kernel
> driver error add a flag "-f" to the "xl vcpu-pin" command forcing the
> hypervisor to undo the override.
> 
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>
>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
@ 2016-03-10 10:26   ` Dario Faggioli
  2016-03-10 17:16   ` Wei Liu
                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 27+ messages in thread
From: Dario Faggioli @ 2016-03-10 10:26 UTC (permalink / raw)
  To: Juergen Gross, xen-devel; +Cc: Ian Jackson, Wei Liu, Stefano Stabellini


[-- Attachment #1.1: Type: text/plain, Size: 833 bytes --]

On Thu, 2016-03-10 at 06:27 +0100, Juergen Gross wrote:
> The hypervisor might return EBUSY when trying to remove a cpu from a
> cpupool when a domain running in this cpupool has pinned a vcpu
> temporarily. Do some retries in this case, perhaps the situation
> cleans up.
> 
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>
>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
  2016-03-10 10:26   ` Dario Faggioli
@ 2016-03-10 17:16   ` Wei Liu
  2016-04-12 13:02   ` Olaf Hering
  2016-04-14 16:04   ` Ian Jackson
  3 siblings, 0 replies; 27+ messages in thread
From: Wei Liu @ 2016-03-10 17:16 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel

On Thu, Mar 10, 2016 at 06:27:12AM +0100, Juergen Gross wrote:
> The hypervisor might return EBUSY when trying to remove a cpu from a
> cpupool when a domain running in this cpupool has pinned a vcpu
> temporarily. Do some retries in this case, perhaps the situation
> cleans up.
> 
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

> ---
> V3: adjust coding style as requested by Wei Liu
> 
> V4: minor code modifications as suggested by Dario Faggioli
> ---
>  tools/libxc/xc_cpupool.c | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libxc/xc_cpupool.c b/tools/libxc/xc_cpupool.c
> index c42273e..261b9c9 100644
> --- a/tools/libxc/xc_cpupool.c
> +++ b/tools/libxc/xc_cpupool.c
> @@ -20,6 +20,7 @@
>   */
>  
>  #include <stdarg.h>
> +#include <unistd.h>
>  #include "xc_private.h"
>  
>  static int do_sysctl_save(xc_interface *xch, struct xen_sysctl *sysctl)
> @@ -137,17 +138,34 @@ int xc_cpupool_addcpu(xc_interface *xch,
>      return do_sysctl_save(xch, &sysctl);
>  }
>  
> +/*
> + * The hypervisor might return EBUSY when trying to remove a cpu from a
> + * cpupool when a domain running in this cpupool has pinned a vcpu
> + * temporarily. Do some retries in this case, perhaps the situation
> + * cleans up.
> + */
> +#define NUM_RMCPU_BUSY_RETRIES 5
> +
>  int xc_cpupool_removecpu(xc_interface *xch,
>                           uint32_t poolid,
>                           int cpu)
>  {
> +    unsigned retries;
> +    int err;
>      DECLARE_SYSCTL;
>  
>      sysctl.cmd = XEN_SYSCTL_cpupool_op;
>      sysctl.u.cpupool_op.op = XEN_SYSCTL_CPUPOOL_OP_RMCPU;
>      sysctl.u.cpupool_op.cpupool_id = poolid;
>      sysctl.u.cpupool_op.cpu = (cpu < 0) ? XEN_SYSCTL_CPUPOOL_PAR_ANY : cpu;
> -    return do_sysctl_save(xch, &sysctl);
> +    for ( retries = 0; retries < NUM_RMCPU_BUSY_RETRIES; retries++ ) {
> +        err = do_sysctl_save(xch, &sysctl);
> +        if ( err < 0 && errno == EBUSY )
> +            sleep(1);
> +        else
> +            break;
> +    }
> +    return err;
>  }
>  
>  int xc_cpupool_movedomain(xc_interface *xch,
> -- 
> 2.6.2
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 3/3] libxl: add force option for xl vcpu-pin
  2016-03-10  5:27 ` [PATCH v4 3/3] libxl: add force option for xl vcpu-pin Juergen Gross
  2016-03-10 10:25   ` Dario Faggioli
@ 2016-03-10 17:17   ` Wei Liu
  2016-04-14 16:10   ` Ian Jackson
  2 siblings, 0 replies; 27+ messages in thread
From: Wei Liu @ 2016-03-10 17:17 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Wei Liu, Stefano Stabellini, Ian Jackson, xen-devel

On Thu, Mar 10, 2016 at 06:27:14AM +0100, Juergen Gross wrote:
> In order to be able to undo a vcpu pin override in case of a kernel
> driver error add a flag "-f" to the "xl vcpu-pin" command forcing the
> hypervisor to undo the override.
> 
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-10  5:27 [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
                   ` (2 preceding siblings ...)
  2016-03-10  5:27 ` [PATCH v4 3/3] libxl: add force option for xl vcpu-pin Juergen Gross
@ 2016-03-22  7:36 ` Juergen Gross
  2016-03-24 14:03   ` Wei Liu
  3 siblings, 1 reply; 27+ messages in thread
From: Juergen Gross @ 2016-03-22  7:36 UTC (permalink / raw)
  To: xen-devel, Ian Jackson; +Cc: Dario Faggioli, Wei Liu, Stefano Stabellini

Committers,

anything missing on my side to have this series being committed?
All patches have Acked-by and Reviewed-by tags.


Juergen

On 10/03/16 06:27, Juergen Gross wrote:
> Some hardware (e.g. Dell studio 1555 laptops) require SMIs to be
> called on physical cpu 0 only. Linux drivers like dcdbas or i8k try
> to achieve this by pinning the running thread to cpu 0, but in Dom0
> this is not enough: the vcpu must be pinned to physical cpu 0 via
> Xen, too.
> 
> This patch series enhances xen tools to cope with new reactions of
> the hypervisor due to the new feature.
> 
> Changes in V4:
> - removed patches 1-3 (hypervisor patches) as already accepted
> - patch 1 (was 4): minor code modifications as suggested by Dario Faggioli
> - patch 3 (was 6): change "force" variable to bool as suggested by 
>   Dario Faggioli
> - patch 3 (was 6): add #define to libxl indicating availability of
>   libxl_set_vcpuaffinity_force() as requested by Wei Liu
> - patch 3 (was 6): add force option to xl man page as requested by 
>   Dario Faggioli
> 
> Changes in V3:
> - removed old patch 1 as already accepted
> - new patch 1 to correct error in cpupool_unassign_cpu_helper()
> - patch 2: adjust coding style as requested by Jan Beulich
> - patch 2: rename pin_temp to pin_override as requested by Jan Beulich and
>   suggested by David Vrabel
> - patch 2: change return type of do_pin_temp() to int as requested by
>   Jan Beulich
> - patch 2: swap error checks in do_sched_op() as requested by Jan Beulich
> - patch 2: adjust comment in xen/include/public/sched.h as requested by
>   Jan Beulich
> - new patch 3 to add possibility in hypervisor to undo pin override via tools
> - patch 4 (was 3): adjust coding style as requested by Wei Liu
> - new patch 5 to  print message how to recover from xl cpupool-cpu-remove
>   errors
> - new patch 6 to undo pin override via xl vcpu-pin (force option)
> 
> Changes in V2:
> - add patch 1 to silence messages on suspend/resume
> - add patch 3 to handle EBUSY case when removing cpu from cpupool
> - limit operation to hardware domain as suggested by Jan Beulich
> - some style issues corrected as requested by Jan Beulich
> - use fixed width types in interface as requested by Jan Beulich
> - add compat layer checking as requested by Jan Beulich
> 
> 
> Juergen Gross (3):
>   libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
>   libxl: print message how to recover from xl cpupool-cpu-remove errors
>   libxl: add force option for xl vcpu-pin
> 
>  docs/man/xl.pod.1         |  7 ++++++-
>  tools/libxc/xc_cpupool.c  | 20 +++++++++++++++++++-
>  tools/libxl/libxl.c       | 31 +++++++++++++++++++++++++------
>  tools/libxl/libxl.h       | 10 ++++++++++
>  tools/libxl/xl_cmdimpl.c  | 32 +++++++++++++++++++++++++++-----
>  tools/libxl/xl_cmdtable.c |  3 ++-
>  6 files changed, 89 insertions(+), 14 deletions(-)
> 
> Cc: Dario Faggioli <dario.faggioli@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-22  7:36 ` [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
@ 2016-03-24 14:03   ` Wei Liu
  2016-03-24 17:58     ` Juergen Gross
  0 siblings, 1 reply; 27+ messages in thread
From: Wei Liu @ 2016-03-24 14:03 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Wei Liu, Dario Faggioli, Stefano Stabellini, Ian Jackson, xen-devel

On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote:
> Committers,
> 
> anything missing on my side to have this series being committed?
> All patches have Acked-by and Reviewed-by tags.
> 

Can you provide a branch with all the tags folded in?

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-24 14:03   ` Wei Liu
@ 2016-03-24 17:58     ` Juergen Gross
  2016-03-24 19:41       ` Wei Liu
  0 siblings, 1 reply; 27+ messages in thread
From: Juergen Gross @ 2016-03-24 17:58 UTC (permalink / raw)
  To: Wei Liu; +Cc: Dario Faggioli, Stefano Stabellini, Ian Jackson, xen-devel

On 24/03/16 15:03, Wei Liu wrote:
> On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote:
>> Committers,
>>
>> anything missing on my side to have this series being committed?
>> All patches have Acked-by and Reviewed-by tags.
>>
> 
> Can you provide a branch with all the tags folded in?

Sure:

https://github.com/jgross1/xen.git pin

Juergen

P.S.: Everything okay? Up to now I've been using git only locally or to
      clone an existing repository. I hope I've done it correctly. :-)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-24 17:58     ` Juergen Gross
@ 2016-03-24 19:41       ` Wei Liu
  2016-03-30 10:28         ` Juergen Gross
  0 siblings, 1 reply; 27+ messages in thread
From: Wei Liu @ 2016-03-24 19:41 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Ian Jackson, Dario Faggioli, Stefano Stabellini, Wei Liu, xen-devel

On Thu, Mar 24, 2016 at 06:58:44PM +0100, Juergen Gross wrote:
> On 24/03/16 15:03, Wei Liu wrote:
> > On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote:
> >> Committers,
> >>
> >> anything missing on my side to have this series being committed?
> >> All patches have Acked-by and Reviewed-by tags.
> >>
> > 
> > Can you provide a branch with all the tags folded in?
> 
> Sure:
> 
> https://github.com/jgross1/xen.git pin
> 
> Juergen
> 
> P.S.: Everything okay? Up to now I've been using git only locally or to
>       clone an existing repository. I hope I've done it correctly. :-)

Thanks.  The branch looks OK to me.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-24 19:41       ` Wei Liu
@ 2016-03-30 10:28         ` Juergen Gross
  2016-03-30 14:23           ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 27+ messages in thread
From: Juergen Gross @ 2016-03-30 10:28 UTC (permalink / raw)
  To: Wei Liu; +Cc: Dario Faggioli, xen-devel, Ian Jackson, Stefano Stabellini

On 24/03/16 20:41, Wei Liu wrote:
> On Thu, Mar 24, 2016 at 06:58:44PM +0100, Juergen Gross wrote:
>> On 24/03/16 15:03, Wei Liu wrote:
>>> On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote:
>>>> Committers,
>>>>
>>>> anything missing on my side to have this series being committed?
>>>> All patches have Acked-by and Reviewed-by tags.
>>>>
>>>
>>> Can you provide a branch with all the tags folded in?
>>
>> Sure:
>>
>> https://github.com/jgross1/xen.git pin
>>
>> Juergen
>>
>> P.S.: Everything okay? Up to now I've been using git only locally or to
>>       clone an existing repository. I hope I've done it correctly. :-)
> 
> Thanks.  The branch looks OK to me.

Still not in the repository...


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-30 10:28         ` Juergen Gross
@ 2016-03-30 14:23           ` Konrad Rzeszutek Wilk
  2016-04-14 16:11             ` Ian Jackson
  0 siblings, 1 reply; 27+ messages in thread
From: Konrad Rzeszutek Wilk @ 2016-03-30 14:23 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Ian Jackson, Dario Faggioli, Stefano Stabellini, Wei Liu, xen-devel

On Wed, Mar 30, 2016 at 12:28:01PM +0200, Juergen Gross wrote:
> On 24/03/16 20:41, Wei Liu wrote:
> > On Thu, Mar 24, 2016 at 06:58:44PM +0100, Juergen Gross wrote:
> >> On 24/03/16 15:03, Wei Liu wrote:
> >>> On Tue, Mar 22, 2016 at 08:36:21AM +0100, Juergen Gross wrote:
> >>>> Committers,
> >>>>
> >>>> anything missing on my side to have this series being committed?
> >>>> All patches have Acked-by and Reviewed-by tags.
> >>>>
> >>>
> >>> Can you provide a branch with all the tags folded in?
> >>
> >> Sure:
> >>
> >> https://github.com/jgross1/xen.git pin
> >>
> >> Juergen
> >>
> >> P.S.: Everything okay? Up to now I've been using git only locally or to
> >>       clone an existing repository. I hope I've done it correctly. :-)
> > 
> > Thanks.  The branch looks OK to me.
> 
> Still not in the repository...

Applied.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
  2016-03-10 10:26   ` Dario Faggioli
  2016-03-10 17:16   ` Wei Liu
@ 2016-04-12 13:02   ` Olaf Hering
  2016-04-12 13:45     ` Juergen Gross
  2016-04-14 16:04   ` Ian Jackson
  3 siblings, 1 reply; 27+ messages in thread
From: Olaf Hering @ 2016-04-12 13:02 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Ian Jackson, Stefano Stabellini, Wei Liu, xen-devel

On Thu, Mar 10, Juergen Gross wrote:

> +#define NUM_RMCPU_BUSY_RETRIES 5
> +
>  int xc_cpupool_removecpu(xc_interface *xch,
>                           uint32_t poolid,
>                           int cpu)
>  {
> +    unsigned retries;
> +    int err;
>      DECLARE_SYSCTL;
>  
>      sysctl.cmd = XEN_SYSCTL_cpupool_op;
>      sysctl.u.cpupool_op.op = XEN_SYSCTL_CPUPOOL_OP_RMCPU;
>      sysctl.u.cpupool_op.cpupool_id = poolid;
>      sysctl.u.cpupool_op.cpu = (cpu < 0) ? XEN_SYSCTL_CPUPOOL_PAR_ANY : cpu;
> -    return do_sysctl_save(xch, &sysctl);
> +    for ( retries = 0; retries < NUM_RMCPU_BUSY_RETRIES; retries++ ) {
> +        err = do_sysctl_save(xch, &sysctl);
> +        if ( err < 0 && errno == EBUSY )
> +            sleep(1);
> +        else
> +            break;
> +    }
> +    return err;

This may fail with gcc-4.8, at least with -Og in 13.1:

[  105s] xc_cpupool.c: In function 'xc_cpupool_removecpu':
[  105s] xc_cpupool.c:168:5: error: 'err' may be used uninitialized in this function [-Werror=maybe-uninitialized]
[  105s]      return err;
[  105s]      ^


Olaf

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-04-12 13:02   ` Olaf Hering
@ 2016-04-12 13:45     ` Juergen Gross
  2016-04-12 13:50       ` Olaf Hering
  2016-04-12 13:57       ` Wei Liu
  0 siblings, 2 replies; 27+ messages in thread
From: Juergen Gross @ 2016-04-12 13:45 UTC (permalink / raw)
  To: Olaf Hering; +Cc: Ian Jackson, Stefano Stabellini, Wei Liu, xen-devel

On 12/04/16 15:02, Olaf Hering wrote:
> On Thu, Mar 10, Juergen Gross wrote:
> 
>> +#define NUM_RMCPU_BUSY_RETRIES 5
>> +
>>  int xc_cpupool_removecpu(xc_interface *xch,
>>                           uint32_t poolid,
>>                           int cpu)
>>  {
>> +    unsigned retries;
>> +    int err;
>>      DECLARE_SYSCTL;
>>  
>>      sysctl.cmd = XEN_SYSCTL_cpupool_op;
>>      sysctl.u.cpupool_op.op = XEN_SYSCTL_CPUPOOL_OP_RMCPU;
>>      sysctl.u.cpupool_op.cpupool_id = poolid;
>>      sysctl.u.cpupool_op.cpu = (cpu < 0) ? XEN_SYSCTL_CPUPOOL_PAR_ANY : cpu;
>> -    return do_sysctl_save(xch, &sysctl);
>> +    for ( retries = 0; retries < NUM_RMCPU_BUSY_RETRIES; retries++ ) {
>> +        err = do_sysctl_save(xch, &sysctl);
>> +        if ( err < 0 && errno == EBUSY )
>> +            sleep(1);
>> +        else
>> +            break;
>> +    }
>> +    return err;
> 
> This may fail with gcc-4.8, at least with -Og in 13.1:
> 
> [  105s] xc_cpupool.c: In function 'xc_cpupool_removecpu':
> [  105s] xc_cpupool.c:168:5: error: 'err' may be used uninitialized in this function [-Werror=maybe-uninitialized]
> [  105s]      return err;
> [  105s]      ^

IMO this is a compiler bug. The compiler could detect easily that err
can't be uninitialized at the return statement (e.g. via loop
unrolling).

I can do a patch, of course. The question is whether I should. :-)


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-04-12 13:45     ` Juergen Gross
@ 2016-04-12 13:50       ` Olaf Hering
  2016-04-12 13:57       ` Wei Liu
  1 sibling, 0 replies; 27+ messages in thread
From: Olaf Hering @ 2016-04-12 13:50 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Ian Jackson, Stefano Stabellini, Wei Liu, xen-devel

On Tue, Apr 12, Juergen Gross wrote:

> IMO this is a compiler bug. The compiler could detect easily that err
> can't be uninitialized at the return statement (e.g. via loop
> unrolling).

There is another place like that in blktap2. There was some discussion
about -Og usage in tools, not sure if it will replace -O0 one day.
I will send a patch to fix both cases so that xen is prepared for such
switch.

Olaf

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-04-12 13:45     ` Juergen Gross
  2016-04-12 13:50       ` Olaf Hering
@ 2016-04-12 13:57       ` Wei Liu
  1 sibling, 0 replies; 27+ messages in thread
From: Wei Liu @ 2016-04-12 13:57 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Ian Jackson, Olaf Hering, Stefano Stabellini, Wei Liu, xen-devel

On Tue, Apr 12, 2016 at 03:45:06PM +0200, Juergen Gross wrote:
> On 12/04/16 15:02, Olaf Hering wrote:
> > On Thu, Mar 10, Juergen Gross wrote:
> > 
> >> +#define NUM_RMCPU_BUSY_RETRIES 5
> >> +
> >>  int xc_cpupool_removecpu(xc_interface *xch,
> >>                           uint32_t poolid,
> >>                           int cpu)
> >>  {
> >> +    unsigned retries;
> >> +    int err;
> >>      DECLARE_SYSCTL;
> >>  
> >>      sysctl.cmd = XEN_SYSCTL_cpupool_op;
> >>      sysctl.u.cpupool_op.op = XEN_SYSCTL_CPUPOOL_OP_RMCPU;
> >>      sysctl.u.cpupool_op.cpupool_id = poolid;
> >>      sysctl.u.cpupool_op.cpu = (cpu < 0) ? XEN_SYSCTL_CPUPOOL_PAR_ANY : cpu;
> >> -    return do_sysctl_save(xch, &sysctl);
> >> +    for ( retries = 0; retries < NUM_RMCPU_BUSY_RETRIES; retries++ ) {
> >> +        err = do_sysctl_save(xch, &sysctl);
> >> +        if ( err < 0 && errno == EBUSY )
> >> +            sleep(1);
> >> +        else
> >> +            break;
> >> +    }
> >> +    return err;
> > 
> > This may fail with gcc-4.8, at least with -Og in 13.1:
> > 
> > [  105s] xc_cpupool.c: In function 'xc_cpupool_removecpu':
> > [  105s] xc_cpupool.c:168:5: error: 'err' may be used uninitialized in this function [-Werror=maybe-uninitialized]
> > [  105s]      return err;
> > [  105s]      ^
> 
> IMO this is a compiler bug. The compiler could detect easily that err
> can't be uninitialized at the return statement (e.g. via loop
> unrolling).
> 
> I can do a patch, of course. The question is whether I should. :-)
> 

You should -- and document that it is to make buggy compiler happy.
We've done this before.

Wei.

> 
> Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
  2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
                     ` (2 preceding siblings ...)
  2016-04-12 13:02   ` Olaf Hering
@ 2016-04-14 16:04   ` Ian Jackson
  3 siblings, 0 replies; 27+ messages in thread
From: Ian Jackson @ 2016-04-14 16:04 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Stefano Stabellini, Wei Liu, xen-devel

Juergen Gross writes ("[PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case"):
> The hypervisor might return EBUSY when trying to remove a cpu from a
> cpupool when a domain running in this cpupool has pinned a vcpu
> temporarily. Do some retries in this case, perhaps the situation
> cleans up.

Unfortunately this patch is not acceptable because:

> +    for ( retries = 0; retries < NUM_RMCPU_BUSY_RETRIES; retries++ ) {
> +        err = do_sysctl_save(xch, &sysctl);
> +        if ( err < 0 && errno == EBUSY )
> +            sleep(1);

libxc may be called from within long-running daemons such as libvirt.

In such a system this sleep would enable an uncooperative or buggy
guest to block all toolstack operations for an extended period.

Sorry,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors
  2016-03-10  5:27 ` [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors Juergen Gross
@ 2016-04-14 16:06   ` Ian Jackson
  2016-04-14 17:10     ` Dario Faggioli
  0 siblings, 1 reply; 27+ messages in thread
From: Ian Jackson @ 2016-04-14 16:06 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Stefano Stabellini, Wei Liu, xen-devel

Juergen Gross writes ("[PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors"):
> An error occurring when calling "xl cpupool-cpu-remove" might leave
> the system in a state where a cpu is neither completely free nor in
> a cpupool.

Surely this is a bug.  Can it not be avoided ?

> This can easily be repaired by adding the cpu via
> "xl cpupool-cpu-add" to the cpupool where it was removed from before.
> Print a message telling this the user in case of an error.
...
> -    if (libxl_cpupool_cpuremove_cpumap(ctx, poolid, &cpumap))
> -        fprintf(stderr, "some cpus may not have been removed from %s\n", pool);
> +    if (libxl_cpupool_cpuremove_cpumap(ctx, poolid, &cpumap)) {
> +        fprintf(stderr, "Some cpus may have not or only partially been removed from '%s'.\n", pool);
> +        fprintf(stderr, "If a cpu can't be added to another cpupool, add it to '%s' again and retry.\n", pool);
> +    }

If it can't be avoided then I guess this will have to do but I remain
to be convinced.

In any case, while you're editing this code, can you please wrap your
long lines.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 3/3] libxl: add force option for xl vcpu-pin
  2016-03-10  5:27 ` [PATCH v4 3/3] libxl: add force option for xl vcpu-pin Juergen Gross
  2016-03-10 10:25   ` Dario Faggioli
  2016-03-10 17:17   ` Wei Liu
@ 2016-04-14 16:10   ` Ian Jackson
  2016-04-14 17:18     ` Dario Faggioli
  2 siblings, 1 reply; 27+ messages in thread
From: Ian Jackson @ 2016-04-14 16:10 UTC (permalink / raw)
  To: Juergen Gross; +Cc: Stefano Stabellini, Wei Liu, xen-devel

Juergen Gross writes ("[PATCH v4 3/3] libxl: add force option for xl vcpu-pin"):
> In order to be able to undo a vcpu pin override in case of a kernel
> driver error add a flag "-f" to the "xl vcpu-pin" command forcing the
> hypervisor to undo the override.
...
> +Specifying I<-f> or I<--force> will remove a temporary pinning done by the
> +operating system (normally this should be done by the operating system).
> +In case a temporary pinning is active for a vcpu the affinity of this vcpu
> +can't be changed without this option.

This documentation needs to confirm that this can only happen to dom0,
or other domains which have been granted hardware access.

Assuming that this is actually true.  If not then surely it should be.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-03-30 14:23           ` Konrad Rzeszutek Wilk
@ 2016-04-14 16:11             ` Ian Jackson
  2016-04-14 19:15               ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 27+ messages in thread
From: Ian Jackson @ 2016-04-14 16:11 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk
  Cc: Juergen Gross, Dario Faggioli, Stefano Stabellini, Wei Liu, xen-devel

Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu"):
> Applied.

Damn, I see I am too late with my review.

I will propose to revert some of this. :-/.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors
  2016-04-14 16:06   ` Ian Jackson
@ 2016-04-14 17:10     ` Dario Faggioli
  2016-04-14 17:22       ` Ian Jackson
  0 siblings, 1 reply; 27+ messages in thread
From: Dario Faggioli @ 2016-04-14 17:10 UTC (permalink / raw)
  To: Ian Jackson, Juergen Gross; +Cc: Wei Liu, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 2187 bytes --]

On Thu, 2016-04-14 at 17:06 +0100, Ian Jackson wrote:
> Juergen Gross writes ("[PATCH v4 2/3] libxl: print message how to
> recover from xl cpupool-cpu-remove errors"):
> > 
> > An error occurring when calling "xl cpupool-cpu-remove" might leave
> > the system in a state where a cpu is neither completely free nor in
> > a cpupool.
> Surely this is a bug.  Can it not be avoided ?
> 
Not easily (and in general not with any patch that I'd consider
appropriate for this phase of the release process), as it depends on
transient situations in the hypervisor, such as lock contention on
scheduling data structures.

> > This can easily be repaired by adding the cpu via
> > "xl cpupool-cpu-add" to the cpupool where it was removed from
> > before.
> > Print a message telling this the user in case of an error.
> ...
> > 
> > -    if (libxl_cpupool_cpuremove_cpumap(ctx, poolid, &cpumap))
> > -        fprintf(stderr, "some cpus may not have been removed from
> > %s\n", pool);
> > +    if (libxl_cpupool_cpuremove_cpumap(ctx, poolid, &cpumap)) {
> > +        fprintf(stderr, "Some cpus may have not or only partially
> > been removed from '%s'.\n", pool);
> > +        fprintf(stderr, "If a cpu can't be added to another
> > cpupool, add it to '%s' again and retry.\n", pool);
> > +    }
> If it can't be avoided then I guess this will have to do but I remain
> to be convinced.
> 
And in fact, it's not something that is introduced by this series,
which is, with this patch, just taking the chance to document things
better (although, this series introduces one more way for the issue to
occur).

Doing some retries at levels lower than this would minimize the chance
of the user actually getting to deal with the problem. For eaxmple,
what's done in libxc... but as you pointed out, that introduces other
problems, so I'm not sure. :-/

Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 3/3] libxl: add force option for xl vcpu-pin
  2016-04-14 16:10   ` Ian Jackson
@ 2016-04-14 17:18     ` Dario Faggioli
  0 siblings, 0 replies; 27+ messages in thread
From: Dario Faggioli @ 2016-04-14 17:18 UTC (permalink / raw)
  To: Ian Jackson, Juergen Gross; +Cc: Wei Liu, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1620 bytes --]

On Thu, 2016-04-14 at 17:10 +0100, Ian Jackson wrote:
> Juergen Gross writes ("[PATCH v4 3/3] libxl: add force option for xl
> vcpu-pin"):
> > 
> > 
> > +Specifying I<-f> or I<--force> will remove a temporary pinning
> > done by the
> > +operating system (normally this should be done by the operating
> > system).
> > +In case a temporary pinning is active for a vcpu the affinity of
> > this vcpu
> > +can't be changed without this option.
> This documentation needs to confirm that this can only happen to
> dom0,
> or other domains which have been granted hardware access.
> 
> Assuming that this is actually true.  If not then surely it should
> be.
> 
AFAIUI, it's like that already.

From commit 8fa0fca9f3fdaac1aead9cf61d678a0d8cce02e2:

+    case SCHEDOP_pin_override:
+    {
+        struct sched_pin_override sched_pin_override;
+
+        ret = -EPERM;
+        if ( !is_hardware_domain(current->domain) )
+            break;
+
+        ret = -EFAULT;
+        if ( copy_from_guest(&sched_pin_override, arg, 1) )
+            break;
+
+        ret = vcpu_pin_override(current, sched_pin_override.pcpu);
+
+        break;
+    }

Better saying that in the documentation makes sense, I guess.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors
  2016-04-14 17:10     ` Dario Faggioli
@ 2016-04-14 17:22       ` Ian Jackson
  2016-04-14 17:27         ` Dario Faggioli
  0 siblings, 1 reply; 27+ messages in thread
From: Ian Jackson @ 2016-04-14 17:22 UTC (permalink / raw)
  To: Dario Faggioli; +Cc: Juergen Gross, Wei Liu, xen-devel

Dario Faggioli writes ("Re: [Xen-devel] [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors"):
> Not easily (and in general not with any patch that I'd consider
> appropriate for this phase of the release process), as it depends on
> transient situations in the hypervisor, such as lock contention on
> scheduling data structures.

I think it would be best to carry on this conversation in response to
my fixup mini-series, which contains a docs patch full of `xxx' :-).

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors
  2016-04-14 17:22       ` Ian Jackson
@ 2016-04-14 17:27         ` Dario Faggioli
  0 siblings, 0 replies; 27+ messages in thread
From: Dario Faggioli @ 2016-04-14 17:27 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Juergen Gross, Wei Liu, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 948 bytes --]

On Thu, 2016-04-14 at 18:22 +0100, Ian Jackson wrote:
> Dario Faggioli writes ("Re: [Xen-devel] [PATCH v4 2/3] libxl: print
> message how to recover from xl cpupool-cpu-remove errors"):
> > 
> > Not easily (and in general not with any patch that I'd consider
> > appropriate for this phase of the release process), as it depends
> > on
> > transient situations in the hypervisor, such as lock contention on
> > scheduling data structures.
> I think it would be best to carry on this conversation in response to
> my fixup mini-series, which contains a docs patch full of `xxx' :-).
> 
Yep, sorry, I've just saw it. I'll reply better tomorrow.

Thanks and Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

* Re: [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu
  2016-04-14 16:11             ` Ian Jackson
@ 2016-04-14 19:15               ` Konrad Rzeszutek Wilk
  0 siblings, 0 replies; 27+ messages in thread
From: Konrad Rzeszutek Wilk @ 2016-04-14 19:15 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Juergen Gross, Dario Faggioli, Stefano Stabellini, Wei Liu, xen-devel

On Thu, Apr 14, 2016 at 05:11:05PM +0100, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu"):
> > Applied.
> 
> Damn, I see I am too late with my review.
> 
> I will propose to revert some of this. :-/.

It is the season :-)
> 
> Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2016-04-14 19:15 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-10  5:27 [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
2016-03-10  5:27 ` [PATCH v4 1/3] libxc: do some retries in xc_cpupool_removecpu() for EBUSY case Juergen Gross
2016-03-10 10:26   ` Dario Faggioli
2016-03-10 17:16   ` Wei Liu
2016-04-12 13:02   ` Olaf Hering
2016-04-12 13:45     ` Juergen Gross
2016-04-12 13:50       ` Olaf Hering
2016-04-12 13:57       ` Wei Liu
2016-04-14 16:04   ` Ian Jackson
2016-03-10  5:27 ` [PATCH v4 2/3] libxl: print message how to recover from xl cpupool-cpu-remove errors Juergen Gross
2016-04-14 16:06   ` Ian Jackson
2016-04-14 17:10     ` Dario Faggioli
2016-04-14 17:22       ` Ian Jackson
2016-04-14 17:27         ` Dario Faggioli
2016-03-10  5:27 ` [PATCH v4 3/3] libxl: add force option for xl vcpu-pin Juergen Gross
2016-03-10 10:25   ` Dario Faggioli
2016-03-10 17:17   ` Wei Liu
2016-04-14 16:10   ` Ian Jackson
2016-04-14 17:18     ` Dario Faggioli
2016-03-22  7:36 ` [PATCH v4 0/3] add hypercall option to temporarily pin a vcpu Juergen Gross
2016-03-24 14:03   ` Wei Liu
2016-03-24 17:58     ` Juergen Gross
2016-03-24 19:41       ` Wei Liu
2016-03-30 10:28         ` Juergen Gross
2016-03-30 14:23           ` Konrad Rzeszutek Wilk
2016-04-14 16:11             ` Ian Jackson
2016-04-14 19:15               ` Konrad Rzeszutek Wilk

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).