xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
* [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory
@ 2019-10-04 15:17 Ian Jackson
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800 Ian Jackson
                   ` (6 more replies)
  0 siblings, 7 replies; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Anthony PERARD, Jürgen Groß, Ian Jackson, Wei Liu

libxl_get_required_shadow_memory has always been anomalous.  libxl
ought to default these things itself.  Recently, another analogous
setting, iommu_memkb, was introduced, along with another function
along the same lines.

This API is not very good.  Fixing it was not entirely trivial.
(Thanks to Paul for explaining some of the difficulties and Anthony
for in-principle review of my proposal.)

Ian Jackson (6):
  libxl: Offer API versions 0x040700 and 0x040800
  xl: Pass libxl_domain_config to freemem(), instead of b_info
  libxl: libxl__domain_config_setdefault: New function
  libxl: libxl_domain_need_memory: Make it take a domain_config
  libxl: Move shadow_memkb and iommu_memkb defaulting into libxl
  libxl: Remove/deprecate libxl_get_required_*_memory from the API

 tools/libxl/libxl.h          | 24 ++++++++++-
 tools/libxl/libxl_create.c   | 95 ++++++++++++++++++++++++++++++++++++++------
 tools/libxl/libxl_dom.c      |  7 ++--
 tools/libxl/libxl_internal.h | 10 +++++
 tools/libxl/libxl_mem.c      | 70 ++++++++++++++++++++++++++------
 tools/libxl/libxl_utils.c    | 15 -------
 tools/libxl/libxl_utils.h    |  2 +-
 tools/xl/xl_parse.c          | 15 +------
 tools/xl/xl_vmcontrol.c      |  6 +--
 9 files changed, 183 insertions(+), 61 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
@ 2019-10-04 15:17 ` Ian Jackson
  2019-10-04 15:33   ` Anthony PERARD
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 2/6] xl: Pass libxl_domain_config to freemem(), instead of b_info Ian Jackson
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Anthony PERARD, Jürgen Groß, Ian Jackson, Wei Liu

According to git log -G:

0x040700 was introduced in 304400459ef0 (aka 4.7.0-rc1~481)
  "tools/libxl: rename remus device to checkpoint device"

0x040800 was introduced in 57f8b13c7240 (aka 4.8.0-rc1~437)
  "libxl: memory size in kb requires 64 bit variable"

It is surprising that no-one noticed this.  I wonder if anyone is
using our LIBXL_API_VERSION facility.  If not maybe we should take a
different approach.

Anyway, in the meantime, we should fix this.  Backporting this is
probably a good idea: it won't change the behaviour for existing
callers but it will avoid errors for some older correct uses.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxl/libxl.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 2555e9cd3b..518fc9e47f 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -655,6 +655,7 @@ typedef struct libxl__ctx libxl_ctx;
 #ifdef LIBXL_API_VERSION
 #if LIBXL_API_VERSION != 0x040200 && LIBXL_API_VERSION != 0x040300 && \
     LIBXL_API_VERSION != 0x040400 && LIBXL_API_VERSION != 0x040500 && \
+    LIBXL_API_VERSION != 0x040700 && LIBXL_API_VERSION != 0x040800 && \
     LIBXL_API_VERSION != 0x041300
 #error Unknown LIBXL_API_VERSION
 #endif
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Xen-devel] [XEN PATCH for-4.13 2/6] xl: Pass libxl_domain_config to freemem(), instead of b_info
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800 Ian Jackson
@ 2019-10-04 15:17 ` Ian Jackson
  2019-10-04 17:24   ` Anthony PERARD
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function Ian Jackson
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Jürgen Groß, Ian Jackson, Wei Liu

We are going to change the libxl API in a moment and this change will
make it simpler.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/xl/xl_vmcontrol.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index b20582e15b..d33c6b38c9 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -314,7 +314,7 @@ static int domain_wait_event(uint32_t domid, libxl_event **event_r)
  * Returns true in case there is already, or we manage to free it, enough
  * memory, but also if autoballoon is false.
  */
-static bool freemem(uint32_t domid, libxl_domain_build_info *b_info)
+static bool freemem(uint32_t domid, libxl_domain_config *d_config)
 {
     int rc, retries = 3;
     uint64_t need_memkb, free_memkb;
@@ -322,7 +322,7 @@ static bool freemem(uint32_t domid, libxl_domain_build_info *b_info)
     if (!autoballoon)
         return true;
 
-    rc = libxl_domain_need_memory(ctx, b_info, &need_memkb);
+    rc = libxl_domain_need_memory(ctx, &d_config->b_info, &need_memkb);
     if (rc < 0)
         return false;
 
@@ -879,7 +879,7 @@ start:
         goto error_out;
 
     if (domid_soft_reset == INVALID_DOMID) {
-        if (!freemem(domid, &d_config.b_info)) {
+        if (!freemem(domid, &d_config)) {
             fprintf(stderr, "failed to free memory for the domain\n");
             ret = ERROR_FAIL;
             goto error_out;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800 Ian Jackson
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 2/6] xl: Pass libxl_domain_config to freemem(), instead of b_info Ian Jackson
@ 2019-10-04 15:17 ` Ian Jackson
  2019-10-04 15:53   ` Anthony PERARD
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 4/6] libxl: libxl_domain_need_memory: Make it take a domain_config Ian Jackson
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Anthony PERARD, Jürgen Groß, Ian Jackson, Wei Liu

Break out this into a new function.  We are going to want to call it
from a new call site.

Unfortunately not all of the defaults can be moved into the new
function without changing the order in which things are done.  That
does not seem wise at this stage of the release.  The effect is that
additional calls to libxl__domain_config_setdefault (which are going
to be introduced) do not quite set everything.  But they will do what
is needed.  After Xen 4.13 is done, we should move those settings into
the right order.

No functional change.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxl/libxl_create.c   | 40 ++++++++++++++++++++++++++++------------
 tools/libxl/libxl_internal.h |  3 +++
 2 files changed, 31 insertions(+), 12 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 099761a2d7..457beb6462 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -862,22 +862,14 @@ static void domcreate_destruction_cb(libxl__egc *egc,
                                      libxl__domain_destroy_state *dds,
                                      int rc);
 
-static void initiate_domain_create(libxl__egc *egc,
-                                   libxl__domain_create_state *dcs)
+int libxl__domain_config_setdefault(libxl__gc *gc,
+                                    libxl_domain_config *d_config,
+                                    uint32_t domid)
 {
-    STATE_AO_GC(dcs->ao);
     libxl_ctx *ctx = libxl__gc_owner(gc);
-    uint32_t domid;
-    int i, ret;
+    int ret;
     bool pod_enabled = false;
 
-    /* convenience aliases */
-    libxl_domain_config *const d_config = dcs->guest_config;
-    const int restore_fd = dcs->restore_fd;
-
-    domid = dcs->domid_soft_reset;
-    libxl__domain_build_state_init(&dcs->build_state);
-
     if (d_config->c_info.ssid_label) {
         char *s = d_config->c_info.ssid_label;
         ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
@@ -1008,6 +1000,27 @@ static void initiate_domain_create(libxl__egc *egc,
         goto error_out;
     }
 
+    ret = 0;
+ error_out:
+    return ret;
+}
+
+static void initiate_domain_create(libxl__egc *egc,
+                                   libxl__domain_create_state *dcs)
+{
+    STATE_AO_GC(dcs->ao);
+    uint32_t domid;
+    int i, ret;
+
+    /* convenience aliases */
+    libxl_domain_config *const d_config = dcs->guest_config;
+    const int restore_fd = dcs->restore_fd;
+
+    domid = dcs->domid_soft_reset;
+    libxl__domain_build_state_init(&dcs->build_state);
+
+    libxl__domain_config_setdefault(gc,d_config,domid);
+
     ret = libxl__domain_make(gc, d_config, &dcs->build_state, &domid);
     if (ret) {
         LOGD(ERROR, domid, "cannot make domain: %d", ret);
@@ -1019,6 +1032,9 @@ static void initiate_domain_create(libxl__egc *egc,
     dcs->guest_domid = domid;
     dcs->sdss.dm.guest_domid = 0; /* means we haven't spawned */
 
+    /* post-4.13 todo: move these next bits of defaulting to
+     * libxl__domain_config_setdefault */
+
     /*
      * Set the dm version quite early so that libxl doesn't have to pass the
      * build info around just to know if the domain has a device model or not.
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index d2d5af746b..50ac7b64ed 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1440,6 +1440,9 @@ _hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,
  *     All libxl API functions are expected to have arranged for this
  *     to be called before using any values within these structures.
  */
+_hidden int libxl__domain_config_setdefault(libxl__gc *gc,
+                                            libxl_domain_config *d_config,
+                                            uint32_t domid /* logging only */);
 _hidden int libxl__domain_create_info_setdefault(libxl__gc *gc,
                                         libxl_domain_create_info *c_info);
 _hidden int libxl__domain_build_info_setdefault(libxl__gc *gc,
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Xen-devel] [XEN PATCH for-4.13 4/6] libxl: libxl_domain_need_memory: Make it take a domain_config
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
                   ` (2 preceding siblings ...)
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function Ian Jackson
@ 2019-10-04 15:17 ` Ian Jackson
  2019-10-04 16:19   ` Anthony PERARD
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl Ian Jackson
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Anthony PERARD, Jürgen Groß, Ian Jackson, Wei Liu

This should calculate the extra memory needed for shadow and iommu,
the defaults for which depend on values in c_info.  So we need this to
have the complete domain config available.

And the defaults should actually be updated and stored.  So make it
non-const.

We provide the usual kind of compatibility function for callers
expecting 4.12 and earlier.  This function becomes responsible for the
clone-and-modify of the b_info.

No overall functional change for external libxl callers which use the
API version system to request a particular API version.

Other external libxl callers will need to update their calling code,
and will then find that the new version of this function fills in most
of the defaults in d_config.  Because libxl__domain_config_setdefault
doesn't quite do all of the defaults, that's only partial.  For
present purposes that doesn't matter because none of the missing
settings are used by the memory calculations.  It does mean we need to
document in the API spec that the defaulting is only partial.

This lack of functional change is despite the fact that
numa_place_domain now no longer calls
libxl__domain_build_info_setdefault (via libxl_domain_need_memory).
That is OK because it's idempotent and numa_place_domain's one call
site is libxl__build_pre which is called from libxl__domain_build
which is called from domcreate_bootloader_done, well after the
defaults are set by initiate_domain_create.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxl/libxl.h          | 23 ++++++++++++++-
 tools/libxl/libxl_dom.c      |  7 +++--
 tools/libxl/libxl_internal.h |  4 +++
 tools/libxl/libxl_mem.c      | 66 +++++++++++++++++++++++++++++++++++---------
 tools/xl/xl_vmcontrol.c      |  2 +-
 5 files changed, 84 insertions(+), 18 deletions(-)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 518fc9e47f..49b56fa1a3 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1245,6 +1245,20 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const libxl_mac *src);
  */
 #define LIBXL_HAVE_FN_USING_QMP_ASYNC 1
 
+/*
+ * LIBXL_HAVE_DOMAIN_NEED_MEMORY_CONFIG
+ *
+ * If this is set, libxl_domain_need_memory takes a
+ * libxl_domain_config* (non-const) and uint32_t domid_for_logging
+ * (instead of a const libxl_domain_build_info*).
+ *
+ * If this is set, there is no need to call
+ * libxl_get_required_shadow_memory and instead the caller should
+ * simply leave shadow_memkb set to LIBXL_MEMKB_DEFAULT and allow
+ * libxl to fill in a suitable default in the usual way.
+ */
+#define LIBXL_HAVE_DOMAIN_NEED_MEMORY_CONFIG
+
 typedef char **libxl_string_list;
 void libxl_string_list_dispose(libxl_string_list *sl);
 int libxl_string_list_length(const libxl_string_list *sl);
@@ -1723,8 +1737,13 @@ int libxl_get_memory_target_0x040700(libxl_ctx *ctx, uint32_t domid,
  */
 /* how much free memory in the system a domain needs to be built */
 int libxl_domain_need_memory(libxl_ctx *ctx,
-                             const libxl_domain_build_info *b_info_in,
+                             libxl_domain_config *config
+                             /* ^ will be partially defaulted */,
+                             uint32_t domid_for_logging /* INVALID_DOMID ok */,
                              uint64_t *need_memkb);
+int libxl_domain_need_memory_0x041200(libxl_ctx *ctx,
+                                      const libxl_domain_build_info *b_info_in,
+                                      uint64_t *need_memkb);
 int libxl_domain_need_memory_0x040700(libxl_ctx *ctx,
                                       const libxl_domain_build_info *b_info_in,
                                       uint32_t *need_memkb)
@@ -1754,6 +1773,8 @@ int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs);
 #define libxl_get_memory_target libxl_get_memory_target_0x040700
 #define libxl_domain_need_memory libxl_domain_need_memory_0x040700
 #define libxl_get_free_memory libxl_get_free_memory_0x040700
+#elif defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x041300
+#define libxl_domain_need_memory libxl_domain_need_memory_0x041200
 #endif
 
 int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index c5685b061c..cdb294ab8d 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -140,8 +140,9 @@ static int numa_cmpf(const libxl__numa_candidate *c1,
 
 /* The actual automatic NUMA placement routine */
 static int numa_place_domain(libxl__gc *gc, uint32_t domid,
-                             libxl_domain_build_info *info)
+                             libxl_domain_config *d_config)
 {
+    libxl_domain_build_info *info = &d_config->b_info;
     int found;
     libxl__numa_candidate candidate;
     libxl_bitmap cpumap, cpupool_nodemap, *map;
@@ -195,7 +196,7 @@ static int numa_place_domain(libxl__gc *gc, uint32_t domid,
         }
     }
 
-    rc = libxl_domain_need_memory(CTX, info, &memkb);
+    rc = libxl__domain_need_memory_calculate(gc, info, &memkb);
     if (rc)
         goto out;
     if (libxl_node_bitmap_alloc(CTX, &cpupool_nodemap, 0)) {
@@ -432,7 +433,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
             if (rc)
                 return rc;
 
-            rc = numa_place_domain(gc, domid, info);
+            rc = numa_place_domain(gc, domid, d_config);
             if (rc) {
                 libxl_bitmap_dispose(&cpumap_soft);
                 return rc;
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 50ac7b64ed..01de5576d9 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1450,6 +1450,10 @@ _hidden int libxl__domain_build_info_setdefault(libxl__gc *gc,
 _hidden void libxl__rdm_setdefault(libxl__gc *gc,
                                    libxl_domain_build_info *b_info);
 
+_hidden int libxl__domain_need_memory_calculate(libxl__gc *gc,
+                                      libxl_domain_build_info *b_info,
+                                      uint64_t *need_memkb);
+
 _hidden const char *libxl__device_nic_devname(libxl__gc *gc,
                                               uint32_t domid,
                                               uint32_t devid,
diff --git a/tools/libxl/libxl_mem.c b/tools/libxl/libxl_mem.c
index fd6f33312e..26cf136ac2 100644
--- a/tools/libxl/libxl_mem.c
+++ b/tools/libxl/libxl_mem.c
@@ -446,20 +446,12 @@ int libxl_get_memory_target_0x040700(
     return libxl__memkb_64to32(ctx, rc, my_out_target, out_target);
 }
 
-int libxl_domain_need_memory(libxl_ctx *ctx,
-                             const libxl_domain_build_info *b_info_in,
-                             uint64_t *need_memkb)
+int libxl__domain_need_memory_calculate(libxl__gc *gc,
+                              libxl_domain_build_info *b_info,
+                              uint64_t *need_memkb)
 {
-    GC_INIT(ctx);
-    libxl_domain_build_info b_info[1];
     int rc;
 
-    libxl_domain_build_info_init(b_info);
-    libxl_domain_build_info_copy(ctx, b_info, b_info_in);
-
-    rc = libxl__domain_build_info_setdefault(gc, b_info);
-    if (rc) goto out;
-
     *need_memkb = b_info->target_memkb;
     *need_memkb += b_info->shadow_memkb + b_info->iommu_memkb;
 
@@ -482,9 +474,57 @@ int libxl_domain_need_memory(libxl_ctx *ctx,
     rc = 0;
 out:
     GC_FREE;
-    libxl_domain_build_info_dispose(b_info);
     return rc;
+}
 
+int libxl_domain_need_memory(libxl_ctx *ctx,
+                             libxl_domain_config *d_config,
+                             uint32_t domid_for_logging,
+                             uint64_t *need_memkb)
+{
+    GC_INIT(ctx);
+    int rc;
+
+    rc = libxl__domain_config_setdefault(gc,
+                                         d_config,
+                                         domid_for_logging);
+    if (rc) goto out;
+
+    rc = libxl__domain_need_memory_calculate(gc,
+                                   &d_config->b_info,
+                                   need_memkb);
+    if (rc) goto out;
+
+    rc = 0;
+ out:
+    GC_FREE;
+    return rc;
+}
+
+int libxl_domain_need_memory_0x041200(libxl_ctx *ctx,
+                                      const libxl_domain_build_info *b_info_in,
+                                      uint64_t *need_memkb)
+{
+    GC_INIT(ctx);
+    int rc;
+
+    libxl_domain_build_info b_info[1];
+    libxl_domain_build_info_init(b_info);
+    libxl_domain_build_info_copy(ctx, b_info, b_info_in);
+
+    rc = libxl__domain_build_info_setdefault(gc, b_info);
+    if (rc) goto out;
+
+    rc = libxl__domain_need_memory_calculate(gc,
+                                   b_info,
+                                   need_memkb);
+    if (rc) goto out;
+
+    rc = 0;
+ out:
+    libxl_domain_build_info_dispose(b_info);
+    GC_FREE;
+    return rc;
 }
 
 int libxl_domain_need_memory_0x040700(libxl_ctx *ctx,
@@ -494,7 +534,7 @@ int libxl_domain_need_memory_0x040700(libxl_ctx *ctx,
     uint64_t my_need_memkb;
     int rc;
 
-    rc = libxl_domain_need_memory(ctx, b_info_in, &my_need_memkb);
+    rc = libxl_domain_need_memory_0x041200(ctx, b_info_in, &my_need_memkb);
     return libxl__memkb_64to32(ctx, rc, my_need_memkb, need_memkb);
 }
 
diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c
index d33c6b38c9..e520b1da79 100644
--- a/tools/xl/xl_vmcontrol.c
+++ b/tools/xl/xl_vmcontrol.c
@@ -322,7 +322,7 @@ static bool freemem(uint32_t domid, libxl_domain_config *d_config)
     if (!autoballoon)
         return true;
 
-    rc = libxl_domain_need_memory(ctx, &d_config->b_info, &need_memkb);
+    rc = libxl_domain_need_memory(ctx, d_config, domid, &need_memkb);
     if (rc < 0)
         return false;
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
                   ` (3 preceding siblings ...)
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 4/6] libxl: libxl_domain_need_memory: Make it take a domain_config Ian Jackson
@ 2019-10-04 15:17 ` Ian Jackson
  2019-10-04 17:19   ` Anthony PERARD
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 6/6] libxl: Remove/deprecate libxl_get_required_*_memory from the API Ian Jackson
  2019-10-07  6:45 ` [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Jürgen Groß
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Anthony PERARD, Jürgen Groß, Ian Jackson, Wei Liu

Defaulting is supposed to be done by libxl.  So these calculations
should be here in libxl.  libxl__domain_config_setdefault has all the
necessary information including the values of max_memkb and max_vcpus.

The overall functional effect depends on the caller:

For xl, no change.  The code moves from xl to libxl.

For callers who set one or both shadow_memkb and iommu_memkb (whether
from libxl_get_required_shadow_memory or otherwise) before calling
libxl_domain_need_memory (any version): the new code will leave their
setting(s) unchanged.

For callers who do not call libxl_domain_need_memory at all, and who
fail to set one of these memory values: now they are both are properly
set.  The shadow and iommu memory to be properly accounted for as
intended.

For callers which call libxl_domain_need_memory and request the
current API (4.13) or which track libxl, the default values are also
now right and everything works as intended.

For callers which call libxl_domain_need_memory, and request an old
pre-4.13 libxl API, and which leave one of these memkb settings unset,
we take special measures to preserve the old behaviour.

This means that they don't get the additional iommu memory and are at
risk of the domain running out of memory as a result of f89f555827a6
"remove late (on-demand) construction of IOMMU page tables".  But this
is no worse than the state just after f89f555827a6, which already
broke such callers in that way.  This is perhaps justifiable because
of the API stability warning next to libxl_domain_need_memory.

An alternative would be to drop the special-casing of these callers.
That would cause a discrepancy between libxl_domain_need_memory and
libxl_domain_create: the former would not include the iommu memory and
the latter would.  That seems worse, but it's debateable.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxl/libxl_create.c   | 40 ++++++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_internal.h |  3 +++
 tools/libxl/libxl_mem.c      |  4 ++++
 tools/xl/xl_parse.c          | 15 ++-------------
 4 files changed, 49 insertions(+), 13 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 457beb6462..3c5033b05b 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -250,6 +250,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
     switch (b_info->type) {
     case LIBXL_DOMAIN_TYPE_HVM:
         if (b_info->shadow_memkb == LIBXL_MEMKB_DEFAULT)
+            /* Normally defaulted in libxl__domain_create_info_setdefault */
             b_info->shadow_memkb = 0;
         if (b_info->u.hvm.mmio_hole_memkb == LIBXL_MEMKB_DEFAULT)
             b_info->u.hvm.mmio_hole_memkb = 0;
@@ -395,6 +396,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
         if (b_info->video_memkb == LIBXL_MEMKB_DEFAULT)
             b_info->video_memkb = 0;
         if (b_info->shadow_memkb == LIBXL_MEMKB_DEFAULT)
+            /* Normally defaulted in libxl__domain_create_info_setdefault */
             b_info->shadow_memkb = 0;
         if (b_info->u.pv.slack_memkb == LIBXL_MEMKB_DEFAULT)
             b_info->u.pv.slack_memkb = 0;
@@ -862,6 +864,30 @@ static void domcreate_destruction_cb(libxl__egc *egc,
                                      libxl__domain_destroy_state *dds,
                                      int rc);
 
+static _Bool ok_to_default_memkb_in_create(libxl__gc *gc)
+{
+    /*
+     * This is a fudge.  We are trying to find whether the caller
+     * calls the old version of libxl_domain_need_memory.  If they do
+     * then, because it only gets the b_info, and because it can't
+     * update the b_info (because it's const), it will base its
+     * calculations on defaulting shadow_memkb and iommu_memkb to 0
+     * In that case we probably shouldn't default them differently
+     * during libxl_domain_create.
+     *
+     * The result is that the behaviour with old callers is the same
+     * as in 4.13: no additional memory is allocated for shadow and
+     * iommu (unless the caller set shadow_memkb, eg from a call to
+     * libxl_get_required_shadow_memory).
+     */
+    return CTX->libxl_domain_need_memory_0x041200_called &&
+          !CTX->libxl_domain_need_memory_called;
+    /*
+     * Treat mixed callers as new callers.  Presumably they know what
+     * they are doing.
+     */
+}
+
 int libxl__domain_config_setdefault(libxl__gc *gc,
                                     libxl_domain_config *d_config,
                                     uint32_t domid)
@@ -974,6 +1000,20 @@ int libxl__domain_config_setdefault(libxl__gc *gc,
         goto error_out;
     }
 
+    if (d_config->b_info.shadow_memkb == LIBXL_MEMKB_DEFAULT
+        && ok_to_default_memkb_in_create(gc))
+        d_config->b_info.shadow_memkb =
+            libxl_get_required_shadow_memory(d_config->b_info.max_memkb,
+                                             d_config->b_info.max_vcpus);
+
+    /* No IOMMU reservation is needed if passthrough mode is not 'sync_pt' */
+    if (d_config->b_info.iommu_memkb == LIBXL_MEMKB_DEFAULT
+        && ok_to_default_memkb_in_create(gc))
+        d_config->b_info.iommu_memkb =
+            (d_config->c_info.passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
+            ? libxl_get_required_iommu_memory(d_config->b_info.max_memkb)
+            : 0;
+
     ret = libxl__domain_build_info_setdefault(gc, &d_config->b_info);
     if (ret) {
         LOGD(ERROR, domid, "Unable to set domain build info defaults");
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 01de5576d9..3e306ad4f6 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -637,6 +637,9 @@ struct libxl__ctx {
     LIBXL_LIST_ENTRY(libxl_ctx) sigchld_users_entry;
 
     libxl_version_info version_info;
+
+    _Bool libxl_domain_need_memory_0x041200_called,
+          libxl_domain_need_memory_called;
 };
 
 /*
diff --git a/tools/libxl/libxl_mem.c b/tools/libxl/libxl_mem.c
index 26cf136ac2..0537846ee2 100644
--- a/tools/libxl/libxl_mem.c
+++ b/tools/libxl/libxl_mem.c
@@ -485,6 +485,8 @@ int libxl_domain_need_memory(libxl_ctx *ctx,
     GC_INIT(ctx);
     int rc;
 
+    ctx->libxl_domain_need_memory_called = 1;
+
     rc = libxl__domain_config_setdefault(gc,
                                          d_config,
                                          domid_for_logging);
@@ -508,6 +510,8 @@ int libxl_domain_need_memory_0x041200(libxl_ctx *ctx,
     GC_INIT(ctx);
     int rc;
 
+    ctx->libxl_domain_need_memory_0x041200_called = 1;
+
     libxl_domain_build_info b_info[1];
     libxl_domain_build_info_init(b_info);
     libxl_domain_build_info_copy(ctx, b_info, b_info_in);
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 03a2c54dd2..79871c22d0 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -1572,19 +1572,8 @@ void parse_config_data(const char *config_source,
         exit(-ERROR_FAIL);
     }
 
-    /* libxl_get_required_shadow_memory() and
-     * libxl_get_required_iommu_memory() must be called after final values
-     * (default or specified) for vcpus and memory are set, because the
-     * calculations depend on those values. */
-    b_info->shadow_memkb = !xlu_cfg_get_long(config, "shadow_memory", &l, 0)
-        ? l * 1024
-        : libxl_get_required_shadow_memory(b_info->max_memkb,
-                                           b_info->max_vcpus);
-
-    /* No IOMMU reservation is needed if passthrough mode is not 'sync_pt' */
-    b_info->iommu_memkb = (c_info->passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
-        ? libxl_get_required_iommu_memory(b_info->max_memkb)
-        : 0;
+    if (!xlu_cfg_get_long(config, "shadow_memory", &l, 0))
+        b_info->shadow_memkb = l * 1024;
 
     xlu_cfg_get_defbool(config, "nomigrate", &b_info->disable_migrate, 0);
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [Xen-devel] [XEN PATCH for-4.13 6/6] libxl: Remove/deprecate libxl_get_required_*_memory from the API
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
                   ` (4 preceding siblings ...)
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl Ian Jackson
@ 2019-10-04 15:17 ` Ian Jackson
  2019-10-04 17:24   ` Anthony PERARD
  2019-10-07  6:45 ` [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Jürgen Groß
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 15:17 UTC (permalink / raw)
  To: xen-devel; +Cc: Anthony PERARD, Jürgen Groß, Ian Jackson, Wei Liu

These are now redundant because shadow_memkb and iommu_memkb are now
defaulted automatically by libxl_domain_need_memory and
libxl_domain_create etc.  Callers should not now call these; instead,
they should just let libxl take care of it.

libxl_get_required_shadow_memory was introduced in f89f555827a6
  "remove late (on-demand) construction of IOMMU page tables"
We can freely remove it because it was never in any release.

libxl_get_required_shadow_memory has been in libxl approximately
forever.  It should probably not have survived the creation of
libxl_domain_create, but it seems the API awkwardnesses we see in
recent commits prevented this.  So we have to keep it.  It remains
functional but we can deprecate it.  Hopefully we can get rid of it
completely before we find the need to change the calculation to use
additional information which its arguments do not currently supply.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 tools/libxl/libxl_create.c | 17 ++++++++++++++++-
 tools/libxl/libxl_utils.c  | 15 ---------------
 tools/libxl/libxl_utils.h  |  2 +-
 3 files changed, 17 insertions(+), 17 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 3c5033b05b..653b07646e 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -888,6 +888,21 @@ static _Bool ok_to_default_memkb_in_create(libxl__gc *gc)
      */
 }
 
+static unsigned long libxl__get_required_iommu_memory(unsigned long maxmem_kb)
+{
+    unsigned long iommu_pages = 0, mem_pages = maxmem_kb / 4;
+    unsigned int level;
+
+    /* Assume a 4 level page table with 512 entries per level */
+    for (level = 0; level < 4; level++)
+    {
+        mem_pages = DIV_ROUNDUP(mem_pages, 512);
+        iommu_pages += mem_pages;
+    }
+
+    return iommu_pages * 4;
+}
+
 int libxl__domain_config_setdefault(libxl__gc *gc,
                                     libxl_domain_config *d_config,
                                     uint32_t domid)
@@ -1011,7 +1026,7 @@ int libxl__domain_config_setdefault(libxl__gc *gc,
         && ok_to_default_memkb_in_create(gc))
         d_config->b_info.iommu_memkb =
             (d_config->c_info.passthrough == LIBXL_PASSTHROUGH_SYNC_PT)
-            ? libxl_get_required_iommu_memory(d_config->b_info.max_memkb)
+            ? libxl__get_required_iommu_memory(d_config->b_info.max_memkb)
             : 0;
 
     ret = libxl__domain_build_info_setdefault(gc, &d_config->b_info);
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 405733b7e1..f360f5e228 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -48,21 +48,6 @@ unsigned long libxl_get_required_shadow_memory(unsigned long maxmem_kb, unsigned
     return 4 * (256 * smp_cpus + 2 * (maxmem_kb / 1024));
 }
 
-unsigned long libxl_get_required_iommu_memory(unsigned long maxmem_kb)
-{
-    unsigned long iommu_pages = 0, mem_pages = maxmem_kb / 4;
-    unsigned int level;
-
-    /* Assume a 4 level page table with 512 entries per level */
-    for (level = 0; level < 4; level++)
-    {
-        mem_pages = DIV_ROUNDUP(mem_pages, 512);
-        iommu_pages += mem_pages;
-    }
-
-    return iommu_pages * 4;
-}
-
 char *libxl_domid_to_name(libxl_ctx *ctx, uint32_t domid)
 {
     unsigned int len;
diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h
index 630ccbe28a..46918aea84 100644
--- a/tools/libxl/libxl_utils.h
+++ b/tools/libxl/libxl_utils.h
@@ -24,7 +24,7 @@ const
 char *libxl_basename(const char *name); /* returns string from strdup */
 
 unsigned long libxl_get_required_shadow_memory(unsigned long maxmem_kb, unsigned int smp_cpus);
-unsigned long libxl_get_required_iommu_memory(unsigned long maxmem_kb);
+  /* deprecated; see LIBXL_HAVE_DOMAIN_NEED_MEMORY_CONFIG in libxl.h */
 int libxl_name_to_domid(libxl_ctx *ctx, const char *name, uint32_t *domid);
 int libxl_domain_qualifier_to_domid(libxl_ctx *ctx, const char *name, uint32_t *domid);
 char *libxl_domid_to_name(libxl_ctx *ctx, uint32_t domid);
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800 Ian Jackson
@ 2019-10-04 15:33   ` Anthony PERARD
  2019-10-04 16:08     ` Ian Jackson
  0 siblings, 1 reply; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 15:33 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 04:17:02PM +0100, Ian Jackson wrote:
> According to git log -G:
> 
> 0x040700 was introduced in 304400459ef0 (aka 4.7.0-rc1~481)
>   "tools/libxl: rename remus device to checkpoint device"
> 
> 0x040800 was introduced in 57f8b13c7240 (aka 4.8.0-rc1~437)
>   "libxl: memory size in kb requires 64 bit variable"
> 
> It is surprising that no-one noticed this.  I wonder if anyone is
> using our LIBXL_API_VERSION facility.  If not maybe we should take a
> different approach.

Libvirt does use it. I think it's currently set to 4.5.

> Anyway, in the meantime, we should fix this.  Backporting this is
> probably a good idea: it won't change the behaviour for existing
> callers but it will avoid errors for some older correct uses.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

> ---
>  tools/libxl/libxl.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 2555e9cd3b..518fc9e47f 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -655,6 +655,7 @@ typedef struct libxl__ctx libxl_ctx;
>  #ifdef LIBXL_API_VERSION
>  #if LIBXL_API_VERSION != 0x040200 && LIBXL_API_VERSION != 0x040300 && \
>      LIBXL_API_VERSION != 0x040400 && LIBXL_API_VERSION != 0x040500 && \
> +    LIBXL_API_VERSION != 0x040700 && LIBXL_API_VERSION != 0x040800 && \
>      LIBXL_API_VERSION != 0x041300
>  #error Unknown LIBXL_API_VERSION
>  #endif

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function Ian Jackson
@ 2019-10-04 15:53   ` Anthony PERARD
  2019-10-04 16:04     ` Ian Jackson
  0 siblings, 1 reply; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 15:53 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 04:17:04PM +0100, Ian Jackson wrote:
> Break out this into a new function.  We are going to want to call it
> from a new call site.
> 
> Unfortunately not all of the defaults can be moved into the new
> function without changing the order in which things are done.  That
> does not seem wise at this stage of the release.  The effect is that
> additional calls to libxl__domain_config_setdefault (which are going
> to be introduced) do not quite set everything.  But they will do what
> is needed.  After Xen 4.13 is done, we should move those settings into
> the right order.
> 
> No functional change.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
>  tools/libxl/libxl_create.c   | 40 ++++++++++++++++++++++++++++------------
>  tools/libxl/libxl_internal.h |  3 +++
>  2 files changed, 31 insertions(+), 12 deletions(-)
> 
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index 099761a2d7..457beb6462 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -862,22 +862,14 @@ static void domcreate_destruction_cb(libxl__egc *egc,
>                                       libxl__domain_destroy_state *dds,
>                                       int rc);
>  
> -static void initiate_domain_create(libxl__egc *egc,
> -                                   libxl__domain_create_state *dcs)
> +int libxl__domain_config_setdefault(libxl__gc *gc,
> +                                    libxl_domain_config *d_config,
> +                                    uint32_t domid)
>  {
> -    STATE_AO_GC(dcs->ao);
>      libxl_ctx *ctx = libxl__gc_owner(gc);
> -    uint32_t domid;
> -    int i, ret;
> +    int ret;
>      bool pod_enabled = false;
>  
> -    /* convenience aliases */
> -    libxl_domain_config *const d_config = dcs->guest_config;
> -    const int restore_fd = dcs->restore_fd;
> -
> -    domid = dcs->domid_soft_reset;
> -    libxl__domain_build_state_init(&dcs->build_state);
> -
>      if (d_config->c_info.ssid_label) {
>          char *s = d_config->c_info.ssid_label;
>          ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
> @@ -1008,6 +1000,27 @@ static void initiate_domain_create(libxl__egc *egc,
>          goto error_out;
>      }
>  
> +    ret = 0;
> + error_out:
> +    return ret;
> +}
> +
> +static void initiate_domain_create(libxl__egc *egc,
> +                                   libxl__domain_create_state *dcs)
> +{
> +    STATE_AO_GC(dcs->ao);
> +    uint32_t domid;
> +    int i, ret;
> +
> +    /* convenience aliases */
> +    libxl_domain_config *const d_config = dcs->guest_config;
> +    const int restore_fd = dcs->restore_fd;
> +
> +    domid = dcs->domid_soft_reset;
> +    libxl__domain_build_state_init(&dcs->build_state);
> +
> +    libxl__domain_config_setdefault(gc,d_config,domid);

Shouldn't you check for error ?

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function
  2019-10-04 15:53   ` Anthony PERARD
@ 2019-10-04 16:04     ` Ian Jackson
  2019-10-04 17:25       ` Anthony PERARD
  0 siblings, 1 reply; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 16:04 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: Jürgen Groß, xen-devel, Wei Liu

Anthony PERARD writes ("Re: [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function"):
> On Fri, Oct 04, 2019 at 04:17:04PM +0100, Ian Jackson wrote:
> > +    libxl__domain_config_setdefault(gc,d_config,domid);
> 
> Shouldn't you check for error ?

Blimey, yes!  Thanks!

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800
  2019-10-04 15:33   ` Anthony PERARD
@ 2019-10-04 16:08     ` Ian Jackson
  0 siblings, 0 replies; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 16:08 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: Jürgen Groß, xen-devel, Wei Liu

Anthony PERARD writes ("Re: [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800"):
> On Fri, Oct 04, 2019 at 04:17:02PM +0100, Ian Jackson wrote:
> > It is surprising that no-one noticed this.  I wonder if anyone is
> > using our LIBXL_API_VERSION facility.  If not maybe we should take a
> > different approach.
> 
> Libvirt does use it. I think it's currently set to 4.5.

Ah.  I have adjusted the commit message somewhat.

> > Anyway, in the meantime, we should fix this.  Backporting this is
> > probably a good idea: it won't change the behaviour for existing
> > callers but it will avoid errors for some older correct uses.
> > 
> > Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
> 
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 4/6] libxl: libxl_domain_need_memory: Make it take a domain_config
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 4/6] libxl: libxl_domain_need_memory: Make it take a domain_config Ian Jackson
@ 2019-10-04 16:19   ` Anthony PERARD
  0 siblings, 0 replies; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 16:19 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 04:17:05PM +0100, Ian Jackson wrote:
> diff --git a/tools/libxl/libxl_mem.c b/tools/libxl/libxl_mem.c
> index fd6f33312e..26cf136ac2 100644
> --- a/tools/libxl/libxl_mem.c
> +++ b/tools/libxl/libxl_mem.c
> @@ -446,20 +446,12 @@ int libxl_get_memory_target_0x040700(
>      return libxl__memkb_64to32(ctx, rc, my_out_target, out_target);
>  }
>  
> -int libxl_domain_need_memory(libxl_ctx *ctx,
> -                             const libxl_domain_build_info *b_info_in,
> -                             uint64_t *need_memkb)
> +int libxl__domain_need_memory_calculate(libxl__gc *gc,
> +                              libxl_domain_build_info *b_info,
> +                              uint64_t *need_memkb)
>  {
> -    GC_INIT(ctx);
> -    libxl_domain_build_info b_info[1];
>      int rc;
>  
> -    libxl_domain_build_info_init(b_info);
> -    libxl_domain_build_info_copy(ctx, b_info, b_info_in);
> -
> -    rc = libxl__domain_build_info_setdefault(gc, b_info);
> -    if (rc) goto out;
> -
>      *need_memkb = b_info->target_memkb;
>      *need_memkb += b_info->shadow_memkb + b_info->iommu_memkb;
>  
> @@ -482,9 +474,57 @@ int libxl_domain_need_memory(libxl_ctx *ctx,
>      rc = 0;
>  out:
>      GC_FREE;

This GC_FREE should be removed.

> -    libxl_domain_build_info_dispose(b_info);
>      return rc;
> +}
>  

The rest looks fine. So with the extra GC_FREE removed:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl Ian Jackson
@ 2019-10-04 17:19   ` Anthony PERARD
  2019-10-04 17:23     ` Ian Jackson
  0 siblings, 1 reply; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 17:19 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 04:17:06PM +0100, Ian Jackson wrote:
> @@ -862,6 +864,30 @@ static void domcreate_destruction_cb(libxl__egc *egc,
>                                       libxl__domain_destroy_state *dds,
>                                       int rc);
>  
> +static _Bool ok_to_default_memkb_in_create(libxl__gc *gc)

Is there a reason to use _Bool instead of `bool'? It would be the first
_Bool in libxl.

> +{
> +    /*
> +     * This is a fudge.  We are trying to find whether the caller
> +     * calls the old version of libxl_domain_need_memory.  If they do
> +     * then, because it only gets the b_info, and because it can't
> +     * update the b_info (because it's const), it will base its
> +     * calculations on defaulting shadow_memkb and iommu_memkb to 0
> +     * In that case we probably shouldn't default them differently
> +     * during libxl_domain_create.
> +     *
> +     * The result is that the behaviour with old callers is the same
> +     * as in 4.13: no additional memory is allocated for shadow and
> +     * iommu (unless the caller set shadow_memkb, eg from a call to
> +     * libxl_get_required_shadow_memory).
> +     */
> +    return CTX->libxl_domain_need_memory_0x041200_called &&
> +          !CTX->libxl_domain_need_memory_called;

I think the logic here is inverted.

With this patch applied, we have:
xl
    calls libxl_domain_need_memory()
        libxl__domain_config_setdefault()
            because shadow_memkb is default and that function return 0
            then shadow_memkb is still default
    create_domain()
        libxl__domain_config_setdefault()
            same thing, shadow_memkb is untouch, so still default
            libxl__domain_build_info_setdefault()
                shadow_memkb is default
                    -> shadow_memkb = 0

This is different what is expected by the commit message.
> For xl, no change.  The code moves from xl to libxl.

Instead, if xl calls libxl_domain_need_memory_0x041200():
    create_domain()
        libxl__domain_config_setdefault()
            if shadow_memkb is default; ok_to_default_memkb_in_create is 1
                so shadow_memkb is set to a value different that the one
                calculated by need_memory_0x04100.

Or did I miss something?

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl
  2019-10-04 17:19   ` Anthony PERARD
@ 2019-10-04 17:23     ` Ian Jackson
  0 siblings, 0 replies; 19+ messages in thread
From: Ian Jackson @ 2019-10-04 17:23 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: Jürgen Groß, xen-devel, Wei Liu

Anthony PERARD writes ("Re: [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl"):
> On Fri, Oct 04, 2019 at 04:17:06PM +0100, Ian Jackson wrote:
> > @@ -862,6 +864,30 @@ static void domcreate_destruction_cb(libxl__egc *egc,
> >                                       libxl__domain_destroy_state *dds,
> >                                       int rc);
> >  
> > +static _Bool ok_to_default_memkb_in_create(libxl__gc *gc)
> 
> Is there a reason to use _Bool instead of `bool'? It would be the first
> _Bool in libxl.

I had the wrong head on.  (I've been writing code recently where
`bool' wasn't available.)

> > +    return CTX->libxl_domain_need_memory_0x041200_called &&
> > +          !CTX->libxl_domain_need_memory_called;
> 
> I think the logic here is inverted.

You are right.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 6/6] libxl: Remove/deprecate libxl_get_required_*_memory from the API
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 6/6] libxl: Remove/deprecate libxl_get_required_*_memory from the API Ian Jackson
@ 2019-10-04 17:24   ` Anthony PERARD
  0 siblings, 0 replies; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 17:24 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 04:17:07PM +0100, Ian Jackson wrote:
> These are now redundant because shadow_memkb and iommu_memkb are now
> defaulted automatically by libxl_domain_need_memory and
> libxl_domain_create etc.  Callers should not now call these; instead,
> they should just let libxl take care of it.
> 
> libxl_get_required_shadow_memory was introduced in f89f555827a6
>   "remove late (on-demand) construction of IOMMU page tables"
> We can freely remove it because it was never in any release.
> 
> libxl_get_required_shadow_memory has been in libxl approximately
> forever.  It should probably not have survived the creation of
> libxl_domain_create, but it seems the API awkwardnesses we see in
> recent commits prevented this.  So we have to keep it.  It remains
> functional but we can deprecate it.  Hopefully we can get rid of it
> completely before we find the need to change the calculation to use
> additional information which its arguments do not currently supply.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 2/6] xl: Pass libxl_domain_config to freemem(), instead of b_info
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 2/6] xl: Pass libxl_domain_config to freemem(), instead of b_info Ian Jackson
@ 2019-10-04 17:24   ` Anthony PERARD
  0 siblings, 0 replies; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 17:24 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 04:17:03PM +0100, Ian Jackson wrote:
> We are going to change the libxl API in a moment and this change will
> make it simpler.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function
  2019-10-04 16:04     ` Ian Jackson
@ 2019-10-04 17:25       ` Anthony PERARD
  2019-10-07  9:57         ` Ian Jackson
  0 siblings, 1 reply; 19+ messages in thread
From: Anthony PERARD @ 2019-10-04 17:25 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Jürgen Groß, xen-devel, Wei Liu

On Fri, Oct 04, 2019 at 05:04:27PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("Re: [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function"):
> > On Fri, Oct 04, 2019 at 04:17:04PM +0100, Ian Jackson wrote:
> > > +    libxl__domain_config_setdefault(gc,d_config,domid);
> > 
> > Shouldn't you check for error ?
> 
> Blimey, yes!  Thanks!

With that fixed:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory
  2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
                   ` (5 preceding siblings ...)
  2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 6/6] libxl: Remove/deprecate libxl_get_required_*_memory from the API Ian Jackson
@ 2019-10-07  6:45 ` Jürgen Groß
  6 siblings, 0 replies; 19+ messages in thread
From: Jürgen Groß @ 2019-10-07  6:45 UTC (permalink / raw)
  To: Ian Jackson, xen-devel; +Cc: AnthonyPERARD, Wei Liu

On 04.10.19 17:17, Ian Jackson wrote:
> libxl_get_required_shadow_memory has always been anomalous.  libxl
> ought to default these things itself.  Recently, another analogous
> setting, iommu_memkb, was introduced, along with another function
> along the same lines.
> 
> This API is not very good.  Fixing it was not entirely trivial.
> (Thanks to Paul for explaining some of the difficulties and Anthony
> for in-principle review of my proposal.)
> 
> Ian Jackson (6):
>    libxl: Offer API versions 0x040700 and 0x040800
>    xl: Pass libxl_domain_config to freemem(), instead of b_info
>    libxl: libxl__domain_config_setdefault: New function
>    libxl: libxl_domain_need_memory: Make it take a domain_config
>    libxl: Move shadow_memkb and iommu_memkb defaulting into libxl
>    libxl: Remove/deprecate libxl_get_required_*_memory from the API
> 
>   tools/libxl/libxl.h          | 24 ++++++++++-
>   tools/libxl/libxl_create.c   | 95 ++++++++++++++++++++++++++++++++++++++------
>   tools/libxl/libxl_dom.c      |  7 ++--
>   tools/libxl/libxl_internal.h | 10 +++++
>   tools/libxl/libxl_mem.c      | 70 ++++++++++++++++++++++++++------
>   tools/libxl/libxl_utils.c    | 15 -------
>   tools/libxl/libxl_utils.h    |  2 +-
>   tools/xl/xl_parse.c          | 15 +------
>   tools/xl/xl_vmcontrol.c      |  6 +--
>   9 files changed, 183 insertions(+), 61 deletions(-)
> 

For the series:

Release-acked-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function
  2019-10-04 17:25       ` Anthony PERARD
@ 2019-10-07  9:57         ` Ian Jackson
  0 siblings, 0 replies; 19+ messages in thread
From: Ian Jackson @ 2019-10-07  9:57 UTC (permalink / raw)
  To: Anthony PERARD; +Cc: Jürgen Groß, xen-devel, Wei Liu

Anthony PERARD writes ("Re: [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function"):
> On Fri, Oct 04, 2019 at 05:04:27PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("Re: [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function"):
> > > Shouldn't you check for error ?
> > 
> > Blimey, yes!  Thanks!
> 
> With that fixed:
> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks.

I think the things you spotted in this review means I should go
through this series again, and try to test it somehow.  I will do that
and then send a v2.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2019-10-07  9:57 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-04 15:17 [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Ian Jackson
2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 1/6] libxl: Offer API versions 0x040700 and 0x040800 Ian Jackson
2019-10-04 15:33   ` Anthony PERARD
2019-10-04 16:08     ` Ian Jackson
2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 2/6] xl: Pass libxl_domain_config to freemem(), instead of b_info Ian Jackson
2019-10-04 17:24   ` Anthony PERARD
2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 3/6] libxl: libxl__domain_config_setdefault: New function Ian Jackson
2019-10-04 15:53   ` Anthony PERARD
2019-10-04 16:04     ` Ian Jackson
2019-10-04 17:25       ` Anthony PERARD
2019-10-07  9:57         ` Ian Jackson
2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 4/6] libxl: libxl_domain_need_memory: Make it take a domain_config Ian Jackson
2019-10-04 16:19   ` Anthony PERARD
2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 5/6] libxl: Move shadow_memkb and iommu_memkb defaulting into libxl Ian Jackson
2019-10-04 17:19   ` Anthony PERARD
2019-10-04 17:23     ` Ian Jackson
2019-10-04 15:17 ` [Xen-devel] [XEN PATCH for-4.13 6/6] libxl: Remove/deprecate libxl_get_required_*_memory from the API Ian Jackson
2019-10-04 17:24   ` Anthony PERARD
2019-10-07  6:45 ` [Xen-devel] [XEN PATCH for-4.13 0/6] Drop/deprecate libxl_get_required_*_memory Jürgen Groß

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).