[PATCH 0 of 12] v4: xl/libxl autoballooning

[PATCH 0 of 12] v4: xl/libxl autoballooning

[Stefano Stabellini]

Hi all,
this patch series adds some new memory management functions to libxl
that are used by xl to autoballoon dom0 down by the amount of memory
needed by the new domain build.
In order to avoid conflicts with other xl instances during the memory
accounting operation I am introducing a global filelock.
I am also introducing a global configuration file for xl to
enable/disable autoballooning.

Changes compares to v3:

- following the discussion on xen-devel I decided to keep max_memkb and
the enforce parameter to libxl_set_memory_target;

- I removed libxl_set_relative_memory_target and added a relative flag
to libxl_set_memory_target instead;

- I added a document that explain the libxl domain memory model;

- I added a small patch to avoid changing static-max after domain
creation;

- I added a small patch to parse the maxmem option in VM config files.

Cheers,

Stefano

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

[PATCH 1 of 12] libxl: add current_memkb to libxl_dominfo

[stefano.stabellini]

currently xcinfo2xlinfo reads tot_pages and uses that data to calculate
max_memkb, while tot_pages is the memory currently used by the domain
and max_pages is the theoretical maximum.
This patch calculates max_memkb from max_pages and introduces
current_memkb to store the current memory usage.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r eff592364826 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Wed Sep 01 11:23:49 2010 +0100
+++ b/tools/libxl/libxl.c	Thu Sep 02 10:51:45 2010 +0100
@@ -543,7 +543,8 @@ static void xcinfo2xlinfo(const xc_domai
     else
         xlinfo->shutdown_reason  = ~0;
 
-    xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
+    xlinfo->current_memkb = PAGE_TO_MEMKB(xcinfo->tot_pages);
+    xlinfo->max_memkb = PAGE_TO_MEMKB(xcinfo->max_pages);
     xlinfo->cpu_time = xcinfo->cpu_time;
     xlinfo->vcpu_max_id = xcinfo->max_vcpu_id;
     xlinfo->vcpu_online = xcinfo->nr_online_vcpus;
diff -r eff592364826 tools/libxl/libxl.idl
--- a/tools/libxl/libxl.idl	Wed Sep 01 11:23:49 2010 +0100
+++ b/tools/libxl/libxl.idl	Thu Sep 02 10:51:45 2010 +0100
@@ -36,6 +36,7 @@ libxl_dominfo = Struct("dominfo",[
 
 Otherwise set to a value guaranteed not to clash with any valid
 SHUTDOWN_* constant."""),
+    ("current_memkb",   uint64),
     ("max_memkb",   uint64),
     ("cpu_time",    uint64),
     ("vcpu_max_id", uint32),
diff -r eff592364826 tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Wed Sep 01 11:23:49 2010 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Thu Sep 02 10:51:45 2010 +0100
@@ -2209,7 +2205,7 @@ static void list_domains(int verbose, co
         printf("%-40s %5d %5lu %5d     %c%c%c%c%c%c  %8.1f",
                 domname,
                 info[i].domid,
-                (unsigned long) (info[i].max_memkb / 1024),
+                (unsigned long) (info[i].current_memkb / 1024),
                 info[i].vcpu_online,
                 info[i].running ? 'r' : '-',
                 info[i].blocked ? 'b' : '-',

[PATCH 2 of 12] libxl: do not modify static-max after domain creation

[stefano.stabellini]

static-max represents the maximum memory the domain can ever have,
therefore it shouldn't be changed after domain creation.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 13811e12adf5 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Thu Sep 02 11:13:49 2010 +0100
+++ b/tools/libxl/libxl.c	Thu Sep 02 11:14:24 2010 +0100
@@ -2746,9 +2746,6 @@ int libxl_domain_setmaxmem(libxl_ctx *ct
         goto out;
     }
 
-    if (domid != 0)
-        libxl_xs_write(&gc, XBT_NULL, libxl_sprintf(&gc, "%s/memory/static-max", dompath), "%"PRIu32, max_memkb);
-
     rc = 0;
 out:
     libxl_free_all(&gc);

[PATCH 3 of 12] libxl: use transactions in libxl_set_memory_target

[stefano.stabellini]

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 33e3cd66e5ca tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Thu Sep 02 11:14:24 2010 +0100
+++ b/tools/libxl/libxl.c	Thu Sep 02 11:14:44 2010 +0100
@@ -2752,58 +2752,66 @@ out:
     return rc;
 }
 
-int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb, int enforce)
+int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t
+        target_memkb, int enforce)
 {
     libxl_gc gc = LIBXL_INIT_GC(ctx);
-    int rc = 1;
-    uint32_t memorykb = 0, videoram = 0;
-    char *memmax, *endptr, *videoram_s = NULL;
+    int rc = 1, abort = 0;
+    uint32_t videoram = 0;
+    char *videoram_s = NULL;
     char *dompath = libxl_xs_get_dompath(&gc, domid);
     xc_domaininfo_t info;
     libxl_dominfo ptr;
     char *uuid;
-
-    if (domid) {
-        memmax = libxl_xs_read(&gc, XBT_NULL, libxl_sprintf(&gc, "%s/memory/static-max", dompath));
-        if (!memmax) {
+    xs_transaction_t t;
+
+retry_transaction:
+    t = xs_transaction_start(ctx->xsh);
+
+    videoram_s = libxl_xs_read(&gc, t, libxl_sprintf(&gc, "%s/memory/videoram",
+                dompath));
+    videoram = videoram_s ? atoi(videoram_s) : 0;
+
+    if (enforce) {
+        rc = xc_domain_setmaxmem(ctx->xch, domid, target_memkb +
+                LIBXL_MAXMEM_CONSTANT);
+        if (rc != 0) {
             XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
-                "cannot get memory info from %s/memory/static-max\n", dompath);
-            goto out;
-        }
-        memorykb = strtoul(memmax, &endptr, 10);
-        if (*endptr != '\0') {
-            XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
-                "invalid max memory %s from %s/memory/static-max\n", memmax, dompath);
-            goto out;
-        }
-
-        if (target_memkb > memorykb) {
-            XL_LOG(ctx, XL_LOG_ERROR,
-                "memory_dynamic_max must be less than or equal to memory_static_max\n");
+                    "xc_domain_setmaxmem domid=%d memkb=%d failed "
+                    "rc=%d\n", domid, target_memkb + LIBXL_MAXMEM_CONSTANT, rc);
+            abort = 1;
             goto out;
         }
     }
 
-    videoram_s = libxl_xs_read(&gc, XBT_NULL, libxl_sprintf(&gc, "%s/memory/videoram", dompath));
-    videoram = videoram_s ? atoi(videoram_s) : 0;
-
-    libxl_xs_write(&gc, XBT_NULL, libxl_sprintf(&gc, "%s/memory/target", dompath), "%"PRIu32, target_memkb);
-
+    rc = xc_domain_memory_set_pod_target(ctx->xch, domid, (target_memkb -
+                videoram) / 4, NULL, NULL, NULL);
+    if (rc != 0) {
+        XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                "xc_domain_memory_set_pod_target domid=%d, memkb=%d "
+                "failed rc=%d\n", domid, (target_memkb - videoram) / 4,
+                rc);
+        abort = 1;
+        goto out;
+    }
+
+    libxl_xs_write(&gc, t, libxl_sprintf(&gc, "%s/memory/target", dompath),
+            "%"PRIu32, target_memkb);
     rc = xc_domain_getinfolist(ctx->xch, domid, 1, &info);
-    if (rc != 1 || info.domain != domid)
+    if (rc != 1 || info.domain != domid) {
+        abort = 1;
         goto out;
+    }
     xcinfo2xlinfo(&info, &ptr);
     uuid = libxl_uuid2string(&gc, ptr.uuid);
-    libxl_xs_write(&gc, XBT_NULL, libxl_sprintf(&gc, "/vm/%s/memory", uuid), "%"PRIu32, target_memkb / 1024);
-
-    if (enforce || !domid)
-        memorykb = target_memkb;
-    rc = xc_domain_setmaxmem(ctx->xch, domid, memorykb + LIBXL_MAXMEM_CONSTANT);
-    if (rc != 0)
-        goto out;
-    rc = xc_domain_memory_set_pod_target(ctx->xch, domid, (target_memkb - videoram) / 4, NULL, NULL, NULL);
+    libxl_xs_write(&gc, t, libxl_sprintf(&gc, "/vm/%s/memory", uuid), "%"PRIu32,
+            target_memkb / 1024);
 
 out:
+    if (!xs_transaction_end(ctx->xsh, t, abort) && !abort)
+        if (errno == EAGAIN)
+            goto retry_transaction;
+
     libxl_free_all(&gc);
     return rc;
 }

[PATCH 4 of 12] libxl: add a "relative" parameter to libxl_set_memory_target

[stefano.stabellini]

Introduce a relative parameter to flag that target_memkb is a
relative amount of memory.
The first time we are reading/writing dom0 memory target, fill the
informations in xenstore if they are missing.
Introduce libxl_get_memory_target.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 503200922947 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Fri Sep 03 11:39:12 2010 +0100
+++ b/tools/libxl/libxl.c	Fri Sep 03 11:46:36 2010 +0100
@@ -2752,13 +2752,61 @@ out:
     return rc;
 }
 
-int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t
-        target_memkb, int enforce)
+static int libxl__fill_dom0_memory_info(libxl_gc *gc, uint32_t *target_memkb)
+{
+    int rc;
+    libxl_dominfo info;
+    char *target = NULL, *endptr = NULL;
+    char *target_path = "/local/domain/0/memory/target";
+    char *max_path = "/local/domain/0/memory/static-max";
+    xs_transaction_t t;
+    libxl_ctx *ctx = libxl_gc_owner(gc);
+
+retry_transaction:
+    t = xs_transaction_start(ctx->xsh);
+
+    target = libxl_xs_read(gc, t, target_path);
+    if (target) {
+        *target_memkb = strtoul(target, &endptr, 10);
+        if (*endptr != '\0') {
+            XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                    "invalid memory target %s from %s\n", target, target_path);
+            rc = ERROR_FAIL;
+            goto out;
+        }
+        rc = 0;
+        goto out;
+    }
+
+    rc = libxl_domain_info(ctx, &info, 0);
+    if (rc < 0)
+        return rc;
+
+    libxl_xs_write(gc, t, target_path, "%"PRIu32,
+            (uint32_t) info.current_memkb);
+    libxl_xs_write(gc, t, max_path, "%"PRIu32,
+            (uint32_t) info.max_memkb);
+
+    *target_memkb = (uint32_t) info.current_memkb;
+    rc = 0;
+
+out:
+    if (!xs_transaction_end(ctx->xsh, t, 0))
+        if (errno == EAGAIN)
+            goto retry_transaction;
+
+
+    return rc;
+}
+
+int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid,
+        int32_t target_memkb, int relative, int enforce)
 {
     libxl_gc gc = LIBXL_INIT_GC(ctx);
     int rc = 1, abort = 0;
-    uint32_t videoram = 0;
-    char *videoram_s = NULL;
+    uint32_t memorykb = 0, videoram = 0;
+    uint32_t current_target_memkb = 0, new_target_memkb = 0;
+    char *memmax, *endptr, *videoram_s = NULL, *target = NULL;
     char *dompath = libxl_xs_get_dompath(&gc, domid);
     xc_domaininfo_t info;
     libxl_dominfo ptr;
@@ -2768,35 +2816,92 @@ int libxl_set_memory_target(libxl_ctx *c
 retry_transaction:
     t = xs_transaction_start(ctx->xsh);
 
-    videoram_s = libxl_xs_read(&gc, t, libxl_sprintf(&gc, "%s/memory/videoram",
-                dompath));
+    target = libxl_xs_read(&gc, t, libxl_sprintf(&gc,
+                "%s/memory/target", dompath));
+    if (!target && !domid) {
+        xs_transaction_end(ctx->xsh, t, 1);
+        rc = libxl__fill_dom0_memory_info(&gc, &current_target_memkb);
+        if (rc < 0) {
+            abort = 1;
+            goto out;
+        }
+        goto retry_transaction;
+    } else if (!target) {
+        XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                "cannot get target memory info from %s/memory/target\n",
+                dompath);
+        abort = 1;
+        goto out;
+    } else {
+        current_target_memkb = strtoul(target, &endptr, 10);
+        if (*endptr != '\0') {
+            XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                    "invalid memory target %s from %s/memory/target\n",
+                    target, dompath);
+            abort = 1;
+            goto out;
+        }
+    }
+    memmax = libxl_xs_read(&gc, t, libxl_sprintf(&gc,
+                "%s/memory/static-max", dompath));
+    if (!memmax) {
+        XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                "cannot get memory info from %s/memory/static-max\n",
+                dompath);
+        abort = 1;
+        goto out;
+    }
+    memorykb = strtoul(memmax, &endptr, 10);
+    if (*endptr != '\0') {
+        XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                "invalid max memory %s from %s/memory/static-max\n",
+                memmax, dompath);
+        abort = 1;
+        goto out;
+    }
+
+    if (relative)
+        new_target_memkb = current_target_memkb + target_memkb;
+    else
+        new_target_memkb = target_memkb;
+    if (new_target_memkb > memorykb) {
+        XL_LOG(ctx, XL_LOG_ERROR,
+                "memory_dynamic_max must be less than or equal to"
+                " memory_static_max\n");
+        abort = 1;
+        goto out;
+    }
+
+    videoram_s = libxl_xs_read(&gc, t, libxl_sprintf(&gc,
+                "%s/memory/videoram", dompath));
     videoram = videoram_s ? atoi(videoram_s) : 0;
 
     if (enforce) {
-        rc = xc_domain_setmaxmem(ctx->xch, domid, target_memkb +
+        memorykb = new_target_memkb;
+        rc = xc_domain_setmaxmem(ctx->xch, domid, memorykb +
                 LIBXL_MAXMEM_CONSTANT);
         if (rc != 0) {
             XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
                     "xc_domain_setmaxmem domid=%d memkb=%d failed "
-                    "rc=%d\n", domid, target_memkb + LIBXL_MAXMEM_CONSTANT, rc);
+                    "rc=%d\n", domid, memorykb + LIBXL_MAXMEM_CONSTANT, rc);
             abort = 1;
             goto out;
         }
     }
 
-    rc = xc_domain_memory_set_pod_target(ctx->xch, domid, (target_memkb -
-                videoram) / 4, NULL, NULL, NULL);
+    rc = xc_domain_memory_set_pod_target(ctx->xch, domid,
+            (new_target_memkb - videoram) / 4, NULL, NULL, NULL);
     if (rc != 0) {
         XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
                 "xc_domain_memory_set_pod_target domid=%d, memkb=%d "
-                "failed rc=%d\n", domid, (target_memkb - videoram) / 4,
+                "failed rc=%d\n", domid, (new_target_memkb - videoram) / 4,
                 rc);
         abort = 1;
         goto out;
     }
 
-    libxl_xs_write(&gc, t, libxl_sprintf(&gc, "%s/memory/target", dompath),
-            "%"PRIu32, target_memkb);
+    libxl_xs_write(&gc, t, libxl_sprintf(&gc, "%s/memory/target",
+                dompath), "%"PRIu32, new_target_memkb);
     rc = xc_domain_getinfolist(ctx->xch, domid, 1, &info);
     if (rc != 1 || info.domain != domid) {
         abort = 1;
@@ -2804,8 +2909,8 @@ retry_transaction:
     }
     xcinfo2xlinfo(&info, &ptr);
     uuid = libxl_uuid2string(&gc, ptr.uuid);
-    libxl_xs_write(&gc, t, libxl_sprintf(&gc, "/vm/%s/memory", uuid), "%"PRIu32,
-            target_memkb / 1024);
+    libxl_xs_write(&gc, t, libxl_sprintf(&gc, "/vm/%s/memory", uuid),
+            "%"PRIu32, new_target_memkb / 1024);
 
 out:
     if (!xs_transaction_end(ctx->xsh, t, abort) && !abort)
@@ -2816,6 +2921,42 @@ out:
     return rc;
 }
 
+int libxl_get_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t *out_target)
+{
+    libxl_gc gc = LIBXL_INIT_GC(ctx);
+    int rc = 1;
+    char *target = NULL, *endptr = NULL;
+    char *dompath = libxl_xs_get_dompath(&gc, domid);
+    uint32_t target_memkb;
+
+    target = libxl_xs_read(&gc, XBT_NULL, libxl_sprintf(&gc,
+                "%s/memory/target", dompath));
+    if (!target && !domid) {
+        rc = libxl__fill_dom0_memory_info(&gc, &target_memkb);
+        if (rc < 0)
+            goto out;
+    } else if (!target) {
+        XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                "cannot get target memory info from %s/memory/target\n",
+                dompath);
+        goto out;
+    } else {
+        target_memkb = strtoul(target, &endptr, 10);
+        if (*endptr != '\0') {
+            XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
+                    "invalid memory target %s from %s/memory/target\n",
+                    target, dompath);
+            goto out;
+        }
+    }
+    *out_target = target_memkb;
+    rc = 0;
+
+out:
+    libxl_free_all(&gc);
+    return rc;
+}
+
 int libxl_button_press(libxl_ctx *ctx, uint32_t domid, libxl_button button)
 {
     int rc = -1;
diff -r 503200922947 tools/libxl/libxl.h
--- a/tools/libxl/libxl.h	Fri Sep 03 11:39:12 2010 +0100
+++ b/tools/libxl/libxl.h	Fri Sep 03 11:46:36 2010 +0100
@@ -315,7 +315,8 @@ int libxl_domain_unpause(libxl_ctx *ctx,
 int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const char *filename);
 
 int libxl_domain_setmaxmem(libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb);
-int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb, int enforce);
+int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, int32_t target_memkb, int relative, int enforce);
+int libxl_get_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t *out_target);
 
 int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
 int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_constype type);
diff -r 503200922947 tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Fri Sep 03 11:39:12 2010 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Fri Sep 03 11:46:36 2010 +0100
@@ -1766,7 +1766,7 @@ static void set_memory_target(char *p, c
         exit(3);
     }
 
-    libxl_set_memory_target(&ctx, domid, memorykb, /* enforce */ 1);
+    libxl_set_memory_target(&ctx, domid, memorykb, 0, /* enforce */ 1);
 }
 
 int main_memset(int argc, char **argv)

[PATCH 5 of 12] libxl: adds few more memory operations

[stefano.stabellini]

libxl_domain_need_memory: calculate how much memory a domain needs in
order to be built and start correctly.

libxl_get_free_memory: calculate the total free memory in the system.

libxl_wait_for_free_memory: wait for a certain amount of memory to
become free in the system.

libxl_wait_for_memory_target: wait for a domain to reach its memory
target.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 0c3fa43db985 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Fri Sep 03 11:47:07 2010 +0100
+++ b/tools/libxl/libxl.c	Fri Sep 03 11:47:45 2010 +0100
@@ -2756,9 +2756,11 @@ static int libxl__fill_dom0_memory_info(
 {
     int rc;
     libxl_dominfo info;
+    libxl_physinfo physinfo;
     char *target = NULL, *endptr = NULL;
     char *target_path = "/local/domain/0/memory/target";
     char *max_path = "/local/domain/0/memory/static-max";
+    char *free_mem_slack_path = "/local/domain/0/memory/freemem-slack";
     xs_transaction_t t;
     libxl_ctx *ctx = libxl_gc_owner(gc);
 
@@ -2782,10 +2784,16 @@ retry_transaction:
     if (rc < 0)
         return rc;
 
+    rc = libxl_get_physinfo(ctx, &physinfo);
+    if (rc < 0)
+        return rc;
+
     libxl_xs_write(gc, t, target_path, "%"PRIu32,
             (uint32_t) info.current_memkb);
     libxl_xs_write(gc, t, max_path, "%"PRIu32,
             (uint32_t) info.max_memkb);
+    libxl_xs_write(gc, t, free_mem_slack_path, "%"PRIu32, (uint32_t)
+            (PAGE_TO_MEMKB(physinfo.total_pages) - info.current_memkb));
 
     *target_memkb = (uint32_t) info.current_memkb;
     rc = 0;
@@ -2799,6 +2807,33 @@ out:
     return rc;
 }
 
+/* returns how much memory should be left free in the system */
+static int libxl__get_free_memory_slack(libxl_gc *gc, uint32_t *free_mem_slack)
+{
+    int rc;
+    char *free_mem_slack_path = "/local/domain/0/memory/freemem-slack";
+    char *free_mem_slack_s, *endptr;
+    uint32_t target_memkb;
+
+retry:
+    free_mem_slack_s = libxl_xs_read(gc, XBT_NULL, free_mem_slack_path);
+    if (!free_mem_slack_s) {
+        rc = libxl__fill_dom0_memory_info(gc, &target_memkb);
+        if (rc < 0)
+            return rc;
+        goto retry;
+    } else {
+        *free_mem_slack = strtoul(free_mem_slack_s, &endptr, 10);
+        if (*endptr != '\0') {
+            XL_LOG_ERRNO(gc->owner, XL_LOG_ERROR,
+                    "invalid free_mem_slack %s from %s\n",
+                    free_mem_slack_s, free_mem_slack_path);
+            return ERROR_FAIL;
+        }
+    }
+    return 0;
+}
+
 int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid,
         int32_t target_memkb, int relative, int enforce)
 {
@@ -2957,6 +2992,102 @@ out:
     return rc;
 }
 
+int libxl_domain_need_memory(libxl_ctx *ctx, libxl_domain_build_info *b_info,
+        libxl_device_model_info *dm_info, uint32_t *need_memkb)
+{
+    *need_memkb = b_info->target_memkb;
+    if (b_info->hvm) {
+        *need_memkb += b_info->shadow_memkb + LIBXL_HVM_EXTRA_MEMORY;
+        if (strstr(dm_info->device_model, "stubdom-dm"))
+            *need_memkb += 32 * 1024;
+    } else
+        *need_memkb += LIBXL_PV_EXTRA_MEMORY;
+    if (*need_memkb % (2 * 1024))
+        *need_memkb += (2 * 1024) - (*need_memkb % (2 * 1024));
+    return 0;
+}
+
+int libxl_get_free_memory(libxl_ctx *ctx, uint32_t *memkb)
+{
+    int rc = 0;
+    libxl_physinfo info;
+    uint32_t freemem_slack;
+    libxl_gc gc = LIBXL_INIT_GC(ctx);
+
+    rc = libxl_get_physinfo(ctx, &info);
+    if (rc < 0)
+        goto out;
+    rc = libxl__get_free_memory_slack(&gc, &freemem_slack);
+    if (rc < 0)
+        goto out;
+
+    if ((info.free_pages + info.scrub_pages) * 4 > freemem_slack)
+        *memkb = (info.free_pages + info.scrub_pages) * 4 - freemem_slack;
+    else
+        *memkb = 0;
+
+out:
+    libxl_free_all(&gc);
+    return rc;
+}
+
+int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, uint32_t
+        memory_kb, int wait_secs)
+{
+    int rc = 0;
+    libxl_physinfo info;
+    uint32_t freemem_slack;
+    libxl_gc gc = LIBXL_INIT_GC(ctx);
+
+    rc = libxl__get_free_memory_slack(&gc, &freemem_slack);
+    if (rc < 0)
+        goto out;
+    while (wait_secs > 0) {
+        rc = libxl_get_physinfo(ctx, &info);
+        if (rc < 0)
+            goto out;
+        if (info.free_pages * 4 - freemem_slack >= memory_kb) {
+            rc = 0;
+            goto out;
+        }
+        wait_secs--;
+        sleep(1);
+    }
+    rc = ERROR_NOMEM;
+
+out:
+    libxl_free_all(&gc);
+    return rc;
+}
+
+int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs)
+{
+    int rc = 0;
+    uint32_t target_memkb = 0;
+    libxl_dominfo info;
+
+    do {
+        wait_secs--;
+        sleep(1);
+
+        rc = libxl_get_memory_target(ctx, domid, &target_memkb);
+        if (rc < 0)
+            goto out;
+
+        rc = libxl_domain_info(ctx, &info, domid);
+        if (rc < 0)
+            return rc;
+    } while (wait_secs > 0 && info.current_memkb > target_memkb);
+
+    if (info.current_memkb <= target_memkb)
+        rc = 0;
+    else
+        rc = ERROR_FAIL;
+
+out:
+    return 0;
+}
+
 int libxl_button_press(libxl_ctx *ctx, uint32_t domid, libxl_button button)
 {
     int rc = -1;
diff -r 0c3fa43db985 tools/libxl/libxl.h
--- a/tools/libxl/libxl.h	Fri Sep 03 11:47:07 2010 +0100
+++ b/tools/libxl/libxl.h	Fri Sep 03 11:47:45 2010 +0100
@@ -317,6 +317,15 @@ int libxl_domain_core_dump(libxl_ctx *ct
 int libxl_domain_setmaxmem(libxl_ctx *ctx, uint32_t domid, uint32_t target_memkb);
 int libxl_set_memory_target(libxl_ctx *ctx, uint32_t domid, int32_t target_memkb, int relative, int enforce);
 int libxl_get_memory_target(libxl_ctx *ctx, uint32_t domid, uint32_t *out_target);
+/* how much free memory in the system a domain needs to be built */
+int libxl_domain_need_memory(libxl_ctx *ctx, libxl_domain_build_info *b_info,
+        libxl_device_model_info *dm_info, uint32_t *need_memkb);
+/* how much free memory is available in the system */
+int libxl_get_free_memory(libxl_ctx *ctx, uint32_t *memkb);
+/* wait for a given amount of memory to be free in the system */
+int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, uint32_t memory_kb, int wait_secs);
+/* wait for the memory target of a domain to be reached */
+int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs);
 
 int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
 int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_constype type);
diff -r 0c3fa43db985 tools/libxl/libxl_internal.h
--- a/tools/libxl/libxl_internal.h	Fri Sep 03 11:47:07 2010 +0100
+++ b/tools/libxl/libxl_internal.h	Fri Sep 03 11:47:45 2010 +0100
@@ -41,6 +41,8 @@
 #define LIBXL_XENCONSOLE_LIMIT 1048576
 #define LIBXL_XENCONSOLE_PROTOCOL "vt100"
 #define LIBXL_MAXMEM_CONSTANT 1024
+#define LIBXL_PV_EXTRA_MEMORY 1024
+#define LIBXL_HVM_EXTRA_MEMORY 2048
 #define QEMU_SIGNATURE "QemuDeviceModelRecord"
 
 #define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))

[PATCH 6 of 12] libxl: introduce the concept of dom0 minimum memory

[stefano.stabellini]

Introduce a minimum value for the memory assigned to dom0.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r c46a6c796bb5 tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Fri Sep 03 11:47:46 2010 +0100
+++ b/tools/libxl/libxl.c	Fri Sep 03 11:49:10 2010 +0100
@@ -2906,6 +2906,12 @@ retry_transaction:
         abort = 1;
         goto out;
     }
+    if (!domid && new_target_memkb < LIBXL_MIN_DOM0_MEM) {
+        XL_LOG(ctx, XL_LOG_ERROR,
+                "new target for dom0 is below the minimum threshold\n");
+        abort = 1;
+        goto out;
+    }
 
     videoram_s = libxl_xs_read(&gc, t, libxl_sprintf(&gc,
                 "%s/memory/videoram", dompath));
diff -r c46a6c796bb5 tools/libxl/libxl_internal.h
--- a/tools/libxl/libxl_internal.h	Fri Sep 03 11:47:46 2010 +0100
+++ b/tools/libxl/libxl_internal.h	Fri Sep 03 11:49:10 2010 +0100
@@ -43,6 +43,7 @@
 #define LIBXL_MAXMEM_CONSTANT 1024
 #define LIBXL_PV_EXTRA_MEMORY 1024
 #define LIBXL_HVM_EXTRA_MEMORY 2048
+#define LIBXL_MIN_DOM0_MEM (128*1024)
 #define QEMU_SIGNATURE "QemuDeviceModelRecord"
 
 #define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))

[PATCH 7 of 12] xl: add a global configuration file

[stefano.stabellini]

Add a global configuration file: /etc/xen/xl.conf; the only option
currently parsed is autoballoon that is 1 by default.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 434f3315185a tools/examples/Makefile
--- a/tools/examples/Makefile	Fri Aug 27 12:43:18 2010 +0100
+++ b/tools/examples/Makefile	Fri Aug 27 14:09:51 2010 +0100
@@ -21,6 +21,7 @@ XEN_CONFIGS += xmexample.nbd
 XEN_CONFIGS += xmexample.vti
 XEN_CONFIGS += xend-pci-quirks.sxp
 XEN_CONFIGS += xend-pci-permissive.sxp
+XEN_CONFIGS += xl.conf
 
 .PHONY: all
 all:
diff -r 434f3315185a tools/examples/xl.conf
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/examples/xl.conf	Fri Aug 27 14:09:51 2010 +0100
@@ -0,0 +1,5 @@
+## Global XL config file ##
+
+# automatically balloon down dom0 when xen doesn't have enough free
+# memory to create a domain
+autoballon=1
diff -r 434f3315185a tools/libxl/xl.c
--- a/tools/libxl/xl.c	Fri Aug 27 12:43:18 2010 +0100
+++ b/tools/libxl/xl.c	Fri Aug 27 14:09:51 2010 +0100
@@ -29,18 +29,49 @@
 
 #include "libxl.h"
 #include "libxl_utils.h"
+#include "libxlutil.h"
 #include "xl.h"
 
 xentoollog_logger_stdiostream *logger;
+int autoballoon = 1;
 
 static xentoollog_level minmsglevel = XTL_PROGRESS;
 
+static void parse_global_config(const char *configfile,
+                              const char *configfile_data,
+                              int configfile_len)
+{
+    long l;
+    XLU_Config *config;
+    int e;
+
+    config = xlu_cfg_init(stderr, configfile);
+    if (!config) {
+        fprintf(stderr, "Failed to allocate for configuration\n");
+        exit(1);
+    }
+
+    e = xlu_cfg_readdata(config, configfile_data, configfile_len);
+    if (e) {
+        fprintf(stderr, "Failed to parse config file: %s\n", strerror(e));
+        exit(1);
+    }
+
+    if (!xlu_cfg_get_long (config, "autoballoon", &l))
+        autoballoon = l;
+
+    xlu_cfg_destroy(config);
+}
+ 
 int main(int argc, char **argv)
 {
     int opt = 0;
     char *cmd = 0;
     struct cmd_spec *cspec;
     int ret;
+    char *config_file;
+    void *config_data = 0;
+    int config_len = 0;
 
     while ((opt = getopt(argc, argv, "+v")) >= 0) {
         switch (opt) {
@@ -69,6 +100,21 @@ int main(int argc, char **argv)
         exit(1);
     }
 
+    /* Read global config file options */
+    ret = asprintf(&config_file, "%s/xl.conf", libxl_xen_config_dir_path());
+    if (ret < 0) {
+        fprintf(stderr, "memory allocation failed ret=%d, errno=%d\n", ret, errno);
+        exit(1);
+    }
+
+    ret = libxl_read_file_contents(&ctx, config_file,
+            &config_data, &config_len);
+    if (ret)
+        fprintf(stderr, "Failed to read config file: %s: %s\n",
+                config_file, strerror(errno));
+    parse_global_config(config_file, config_data, config_len);
+    free(config_file);
+
     /* Reset options for per-command use of getopt. */
     argv += optind;
     argc -= optind;
diff -r 434f3315185a tools/libxl/xl.h
--- a/tools/libxl/xl.h	Fri Aug 27 12:43:18 2010 +0100
+++ b/tools/libxl/xl.h	Fri Aug 27 14:09:51 2010 +0100
@@ -90,4 +90,7 @@ struct cmd_spec *cmdtable_lookup(const c
 extern libxl_ctx ctx;
 extern xentoollog_logger_stdiostream *logger;
 
+/* global options */
+extern int autoballoon;
+
 #endif /* XL_H */

[PATCH 8 of 12] introduce XEN_LOCK_DIR

[stefano.stabellini]

Introduce a OS dependent global variable XEN_LOCK_DIR that points at the
system directory that contains lock files.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 29dbbdc79759 Config.mk
--- a/Config.mk	Fri Aug 27 14:17:58 2010 +0100
+++ b/Config.mk	Fri Aug 27 14:23:13 2010 +0100
@@ -114,6 +114,7 @@ define buildmakevars2file-closure
 	echo "XENFIRMWAREDIR=\"$(XENFIRMWAREDIR)\"" >> $(1);           \
 	echo "XEN_CONFIG_DIR=\"$(XEN_CONFIG_DIR)\"" >> $(1);           \
 	echo "XEN_SCRIPT_DIR=\"$(XEN_SCRIPT_DIR)\"" >> $(1)
+	echo "XEN_LOCK_DIR=\"$(XEN_LOCK_DIR)\"" >> $(1)
 endef
 
 ifeq ($(debug),y)
diff -r 29dbbdc79759 config/NetBSD.mk
--- a/config/NetBSD.mk	Fri Aug 27 14:17:58 2010 +0100
+++ b/config/NetBSD.mk	Fri Aug 27 14:23:13 2010 +0100
@@ -7,4 +7,10 @@ LIBLEAFDIR_x86_64 = lib
 LIBEXEC = $(PREFIX)/libexec
 PRIVATE_BINDIR = $(BINDIR)
 
+ifeq ($(PREFIX),/usr)
+XEN_LOCK_DIR = /var/lib
+else
+XEN_LOCK_DIR = $(PREFIX)/var/lib
+endif
+
 WGET = ftp
diff -r 29dbbdc79759 config/StdGNU.mk
--- a/config/StdGNU.mk	Fri Aug 27 14:17:58 2010 +0100
+++ b/config/StdGNU.mk	Fri Aug 27 14:23:13 2010 +0100
@@ -45,8 +45,10 @@ PRIVATE_BINDIR = $(PRIVATE_PREFIX)/bin
 
 ifeq ($(PREFIX),/usr)
 CONFIG_DIR = /etc
+XEN_LOCK_DIR = /var/lock
 else
 CONFIG_DIR = $(PREFIX)/etc
+XEN_LOCK_DIR = $(PREFIX)/var/lock
 endif
 
 SYSCONFIG_DIR = $(CONFIG_DIR)/sysconfig
diff -r 29dbbdc79759 tools/libxl/libxl.h
--- a/tools/libxl/libxl.h	Fri Aug 27 14:17:58 2010 +0100
+++ b/tools/libxl/libxl.h	Fri Aug 27 14:23:13 2010 +0100
@@ -495,6 +495,7 @@ const char *libxl_private_bindir_path(vo
 const char *libxl_xenfirmwaredir_path(void);
 const char *libxl_xen_config_dir_path(void);
 const char *libxl_xen_script_dir_path(void);
+const char *libxl_lock_dir_path(void);
 
 #endif /* LIBXL_H */
 
diff -r 29dbbdc79759 tools/libxl/libxl_paths.c
--- a/tools/libxl/libxl_paths.c	Fri Aug 27 14:17:58 2010 +0100
+++ b/tools/libxl/libxl_paths.c	Fri Aug 27 14:23:13 2010 +0100
@@ -60,3 +60,7 @@ const char *libxl_xen_script_dir_path(vo
     return XEN_SCRIPT_DIR;
 }
 
+const char *libxl_lock_dir_path(void)
+{
+    return XEN_LOCK_DIR;
+}

[PATCH 9 of 12] xl: free memory before building a domain

[stefano.stabellini]

Free the needed amount of memory before proceeding with the domain
build.
Use a filelock to prevent other xl instances from conflicting during
this operation.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 34af3695004d tools/examples/xl.conf
--- a/tools/examples/xl.conf	Fri Sep 03 11:49:34 2010 +0100
+++ b/tools/examples/xl.conf	Fri Sep 03 11:50:14 2010 +0100
@@ -3,3 +3,6 @@
 # automatically balloon down dom0 when xen doesn't have enough free
 # memory to create a domain
 autoballon=1
+
+# full path of the lockfile used by xl during domain creation
+#lockfile="/var/lock/xl"
diff -r 34af3695004d tools/libxl/xl.c
--- a/tools/libxl/xl.c	Fri Sep 03 11:49:34 2010 +0100
+++ b/tools/libxl/xl.c	Fri Sep 03 11:50:14 2010 +0100
@@ -34,6 +34,7 @@
 
 xentoollog_logger_stdiostream *logger;
 int autoballoon = 1;
+char *lockfile;
 
 static xentoollog_level minmsglevel = XTL_PROGRESS;
 
@@ -44,6 +45,7 @@ static void parse_global_config(const ch
     long l;
     XLU_Config *config;
     int e;
+    const char *buf;
 
     config = xlu_cfg_init(stderr, configfile);
     if (!config) {
@@ -60,6 +62,16 @@ static void parse_global_config(const ch
     if (!xlu_cfg_get_long (config, "autoballoon", &l))
         autoballoon = l;
 
+    if (!xlu_cfg_get_string (config, "lockfile", &buf))
+        lockfile = strdup(buf);
+    else {
+        e = asprintf(&lockfile, "%s/xl", (char *)libxl_lock_dir_path());
+        if (e < 0) {
+            fprintf(stderr, "asprintf memory allocation failed\n");
+            exit(1);
+        }
+    }
+
     xlu_cfg_destroy(config);
 }
  
diff -r 34af3695004d tools/libxl/xl.h
--- a/tools/libxl/xl.h	Fri Sep 03 11:49:34 2010 +0100
+++ b/tools/libxl/xl.h	Fri Sep 03 11:50:14 2010 +0100
@@ -92,5 +92,6 @@ extern xentoollog_logger_stdiostream *lo
 
 /* global options */
 extern int autoballoon;
+extern char *lockfile;
 
 #endif /* XL_H */
diff -r 34af3695004d tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Fri Sep 03 11:49:34 2010 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Fri Sep 03 11:50:14 2010 +0100
@@ -68,6 +68,7 @@ libxl_ctx ctx;
 /* when we operate on a domain, it is this one: */
 static uint32_t domid;
 static const char *common_domname;
+static int fd_lock = -1;
 
 
 static const char savefileheader_magic[32]=
@@ -234,6 +235,65 @@ static void find_domain(const char *p)
     common_domname = was_name ? p : libxl_domid_to_name(&ctx, domid);
 }
 
+static int acquire_lock(void)
+{
+    int rc;
+    struct flock fl;
+
+    /* lock already acquired */
+    if (fd_lock >= 0)
+        return ERROR_INVAL;
+
+    fl.l_type = F_WRLCK;
+    fl.l_whence = SEEK_SET;
+    fl.l_start = 0;
+    fl.l_len = 0;
+    fd_lock = open(lockfile, O_WRONLY|O_CREAT, S_IWUSR);
+    if (fd_lock < 0) {
+        fprintf(stderr, "cannot open the lockfile %s errno=%d\n", lockfile, errno);
+        return ERROR_FAIL;
+    }
+get_lock:
+    rc = fcntl(fd_lock, F_SETLKW, &fl);
+    if (rc < 0 && errno == EINTR)
+        goto get_lock;
+    if (rc < 0) {
+        fprintf(stderr, "cannot acquire lock %s errno=%d\n", lockfile, errno);
+        rc = ERROR_FAIL;
+    } else
+        rc = 0;
+    return rc;
+}
+
+static int release_lock(void)
+{
+    int rc;
+    struct flock fl;
+
+    /* lock not acquired */
+    if (fd_lock < 0)
+        return ERROR_INVAL;
+
+release_lock:
+    fl.l_type = F_UNLCK;
+    fl.l_whence = SEEK_SET;
+    fl.l_start = 0;
+    fl.l_len = 0;
+
+    rc = fcntl(fd_lock, F_SETLKW, &fl);
+    if (rc < 0 && errno == EINTR)
+        goto release_lock;
+    if (rc < 0) {
+        fprintf(stderr, "cannot release lock %s, errno=%d\n", lockfile, errno);
+        rc = ERROR_FAIL;
+    } else
+        rc = 0;
+    close(fd_lock);
+    fd_lock = -1;
+
+    return rc;
+}
+
 #define LOG(_f, _a...)   dolog(__FILE__, __LINE__, __func__, _f "\n", ##_a)
 
 static void dolog(const char *file, int line, const char *func, char *fmt, ...)
@@ -1223,6 +1283,48 @@ struct domain_create {
     char **migration_domname_r;
 };
 
+static int freemem(libxl_domain_build_info *b_info, libxl_device_model_info *dm_info)
+{
+    int rc, retries = 3;
+    uint32_t need_memkb, free_memkb;
+
+    if (!autoballoon)
+        return 0;
+
+    rc = libxl_domain_need_memory(&ctx, b_info, dm_info, &need_memkb);
+    if (rc < 0)
+        return rc;
+
+    do {
+        rc = libxl_get_free_memory(&ctx, &free_memkb);
+        if (rc < 0)
+            return rc;
+
+        if (free_memkb >= need_memkb)
+            return 0;
+
+        rc = libxl_set_memory_target(&ctx, 0, free_memkb - need_memkb, 1, 0);
+        if (rc < 0)
+            return rc;
+
+        rc = libxl_wait_for_free_memory(&ctx, domid, need_memkb, 10);
+        if (!rc)
+            return 0;
+        else if (rc != ERROR_NOMEM)
+            return rc;
+
+        /* the memory target has been reached but the free memory is still
+         * not enough: loop over again */
+        rc = libxl_wait_for_memory_target(&ctx, 0, 1);
+        if (rc < 0)
+            return rc;
+
+        retries--;
+    } while (retries > 0);
+
+    return ERROR_NOMEM;
+}
+
 static int create_domain(struct domain_create *dom_info)
 {
     struct domain_config d_config;
@@ -1381,6 +1483,17 @@ static int create_domain(struct domain_c
 start:
     domid = 0;
 
+    rc = acquire_lock();
+    if (rc < 0)
+        goto error_out;
+
+    ret = freemem(&d_config.b_info, &dm_info);
+    if (ret < 0) {
+        fprintf(stderr, "failed to free memory for the domain\n");
+        ret = ERROR_FAIL;
+        goto error_out;
+    }
+
     ret = libxl_domain_make(&ctx, &d_config.c_info, &domid);
     if (ret) {
         fprintf(stderr, "cannot make domain: %d\n", ret);
@@ -1489,6 +1602,8 @@ start:
             goto error_out;
     }
 
+    release_lock();
+
     if (!paused)
         libxl_domain_unpause(&ctx, domid);
 
@@ -1626,6 +1741,7 @@ start:
     }
 
 error_out:
+    release_lock();
     if (domid)
         libxl_domain_destroy(&ctx, domid, 0);

[PATCH 10 of 12] libxl: proper accounting for the videoram

[stefano.stabellini]

Remove the videoram from the memory target of the domains consistently,
leave the total amount of memory in maxmem.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 6624c7399d5f tools/libxl/libxl.c
--- a/tools/libxl/libxl.c	Fri Sep 03 11:50:14 2010 +0100
+++ b/tools/libxl/libxl.c	Fri Sep 03 11:51:22 2010 +0100
@@ -2930,12 +2930,13 @@ retry_transaction:
         }
     }
 
+    new_target_memkb -= videoram;
     rc = xc_domain_memory_set_pod_target(ctx->xch, domid,
-            (new_target_memkb - videoram) / 4, NULL, NULL, NULL);
+            new_target_memkb / 4, NULL, NULL, NULL);
     if (rc != 0) {
         XL_LOG_ERRNO(ctx, XL_LOG_ERROR,
                 "xc_domain_memory_set_pod_target domid=%d, memkb=%d "
-                "failed rc=%d\n", domid, (new_target_memkb - videoram) / 4,
+                "failed rc=%d\n", domid, new_target_memkb / 4,
                 rc);
         abort = 1;
         goto out;
diff -r 6624c7399d5f tools/libxl/libxl_dom.c
--- a/tools/libxl/libxl_dom.c	Fri Sep 03 11:50:14 2010 +0100
+++ b/tools/libxl/libxl_dom.c	Fri Sep 03 11:51:22 2010 +0100
@@ -100,7 +100,7 @@ int build_post(libxl_ctx *ctx, uint32_t 
     ents[0] = "memory/static-max";
     ents[1] = libxl_sprintf(&gc, "%d", info->max_memkb);
     ents[2] = "memory/target";
-    ents[3] = libxl_sprintf(&gc, "%d", info->target_memkb);
+    ents[3] = libxl_sprintf(&gc, "%d", info->target_memkb - info->video_memkb);
     ents[4] = "memory/videoram";
     ents[5] = libxl_sprintf(&gc, "%d", info->video_memkb);
     ents[6] = "domid";

[PATCH 11 of 12] xl: parse maxmem parameter in VM config files

[stefano.stabellini]

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 92b36bfe3088 tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c	Thu Sep 02 15:39:48 2010 +0100
+++ b/tools/libxl/xl_cmdimpl.c	Thu Sep 02 17:47:23 2010 +0100
@@ -692,6 +692,9 @@ static void parse_config_data(const char
         b_info->target_memkb = b_info->max_memkb;
     }
 
+    if (!xlu_cfg_get_long (config, "maxmem", &l))
+        b_info->max_memkb = l * 1024;
+
     if (xlu_cfg_get_string (config, "on_poweroff", &buf))
         buf = "destroy";
     if (!parse_action_on_shutdown(buf, &d_config->on_poweroff)) {

[PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model

[stefano.stabellini]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 5240 bytes --]

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r 12923cf32127 tools/libxl/libxl_memory.txt
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/libxl/libxl_memory.txt	Fri Sep 03 12:10:58 2010 +0100
@@ -0,0 +1,70 @@
+/* === Domain memory breakdown: HVM guests ==================================
+
+             ╤  ╔══════════╗                                     ╤            
+             │  ║ shadow   ║                                     │            
+             │  ╠══════════╣                                     │            
+    overhead │  ║ extra    ║                                     │            
+             │  ║ external ║                                     │            
+             │  ╠══════════╣                          ╤          │            
+             │  ║ extra    ║                          │          │            
+             │  ║ internal ║                          │          │            
+             ╪  ╠══════════╣                ╤         │          │ footprint  
+             │  ║ video    ║                │         │          │            
+             │  ╠══════════╣  ╤    ╤        │         │ xen      │            
+             │  ║          ║  │    │        │ actual  │ maximum  │            
+             │  ║          ║  │    │        │ target  │          │            
+             │  ║ guest    ║  │    │ build  │         │          │            
+             │  ║          ║  │    │ start  │         │          │            
+      static │  ║          ║  │    │        │         │          │            
+     maximum │  ╟──────────╢  │    ╧        ╧         ╧          ╧            
+             │  ║          ║  │                                               
+             │  ║          ║  │                                               
+             │  ║ balloon  ║  │ build                                         
+             │  ║          ║  │ maximum                                       
+             │  ║          ║  │                                               
+             ╧  ╚══════════╝  ╧                                               
+
+
+    extra internal = LIBXL_MAXMEM_CONSTANT
+    extra external = LIBXL_HVM_EXTRA_MEMORY
+    shadow = libxl_domain_build_info.shadow_memkb
+    static maximum = libxl_domain_build_info.max_memkb
+    video = libxl_domain_build_info.video_memkb
+    build start = libxl_domain_build_info.target_memkb
+    libxl_domain_setmaxmem -> xen maximum
+    libxl_set_memory_target -> actual target
+
+
+ === Domain memory breakdown: PV guests ==================================
+
+
+             ╤  ╔══════════╗                                     ╤            
+    overhead │  ║ extra    ║                                     │            
+             │  ║ external ║                                     │            
+             │  ╠══════════╣                          ╤          │            
+             │  ║ extra    ║                          │          │            
+             │  ║ internal ║                          │          │            
+             ╪  ╠══════════╣  ╤    ╤        ╤         │          │ footprint  
+             │  ║          ║  │    │        │         │ xen      │            
+             │  ║          ║  │    │        │ actual  │ maximum  │ 
+             │  ║ guest    ║  │    │ build  │ target  │          │            
+             │  ║          ║  │    │ start  │         │          │            
+      static │  ║          ║  │    │        │         │          │            
+     maximum │  ╟──────────╢  │    ╧        ╧         ╧          ╧            
+             │  ║          ║  │                                               
+             │  ║          ║  │                                               
+             │  ║ balloon  ║  │ build                                         
+             │  ║          ║  │ maximum                                       
+             │  ║          ║  │                                               
+             ╧  ╚══════════╝  ╧                                               
+
+
+    extra internal = LIBXL_MAXMEM_CONSTANT
+    extra external = LIBXL_PV_EXTRA_MEMORY
+    static maximum = libxl_domain_build_info.max_memkb
+    build start = libxl_domain_build_info.target_memkb
+    libxl_domain_setmaxmem -> xen maximum
+    libxl_set_memory_target -> actual target
+
+
+ ========================================================================= */

Re: [PATCH 7 of 12] xl: add a global configuration file

[Ian Jackson]

stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH 7 of 12] xl: add a global configuration file"):
> Add a global configuration file: /etc/xen/xl.conf; the only option
> currently parsed is autoballoon that is 1 by default.

You've changed your patch 8 to not have an uncommented value to set
the default to the default in the config file, but not this one ...

Ian.

Re: [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model

[Ian Jackson]

stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model"):
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

This patch is full of high-bit-set characters.

Ian.

Re: [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model

[Stefano Stabellini]

On Fri, 3 Sep 2010, Ian Jackson wrote:
> stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model"):
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> 
> This patch is full of high-bit-set characters.
 
This is a document, not code, I think it is fine to use UTF-8 here,
especially if non-ASCII characters are used to create a nice and useful
diagram (BTW the diagram comes from one of the xapi docs).

Re: [PATCH 7 of 12] xl: add a global configuration file

[Stefano Stabellini]

On Fri, 3 Sep 2010, Ian Jackson wrote:
> stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH 7 of 12] xl: add a global configuration file"):
> > Add a global configuration file: /etc/xen/xl.conf; the only option
> > currently parsed is autoballoon that is 1 by default.
> 
> You've changed your patch 8 to not have an uncommented value to set
> the default to the default in the config file, but not this one ...
> 
 
thanks for spotting this, I am appending an updated version (you might
need to fix PATCH 9 because it might have problems applying after this
change, but the fix is so trivial that I am not going to post another
update of the series just for that):

---

Add a global configuration file: /etc/xen/xl.conf; the only option
currently parsed is autoballoon that is 1 by default.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

diff -r b889336acffb tools/examples/Makefile
--- a/tools/examples/Makefile	Mon Sep 06 11:49:29 2010 +0100
+++ b/tools/examples/Makefile	Mon Sep 06 11:50:20 2010 +0100
@@ -21,6 +21,7 @@ XEN_CONFIGS += xmexample.nbd
 XEN_CONFIGS += xmexample.vti
 XEN_CONFIGS += xend-pci-quirks.sxp
 XEN_CONFIGS += xend-pci-permissive.sxp
+XEN_CONFIGS += xl.conf
 
 .PHONY: all
 all:
diff -r b889336acffb tools/examples/xl.conf
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/examples/xl.conf	Mon Sep 06 11:50:20 2010 +0100
@@ -0,0 +1,5 @@
+## Global XL config file ##
+
+# automatically balloon down dom0 when xen doesn't have enough free
+# memory to create a domain
+#autoballon=1
diff -r b889336acffb tools/libxl/xl.c
--- a/tools/libxl/xl.c	Mon Sep 06 11:49:29 2010 +0100
+++ b/tools/libxl/xl.c	Mon Sep 06 11:50:20 2010 +0100
@@ -29,18 +29,49 @@
 
 #include "libxl.h"
 #include "libxl_utils.h"
+#include "libxlutil.h"
 #include "xl.h"
 
 xentoollog_logger_stdiostream *logger;
+int autoballoon = 1;
 
 static xentoollog_level minmsglevel = XTL_PROGRESS;
 
+static void parse_global_config(const char *configfile,
+                              const char *configfile_data,
+                              int configfile_len)
+{
+    long l;
+    XLU_Config *config;
+    int e;
+
+    config = xlu_cfg_init(stderr, configfile);
+    if (!config) {
+        fprintf(stderr, "Failed to allocate for configuration\n");
+        exit(1);
+    }
+
+    e = xlu_cfg_readdata(config, configfile_data, configfile_len);
+    if (e) {
+        fprintf(stderr, "Failed to parse config file: %s\n", strerror(e));
+        exit(1);
+    }
+
+    if (!xlu_cfg_get_long (config, "autoballoon", &l))
+        autoballoon = l;
+
+    xlu_cfg_destroy(config);
+}
+ 
 int main(int argc, char **argv)
 {
     int opt = 0;
     char *cmd = 0;
     struct cmd_spec *cspec;
     int ret;
+    char *config_file;
+    void *config_data = 0;
+    int config_len = 0;
 
     while ((opt = getopt(argc, argv, "+v")) >= 0) {
         switch (opt) {
@@ -69,6 +100,21 @@ int main(int argc, char **argv)
         exit(1);
     }
 
+    /* Read global config file options */
+    ret = asprintf(&config_file, "%s/xl.conf", libxl_xen_config_dir_path());
+    if (ret < 0) {
+        fprintf(stderr, "memory allocation failed ret=%d, errno=%d\n", ret, errno);
+        exit(1);
+    }
+
+    ret = libxl_read_file_contents(&ctx, config_file,
+            &config_data, &config_len);
+    if (ret)
+        fprintf(stderr, "Failed to read config file: %s: %s\n",
+                config_file, strerror(errno));
+    parse_global_config(config_file, config_data, config_len);
+    free(config_file);
+
     /* Reset options for per-command use of getopt. */
     argv += optind;
     argc -= optind;
diff -r b889336acffb tools/libxl/xl.h
--- a/tools/libxl/xl.h	Mon Sep 06 11:49:29 2010 +0100
+++ b/tools/libxl/xl.h	Mon Sep 06 11:50:20 2010 +0100
@@ -90,4 +90,7 @@ struct cmd_spec *cmdtable_lookup(const c
 extern libxl_ctx ctx;
 extern xentoollog_logger_stdiostream *logger;
 
+/* global options */
+extern int autoballoon;
+
 #endif /* XL_H */

Re: [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model

[Ian Jackson]

Stefano Stabellini writes ("Re: [Xen-devel] [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model"):
> On Fri, 3 Sep 2010, Ian Jackson wrote:
> > stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model"):
> > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > 
> > This patch is full of high-bit-set characters.
>  
> This is a document, not code, I think it is fine to use UTF-8 here,
> especially if non-ASCII characters are used to create a nice and useful
> diagram (BTW the diagram comes from one of the xapi docs).

There is no need to make this document difficult to access for people
who are for any reason not using utf-8.

Would you like me to make a version which uses traditional ascii-art
for the boxes etc. ?

Ian.

Re: [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model

[Stefano Stabellini]

On Tue, 7 Sep 2010, Ian Jackson wrote:
> Stefano Stabellini writes ("Re: [Xen-devel] [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model"):
> > On Fri, 3 Sep 2010, Ian Jackson wrote:
> > > stefano.stabellini@eu.citrix.com writes ("[Xen-devel] [PATCH 12 of 12] libxl: add a detailed explanation of the libxl domain memory model"):
> > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > 
> > > This patch is full of high-bit-set characters.
> >  
> > This is a document, not code, I think it is fine to use UTF-8 here,
> > especially if non-ASCII characters are used to create a nice and useful
> > diagram (BTW the diagram comes from one of the xapi docs).
> 
> There is no need to make this document difficult to access for people
> who are for any reason not using utf-8.
> 
> Would you like me to make a version which uses traditional ascii-art
> for the boxes etc. ?
 
sure, go ahead.