* [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
@ 2014-05-01 12:57 ` Wei Liu
2014-05-06 12:45 ` Ian Campbell
2014-05-01 12:57 ` [PATCH V4 02/24] libxl: fix memory leak in libxl_cpuid_dispose Wei Liu
` (23 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:57 UTC (permalink / raw)
To: xen-devel
Cc: Wei Liu, ian.campbell, Dario Faggioli, ian.jackson,
Daniel De Graaf, Juergen Gross
This patch pushes parsing of "init_seclabel", "seclabel",
"device_model_stubdomain_seclabel" and "pool" down to libxl level.
Originally the parsing is done in xl level, which is not ideal because
libxl won't have the truely relavent information. Now libxl holds
important information by itself. This is useful when we do serialization
and deserialization of domain configurations on libxl level.
The libxl IDL is extended to hold the string of labels and pool name.
And if there those strings are present they take precedence over the
numeric representations.
As all relavent structures have a field called X_name / X_label now, a
string is also copied there so that we can use it directly.
In order to be compatible with users of older versions of libxl, this
patch also defines LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING. If it is
defined, those strings are available. And if those strings are not NULL,
libxl will do the parsing and ignore the numeric values.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Juergen Gross <juergen.gross@ts.fujitsu.com>
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
tools/libxl/libxl.c | 19 ++++--
tools/libxl/libxl.h | 12 ++++
tools/libxl/libxl_create.c | 57 ++++++++++++++++
tools/libxl/libxl_dm.c | 4 ++
tools/libxl/libxl_types.idl | 6 ++
tools/libxl/libxl_utils.c | 31 +++++++++
tools/libxl/libxl_utils.h | 3 +
tools/libxl/xl_cmdimpl.c | 150 ++++++++++---------------------------------
tools/libxl/xl_sxp.c | 7 +-
9 files changed, 164 insertions(+), 125 deletions(-)
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 30b0b06..9a90f40 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -520,12 +520,18 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid,
return 0;
}
-static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
+static void xcinfo2xlinfo(libxl_ctx *ctx,
+ const xc_domaininfo_t *xcinfo,
libxl_dominfo *xlinfo)
{
+ size_t size;
+
memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
xlinfo->domid = xcinfo->domain;
xlinfo->ssidref = xcinfo->ssidref;
+ if (libxl_flask_sid_to_context(ctx, xlinfo->ssidref,
+ &xlinfo->ssid_label, &size) < 0)
+ xlinfo->ssid_label = NULL;
xlinfo->dying = !!(xcinfo->flags&XEN_DOMINF_dying);
xlinfo->shutdown = !!(xcinfo->flags&XEN_DOMINF_shutdown);
@@ -572,7 +578,7 @@ libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out)
}
for (i = 0; i < ret; i++) {
- xcinfo2xlinfo(&info[i], &ptr[i]);
+ xcinfo2xlinfo(ctx, &info[i], &ptr[i]);
}
*nb_domain_out = ret;
return ptr;
@@ -591,7 +597,7 @@ int libxl_domain_info(libxl_ctx *ctx, libxl_dominfo *info_r,
if (ret==0 || xcinfo.domain != domid) return ERROR_INVAL;
if (info_r)
- xcinfo2xlinfo(&xcinfo, info_r);
+ xcinfo2xlinfo(ctx, &xcinfo, info_r);
return 0;
}
@@ -619,6 +625,11 @@ static int cpupool_info(libxl__gc *gc,
}
info->poolid = xcinfo->cpupool_id;
+ info->pool_name = libxl_cpupoolid_to_name(CTX, info->poolid);
+ if (!info->pool_name) {
+ rc = ERROR_NOMEM;
+ goto out;
+ }
info->sched = xcinfo->sched_id;
info->n_dom = xcinfo->n_dom;
rc = libxl_cpu_bitmap_alloc(CTX, &info->cpumap, 0);
@@ -4156,7 +4167,7 @@ retry_transaction:
abort_transaction = 1;
goto out;
}
- xcinfo2xlinfo(&info, &ptr);
+ xcinfo2xlinfo(ctx, &info, &ptr);
uuid = libxl__uuid2string(gc, ptr.uuid);
libxl__xs_write(gc, t, libxl__sprintf(gc, "/vm/%s/memory", uuid),
"%"PRIu32, new_target_memkb / 1024);
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index b2c3015..2f14d26 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -462,6 +462,18 @@
#define LIBXL_EXTERNAL_CALLERS_ONLY /* disappears for callers outside libxl */
#endif
+/*
+ * LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING
+ *
+ * If this is defined, then libxl IDL contains strings of XSM security
+ * labels and CPU pool name.
+ *
+ * If those strings are not NULL, libxl will overwrite the numeric
+ * representations of these configurations regardless if they've been
+ * set by the caller, because libxl will do the parsing by itself.
+ */
+#define LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING 1
+
typedef uint8_t libxl_mac[6];
#define LIBXL_MAC_FMT "%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx"
#define LIBXL_MAC_FMTLEN ((2*6)+5) /* 6 hex bytes plus 5 colons */
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index d015cf4..fe3bdd2 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -724,6 +724,63 @@ static void initiate_domain_create(libxl__egc *egc,
domid = 0;
+ if (d_config->c_info.ssid_label) {
+ char *s = d_config->c_info.ssid_label;
+ ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
+ &d_config->c_info.ssidref);
+ if (ret) {
+ if (errno == ENOSYS) {
+ LOG(WARN, "XSM Disabled: init_seclabel not supported");
+ ret = 0;
+ } else {
+ LOG(ERROR, "Invalid init_seclabel: %s", s);
+ goto error_out;
+ }
+ }
+ }
+
+ if (d_config->b_info.exec_ssid_label) {
+ char *s = d_config->b_info.exec_ssid_label;
+ ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
+ &d_config->b_info.exec_ssidref);
+ if (ret) {
+ if (errno == ENOSYS) {
+ LOG(WARN, "XSM Disabled: seclabel not supported");
+ ret = 0;
+ } else {
+ LOG(ERROR, "Invalid seclabel: %s", s);
+ goto error_out;
+ }
+ }
+ }
+
+ if (d_config->b_info.device_model_ssid_label) {
+ char *s = d_config->b_info.device_model_ssid_label;
+ ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
+ &d_config->b_info.device_model_ssidref);
+ if (ret) {
+ if (errno == ENOSYS) {
+ LOG(WARN,"XSM Disabled: device_model_stubdomain_seclabel not supported");
+ ret = 0;
+ } else {
+ LOG(ERROR, "Invalid device_model_stubdomain_seclabel: %s", s);
+ goto error_out;
+ }
+ }
+ }
+
+ if (d_config->c_info.pool_name) {
+ d_config->c_info.poolid = -1;
+ libxl_cpupool_qualifier_to_cpupoolid(ctx, d_config->c_info.pool_name,
+ &d_config->c_info.poolid,
+ NULL);
+ }
+ if (!libxl_cpupoolid_is_valid(ctx, d_config->c_info.poolid)) {
+ LOG(ERROR, "Illegal pool specified: %s", d_config->c_info.pool_name);
+ ret = ERROR_INVAL;
+ goto error_out;
+ }
+
/* If target_memkb is smaller than max_memkb, the subsequent call
* to libxc when building HVM domain will enable PoD mode.
*/
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 8abed7b..ea1ce38 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -909,7 +909,11 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
dm_config->c_info.type = LIBXL_DOMAIN_TYPE_PV;
dm_config->c_info.name = libxl__stub_dm_name(gc,
libxl__domid_to_name(gc, guest_domid));
+ /* When we are here to launch stubdom, ssidref is a valid value
+ * already, no need to parse it again.
+ */
dm_config->c_info.ssidref = guest_config->b_info.device_model_ssidref;
+ dm_config->c_info.ssid_label = NULL;
libxl_uuid_generate(&dm_config->c_info.uuid);
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 0f7bbf8..5f4341f 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -215,6 +215,7 @@ libxl_dominfo = Struct("dominfo",[
("uuid", libxl_uuid),
("domid", libxl_domid),
("ssidref", uint32),
+ ("ssid_label", string),
("running", bool),
("blocked", bool),
("paused", bool),
@@ -240,6 +241,7 @@ libxl_dominfo = Struct("dominfo",[
libxl_cpupoolinfo = Struct("cpupoolinfo", [
("poolid", uint32),
+ ("pool_name", string),
("sched", libxl_scheduler),
("n_dom", uint32),
("cpumap", libxl_bitmap)
@@ -270,11 +272,13 @@ libxl_domain_create_info = Struct("domain_create_info",[
("hap", libxl_defbool),
("oos", libxl_defbool),
("ssidref", uint32),
+ ("ssid_label", string),
("name", string),
("uuid", libxl_uuid),
("xsdata", libxl_key_value_list),
("platformdata", libxl_key_value_list),
("poolid", uint32),
+ ("pool_name", string),
("run_hotplug_scripts",libxl_defbool),
("pvh", libxl_defbool),
("driver_domain",libxl_defbool),
@@ -309,6 +313,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
("shadow_memkb", MemKB),
("rtc_timeoffset", uint32),
("exec_ssidref", uint32),
+ ("exec_ssid_label", string),
("localtime", libxl_defbool),
("disable_migrate", libxl_defbool),
("cpuid", libxl_cpuid_policy_list),
@@ -319,6 +324,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
# if you set device_model you must set device_model_version too
("device_model", string),
("device_model_ssidref", uint32),
+ ("device_model_ssid_label", string),
# extra parameters pass directly to qemu, NULL terminated
("extra", libxl_string_list),
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 1f334f2..476921e 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -108,6 +108,37 @@ int libxl_domain_qualifier_to_domid(libxl_ctx *ctx, const char *name,
return rv;
}
+static int qualifier_to_id(const char *p, uint32_t *id_r)
+{
+ int i, alldigit;
+
+ alldigit = 1;
+ for (i = 0; p[i]; i++) {
+ if (!isdigit((uint8_t)p[i])) {
+ alldigit = 0;
+ break;
+ }
+ }
+
+ if (i > 0 && alldigit) {
+ *id_r = strtoul(p, NULL, 10);
+ return 0;
+ } else {
+ /* check here if it's a uuid and do proper conversion */
+ }
+ return 1;
+}
+
+int libxl_cpupool_qualifier_to_cpupoolid(libxl_ctx *ctx, const char *p,
+ uint32_t *poolid_r,
+ int *was_name_r)
+{
+ int was_name;
+
+ was_name = qualifier_to_id(p, poolid_r);
+ if (was_name_r) *was_name_r = was_name;
+ return was_name ? libxl_name_to_cpupoolid(ctx, p, poolid_r) : 0;
+}
char *libxl_cpupoolid_to_name(libxl_ctx *ctx, uint32_t poolid)
{
diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h
index e37fb89..8bfb81b 100644
--- a/tools/libxl/libxl_utils.h
+++ b/tools/libxl/libxl_utils.h
@@ -23,6 +23,9 @@ unsigned long libxl_get_required_shadow_memory(unsigned long maxmem_kb, unsigned
int libxl_name_to_domid(libxl_ctx *ctx, const char *name, uint32_t *domid);
int libxl_domain_qualifier_to_domid(libxl_ctx *ctx, const char *name, uint32_t *domid);
char *libxl_domid_to_name(libxl_ctx *ctx, uint32_t domid);
+int libxl_cpupool_qualifier_to_cpupoolid(libxl_ctx *ctx, const char *p,
+ uint32_t *poolid_r,
+ int *was_name_r);
int libxl_name_to_cpupoolid(libxl_ctx *ctx, const char *name, uint32_t *poolid);
char *libxl_cpupoolid_to_name(libxl_ctx *ctx, uint32_t poolid);
int libxl_cpupoolid_is_valid(libxl_ctx *ctx, uint32_t poolid);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 0b38b32..289ea9a 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -158,38 +158,6 @@ struct domain_create {
};
-
-static int qualifier_to_id(const char *p, uint32_t *id_r)
-{
- int i, alldigit;
-
- alldigit = 1;
- for (i = 0; p[i]; i++) {
- if (!isdigit((uint8_t)p[i])) {
- alldigit = 0;
- break;
- }
- }
-
- if (i > 0 && alldigit) {
- *id_r = strtoul(p, NULL, 10);
- return 0;
- } else {
- /* check here if it's a uuid and do proper conversion */
- }
- return 1;
-}
-
-static int cpupool_qualifier_to_cpupoolid(const char *p, uint32_t *poolid_r,
- int *was_name_r)
-{
- int was_name;
-
- was_name = qualifier_to_id(p, poolid_r);
- if (was_name_r) *was_name_r = was_name;
- return was_name ? libxl_name_to_cpupoolid(ctx, p, poolid_r) : 0;
-}
-
static uint32_t find_domain(const char *p) __attribute__((warn_unused_result));
static uint32_t find_domain(const char *p)
{
@@ -759,35 +727,14 @@ static void parse_config_data(const char *config_source,
exit(1);
}
- if (!xlu_cfg_get_string (config, "init_seclabel", &buf, 0)) {
- e = libxl_flask_context_to_sid(ctx, (char *)buf, strlen(buf),
- &c_info->ssidref);
- if (e) {
- if (errno == ENOSYS) {
- fprintf(stderr, "XSM Disabled: init_seclabel not supported\n");
- } else {
- fprintf(stderr, "Invalid init_seclabel: %s\n", buf);
- exit(1);
- }
- }
- }
+ if (!xlu_cfg_get_string (config, "init_seclabel", &buf, 0))
+ c_info->ssid_label = strdup(buf);
if (!xlu_cfg_get_string (config, "seclabel", &buf, 0)) {
- uint32_t ssidref;
- e = libxl_flask_context_to_sid(ctx, (char *)buf, strlen(buf),
- &ssidref);
- if (e) {
- if (errno == ENOSYS) {
- fprintf(stderr, "XSM Disabled: seclabel not supported\n");
- } else {
- fprintf(stderr, "Invalid seclabel: %s\n", buf);
- exit(1);
- }
- } else if (c_info->ssidref) {
- b_info->exec_ssidref = ssidref;
- } else {
- c_info->ssidref = ssidref;
- }
+ if (c_info->ssid_label)
+ b_info->exec_ssid_label = strdup(buf);
+ else
+ c_info->ssid_label = strdup(buf);
}
libxl_defbool_set(&c_info->run_hotplug_scripts, run_hotplug_scripts);
@@ -815,14 +762,8 @@ static void parse_config_data(const char *config_source,
xlu_cfg_get_defbool(config, "oos", &c_info->oos, 0);
- if (!xlu_cfg_get_string (config, "pool", &buf, 0)) {
- c_info->poolid = -1;
- cpupool_qualifier_to_cpupoolid(buf, &c_info->poolid, NULL);
- }
- if (!libxl_cpupoolid_is_valid(ctx, c_info->poolid)) {
- fprintf(stderr, "Illegal pool specified\n");
- exit(1);
- }
+ if (!xlu_cfg_get_string (config, "pool", &buf, 0))
+ c_info->pool_name = strdup(buf);
libxl_domain_build_info_init_type(b_info, c_info->type);
if (blkdev_start)
@@ -1614,20 +1555,9 @@ skip_vfb:
&b_info->device_model_stubdomain, 0);
if (!xlu_cfg_get_string (config, "device_model_stubdomain_seclabel",
- &buf, 0)) {
- e = libxl_flask_context_to_sid(ctx, (char *)buf, strlen(buf),
- &b_info->device_model_ssidref);
- if (e) {
- if (errno == ENOSYS) {
- fprintf(stderr, "XSM Disabled:"
- " device_model_stubdomain_seclabel not supported\n");
- } else {
- fprintf(stderr, "Invalid device_model_stubdomain_seclabel:"
- " %s\n", buf);
- exit(1);
- }
- }
- }
+ &buf, 0))
+ b_info->device_model_ssid_label = strdup(buf);
+
#define parse_extra_args(type) \
e = xlu_cfg_get_list_as_string_list(config, "device_model_args"#type, \
&b_info->extra##type, 0); \
@@ -3339,15 +3269,8 @@ static void list_domains(int verbose, int context, int claim, int numa,
}
if (claim)
printf(" %5lu", (unsigned long)info[i].outstanding_memkb / 1024);
- if (verbose || context) {
- int rc;
- size_t size;
- char *buf = NULL;
- rc = libxl_flask_sid_to_context(ctx, info[i].ssidref, &buf,
- &size);
- printf(" %16s", rc < 0 ? "-" : buf);
- free(buf);
- }
+ if (verbose || context)
+ printf(" %16s", info[i].ssid_label ? : "-");
if (numa) {
libxl_domain_get_nodeaffinity(ctx, info[i].domid, &nodemap);
@@ -5223,7 +5146,7 @@ static int sched_domain_output(libxl_scheduler sched, int (*output)(int),
int rc = 0;
if (cpupool) {
- if (cpupool_qualifier_to_cpupoolid(cpupool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, cpupool, &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", cpupool);
return -ERROR_FAIL;
@@ -5343,7 +5266,8 @@ int main_sched_credit(int argc, char **argv)
uint32_t poolid = 0;
if (cpupool) {
- if (cpupool_qualifier_to_cpupoolid(cpupool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, cpupool,
+ &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", cpupool);
return -ERROR_FAIL;
@@ -6811,27 +6735,21 @@ int main_cpupoollist(int argc, char **argv)
for (p = 0; p < n_pools; p++) {
if (!ret && (!pool || (poolinfo[p].poolid == poolid))) {
- name = libxl_cpupoolid_to_name(ctx, poolinfo[p].poolid);
- if (!name) {
- fprintf(stderr, "error getting cpupool info\n");
- ret = -ERROR_NOMEM;
- } else {
- printf("%-19s", name);
- free(name);
- n = 0;
- libxl_for_each_bit(c, poolinfo[p].cpumap)
- if (libxl_bitmap_test(&poolinfo[p].cpumap, c)) {
- if (n && opt_cpus) printf(",");
- if (opt_cpus) printf("%d", c);
- n++;
- }
- if (!opt_cpus) {
- printf("%3d %9s y %4d", n,
- libxl_scheduler_to_string(poolinfo[p].sched),
- poolinfo[p].n_dom);
+ name = poolinfo[p].pool_name;
+ printf("%-19s", name);
+ n = 0;
+ libxl_for_each_bit(c, poolinfo[p].cpumap)
+ if (libxl_bitmap_test(&poolinfo[p].cpumap, c)) {
+ if (n && opt_cpus) printf(",");
+ if (opt_cpus) printf("%d", c);
+ n++;
}
- printf("\n");
+ if (!opt_cpus) {
+ printf("%3d %9s y %4d", n,
+ libxl_scheduler_to_string(poolinfo[p].sched),
+ poolinfo[p].n_dom);
}
+ printf("\n");
}
}
@@ -6852,7 +6770,7 @@ int main_cpupooldestroy(int argc, char **argv)
pool = argv[optind];
- if (cpupool_qualifier_to_cpupoolid(pool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, pool, &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", pool);
return -ERROR_FAIL;
@@ -6874,7 +6792,7 @@ int main_cpupoolrename(int argc, char **argv)
pool = argv[optind++];
- if (cpupool_qualifier_to_cpupoolid(pool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, pool, &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", pool);
return -ERROR_FAIL;
@@ -6912,7 +6830,7 @@ int main_cpupoolcpuadd(int argc, char **argv)
cpu = atoi(argv[optind]);
}
- if (cpupool_qualifier_to_cpupoolid(pool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, pool, &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", pool);
return -ERROR_FAIL;
@@ -6957,7 +6875,7 @@ int main_cpupoolcpuremove(int argc, char **argv)
cpu = atoi(argv[optind]);
}
- if (cpupool_qualifier_to_cpupoolid(pool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, pool, &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", pool);
return -ERROR_FAIL;
@@ -7001,7 +6919,7 @@ int main_cpupoolmigrate(int argc, char **argv)
return -ERROR_FAIL;
}
- if (cpupool_qualifier_to_cpupoolid(pool, &poolid, NULL) ||
+ if (libxl_cpupool_qualifier_to_cpupoolid(ctx, pool, &poolid, NULL) ||
!libxl_cpupoolid_is_valid(ctx, poolid)) {
fprintf(stderr, "unknown cpupool \'%s\'\n", pool);
return -ERROR_FAIL;
diff --git a/tools/libxl/xl_sxp.c b/tools/libxl/xl_sxp.c
index a16a025..48eb608 100644
--- a/tools/libxl/xl_sxp.c
+++ b/tools/libxl/xl_sxp.c
@@ -37,7 +37,6 @@ void printf_info_sexp(int domid, libxl_domain_config *d_config)
libxl_domain_create_info *c_info = &d_config->c_info;
libxl_domain_build_info *b_info = &d_config->b_info;
- char *pool;
printf("(domain\n\t(domid %d)\n", domid);
printf("\t(create_info)\n");
@@ -55,10 +54,8 @@ void printf_info_sexp(int domid, libxl_domain_config *d_config)
} else {
printf("\t(uuid <unknown>)\n");
}
- pool = libxl_cpupoolid_to_name(ctx, c_info->poolid);
- if (pool)
- printf("\t(cpupool %s)\n", pool);
- free(pool);
+ if (c_info->pool_name)
+ printf("\t(cpupool %s)\n", c_info->pool_name);
if (c_info->xsdata)
printf("\t(xsdata contains data)\n");
else
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl
2014-05-01 12:57 ` [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl Wei Liu
@ 2014-05-06 12:45 ` Ian Campbell
2014-05-06 13:03 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 12:45 UTC (permalink / raw)
To: Wei Liu
Cc: Daniel De Graaf, Dario Faggioli, ian.jackson, Juergen Gross, xen-devel
On Thu, 2014-05-01 at 13:57 +0100, Wei Liu wrote:
> This patch pushes parsing of "init_seclabel", "seclabel",
> "device_model_stubdomain_seclabel" and "pool" down to libxl level.
>
> Originally the parsing is done in xl level, which is not ideal because
> libxl won't have the truely relavent information. Now libxl holds
> important information by itself. This is useful when we do serialization
> and deserialization of domain configurations on libxl level.
>
> The libxl IDL is extended to hold the string of labels and pool name.
> And if there those strings are present they take precedence over the
> numeric representations.
>
> As all relavent structures have a field called X_name / X_label now, a
"relevant" (several of these).
> string is also copied there so that we can use it directly.
>
> In order to be compatible with users of older versions of libxl, this
> patch also defines LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING. If it is
> defined, those strings are available. And if those strings are not NULL,
> libxl will do the parsing and ignore the numeric values.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> Cc: Dario Faggioli <dario.faggioli@citrix.com>
> Cc: Juergen Gross <juergen.gross@ts.fujitsu.com>
> Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> ---
> tools/libxl/libxl.c | 19 ++++--
> tools/libxl/libxl.h | 12 ++++
> tools/libxl/libxl_create.c | 57 ++++++++++++++++
> tools/libxl/libxl_dm.c | 4 ++
> tools/libxl/libxl_types.idl | 6 ++
> tools/libxl/libxl_utils.c | 31 +++++++++
> tools/libxl/libxl_utils.h | 3 +
> tools/libxl/xl_cmdimpl.c | 150 ++++++++++---------------------------------
> tools/libxl/xl_sxp.c | 7 +-
> 9 files changed, 164 insertions(+), 125 deletions(-)
>
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 30b0b06..9a90f40 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -520,12 +520,18 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid,
> return 0;
> }
>
> -static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
> +static void xcinfo2xlinfo(libxl_ctx *ctx,
> + const xc_domaininfo_t *xcinfo,
> libxl_dominfo *xlinfo)
> {
> + size_t size;
> +
> memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
> xlinfo->domid = xcinfo->domain;
> xlinfo->ssidref = xcinfo->ssidref;
> + if (libxl_flask_sid_to_context(ctx, xlinfo->ssidref,
> + &xlinfo->ssid_label, &size) < 0)
> + xlinfo->ssid_label = NULL;
Is this a critical error?
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index b2c3015..2f14d26 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -462,6 +462,18 @@
> #define LIBXL_EXTERNAL_CALLERS_ONLY /* disappears for callers outside libxl */
> #endif
>
> +/*
> + * LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING
> + *
> + * If this is defined, then libxl IDL contains strings of XSM security
> + * labels and CPU pool name.
> + *
> + * If those strings are not NULL, libxl will overwrite the numeric
> + * representations of these configurations regardless if they've been
> + * set by the caller, because libxl will do the parsing by itself.
Can you reference the struct and field names explicitly please.
I think it would be fine to have 2 #defines here and a more conventional
name would be LIBXL_HAVE_<STRUCT>_<FIELD>, which splitting would allow.
> + */
> +#define LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING 1
> +
> typedef uint8_t libxl_mac[6];
> #define LIBXL_MAC_FMT "%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx"
> #define LIBXL_MAC_FMTLEN ((2*6)+5) /* 6 hex bytes plus 5 colons */
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index d015cf4..fe3bdd2 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -724,6 +724,63 @@ static void initiate_domain_create(libxl__egc *egc,
>
> domid = 0;
I guess this is just code motion, so I'm happy with it going in, even if
the error behaviour of libxl_flask_context_to_size strikes me as a bit
odd (libxl doesn't normally use errno).
> + if (d_config->c_info.ssid_label) {
> + char *s = d_config->c_info.ssid_label;
> + ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
> + &d_config->c_info.ssidref);
> + if (ret) {
> + if (errno == ENOSYS) {
> + LOG(WARN, "XSM Disabled: init_seclabel not supported");
> + ret = 0;
> + } else {
> + LOG(ERROR, "Invalid init_seclabel: %s", s);
> + goto error_out;
> + }
> + }
> + }
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 0f7bbf8..5f4341f 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -215,6 +215,7 @@ libxl_dominfo = Struct("dominfo",[
> ("uuid", libxl_uuid),
> ("domid", libxl_domid),
> ("ssidref", uint32),
> + ("ssid_label", string),
Alignment (or perhaps hard/soft tabs?) There were a few of these.
> diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
> index 1f334f2..476921e 100644
> --- a/tools/libxl/libxl_utils.c
> +++ b/tools/libxl/libxl_utils.c
All just code motion? (Can the code motion aspect here be easily split
out?)
[...]
> + if (!xlu_cfg_get_string (config, "init_seclabel", &buf, 0))
> + c_info->ssid_label = strdup(buf);
You might find xlu_cfg_replace_string useful at various points here.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl
2014-05-06 12:45 ` Ian Campbell
@ 2014-05-06 13:03 ` Wei Liu
2014-05-07 9:50 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 13:03 UTC (permalink / raw)
To: Ian Campbell
Cc: Wei Liu, Dario Faggioli, ian.jackson, xen-devel, Daniel De Graaf,
Juergen Gross
On Tue, May 06, 2014 at 01:45:30PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:57 +0100, Wei Liu wrote:
> > This patch pushes parsing of "init_seclabel", "seclabel",
> > "device_model_stubdomain_seclabel" and "pool" down to libxl level.
> >
> > Originally the parsing is done in xl level, which is not ideal because
> > libxl won't have the truely relavent information. Now libxl holds
> > important information by itself. This is useful when we do serialization
> > and deserialization of domain configurations on libxl level.
> >
> > The libxl IDL is extended to hold the string of labels and pool name.
> > And if there those strings are present they take precedence over the
> > numeric representations.
> >
> > As all relavent structures have a field called X_name / X_label now, a
>
> "relevant" (several of these).
>
Good catch, thanks.
> > string is also copied there so that we can use it directly.
> >
> > In order to be compatible with users of older versions of libxl, this
> > patch also defines LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING. If it is
> > defined, those strings are available. And if those strings are not NULL,
> > libxl will do the parsing and ignore the numeric values.
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > Cc: Dario Faggioli <dario.faggioli@citrix.com>
> > Cc: Juergen Gross <juergen.gross@ts.fujitsu.com>
> > Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
> > ---
> > tools/libxl/libxl.c | 19 ++++--
> > tools/libxl/libxl.h | 12 ++++
> > tools/libxl/libxl_create.c | 57 ++++++++++++++++
> > tools/libxl/libxl_dm.c | 4 ++
> > tools/libxl/libxl_types.idl | 6 ++
> > tools/libxl/libxl_utils.c | 31 +++++++++
> > tools/libxl/libxl_utils.h | 3 +
> > tools/libxl/xl_cmdimpl.c | 150 ++++++++++---------------------------------
> > tools/libxl/xl_sxp.c | 7 +-
> > 9 files changed, 164 insertions(+), 125 deletions(-)
> >
> > diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> > index 30b0b06..9a90f40 100644
> > --- a/tools/libxl/libxl.c
> > +++ b/tools/libxl/libxl.c
> > @@ -520,12 +520,18 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid,
> > return 0;
> > }
> >
> > -static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo,
> > +static void xcinfo2xlinfo(libxl_ctx *ctx,
> > + const xc_domaininfo_t *xcinfo,
> > libxl_dominfo *xlinfo)
> > {
> > + size_t size;
> > +
> > memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
> > xlinfo->domid = xcinfo->domain;
> > xlinfo->ssidref = xcinfo->ssidref;
> > + if (libxl_flask_sid_to_context(ctx, xlinfo->ssidref,
> > + &xlinfo->ssid_label, &size) < 0)
> > + xlinfo->ssid_label = NULL;
>
> Is this a critical error?
>
I'm not very sure about this. My thought was that not displaying the
name of the label is not critical, but on the other hand you can argue
that showing incorrect information is just wrong...
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index b2c3015..2f14d26 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -462,6 +462,18 @@
> > #define LIBXL_EXTERNAL_CALLERS_ONLY /* disappears for callers outside libxl */
> > #endif
> >
> > +/*
> > + * LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING
> > + *
> > + * If this is defined, then libxl IDL contains strings of XSM security
> > + * labels and CPU pool name.
> > + *
> > + * If those strings are not NULL, libxl will overwrite the numeric
> > + * representations of these configurations regardless if they've been
> > + * set by the caller, because libxl will do the parsing by itself.
>
> Can you reference the struct and field names explicitly please.
>
> I think it would be fine to have 2 #defines here and a more conventional
> name would be LIBXL_HAVE_<STRUCT>_<FIELD>, which splitting would allow.
>
OK.
> > + */
> > +#define LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING 1
> > +
> > typedef uint8_t libxl_mac[6];
> > #define LIBXL_MAC_FMT "%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx"
> > #define LIBXL_MAC_FMTLEN ((2*6)+5) /* 6 hex bytes plus 5 colons */
> > diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> > index d015cf4..fe3bdd2 100644
> > --- a/tools/libxl/libxl_create.c
> > +++ b/tools/libxl/libxl_create.c
> > @@ -724,6 +724,63 @@ static void initiate_domain_create(libxl__egc *egc,
> >
> > domid = 0;
>
> I guess this is just code motion, so I'm happy with it going in, even if
> the error behaviour of libxl_flask_context_to_size strikes me as a bit
> odd (libxl doesn't normally use errno).
>
Right, it's just code motion.
> > + if (d_config->c_info.ssid_label) {
> > + char *s = d_config->c_info.ssid_label;
> > + ret = libxl_flask_context_to_sid(ctx, s, strlen(s),
> > + &d_config->c_info.ssidref);
> > + if (ret) {
> > + if (errno == ENOSYS) {
> > + LOG(WARN, "XSM Disabled: init_seclabel not supported");
> > + ret = 0;
> > + } else {
> > + LOG(ERROR, "Invalid init_seclabel: %s", s);
> > + goto error_out;
> > + }
> > + }
> > + }
>
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index 0f7bbf8..5f4341f 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -215,6 +215,7 @@ libxl_dominfo = Struct("dominfo",[
> > ("uuid", libxl_uuid),
> > ("domid", libxl_domid),
> > ("ssidref", uint32),
> > + ("ssid_label", string),
>
> Alignment (or perhaps hard/soft tabs?) There were a few of these.
>
I will check.
> > diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
> > index 1f334f2..476921e 100644
> > --- a/tools/libxl/libxl_utils.c
> > +++ b/tools/libxl/libxl_utils.c
>
> All just code motion? (Can the code motion aspect here be easily split
> out?)
>
Correct. I can do a prerequisite patch to move this part around.
> [...]
> > + if (!xlu_cfg_get_string (config, "init_seclabel", &buf, 0))
> > + c_info->ssid_label = strdup(buf);
>
> You might find xlu_cfg_replace_string useful at various points here.
>
Thanks for the hint.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl
2014-05-06 13:03 ` Wei Liu
@ 2014-05-07 9:50 ` Ian Campbell
2014-05-07 10:30 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 9:50 UTC (permalink / raw)
To: Wei Liu
Cc: Daniel De Graaf, Dario Faggioli, ian.jackson, Juergen Gross, xen-devel
On Tue, 2014-05-06 at 14:03 +0100, Wei Liu wrote:
> > > + size_t size;
> > > +
> > > memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
> > > xlinfo->domid = xcinfo->domain;
> > > xlinfo->ssidref = xcinfo->ssidref;
> > > + if (libxl_flask_sid_to_context(ctx, xlinfo->ssidref,
> > > + &xlinfo->ssid_label, &size) < 0)
> > > + xlinfo->ssid_label = NULL;
> >
> > Is this a critical error?
> >
>
> I'm not very sure about this. My thought was that not displaying the
> name of the label is not critical, but on the other hand you can argue
> that showing incorrect information is just wrong...
Isn't this used when roundtripping things through the JSON though --
i.e. lookup the name and then reparse on the other end? If you fail to
do the lookup -> boom?
Are there circumstances where it is expected that it isn't possible to
translate an ssid ref back to a name? Perhaps if it has been deleted
since the guest was created? If there isn't such a possibility then that
would argue for it being an error. (Maybe this is one for Daniel)
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl
2014-05-07 9:50 ` Ian Campbell
@ 2014-05-07 10:30 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-07 10:30 UTC (permalink / raw)
To: Ian Campbell
Cc: Wei Liu, Dario Faggioli, ian.jackson, xen-devel, Daniel De Graaf,
Juergen Gross
On Wed, May 07, 2014 at 10:50:58AM +0100, Ian Campbell wrote:
> On Tue, 2014-05-06 at 14:03 +0100, Wei Liu wrote:
> > > > + size_t size;
> > > > +
> > > > memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t));
> > > > xlinfo->domid = xcinfo->domain;
> > > > xlinfo->ssidref = xcinfo->ssidref;
> > > > + if (libxl_flask_sid_to_context(ctx, xlinfo->ssidref,
> > > > + &xlinfo->ssid_label, &size) < 0)
> > > > + xlinfo->ssid_label = NULL;
> > >
> > > Is this a critical error?
> > >
> >
> > I'm not very sure about this. My thought was that not displaying the
> > name of the label is not critical, but on the other hand you can argue
> > that showing incorrect information is just wrong...
>
> Isn't this used when roundtripping things through the JSON though --
> i.e. lookup the name and then reparse on the other end? If you fail to
> do the lookup -> boom?
>
No. This is only used for showing information, not for domain creation.
So it seems less critical to me.
Wei.
> Are there circumstances where it is expected that it isn't possible to
> translate an ssid ref back to a name? Perhaps if it has been deleted
> since the guest was created? If there isn't such a possibility then that
> would argue for it being an error. (Maybe this is one for Daniel)
>
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 02/24] libxl: fix memory leak in libxl_cpuid_dispose
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
2014-05-01 12:57 ` [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl Wei Liu
@ 2014-05-01 12:57 ` Wei Liu
2014-05-06 12:47 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types Wei Liu
` (22 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:57 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
libxl_cpuid_policy_list is not allocated with GC-aware allocation so it
needs to be freed manually, just like what libxl_string_list_dispose and
libxl_key_value_list_dispose do.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_cpuid.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index dd21b78..8a59c4d 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -26,6 +26,7 @@ void libxl_cpuid_dispose(libxl_cpuid_policy_list *p_cpuid_list)
if (cpuid_list[i].policy[j] != NULL)
free(cpuid_list[i].policy[j]);
}
+ free(cpuid_list);
return;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 02/24] libxl: fix memory leak in libxl_cpuid_dispose
2014-05-01 12:57 ` [PATCH V4 02/24] libxl: fix memory leak in libxl_cpuid_dispose Wei Liu
@ 2014-05-06 12:47 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 12:47 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:57 +0100, Wei Liu wrote:
> libxl_cpuid_policy_list is not allocated with GC-aware allocation so it
> needs to be freed manually, just like what libxl_string_list_dispose and
> libxl_key_value_list_dispose do.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
The number of indirections in these functions always confuses me, but I
suppose I agree:
Acked-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> tools/libxl/libxl_cpuid.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> index dd21b78..8a59c4d 100644
> --- a/tools/libxl/libxl_cpuid.c
> +++ b/tools/libxl/libxl_cpuid.c
> @@ -26,6 +26,7 @@ void libxl_cpuid_dispose(libxl_cpuid_policy_list *p_cpuid_list)
> if (cpuid_list[i].policy[j] != NULL)
> free(cpuid_list[i].policy[j]);
> }
> + free(cpuid_list);
> return;
> }
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
2014-05-01 12:57 ` [PATCH V4 01/24] xl / libxl: push parsing of SSID and CPU pool ID down to libxl Wei Liu
2014-05-01 12:57 ` [PATCH V4 02/24] libxl: fix memory leak in libxl_cpuid_dispose Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 12:49 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 04/24] libxl.h: move / add some libxl defbool #define here Wei Liu
` (21 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.h | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 2f14d26..b33460b 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -233,6 +233,11 @@
* libxl_types.idl). The library provides a common set of methods for
* initialising and freeing these types.
*
+ * The paradigm of using libxl types is that user always calls "init"
+ * function before using a type, and always calls "dispose" after using
+ * a type to clean up, regardless of the operations on this type
+ * succeed or fail. See xl code for examples.
+ *
* void libxl_<type>_init(<type> *p):
*
* Initialises the members of "p" to all defaults. These may either
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types
2014-05-01 12:58 ` [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types Wei Liu
@ 2014-05-06 12:49 ` Ian Campbell
2014-05-06 13:12 ` Ian Jackson
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 12:49 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
I think this reflects reality even if it's not what we would actually
have wanted.
Acked-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> tools/libxl/libxl.h | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 2f14d26..b33460b 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -233,6 +233,11 @@
> * libxl_types.idl). The library provides a common set of methods for
> * initialising and freeing these types.
> *
> + * The paradigm of using libxl types is that user always calls "init"
> + * function before using a type, and always calls "dispose" after using
> + * a type to clean up, regardless of the operations on this type
> + * succeed or fail. See xl code for examples.
> + *
> * void libxl_<type>_init(<type> *p):
> *
> * Initialises the members of "p" to all defaults. These may either
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types
2014-05-06 12:49 ` Ian Campbell
@ 2014-05-06 13:12 ` Ian Jackson
2014-05-06 13:49 ` Wei Liu
2014-05-07 9:53 ` Ian Campbell
0 siblings, 2 replies; 120+ messages in thread
From: Ian Jackson @ 2014-05-06 13:12 UTC (permalink / raw)
To: Ian Campbell; +Cc: Wei Liu, xen-devel
Ian Campbell writes ("Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types"):
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>
> I think this reflects reality even if it's not what we would actually
> have wanted.
If we intend to change it, we should say so.
> > + * The paradigm of using libxl types is that user always calls "init"
> > + * function before using a type, and always calls "dispose" after using
> > + * a type to clean up, regardless of the operations on this type
> > + * succeed or fail. See xl code for examples.
Here's a version which reads slightly better to a native speaker and
is IMO slightly less ambiguous:
* IDL-generated libxl types should be used as follows: the user must
* always call the "init" function before using a type, even if the
* variable is simply being passed by reference as an out parameter
* to a libxl function. The user must always calls "dispose" exactly
* once afterwards, to clean up, regardless of whether operations on
* this object succeeded or failed. See the xl code for examples.
I would add:
* "init" is idempotent. We intend that "dispose" will become
* idempotent, but this is not currently the case.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types
2014-05-06 13:12 ` Ian Jackson
@ 2014-05-06 13:49 ` Wei Liu
2014-05-07 9:53 ` Ian Campbell
1 sibling, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 13:49 UTC (permalink / raw)
To: Ian Jackson; +Cc: Wei Liu, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 02:12:20PM +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types"):
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> >
> > I think this reflects reality even if it's not what we would actually
> > have wanted.
>
> If we intend to change it, we should say so.
>
> > > + * The paradigm of using libxl types is that user always calls "init"
> > > + * function before using a type, and always calls "dispose" after using
> > > + * a type to clean up, regardless of the operations on this type
> > > + * succeed or fail. See xl code for examples.
>
> Here's a version which reads slightly better to a native speaker and
> is IMO slightly less ambiguous:
>
> * IDL-generated libxl types should be used as follows: the user must
> * always call the "init" function before using a type, even if the
> * variable is simply being passed by reference as an out parameter
> * to a libxl function. The user must always calls "dispose" exactly
> * once afterwards, to clean up, regardless of whether operations on
> * this object succeeded or failed. See the xl code for examples.
>
> I would add:
>
> * "init" is idempotent. We intend that "dispose" will become
> * idempotent, but this is not currently the case.
>
I will use your version. Thanks.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types
2014-05-06 13:12 ` Ian Jackson
2014-05-06 13:49 ` Wei Liu
@ 2014-05-07 9:53 ` Ian Campbell
1 sibling, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 9:53 UTC (permalink / raw)
To: Ian Jackson; +Cc: Wei Liu, xen-devel
On Tue, 2014-05-06 at 14:12 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types"):
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> >
> > I think this reflects reality even if it's not what we would actually
> > have wanted.
>
> If we intend to change it, we should say so.
>
> > > + * The paradigm of using libxl types is that user always calls "init"
> > > + * function before using a type, and always calls "dispose" after using
> > > + * a type to clean up, regardless of the operations on this type
> > > + * succeed or fail. See xl code for examples.
>
> Here's a version which reads slightly better to a native speaker and
> is IMO slightly less ambiguous:
>
> * IDL-generated libxl types should be used as follows: the user must
> * always call the "init" function before using a type, even if the
> * variable is simply being passed by reference as an out parameter
> * to a libxl function. The user must always calls "dispose" exactly
> * once afterwards, to clean up, regardless of whether operations on
> * this object succeeded or failed. See the xl code for examples.
>
> I would add:
>
> * "init" is idempotent. We intend that "dispose" will become
> * idempotent, but this is not currently the case.
I suppose this last bit corresponds to "if we intend to change it" so
should be included iff that is the case.
I can see that it would be desirable to make that change, but I'm not
sure how any application could ever make use of it, since older versions
will crash horribly, and if they were to use a #ifdef
LIBXL_HAVE_IDEMPOTENT_DISPOSE, well, then they could just have called it
once.
I suppose if they only care about Xen >= the version with the change
they could make use of it.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 04/24] libxl.h: move / add some libxl defbool #define here
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (2 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 03/24] libxl.h: document the paradigm of using libxl types Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 12:50 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 05/24] libxl IDL: rename json_fn to json_gen_fn Wei Liu
` (20 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
They will be used by both JSON generator and parser so they should be in
header file.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.c | 10 +++-------
tools/libxl/libxl.h | 6 ++++++
2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 9a90f40..dbba7ca 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -232,10 +232,6 @@ void libxl_key_value_list_dispose(libxl_key_value_list *pkvl)
free(kvl);
}
-#define LIBXL__DEFBOOL_DEFAULT (0)
-#define LIBXL__DEFBOOL_FALSE (-1)
-#define LIBXL__DEFBOOL_TRUE (1)
-
void libxl_defbool_set(libxl_defbool *db, bool b)
{
db->val = b ? LIBXL__DEFBOOL_TRUE : LIBXL__DEFBOOL_FALSE;
@@ -266,11 +262,11 @@ bool libxl_defbool_val(libxl_defbool db)
const char *libxl_defbool_to_string(libxl_defbool b)
{
if (b.val < 0)
- return "False";
+ return LIBXL__DEFBOOL_STR_FALSE;
else if (b.val > 0)
- return "True";
+ return LIBXL__DEFBOOL_STR_TRUE;
else
- return "<default>";
+ return LIBXL__DEFBOOL_STR_DEFAULT;
}
/******************************************************************************/
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index b33460b..24f422d 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -538,6 +538,12 @@ typedef LIBXL_TAILQ_ENTRY(struct libxl_event) libxl_ev_link;
* To allow users of the library to naively select all defaults this
* state is represented as 0. False is < 0 and True is > 0.
*/
+#define LIBXL__DEFBOOL_DEFAULT (0)
+#define LIBXL__DEFBOOL_FALSE (-1)
+#define LIBXL__DEFBOOL_TRUE (1)
+#define LIBXL__DEFBOOL_STR_DEFAULT "<default>"
+#define LIBXL__DEFBOOL_STR_FALSE "False"
+#define LIBXL__DEFBOOL_STR_TRUE "True"
typedef struct {
int val;
} libxl_defbool;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 05/24] libxl IDL: rename json_fn to json_gen_fn
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (3 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 04/24] libxl.h: move / add some libxl defbool #define here Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-01 12:58 ` [PATCH V4 06/24] libxl_json: introduce libx__object_from_json Wei Liu
` (19 subsequent siblings)
24 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
This json_fn is in fact used to generate string representation of a json
data structure. We will introduce another json function to parse json
data structure in later changeset, so rename json_fn to json_gen_fn to
clarify.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxl/gentest.py | 4 ++--
tools/libxl/gentypes.py | 12 ++++++------
tools/libxl/idl.py | 10 +++++-----
tools/libxl/idl.txt | 4 ++--
tools/libxl/libxl_types.idl | 4 ++--
tools/libxl/libxl_types_internal.idl | 2 +-
6 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index 722b7f4..eb9a21b 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -52,7 +52,7 @@ def gen_rand_init(ty, v, indent = " ", parent = None):
s += " break;\n"
s += "}\n"
elif isinstance(ty, idl.Struct) \
- and (parent is None or ty.json_fn is None):
+ and (parent is None or ty.json_gen_fn is None):
for f in [f for f in ty.fields if not f.const]:
(nparent,fexpr) = ty.member(v, f, parent is None)
s += gen_rand_init(f.type, fexpr, "", nparent)
@@ -243,7 +243,7 @@ int main(int argc, char **argv)
f.write(" printf(\"Testing TYPE_to_json()\\n\");\n")
f.write(" printf(\"----------------------\\n\");\n")
f.write(" printf(\"\\n\");\n")
- for ty in [t for t in types if t.json_fn is not None]:
+ for ty in [t for t in types if t.json_gen_fn is not None]:
arg = ty.typename + "_val"
f.write(" %s_rand_init(%s);\n" % (ty.typename, \
ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 917e2c2..61a2b3d 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -229,7 +229,7 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
s += " goto out;\n"
s += " break;\n"
s += "}\n"
- elif isinstance(ty, idl.Struct) and (parent is None or ty.json_fn is None):
+ elif isinstance(ty, idl.Struct) and (parent is None or ty.json_gen_fn is None):
s += "s = yajl_gen_map_open(hand);\n"
s += "if (s != yajl_gen_status_ok)\n"
s += " goto out;\n"
@@ -243,8 +243,8 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
s += "if (s != yajl_gen_status_ok)\n"
s += " goto out;\n"
else:
- if ty.json_fn is not None:
- s += "s = %s(hand, %s);\n" % (ty.json_fn, ty.pass_arg(v, parent is None))
+ if ty.json_gen_fn is not None:
+ s += "s = %s(hand, %s);\n" % (ty.json_gen_fn, ty.pass_arg(v, parent is None))
s += "if (s != yajl_gen_status_ok)\n"
s += " goto out;\n"
@@ -341,7 +341,7 @@ if __name__ == '__main__':
f.write("%svoid %s(%s, %s);\n" % (ty.hidden(), ty.init_fn + "_" + ku.keyvar.name,
ty.make_arg("p"),
ku.keyvar.type.make_arg(ku.keyvar.name)))
- if ty.json_fn is not None:
+ if ty.json_gen_fn is not None:
f.write("%schar *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
if isinstance(ty, idl.Enumeration):
f.write("%sconst char *%s_to_string(%s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
@@ -369,7 +369,7 @@ if __name__ == '__main__':
""" % (header_json_define, header_json_define, " ".join(sys.argv)))
- for ty in [ty for ty in types if ty.json_fn is not None]:
+ for ty in [ty for ty in types if ty.json_gen_fn is not None]:
f.write("%syajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
f.write("\n")
@@ -426,7 +426,7 @@ if __name__ == '__main__':
f.write("}\n")
f.write("\n")
- for ty in [t for t in types if t.json_fn is not None]:
+ for ty in [t for t in types if t.json_gen_fn is not None]:
f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s)\n" % (ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
f.write("{\n")
f.write(libxl_C_type_gen_json(ty, "p"))
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index e4dc79b..92133a3 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -65,9 +65,9 @@ class Type(object):
self.autogenerate_init_fn = kwargs.setdefault('autogenerate_init_fn', False)
if self.typename is not None and not self.private:
- self.json_fn = kwargs.setdefault('json_fn', self.typename + "_gen_json")
+ self.json_gen_fn = kwargs.setdefault('json_gen_fn', self.typename + "_gen_json")
else:
- self.json_fn = kwargs.setdefault('json_fn', None)
+ self.json_gen_fn = kwargs.setdefault('json_gen_fn', None)
self.autogenerate_json = kwargs.setdefault('autogenerate_json', True)
@@ -118,7 +118,7 @@ class Number(Builtin):
kwargs.setdefault('namespace', None)
kwargs.setdefault('dispose_fn', None)
kwargs.setdefault('signed', False)
- kwargs.setdefault('json_fn', "yajl_gen_integer")
+ kwargs.setdefault('json_gen_fn', "yajl_gen_integer")
self.signed = kwargs['signed']
Builtin.__init__(self, ctype, **kwargs)
@@ -256,7 +256,7 @@ class KeyedUnion(Aggregate):
void = Builtin("void *", namespace = None)
bool = Builtin("bool", namespace = None,
- json_fn = "yajl_gen_bool",
+ json_gen_fn = "yajl_gen_bool",
autogenerate_json = False)
size_t = Number("size_t", namespace = None)
@@ -269,7 +269,7 @@ uint32 = UInt(32)
uint64 = UInt(64)
string = Builtin("char *", namespace = None, dispose_fn = "free",
- json_fn = "libxl__string_gen_json",
+ json_gen_fn = "libxl__string_gen_json",
autogenerate_json = False)
class Array(Type):
diff --git a/tools/libxl/idl.txt b/tools/libxl/idl.txt
index 439aede..6a53dd8 100644
--- a/tools/libxl/idl.txt
+++ b/tools/libxl/idl.txt
@@ -60,14 +60,14 @@ Type.autogenerate_init_fn: (default: True if dir in [IN, BOTH])
Indicates if the above named Type.init_fn should be
autogenerated.
-Type.json_fn: (default: typename + "_gen_json" or None if type == None)
+Type.json_gen_fn: (default: typename + "_gen_json" or None if type == None)
The name of the C function which will generate a YAJL data structure
representing this type.
Type.autogenerate_json: (default: True)
- Indicates if the above named Type.json_fn should be autogenerated.
+ Indicates if the above named Type.json_gen_fn should be autogenerated.
Other simple type-Classes
-------------------------
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 5f4341f..5893260 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -7,8 +7,8 @@ namespace("libxl_")
libxl_defbool = Builtin("defbool", passby=PASS_BY_REFERENCE)
-libxl_domid = Builtin("domid", json_fn = "yajl_gen_integer", autogenerate_json = False)
-libxl_devid = Builtin("devid", json_fn = "yajl_gen_integer", autogenerate_json = False, signed = True, init_val="-1")
+libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False)
+libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False, signed = True, init_val="-1")
libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE)
libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE)
libxl_bitmap = Builtin("bitmap", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index cb9444f..a964851 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -1,7 +1,7 @@
namespace("libxl__")
hidden(True)
-libxl_domid = Builtin("domid", namespace="libxl_", json_fn = "yajl_gen_integer")
+libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer")
libxl__qmp_message_type = Enumeration("qmp_message_type", [
(1, "QMP"),
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 06/24] libxl_json: introduce libx__object_from_json
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (4 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 05/24] libxl IDL: rename json_fn to json_gen_fn Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 12:53 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 07/24] libxl_internal: make JSON_* types a bit-field Wei Liu
` (18 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Given a JSON string, we need to convert it to libxl_FOO struct.
The approach is:
JSON string -> libxl__json_object -> libxl_FOO struct
With this approach we can make use of libxl's infrastructure to do the
first half (JSON string -> libxl__json_object).
Second half is done by auto-generated code by libxl's IDL
infrastructure. IDL patch(es) will come later.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_internal.h | 8 ++++++++
tools/libxl/libxl_json.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 38 insertions(+)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index c2b73c4..0c10bf7 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1640,6 +1640,14 @@ typedef struct libxl__json_object {
struct libxl__json_object *parent;
} libxl__json_object;
+typedef int (*libxl__json_parse_callback)(libxl__gc *gc,
+ libxl__json_object *o,
+ void *p);
+_hidden int libxl__object_from_json(libxl_ctx *ctx, const char *type,
+ libxl__json_parse_callback parse,
+ void *p,
+ const char *s);
+
typedef struct {
char *map_key;
libxl__json_object *obj;
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 3ea56a4..b11ad4b 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -794,6 +794,36 @@ out:
return ret;
}
+int libxl__object_from_json(libxl_ctx *ctx, const char *type,
+ libxl__json_parse_callback parse,
+ void *p, const char *s)
+{
+ GC_INIT(ctx);
+ libxl__json_object *o;
+ int rc;
+
+ o = libxl__json_parse(gc, s);
+ if (!o) {
+ LOG(ERROR,
+ "unable to generate libxl__json_object from JSON representation of %s.",
+ type);
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ rc = parse(gc, o, p);
+ if (rc) {
+ LOG(ERROR, "unable to convert libxl__json_object to %s.", type);
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ rc = 0;
+out:
+ GC_FREE;
+ return rc;
+}
+
/*
* Local variables:
* mode: C
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 06/24] libxl_json: introduce libx__object_from_json
2014-05-01 12:58 ` [PATCH V4 06/24] libxl_json: introduce libx__object_from_json Wei Liu
@ 2014-05-06 12:53 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 12:53 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
Typo in $subject "libx" vs "libxl".
> Given a JSON string, we need to convert it to libxl_FOO struct.
>
> The approach is:
> JSON string -> libxl__json_object -> libxl_FOO struct
>
> With this approach we can make use of libxl's infrastructure to do the
> first half (JSON string -> libxl__json_object).
>
> Second half is done by auto-generated code by libxl's IDL
> infrastructure. IDL patch(es) will come later.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> tools/libxl/libxl_internal.h | 8 ++++++++
> tools/libxl/libxl_json.c | 30 ++++++++++++++++++++++++++++++
> 2 files changed, 38 insertions(+)
>
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index c2b73c4..0c10bf7 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -1640,6 +1640,14 @@ typedef struct libxl__json_object {
> struct libxl__json_object *parent;
> } libxl__json_object;
>
> +typedef int (*libxl__json_parse_callback)(libxl__gc *gc,
> + libxl__json_object *o,
> + void *p);
> +_hidden int libxl__object_from_json(libxl_ctx *ctx, const char *type,
> + libxl__json_parse_callback parse,
> + void *p,
> + const char *s);
> +
> typedef struct {
> char *map_key;
> libxl__json_object *obj;
> diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
> index 3ea56a4..b11ad4b 100644
> --- a/tools/libxl/libxl_json.c
> +++ b/tools/libxl/libxl_json.c
> @@ -794,6 +794,36 @@ out:
> return ret;
> }
>
> +int libxl__object_from_json(libxl_ctx *ctx, const char *type,
> + libxl__json_parse_callback parse,
> + void *p, const char *s)
> +{
> + GC_INIT(ctx);
> + libxl__json_object *o;
> + int rc;
> +
> + o = libxl__json_parse(gc, s);
> + if (!o) {
> + LOG(ERROR,
> + "unable to generate libxl__json_object from JSON representation of %s.",
> + type);
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + rc = parse(gc, o, p);
> + if (rc) {
> + LOG(ERROR, "unable to convert libxl__json_object to %s.", type);
> + rc = ERROR_FAIL;
> + goto out;
> + }
> +
> + rc = 0;
> +out:
> + GC_FREE;
> + return rc;
> +}
> +
> /*
> * Local variables:
> * mode: C
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 07/24] libxl_internal: make JSON_* types a bit-field
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (5 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 06/24] libxl_json: introduce libx__object_from_json Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-01 12:58 ` [PATCH V4 08/24] libxl_internal: introduce libxl__json_object_is_{null, number, double} Wei Liu
` (17 subsequent siblings)
24 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Libxl can generate number as type JSON_INTEGER, JSON_DOUBLE or
JSON_NUMBER, string as type JSON_STRING or JSON_NULL (if string is
null).
So make JSON_* type a bit-field and use it in libxl__json_map_get. This is
useful when parsing a libxl__json_object to libxl_FOO struct. We can
enforce type checking on libxl__json_object in an easy way.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxl/libxl_internal.h | 18 +++++++++---------
tools/libxl/libxl_json.c | 2 +-
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 0c10bf7..f37a102 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1613,16 +1613,16 @@ _hidden yajl_gen_status libxl__yajl_gen_asciiz(yajl_gen hand, const char *str);
_hidden yajl_gen_status libxl__yajl_gen_enum(yajl_gen hand, const char *str);
typedef enum {
- JSON_NULL,
- JSON_BOOL,
- JSON_INTEGER,
- JSON_DOUBLE,
+ JSON_NULL = (1 << 0),
+ JSON_BOOL = (1 << 1),
+ JSON_INTEGER = (1 << 2),
+ JSON_DOUBLE = (1 << 3),
/* number is store in string, it's too big to be a long long or a double */
- JSON_NUMBER,
- JSON_STRING,
- JSON_MAP,
- JSON_ARRAY,
- JSON_ANY
+ JSON_NUMBER = (1 << 4),
+ JSON_STRING = (1 << 5),
+ JSON_MAP = (1 << 6),
+ JSON_ARRAY = (1 << 7),
+ JSON_ANY = 255 /* this is a mask of all values above, adjust as needed */
} libxl__json_node_type;
typedef struct libxl__json_object {
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index b11ad4b..27cce9c 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -363,7 +363,7 @@ const libxl__json_object *libxl__json_map_get(const char *key,
return NULL;
if (strcmp(key, node->map_key) == 0) {
if (expected_type == JSON_ANY
- || (node->obj && node->obj->type == expected_type)) {
+ || (node->obj && (node->obj->type & expected_type))) {
return node->obj;
} else {
return NULL;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 08/24] libxl_internal: introduce libxl__json_object_is_{null, number, double}
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (6 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 07/24] libxl_internal: make JSON_* types a bit-field Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-01 12:58 ` [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types Wei Liu
` (16 subsequent siblings)
24 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
... which return true if json object is valid and of type
JSON_{NULL,NUMBER,DOUBLE}.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxl/libxl_internal.h | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index f37a102..9074d40 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1655,6 +1655,10 @@ typedef struct {
typedef struct libxl__yajl_ctx libxl__yajl_ctx;
+static inline bool libxl__json_object_is_null(const libxl__json_object *o)
+{
+ return o != NULL && o->type == JSON_NULL;
+}
static inline bool libxl__json_object_is_bool(const libxl__json_object *o)
{
return o != NULL && o->type == JSON_BOOL;
@@ -1667,6 +1671,14 @@ static inline bool libxl__json_object_is_integer(const libxl__json_object *o)
{
return o != NULL && o->type == JSON_INTEGER;
}
+static inline bool libxl__json_object_is_double(const libxl__json_object *o)
+{
+ return o != NULL && o->type == JSON_DOUBLE;
+}
+static inline bool libxl__json_object_is_number(const libxl__json_object *o)
+{
+ return o != NULL && o->type == JSON_NUMBER;
+}
static inline bool libxl__json_object_is_map(const libxl__json_object *o)
{
return o != NULL && o->type == JSON_MAP;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (7 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 08/24] libxl_internal: introduce libxl__json_object_is_{null, number, double} Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 12:57 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 10/24] libxl_json: allow basic JSON type objects generation Wei Liu
` (15 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
This changeset introduces following functions:
* libxl_defbool_parse_json
* libxl__bool_parse_json
* libxl_uuid_parse_json
* libxl_mac_parse_json
* libxl_bitmap_parse_json
* libxl_cpuid_policy_list_parse_json
* libxl_string_list_parse_json
* libxl_key_value_list_parse_json
* libxl_hwcap_parse_json
* libxl__int_parse_json
* libxl__uint{8,16,32,64}_parse_json
* libxl__string_parse_json
They will be used in later patch to convert the libxl__json_object
tree of a builtin type to libxl_FOO struct.
Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
yajl_gen_integer to generate JSON object.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_cpuid.c | 86 +++++++++++++---
tools/libxl/libxl_json.c | 236 +++++++++++++++++++++++++++++++++++++++++++
tools/libxl/libxl_json.h | 36 ++++++-
tools/libxl/libxl_nocpuid.c | 7 ++
4 files changed, 348 insertions(+), 17 deletions(-)
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 8a59c4d..45d2d9a 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -337,29 +337,29 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
(const char**)(cpuid[i].policy), cpuid_res);
}
+static const char *input_names[2] = { "leaf", "subleaf" };
+static const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
+/*
+ * Aiming for:
+ * [
+ * { 'leaf': 'val-eax',
+ * 'subleaf': 'val-ecx',
+ * 'eax': 'filter',
+ * 'ebx': 'filter',
+ * 'ecx': 'filter',
+ * 'edx': 'filter' },
+ * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
+ * ... etc ...
+ * ]
+ */
+
yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
libxl_cpuid_policy_list *pcpuid)
{
libxl_cpuid_policy_list cpuid = *pcpuid;
yajl_gen_status s;
- const char *input_names[2] = { "leaf", "subleaf" };
- const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
int i, j;
- /*
- * Aiming for:
- * [
- * { 'leaf': 'val-eax',
- * 'subleaf': 'val-ecx',
- * 'eax': 'filter',
- * 'ebx': 'filter',
- * 'ecx': 'filter',
- * 'edx': 'filter' },
- * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
- * ... etc ...
- * ]
- */
-
s = yajl_gen_array_open(hand);
if (s != yajl_gen_status_ok) goto out;
@@ -397,6 +397,60 @@ out:
return s;
}
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p)
+{
+ int i, size;
+ libxl_cpuid_policy_list l;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ if (!o->u.array->count)
+ return 0;
+
+ size = o->u.array->count;
+ /* need one extra slot as sentinel */
+ l = *p = libxl__calloc(NOGC, size + 1, sizeof(libxl_cpuid_policy));
+
+ l[size].input[0] = XEN_CPUID_INPUT_UNUSED;
+ l[size].input[1] = XEN_CPUID_INPUT_UNUSED;
+
+ for (i = 0; i < size; i++) {
+ const libxl__json_object *t;
+ int j;
+
+ if (flexarray_get(o->u.array, i, (void**)&t) != 0)
+ return ERROR_FAIL;
+
+ if (!libxl__json_object_is_map(t))
+ return ERROR_FAIL;
+
+ for (j = 0; j < ARRAY_SIZE(l[0].input); j++) {
+ const libxl__json_object *r;
+
+ r = libxl__json_map_get(input_names[j], t, JSON_INTEGER);
+ if (!r)
+ l[i].input[j] = XEN_CPUID_INPUT_UNUSED;
+ else
+ l[i].input[j] = r->u.i;
+ }
+
+ for (j = 0; j < ARRAY_SIZE(l[0].policy); j++) {
+ const libxl__json_object *r;
+
+ r = libxl__json_map_get(policy_names[j], t, JSON_STRING);
+ if (!r)
+ l[i].policy[j] = NULL;
+ else
+ l[i].policy[j] = libxl__strdup(NOGC, r->u.string);
+ }
+ }
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 27cce9c..2cab951 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -100,6 +100,38 @@ yajl_gen_status libxl_defbool_gen_json(yajl_gen hand,
return libxl__yajl_gen_asciiz(hand, libxl_defbool_to_string(*db));
}
+int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_defbool *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_DEFAULT,
+ strlen(LIBXL__DEFBOOL_STR_DEFAULT)))
+ p->val = LIBXL__DEFBOOL_DEFAULT;
+ else if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_TRUE,
+ strlen(LIBXL__DEFBOOL_STR_TRUE)))
+ p->val = LIBXL__DEFBOOL_TRUE;
+ else if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_FALSE,
+ strlen(LIBXL__DEFBOOL_STR_FALSE)))
+ p->val = LIBXL__DEFBOOL_FALSE;
+ else
+ return ERROR_FAIL;
+
+ return 0;
+}
+
+int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ bool *p)
+{
+ if (!libxl__json_object_is_bool(o))
+ return ERROR_FAIL;
+
+ *p = o->u.b;
+
+ return 0;
+}
+
yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
libxl_uuid *uuid)
{
@@ -108,6 +140,15 @@ yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN);
}
+int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_uuid *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ return libxl_uuid_from_string(p, o->u.string);
+}
+
yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand,
libxl_bitmap *bitmap)
{
@@ -128,6 +169,36 @@ out:
return s;
}
+int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_bitmap *p)
+{
+ int i;
+ int size;
+ const libxl__json_object *t;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ if (!o->u.array->count) {
+ libxl_bitmap_init(p);
+ return 0;
+ }
+
+ t = libxl__json_array_get(o, o->u.array->count - 1);
+ size = t->u.i + 1;
+
+ libxl_bitmap_alloc(CTX, p, size);
+
+ for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
+ if (!libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+
+ libxl_bitmap_set(p, t->u.i);
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
libxl_key_value_list *pkvl)
{
@@ -155,6 +226,40 @@ out:
return s;
}
+int libxl_key_value_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_key_value_list *p)
+{
+ libxl__json_map_node *node = NULL;
+ flexarray_t *maps = NULL;
+ int i, size;
+ libxl_key_value_list kvl;
+
+ if (!libxl__json_object_is_map(o))
+ return ERROR_FAIL;
+
+ maps = o->u.map;
+ size = maps->count * 2;
+ kvl = *p = libxl__calloc(NOGC, size, sizeof(char *));
+
+ for (i = 0; i < maps->count; i++) {
+ int idx = i * 2;
+ if (flexarray_get(maps, i, (void**)&node) != 0)
+ return ERROR_FAIL;
+
+ if (!libxl__json_object_is_string(node->obj) &&
+ !libxl__json_object_is_null(node->obj))
+ return ERROR_FAIL;
+
+ kvl[idx] = libxl__strdup(NOGC, node->map_key);
+ if (libxl__json_object_is_string(node->obj))
+ kvl[idx+1] = libxl__strdup(NOGC, node->obj->u.string);
+ else
+ kvl[idx+1] = NULL;
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl)
{
libxl_string_list l = *pl;
@@ -176,6 +281,38 @@ out:
return s;
}
+int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_string_list *p)
+{
+ const libxl__json_object *t;
+ libxl_string_list l;
+ flexarray_t *array = NULL;
+ int i, size;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ array = o->u.array;
+ size = array->count;
+
+ if (size == 0) {
+ *p = NULL;
+ return 0;
+ }
+
+ /* need one extra slot as sentinel */
+ l = *p = libxl__calloc(NOGC, size + 1, sizeof(char *));
+
+ for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
+ if (!libxl__json_object_is_string(t))
+ return ERROR_FAIL;
+
+ l[i] = libxl__strdup(NOGC, t->u.string);
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
{
char buf[LIBXL_MAC_FMTLEN+1];
@@ -183,6 +320,15 @@ yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN);
}
+int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_mac *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ return libxl__parse_mac(o->u.string, *p);
+}
+
yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand,
libxl_hwcap *p)
{
@@ -201,6 +347,27 @@ out:
return s;
}
+int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_hwcap *p)
+{
+ int i;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ for (i = 0; i<4; i++) {
+ const libxl__json_object *t;
+
+ t = libxl__json_array_get(o, i);
+ if (!t || !libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+
+ (*p)[i] = t->u.i;
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl__string_gen_json(yajl_gen hand,
const char *p)
{
@@ -210,6 +377,20 @@ yajl_gen_status libxl__string_gen_json(yajl_gen hand,
return yajl_gen_null(hand);
}
+int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ char **p)
+{
+ if (!libxl__json_object_is_string(o) && !libxl__json_object_is_null(o))
+ return ERROR_FAIL;
+
+ if (libxl__json_object_is_null(o))
+ *p = NULL;
+ else
+ *p = libxl__strdup(NOGC, o->u.string);
+
+ return 0;
+}
+
/*
* libxl__json_object helper functions
*/
@@ -824,6 +1005,61 @@ out:
return rc;
}
+int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((int *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint8_t *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint16_t *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint32_t *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint64_t *)p) = o->u.i;
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_json.h b/tools/libxl/libxl_json.h
index a4dd8fc..924b2aa 100644
--- a/tools/libxl/libxl_json.h
+++ b/tools/libxl/libxl_json.h
@@ -22,17 +22,51 @@
# include <yajl/yajl_version.h>
#endif
+typedef struct libxl__gc libxl__gc;
+typedef struct libxl__json_object libxl__json_object;
+
yajl_gen_status libxl_defbool_gen_json(yajl_gen hand, libxl_defbool *p);
-yajl_gen_status libxl_domid_gen_json(yajl_gen hand, libxl_domid *p);
+int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_defbool *p);
+int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ bool *p);
yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, libxl_uuid *p);
+int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_uuid *p);
yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *p);
+int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_mac *p);
yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand, libxl_bitmap *p);
+int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_bitmap *p);
yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
libxl_cpuid_policy_list *p);
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p);
yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *p);
+int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_string_list *p);
yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
libxl_key_value_list *p);
+int libxl_key_value_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_key_value_list *p);
yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, libxl_hwcap *p);
+int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_hwcap *p);
+int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ char **p);
#include <_libxl_types_json.h>
diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
index 5f7cb6a..eb525fc 100644
--- a/tools/libxl/libxl_nocpuid.c
+++ b/tools/libxl/libxl_nocpuid.c
@@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
return 0;
}
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p)
+{
+ return 0;
+}
+
/*
* Local variables:
* mode: C
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-01 12:58 ` [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types Wei Liu
@ 2014-05-06 12:57 ` Ian Campbell
2014-05-06 13:06 ` Wei Liu
2014-05-06 13:57 ` Anthony PERARD
0 siblings, 2 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 12:57 UTC (permalink / raw)
To: Wei Liu, Anthony Perard; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
Anthony, are you OK with this one now?
> This changeset introduces following functions:
> * libxl_defbool_parse_json
> * libxl__bool_parse_json
> * libxl_uuid_parse_json
> * libxl_mac_parse_json
> * libxl_bitmap_parse_json
> * libxl_cpuid_policy_list_parse_json
> * libxl_string_list_parse_json
> * libxl_key_value_list_parse_json
> * libxl_hwcap_parse_json
> * libxl__int_parse_json
> * libxl__uint{8,16,32,64}_parse_json
> * libxl__string_parse_json
>
> They will be used in later patch to convert the libxl__json_object
> tree of a builtin type to libxl_FOO struct.
>
> Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
"declaration"
> yajl_gen_integer to generate JSON object.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
With one small question:
[...]
> diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
> index 5f7cb6a..eb525fc 100644
> --- a/tools/libxl/libxl_nocpuid.c
> +++ b/tools/libxl/libxl_nocpuid.c
> @@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> return 0;
> }
>
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p)
> +{
> + return 0;
Not an error of some sort? Or even an assert?
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 12:57 ` Ian Campbell
@ 2014-05-06 13:06 ` Wei Liu
2014-05-07 9:57 ` Ian Campbell
2014-05-06 13:57 ` Anthony PERARD
1 sibling, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 13:06 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony Perard, ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 01:57:38PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
>
> Anthony, are you OK with this one now?
>
> > This changeset introduces following functions:
> > * libxl_defbool_parse_json
> > * libxl__bool_parse_json
> > * libxl_uuid_parse_json
> > * libxl_mac_parse_json
> > * libxl_bitmap_parse_json
> > * libxl_cpuid_policy_list_parse_json
> > * libxl_string_list_parse_json
> > * libxl_key_value_list_parse_json
> > * libxl_hwcap_parse_json
> > * libxl__int_parse_json
> > * libxl__uint{8,16,32,64}_parse_json
> > * libxl__string_parse_json
> >
> > They will be used in later patch to convert the libxl__json_object
> > tree of a builtin type to libxl_FOO struct.
> >
> > Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
>
> "declaration"
>
Thanks.
> > yajl_gen_integer to generate JSON object.
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
>
> With one small question:
>
> [...]
> > diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
> > index 5f7cb6a..eb525fc 100644
> > --- a/tools/libxl/libxl_nocpuid.c
> > +++ b/tools/libxl/libxl_nocpuid.c
> > @@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> > return 0;
> > }
> >
> > +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> > + const libxl__json_object *o,
> > + libxl_cpuid_policy_list *p)
> > +{
> > + return 0;
>
> Not an error of some sort? Or even an assert?
>
I didn't see assertion in libxl_cpuid_policy_list_gen_json either, so I
presumed it is not necessary and returning "success" is enough.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 13:06 ` Wei Liu
@ 2014-05-07 9:57 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 9:57 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony Perard, ian.jackson, xen-devel
On Tue, 2014-05-06 at 14:06 +0100, Wei Liu wrote:
> > > yajl_gen_integer to generate JSON object.
> > >
> > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> >
> > Acked-by: Ian Campbell <ian.campbell@citrix.com>
> >
> > With one small question:
> >
> > [...]
> > > diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
> > > index 5f7cb6a..eb525fc 100644
> > > --- a/tools/libxl/libxl_nocpuid.c
> > > +++ b/tools/libxl/libxl_nocpuid.c
> > > @@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> > > return 0;
> > > }
> > >
> > > +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> > > + const libxl__json_object *o,
> > > + libxl_cpuid_policy_list *p)
> > > +{
> > > + return 0;
> >
> > Not an error of some sort? Or even an assert?
> >
>
> I didn't see assertion in libxl_cpuid_policy_list_gen_json either, so I
> presumed it is not necessary and returning "success" is enough.
Generating a "NULL" json from a NULL cpuid policy is one thing, but
parsing a valid cpuid policy into NULL might be more surprising? OTOH
libxl_cpuid_parse_config just returns success, and I suppose this is
equivalent.
OK, my ack stands.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 12:57 ` Ian Campbell
2014-05-06 13:06 ` Wei Liu
@ 2014-05-06 13:57 ` Anthony PERARD
2014-05-06 14:04 ` Wei Liu
1 sibling, 1 reply; 120+ messages in thread
From: Anthony PERARD @ 2014-05-06 13:57 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 01:57:38PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
>
> Anthony, are you OK with this one now?
Well, I think I commented the V3 just after the V4 was sent, so every
comments still applies.
--
Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 13:57 ` Anthony PERARD
@ 2014-05-06 14:04 ` Wei Liu
2014-05-06 14:35 ` Anthony PERARD
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:04 UTC (permalink / raw)
To: Anthony PERARD; +Cc: ian.jackson, Wei Liu, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 02:57:13PM +0100, Anthony PERARD wrote:
> On Tue, May 06, 2014 at 01:57:38PM +0100, Ian Campbell wrote:
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> >
> > Anthony, are you OK with this one now?
>
> Well, I think I commented the V3 just after the V4 was sent, so every
> comments still applies.
>
I have this one in queue:
>From 28f45d7e0edd26187285d4c3dea507776f2523d7 Mon Sep 17 00:00:00 2001
From: Wei Liu <wei.liu2@citrix.com>
Date: Tue, 8 Apr 2014 15:27:04 +0100
Subject: [PATCH] libxl_json: introduce parser functions for builtin types
This changeset introduces following functions:
* libxl_defbool_parse_json
* libxl__bool_parse_json
* libxl_uuid_parse_json
* libxl_mac_parse_json
* libxl_bitmap_parse_json
* libxl_cpuid_policy_list_parse_json
* libxl_string_list_parse_json
* libxl_key_value_list_parse_json
* libxl_hwcap_parse_json
* libxl__int_parse_json
* libxl__uint{8,16,32,64}_parse_json
* libxl__string_parse_json
They will be used in later patch to convert the libxl__json_object
tree of a builtin type to libxl_FOO struct.
Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
yajl_gen_integer to generate JSON object.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
---
tools/libxl/libxl_cpuid.c | 89 +++++++++++++---
tools/libxl/libxl_json.c | 239 +++++++++++++++++++++++++++++++++++++++++++
tools/libxl/libxl_json.h | 36 ++++++-
tools/libxl/libxl_nocpuid.c | 7 ++
4 files changed, 354 insertions(+), 17 deletions(-)
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 8a59c4d..7ac5b74 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -337,29 +337,29 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
(const char**)(cpuid[i].policy), cpuid_res);
}
+static const char *input_names[2] = { "leaf", "subleaf" };
+static const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
+/*
+ * Aiming for:
+ * [
+ * { 'leaf': 'val-eax',
+ * 'subleaf': 'val-ecx',
+ * 'eax': 'filter',
+ * 'ebx': 'filter',
+ * 'ecx': 'filter',
+ * 'edx': 'filter' },
+ * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
+ * ... etc ...
+ * ]
+ */
+
yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
libxl_cpuid_policy_list *pcpuid)
{
libxl_cpuid_policy_list cpuid = *pcpuid;
yajl_gen_status s;
- const char *input_names[2] = { "leaf", "subleaf" };
- const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
int i, j;
- /*
- * Aiming for:
- * [
- * { 'leaf': 'val-eax',
- * 'subleaf': 'val-ecx',
- * 'eax': 'filter',
- * 'ebx': 'filter',
- * 'ecx': 'filter',
- * 'edx': 'filter' },
- * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
- * ... etc ...
- * ]
- */
-
s = yajl_gen_array_open(hand);
if (s != yajl_gen_status_ok) goto out;
@@ -397,6 +397,63 @@ out:
return s;
}
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p)
+{
+ int i, size;
+ libxl_cpuid_policy_list l;
+ flexarray_t *array;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ array = libxl__json_object_get_array(o);
+ if (!array->count)
+ return 0;
+
+ size = array->count;
+ /* need one extra slot as sentinel */
+ l = *p = libxl__calloc(NOGC, size + 1, sizeof(libxl_cpuid_policy));
+
+ l[size].input[0] = XEN_CPUID_INPUT_UNUSED;
+ l[size].input[1] = XEN_CPUID_INPUT_UNUSED;
+
+ for (i = 0; i < size; i++) {
+ const libxl__json_object *t;
+ int j;
+
+ if (flexarray_get(array, i, (void**)&t) != 0)
+ return ERROR_FAIL;
+
+ if (!libxl__json_object_is_map(t))
+ return ERROR_FAIL;
+
+ for (j = 0; j < ARRAY_SIZE(l[0].input); j++) {
+ const libxl__json_object *r;
+
+ r = libxl__json_map_get(input_names[j], t, JSON_INTEGER);
+ if (!r)
+ l[i].input[j] = XEN_CPUID_INPUT_UNUSED;
+ else
+ l[i].input[j] = libxl__json_object_get_integer(r);
+ }
+
+ for (j = 0; j < ARRAY_SIZE(l[0].policy); j++) {
+ const libxl__json_object *r;
+
+ r = libxl__json_map_get(policy_names[j], t, JSON_STRING);
+ if (!r)
+ l[i].policy[j] = NULL;
+ else
+ l[i].policy[j] =
+ libxl__strdup(NOGC, libxl__json_object_get_string(r));
+ }
+ }
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 27cce9c..8228fcb 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -100,6 +100,42 @@ yajl_gen_status libxl_defbool_gen_json(yajl_gen hand,
return libxl__yajl_gen_asciiz(hand, libxl_defbool_to_string(*db));
}
+int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_defbool *p)
+{
+ const char *s;
+
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ s = libxl__json_object_get_string(o);
+
+ if (!strncmp(s, LIBXL__DEFBOOL_STR_DEFAULT,
+ strlen(LIBXL__DEFBOOL_STR_DEFAULT)))
+ p->val = LIBXL__DEFBOOL_DEFAULT;
+ else if (!strncmp(s, LIBXL__DEFBOOL_STR_TRUE,
+ strlen(LIBXL__DEFBOOL_STR_TRUE)))
+ p->val = LIBXL__DEFBOOL_TRUE;
+ else if (!strncmp(s, LIBXL__DEFBOOL_STR_FALSE,
+ strlen(LIBXL__DEFBOOL_STR_FALSE)))
+ p->val = LIBXL__DEFBOOL_FALSE;
+ else
+ return ERROR_FAIL;
+
+ return 0;
+}
+
+int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ bool *p)
+{
+ if (!libxl__json_object_is_bool(o))
+ return ERROR_FAIL;
+
+ *p = libxl__json_object_get_bool(o);
+
+ return 0;
+}
+
yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
libxl_uuid *uuid)
{
@@ -108,6 +144,15 @@ yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN);
}
+int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_uuid *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ return libxl_uuid_from_string(p, o->u.string);
+}
+
yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand,
libxl_bitmap *bitmap)
{
@@ -128,6 +173,40 @@ out:
return s;
}
+int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_bitmap *p)
+{
+ int i;
+ int size;
+ const libxl__json_object *t;
+ flexarray_t *array;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ array = libxl__json_object_get_array(o);
+ if (!array->count) {
+ libxl_bitmap_init(p);
+ return 0;
+ }
+
+ t = libxl__json_array_get(o, array->count - 1);
+ if (!libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+ size = libxl__json_object_get_integer(t) + 1;
+
+ libxl_bitmap_alloc(CTX, p, size);
+
+ for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
+ if (!libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+
+ libxl_bitmap_set(p, libxl__json_object_get_integer(t));
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
libxl_key_value_list *pkvl)
{
@@ -155,6 +234,41 @@ out:
return s;
}
+int libxl_key_value_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_key_value_list *p)
+{
+ libxl__json_map_node *node = NULL;
+ flexarray_t *maps = NULL;
+ int i, size;
+ libxl_key_value_list kvl;
+
+ if (!libxl__json_object_is_map(o))
+ return ERROR_FAIL;
+
+ maps = libxl__json_object_get_map(o);
+ size = maps->count * 2;
+ kvl = *p = libxl__calloc(NOGC, size, sizeof(char *));
+
+ for (i = 0; i < maps->count; i++) {
+ int idx = i * 2;
+ if (flexarray_get(maps, i, (void**)&node) != 0)
+ return ERROR_FAIL;
+
+ if (!libxl__json_object_is_string(node->obj) &&
+ !libxl__json_object_is_null(node->obj))
+ return ERROR_FAIL;
+
+ kvl[idx] = libxl__strdup(NOGC, node->map_key);
+ if (libxl__json_object_is_string(node->obj))
+ kvl[idx+1] =
+ libxl__strdup(NOGC, libxl__json_object_get_string(node->obj));
+ else
+ kvl[idx+1] = NULL;
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl)
{
libxl_string_list l = *pl;
@@ -176,6 +290,38 @@ out:
return s;
}
+int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_string_list *p)
+{
+ const libxl__json_object *t;
+ libxl_string_list l;
+ flexarray_t *array = NULL;
+ int i, size;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ array = libxl__json_object_get_array(o);
+ size = array->count;
+
+ if (size == 0) {
+ *p = NULL;
+ return 0;
+ }
+
+ /* need one extra slot as sentinel */
+ l = *p = libxl__calloc(NOGC, size + 1, sizeof(char *));
+
+ for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
+ if (!libxl__json_object_is_string(t))
+ return ERROR_FAIL;
+
+ l[i] = libxl__strdup(NOGC, libxl__json_object_get_string(t));
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
{
char buf[LIBXL_MAC_FMTLEN+1];
@@ -183,6 +329,15 @@ yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN);
}
+int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_mac *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ return libxl__parse_mac(libxl__json_object_get_string(o), *p);
+}
+
yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand,
libxl_hwcap *p)
{
@@ -201,6 +356,27 @@ out:
return s;
}
+int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_hwcap *p)
+{
+ int i;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ for (i = 0; i<4; i++) {
+ const libxl__json_object *t;
+
+ t = libxl__json_array_get(o, i);
+ if (!t || !libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+
+ (*p)[i] = libxl__json_object_get_integer(t);
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl__string_gen_json(yajl_gen hand,
const char *p)
{
@@ -210,6 +386,20 @@ yajl_gen_status libxl__string_gen_json(yajl_gen hand,
return yajl_gen_null(hand);
}
+int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ char **p)
+{
+ if (!libxl__json_object_is_string(o) && !libxl__json_object_is_null(o))
+ return ERROR_FAIL;
+
+ if (libxl__json_object_is_null(o))
+ *p = NULL;
+ else
+ *p = libxl__strdup(NOGC, libxl__json_object_get_string(o));
+
+ return 0;
+}
+
/*
* libxl__json_object helper functions
*/
@@ -824,6 +1014,55 @@ out:
return rc;
}
+int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ long long i;
+
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ i = libxl__json_object_get_integer(o);
+
+ if (i > INT_MAX || i < INT_MIN)
+ return ERROR_FAIL;
+
+ *((int *)p) = i;
+
+ return 0;
+}
+
+/* Macro to generate:
+ * libxl__uint8_parse_json
+ * libxl__uint16_parse_json
+ * libxl__uint32_parse_json
+ * libxl__uint64_parse_json
+ */
+#define PARSE_UINT(width) \
+ int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
+ const libxl__json_object *o,\
+ void *p) \
+ { \
+ long long i; \
+ \
+ if (!libxl__json_object_is_integer(o)) \
+ return ERROR_FAIL; \
+ \
+ i = libxl__json_object_get_integer(o); \
+ \
+ if (i > UINT ## width ## _MAX) \
+ return ERROR_FAIL; \
+ \
+ *((uint ## width ## _t *)p) = i; \
+ \
+ return 0; \
+ }
+
+PARSE_UINT(8);
+PARSE_UINT(16);
+PARSE_UINT(32);
+PARSE_UINT(64);
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_json.h b/tools/libxl/libxl_json.h
index a4dd8fc..924b2aa 100644
--- a/tools/libxl/libxl_json.h
+++ b/tools/libxl/libxl_json.h
@@ -22,17 +22,51 @@
# include <yajl/yajl_version.h>
#endif
+typedef struct libxl__gc libxl__gc;
+typedef struct libxl__json_object libxl__json_object;
+
yajl_gen_status libxl_defbool_gen_json(yajl_gen hand, libxl_defbool *p);
-yajl_gen_status libxl_domid_gen_json(yajl_gen hand, libxl_domid *p);
+int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_defbool *p);
+int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ bool *p);
yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, libxl_uuid *p);
+int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_uuid *p);
yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *p);
+int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_mac *p);
yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand, libxl_bitmap *p);
+int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_bitmap *p);
yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
libxl_cpuid_policy_list *p);
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p);
yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *p);
+int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_string_list *p);
yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
libxl_key_value_list *p);
+int libxl_key_value_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_key_value_list *p);
yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, libxl_hwcap *p);
+int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_hwcap *p);
+int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ char **p);
#include <_libxl_types_json.h>
diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
index 5f7cb6a..eb525fc 100644
--- a/tools/libxl/libxl_nocpuid.c
+++ b/tools/libxl/libxl_nocpuid.c
@@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
return 0;
}
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p)
+{
+ return 0;
+}
+
/*
* Local variables:
* mode: C
--
1.7.10.4
> --
> Anthony PERARD
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 14:04 ` Wei Liu
@ 2014-05-06 14:35 ` Anthony PERARD
2014-05-06 14:45 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Anthony PERARD @ 2014-05-06 14:35 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 03:04:20PM +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 02:57:13PM +0100, Anthony PERARD wrote:
> > On Tue, May 06, 2014 at 01:57:38PM +0100, Ian Campbell wrote:
> > > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > >
> > > Anthony, are you OK with this one now?
> >
> > Well, I think I commented the V3 just after the V4 was sent, so every
> > comments still applies.
> >
>
> I have this one in queue:
>
>
> From 28f45d7e0edd26187285d4c3dea507776f2523d7 Mon Sep 17 00:00:00 2001
> From: Wei Liu <wei.liu2@citrix.com>
> Date: Tue, 8 Apr 2014 15:27:04 +0100
> Subject: [PATCH] libxl_json: introduce parser functions for builtin types
>
> This changeset introduces following functions:
> * libxl_defbool_parse_json
> * libxl__bool_parse_json
> * libxl_uuid_parse_json
> * libxl_mac_parse_json
> * libxl_bitmap_parse_json
> * libxl_cpuid_policy_list_parse_json
> * libxl_string_list_parse_json
> * libxl_key_value_list_parse_json
> * libxl_hwcap_parse_json
> * libxl__int_parse_json
> * libxl__uint{8,16,32,64}_parse_json
> * libxl__string_parse_json
>
> They will be used in later patch to convert the libxl__json_object
> tree of a builtin type to libxl_FOO struct.
>
> Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
> yajl_gen_integer to generate JSON object.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> ---
> tools/libxl/libxl_cpuid.c | 89 +++++++++++++---
> tools/libxl/libxl_json.c | 239 +++++++++++++++++++++++++++++++++++++++++++
> tools/libxl/libxl_json.h | 36 ++++++-
> tools/libxl/libxl_nocpuid.c | 7 ++
> 4 files changed, 354 insertions(+), 17 deletions(-)
>
> diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> index 8a59c4d..7ac5b74 100644
> --- a/tools/libxl/libxl_cpuid.c
> +++ b/tools/libxl/libxl_cpuid.c
> @@ -337,29 +337,29 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
> (const char**)(cpuid[i].policy), cpuid_res);
> }
>
> +static const char *input_names[2] = { "leaf", "subleaf" };
> +static const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
> +/*
> + * Aiming for:
> + * [
> + * { 'leaf': 'val-eax',
> + * 'subleaf': 'val-ecx',
> + * 'eax': 'filter',
> + * 'ebx': 'filter',
> + * 'ecx': 'filter',
> + * 'edx': 'filter' },
> + * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
> + * ... etc ...
> + * ]
> + */
> +
> yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> libxl_cpuid_policy_list *pcpuid)
> {
> libxl_cpuid_policy_list cpuid = *pcpuid;
> yajl_gen_status s;
> - const char *input_names[2] = { "leaf", "subleaf" };
> - const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
> int i, j;
>
> - /*
> - * Aiming for:
> - * [
> - * { 'leaf': 'val-eax',
> - * 'subleaf': 'val-ecx',
> - * 'eax': 'filter',
> - * 'ebx': 'filter',
> - * 'ecx': 'filter',
> - * 'edx': 'filter' },
> - * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
> - * ... etc ...
> - * ]
> - */
> -
> s = yajl_gen_array_open(hand);
> if (s != yajl_gen_status_ok) goto out;
>
> @@ -397,6 +397,63 @@ out:
> return s;
> }
>
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p)
> +{
> + int i, size;
> + libxl_cpuid_policy_list l;
> + flexarray_t *array;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + array = libxl__json_object_get_array(o);
> + if (!array->count)
> + return 0;
> +
> + size = array->count;
> + /* need one extra slot as sentinel */
> + l = *p = libxl__calloc(NOGC, size + 1, sizeof(libxl_cpuid_policy));
> +
> + l[size].input[0] = XEN_CPUID_INPUT_UNUSED;
> + l[size].input[1] = XEN_CPUID_INPUT_UNUSED;
> +
> + for (i = 0; i < size; i++) {
> + const libxl__json_object *t;
> + int j;
> +
> + if (flexarray_get(array, i, (void**)&t) != 0)
> + return ERROR_FAIL;
> +
> + if (!libxl__json_object_is_map(t))
> + return ERROR_FAIL;
> +
> + for (j = 0; j < ARRAY_SIZE(l[0].input); j++) {
> + const libxl__json_object *r;
> +
> + r = libxl__json_map_get(input_names[j], t, JSON_INTEGER);
> + if (!r)
> + l[i].input[j] = XEN_CPUID_INPUT_UNUSED;
> + else
> + l[i].input[j] = libxl__json_object_get_integer(r);
> + }
> +
> + for (j = 0; j < ARRAY_SIZE(l[0].policy); j++) {
> + const libxl__json_object *r;
> +
> + r = libxl__json_map_get(policy_names[j], t, JSON_STRING);
> + if (!r)
> + l[i].policy[j] = NULL;
> + else
> + l[i].policy[j] =
> + libxl__strdup(NOGC, libxl__json_object_get_string(r));
> + }
> + }
> +
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
> index 27cce9c..8228fcb 100644
> --- a/tools/libxl/libxl_json.c
> +++ b/tools/libxl/libxl_json.c
> @@ -100,6 +100,42 @@ yajl_gen_status libxl_defbool_gen_json(yajl_gen hand,
> return libxl__yajl_gen_asciiz(hand, libxl_defbool_to_string(*db));
> }
>
> +int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_defbool *p)
> +{
> + const char *s;
> +
> + if (!libxl__json_object_is_string(o))
> + return ERROR_FAIL;
> +
> + s = libxl__json_object_get_string(o);
> +
> + if (!strncmp(s, LIBXL__DEFBOOL_STR_DEFAULT,
> + strlen(LIBXL__DEFBOOL_STR_DEFAULT)))
> + p->val = LIBXL__DEFBOOL_DEFAULT;
> + else if (!strncmp(s, LIBXL__DEFBOOL_STR_TRUE,
> + strlen(LIBXL__DEFBOOL_STR_TRUE)))
> + p->val = LIBXL__DEFBOOL_TRUE;
> + else if (!strncmp(s, LIBXL__DEFBOOL_STR_FALSE,
> + strlen(LIBXL__DEFBOOL_STR_FALSE)))
> + p->val = LIBXL__DEFBOOL_FALSE;
> + else
> + return ERROR_FAIL;
> +
> + return 0;
> +}
> +
> +int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + bool *p)
> +{
> + if (!libxl__json_object_is_bool(o))
> + return ERROR_FAIL;
> +
> + *p = libxl__json_object_get_bool(o);
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
> libxl_uuid *uuid)
> {
> @@ -108,6 +144,15 @@ yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
> return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN);
> }
>
> +int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_uuid *p)
> +{
> + if (!libxl__json_object_is_string(o))
> + return ERROR_FAIL;
> +
> + return libxl_uuid_from_string(p, o->u.string);
> +}
> +
> yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand,
> libxl_bitmap *bitmap)
> {
> @@ -128,6 +173,40 @@ out:
> return s;
> }
>
> +int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_bitmap *p)
> +{
> + int i;
> + int size;
> + const libxl__json_object *t;
> + flexarray_t *array;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + array = libxl__json_object_get_array(o);
> + if (!array->count) {
> + libxl_bitmap_init(p);
> + return 0;
> + }
> +
> + t = libxl__json_array_get(o, array->count - 1);
> + if (!libxl__json_object_is_integer(t))
> + return ERROR_FAIL;
> + size = libxl__json_object_get_integer(t) + 1;
> +
> + libxl_bitmap_alloc(CTX, p, size);
> +
> + for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
> + if (!libxl__json_object_is_integer(t))
> + return ERROR_FAIL;
> +
> + libxl_bitmap_set(p, libxl__json_object_get_integer(t));
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
> libxl_key_value_list *pkvl)
> {
> @@ -155,6 +234,41 @@ out:
> return s;
> }
>
> +int libxl_key_value_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_key_value_list *p)
> +{
> + libxl__json_map_node *node = NULL;
> + flexarray_t *maps = NULL;
> + int i, size;
> + libxl_key_value_list kvl;
> +
> + if (!libxl__json_object_is_map(o))
> + return ERROR_FAIL;
> +
> + maps = libxl__json_object_get_map(o);
> + size = maps->count * 2;
> + kvl = *p = libxl__calloc(NOGC, size, sizeof(char *));
> +
> + for (i = 0; i < maps->count; i++) {
> + int idx = i * 2;
> + if (flexarray_get(maps, i, (void**)&node) != 0)
> + return ERROR_FAIL;
> +
> + if (!libxl__json_object_is_string(node->obj) &&
> + !libxl__json_object_is_null(node->obj))
> + return ERROR_FAIL;
> +
> + kvl[idx] = libxl__strdup(NOGC, node->map_key);
> + if (libxl__json_object_is_string(node->obj))
> + kvl[idx+1] =
> + libxl__strdup(NOGC, libxl__json_object_get_string(node->obj));
> + else
> + kvl[idx+1] = NULL;
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl)
> {
> libxl_string_list l = *pl;
> @@ -176,6 +290,38 @@ out:
> return s;
> }
>
> +int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_string_list *p)
> +{
> + const libxl__json_object *t;
> + libxl_string_list l;
> + flexarray_t *array = NULL;
> + int i, size;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + array = libxl__json_object_get_array(o);
> + size = array->count;
> +
> + if (size == 0) {
> + *p = NULL;
> + return 0;
> + }
> +
> + /* need one extra slot as sentinel */
> + l = *p = libxl__calloc(NOGC, size + 1, sizeof(char *));
> +
> + for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
> + if (!libxl__json_object_is_string(t))
> + return ERROR_FAIL;
> +
> + l[i] = libxl__strdup(NOGC, libxl__json_object_get_string(t));
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
> {
> char buf[LIBXL_MAC_FMTLEN+1];
> @@ -183,6 +329,15 @@ yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
> return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN);
> }
>
> +int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_mac *p)
> +{
> + if (!libxl__json_object_is_string(o))
> + return ERROR_FAIL;
> +
> + return libxl__parse_mac(libxl__json_object_get_string(o), *p);
> +}
> +
> yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand,
> libxl_hwcap *p)
> {
> @@ -201,6 +356,27 @@ out:
> return s;
> }
>
> +int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_hwcap *p)
> +{
> + int i;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + for (i = 0; i<4; i++) {
> + const libxl__json_object *t;
> +
> + t = libxl__json_array_get(o, i);
> + if (!t || !libxl__json_object_is_integer(t))
> + return ERROR_FAIL;
> +
> + (*p)[i] = libxl__json_object_get_integer(t);
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl__string_gen_json(yajl_gen hand,
> const char *p)
> {
> @@ -210,6 +386,20 @@ yajl_gen_status libxl__string_gen_json(yajl_gen hand,
> return yajl_gen_null(hand);
> }
>
> +int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + char **p)
> +{
> + if (!libxl__json_object_is_string(o) && !libxl__json_object_is_null(o))
> + return ERROR_FAIL;
> +
> + if (libxl__json_object_is_null(o))
> + *p = NULL;
> + else
> + *p = libxl__strdup(NOGC, libxl__json_object_get_string(o));
> +
> + return 0;
> +}
> +
> /*
> * libxl__json_object helper functions
> */
> @@ -824,6 +1014,55 @@ out:
> return rc;
> }
>
> +int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + long long i;
> +
> + if (!libxl__json_object_is_integer(o))
> + return ERROR_FAIL;
> +
> + i = libxl__json_object_get_integer(o);
> +
> + if (i > INT_MAX || i < INT_MIN)
> + return ERROR_FAIL;
> +
> + *((int *)p) = i;
> +
> + return 0;
> +}
> +
> +/* Macro to generate:
> + * libxl__uint8_parse_json
> + * libxl__uint16_parse_json
> + * libxl__uint32_parse_json
> + * libxl__uint64_parse_json
> + */
> +#define PARSE_UINT(width) \
> + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> + const libxl__json_object *o,\
> + void *p) \
> + { \
> + long long i; \
> + \
> + if (!libxl__json_object_is_integer(o)) \
> + return ERROR_FAIL; \
> + \
> + i = libxl__json_object_get_integer(o); \
> + \
> + if (i > UINT ## width ## _MAX) \
> + return ERROR_FAIL; \
My guest is this will always be false for uint64 and maybe for uint32.
The value return by get_interger can only be <= LLONG_MAX which I
suppose is still < UINT64_MAX.
Also, 'i' might be < 0.
If json contain a value > LLONG_MAX, the value will be store as a string
with the type number.
> + \
> + *((uint ## width ## _t *)p) = i; \
> + \
> + return 0; \
> + }
> +
> +PARSE_UINT(8);
> +PARSE_UINT(16);
> +PARSE_UINT(32);
> +PARSE_UINT(64);
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/libxl_json.h b/tools/libxl/libxl_json.h
> index a4dd8fc..924b2aa 100644
> --- a/tools/libxl/libxl_json.h
> +++ b/tools/libxl/libxl_json.h
> @@ -22,17 +22,51 @@
> # include <yajl/yajl_version.h>
> #endif
>
> +typedef struct libxl__gc libxl__gc;
> +typedef struct libxl__json_object libxl__json_object;
> +
> yajl_gen_status libxl_defbool_gen_json(yajl_gen hand, libxl_defbool *p);
> -yajl_gen_status libxl_domid_gen_json(yajl_gen hand, libxl_domid *p);
> +int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_defbool *p);
> +int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + bool *p);
> yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, libxl_uuid *p);
> +int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_uuid *p);
> yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *p);
> +int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_mac *p);
> yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand, libxl_bitmap *p);
> +int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_bitmap *p);
> yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> libxl_cpuid_policy_list *p);
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p);
> yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *p);
> +int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_string_list *p);
> yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
> libxl_key_value_list *p);
> +int libxl_key_value_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_key_value_list *p);
> yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, libxl_hwcap *p);
> +int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_hwcap *p);
> +int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + char **p);
>
> #include <_libxl_types_json.h>
>
> diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
> index 5f7cb6a..eb525fc 100644
> --- a/tools/libxl/libxl_nocpuid.c
> +++ b/tools/libxl/libxl_nocpuid.c
> @@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> return 0;
> }
>
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p)
> +{
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
This new patch looks fine, beyond the comment on PARSE_UINT().
--
Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 14:35 ` Anthony PERARD
@ 2014-05-06 14:45 ` Wei Liu
2014-05-06 14:49 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:45 UTC (permalink / raw)
To: Anthony PERARD; +Cc: ian.jackson, Wei Liu, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 03:35:27PM +0100, Anthony PERARD wrote:
[...]
> > +/* Macro to generate:
> > + * libxl__uint8_parse_json
> > + * libxl__uint16_parse_json
> > + * libxl__uint32_parse_json
> > + * libxl__uint64_parse_json
> > + */
> > +#define PARSE_UINT(width) \
> > + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> > + const libxl__json_object *o,\
> > + void *p) \
> > + { \
> > + long long i; \
> > + \
> > + if (!libxl__json_object_is_integer(o)) \
> > + return ERROR_FAIL; \
> > + \
> > + i = libxl__json_object_get_integer(o); \
> > + \
> > + if (i > UINT ## width ## _MAX) \
> > + return ERROR_FAIL; \
>
> My guest is this will always be false for uint64 and maybe for uint32.
> The value return by get_interger can only be <= LLONG_MAX which I
> suppose is still < UINT64_MAX.
>
I was just being lazy about it.
> Also, 'i' might be < 0.
>
Phew, I knew this but somehow I thought it was OK. :-(
So the two things combined, the check should be
(i > 0 && i < UINT ## width ## _MAX)
What do you think?
> If json contain a value > LLONG_MAX, the value will be store as a string
> with the type number.
>
This is also true, but I coded like this on purpose. You won't be able
to convert that string anyway because that's why it's stored as string
in the first place. We should just return ERROR_FAIL in this case, which
we do already with the check libxl__json_object_is_integer.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 14:45 ` Wei Liu
@ 2014-05-06 14:49 ` Wei Liu
2014-05-06 15:14 ` Anthony PERARD
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:49 UTC (permalink / raw)
To: Anthony PERARD; +Cc: ian.jackson, Wei Liu, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 03:45:47PM +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 03:35:27PM +0100, Anthony PERARD wrote:
> [...]
> > > +/* Macro to generate:
> > > + * libxl__uint8_parse_json
> > > + * libxl__uint16_parse_json
> > > + * libxl__uint32_parse_json
> > > + * libxl__uint64_parse_json
> > > + */
> > > +#define PARSE_UINT(width) \
> > > + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> > > + const libxl__json_object *o,\
> > > + void *p) \
> > > + { \
> > > + long long i; \
> > > + \
> > > + if (!libxl__json_object_is_integer(o)) \
> > > + return ERROR_FAIL; \
> > > + \
> > > + i = libxl__json_object_get_integer(o); \
> > > + \
> > > + if (i > UINT ## width ## _MAX) \
> > > + return ERROR_FAIL; \
> >
> > My guest is this will always be false for uint64 and maybe for uint32.
> > The value return by get_interger can only be <= LLONG_MAX which I
> > suppose is still < UINT64_MAX.
> >
>
> I was just being lazy about it.
>
> > Also, 'i' might be < 0.
> >
>
> Phew, I knew this but somehow I thought it was OK. :-(
>
> So the two things combined, the check should be
>
> (i > 0 && i < UINT ## width ## _MAX)
>
> What do you think?
>
> > If json contain a value > LLONG_MAX, the value will be store as a string
> > with the type number.
> >
>
> This is also true, but I coded like this on purpose. You won't be able
> to convert that string anyway because that's why it's stored as string
> in the first place. We should just return ERROR_FAIL in this case, which
> we do already with the check libxl__json_object_is_integer.
>
Of course with the sole exception of uint64_t parser. This is a special
case that needs to be handled.
Wei.
> Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 14:49 ` Wei Liu
@ 2014-05-06 15:14 ` Anthony PERARD
2014-05-06 15:42 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Anthony PERARD @ 2014-05-06 15:14 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 03:49:06PM +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 03:45:47PM +0100, Wei Liu wrote:
> > On Tue, May 06, 2014 at 03:35:27PM +0100, Anthony PERARD wrote:
> > [...]
> > > > +/* Macro to generate:
> > > > + * libxl__uint8_parse_json
> > > > + * libxl__uint16_parse_json
> > > > + * libxl__uint32_parse_json
> > > > + * libxl__uint64_parse_json
> > > > + */
> > > > +#define PARSE_UINT(width) \
> > > > + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> > > > + const libxl__json_object *o,\
> > > > + void *p) \
> > > > + { \
> > > > + long long i; \
> > > > + \
> > > > + if (!libxl__json_object_is_integer(o)) \
> > > > + return ERROR_FAIL; \
> > > > + \
> > > > + i = libxl__json_object_get_integer(o); \
> > > > + \
> > > > + if (i > UINT ## width ## _MAX) \
> > > > + return ERROR_FAIL; \
> > >
> > > My guest is this will always be false for uint64 and maybe for uint32.
> > > The value return by get_interger can only be <= LLONG_MAX which I
> > > suppose is still < UINT64_MAX.
> > >
> >
> > I was just being lazy about it.
> >
> > > Also, 'i' might be < 0.
> > >
> >
> > Phew, I knew this but somehow I thought it was OK. :-(
> >
> > So the two things combined, the check should be
> >
> > (i > 0 && i < UINT ## width ## _MAX)
> >
> > What do you think?
0 is also valid ;), and I think UINTX_MAX is valid too,
so (i >= 0 && i <= UINT ## width ## _MAX) should do.
Or:
if (i < 0 || i > UINT ## width ## _MAX)
return ERROR_FAIL;
> > > If json contain a value > LLONG_MAX, the value will be store as a string
> > > with the type number.
> > >
> >
> > This is also true, but I coded like this on purpose. You won't be able
> > to convert that string anyway because that's why it's stored as string
> > in the first place. We should just return ERROR_FAIL in this case, which
> > we do already with the check libxl__json_object_is_integer.
> >
>
> Of course with the sole exception of uint64_t parser. This is a special
> case that needs to be handled.
Right.
--
Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 15:14 ` Anthony PERARD
@ 2014-05-06 15:42 ` Wei Liu
2014-05-06 15:59 ` Anthony PERARD
2014-05-12 16:43 ` Wei Liu
0 siblings, 2 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 15:42 UTC (permalink / raw)
To: Anthony PERARD; +Cc: ian.jackson, Wei Liu, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 04:14:53PM +0100, Anthony PERARD wrote:
> On Tue, May 06, 2014 at 03:49:06PM +0100, Wei Liu wrote:
> > On Tue, May 06, 2014 at 03:45:47PM +0100, Wei Liu wrote:
> > > On Tue, May 06, 2014 at 03:35:27PM +0100, Anthony PERARD wrote:
> > > [...]
> > > > > +/* Macro to generate:
> > > > > + * libxl__uint8_parse_json
> > > > > + * libxl__uint16_parse_json
> > > > > + * libxl__uint32_parse_json
> > > > > + * libxl__uint64_parse_json
> > > > > + */
> > > > > +#define PARSE_UINT(width) \
> > > > > + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> > > > > + const libxl__json_object *o,\
> > > > > + void *p) \
> > > > > + { \
> > > > > + long long i; \
> > > > > + \
> > > > > + if (!libxl__json_object_is_integer(o)) \
> > > > > + return ERROR_FAIL; \
> > > > > + \
> > > > > + i = libxl__json_object_get_integer(o); \
> > > > > + \
> > > > > + if (i > UINT ## width ## _MAX) \
> > > > > + return ERROR_FAIL; \
> > > >
> > > > My guest is this will always be false for uint64 and maybe for uint32.
> > > > The value return by get_interger can only be <= LLONG_MAX which I
> > > > suppose is still < UINT64_MAX.
> > > >
> > >
> > > I was just being lazy about it.
> > >
> > > > Also, 'i' might be < 0.
> > > >
> > >
> > > Phew, I knew this but somehow I thought it was OK. :-(
> > >
> > > So the two things combined, the check should be
> > >
> > > (i > 0 && i < UINT ## width ## _MAX)
> > >
> > > What do you think?
>
> 0 is also valid ;), and I think UINTX_MAX is valid too,
> so (i >= 0 && i <= UINT ## width ## _MAX) should do.
> Or:
> if (i < 0 || i > UINT ## width ## _MAX)
> return ERROR_FAIL;
>
>
> > > > If json contain a value > LLONG_MAX, the value will be store as a string
> > > > with the type number.
> > > >
> > >
> > > This is also true, but I coded like this on purpose. You won't be able
> > > to convert that string anyway because that's why it's stored as string
> > > in the first place. We should just return ERROR_FAIL in this case, which
> > > we do already with the check libxl__json_object_is_integer.
> > >
> >
> > Of course with the sole exception of uint64_t parser. This is a special
> > case that needs to be handled.
>
> Right.
>
For your convenient I paste the relevant code snippet here:
+/* Macro to generate:
+ * libxl__uint8_parse_json
+ * libxl__uint16_parse_json
+ * libxl__uint32_parse_json
+ */
+#define PARSE_UINT(width) \
+ int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
+ const libxl__json_object *o,\
+ void *p) \
+ { \
+ long long i; \
+ \
+ if (!libxl__json_object_is_integer(o)) \
+ return ERROR_FAIL; \
+ \
+ i = libxl__json_object_get_integer(o); \
+ \
+ if (i < 0 || i > UINT ## width ## _MAX) \
+ return ERROR_FAIL; \
+ \
+ *((uint ## width ## _t *)p) = i; \
+ \
+ return 0; \
+ }
+
+PARSE_UINT(8);
+PARSE_UINT(16);
+PARSE_UINT(32);
+
+int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o) &&
+ !libxl__json_object_is_number(o))
+ return ERROR_FAIL;
+
+ if (libxl__json_object_is_integer(o)) {
+ long long i = libxl__json_object_get_integer(o);
+
+ if (i < 0)
+ return ERROR_FAIL;
+
+ *((uint64_t *)p) = i;
+ } else {
+ const char *s;
+ unsigned long long i;
+ int saved_errno = errno;
+
+ s = libxl__json_object_get_number(o);
+
+ errno = 0;
+ i = strtoull(s, NULL, 10);
+
+ if (i == ULLONG_MAX && errno == ERANGE)
+ return ERROR_FAIL;
+
+ errno = saved_errno;
+ *((uint64_t *)p) = i;
+ }
+
+ return 0;
+}
> --
> Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 15:42 ` Wei Liu
@ 2014-05-06 15:59 ` Anthony PERARD
2014-05-06 16:13 ` Wei Liu
2014-05-12 16:43 ` Wei Liu
1 sibling, 1 reply; 120+ messages in thread
From: Anthony PERARD @ 2014-05-06 15:59 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 04:14:53PM +0100, Anthony PERARD wrote:
> > On Tue, May 06, 2014 at 03:49:06PM +0100, Wei Liu wrote:
> > > On Tue, May 06, 2014 at 03:45:47PM +0100, Wei Liu wrote:
> > > > On Tue, May 06, 2014 at 03:35:27PM +0100, Anthony PERARD wrote:
> > > > [...]
> > > > > > +/* Macro to generate:
> > > > > > + * libxl__uint8_parse_json
> > > > > > + * libxl__uint16_parse_json
> > > > > > + * libxl__uint32_parse_json
> > > > > > + * libxl__uint64_parse_json
> > > > > > + */
> > > > > > +#define PARSE_UINT(width) \
> > > > > > + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> > > > > > + const libxl__json_object *o,\
> > > > > > + void *p) \
> > > > > > + { \
> > > > > > + long long i; \
> > > > > > + \
> > > > > > + if (!libxl__json_object_is_integer(o)) \
> > > > > > + return ERROR_FAIL; \
> > > > > > + \
> > > > > > + i = libxl__json_object_get_integer(o); \
> > > > > > + \
> > > > > > + if (i > UINT ## width ## _MAX) \
> > > > > > + return ERROR_FAIL; \
> > > > >
> > > > > My guest is this will always be false for uint64 and maybe for uint32.
> > > > > The value return by get_interger can only be <= LLONG_MAX which I
> > > > > suppose is still < UINT64_MAX.
> > > > >
> > > >
> > > > I was just being lazy about it.
> > > >
> > > > > Also, 'i' might be < 0.
> > > > >
> > > >
> > > > Phew, I knew this but somehow I thought it was OK. :-(
> > > >
> > > > So the two things combined, the check should be
> > > >
> > > > (i > 0 && i < UINT ## width ## _MAX)
> > > >
> > > > What do you think?
> >
> > 0 is also valid ;), and I think UINTX_MAX is valid too,
> > so (i >= 0 && i <= UINT ## width ## _MAX) should do.
> > Or:
> > if (i < 0 || i > UINT ## width ## _MAX)
> > return ERROR_FAIL;
> >
> >
> > > > > If json contain a value > LLONG_MAX, the value will be store as a string
> > > > > with the type number.
> > > > >
> > > >
> > > > This is also true, but I coded like this on purpose. You won't be able
> > > > to convert that string anyway because that's why it's stored as string
> > > > in the first place. We should just return ERROR_FAIL in this case, which
> > > > we do already with the check libxl__json_object_is_integer.
> > > >
> > >
> > > Of course with the sole exception of uint64_t parser. This is a special
> > > case that needs to be handled.
> >
> > Right.
> >
>
> For your convenient I paste the relevant code snippet here:
>
> +/* Macro to generate:
> + * libxl__uint8_parse_json
> + * libxl__uint16_parse_json
> + * libxl__uint32_parse_json
> + */
> +#define PARSE_UINT(width) \
> + int libxl__uint ## width ## _parse_json(libxl__gc *gc, \
> + const libxl__json_object *o,\
> + void *p) \
> + { \
> + long long i; \
> + \
> + if (!libxl__json_object_is_integer(o)) \
> + return ERROR_FAIL; \
> + \
> + i = libxl__json_object_get_integer(o); \
> + \
> + if (i < 0 || i > UINT ## width ## _MAX) \
> + return ERROR_FAIL; \
> + \
> + *((uint ## width ## _t *)p) = i; \
> + \
> + return 0; \
> + }
> +
> +PARSE_UINT(8);
> +PARSE_UINT(16);
> +PARSE_UINT(32);
> +
> +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o) &&
> + !libxl__json_object_is_number(o))
> + return ERROR_FAIL;
> +
> + if (libxl__json_object_is_integer(o)) {
> + long long i = libxl__json_object_get_integer(o);
> +
> + if (i < 0)
> + return ERROR_FAIL;
> +
> + *((uint64_t *)p) = i;
> + } else {
> + const char *s;
> + unsigned long long i;
> + int saved_errno = errno;
> +
> + s = libxl__json_object_get_number(o);
> +
> + errno = 0;
> + i = strtoull(s, NULL, 10);
> +
> + if (i == ULLONG_MAX && errno == ERANGE)
> + return ERROR_FAIL;
> +
> + errno = saved_errno;
> + *((uint64_t *)p) = i;
> + }
> +
> + return 0;
> +}
That look good now.
Ack on the all patch.
--
Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-06 15:42 ` Wei Liu
2014-05-06 15:59 ` Anthony PERARD
@ 2014-05-12 16:43 ` Wei Liu
2014-05-13 8:26 ` Ian Campbell
1 sibling, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-12 16:43 UTC (permalink / raw)
To: Anthony PERARD; +Cc: ian.jackson, Wei Liu, Ian Campbell, xen-devel
On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
[...]
> +
> +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o) &&
> + !libxl__json_object_is_number(o))
> + return ERROR_FAIL;
> +
> + if (libxl__json_object_is_integer(o)) {
> + long long i = libxl__json_object_get_integer(o);
> +
> + if (i < 0)
> + return ERROR_FAIL;
> +
Just found out that current API will generate -1 for some fields (say,
video_memkb), so we have to loosen this check.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-12 16:43 ` Wei Liu
@ 2014-05-13 8:26 ` Ian Campbell
2014-05-13 9:19 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-13 8:26 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony PERARD, ian.jackson, xen-devel
On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> [...]
> > +
> > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > + void *p)
> > +{
> > + if (!libxl__json_object_is_integer(o) &&
> > + !libxl__json_object_is_number(o))
> > + return ERROR_FAIL;
> > +
> > + if (libxl__json_object_is_integer(o)) {
> > + long long i = libxl__json_object_get_integer(o);
> > +
> > + if (i < 0)
> > + return ERROR_FAIL;
> > +
>
> Just found out that current API will generate -1 for some fields (say,
> video_memkb), so we have to loosen this check.
Is -1 the flag for "default" in these cases?
I wonder -- should we omit fields which are set to default from the json
altogether rather than encode them as some specific value? (the actual
value is notionally somewhat libxl internal). This would be a case of
comparing them against their IDL init_val in the gen_json function I
think.
Thoughts?
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 8:26 ` Ian Campbell
@ 2014-05-13 9:19 ` Wei Liu
2014-05-13 9:32 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-13 9:19 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony PERARD, ian.jackson, Wei Liu, xen-devel
On Tue, May 13, 2014 at 09:26:06AM +0100, Ian Campbell wrote:
> On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> > On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> > [...]
> > > +
> > > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > > + void *p)
> > > +{
> > > + if (!libxl__json_object_is_integer(o) &&
> > > + !libxl__json_object_is_number(o))
> > > + return ERROR_FAIL;
> > > +
> > > + if (libxl__json_object_is_integer(o)) {
> > > + long long i = libxl__json_object_get_integer(o);
> > > +
> > > + if (i < 0)
> > > + return ERROR_FAIL;
> > > +
> >
> > Just found out that current API will generate -1 for some fields (say,
> > video_memkb), so we have to loosen this check.
>
> Is -1 the flag for "default" in these cases?
>
Yes, that's actually ~0 (LIBXL_MEMKB_DEFAULT) in libxl.
MemKB type is actually uint64 but the generator generates int64 type,
hence the -1.
However, there is other instances that some field has uint64 type and
actually generates -1 value.
The discussion below is orthogonal to the problem I found.
> I wonder -- should we omit fields which are set to default from the json
> altogether rather than encode them as some specific value? (the actual
> value is notionally somewhat libxl internal). This would be a case of
LIBXL_MEMKB_DEFAULT is in libxl.h, as with some other default
values, so that the value is actually visible to external users. Not
saying that the actual value affects that much, just to clarify.
There're some defaults that are not visible to external world though.
> comparing them against their IDL init_val in the gen_json function I
> think.
>
Or the other way around, export all default values to libxl.h so that
libxl users can interpret them as well.
Not sure what's the best approach.
Wei.
> Thoughts?
>
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 9:19 ` Wei Liu
@ 2014-05-13 9:32 ` Ian Campbell
2014-05-13 10:40 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-13 9:32 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony PERARD, ian.jackson, xen-devel
On Tue, 2014-05-13 at 10:19 +0100, Wei Liu wrote:
> On Tue, May 13, 2014 at 09:26:06AM +0100, Ian Campbell wrote:
> > On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> > > On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> > > [...]
> > > > +
> > > > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > > > + void *p)
> > > > +{
> > > > + if (!libxl__json_object_is_integer(o) &&
> > > > + !libxl__json_object_is_number(o))
> > > > + return ERROR_FAIL;
> > > > +
> > > > + if (libxl__json_object_is_integer(o)) {
> > > > + long long i = libxl__json_object_get_integer(o);
> > > > +
> > > > + if (i < 0)
> > > > + return ERROR_FAIL;
> > > > +
> > >
> > > Just found out that current API will generate -1 for some fields (say,
> > > video_memkb), so we have to loosen this check.
> >
> > Is -1 the flag for "default" in these cases?
> >
>
> Yes, that's actually ~0 (LIBXL_MEMKB_DEFAULT) in libxl.
>
> MemKB type is actually uint64 but the generator generates int64 type,
> hence the -1.
Is this a separate (latent) bug? What happens for uint64 values >
INT64_MAX but < UINT64_MAX?
> However, there is other instances that some field has uint64 type and
> actually generates -1 value.
>
> The discussion below is orthogonal to the problem I found.
>
> > I wonder -- should we omit fields which are set to default from the json
> > altogether rather than encode them as some specific value? (the actual
> > value is notionally somewhat libxl internal). This would be a case of
>
> LIBXL_MEMKB_DEFAULT is in libxl.h, as with some other default
> values, so that the value is actually visible to external users. Not
> saying that the actual value affects that much, just to clarify.
I think this counts as being "invisible", in that it would be wrong for
the user to ever hard code -1 or ~0 here -- they should always use
LIBXL_MEMKB_DEFAULT and not care/know what the value happens to be. In
principal we could change it if we wanted (ok, there probably isn't any
other sane value in this case)
> There're some defaults that are not visible to external world though.
>
> > comparing them against their IDL init_val in the gen_json function I
> > think.
> >
>
> Or the other way around, export all default values to libxl.h so that
> libxl users can interpret them as well.
>
> Not sure what's the best approach.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 9:32 ` Ian Campbell
@ 2014-05-13 10:40 ` Wei Liu
2014-05-13 10:45 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-13 10:40 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony PERARD, ian.jackson, Wei Liu, xen-devel
On Tue, May 13, 2014 at 10:32:57AM +0100, Ian Campbell wrote:
> On Tue, 2014-05-13 at 10:19 +0100, Wei Liu wrote:
> > On Tue, May 13, 2014 at 09:26:06AM +0100, Ian Campbell wrote:
> > > On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> > > > On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> > > > [...]
> > > > > +
> > > > > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > > > > + void *p)
> > > > > +{
> > > > > + if (!libxl__json_object_is_integer(o) &&
> > > > > + !libxl__json_object_is_number(o))
> > > > > + return ERROR_FAIL;
> > > > > +
> > > > > + if (libxl__json_object_is_integer(o)) {
> > > > > + long long i = libxl__json_object_get_integer(o);
> > > > > +
> > > > > + if (i < 0)
> > > > > + return ERROR_FAIL;
> > > > > +
> > > >
> > > > Just found out that current API will generate -1 for some fields (say,
> > > > video_memkb), so we have to loosen this check.
> > >
> > > Is -1 the flag for "default" in these cases?
> > >
> >
> > Yes, that's actually ~0 (LIBXL_MEMKB_DEFAULT) in libxl.
> >
> > MemKB type is actually uint64 but the generator generates int64 type,
> > hence the -1.
>
> Is this a separate (latent) bug? What happens for uint64 values >
> INT64_MAX but < UINT64_MAX?
>
It's buggy I think. After the transformation (generator -> string ->
parser) it gets sign-extended when it really shouldn't.
And this bug affects all fields which have uint64 type. Sigh. This is a
limitation in yajl (yajl_gen_integer) not able to generate uint64 value.
Parser (json_callback_number) can cope with values that it cannot
convert to long long.
> > However, there is other instances that some field has uint64 type and
> > actually generates -1 value.
> >
> > The discussion below is orthogonal to the problem I found.
> >
> > > I wonder -- should we omit fields which are set to default from the json
> > > altogether rather than encode them as some specific value? (the actual
> > > value is notionally somewhat libxl internal). This would be a case of
> >
> > LIBXL_MEMKB_DEFAULT is in libxl.h, as with some other default
> > values, so that the value is actually visible to external users. Not
> > saying that the actual value affects that much, just to clarify.
>
> I think this counts as being "invisible", in that it would be wrong for
> the user to ever hard code -1 or ~0 here -- they should always use
> LIBXL_MEMKB_DEFAULT and not care/know what the value happens to be. In
> principal we could change it if we wanted (ok, there probably isn't any
> other sane value in this case)
>
Fine with me to not generate JSON for all those "default" values. The
parser should be able to cope with missing fields anyway.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 10:40 ` Wei Liu
@ 2014-05-13 10:45 ` Ian Campbell
2014-05-13 10:51 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-13 10:45 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony PERARD, ian.jackson, xen-devel
On Tue, 2014-05-13 at 11:40 +0100, Wei Liu wrote:
> On Tue, May 13, 2014 at 10:32:57AM +0100, Ian Campbell wrote:
> > On Tue, 2014-05-13 at 10:19 +0100, Wei Liu wrote:
> > > On Tue, May 13, 2014 at 09:26:06AM +0100, Ian Campbell wrote:
> > > > On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> > > > > On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> > > > > [...]
> > > > > > +
> > > > > > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > > > > > + void *p)
> > > > > > +{
> > > > > > + if (!libxl__json_object_is_integer(o) &&
> > > > > > + !libxl__json_object_is_number(o))
> > > > > > + return ERROR_FAIL;
> > > > > > +
> > > > > > + if (libxl__json_object_is_integer(o)) {
> > > > > > + long long i = libxl__json_object_get_integer(o);
> > > > > > +
> > > > > > + if (i < 0)
> > > > > > + return ERROR_FAIL;
> > > > > > +
> > > > >
> > > > > Just found out that current API will generate -1 for some fields (say,
> > > > > video_memkb), so we have to loosen this check.
> > > >
> > > > Is -1 the flag for "default" in these cases?
> > > >
> > >
> > > Yes, that's actually ~0 (LIBXL_MEMKB_DEFAULT) in libxl.
> > >
> > > MemKB type is actually uint64 but the generator generates int64 type,
> > > hence the -1.
> >
> > Is this a separate (latent) bug? What happens for uint64 values >
> > INT64_MAX but < UINT64_MAX?
> >
>
> It's buggy I think. After the transformation (generator -> string ->
> parser) it gets sign-extended when it really shouldn't.
>
> And this bug affects all fields which have uint64 type. Sigh. This is a
> limitation in yajl (yajl_gen_integer) not able to generate uint64 value.
> Parser (json_callback_number) can cope with values that it cannot
> convert to long long.
Oh dear. Are we supposed to be using yajl_gen_number then?
> > > However, there is other instances that some field has uint64 type and
> > > actually generates -1 value.
> > >
> > > The discussion below is orthogonal to the problem I found.
> > >
> > > > I wonder -- should we omit fields which are set to default from the json
> > > > altogether rather than encode them as some specific value? (the actual
> > > > value is notionally somewhat libxl internal). This would be a case of
> > >
> > > LIBXL_MEMKB_DEFAULT is in libxl.h, as with some other default
> > > values, so that the value is actually visible to external users. Not
> > > saying that the actual value affects that much, just to clarify.
> >
> > I think this counts as being "invisible", in that it would be wrong for
> > the user to ever hard code -1 or ~0 here -- they should always use
> > LIBXL_MEMKB_DEFAULT and not care/know what the value happens to be. In
> > principal we could change it if we wanted (ok, there probably isn't any
> > other sane value in this case)
> >
>
> Fine with me to not generate JSON for all those "default" values. The
> parser should be able to cope with missing fields anyway.
Indeed, in fact now I think of it if we want people to be able to use
JSON as a domain config format it is required to do so.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 10:45 ` Ian Campbell
@ 2014-05-13 10:51 ` Wei Liu
2014-05-13 10:55 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-13 10:51 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony PERARD, ian.jackson, Wei Liu, xen-devel
On Tue, May 13, 2014 at 11:45:04AM +0100, Ian Campbell wrote:
> On Tue, 2014-05-13 at 11:40 +0100, Wei Liu wrote:
> > On Tue, May 13, 2014 at 10:32:57AM +0100, Ian Campbell wrote:
> > > On Tue, 2014-05-13 at 10:19 +0100, Wei Liu wrote:
> > > > On Tue, May 13, 2014 at 09:26:06AM +0100, Ian Campbell wrote:
> > > > > On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> > > > > > On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> > > > > > [...]
> > > > > > > +
> > > > > > > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > > > > > > + void *p)
> > > > > > > +{
> > > > > > > + if (!libxl__json_object_is_integer(o) &&
> > > > > > > + !libxl__json_object_is_number(o))
> > > > > > > + return ERROR_FAIL;
> > > > > > > +
> > > > > > > + if (libxl__json_object_is_integer(o)) {
> > > > > > > + long long i = libxl__json_object_get_integer(o);
> > > > > > > +
> > > > > > > + if (i < 0)
> > > > > > > + return ERROR_FAIL;
> > > > > > > +
> > > > > >
> > > > > > Just found out that current API will generate -1 for some fields (say,
> > > > > > video_memkb), so we have to loosen this check.
> > > > >
> > > > > Is -1 the flag for "default" in these cases?
> > > > >
> > > >
> > > > Yes, that's actually ~0 (LIBXL_MEMKB_DEFAULT) in libxl.
> > > >
> > > > MemKB type is actually uint64 but the generator generates int64 type,
> > > > hence the -1.
> > >
> > > Is this a separate (latent) bug? What happens for uint64 values >
> > > INT64_MAX but < UINT64_MAX?
> > >
> >
> > It's buggy I think. After the transformation (generator -> string ->
> > parser) it gets sign-extended when it really shouldn't.
> >
> > And this bug affects all fields which have uint64 type. Sigh. This is a
> > limitation in yajl (yajl_gen_integer) not able to generate uint64 value.
> > Parser (json_callback_number) can cope with values that it cannot
> > convert to long long.
>
> Oh dear. Are we supposed to be using yajl_gen_number then?
>
I think so. Some more patches to my queue. Looks like this work is never
coming to an end. :-)
And presumably if we are to make this change we also need to update
libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
for the name?
Now looking back to my libxl__uint64_parse_json it's actually not buggy!
Huh.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 10:51 ` Wei Liu
@ 2014-05-13 10:55 ` Ian Campbell
2014-05-13 11:03 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-13 10:55 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony PERARD, ian.jackson, xen-devel
On Tue, 2014-05-13 at 11:51 +0100, Wei Liu wrote:
> On Tue, May 13, 2014 at 11:45:04AM +0100, Ian Campbell wrote:
> > On Tue, 2014-05-13 at 11:40 +0100, Wei Liu wrote:
> > > On Tue, May 13, 2014 at 10:32:57AM +0100, Ian Campbell wrote:
> > > > On Tue, 2014-05-13 at 10:19 +0100, Wei Liu wrote:
> > > > > On Tue, May 13, 2014 at 09:26:06AM +0100, Ian Campbell wrote:
> > > > > > On Mon, 2014-05-12 at 17:43 +0100, Wei Liu wrote:
> > > > > > > On Tue, May 06, 2014 at 04:42:20PM +0100, Wei Liu wrote:
> > > > > > > [...]
> > > > > > > > +
> > > > > > > > +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> > > > > > > > + void *p)
> > > > > > > > +{
> > > > > > > > + if (!libxl__json_object_is_integer(o) &&
> > > > > > > > + !libxl__json_object_is_number(o))
> > > > > > > > + return ERROR_FAIL;
> > > > > > > > +
> > > > > > > > + if (libxl__json_object_is_integer(o)) {
> > > > > > > > + long long i = libxl__json_object_get_integer(o);
> > > > > > > > +
> > > > > > > > + if (i < 0)
> > > > > > > > + return ERROR_FAIL;
> > > > > > > > +
> > > > > > >
> > > > > > > Just found out that current API will generate -1 for some fields (say,
> > > > > > > video_memkb), so we have to loosen this check.
> > > > > >
> > > > > > Is -1 the flag for "default" in these cases?
> > > > > >
> > > > >
> > > > > Yes, that's actually ~0 (LIBXL_MEMKB_DEFAULT) in libxl.
> > > > >
> > > > > MemKB type is actually uint64 but the generator generates int64 type,
> > > > > hence the -1.
> > > >
> > > > Is this a separate (latent) bug? What happens for uint64 values >
> > > > INT64_MAX but < UINT64_MAX?
> > > >
> > >
> > > It's buggy I think. After the transformation (generator -> string ->
> > > parser) it gets sign-extended when it really shouldn't.
> > >
> > > And this bug affects all fields which have uint64 type. Sigh. This is a
> > > limitation in yajl (yajl_gen_integer) not able to generate uint64 value.
> > > Parser (json_callback_number) can cope with values that it cannot
> > > convert to long long.
> >
> > Oh dear. Are we supposed to be using yajl_gen_number then?
> >
>
> I think so. Some more patches to my queue. Looks like this work is never
> coming to an end. :-)
Sorry about that. I expected there to be some yakks in this work,
interesting they haven't turned out to be the ones I was expecting.
> And presumably if we are to make this change we also need to update
> libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> for the name?
Does this actually change the output syntax at all? If it is just a bug
fix then we don't need to announce it I think. Any consumers of the JSON
will have to cope with the buggy version (if they want to) irrespective
of any #define.
> Now looking back to my libxl__uint64_parse_json it's actually not buggy!
> Huh.
Great!
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 10:55 ` Ian Campbell
@ 2014-05-13 11:03 ` Wei Liu
2014-05-13 11:13 ` Ian Campbell
2014-05-13 11:28 ` Anthony PERARD
0 siblings, 2 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-13 11:03 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony PERARD, ian.jackson, Wei Liu, xen-devel
On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
[...]
> > And presumably if we are to make this change we also need to update
> > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > for the name?
>
> Does this actually change the output syntax at all? If it is just a bug
> fix then we don't need to announce it I think. Any consumers of the JSON
> will have to cope with the buggy version (if they want to) irrespective
> of any #define.
>
I will need to check. Presumably it generates a string or something.
We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
other uint types are not affect.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 11:03 ` Wei Liu
@ 2014-05-13 11:13 ` Ian Campbell
2014-05-13 11:38 ` Wei Liu
2014-05-13 11:28 ` Anthony PERARD
1 sibling, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-13 11:13 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony PERARD, ian.jackson, xen-devel
On Tue, 2014-05-13 at 12:03 +0100, Wei Liu wrote:
> On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
> [...]
> > > And presumably if we are to make this change we also need to update
> > > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > > for the name?
> >
> > Does this actually change the output syntax at all? If it is just a bug
> > fix then we don't need to announce it I think. Any consumers of the JSON
> > will have to cope with the buggy version (if they want to) irrespective
> > of any #define.
> >
>
> I will need to check. Presumably it generates a string or something.
Isn't JSON duck typed though? IOW a correct parser should treat {foo: 1}
and {foo: "1"} the same, if the field foo has numerical semantics?
What does libyajl's parser do?
> We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
> other uint types are not affect.
>
> Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 11:13 ` Ian Campbell
@ 2014-05-13 11:38 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-13 11:38 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony PERARD, ian.jackson, Wei Liu, xen-devel
On Tue, May 13, 2014 at 12:13:29PM +0100, Ian Campbell wrote:
> On Tue, 2014-05-13 at 12:03 +0100, Wei Liu wrote:
> > On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
> > [...]
> > > > And presumably if we are to make this change we also need to update
> > > > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > > > for the name?
> > >
> > > Does this actually change the output syntax at all? If it is just a bug
> > > fix then we don't need to announce it I think. Any consumers of the JSON
> > > will have to cope with the buggy version (if they want to) irrespective
> > > of any #define.
> > >
> >
> > I will need to check. Presumably it generates a string or something.
>
> Isn't JSON duck typed though? IOW a correct parser should treat {foo: 1}
> and {foo: "1"} the same, if the field foo has numerical semantics?
>
My initial test shows that for -1 it generates { foo :
18446744073709551615 } which is actually still a number.
So we can safely assume that this is a bugfix. No #define in libxl.h
needed.
> What does libyajl's parser do?
>
It doesn't seem to be very well documented.
Wei.
> > We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
> > other uint types are not affect.
> >
> > Wei.
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 11:03 ` Wei Liu
2014-05-13 11:13 ` Ian Campbell
@ 2014-05-13 11:28 ` Anthony PERARD
2014-05-13 11:40 ` Wei Liu
1 sibling, 1 reply; 120+ messages in thread
From: Anthony PERARD @ 2014-05-13 11:28 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, Ian Campbell, xen-devel
On Tue, May 13, 2014 at 12:03:59PM +0100, Wei Liu wrote:
> On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
> [...]
> > > And presumably if we are to make this change we also need to update
> > > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > > for the name?
> >
> > Does this actually change the output syntax at all? If it is just a bug
> > fix then we don't need to announce it I think. Any consumers of the JSON
> > will have to cope with the buggy version (if they want to) irrespective
> > of any #define.
> >
>
> I will need to check. Presumably it generates a string or something.
>
> We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
> other uint types are not affect.
>From the JSON format perspective, every numbers are signed decimal, I
think (after a quick look on wikipedia). The parser may make a difference
between signed/unsigned decimal/interger.
For the default value, it might make sense to not include them, since
there are basicly not set values.
--
Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 11:28 ` Anthony PERARD
@ 2014-05-13 11:40 ` Wei Liu
2014-05-13 11:54 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-13 11:40 UTC (permalink / raw)
To: Anthony PERARD; +Cc: ian.jackson, Wei Liu, Ian Campbell, xen-devel
On Tue, May 13, 2014 at 12:28:42PM +0100, Anthony PERARD wrote:
> On Tue, May 13, 2014 at 12:03:59PM +0100, Wei Liu wrote:
> > On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
> > [...]
> > > > And presumably if we are to make this change we also need to update
> > > > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > > > for the name?
> > >
> > > Does this actually change the output syntax at all? If it is just a bug
> > > fix then we don't need to announce it I think. Any consumers of the JSON
> > > will have to cope with the buggy version (if they want to) irrespective
> > > of any #define.
> > >
> >
> > I will need to check. Presumably it generates a string or something.
> >
> > We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
> > other uint types are not affect.
>
> >From the JSON format perspective, every numbers are signed decimal, I
> think (after a quick look on wikipedia). The parser may make a difference
> between signed/unsigned decimal/interger.
>
> For the default value, it might make sense to not include them, since
> there are basicly not set values.
>
Can you confirm that the existing consumer (QEMU?) will not care if those
values are absent in JSON object? If QEMU uses our parser then it should
be fine.
Wei.
> --
> Anthony PERARD
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 11:40 ` Wei Liu
@ 2014-05-13 11:54 ` Ian Campbell
2014-05-13 11:58 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-13 11:54 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony PERARD, ian.jackson, xen-devel
On Tue, 2014-05-13 at 12:40 +0100, Wei Liu wrote:
> On Tue, May 13, 2014 at 12:28:42PM +0100, Anthony PERARD wrote:
> > On Tue, May 13, 2014 at 12:03:59PM +0100, Wei Liu wrote:
> > > On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
> > > [...]
> > > > > And presumably if we are to make this change we also need to update
> > > > > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > > > > for the name?
> > > >
> > > > Does this actually change the output syntax at all? If it is just a bug
> > > > fix then we don't need to announce it I think. Any consumers of the JSON
> > > > will have to cope with the buggy version (if they want to) irrespective
> > > > of any #define.
> > > >
> > >
> > > I will need to check. Presumably it generates a string or something.
> > >
> > > We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
> > > other uint types are not affect.
> >
> > >From the JSON format perspective, every numbers are signed decimal, I
> > think (after a quick look on wikipedia). The parser may make a difference
> > between signed/unsigned decimal/interger.
> >
> > For the default value, it might make sense to not include them, since
> > there are basicly not set values.
> >
>
> Can you confirm that the existing consumer (QEMU?) will not care if those
> values are absent in JSON object? If QEMU uses our parser then it should
> be fine.
The objects passed to qemu are not libxl IDL/autogenerated ones, are
they? They are RPC calls which are crafted via some other mechanism.
IOW I don't think this change is going to impact those at all.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types
2014-05-13 11:54 ` Ian Campbell
@ 2014-05-13 11:58 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-13 11:58 UTC (permalink / raw)
To: Ian Campbell; +Cc: Anthony PERARD, ian.jackson, Wei Liu, xen-devel
On Tue, May 13, 2014 at 12:54:25PM +0100, Ian Campbell wrote:
> On Tue, 2014-05-13 at 12:40 +0100, Wei Liu wrote:
> > On Tue, May 13, 2014 at 12:28:42PM +0100, Anthony PERARD wrote:
> > > On Tue, May 13, 2014 at 12:03:59PM +0100, Wei Liu wrote:
> > > > On Tue, May 13, 2014 at 11:55:30AM +0100, Ian Campbell wrote:
> > > > [...]
> > > > > > And presumably if we are to make this change we also need to update
> > > > > > libxl.h to have LIBXL_JSON_GENERATE_NUMBER-ish? What would you suggest
> > > > > > for the name?
> > > > >
> > > > > Does this actually change the output syntax at all? If it is just a bug
> > > > > fix then we don't need to announce it I think. Any consumers of the JSON
> > > > > will have to cope with the buggy version (if they want to) irrespective
> > > > > of any #define.
> > > > >
> > > >
> > > > I will need to check. Presumably it generates a string or something.
> > > >
> > > > We can treat it as a bug fix if we only use yajl_gen_number of uint64_t,
> > > > other uint types are not affect.
> > >
> > > >From the JSON format perspective, every numbers are signed decimal, I
> > > think (after a quick look on wikipedia). The parser may make a difference
> > > between signed/unsigned decimal/interger.
> > >
> > > For the default value, it might make sense to not include them, since
> > > there are basicly not set values.
> > >
> >
> > Can you confirm that the existing consumer (QEMU?) will not care if those
> > values are absent in JSON object? If QEMU uses our parser then it should
> > be fine.
>
> The objects passed to qemu are not libxl IDL/autogenerated ones, are
> they? They are RPC calls which are crafted via some other mechanism.
>
If QEMU will not use libxl IDL structures then it's also fine.
> IOW I don't think this change is going to impact those at all.
>
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 10/24] libxl_json: allow basic JSON type objects generation
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (8 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 09/24] libxl_json: introduce parser functions for builtin types Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 13:02 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output Wei Liu
` (14 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
The original logic is that basic JSON types (number, string and null)
must be an element of JSON map or array. This assumption doesn't hold
true anymore when we need to return basic JSON types.
Returning basic JSON types is required for parsing number, string and
null objects back into libxl__json_object.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_internal.h | 2 +-
tools/libxl/libxl_json.c | 87 ++++++++++++++++++------------------------
2 files changed, 38 insertions(+), 51 deletions(-)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 9074d40..6e3d19d 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1735,7 +1735,7 @@ _hidden libxl__json_object *libxl__json_object_alloc(libxl__gc *gc_opt,
libxl__json_node_type type);
_hidden int libxl__json_object_append_to(libxl__gc *gc_opt,
libxl__json_object *obj,
- libxl__json_object *dst);
+ libxl__yajl_ctx *ctx);
_hidden libxl__json_object *libxl__json_array_get(const libxl__json_object *o,
int i);
_hidden
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 2cab951..cb8490b 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -415,36 +415,43 @@ libxl__json_object *libxl__json_object_alloc(libxl__gc *gc,
return obj;
}
-int libxl__json_object_append_to(libxl__gc *gc,
- libxl__json_object *obj,
- libxl__json_object *dst)
+int libxl__json_object_append_to(libxl__gc *gc, libxl__json_object *obj,
+ libxl__yajl_ctx *ctx)
{
- assert(dst != NULL);
+ libxl__json_object *dst = ctx->current;
- switch (dst->type) {
- case JSON_MAP: {
- libxl__json_map_node *last;
+ if (dst) {
+ switch (dst->type) {
+ case JSON_MAP: {
+ libxl__json_map_node *last;
- if (dst->u.map->count == 0) {
+ if (dst->u.map->count == 0) {
+ LIBXL__LOG(libxl__gc_owner(gc), LIBXL__LOG_ERROR,
+ "Try to add a value to an empty map (with no key)");
+ return ERROR_FAIL;
+ }
+ flexarray_get(dst->u.map, dst->u.map->count - 1, (void**)&last);
+ last->obj = obj;
+ break;
+ }
+ case JSON_ARRAY:
+ flexarray_append(dst->u.array, obj);
+ break;
+ default:
LIBXL__LOG(libxl__gc_owner(gc), LIBXL__LOG_ERROR,
- "Try to add a value to an empty map (with no key)");
- return -1;
+ "Try append an object is not a map/array (%i)\n",
+ dst->type);
+ return ERROR_FAIL;
}
- flexarray_get(dst->u.map, dst->u.map->count - 1, (void**)&last);
- last->obj = obj;
- break;
- }
- case JSON_ARRAY:
- flexarray_append(dst->u.array, obj);
- break;
- default:
- LIBXL__LOG(libxl__gc_owner(gc), LIBXL__LOG_ERROR,
- "Try append an object is not a map/array (%i)\n",
- dst->type);
- return -1;
}
obj->parent = dst;
+
+ if (libxl__json_object_is_map(obj) || libxl__json_object_is_array(obj))
+ ctx->current = obj;
+ if (ctx->head == NULL)
+ ctx->head = obj;
+
return 0;
}
@@ -630,9 +637,8 @@ static int json_callback_null(void *opaque)
obj = libxl__json_object_alloc(ctx->gc, JSON_NULL);
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -647,9 +653,8 @@ static int json_callback_boolean(void *opaque, int boolean)
obj = libxl__json_object_alloc(ctx->gc, JSON_BOOL);
obj->u.b = boolean;
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -704,9 +709,8 @@ error:
obj->u.string = t;
out:
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -728,9 +732,8 @@ static int json_callback_string(void *opaque, const unsigned char *str,
obj = libxl__json_object_alloc(ctx->gc, JSON_STRING);
obj->u.string = t;
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -776,16 +779,8 @@ static int json_callback_start_map(void *opaque)
obj = libxl__json_object_alloc(ctx->gc, JSON_MAP);
- if (ctx->current) {
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
- return 0;
- }
- }
-
- ctx->current = obj;
- if (ctx->head == NULL) {
- ctx->head = obj;
- }
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
+ return 0;
return 1;
}
@@ -816,16 +811,8 @@ static int json_callback_start_array(void *opaque)
obj = libxl__json_object_alloc(ctx->gc, JSON_ARRAY);
- if (ctx->current) {
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
- return 0;
- }
- }
-
- ctx->current = obj;
- if (ctx->head == NULL) {
- ctx->head = obj;
- }
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
+ return 0;
return 1;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 10/24] libxl_json: allow basic JSON type objects generation
2014-05-01 12:58 ` [PATCH V4 10/24] libxl_json: allow basic JSON type objects generation Wei Liu
@ 2014-05-06 13:02 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 13:02 UTC (permalink / raw)
To: Wei Liu; +Cc: Anthony Perard, ian.jackson, xen-devel
CCing Anthony.
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> The original logic is that basic JSON types (number, string and null)
> must be an element of JSON map or array. This assumption doesn't hold
> true anymore when we need to return basic JSON types.
>
> Returning basic JSON types is required for parsing number, string and
> null objects back into libxl__json_object.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
I think passing (ctx->gc, obj, ctx) as arguments is a bit odd, but I can
see how it makes sense from the PoV of the conventions here (and given
that the ctx is a libxl__yajl_ctx not a libxl__ctx):
Acked-by: Ian Campbell <ian.campbell@citrix.com>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (9 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 10/24] libxl_json: allow basic JSON type objects generation Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 13:03 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 12/24] libxl IDL: generate code to parse libxl__json_object to libxl_FOO struct Wei Liu
` (13 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Parser relies on the discriminator to go to correct branch.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentypes.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 61a2b3d..8d7183a 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -229,6 +229,11 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
s += " goto out;\n"
s += " break;\n"
s += "}\n"
+ s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" \
+ % (ty.keyvar.name, ty.keyvar.name)
+ s += "if (s != yajl_gen_status_ok)\n"
+ s += " goto out;\n"
+ s += libxl_C_type_gen_json(ty.keyvar.type, (parent + ty.keyvar.name), indent, parent)
elif isinstance(ty, idl.Struct) and (parent is None or ty.json_gen_fn is None):
s += "s = yajl_gen_map_open(hand);\n"
s += "if (s != yajl_gen_status_ok)\n"
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output
2014-05-01 12:58 ` [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output Wei Liu
@ 2014-05-06 13:03 ` Ian Campbell
2014-05-06 13:09 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 13:03 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> Parser relies on the discriminator to go to correct branch.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix>
But, an idea: What about calling the field which refers to the union by
it's discriminated name, i.e. "u.pv": { map of things } instead of
"type": "pv", "u" { map of things } ?
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output
2014-05-06 13:03 ` Ian Campbell
@ 2014-05-06 13:09 ` Wei Liu
2014-05-07 10:00 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 13:09 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 02:03:49PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > Parser relies on the discriminator to go to correct branch.
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>
> Acked-by: Ian Campbell <ian.campbell@citrix>
>
> But, an idea: What about calling the field which refers to the union by
> it's discriminated name, i.e. "u.pv": { map of things } instead of
> "type": "pv", "u" { map of things } ?
>
>
But that breaks API, doesn't it? If we can safely assume that there's no
existing user of this JSON API, changing it won't be hard.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output
2014-05-06 13:09 ` Wei Liu
@ 2014-05-07 10:00 ` Ian Campbell
2014-05-07 10:35 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:00 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Tue, 2014-05-06 at 14:09 +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 02:03:49PM +0100, Ian Campbell wrote:
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > Parser relies on the discriminator to go to correct branch.
> > >
> > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> >
> > Acked-by: Ian Campbell <ian.campbell@citrix>
> >
> > But, an idea: What about calling the field which refers to the union by
> > it's discriminated name, i.e. "u.pv": { map of things } instead of
> > "type": "pv", "u" { map of things } ?
> But that breaks API, doesn't it?
Doh, yes.
> If we can safely assume that there's no
> existing user of this JSON API, changing it won't be hard.
Is it even possible to sanely use things as they are? The existence of
this patch would seem to suggest not?
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output
2014-05-07 10:00 ` Ian Campbell
@ 2014-05-07 10:35 ` Wei Liu
2014-05-07 10:45 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-07 10:35 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Wed, May 07, 2014 at 11:00:26AM +0100, Ian Campbell wrote:
> On Tue, 2014-05-06 at 14:09 +0100, Wei Liu wrote:
> > On Tue, May 06, 2014 at 02:03:49PM +0100, Ian Campbell wrote:
> > > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > > Parser relies on the discriminator to go to correct branch.
> > > >
> > > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > >
> > > Acked-by: Ian Campbell <ian.campbell@citrix>
> > >
> > > But, an idea: What about calling the field which refers to the union by
> > > it's discriminated name, i.e. "u.pv": { map of things } instead of
> > > "type": "pv", "u" { map of things } ?
>
> > But that breaks API, doesn't it?
>
> Doh, yes.
>
> > If we can safely assume that there's no
> > existing user of this JSON API, changing it won't be hard.
>
> Is it even possible to sanely use things as they are? The existence of
> this patch would seem to suggest not?
>
There's actual a horrible way to hack around current implementation: if
you calculate the intersection of those fields in "pv" and "hvm" you can
actually discriminate things. Not sure if a sane person would do it
though! However I would rather stay on the safe side.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output
2014-05-07 10:35 ` Wei Liu
@ 2014-05-07 10:45 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:45 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Wed, 2014-05-07 at 11:35 +0100, Wei Liu wrote:
> On Wed, May 07, 2014 at 11:00:26AM +0100, Ian Campbell wrote:
> > On Tue, 2014-05-06 at 14:09 +0100, Wei Liu wrote:
> > > On Tue, May 06, 2014 at 02:03:49PM +0100, Ian Campbell wrote:
> > > > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > > > Parser relies on the discriminator to go to correct branch.
> > > > >
> > > > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > > >
> > > > Acked-by: Ian Campbell <ian.campbell@citrix>
> > > >
> > > > But, an idea: What about calling the field which refers to the union by
> > > > it's discriminated name, i.e. "u.pv": { map of things } instead of
> > > > "type": "pv", "u" { map of things } ?
> >
> > > But that breaks API, doesn't it?
> >
> > Doh, yes.
> >
> > > If we can safely assume that there's no
> > > existing user of this JSON API, changing it won't be hard.
> >
> > Is it even possible to sanely use things as they are? The existence of
> > this patch would seem to suggest not?
> >
>
> There's actual a horrible way to hack around current implementation: if
> you calculate the intersection of those fields in "pv" and "hvm" you can
> actually discriminate things. Not sure if a sane person would do it
> though!
I think it is a pretty safe bet noone is doing this! (or at least they
probably have no illusions about it being sane!)
> However I would rather stay on the safe side.
I'm leaning towards we change this, although rather than "u.pv" as I
suggested earlier I think "type.pv" (so "discriminator field name"."enum
name") would be better.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 12/24] libxl IDL: generate code to parse libxl__json_object to libxl_FOO struct
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (10 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 11/24] libxl/gentypes.py: include discriminator in JSON output Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 13:45 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 13/24] libxl/gentest.py: test JSON parser Wei Liu
` (12 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
libxl_FOO_parse_json functions are generated.
Note that these functions are used to parse libxl__json_object to
libxl__FOO struct. They don't consume JSON string.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentypes.py | 114 ++++++++++++++++++++++++++++++++++
tools/libxl/idl.py | 15 +++++
tools/libxl/idl.txt | 7 ++-
tools/libxl/libxl_types.idl | 31 +++++----
tools/libxl/libxl_types_internal.idl | 4 +-
5 files changed, 156 insertions(+), 15 deletions(-)
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 8d7183a..3e1140d 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -270,6 +270,99 @@ def libxl_C_type_to_json(ty, v, indent = " "):
s = indent + s
return s.replace("\n", "\n%s" % indent).rstrip(indent)
+def libxl_C_type_parse_json(ty, w, v, indent = " ", parent = None):
+ s = ""
+ if parent is None:
+ s += "int rc = 0;\n"
+ s += "const libxl__json_object *x = o;\n"
+
+ if isinstance(ty, idl.Array):
+ if parent is None:
+ raise Exception("Array type must have a parent")
+ lenvar = parent + ty.lenvar.name
+ s += "{\n"
+ s += " libxl__json_object *t;\n"
+ s += " int i;\n"
+ s += " if (!libxl__json_object_is_array(x)) {\n"
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += " %s = x->u.array->count;\n" % lenvar
+ s += " %s = libxl__calloc(NOGC, %s, sizeof(*%s));\n" % (v, lenvar, v)
+ s += " if (!%s && %s != 0) {\n" % (v, lenvar)
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += " for (i=0; (t=libxl__json_array_get(x,i)); i++) {\n"
+ s += libxl_C_type_parse_json(ty.elem_type, "t", v+"[i]",
+ indent + " ", parent)
+ s += " }\n"
+ s += " if (i != %s) {\n" % lenvar
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += "}\n"
+ elif isinstance(ty, idl.Enumeration):
+ s += "{\n"
+ s += " const char *enum_str;\n"
+ s += " if (!libxl__json_object_is_string(x)) {\n"
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += " enum_str = libxl__json_object_get_string(x);\n"
+ s += " rc = %s_from_string(enum_str, %s);\n" % (ty.typename, ty.pass_arg(v, parent is None, idl.PASS_BY_REFERENCE))
+ s += " if (rc)\n"
+ s += " goto out;\n"
+ s += "}\n"
+ elif isinstance(ty, idl.KeyedUnion):
+ if parent is None:
+ raise Exception("KeyedUnion type must have a parent")
+ s += "switch (%s) {\n" % (parent + ty.keyvar.name)
+ for f in ty.fields:
+ (nparent,fexpr) = ty.member(v, f, parent is None)
+ s += "case %s:\n" % f.enumname
+ if f.type is not None:
+ s += libxl_C_type_parse_json(f.type, w, fexpr, indent + " ", nparent)
+ s += " break;\n"
+ s += "}\n"
+ elif isinstance(ty, idl.Struct) and (parent is None or ty.json_parse_fn is None):
+ for f in [f for f in ty.fields if not f.const and not f.type.private]:
+ if isinstance(f.type, idl.KeyedUnion):
+ s += "x = libxl__json_map_get(\"%s\", %s, %s);\n" % \
+ (f.type.keyvar.name, w, f.type.keyvar.type.json_parse_type)
+ s += "if (x) {\n"
+ (nparent,fexpr) = ty.member(v, f.type.keyvar, parent is None)
+ s += libxl_C_type_parse_json(f.type.keyvar.type, "x", fexpr, " ", nparent)
+ s += "}\n"
+ s += "x = libxl__json_map_get(\"%s\", %s, %s);\n" % (f.name, w, f.type.json_parse_type)
+ s += "if (x) {\n"
+ (nparent,fexpr) = ty.member(v, f, parent is None)
+ s += libxl_C_type_parse_json(f.type, "x", fexpr, " ", nparent)
+ s += " x = x->parent;\n"
+ s += "}\n"
+ else:
+ if ty.json_parse_fn is not None:
+ s += "rc = %s(gc, %s, &%s);\n" % (ty.json_parse_fn, w, v)
+ s += "if (rc)\n"
+ s += " goto out;\n"
+
+ if parent is None:
+ s += "out:\n"
+ s += "return rc;\n"
+
+ if s != "":
+ s = indent +s
+ return s.replace("\n", "\n%s" % indent).rstrip(indent)
+
+def libxl_C_type_from_json(ty, v, w, indent = " "):
+ s = ""
+ parse = "(libxl__json_parse_callback)&%s_parse_json" % ty.typename
+ s += "return libxl__object_from_json(ctx, \"%s\", %s, %s, %s);\n" % (ty.typename, parse, v, w)
+
+ if s != "":
+ s = indent + s
+ return s.replace("\n", "\n%s" % indent).rstrip(indent)
+
def libxl_C_enum_to_string(ty, e, indent = " "):
s = ""
s += "switch(%s) {\n" % e
@@ -348,6 +441,8 @@ if __name__ == '__main__':
ku.keyvar.type.make_arg(ku.keyvar.name)))
if ty.json_gen_fn is not None:
f.write("%schar *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
+ if ty.json_parse_fn is not None:
+ f.write("%sint %s_from_json(libxl_ctx *ctx, %s, const char *s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
if isinstance(ty, idl.Enumeration):
f.write("%sconst char *%s_to_string(%s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
f.write("%sint %s_from_string(const char *s, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("e", passby=idl.PASS_BY_REFERENCE)))
@@ -377,6 +472,10 @@ if __name__ == '__main__':
for ty in [ty for ty in types if ty.json_gen_fn is not None]:
f.write("%syajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+ for ty in [ty for ty in types if ty.json_parse_fn is not None]:
+ f.write("%sint %s_parse_json(libxl__gc *gc, const libxl__json_object *o, %s);\n" % \
+ (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+
f.write("\n")
f.write("""#endif /* %s */\n""" % header_json_define)
f.close()
@@ -444,4 +543,19 @@ if __name__ == '__main__':
f.write("}\n")
f.write("\n")
+ for ty in [t for t in types if t.json_parse_fn is not None]:
+ f.write("int %s_parse_json(libxl__gc *gc, const libxl__json_object *%s, %s)\n" % (ty.typename,"o",ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+ f.write("{\n")
+ f.write(libxl_C_type_parse_json(ty, "o", "p"))
+ f.write("}\n")
+ f.write("\n")
+
+ f.write("int %s_from_json(libxl_ctx *ctx, %s, const char *s)\n" % (ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+ f.write("{\n")
+ if not isinstance(ty, idl.Enumeration):
+ f.write(" %s_init(p);\n" % ty.typename)
+ f.write(libxl_C_type_from_json(ty, "p", "s"))
+ f.write("}\n")
+ f.write("\n")
+
f.close()
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index 92133a3..9ce1e33 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -66,8 +66,12 @@ class Type(object):
if self.typename is not None and not self.private:
self.json_gen_fn = kwargs.setdefault('json_gen_fn', self.typename + "_gen_json")
+ self.json_parse_type = kwargs.setdefault('json_parse_type', "JSON_ANY")
+ self.json_parse_fn = kwargs.setdefault('json_parse_fn', self.typename + "_parse_json")
else:
self.json_gen_fn = kwargs.setdefault('json_gen_fn', None)
+ self.json_parse_type = kwargs.setdefault('json_parse_type', None)
+ self.json_parse_fn = kwargs.setdefault('json_parse_fn', None)
self.autogenerate_json = kwargs.setdefault('autogenerate_json', True)
@@ -119,6 +123,9 @@ class Number(Builtin):
kwargs.setdefault('dispose_fn', None)
kwargs.setdefault('signed', False)
kwargs.setdefault('json_gen_fn', "yajl_gen_integer")
+ kwargs.setdefault('json_parse_type', "JSON_INTEGER")
+ # json_parse_fn might be overriden on specific type
+ kwargs.setdefault('json_parse_fn', "libxl__int_parse_json")
self.signed = kwargs['signed']
Builtin.__init__(self, ctype, **kwargs)
@@ -126,6 +133,7 @@ class UInt(Number):
def __init__(self, w, **kwargs):
kwargs.setdefault('namespace', None)
kwargs.setdefault('dispose_fn', None)
+ kwargs.setdefault('json_parse_fn', "libxl__uint%d_parse_json" % w)
Number.__init__(self, "uint%d_t" % w, **kwargs)
self.width = w
@@ -142,6 +150,7 @@ class EnumerationValue(object):
class Enumeration(Type):
def __init__(self, typename, values, **kwargs):
kwargs.setdefault('dispose_fn', None)
+ kwargs.setdefault('json_parse_type', "JSON_STRING")
Type.__init__(self, typename, **kwargs)
self.value_namespace = kwargs.setdefault('value_namespace',
@@ -171,6 +180,7 @@ class Field(object):
class Aggregate(Type):
"""A type containing a collection of other types"""
def __init__(self, kind, typename, fields, **kwargs):
+ kwargs.setdefault('json_parse_type', "JSON_MAP")
Type.__init__(self, typename, **kwargs)
if self.typename is not None:
@@ -257,6 +267,8 @@ class KeyedUnion(Aggregate):
void = Builtin("void *", namespace = None)
bool = Builtin("bool", namespace = None,
json_gen_fn = "yajl_gen_bool",
+ json_parse_type = "JSON_BOOL",
+ json_parse_fn = "libxl__bool_parse_json",
autogenerate_json = False)
size_t = Number("size_t", namespace = None)
@@ -270,12 +282,15 @@ uint64 = UInt(64)
string = Builtin("char *", namespace = None, dispose_fn = "free",
json_gen_fn = "libxl__string_gen_json",
+ json_parse_type = "JSON_STRING | JSON_NULL",
+ json_parse_fn = "libxl__string_parse_json",
autogenerate_json = False)
class Array(Type):
"""An array of the same type"""
def __init__(self, elem_type, lenvar_name, **kwargs):
kwargs.setdefault('dispose_fn', 'free')
+ kwargs.setdefault('json_parse_type', 'JSON_ARRAY')
Type.__init__(self, namespace=elem_type.namespace, typename=elem_type.rawname + " *", **kwargs)
lv_kwargs = dict([(x.lstrip('lenvar_'),y) for (x,y) in kwargs.items() if x.startswith('lenvar_')])
diff --git a/tools/libxl/idl.txt b/tools/libxl/idl.txt
index 6a53dd8..484d5d7 100644
--- a/tools/libxl/idl.txt
+++ b/tools/libxl/idl.txt
@@ -65,9 +65,14 @@ Type.json_gen_fn: (default: typename + "_gen_json" or None if type == None)
The name of the C function which will generate a YAJL data structure
representing this type.
+Type.json_parse_fn: (default: typename + "_parse_json" or None if type == None)
+
+ The name of the C function which will parse a libxl JSON structure
+ representing this type to C type.
+
Type.autogenerate_json: (default: True)
- Indicates if the above named Type.json_gen_fn should be autogenerated.
+ Indicates if the above named Type.json_*_fn should be autogenerated.
Other simple type-Classes
-------------------------
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 5893260..6cf99fd 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -5,18 +5,23 @@
namespace("libxl_")
-libxl_defbool = Builtin("defbool", passby=PASS_BY_REFERENCE)
-
-libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False)
-libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False, signed = True, init_val="-1")
-libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE)
-libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE)
-libxl_bitmap = Builtin("bitmap", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
-libxl_cpuid_policy_list = Builtin("cpuid_policy_list", dispose_fn="libxl_cpuid_dispose", passby=PASS_BY_REFERENCE)
-
-libxl_string_list = Builtin("string_list", dispose_fn="libxl_string_list_dispose", passby=PASS_BY_REFERENCE)
-libxl_key_value_list = Builtin("key_value_list", dispose_fn="libxl_key_value_list_dispose", passby=PASS_BY_REFERENCE)
-libxl_hwcap = Builtin("hwcap", passby=PASS_BY_REFERENCE)
+libxl_defbool = Builtin("defbool", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+
+libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", json_parse_fn = "libxl__uint32_parse_json",
+ json_parse_type = "JSON_INTEGER", autogenerate_json = False)
+libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", json_parse_fn = "libxl__int_parse_json",
+ json_parse_type = "JSON_INTEGER", autogenerate_json = False, signed = True, init_val="-1")
+libxl_uuid = Builtin("uuid", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+libxl_mac = Builtin("mac", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+libxl_bitmap = Builtin("bitmap", json_parse_type="JSON_ARRAY", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
+libxl_cpuid_policy_list = Builtin("cpuid_policy_list", json_parse_type="JSON_ARRAY",
+ dispose_fn="libxl_cpuid_dispose", passby=PASS_BY_REFERENCE)
+
+libxl_string_list = Builtin("string_list", json_parse_type="JSON_ARRAY",
+ dispose_fn="libxl_string_list_dispose", passby=PASS_BY_REFERENCE)
+libxl_key_value_list = Builtin("key_value_list", json_parse_type="JSON_MAP",
+ dispose_fn="libxl_key_value_list_dispose", passby=PASS_BY_REFERENCE)
+libxl_hwcap = Builtin("hwcap", json_parse_type="JSON_ARRAY", passby=PASS_BY_REFERENCE)
#
# Specific integer types
@@ -581,7 +586,7 @@ libxl_event_type = Enumeration("event_type", [
libxl_ev_user = UInt(64)
-libxl_ev_link = Builtin("ev_link", passby=PASS_BY_REFERENCE, private=True)
+libxl_ev_link = Builtin("ev_link", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE, private=True)
libxl_event = Struct("event",[
("link", libxl_ev_link),
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index a964851..17533f1 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -1,7 +1,9 @@
namespace("libxl__")
hidden(True)
-libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer")
+libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer",
+ json_parse_fn = "libxl__uint32_parse_json", json_parse_type = "JSON_INTEGER",
+ autogenerate_json = False)
libxl__qmp_message_type = Enumeration("qmp_message_type", [
(1, "QMP"),
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 13/24] libxl/gentest.py: test JSON parser
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (11 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 12/24] libxl IDL: generate code to parse libxl__json_object to libxl_FOO struct Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 13:46 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 14/24] libxl: introduce libxl_key_value_list_length Wei Liu
` (11 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
The test is done in following steps:
1. initialise libxl_FOO struct
2. generate JSON string A for libxl_FOO struct FOO1
3. convert JSON string A to libxl_FOO struct FOO2
4. generate JSON string B for libxl_FOO struct FOO2
5. compare A and B
If A and B are identical then we are good.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentest.py | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index eb9a21b..b92d092 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -225,10 +225,11 @@ int main(int argc, char **argv)
""")
for ty in types:
- f.write(" %s %s_val;\n" % (ty.typename, ty.typename))
+ f.write(" %s %s_val, %s_val_new;\n" % \
+ (ty.typename, ty.typename, ty.typename))
f.write("""
int rc;
- char *s;
+ char *s, *new_s;
xentoollog_logger_stdiostream *logger;
libxl_ctx *ctx;
@@ -240,20 +241,36 @@ int main(int argc, char **argv)
exit(1);
}
""")
- f.write(" printf(\"Testing TYPE_to_json()\\n\");\n")
+ f.write(" printf(\"Testing TYPE_to/from_json()\\n\");\n")
f.write(" printf(\"----------------------\\n\");\n")
f.write(" printf(\"\\n\");\n")
for ty in [t for t in types if t.json_gen_fn is not None]:
arg = ty.typename + "_val"
f.write(" %s_rand_init(%s);\n" % (ty.typename, \
ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
+ if not isinstance(ty, idl.Enumeration):
+ f.write(" %s_init(%s_new);\n" % (ty.typename, \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
f.write(" s = %s_to_json(ctx, %s);\n" % \
(ty.typename, ty.pass_arg(arg, isref=False)))
f.write(" printf(\"%%s: %%s\\n\", \"%s\", s);\n" % ty.typename)
f.write(" if (s == NULL) abort();\n")
+ f.write(" rc = %s_from_json(ctx, &%s_val_new, s);\n" % \
+ (ty.typename, ty.typename))
+ f.write(" if (rc) abort();\n")
+ f.write(" new_s = %s_to_json(ctx, %s_new);\n" % \
+ (ty.typename, ty.pass_arg(arg, isref=False)))
+ f.write(" if (new_s == NULL) abort();\n")
+ f.write(" if (strcmp(s, new_s)) {\n")
+ f.write(" printf(\"Huh? Regenerated string different from original string.\\n\");\n")
+ f.write(" printf(\"Regenerated string: %s\\n\", new_s);\n")
+ f.write(" abort();\n")
+ f.write(" }\n")
f.write(" free(s);\n")
+ f.write(" free(new_s);\n")
if ty.dispose_fn is not None:
f.write(" %s(&%s_val);\n" % (ty.dispose_fn, ty.typename))
+ f.write(" %s(&%s_val_new);\n" % (ty.dispose_fn, ty.typename))
f.write("\n")
f.write(" printf(\"Testing Enumerations\\n\");\n")
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 13/24] libxl/gentest.py: test JSON parser
2014-05-01 12:58 ` [PATCH V4 13/24] libxl/gentest.py: test JSON parser Wei Liu
@ 2014-05-06 13:46 ` Ian Campbell
2014-05-06 14:11 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 13:46 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> The test is done in following steps:
>
> 1. initialise libxl_FOO struct
> 2. generate JSON string A for libxl_FOO struct FOO1
> 3. convert JSON string A to libxl_FOO struct FOO2
> 4. generate JSON string B for libxl_FOO struct FOO2
> 5. compare A and B
>
> If A and B are identical then we are good.
I presume you have run this and we are good?
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 13/24] libxl/gentest.py: test JSON parser
2014-05-06 13:46 ` Ian Campbell
@ 2014-05-06 14:11 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:11 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 02:46:41PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > The test is done in following steps:
> >
> > 1. initialise libxl_FOO struct
> > 2. generate JSON string A for libxl_FOO struct FOO1
> > 3. convert JSON string A to libxl_FOO struct FOO2
> > 4. generate JSON string B for libxl_FOO struct FOO2
> > 5. compare A and B
> >
> > If A and B are identical then we are good.
>
> I presume you have run this and we are good?
>
Yes, I ran it with a script for several hours. It worked fine. I also
deliberately increased the lenght of array to stress it more.
$!/bin/sh
XL=/local/scratch/xen-toolstack.git/tools/libxl
while true;
do
touch $XL/gentest.py
make -C $XL/ >/dev/null
scp $XL/testidl root@h16:~
ssh root@h16 /root/testidl
if [ $? -ne 0 ]; then
echo "FAIL"
exit 255
fi
done
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>
> Acked-by: Ian Campbell <ian.campbell@citrix.com>
>
Thanks
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 14/24] libxl: introduce libxl_key_value_list_length
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (12 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 13/24] libxl/gentest.py: test JSON parser Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 13:50 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 15/24] libxl: introduce libxl_cpuid_policy_list_length Wei Liu
` (10 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.c | 12 ++++++++++++
tools/libxl/libxl.h | 1 +
2 files changed, 13 insertions(+)
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index dbba7ca..08c4f54 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -216,6 +216,18 @@ int libxl_string_list_length(const libxl_string_list *psl)
return i;
}
+int libxl_key_value_list_length(libxl_key_value_list *pkvl)
+{
+ int i = 0;
+ libxl_key_value_list kvl = *pkvl;
+
+ if (kvl)
+ while (kvl[i])
+ i += 2;
+
+ return i / 2;
+}
+
void libxl_key_value_list_dispose(libxl_key_value_list *pkvl)
{
int i;
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 24f422d..7d0351b 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -490,6 +490,7 @@ int libxl_string_list_length(const libxl_string_list *sl);
typedef char **libxl_key_value_list;
void libxl_key_value_list_dispose(libxl_key_value_list *kvl);
+int libxl_key_value_list_length(libxl_key_value_list *kvl);
typedef uint32_t libxl_hwcap[8];
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 14/24] libxl: introduce libxl_key_value_list_length
2014-05-01 12:58 ` [PATCH V4 14/24] libxl: introduce libxl_key_value_list_length Wei Liu
@ 2014-05-06 13:50 ` Ian Campbell
2014-05-06 14:13 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 13:50 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/libxl.c | 12 ++++++++++++
> tools/libxl/libxl.h | 1 +
> 2 files changed, 13 insertions(+)
>
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index dbba7ca..08c4f54 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -216,6 +216,18 @@ int libxl_string_list_length(const libxl_string_list *psl)
> return i;
> }
>
> +int libxl_key_value_list_length(libxl_key_value_list *pkvl)
> +{
> + int i = 0;
> + libxl_key_value_list kvl = *pkvl;
> +
> + if (kvl)
> + while (kvl[i])
> + i += 2;
> +
> + return i / 2;
"+= 2" *and* "/ 2"?
Oh I see, values can be NULL.
It might sound stupid but:
while(kvl[i]) {
i++; /* Key */
i++; /* Values, could be NULL */
might be clearer?
Or:
while (kvl[2*i]) /* Only check keys */
i++
return i;
The key think about both proposals is the presence of a comment ;-)
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 14/24] libxl: introduce libxl_key_value_list_length
2014-05-06 13:50 ` Ian Campbell
@ 2014-05-06 14:13 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:13 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 02:50:04PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/libxl.c | 12 ++++++++++++
> > tools/libxl/libxl.h | 1 +
> > 2 files changed, 13 insertions(+)
> >
> > diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> > index dbba7ca..08c4f54 100644
> > --- a/tools/libxl/libxl.c
> > +++ b/tools/libxl/libxl.c
> > @@ -216,6 +216,18 @@ int libxl_string_list_length(const libxl_string_list *psl)
> > return i;
> > }
> >
> > +int libxl_key_value_list_length(libxl_key_value_list *pkvl)
> > +{
> > + int i = 0;
> > + libxl_key_value_list kvl = *pkvl;
> > +
> > + if (kvl)
> > + while (kvl[i])
> > + i += 2;
> > +
> > + return i / 2;
>
> "+= 2" *and* "/ 2"?
>
> Oh I see, values can be NULL.
>
> It might sound stupid but:
> while(kvl[i]) {
> i++; /* Key */
> i++; /* Values, could be NULL */
> might be clearer?
>
> Or:
> while (kvl[2*i]) /* Only check keys */
> i++
> return i;
>
> The key think about both proposals is the presence of a comment ;-)
>
NP. I will use the second form.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 15/24] libxl: introduce libxl_cpuid_policy_list_length
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (13 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 14/24] libxl: introduce libxl_key_value_list_length Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 13:51 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 16/24] libxl: copy function for builtin types Wei Liu
` (9 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.h | 1 +
tools/libxl/libxl_cpuid.c | 11 +++++++++++
2 files changed, 12 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 7d0351b..3755bad 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -510,6 +510,7 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
typedef struct libxl__cpuid_policy libxl_cpuid_policy;
typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
+int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l);
#define LIBXL_PCI_FUNC_ALL (~0U)
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 45d2d9a..7f27c67 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -451,6 +451,17 @@ int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
return 0;
}
+int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
+{
+ int i = 0;
+
+ if (*l)
+ while ((*l)[i].input[0] != XEN_CPUID_INPUT_UNUSED)
+ i++;
+
+ return i;
+}
+
/*
* Local variables:
* mode: C
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 15/24] libxl: introduce libxl_cpuid_policy_list_length
2014-05-01 12:58 ` [PATCH V4 15/24] libxl: introduce libxl_cpuid_policy_list_length Wei Liu
@ 2014-05-06 13:51 ` Ian Campbell
2014-05-06 14:17 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 13:51 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/libxl.h | 1 +
> tools/libxl/libxl_cpuid.c | 11 +++++++++++
> 2 files changed, 12 insertions(+)
>
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 7d0351b..3755bad 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -510,6 +510,7 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
> typedef struct libxl__cpuid_policy libxl_cpuid_policy;
> typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
> void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
> +int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l);
>
> #define LIBXL_PCI_FUNC_ALL (~0U)
>
> diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> index 45d2d9a..7f27c67 100644
> --- a/tools/libxl/libxl_cpuid.c
> +++ b/tools/libxl/libxl_cpuid.c
> @@ -451,6 +451,17 @@ int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> return 0;
> }
>
> +int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> +{
> + int i = 0;
> +
> + if (*l)
> + while ((*l)[i].input[0] != XEN_CPUID_INPUT_UNUSED)
Other functions of this type would assign *l to a temporary, I think
this should do the same. The prevalent style would make the parameter
*pl and then l = *pl.
> + i++;
> +
> + return i;
> +}
> +
> /*
> * Local variables:
> * mode: C
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 15/24] libxl: introduce libxl_cpuid_policy_list_length
2014-05-06 13:51 ` Ian Campbell
@ 2014-05-06 14:17 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:17 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 02:51:48PM +0100, Ian Campbell wrote:
[...]
> >
> > +int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> > +{
> > + int i = 0;
> > +
> > + if (*l)
> > + while ((*l)[i].input[0] != XEN_CPUID_INPUT_UNUSED)
>
> Other functions of this type would assign *l to a temporary, I think
> this should do the same. The prevalent style would make the parameter
> *pl and then l = *pl.
>
No problem. ;-)
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (14 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 15/24] libxl: introduce libxl_cpuid_policy_list_length Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 14:03 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 17/24] libxl IDL: generate deep copy functions Wei Liu
` (8 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
These functions will be used in later patch to deep-copy a structure.
Functions introduced:
* libxl_string_list_copy
* libxl_key_value_list_copy
* libxl_hwcap_copy
* libxl_mac_copy
* libxl_cpuid_policy_list_copy
* libxl_string_copy
* libxl_bitmap_copy_alloc
* libxl_uuid_copy_ctx
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.c | 67 +++++++++++++++++++++++++++++++++++++++++++
tools/libxl/libxl.h | 14 +++++++--
tools/libxl/libxl_cpuid.c | 33 +++++++++++++++++++++
tools/libxl/libxl_nocpuid.c | 6 ++++
tools/libxl/libxl_utils.c | 25 ++++++++++++++++
tools/libxl/libxl_utils.h | 4 +++
tools/libxl/libxl_uuid.c | 12 ++++++++
tools/libxl/libxl_uuid.h | 4 +++
8 files changed, 163 insertions(+), 2 deletions(-)
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 08c4f54..3e79f61 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -205,6 +205,29 @@ void libxl_string_list_dispose(libxl_string_list *psl)
free(sl);
}
+void libxl_string_list_copy(libxl_ctx *ctx,
+ libxl_string_list *dst,
+ libxl_string_list *src)
+{
+ GC_INIT(ctx);
+ int i, len;
+
+ if (!*src) {
+ *dst = NULL;
+ goto out;
+ }
+
+ len = libxl_string_list_length(src);
+ /* one extra slot for sentinel */
+ *dst = libxl__calloc(NOGC, len + 1, sizeof(char *));
+
+ for (i = 0; i < len; i++)
+ (*dst)[i] = libxl__strdup(NOGC, (*src)[i]);
+
+out:
+ GC_FREE;
+}
+
int libxl_string_list_length(const libxl_string_list *psl)
{
int i = 0;
@@ -244,6 +267,34 @@ void libxl_key_value_list_dispose(libxl_key_value_list *pkvl)
free(kvl);
}
+void libxl_key_value_list_copy(libxl_ctx *ctx,
+ libxl_key_value_list *dst,
+ libxl_key_value_list *src)
+{
+ GC_INIT(ctx);
+ int i, len;
+
+ if (*src == NULL) {
+ *dst = NULL;
+ goto out;
+ }
+
+ len = libxl_key_value_list_length(src);
+ /* one extra slot for sentinel */
+ *dst = libxl__calloc(NOGC, len * 2 + 1, sizeof(char *));
+
+ for (i = 0; i < len * 2; i += 2) {
+ (*dst)[i] = libxl__strdup(NOGC, (*src)[i]);
+ if ((*src)[i+1])
+ (*dst)[i+1] = libxl__strdup(NOGC, (*src)[i+1]);
+ else
+ (*dst)[i+1] = NULL;
+ }
+
+out:
+ GC_FREE;
+}
+
void libxl_defbool_set(libxl_defbool *db, bool b)
{
db->val = b ? LIBXL__DEFBOOL_TRUE : LIBXL__DEFBOOL_FALSE;
@@ -5680,6 +5731,22 @@ int libxl_fd_set_cloexec(libxl_ctx *ctx, int fd, int cloexec)
int libxl_fd_set_nonblock(libxl_ctx *ctx, int fd, int nonblock)
{ return fd_set_flags(ctx,fd, F_GETFL,F_SETFL,"FL", O_NONBLOCK, nonblock); }
+
+void libxl_hwcap_copy(libxl_ctx *ctx,libxl_hwcap *dst, libxl_hwcap *src)
+{
+ int i;
+
+ for (i = 0; i < 8; i++)
+ (*dst)[i] = (*src)[i];
+}
+
+void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, libxl_mac *src)
+{
+ int i;
+
+ for (i = 0; i < 6; i++)
+ (*dst)[i] = (*src)[i];
+}
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 3755bad..c6a9a0d 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -479,20 +479,29 @@
*/
#define LIBXL_HAVE_SSID_AND_CPUPOOLID_PARSING 1
+typedef struct libxl__ctx libxl_ctx;
+
typedef uint8_t libxl_mac[6];
#define LIBXL_MAC_FMT "%02hhx:%02hhx:%02hhx:%02hhx:%02hhx:%02hhx"
#define LIBXL_MAC_FMTLEN ((2*6)+5) /* 6 hex bytes plus 5 colons */
#define LIBXL_MAC_BYTES(mac) mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]
+void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, libxl_mac *src);
typedef char **libxl_string_list;
void libxl_string_list_dispose(libxl_string_list *sl);
int libxl_string_list_length(const libxl_string_list *sl);
+void libxl_string_list_copy(libxl_ctx *ctx, libxl_string_list *dst,
+ libxl_string_list *src);
typedef char **libxl_key_value_list;
void libxl_key_value_list_dispose(libxl_key_value_list *kvl);
int libxl_key_value_list_length(libxl_key_value_list *kvl);
+void libxl_key_value_list_copy(libxl_ctx *ctx,
+ libxl_key_value_list *dst,
+ libxl_key_value_list *src);
typedef uint32_t libxl_hwcap[8];
+void libxl_hwcap_copy(libxl_ctx *ctx, libxl_hwcap *dst, libxl_hwcap *src);
typedef uint64_t libxl_ev_user;
@@ -511,6 +520,9 @@ typedef struct libxl__cpuid_policy libxl_cpuid_policy;
typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l);
+void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
+ libxl_cpuid_policy_list *dst,
+ libxl_cpuid_policy_list *src);
#define LIBXL_PCI_FUNC_ALL (~0U)
@@ -561,8 +573,6 @@ bool libxl_defbool_val(libxl_defbool db);
const char *libxl_defbool_to_string(libxl_defbool b);
-typedef struct libxl__ctx libxl_ctx;
-
#define LIBXL_TIMER_MODE_DEFAULT -1
#define LIBXL_MEMKB_DEFAULT ~0ULL
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 7f27c67..aab9d0a 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -462,6 +462,39 @@ int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
return i;
}
+void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
+ libxl_cpuid_policy_list *dst,
+ libxl_cpuid_policy_list *src)
+{
+ GC_INIT(ctx);
+ int i, j, len;
+
+ if (*src == NULL) {
+ *dst = NULL;
+ goto out;
+ }
+
+ len = libxl_cpuid_policy_list_length(src);
+ /* one extra slot for sentinel */
+ *dst = libxl__calloc(NOGC, len + 1, sizeof(libxl_cpuid_policy));
+ (*dst)[len].input[0] = XEN_CPUID_INPUT_UNUSED;
+ (*dst)[len].input[1] = XEN_CPUID_INPUT_UNUSED;
+
+ for (i = 0; i < len; i++) {
+ for (j = 0; j < 2; j++)
+ (*dst)[i].input[j] = (*src)[i].input[j];
+ for (j = 0; j < 4; j++)
+ if ((*src)[i].policy[j])
+ (*dst)[i].policy[j] =
+ libxl__strdup(NOGC, (*src)[i].policy[j]);
+ else
+ (*dst)[i].policy[j] = NULL;
+ }
+
+out:
+ GC_FREE;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
index eb525fc..698d252 100644
--- a/tools/libxl/libxl_nocpuid.c
+++ b/tools/libxl/libxl_nocpuid.c
@@ -51,6 +51,12 @@ int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
return 0;
}
+void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
+ libxl_cpuid_policy_list *dst,
+ libxl_cpuid_policy_list *src)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 476921e..0f85579 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -614,6 +614,19 @@ void libxl_bitmap_copy(libxl_ctx *ctx, libxl_bitmap *dptr,
memcpy(dptr->map, sptr->map, sz * sizeof(*dptr->map));
}
+void libxl_bitmap_copy_alloc(libxl_ctx *ctx,
+ libxl_bitmap *dptr,
+ const libxl_bitmap *sptr)
+{
+ GC_INIT(ctx);
+
+ dptr->map = libxl__calloc(NOGC, sptr->size, sizeof(*sptr->map));
+ dptr->size = sptr->size;
+ memcpy(dptr->map, sptr->map, sptr->size * sizeof(*sptr->map));
+
+ GC_FREE;
+}
+
int libxl_bitmap_is_full(const libxl_bitmap *bitmap)
{
int i;
@@ -1013,6 +1026,18 @@ int libxl_domid_valid_guest(uint32_t domid)
return domid > 0 && domid < DOMID_FIRST_RESERVED;
}
+void libxl_string_copy(libxl_ctx *ctx, char **dst, char **src)
+{
+ GC_INIT(ctx);
+
+ if (*src)
+ *dst = libxl__strdup(NOGC, *src);
+ else
+ *dst = NULL;
+
+ GC_FREE;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h
index 8bfb81b..cc528d2 100644
--- a/tools/libxl/libxl_utils.h
+++ b/tools/libxl/libxl_utils.h
@@ -76,6 +76,8 @@ int libxl_devid_to_device_vtpm(libxl_ctx *ctx, uint32_t domid,
int libxl_bitmap_alloc(libxl_ctx *ctx, libxl_bitmap *bitmap, int n_bits);
/* Allocated bimap is from malloc, libxl_bitmap_dispose() to be
* called by the application when done. */
+void libxl_bitmap_copy_alloc(libxl_ctx *ctx, libxl_bitmap *dptr,
+ const libxl_bitmap *sptr);
void libxl_bitmap_copy(libxl_ctx *ctx, libxl_bitmap *dptr,
const libxl_bitmap *sptr);
int libxl_bitmap_is_full(const libxl_bitmap *bitmap);
@@ -121,6 +123,8 @@ int libxl_cpumap_to_nodemap(libxl_ctx *ctx,
return (s + 1023) / 1024;
}
+void libxl_string_copy(libxl_ctx *ctx, char **dst, char **src);
+
#endif
/*
diff --git a/tools/libxl/libxl_uuid.c b/tools/libxl/libxl_uuid.c
index ecc29c7..1941883 100644
--- a/tools/libxl/libxl_uuid.c
+++ b/tools/libxl/libxl_uuid.c
@@ -40,6 +40,12 @@ void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src)
uuid_copy(dst->uuid, src->uuid);
}
+void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
+ const libxl_uuid *src)
+{
+ uuid_copy(dst->uuid, src->uuid);
+}
+
void libxl_uuid_clear(libxl_uuid *uuid)
{
uuid_clear(uuid->uuid);
@@ -87,6 +93,12 @@ void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src)
memcpy(dst->uuid, src->uuid, sizeof(dst->uuid));
}
+void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
+ const libxl_uuid *src)
+{
+ memcpy(dst->uuid, src->uuid, sizeof(dst->uuid));
+}
+
void libxl_uuid_clear(libxl_uuid *uuid)
{
memset(uuid->uuid, 0, sizeof(uuid->uuid));
diff --git a/tools/libxl/libxl_uuid.h b/tools/libxl/libxl_uuid.h
index 93c65a7..541f0f8 100644
--- a/tools/libxl/libxl_uuid.h
+++ b/tools/libxl/libxl_uuid.h
@@ -53,10 +53,14 @@ typedef struct {
#endif
+typedef struct libxl__ctx libxl_ctx;
+
int libxl_uuid_is_nil(libxl_uuid *uuid);
void libxl_uuid_generate(libxl_uuid *uuid);
int libxl_uuid_from_string(libxl_uuid *uuid, const char *in);
void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src);
+void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
+ const libxl_uuid *src);
void libxl_uuid_clear(libxl_uuid *uuid);
int libxl_uuid_compare(libxl_uuid *uuid1, libxl_uuid *uuid2);
uint8_t *libxl_uuid_bytearray(libxl_uuid *uuid);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-01 12:58 ` [PATCH V4 16/24] libxl: copy function for builtin types Wei Liu
@ 2014-05-06 14:03 ` Ian Campbell
2014-05-06 14:36 ` Wei Liu
2014-05-07 10:47 ` Ian Jackson
0 siblings, 2 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 14:03 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> These functions will be used in later patch to deep-copy a structure.
Please can you document this in libxl.h alongside _init and _dispose
etc. Can you do parse_json and gen_json too (I know the second isn't
your mess, it's mine, sorry!).
> diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> index 7f27c67..aab9d0a 100644
> --- a/tools/libxl/libxl_cpuid.c
> +++ b/tools/libxl/libxl_cpuid.c
> @@ -462,6 +462,39 @@ int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> return i;
> }
>
> +void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
> + libxl_cpuid_policy_list *dst,
> + libxl_cpuid_policy_list *src)
Just picking on this one at random, should they return an int even if
all the current uses can only return success? Is there any conceivable
way for any of these functions to fail (other than enomem which is
handled already). e.g. ERROR_INVALID because the input is bogus perhaps.
For consistency if any one can fail I think they should all return an
error code.
> diff --git a/tools/libxl/libxl_uuid.h b/tools/libxl/libxl_uuid.h
> index 93c65a7..541f0f8 100644
> --- a/tools/libxl/libxl_uuid.h
> +++ b/tools/libxl/libxl_uuid.h
> @@ -53,10 +53,14 @@ typedef struct {
>
> #endif
>
> +typedef struct libxl__ctx libxl_ctx;
> +
> int libxl_uuid_is_nil(libxl_uuid *uuid);
> void libxl_uuid_generate(libxl_uuid *uuid);
> int libxl_uuid_from_string(libxl_uuid *uuid, const char *in);
> void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src);
> +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> + const libxl_uuid *src);
Hrm, this is rather unfortunate.
All of these only take a ctx so they can use NOGC. I wonder if a #define
INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
nicer than this?
Ian? What do you think?
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-06 14:03 ` Ian Campbell
@ 2014-05-06 14:36 ` Wei Liu
2014-05-07 10:05 ` Ian Campbell
2014-05-07 10:47 ` Ian Jackson
1 sibling, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:36 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 03:03:13PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > These functions will be used in later patch to deep-copy a structure.
>
> Please can you document this in libxl.h alongside _init and _dispose
> etc. Can you do parse_json and gen_json too (I know the second isn't
> your mess, it's mine, sorry!).
No problem. Now this series is one patch longer (I plan to document
gen_json in a separate patch)! Hope that won't be too inconvenient for
you. :-)
> > diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> > index 7f27c67..aab9d0a 100644
> > --- a/tools/libxl/libxl_cpuid.c
> > +++ b/tools/libxl/libxl_cpuid.c
> > @@ -462,6 +462,39 @@ int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> > return i;
> > }
> >
> > +void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
> > + libxl_cpuid_policy_list *dst,
> > + libxl_cpuid_policy_list *src)
>
> Just picking on this one at random, should they return an int even if
> all the current uses can only return success? Is there any conceivable
> way for any of these functions to fail (other than enomem which is
> handled already). e.g. ERROR_INVALID because the input is bogus perhaps.
Shouldn't the input already be sanatized at this point?
Or, does it really matter if the input is bogus? Is it really copy
function's job to sanitize input? I would expect the user of this struct
to do the right thing when it encounters bogus input, not the copy
function.
> For consistency if any one can fail I think they should all return an
> error code.
>
I can do this. But if we don't sanitize the input here, the only thing
that can fail is memory allocation, and libxl__calloc calls _exit, we
cannot return to caller anyway.
Wei.
> > diff --git a/tools/libxl/libxl_uuid.h b/tools/libxl/libxl_uuid.h
> > index 93c65a7..541f0f8 100644
> > --- a/tools/libxl/libxl_uuid.h
> > +++ b/tools/libxl/libxl_uuid.h
> > @@ -53,10 +53,14 @@ typedef struct {
> >
> > #endif
> >
> > +typedef struct libxl__ctx libxl_ctx;
> > +
> > int libxl_uuid_is_nil(libxl_uuid *uuid);
> > void libxl_uuid_generate(libxl_uuid *uuid);
> > int libxl_uuid_from_string(libxl_uuid *uuid, const char *in);
> > void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src);
> > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > + const libxl_uuid *src);
>
> Hrm, this is rather unfortunate.
>
> All of these only take a ctx so they can use NOGC. I wonder if a #define
> INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> nicer than this?
>
> Ian? What do you think?
>
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-06 14:36 ` Wei Liu
@ 2014-05-07 10:05 ` Ian Campbell
2014-05-07 10:19 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:05 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
(Ian J -- discussion about NOGC at the bottom may be of interest)
On Tue, 2014-05-06 at 15:36 +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 03:03:13PM +0100, Ian Campbell wrote:
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > These functions will be used in later patch to deep-copy a structure.
> >
> > Please can you document this in libxl.h alongside _init and _dispose
> > etc. Can you do parse_json and gen_json too (I know the second isn't
> > your mess, it's mine, sorry!).
>
> No problem. Now this series is one patch longer (I plan to document
> gen_json in a separate patch)! Hope that won't be too inconvenient for
> you. :-)
What's one more patch :-P
>
> > > diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> > > index 7f27c67..aab9d0a 100644
> > > --- a/tools/libxl/libxl_cpuid.c
> > > +++ b/tools/libxl/libxl_cpuid.c
> > > @@ -462,6 +462,39 @@ int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> > > return i;
> > > }
> > >
> > > +void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
> > > + libxl_cpuid_policy_list *dst,
> > > + libxl_cpuid_policy_list *src)
> >
> > Just picking on this one at random, should they return an int even if
> > all the current uses can only return success? Is there any conceivable
> > way for any of these functions to fail (other than enomem which is
> > handled already). e.g. ERROR_INVALID because the input is bogus perhaps.
>
> Shouldn't the input already be sanatized at this point?
By what?
> Or, does it really matter if the input is bogus? Is it really copy
> function's job to sanitize input? I would expect the user of this struct
> to do the right thing when it encounters bogus input, not the copy
> function.
The caller may not know it is bogus though, it may only come to light
when the copy tries and fails.
Perhaps consider the case where to caller is buggy (so it by definition
doesn't know the input is buggy)? For example what do you do if the
discriminatator of a keyed enum is invalid? I suppose you could do
nothing with the union.
> > For consistency if any one can fail I think they should all return an
> > error code.
> >
>
> I can do this. But if we don't sanitize the input here, the only thing
> that can fail is memory allocation, and libxl__calloc calls _exit, we
> cannot return to caller anyway.
>
> Wei.
Did you see this final comment?:
> > > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > > + const libxl_uuid *src);
> >
> > Hrm, this is rather unfortunate.
> >
> > All of these only take a ctx so they can use NOGC. I wonder if a #define
> > INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> > nicer than this?
> >
> > Ian? What do you think?
> >
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-07 10:05 ` Ian Campbell
@ 2014-05-07 10:19 ` Wei Liu
2014-05-07 10:40 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-07 10:19 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Wed, May 07, 2014 at 11:05:14AM +0100, Ian Campbell wrote:
> (Ian J -- discussion about NOGC at the bottom may be of interest)
>
> On Tue, 2014-05-06 at 15:36 +0100, Wei Liu wrote:
> > On Tue, May 06, 2014 at 03:03:13PM +0100, Ian Campbell wrote:
> > > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > > These functions will be used in later patch to deep-copy a structure.
> > >
> > > Please can you document this in libxl.h alongside _init and _dispose
> > > etc. Can you do parse_json and gen_json too (I know the second isn't
> > > your mess, it's mine, sorry!).
> >
> > No problem. Now this series is one patch longer (I plan to document
> > gen_json in a separate patch)! Hope that won't be too inconvenient for
> > you. :-)
>
> What's one more patch :-P
>
> >
> > > > diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> > > > index 7f27c67..aab9d0a 100644
> > > > --- a/tools/libxl/libxl_cpuid.c
> > > > +++ b/tools/libxl/libxl_cpuid.c
> > > > @@ -462,6 +462,39 @@ int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> > > > return i;
> > > > }
> > > >
> > > > +void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
> > > > + libxl_cpuid_policy_list *dst,
> > > > + libxl_cpuid_policy_list *src)
> > >
> > > Just picking on this one at random, should they return an int even if
> > > all the current uses can only return success? Is there any conceivable
> > > way for any of these functions to fail (other than enomem which is
> > > handled already). e.g. ERROR_INVALID because the input is bogus perhaps.
> >
> > Shouldn't the input already be sanatized at this point?
>
> By what?
>
By the caller. How come there's unsanatized value in src? If there's,
then the problem should be addressed by other part of the code base.
> > Or, does it really matter if the input is bogus? Is it really copy
> > function's job to sanitize input? I would expect the user of this struct
> > to do the right thing when it encounters bogus input, not the copy
> > function.
>
> The caller may not know it is bogus though, it may only come to light
> when the copy tries and fails.
>
Even if the caller is buggy and corrupts the input value, I think the
consumer in libxl (say, libxl_domain_create_new) should be able to
handle this. That is, my intent is to fail in actual consumer of the
structure but not the copy function.
> Perhaps consider the case where to caller is buggy (so it by definition
> doesn't know the input is buggy)? For example what do you do if the
> discriminatator of a keyed enum is invalid? I suppose you could do
> nothing with the union.
>
"keyed enum"? Did you mean "keyed union"?
Then that union is not touched at all.
> > > For consistency if any one can fail I think they should all return an
> > > error code.
> > >
> >
> > I can do this. But if we don't sanitize the input here, the only thing
> > that can fail is memory allocation, and libxl__calloc calls _exit, we
> > cannot return to caller anyway.
> >
> > Wei.
>
> Did you see this final comment?:
>
> > > > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > > > + const libxl_uuid *src);
> > >
> > > Hrm, this is rather unfortunate.
> > >
> > > All of these only take a ctx so they can use NOGC. I wonder if a #define
> > > INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> > > nicer than this?
> > >
> > > Ian? What do you think?
> > >
>
Yes I saw this, but I thought it was addressed to Ian J so I somehow
skipped it. I think INIT_NOGC macro will do. If both of you agree I
will write yet another patch for this. :-P
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-07 10:19 ` Wei Liu
@ 2014-05-07 10:40 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:40 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Wed, 2014-05-07 at 11:19 +0100, Wei Liu wrote:
> On Wed, May 07, 2014 at 11:05:14AM +0100, Ian Campbell wrote:
> > (Ian J -- discussion about NOGC at the bottom may be of interest)
> >
> > On Tue, 2014-05-06 at 15:36 +0100, Wei Liu wrote:
> > > On Tue, May 06, 2014 at 03:03:13PM +0100, Ian Campbell wrote:
> > > > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > > > These functions will be used in later patch to deep-copy a structure.
> > > >
> > > > Please can you document this in libxl.h alongside _init and _dispose
> > > > etc. Can you do parse_json and gen_json too (I know the second isn't
> > > > your mess, it's mine, sorry!).
> > >
> > > No problem. Now this series is one patch longer (I plan to document
> > > gen_json in a separate patch)! Hope that won't be too inconvenient for
> > > you. :-)
> >
> > What's one more patch :-P
> >
> > >
> > > > > diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> > > > > index 7f27c67..aab9d0a 100644
> > > > > --- a/tools/libxl/libxl_cpuid.c
> > > > > +++ b/tools/libxl/libxl_cpuid.c
> > > > > @@ -462,6 +462,39 @@ int libxl_cpuid_policy_list_length(libxl_cpuid_policy_list *l)
> > > > > return i;
> > > > > }
> > > > >
> > > > > +void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
> > > > > + libxl_cpuid_policy_list *dst,
> > > > > + libxl_cpuid_policy_list *src)
> > > >
> > > > Just picking on this one at random, should they return an int even if
> > > > all the current uses can only return success? Is there any conceivable
> > > > way for any of these functions to fail (other than enomem which is
> > > > handled already). e.g. ERROR_INVALID because the input is bogus perhaps.
> > >
> > > Shouldn't the input already be sanatized at this point?
> >
> > By what?
> >
>
> By the caller. How come there's unsanatized value in src? If there's,
> then the problem should be addressed by other part of the code base.
>
> > > Or, does it really matter if the input is bogus? Is it really copy
> > > function's job to sanitize input? I would expect the user of this struct
> > > to do the right thing when it encounters bogus input, not the copy
> > > function.
> >
> > The caller may not know it is bogus though, it may only come to light
> > when the copy tries and fails.
> >
>
> Even if the caller is buggy and corrupts the input value, I think the
> consumer in libxl (say, libxl_domain_create_new) should be able to
> handle this. That is, my intent is to fail in actual consumer of the
> structure but not the copy function.
I suppose that makes sense.
> > Perhaps consider the case where to caller is buggy (so it by definition
> > doesn't know the input is buggy)? For example what do you do if the
> > discriminatator of a keyed enum is invalid? I suppose you could do
> > nothing with the union.
> >
>
> "keyed enum"? Did you mean "keyed union"?
Yes.
>
> Then that union is not touched at all.
>
> > > > For consistency if any one can fail I think they should all return an
> > > > error code.
> > > >
> > >
> > > I can do this. But if we don't sanitize the input here, the only thing
> > > that can fail is memory allocation, and libxl__calloc calls _exit, we
> > > cannot return to caller anyway.
> > >
> > > Wei.
> >
> > Did you see this final comment?:
> >
> > > > > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > > > > + const libxl_uuid *src);
> > > >
> > > > Hrm, this is rather unfortunate.
> > > >
> > > > All of these only take a ctx so they can use NOGC. I wonder if a #define
> > > > INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> > > > nicer than this?
> > > >
> > > > Ian? What do you think?
> > > >
> >
>
> Yes I saw this, but I thought it was addressed to Ian J so I somehow
> skipped it. I think INIT_NOGC macro will do. If both of you agree I
> will write yet another patch for this. :-P
>
> Wei.
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-06 14:03 ` Ian Campbell
2014-05-06 14:36 ` Wei Liu
@ 2014-05-07 10:47 ` Ian Jackson
2014-05-07 10:57 ` Ian Campbell
1 sibling, 1 reply; 120+ messages in thread
From: Ian Jackson @ 2014-05-07 10:47 UTC (permalink / raw)
To: Ian Campbell; +Cc: Wei Liu, xen-devel
Ian Campbell writes ("Re: [PATCH V4 16/24] libxl: copy function for builtin types"):
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > +typedef struct libxl__ctx libxl_ctx;
> > +
> > int libxl_uuid_is_nil(libxl_uuid *uuid);
> > void libxl_uuid_generate(libxl_uuid *uuid);
> > int libxl_uuid_from_string(libxl_uuid *uuid, const char *in);
> > void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src);
> > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > + const libxl_uuid *src);
>
> Hrm, this is rather unfortunate.
>
> All of these only take a ctx so they can use NOGC. I wonder if a #define
> INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> nicer than this?
The problem with this is that the caller's log functions won't be
respected. I think it would be better to make all of the copy
functions take a ctx.
Alternatively we should at least document this restriction, but also
then we shouldn't ever call the copy functions from within libxl which
(without reading the rest of the series) I think would probably defeat
the point.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-07 10:47 ` Ian Jackson
@ 2014-05-07 10:57 ` Ian Campbell
2014-05-07 12:55 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:57 UTC (permalink / raw)
To: Ian Jackson; +Cc: Wei Liu, xen-devel
On Wed, 2014-05-07 at 11:47 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [PATCH V4 16/24] libxl: copy function for builtin types"):
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > +typedef struct libxl__ctx libxl_ctx;
> > > +
> > > int libxl_uuid_is_nil(libxl_uuid *uuid);
> > > void libxl_uuid_generate(libxl_uuid *uuid);
> > > int libxl_uuid_from_string(libxl_uuid *uuid, const char *in);
> > > void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src);
> > > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > > + const libxl_uuid *src);
> >
> > Hrm, this is rather unfortunate.
> >
> > All of these only take a ctx so they can use NOGC. I wonder if a #define
> > INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> > nicer than this?
>
> The problem with this is that the caller's log functions won't be
> respected.
Ah yes, I'd forgotten this.
> I think it would be better to make all of the copy
> functions take a ctx.
Right. I was mostly annoyed by libxl_uuid_copy vs. libxl_uuid_copy_ctx.
I think we can probably use LIBXL_API_VERSION to add this parameter,
similar to how we did for libxl_domain_create_restore. That would mean
libxl_uuid_copy would have to accept ctx==NULL, but it seems unlikely to
need to log anyway.
Something like:
#define LIBXL_HAVE_UUID_COPY_CTX_PARAM
void libxl_uuid_copy(libxl_ctx *ctx, libxl_uuid *dst,
const libxl_uuid *src);
#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x040400
#define libxl_uuid_copy(dst, src) libxl_uuid_copy(NULL, dst, src)
#endif
ought to do it I think? Perhaps we should do all of libxl_uuid_* for
consistency?
> Alternatively we should at least document this restriction, but also
> then we shouldn't ever call the copy functions from within libxl which
> (without reading the rest of the series) I think would probably defeat
> the point.
I think they are currently all used from xl, but one of my comments
might change that.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-07 10:57 ` Ian Campbell
@ 2014-05-07 12:55 ` Wei Liu
2014-05-07 13:10 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-07 12:55 UTC (permalink / raw)
To: Ian Campbell; +Cc: Wei Liu, Ian Jackson, xen-devel
On Wed, May 07, 2014 at 11:57:59AM +0100, Ian Campbell wrote:
> On Wed, 2014-05-07 at 11:47 +0100, Ian Jackson wrote:
> > Ian Campbell writes ("Re: [PATCH V4 16/24] libxl: copy function for builtin types"):
> > > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > > +typedef struct libxl__ctx libxl_ctx;
> > > > +
> > > > int libxl_uuid_is_nil(libxl_uuid *uuid);
> > > > void libxl_uuid_generate(libxl_uuid *uuid);
> > > > int libxl_uuid_from_string(libxl_uuid *uuid, const char *in);
> > > > void libxl_uuid_copy(libxl_uuid *dst, const libxl_uuid *src);
> > > > +void libxl_uuid_copy_ctx(libxl_ctx *ctx, libxl_uuid *dst,
> > > > + const libxl_uuid *src);
> > >
> > > Hrm, this is rather unfortunate.
> > >
> > > All of these only take a ctx so they can use NOGC. I wonder if a #define
> > > INIT_NOGC which provides a suitable gc (with maxsize == -1) might be
> > > nicer than this?
> >
> > The problem with this is that the caller's log functions won't be
> > respected.
>
> Ah yes, I'd forgotten this.
>
> > I think it would be better to make all of the copy
> > functions take a ctx.
>
> Right. I was mostly annoyed by libxl_uuid_copy vs. libxl_uuid_copy_ctx.
>
So was I. I felt so guilty when I needed to invent a new variant with
ugly name.
> I think we can probably use LIBXL_API_VERSION to add this parameter,
> similar to how we did for libxl_domain_create_restore. That would mean
> libxl_uuid_copy would have to accept ctx==NULL, but it seems unlikely to
> need to log anyway.
>
> Something like:
>
> #define LIBXL_HAVE_UUID_COPY_CTX_PARAM
>
> void libxl_uuid_copy(libxl_ctx *ctx, libxl_uuid *dst,
> const libxl_uuid *src);
>
> #if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x040400
> #define libxl_uuid_copy(dst, src) libxl_uuid_copy(NULL, dst, src)
> #endif
>
This approach looks reasonable. One question, should I bump
LIBXL_API_VERSION to 0x040500?
> ought to do it I think? Perhaps we should do all of libxl_uuid_* for
> consistency?
>
I don't think so. Looking at the existing API not every function takes a
ctx. Other functions which don't involve memory allocation should be
fine without a ctx.
Wei.
> > Alternatively we should at least document this restriction, but also
> > then we shouldn't ever call the copy functions from within libxl which
> > (without reading the rest of the series) I think would probably defeat
> > the point.
>
> I think they are currently all used from xl, but one of my comments
> might change that.
>
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 16/24] libxl: copy function for builtin types
2014-05-07 12:55 ` Wei Liu
@ 2014-05-07 13:10 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 13:10 UTC (permalink / raw)
To: Wei Liu; +Cc: Ian Jackson, xen-devel
> > I think we can probably use LIBXL_API_VERSION to add this parameter,
> > similar to how we did for libxl_domain_create_restore. That would mean
> > libxl_uuid_copy would have to accept ctx==NULL, but it seems unlikely to
> > need to log anyway.
> >
> > Something like:
> >
> > #define LIBXL_HAVE_UUID_COPY_CTX_PARAM
> >
> > void libxl_uuid_copy(libxl_ctx *ctx, libxl_uuid *dst,
> > const libxl_uuid *src);
> >
> > #if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x040400
> > #define libxl_uuid_copy(dst, src) libxl_uuid_copy(NULL, dst, src)
> > #endif
> >
>
> This approach looks reasonable. One question, should I bump
> LIBXL_API_VERSION to 0x040500?
No, this is something which the calling application is required to
define if it wants a particular version of the interface.
There is a check in libxl.h somewhere which defines the valid versions,
you should update that though.
>
> > ought to do it I think? Perhaps we should do all of libxl_uuid_* for
> > consistency?
> >
>
> I don't think so. Looking at the existing API not every function takes a
> ctx.
I think most of them do (for consistency).
> Other functions which don't involve memory allocation should be
> fine without a ctx.
The problem comes in the future when we discover a need to log or
allocate from this function (as you've just discovered).
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 17/24] libxl IDL: generate deep copy functions
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (15 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 16/24] libxl: copy function for builtin types Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 14:06 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 18/24] libxl/gentest.py: test " Wei Liu
` (7 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Introduces copy_fn for a type.
For most builtin types we can use direct assignment. For those builtin
types which cannot use direct assignment we use the copy functions in
previous patch.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentypes.py | 67 ++++++++++++++++++++++++++++++++++
tools/libxl/idl.py | 13 ++++++-
tools/libxl/idl.txt | 12 ++++++
tools/libxl/libxl_types.idl | 26 ++++++++-----
tools/libxl/libxl_types_internal.idl | 2 +-
5 files changed, 108 insertions(+), 12 deletions(-)
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 3e1140d..919d2d3 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -100,6 +100,61 @@ def libxl_C_type_dispose(ty, v, indent = " ", parent = None):
s = indent + s
return s.replace("\n", "\n%s" % indent).rstrip(indent)
+def libxl_C_type_copy(ty, v, w, indent = " ", vparent = None, wparent = None):
+ s = ""
+
+ if vparent is None:
+ s += "GC_INIT(ctx);\n";
+
+ if isinstance(ty, idl.KeyedUnion):
+ if vparent is None or wparent is None:
+ raise Exception("KeyedUnion type must have a parent")
+ s += "%s = %s;\n" % ((vparent + ty.keyvar.name), (wparent + ty.keyvar.name))
+ s += "switch (%s) {\n" % (wparent + ty.keyvar.name)
+ for f in ty.fields:
+ (vnparent,vfexpr) = ty.member(v, f, vparent is None)
+ (wnparent,wfexpr) = ty.member(w, f, wparent is None)
+ s += "case %s:\n" % f.enumname
+ if f.type is not None:
+ s += libxl_C_type_copy(f.type, vfexpr, wfexpr, indent + " ",
+ vnparent, wnparent)
+ s += " break;\n"
+ s += "}\n"
+ elif isinstance(ty, idl.Array):
+ if vparent is None or wparent is None:
+ raise Exception("Array type must have a parent")
+ s += "%s = libxl__calloc(NOGC, %s, sizeof(*%s));\n" % (ty.pass_arg(v, vparent is None),
+ (wparent + ty.lenvar.name),
+ ty.pass_arg(w, wparent is None))
+ s += "%s = %s;\n" % ((vparent + ty.lenvar.name), (wparent + ty.lenvar.name))
+ s += "{\n"
+ s += " int i;\n"
+ s += " for (i=0; i<%s; i++)\n" % (wparent + ty.lenvar.name)
+ s += libxl_C_type_copy(ty.elem_type, v+"[i]", w+"[i]",
+ indent + " ", vparent, wparent)
+ s += "}\n"
+ elif isinstance(ty, idl.Struct) and ((vparent is None and wparent is None) or ty.copy_fn is None):
+ for f in [f for f in ty.fields if not f.const and not f.type.private]:
+ (vnparent,vfexpr) = ty.member(v, f, vparent is None)
+ (wnparent,wfexpr) = ty.member(w, f, wparent is None)
+ s += libxl_C_type_copy(f.type, vfexpr, wfexpr, "", vnparent, wnparent)
+ else:
+ if ty.copy_fn is not None:
+ s += "%s(ctx, %s, %s);\n" % (ty.copy_fn,
+ ty.pass_arg(v, vparent is None, passby=idl.PASS_BY_REFERENCE),
+ ty.pass_arg(w, wparent is None, passby=idl.PASS_BY_REFERENCE))
+
+ else:
+ s += "%s = %s;\n" % (ty.pass_arg(v, vparent is None, passby=idl.PASS_BY_VALUE),
+ ty.pass_arg(w, wparent is None, passby=idl.PASS_BY_VALUE))
+
+ if vparent is None:
+ s += "GC_FREE;\n"
+
+ if s != "":
+ s = indent + s
+ return s.replace("\n", "\n%s" % indent).rstrip(indent)
+
def libxl_init_members(ty, nesting = 0):
"""Returns a list of members of ty which require a separate init"""
@@ -430,6 +485,9 @@ if __name__ == '__main__':
f.write(libxl_C_type_define(ty) + ";\n")
if ty.dispose_fn is not None:
f.write("%svoid %s(%s);\n" % (ty.hidden(), ty.dispose_fn, ty.make_arg("p")))
+ if ty.copy_fn is not None:
+ f.write("%svoid %s(libxl_ctx *ctx, %s, %s);\n" % (ty.hidden(), ty.copy_fn,
+ ty.make_arg("dst"), ty.make_arg("src")))
if ty.init_fn is not None:
f.write("%svoid %s(%s);\n" % (ty.hidden(), ty.init_fn, ty.make_arg("p")))
for field in libxl_init_members(ty):
@@ -509,6 +567,15 @@ if __name__ == '__main__':
f.write(" memset(p, LIBXL_DTOR_POISON, sizeof(*p));\n")
f.write("}\n")
f.write("\n")
+
+ for ty in [t for t in types if t.copy_fn and t.autogenerate_copy_fn]:
+ f.write("void %s(libxl_ctx *ctx, %s, %s)\n" % (ty.copy_fn,
+ ty.make_arg("dst", passby=idl.PASS_BY_REFERENCE),
+ ty.make_arg("src", passby=idl.PASS_BY_REFERENCE)))
+ f.write("{\n")
+ f.write(libxl_C_type_copy(ty, "dst", "src"))
+ f.write("}\n")
+ f.write("\n")
for ty in [t for t in types if t.init_fn is not None and t.autogenerate_init_fn]:
f.write(libxl_C_type_init(ty))
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index 9ce1e33..3498af1 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -60,6 +60,13 @@ class Type(object):
self.autogenerate_dispose_fn = kwargs.setdefault('autogenerate_dispose_fn', True)
+ if self.typename is not None:
+ self.copy_fn = kwargs.setdefault('copy_fn', self.typename + "_copy")
+ else:
+ self.copy_fn = kwargs.setdefault('copy_fn', None)
+
+ self.autogenerate_copy_fn = kwargs.setdefault('autogenerate_copy_fn', True)
+
self.init_fn = kwargs.setdefault('init_fn', None)
self.init_val = kwargs.setdefault('init_val', None)
self.autogenerate_init_fn = kwargs.setdefault('autogenerate_init_fn', False)
@@ -121,6 +128,7 @@ class Number(Builtin):
def __init__(self, ctype, **kwargs):
kwargs.setdefault('namespace', None)
kwargs.setdefault('dispose_fn', None)
+ kwargs.setdefault('copy_fn', None)
kwargs.setdefault('signed', False)
kwargs.setdefault('json_gen_fn', "yajl_gen_integer")
kwargs.setdefault('json_parse_type', "JSON_INTEGER")
@@ -134,6 +142,7 @@ class UInt(Number):
kwargs.setdefault('namespace', None)
kwargs.setdefault('dispose_fn', None)
kwargs.setdefault('json_parse_fn', "libxl__uint%d_parse_json" % w)
+ kwargs.setdefault('copy_fn', None)
Number.__init__(self, "uint%d_t" % w, **kwargs)
self.width = w
@@ -150,6 +159,7 @@ class EnumerationValue(object):
class Enumeration(Type):
def __init__(self, typename, values, **kwargs):
kwargs.setdefault('dispose_fn', None)
+ kwargs.setdefault('copy_fn', None)
kwargs.setdefault('json_parse_type', "JSON_STRING")
Type.__init__(self, typename, **kwargs)
@@ -266,6 +276,7 @@ class KeyedUnion(Aggregate):
void = Builtin("void *", namespace = None)
bool = Builtin("bool", namespace = None,
+ copy_fn=None,
json_gen_fn = "yajl_gen_bool",
json_parse_type = "JSON_BOOL",
json_parse_fn = "libxl__bool_parse_json",
@@ -280,7 +291,7 @@ uint16 = UInt(16)
uint32 = UInt(32)
uint64 = UInt(64)
-string = Builtin("char *", namespace = None, dispose_fn = "free",
+string = Builtin("char *", namespace = None, copy_fn = "libxl_string_copy", dispose_fn = "free",
json_gen_fn = "libxl__string_gen_json",
json_parse_type = "JSON_STRING | JSON_NULL",
json_parse_fn = "libxl__string_parse_json",
diff --git a/tools/libxl/idl.txt b/tools/libxl/idl.txt
index 484d5d7..8b54aeb 100644
--- a/tools/libxl/idl.txt
+++ b/tools/libxl/idl.txt
@@ -44,6 +44,18 @@ Type.autogenerate_dispose_fn: (default: True)
Indicates if the above named Type.dispose_fn should be
autogenerated.
+Type.copy_fn: (default: typename + "_copy" or None if type == None)
+
+ The name of the C function which will deep copy all fields within
+ this type.
+
+Type.autogenerate_copy_fn: (default: True)
+
+ Indicates if the above named Type.copy_fn should be
+ autogenerated.
+
+Type.autogenerate_copy_fn
+
Type.init_val: (default: None)
C expression for the value to initialise instances of this type to.
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 6cf99fd..1bd4185 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -5,23 +5,29 @@
namespace("libxl_")
-libxl_defbool = Builtin("defbool", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+libxl_defbool = Builtin("defbool", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE, copy_fn=None)
libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", json_parse_fn = "libxl__uint32_parse_json",
- json_parse_type = "JSON_INTEGER", autogenerate_json = False)
+ json_parse_type = "JSON_INTEGER", autogenerate_json = False, copy_fn=None)
libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", json_parse_fn = "libxl__int_parse_json",
- json_parse_type = "JSON_INTEGER", autogenerate_json = False, signed = True, init_val="-1")
-libxl_uuid = Builtin("uuid", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
-libxl_mac = Builtin("mac", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
-libxl_bitmap = Builtin("bitmap", json_parse_type="JSON_ARRAY", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
+ json_parse_type = "JSON_INTEGER", autogenerate_json = False, signed = True, init_val="-1",
+ copy_fn=None)
+libxl_uuid = Builtin("uuid", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE, copy_fn="libxl_uuid_copy_ctx")
+libxl_mac = Builtin("mac", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE, copy_fn="libxl_mac_copy")
+libxl_bitmap = Builtin("bitmap", json_parse_type="JSON_ARRAY", dispose_fn="libxl_bitmap_dispose",
+ copy_fn='libxl_bitmap_copy_alloc', passby=PASS_BY_REFERENCE)
libxl_cpuid_policy_list = Builtin("cpuid_policy_list", json_parse_type="JSON_ARRAY",
- dispose_fn="libxl_cpuid_dispose", passby=PASS_BY_REFERENCE)
+ dispose_fn="libxl_cpuid_dispose", passby=PASS_BY_REFERENCE,
+ copy_fn="libxl_cpuid_policy_list_copy")
libxl_string_list = Builtin("string_list", json_parse_type="JSON_ARRAY",
- dispose_fn="libxl_string_list_dispose", passby=PASS_BY_REFERENCE)
+ dispose_fn="libxl_string_list_dispose", passby=PASS_BY_REFERENCE,
+ copy_fn="libxl_string_list_copy")
libxl_key_value_list = Builtin("key_value_list", json_parse_type="JSON_MAP",
- dispose_fn="libxl_key_value_list_dispose", passby=PASS_BY_REFERENCE)
-libxl_hwcap = Builtin("hwcap", json_parse_type="JSON_ARRAY", passby=PASS_BY_REFERENCE)
+ dispose_fn="libxl_key_value_list_dispose", passby=PASS_BY_REFERENCE,
+ copy_fn="libxl_key_value_list_copy")
+libxl_hwcap = Builtin("hwcap", json_parse_type="JSON_ARRAY", passby=PASS_BY_REFERENCE,
+ copy_fn="libxl_hwcap_copy")
#
# Specific integer types
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index 17533f1..800361b 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -3,7 +3,7 @@ hidden(True)
libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer",
json_parse_fn = "libxl__uint32_parse_json", json_parse_type = "JSON_INTEGER",
- autogenerate_json = False)
+ autogenerate_json = False, copy_fn = None)
libxl__qmp_message_type = Enumeration("qmp_message_type", [
(1, "QMP"),
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 18/24] libxl/gentest.py: test deep copy functions
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (16 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 17/24] libxl IDL: generate deep copy functions Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 14:06 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 19/24] xl: introduce and use "xl-json" format Wei Liu
` (6 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
The test is done as followed:
1. initialise libxl_FOO struct A
2. deep-copy A to B
3. generate JSON string for A and B
4. compare two JSON strings
If two strings are identical then we're good.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentest.py | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index b92d092..07ffd31 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -273,6 +273,43 @@ int main(int argc, char **argv)
f.write(" %s(&%s_val_new);\n" % (ty.dispose_fn, ty.typename))
f.write("\n")
+ f.write(" printf(\"Testing TYPE_copy()\\n\");\n")
+ f.write(" printf(\"----------------------\\n\");\n")
+ f.write(" printf(\"\\n\");\n")
+ for ty in [t for t in types if t.copy_fn is not None]:
+ f.write(" printf(\"Testing %s_copy, \");\n" % ty.typename)
+ arg = ty.typename + "_val"
+ f.write(" %s_init(%s);\n" % (ty.typename, \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
+ f.write(" %s_rand_init(%s);\n" % (ty.typename, \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
+ f.write(" %s_init(%s_new);\n" % (ty.typename, \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
+ f.write(" %s_copy(ctx, %s_new, %s);\n" % (ty.typename, \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE), \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
+ f.write(" s = %s_to_json(ctx, %s);\n" % \
+ (ty.typename, ty.pass_arg(arg, isref=False)))
+ f.write(" if (s == NULL) abort();\n")
+ f.write(" new_s = %s_to_json(ctx, %s_new);\n" % \
+ (ty.typename, ty.pass_arg(arg, isref=False)))
+ f.write(" if (new_s == NULL) abort();\n")
+ f.write(" if (strcmp(s, new_s)) {\n")
+ f.write(" printf(\"Huh? Deep copy for %s failed. Regenerated string different from original string.\\n\");\n" \
+ % ty.typename)
+ f.write(" printf(\"Original string: %s\\n\", s);\n")
+ f.write(" printf(\"Regenerated string: %s\\n\", new_s);\n")
+ f.write(" abort();\n")
+ f.write(" }\n")
+ f.write(" free(s);\n")
+ f.write(" free(new_s);\n")
+ if ty.dispose_fn is not None:
+ f.write(" %s(&%s_val);\n" % (ty.dispose_fn, ty.typename))
+ f.write(" %s(&%s_val_new);\n" % (ty.dispose_fn, ty.typename))
+ f.write(" printf(\"done\\n\");\n")
+ f.write("\n")
+
+ f.write(" printf(\"\\n\");\n")
f.write(" printf(\"Testing Enumerations\\n\");\n")
f.write(" printf(\"--------------------\\n\");\n")
f.write(" printf(\"\\n\");\n")
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (17 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 18/24] libxl/gentest.py: test " Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 14:26 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 20/24] xl: introduce load/save_domain_config Wei Liu
` (5 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Originally xl verbatimly copies the domain config file in its user data
store. This patch adds the functionality to transform text domain config
file to JSON object and save that object in user data store.
Later patches will modify device hotplug commands and memory hotplug
commands to update the saved "xl-json" file as needed. In the end we
always have the up-to-date configurations of a domain saved in user data
store.
What this patch does:
1. add a mandatory flag to save protocol to indicate whether the saved
config is a JSON object
2. register a new private data type "xl-json" in libxl.h
3. modify xl to save / load "xl-json" file where necessary
After this change xl supports both "xl" format and "xl-json" format.
A new flag "-j" is added to various commands, so that users can supply a
config file in JSON format.
Tests done so far (xl.{new,old} denotes xl with{,out} "xl_json"
support):
1. xl.new create then xl.new save, hexdump saved file: domain config
saved in JSON format
2. xl.new create, xl.new save then xl.old restore: failed on mandatory
flag check
3. xl.new create, xl.new save then xl.new restore: succeeded
4. xl.old create, xl.old save then xl.new restore: succeeded
5. xl.new create then local migrate, receiving end xl.new: succeeded
6. xl.old create then local migrate, receiving end xl.new: succeeded
The only drawback is that when restoring a domain, xl cannot
automatically spawn a vncviewer anymore. That's because that option is
part of domain_create info not domain configuration, thus it's not
saved in the JSON config. A warning is printed out and documentation
is updated.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
docs/man/xl.cfg.pod.5 | 3 +-
docs/man/xl.pod.1 | 21 +++
tools/libxl/libxl.h | 1 +
tools/libxl/xl_cmdimpl.c | 313 ++++++++++++++++++++++++++++++++++++---------
tools/libxl/xl_cmdtable.c | 11 +-
5 files changed, 285 insertions(+), 64 deletions(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index c8ce6c1..e6c86b0 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1114,7 +1114,8 @@ other VNC-related settings. The default is to enable this.
=item B<vncviewer=BOOLEAN>
-Automatically spawn a vncviewer when creating/restoring a guest.
+Automatically spawn a vncviewer when creating a guest. This option has
+no effect when restoring a guest and it might be removed in the future.
=item B<vnclisten="ADDRESS[:DISPLAYNUM]">
diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index 30bd4bf..f1a1b9b 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -146,6 +146,11 @@ useful for determining issues with crashing domains and just as a
general convenience since you often want to watch the
domain boot.
+=item B<-j>
+
+The provided I<configfile> is in JSON format. Cannot be used with "key=value"
+at the same time.
+
=item B<key=value>
It is possible to pass I<key=value> pairs on the command line to provide
@@ -197,6 +202,11 @@ B<OPTIONS>
Use the given configuration file.
+=item B<-j>
+
+The provided I<configfile> is in JSON format. Cannot be used with "key=value"
+at the same time.
+
=item B<key=value>
It is possible to pass I<key=value> pairs on the command line to
@@ -423,6 +433,10 @@ Send <config> instead of config file from creation.
Print huge (!) amount of debug during the migration process.
+=item B<-j>
+
+The provided I<config> is in JSON format.
+
=back
=item B<remus> [I<OPTIONS>] I<domain-id> I<host>
@@ -528,6 +542,9 @@ Attach to domain's VNC server, forking a vncviewer process.
Pass VNC password to vncviewer via stdin.
+=item B<-j>
+
+The provided I<ConfigFile> is in JSON format.
=back
@@ -551,6 +568,10 @@ Leave domain running after creating the snapshot.
Leave domain paused after creating the snapshot.
+=item B<-j>
+
+The provided I<ConfigFile> is in JSON format.
+
=back
=item B<sharing> [I<domain-id>]
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index c6a9a0d..970eba2 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1081,6 +1081,7 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
*
* userid Data contents
* "xl" domain config file in xl format, Unix line endings
+ * "xl-json" domain config in JSON format generated by xl
* "libvirt-xml" domain config file in libvirt XML format. See
* http://libvirt.org/formatdomain.html
*
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 289ea9a..7443d86 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -110,6 +110,8 @@ static const char migrate_report[]=
* from target to source
*/
+#define XL_MANDATORY_FLAG_JSON (1U << 0) /* config data is in JSON format */
+#define XL_MANDATORY_FLAG_ALL (XL_MANDATORY_FLAG_JSON)
struct save_file_header {
char magic[32]; /* savefileheader_magic */
/* All uint32_ts are in domain's byte order. */
@@ -151,6 +153,7 @@ struct domain_create {
int console_autoconnect;
int checkpointed_stream;
const char *config_file;
+ int config_in_json;
const char *extra_config; /* extra config string */
const char *restore_file;
int migrate_fd; /* -1 means none */
@@ -693,6 +696,73 @@ static void parse_top_level_sdl_options(XLU_Config *config,
xlu_cfg_replace_string (config, "xauthority", &sdl->xauthority, 0);
}
+static void nic_update_default(libxl_domain_config *d_config)
+{
+ int i;
+ libxl_device_nic *nic;
+
+ for (i = 0; i < d_config->num_nics; i++) {
+ nic = &d_config->nics[i];
+
+ if (default_vifscript) {
+ free(nic->script);
+ nic->script = strdup(default_vifscript);
+ }
+
+ if (default_bridge) {
+ free(nic->bridge);
+ nic->bridge = strdup(default_bridge);
+ }
+
+ if (default_gatewaydev) {
+ free(nic->gatewaydev);
+ nic->gatewaydev = strdup(default_gatewaydev);
+ }
+
+ if (default_vifbackend) {
+ free(nic->backend_domname);
+ nic->backend_domname = strdup(default_vifbackend);
+ }
+ }
+}
+
+static void parse_config_data_json(char *config_data,
+ int config_len,
+ libxl_domain_config *d_config)
+{
+ int ret;
+
+ if (!config_len) {
+ fprintf(stderr, "Config data stream empty\n");
+ exit(1);
+ }
+
+ /* Make sure this string ends with \0 -- the parser expects a NULL
+ * terminated string.
+ */
+ if (config_data[config_len-1] != '\0') {
+ config_data = realloc(config_data, config_len + 1);
+ if (!config_data) {
+ fprintf(stderr, "Failed to realloc config_data\n");
+ exit(1);
+ }
+ config_data[config_len] = '\0';
+ }
+
+ ret = libxl_domain_config_from_json(ctx, d_config, config_data);
+ if (ret) {
+ fprintf(stderr, "Failed to parse config\n");
+ exit(1);
+ }
+
+ if (blkdev_start) {
+ free(d_config->b_info.blkdev_start);
+ d_config->b_info.blkdev_start = strdup(blkdev_start);
+ }
+
+ nic_update_default(d_config);
+}
+
static void parse_config_data(const char *config_source,
const char *config_data,
int config_len,
@@ -930,6 +1000,8 @@ static void parse_config_data(const char *config_source,
b_info->rtc_timeoffset = l;
if (dom_info && !xlu_cfg_get_long(config, "vncviewer", &l, 0)) {
+ fprintf(stderr, "WARNING: \"vncviewer\" option found. It might be removed in future release. "
+ "Use \"-V\" option of \"xl create\" to automatically spawn vncviewer.\n");
/* Command line arguments must take precedence over what's
* specified in the configuration file. */
if (!dom_info->vnc)
@@ -1243,26 +1315,6 @@ static void parse_config_data(const char *config_source,
libxl_device_nic_init(nic);
nic->devid = d_config->num_nics;
- if (default_vifscript) {
- free(nic->script);
- nic->script = strdup(default_vifscript);
- }
-
- if (default_bridge) {
- free(nic->bridge);
- nic->bridge = strdup(default_bridge);
- }
-
- if (default_gatewaydev) {
- free(nic->gatewaydev);
- nic->gatewaydev = strdup(default_gatewaydev);
- }
-
- if (default_vifbackend) {
- free(nic->backend_domname);
- nic->backend_domname = strdup(default_vifbackend);
- }
-
p = strtok(buf2, ",");
if (!p)
goto skip_nic;
@@ -1334,6 +1386,8 @@ skip_nic:
}
}
+ nic_update_default(d_config);
+
if (!xlu_cfg_get_list(config, "vif2", NULL, 0, 0)) {
fprintf(stderr, "WARNING: vif2: netchannel2 is deprecated and not supported by xl\n");
}
@@ -1699,12 +1753,13 @@ skip_vfb:
}
static void reload_domain_config(uint32_t domid,
- uint8_t **config_data, int *config_len)
+ uint8_t **config_data, int *config_len,
+ bool *config_in_json)
{
uint8_t *t_data;
int ret, t_len;
- ret = libxl_userdata_retrieve(ctx, domid, "xl", &t_data, &t_len);
+ ret = libxl_userdata_retrieve(ctx, domid, "xl-json", &t_data, &t_len);
if (ret) {
LOG("failed to retrieve guest configuration (rc=%d). "
"reusing old configuration", ret);
@@ -1714,6 +1769,7 @@ static void reload_domain_config(uint32_t domid,
free(*config_data);
*config_data = t_data;
*config_len = t_len;
+ *config_in_json = true;
}
/* Returns 1 if domain should be restarted,
@@ -1722,6 +1778,7 @@ static void reload_domain_config(uint32_t domid,
static int handle_domain_death(uint32_t *r_domid,
libxl_event *event,
uint8_t **config_data, int *config_len,
+ bool *config_in_json,
libxl_domain_config *d_config)
{
@@ -1779,12 +1836,14 @@ static int handle_domain_death(uint32_t *r_domid,
break;
case LIBXL_ACTION_ON_SHUTDOWN_RESTART_RENAME:
- reload_domain_config(*r_domid, config_data, config_len);
+ reload_domain_config(*r_domid, config_data, config_len,
+ config_in_json);
restart = 2;
break;
case LIBXL_ACTION_ON_SHUTDOWN_RESTART:
- reload_domain_config(*r_domid, config_data, config_len);
+ reload_domain_config(*r_domid, config_data, config_len,
+ config_in_json);
restart = 1;
/* fall-through */
@@ -1965,11 +2024,43 @@ static void evdisable_disk_ejects(libxl_evgen_disk_eject **diskws,
}
}
+/* Selectively update domain configuration struct. This function is
+ * only used when creating domain.
+ *
+ * src contains a libxl_domain_config that is used by libxl to create
+ * a domain. Presumably libxl fills in relevant information when
+ * creating a domain.
+ *
+ * dst contains a vanilla copy of domain configuration from user
+ * supplied config file. It serves as a template.
+ *
+ * The end result is that dst now contains all relevant information to
+ * reconstruct a domain based on user's configurations and libxl's
+ * decision.
+ */
+static void update_domain_config(libxl_domain_config *dst,
+ libxl_domain_config *src)
+{
+ int i;
+
+ /* update network interface information */
+ for (i = 0; i < src->num_nics; i++)
+ libxl_mac_copy(ctx, &dst->nics[i].mac, &src->nics[i].mac);
+
+ /* update vTPM information */
+ for (i = 0; i < src->num_vtpms; i++)
+ libxl_uuid_copy(&dst->vtpms[i].uuid, &src->vtpms[i].uuid);
+
+ /* update guest UUID */
+ libxl_uuid_copy(&dst->c_info.uuid, &src->c_info.uuid);
+}
+
static uint32_t create_domain(struct domain_create *dom_info)
{
uint32_t domid = INVALID_DOMID;
- libxl_domain_config d_config;
+ libxl_domain_config d_config, d_config_saved;
+ char *d_config_json = NULL;
int debug = dom_info->debug;
int daemonize = dom_info->daemonize;
@@ -1990,6 +2081,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
libxl_evgen_disk_eject **diskws = NULL; /* one per disk */
void *config_data = 0;
int config_len = 0;
+ bool config_in_json;
int restore_fd = -1;
const libxl_asyncprogress_how *autoconnect_console_how;
struct save_file_header hdr;
@@ -1997,6 +2089,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
int restoring = (restore_file || (migrate_fd >= 0));
libxl_domain_config_init(&d_config);
+ libxl_domain_config_init(&d_config_saved);
if (restoring) {
uint8_t *optdata_begin = 0;
@@ -2036,7 +2129,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
restore_source, hdr.mandatory_flags, hdr.optional_flags,
hdr.optional_data_len);
- badflags = hdr.mandatory_flags & ~( 0 /* none understood yet */ );
+ badflags = hdr.mandatory_flags & ~XL_MANDATORY_FLAG_ALL;
if (badflags) {
fprintf(stderr, "Savefile has mandatory flag(s) 0x%"PRIx32" "
"which are not supported; need newer xl\n",
@@ -2064,7 +2157,9 @@ static uint32_t create_domain(struct domain_create *dom_info)
optdata_here = optdata_begin;
if (OPTDATA_LEFT) {
- fprintf(stderr, " Savefile contains xl domain config\n");
+ fprintf(stderr, " Savefile contains xl domain config%s\n",
+ !!(hdr.mandatory_flags & XL_MANDATORY_FLAG_JSON)
+ ? " in JSON format" : "");
WITH_OPTDATA(4, {
memcpy(u32buf.b, optdata_here, 4);
config_len = u32buf.u32;
@@ -2104,6 +2199,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
extra_config);
}
config_source=config_file;
+ config_in_json = !!dom_info->config_in_json;
} else {
if (!config_data) {
fprintf(stderr, "Config file not specified and"
@@ -2111,12 +2207,22 @@ static uint32_t create_domain(struct domain_create *dom_info)
return ERROR_INVAL;
}
config_source = "<saved>";
+ config_in_json = !!(hdr.mandatory_flags & XL_MANDATORY_FLAG_JSON);
}
if (!dom_info->quiet)
printf("Parsing config from %s\n", config_source);
- parse_config_data(config_source, config_data, config_len, &d_config, dom_info);
+ if (config_in_json)
+ parse_config_data_json(config_data, config_len, &d_config);
+ else
+ parse_config_data(config_source, config_data, config_len,
+ &d_config, dom_info);
+
+ /* Save a copy of vanilla domain configuration, as libxl routines
+ * will fill in more stuffs.
+ */
+ libxl_domain_config_copy(ctx, &d_config_saved, &d_config);
if (migrate_fd >= 0) {
if (d_config.c_info.name) {
@@ -2211,8 +2317,16 @@ start:
free(vcpu_to_pcpu); vcpu_to_pcpu = NULL;
}
- ret = libxl_userdata_store(ctx, domid, "xl",
- config_data, config_len);
+ update_domain_config(&d_config_saved, &d_config);
+ d_config_json = libxl_domain_config_to_json(ctx, &d_config_saved);
+ if (d_config_json == NULL) {
+ perror("cannot generate JSON object of domain configuration");
+ ret = ERROR_FAIL;
+ goto error_out;
+ }
+ ret = libxl_userdata_store(ctx, domid, "xl-json",
+ (const uint8_t *)d_config_json,
+ strlen(d_config_json));
if (ret) {
perror("cannot save config file");
ret = ERROR_FAIL;
@@ -2277,7 +2391,7 @@ start:
event->u.domain_shutdown.shutdown_reason);
switch (handle_domain_death(&domid, event,
(uint8_t **)&config_data, &config_len,
- &d_config)) {
+ &config_in_json, &d_config)) {
case 2:
if (!preserve_domain(&domid, event, &d_config)) {
/* If we fail then exit leaving the old domain in place. */
@@ -2317,8 +2431,11 @@ start:
/* Reparse the configuration in case it has changed */
libxl_domain_config_dispose(&d_config);
libxl_domain_config_init(&d_config);
- parse_config_data(config_source, config_data, config_len,
- &d_config, dom_info);
+ if (config_in_json)
+ parse_config_data_json(config_data, config_len, &d_config);
+ else
+ parse_config_data(config_source, config_data, config_len,
+ &d_config, dom_info);
/*
* XXX FIXME: If this sleep is not there then domain
@@ -2371,9 +2488,12 @@ out:
close(logfile);
libxl_domain_config_dispose(&d_config);
+ libxl_domain_config_dispose(&d_config_saved);
free(config_data);
+ free(d_config_json);
+
console_child_report(child_console);
if (deathw)
@@ -3124,12 +3244,13 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain)
/* no detailed info available on dom0 */
if (info[i].domid == 0)
continue;
- rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
+ rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl-json",
+ &data, &len);
if (rc)
continue;
CHK_SYSCALL(asprintf(&config_source, "<domid %d data>", info[i].domid));
libxl_domain_config_init(&d_config);
- parse_config_data(config_source, (char *)data, len, &d_config, NULL);
+ parse_config_data_json((char *)data, len, &d_config);
if (default_output_format == OUTPUT_FORMAT_JSON)
s = printf_info_one_json(hand, info[i].domid, &d_config);
else
@@ -3318,7 +3439,8 @@ static void core_dump_domain(uint32_t domid, const char *filename)
static void save_domain_core_begin(uint32_t domid,
const char *override_config_file,
uint8_t **config_data_r,
- int *config_len_r)
+ int *config_len_r,
+ bool *config_in_json)
{
int rc;
@@ -3329,9 +3451,12 @@ static void save_domain_core_begin(uint32_t domid,
rc = libxl_read_file_contents(ctx, override_config_file,
&config_v, config_len_r);
*config_data_r = config_v;
+ /* leave config_in_json untouched */
} else {
- rc = libxl_userdata_retrieve(ctx, domid, "xl",
+ rc = libxl_userdata_retrieve(ctx, domid, "xl-json",
config_data_r, config_len_r);
+ if (!rc)
+ *config_in_json = true;
}
if (rc) {
fputs("Unable to get config file\n",stderr);
@@ -3340,7 +3465,8 @@ static void save_domain_core_begin(uint32_t domid,
}
static void save_domain_core_writeconfig(int fd, const char *source,
- const uint8_t *config_data, int config_len)
+ const uint8_t *config_data, int config_len,
+ bool config_in_json)
{
struct save_file_header hdr;
uint8_t *optdata_begin;
@@ -3364,6 +3490,8 @@ static void save_domain_core_writeconfig(int fd, const char *source,
u32buf.u32 = config_len;
ADD_OPTDATA(u32buf.b, 4);
ADD_OPTDATA(config_data, config_len);
+ if (config_in_json)
+ hdr.mandatory_flags |= XL_MANDATORY_FLAG_JSON;
/* that's the optional data */
@@ -3382,17 +3510,19 @@ static void save_domain_core_writeconfig(int fd, const char *source,
}
static int save_domain(uint32_t domid, const char *filename, int checkpoint,
- int leavepaused, const char *override_config_file)
+ int leavepaused, const char *override_config_file,
+ bool config_in_json)
{
int fd;
uint8_t *config_data;
int config_len;
save_domain_core_begin(domid, override_config_file,
- &config_data, &config_len);
+ &config_data, &config_len, &config_in_json);
if (!config_len) {
fputs(" Savefile will not contain xl domain config\n", stderr);
+ config_in_json = false;
}
fd = open(filename, O_WRONLY|O_CREAT|O_TRUNC, 0644);
@@ -3401,7 +3531,8 @@ static int save_domain(uint32_t domid, const char *filename, int checkpoint,
exit(2);
}
- save_domain_core_writeconfig(fd, filename, config_data, config_len);
+ save_domain_core_writeconfig(fd, filename, config_data, config_len,
+ config_in_json);
int rc = libxl_domain_suspend(ctx, domid, fd, 0, NULL);
close(fd);
@@ -3542,7 +3673,7 @@ static void migration_child_report(int recv_fd) {
static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child,
uint8_t *config_data, int config_len,
- const char *rune)
+ bool config_in_json, const char *rune)
{
int rc = 0;
@@ -3561,12 +3692,13 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child,
}
save_domain_core_writeconfig(send_fd, "migration stream",
- config_data, config_len);
+ config_data, config_len, config_in_json);
}
static void migrate_domain(uint32_t domid, const char *rune, int debug,
- const char *override_config_file)
+ const char *override_config_file,
+ bool config_in_json)
{
pid_t child = -1;
int rc;
@@ -3577,7 +3709,7 @@ static void migrate_domain(uint32_t domid, const char *rune, int debug,
int config_len, flags = LIBXL_SUSPEND_LIVE;
save_domain_core_begin(domid, override_config_file,
- &config_data, &config_len);
+ &config_data, &config_len, &config_in_json);
if (!config_len) {
fprintf(stderr, "No config file stored for running domain and "
@@ -3588,7 +3720,7 @@ static void migrate_domain(uint32_t domid, const char *rune, int debug,
child = create_migration_child(rune, &send_fd, &recv_fd);
migrate_do_preamble(send_fd, recv_fd, child, config_data, config_len,
- rune);
+ config_in_json, rune);
xtl_stdiostream_adjust_flags(logger, XTL_STDIOSTREAM_HIDE_PROGRESS, 0);
@@ -3848,7 +3980,8 @@ int main_restore(int argc, char **argv)
const char *config_file = NULL;
struct domain_create dom_info;
int paused = 0, debug = 0, daemonize = 1, monitor = 1,
- console_autoconnect = 0, vnc = 0, vncautopass = 0;
+ console_autoconnect = 0, vnc = 0, vncautopass = 0,
+ config_in_json = 0;
int opt, rc;
static struct option opts[] = {
{"vncviewer", 0, 0, 'V'},
@@ -3857,7 +3990,7 @@ int main_restore(int argc, char **argv)
{0, 0, 0, 0}
};
- SWITCH_FOREACH_OPT(opt, "FhcpdeVA", opts, "restore", 1) {
+ SWITCH_FOREACH_OPT(opt, "FhcpdeVAj", opts, "restore", 1) {
case 'c':
console_autoconnect = 1;
break;
@@ -3880,6 +4013,9 @@ int main_restore(int argc, char **argv)
case 'A':
vnc = vncautopass = 1;
break;
+ case 'j':
+ config_in_json = 1;
+ break;
}
if (argc-optind == 1) {
@@ -3892,12 +4028,19 @@ int main_restore(int argc, char **argv)
return 2;
}
+ if (!config_file && config_in_json) {
+ fprintf(stderr, "\"-j\" specified but no config file provided\n");
+ help("restore");
+ return 2;
+ }
+
memset(&dom_info, 0, sizeof(dom_info));
dom_info.debug = debug;
dom_info.daemonize = daemonize;
dom_info.monitor = monitor;
dom_info.paused = paused;
dom_info.config_file = config_file;
+ dom_info.config_in_json = config_in_json;
dom_info.restore_file = checkpoint_file;
dom_info.migrate_fd = -1;
dom_info.vnc = vnc;
@@ -3948,17 +4091,21 @@ int main_save(int argc, char **argv)
uint32_t domid;
const char *filename;
const char *config_filename = NULL;
+ bool config_in_json = false;
int checkpoint = 0;
int leavepaused = 0;
int opt;
- SWITCH_FOREACH_OPT(opt, "cp", NULL, "save", 2) {
+ SWITCH_FOREACH_OPT(opt, "cpj", NULL, "save", 2) {
case 'c':
checkpoint = 1;
break;
case 'p':
leavepaused = 1;
break;
+ case 'j':
+ config_in_json = true;
+ break;
}
if (argc-optind > 3) {
@@ -3966,12 +4113,19 @@ int main_save(int argc, char **argv)
return 2;
}
+ if (!config_filename && config_in_json) {
+ fprintf(stderr, "\"-j\" specified but no config file provided\n");
+ help("save");
+ return 2;
+ }
+
domid = find_domain(argv[optind]);
filename = argv[optind + 1];
if ( argc - optind >= 3 )
config_filename = argv[optind + 2];
- save_domain(domid, filename, checkpoint, leavepaused, config_filename);
+ save_domain(domid, filename, checkpoint, leavepaused, config_filename,
+ config_in_json);
return 0;
}
@@ -3979,6 +4133,7 @@ int main_migrate(int argc, char **argv)
{
uint32_t domid;
const char *config_filename = NULL;
+ bool config_in_json = false;
const char *ssh_command = "ssh";
char *rune = NULL;
char *host;
@@ -3989,7 +4144,7 @@ int main_migrate(int argc, char **argv)
{0, 0, 0, 0}
};
- SWITCH_FOREACH_OPT(opt, "FC:s:e", opts, "migrate", 2) {
+ SWITCH_FOREACH_OPT(opt, "FC:s:ej", opts, "migrate", 2) {
case 'C':
config_filename = optarg;
break;
@@ -4003,11 +4158,20 @@ int main_migrate(int argc, char **argv)
daemonize = 0;
monitor = 0;
break;
+ case 'j':
+ config_in_json = true;
+ break;
case 0x100:
debug = 1;
break;
}
+ if (!config_filename && config_in_json) {
+ fprintf(stderr, "\"-j\" specified but no config file provided\n");
+ help("migrate");
+ return 2;
+ }
+
domid = find_domain(argv[optind]);
host = argv[optind + 1];
@@ -4036,7 +4200,7 @@ int main_migrate(int argc, char **argv)
return 1;
}
- migrate_domain(domid, rune, debug, config_filename);
+ migrate_domain(domid, rune, debug, config_filename, config_in_json);
return 0;
}
#endif
@@ -4265,7 +4429,8 @@ int main_create(int argc, char **argv)
char extra_config[1024];
struct domain_create dom_info;
int paused = 0, debug = 0, daemonize = 1, console_autoconnect = 0,
- quiet = 0, monitor = 1, vnc = 0, vncautopass = 0;
+ quiet = 0, monitor = 1, vnc = 0, vncautopass = 0,
+ config_in_json = 0;
int opt, rc;
static struct option opts[] = {
{"dryrun", 0, 0, 'n'},
@@ -4282,7 +4447,7 @@ int main_create(int argc, char **argv)
argc--; argv++;
}
- SWITCH_FOREACH_OPT(opt, "Fhnqf:pcdeVA", opts, "create", 0) {
+ SWITCH_FOREACH_OPT(opt, "Fhnqf:pcdeVAj", opts, "create", 0) {
case 'f':
filename = optarg;
break;
@@ -4314,6 +4479,9 @@ int main_create(int argc, char **argv)
case 'A':
vnc = vncautopass = 1;
break;
+ case 'j':
+ config_in_json = 1;
+ break;
}
extra_config[0] = '\0';
@@ -4329,6 +4497,11 @@ int main_create(int argc, char **argv)
}
}
+ if (config_in_json && strlen(extra_config)) {
+ fprintf(stderr, "\"-j\" specified, key=value extra configs ignored\n");
+ extra_config[0] = '\0';
+ }
+
memset(&dom_info, 0, sizeof(dom_info));
dom_info.debug = debug;
dom_info.daemonize = daemonize;
@@ -4337,6 +4510,7 @@ int main_create(int argc, char **argv)
dom_info.dryrun = dryrun_only;
dom_info.quiet = quiet;
dom_info.config_file = filename;
+ dom_info.config_in_json = config_in_json;
dom_info.extra_config = extra_config;
dom_info.migrate_fd = -1;
dom_info.vnc = vnc;
@@ -4358,6 +4532,7 @@ int main_config_update(int argc, char **argv)
char extra_config[1024];
void *config_data = 0;
int config_len = 0;
+ bool config_in_json = false;
libxl_domain_config d_config;
int opt, rc;
int debug = 0;
@@ -4381,13 +4556,15 @@ int main_config_update(int argc, char **argv)
argc--; argv++;
}
- SWITCH_FOREACH_OPT(opt, "dhqf:", opts, "config_update", 0) {
+ SWITCH_FOREACH_OPT(opt, "dhqf:j", opts, "config_update", 0) {
case 'd':
debug = 1;
break;
case 'f':
filename = optarg;
break;
+ case 'j':
+ config_in_json = true;
}
extra_config[0] = '\0';
@@ -4402,6 +4579,12 @@ int main_config_update(int argc, char **argv)
return 2;
}
}
+
+ if (config_in_json && strlen(extra_config)) {
+ fprintf(stderr, "\"-j\" specified, key=value extra configs ignored\n");
+ extra_config[0] = '\0';
+ }
+
if (filename) {
free(config_data); config_data = 0;
rc = libxl_read_file_contents(ctx, filename,
@@ -4430,19 +4613,26 @@ int main_config_update(int argc, char **argv)
libxl_domain_config_init(&d_config);
- parse_config_data(filename, config_data, config_len, &d_config, NULL);
+ if (config_in_json)
+ parse_config_data_json(config_data, config_len, &d_config);
+ else
+ parse_config_data(filename, config_data, config_len, &d_config, NULL);
if (debug || dryrun_only)
printf_info(default_output_format, -1, &d_config);
if (!dryrun_only) {
+ char *d_config_json = NULL;
fprintf(stderr, "setting dom%d configuration\n", domid);
- rc = libxl_userdata_store(ctx, domid, "xl",
- config_data, config_len);
+ d_config_json = libxl_domain_config_to_json(ctx, &d_config);
+ rc = libxl_userdata_store(ctx, domid, "xl-json",
+ (const uint8_t*)d_config_json,
+ strlen(d_config_json));
if (rc) {
fprintf(stderr, "failed to update configuration\n");
exit(1);
}
+ free(d_config_json);
}
libxl_domain_config_dispose(&d_config);
@@ -7196,6 +7386,7 @@ int main_remus(int argc, char **argv)
pid_t child = -1;
uint8_t *config_data;
int config_len;
+ bool config_in_json;
memset(&r_info, 0, sizeof(libxl_domain_remus_info));
/* Defaults */
@@ -7241,7 +7432,8 @@ int main_remus(int argc, char **argv)
return 1;
}
- save_domain_core_begin(domid, NULL, &config_data, &config_len);
+ save_domain_core_begin(domid, NULL, &config_data, &config_len,
+ &config_in_json);
if (!config_len) {
fprintf(stderr, "No config file stored for running domain and "
@@ -7251,7 +7443,8 @@ int main_remus(int argc, char **argv)
child = create_migration_child(rune, &send_fd, &recv_fd);
- migrate_do_preamble(send_fd, recv_fd, child, config_data, config_len,
+ migrate_do_preamble(send_fd, recv_fd, child,
+ config_data, config_len, true /* config_in_json */,
rune);
if (ssh_command[0])
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 4279b9f..cdc8e14 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -33,7 +33,8 @@ struct cmd_spec cmd_table[] = {
"-e Do not wait in the background for the death of the domain.\n"
"-V, --vncviewer Connect to the VNC display after the domain is created.\n"
"-A, --vncviewer-autopass\n"
- " Pass VNC password to viewer via stdin."
+ " Pass VNC password to viewer via stdin.\n"
+ "-j Config file in \"xl-json\" format."
},
{ "config-update",
&main_config_update, 1, 1,
@@ -43,6 +44,7 @@ struct cmd_spec cmd_table[] = {
"-h Print this help.\n"
"-f FILE, --defconfig=FILE\n Use the given configuration file.\n"
"-d Enable debug messages.\n"
+ "-j <ConfigFile> in \"xl-json\" format."
},
{ "list",
&main_list, 0, 0,
@@ -147,7 +149,8 @@ struct cmd_spec cmd_table[] = {
"[options] <Domain> <CheckpointFile> [<ConfigFile>]",
"-h Print this help.\n"
"-c Leave domain running after creating the snapshot.\n"
- "-p Leave domain paused after creating the snapshot."
+ "-p Leave domain paused after creating the snapshot.\n"
+ "-j Config file is in \"xl-json\" format (ConfigFile cannot be null)."
},
{ "migrate",
&main_migrate, 0, 1,
@@ -155,6 +158,7 @@ struct cmd_spec cmd_table[] = {
"[options] <Domain> <host>",
"-h Print this help.\n"
"-C <config> Send <config> instead of config file from creation.\n"
+ "-j <config> is in \"xl-json\" format.\n"
"-s <sshcommand> Use <sshcommand> instead of ssh. String will be passed\n"
" to sh. If empty, run <host> instead of ssh <host> xl\n"
" migrate-receive [-d -e]\n"
@@ -171,7 +175,8 @@ struct cmd_spec cmd_table[] = {
"-e Do not wait in the background for the death of the domain.\n"
"-d Enable debug messages.\n"
"-V, --vncviewer Connect to the VNC display after the domain is created.\n"
- "-A, --vncviewer-autopass Pass VNC password to viewer via stdin."
+ "-A, --vncviewer-autopass Pass VNC password to viewer via stdin.\n"
+ "-j Config file is in \"xl-json\" format (ConfigFile cannot be null)."
},
{ "migrate-receive",
&main_migrate_receive, 0, 1,
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-01 12:58 ` [PATCH V4 19/24] xl: introduce and use "xl-json" format Wei Liu
@ 2014-05-06 14:26 ` Ian Campbell
2014-05-06 15:17 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 14:26 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> Originally xl verbatimly copies the domain config file in its user data
I'm not 100% sure but I think the correct phrasing is "... copies the
domain config verbatim into its...". And either s/Originally/Currently/
or s/copies/copied/.
> diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> index 30bd4bf..f1a1b9b 100644
> --- a/docs/man/xl.pod.1
> +++ b/docs/man/xl.pod.1
> @@ -146,6 +146,11 @@ useful for determining issues with crashing domains and just as a
> general convenience since you often want to watch the
> domain boot.
>
> +=item B<-j>
> +
> +The provided I<configfile> is in JSON format. Cannot be used with "key=value"
> +at the same time.
Do I understand correctly that the intention here is to introduce and
support JSON format configuration files generally, rather than only as
part of the migration protocol?
I don't necessarily object to this but we should we wary of adding new
things to support just because they appear to be easy to do as a side
effect of some other work, there is a maintenance burden associated with
adding this feature, which might potentially be substantial in the long
run.
Do we think there is much call for this feature?
(I'm not vetoing this, I just want to make sure we've thought it
through...)
(having read through the patch, I'm not sure how "free" this is -- the
use of config_in_json seems to have got its tendrils into a lot of
places)
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index c6a9a0d..970eba2 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -1081,6 +1081,7 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
> *
> * userid Data contents
> * "xl" domain config file in xl format, Unix line endings
> + * "xl-json" domain config in JSON format generated by xl
"generated by libxl" I think. Could even call it "libxl-json".
> * "libvirt-xml" domain config file in libvirt XML format. See
> * http://libvirt.org/formatdomain.html
> *
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 289ea9a..7443d86 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -110,6 +110,8 @@ static const char migrate_report[]=
> * from target to source
> */
>
> +#define XL_MANDATORY_FLAG_JSON (1U << 0) /* config data is in JSON format */
> +#define XL_MANDATORY_FLAG_ALL (XL_MANDATORY_FLAG_JSON)
> struct save_file_header {
> char magic[32]; /* savefileheader_magic */
> /* All uint32_ts are in domain's byte order. */
> @@ -151,6 +153,7 @@ struct domain_create {
> int console_autoconnect;
> int checkpointed_stream;
> const char *config_file;
> + int config_in_json;
> const char *extra_config; /* extra config string */
> const char *restore_file;
> int migrate_fd; /* -1 means none */
> @@ -693,6 +696,73 @@ static void parse_top_level_sdl_options(XLU_Config *config,
> xlu_cfg_replace_string (config, "xauthority", &sdl->xauthority, 0);
> }
>
> +static void nic_update_default(libxl_domain_config *d_config)
> +{
> + int i;
> + libxl_device_nic *nic;
> +
> + for (i = 0; i < d_config->num_nics; i++) {
> + nic = &d_config->nics[i];
> +
> + if (default_vifscript) {
> + free(nic->script);
> + nic->script = strdup(default_vifscript);
> + }
You can use the replace_string() helper for all these I think.
Except -- what about any explicitly configured script (ie. vif =
['script=blargle']), does this not replace it with the default? Should
this be if !nic->script ?
Do you pickle the original config before libxl on the remote end has set
the defaults? Otherwise how can we distinguish a user supplied script
from a libxl supplied default?
> +static void parse_config_data_json(char *config_data,
> + int config_len,
> + libxl_domain_config *d_config)
> +{
> + int ret;
> +
> + if (!config_len) {
> + fprintf(stderr, "Config data stream empty\n");
> + exit(1);
> + }
> +
> + /* Make sure this string ends with \0 -- the parser expects a NULL
> + * terminated string.
Perhaps this guarantee could be pushed down into the helper which slurps
the file in for us in the first place?
(that's assuming that making the parser accept a length instead of a
asciz is not possible)
> + */
> + if (config_data[config_len-1] != '\0') {
> + config_data = realloc(config_data, config_len + 1);
> + if (!config_data) {
> + fprintf(stderr, "Failed to realloc config_data\n");
> + exit(1);
> + }
> + config_data[config_len] = '\0';
> + }
> +
> + ret = libxl_domain_config_from_json(ctx, d_config, config_data);
> + if (ret) {
> + fprintf(stderr, "Failed to parse config\n");
> + exit(1);
> + }
> +
> + if (blkdev_start) {
> + free(d_config->b_info.blkdev_start);
> + d_config->b_info.blkdev_start = strdup(blkdev_start);
> + }
replace_string again.
> +
> + nic_update_default(d_config);
> +}
> +
> static void parse_config_data(const char *config_source,
> const char *config_data,
> int config_len,
> @@ -930,6 +1000,8 @@ static void parse_config_data(const char *config_source,
> b_info->rtc_timeoffset = l;
>
> if (dom_info && !xlu_cfg_get_long(config, "vncviewer", &l, 0)) {
> + fprintf(stderr, "WARNING: \"vncviewer\" option found. It might be removed in future release. "
> + "Use \"-V\" option of \"xl create\" to automatically spawn vncviewer.\n");
We should either decide this is deprecated or not "might be removed in
the future" is not a helpful thing to say I don't think. It either will
or it won't be removed. Are we feeling brave enough to just remove it?
> @@ -1965,11 +2024,43 @@ static void evdisable_disk_ejects(libxl_evgen_disk_eject **diskws,
> }
> }
>
> +/* Selectively update domain configuration struct. This function is
> + * only used when creating domain.
> + *
> + * src contains a libxl_domain_config that is used by libxl to create
> + * a domain. Presumably libxl fills in relevant information when
> + * creating a domain.
s/Presumably/It assumes that/ (or requires that).
> + *
> + * dst contains a vanilla copy of domain configuration from user
"from the user supplied"
> + * supplied config file. It serves as a template.
> + *
> + * The end result is that dst now contains all relevant information to
> + * reconstruct a domain based on user's configurations and libxl's
> + * decision.
"decisions"
> @@ -2111,12 +2207,22 @@ static uint32_t create_domain(struct domain_create *dom_info)
> return ERROR_INVAL;
> }
> config_source = "<saved>";
> + config_in_json = !!(hdr.mandatory_flags & XL_MANDATORY_FLAG_JSON);
> }
>
> if (!dom_info->quiet)
> printf("Parsing config from %s\n", config_source);
>
> - parse_config_data(config_source, config_data, config_len, &d_config, dom_info);
> + if (config_in_json)
> + parse_config_data_json(config_data, config_len, &d_config);
> + else
> + parse_config_data(config_source, config_data, config_len,
> + &d_config, dom_info);
> +
> + /* Save a copy of vanilla domain configuration, as libxl routines
> + * will fill in more stuffs.
s/stuffs/stuff/.
> + */
> + libxl_domain_config_copy(ctx, &d_config_saved, &d_config);
>
> if (migrate_fd >= 0) {
> if (d_config.c_info.name) {
> @@ -3364,6 +3490,8 @@ static void save_domain_core_writeconfig(int fd, const char *source,
> u32buf.u32 = config_len;
> ADD_OPTDATA(u32buf.b, 4);
> ADD_OPTDATA(config_data, config_len);
> + if (config_in_json)
Should this not always be true when saving? Even if the original was in
xl syntax we've by now converted it?
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-06 14:26 ` Ian Campbell
@ 2014-05-06 15:17 ` Wei Liu
2014-05-06 16:35 ` Wei Liu
` (2 more replies)
0 siblings, 3 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 15:17 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 03:26:42PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > Originally xl verbatimly copies the domain config file in its user data
>
> I'm not 100% sure but I think the correct phrasing is "... copies the
> domain config verbatim into its...". And either s/Originally/Currently/
> or s/copies/copied/.
>
Ah, OK. I will fix this.
> > diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> > index 30bd4bf..f1a1b9b 100644
> > --- a/docs/man/xl.pod.1
> > +++ b/docs/man/xl.pod.1
> > @@ -146,6 +146,11 @@ useful for determining issues with crashing domains and just as a
> > general convenience since you often want to watch the
> > domain boot.
> >
> > +=item B<-j>
> > +
> > +The provided I<configfile> is in JSON format. Cannot be used with "key=value"
> > +at the same time.
>
> Do I understand correctly that the intention here is to introduce and
> support JSON format configuration files generally, rather than only as
> part of the migration protocol?
>
Yes. You understanding is correct.
> I don't necessarily object to this but we should we wary of adding new
> things to support just because they appear to be easy to do as a side
> effect of some other work, there is a maintenance burden associated with
> adding this feature, which might potentially be substantial in the long
> run.
>
> Do we think there is much call for this feature?
>
> (I'm not vetoing this, I just want to make sure we've thought it
> through...)
>
IanJ seemed to fond of this so I added it here. But we can wait, I
think. Next version will be much shorter without this. :-)
> (having read through the patch, I'm not sure how "free" this is -- the
> use of config_in_json seems to have got its tendrils into a lot of
> places)
>
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index c6a9a0d..970eba2 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -1081,6 +1081,7 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
> > *
> > * userid Data contents
> > * "xl" domain config file in xl format, Unix line endings
> > + * "xl-json" domain config in JSON format generated by xl
>
> "generated by libxl" I think. Could even call it "libxl-json".
>
In my original design this file is actually a JSON representation of xl
config file, but it's not the case anymore. So I think calling it
libxl-json is sufficient.
On the flip side this file is produced by xl (for now) and only used by
xl only, so I think "xl-json" is also approriate.
Not sure which one is better.
> > * "libvirt-xml" domain config file in libvirt XML format. See
> > * http://libvirt.org/formatdomain.html
> > *
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 289ea9a..7443d86 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -110,6 +110,8 @@ static const char migrate_report[]=
> > * from target to source
> > */
> >
> > +#define XL_MANDATORY_FLAG_JSON (1U << 0) /* config data is in JSON format */
> > +#define XL_MANDATORY_FLAG_ALL (XL_MANDATORY_FLAG_JSON)
> > struct save_file_header {
> > char magic[32]; /* savefileheader_magic */
> > /* All uint32_ts are in domain's byte order. */
> > @@ -151,6 +153,7 @@ struct domain_create {
> > int console_autoconnect;
> > int checkpointed_stream;
> > const char *config_file;
> > + int config_in_json;
> > const char *extra_config; /* extra config string */
> > const char *restore_file;
> > int migrate_fd; /* -1 means none */
> > @@ -693,6 +696,73 @@ static void parse_top_level_sdl_options(XLU_Config *config,
> > xlu_cfg_replace_string (config, "xauthority", &sdl->xauthority, 0);
> > }
> >
> > +static void nic_update_default(libxl_domain_config *d_config)
> > +{
> > + int i;
> > + libxl_device_nic *nic;
> > +
> > + for (i = 0; i < d_config->num_nics; i++) {
> > + nic = &d_config->nics[i];
> > +
> > + if (default_vifscript) {
> > + free(nic->script);
> > + nic->script = strdup(default_vifscript);
> > + }
>
> You can use the replace_string() helper for all these I think.
>
> Except -- what about any explicitly configured script (ie. vif =
> ['script=blargle']), does this not replace it with the default? Should
> this be if !nic->script ?
>
Haven't thought about this. This is actually just code motion. I think
you just discovered a bug. One more patch added to queue!
> Do you pickle the original config before libxl on the remote end has set
> the defaults? Otherwise how can we distinguish a user supplied script
> from a libxl supplied default?
>
> > +static void parse_config_data_json(char *config_data,
> > + int config_len,
> > + libxl_domain_config *d_config)
> > +{
> > + int ret;
> > +
> > + if (!config_len) {
> > + fprintf(stderr, "Config data stream empty\n");
> > + exit(1);
> > + }
> > +
> > + /* Make sure this string ends with \0 -- the parser expects a NULL
> > + * terminated string.
>
> Perhaps this guarantee could be pushed down into the helper which slurps
> the file in for us in the first place?
>
I will see what I can do.
> (that's assuming that making the parser accept a length instead of a
> asciz is not possible)
>
Correct, it's a stream parser. It doesn't accept a length.
> > + */
> > + if (config_data[config_len-1] != '\0') {
> > + config_data = realloc(config_data, config_len + 1);
> > + if (!config_data) {
> > + fprintf(stderr, "Failed to realloc config_data\n");
> > + exit(1);
> > + }
> > + config_data[config_len] = '\0';
> > + }
> > +
> > + ret = libxl_domain_config_from_json(ctx, d_config, config_data);
> > + if (ret) {
> > + fprintf(stderr, "Failed to parse config\n");
> > + exit(1);
> > + }
> > +
> > + if (blkdev_start) {
> > + free(d_config->b_info.blkdev_start);
> > + d_config->b_info.blkdev_start = strdup(blkdev_start);
> > + }
>
> replace_string again.
>
Ack.
> > +
> > + nic_update_default(d_config);
> > +}
> > +
> > static void parse_config_data(const char *config_source,
> > const char *config_data,
> > int config_len,
> > @@ -930,6 +1000,8 @@ static void parse_config_data(const char *config_source,
> > b_info->rtc_timeoffset = l;
> >
> > if (dom_info && !xlu_cfg_get_long(config, "vncviewer", &l, 0)) {
> > + fprintf(stderr, "WARNING: \"vncviewer\" option found. It might be removed in future release. "
> > + "Use \"-V\" option of \"xl create\" to automatically spawn vncviewer.\n");
>
> We should either decide this is deprecated or not "might be removed in
> the future" is not a helpful thing to say I don't think. It either will
> or it won't be removed. Are we feeling brave enough to just remove it?
>
I think it's quite safe to remove it. This functional regression is in
no way critical and can be easily worked around.
> > @@ -1965,11 +2024,43 @@ static void evdisable_disk_ejects(libxl_evgen_disk_eject **diskws,
> > }
> > }
> >
> > +/* Selectively update domain configuration struct. This function is
> > + * only used when creating domain.
> > + *
> > + * src contains a libxl_domain_config that is used by libxl to create
> > + * a domain. Presumably libxl fills in relevant information when
> > + * creating a domain.
>
> s/Presumably/It assumes that/ (or requires that).
>
>
Ack.
> > + *
> > + * dst contains a vanilla copy of domain configuration from user
>
> "from the user supplied"
>
> > + * supplied config file. It serves as a template.
> > + *
> > + * The end result is that dst now contains all relevant information to
> > + * reconstruct a domain based on user's configurations and libxl's
> > + * decision.
>
> "decisions"
>
Ack.
> > @@ -2111,12 +2207,22 @@ static uint32_t create_domain(struct domain_create *dom_info)
> > return ERROR_INVAL;
> > }
> > config_source = "<saved>";
> > + config_in_json = !!(hdr.mandatory_flags & XL_MANDATORY_FLAG_JSON);
> > }
> >
> > if (!dom_info->quiet)
> > printf("Parsing config from %s\n", config_source);
> >
> > - parse_config_data(config_source, config_data, config_len, &d_config, dom_info);
> > + if (config_in_json)
> > + parse_config_data_json(config_data, config_len, &d_config);
> > + else
> > + parse_config_data(config_source, config_data, config_len,
> > + &d_config, dom_info);
> > +
> > + /* Save a copy of vanilla domain configuration, as libxl routines
> > + * will fill in more stuffs.
>
> s/stuffs/stuff/.
>
Ack.
> > + */
> > + libxl_domain_config_copy(ctx, &d_config_saved, &d_config);
> >
> > if (migrate_fd >= 0) {
> > if (d_config.c_info.name) {
> > @@ -3364,6 +3490,8 @@ static void save_domain_core_writeconfig(int fd, const char *source,
> > u32buf.u32 = config_len;
> > ADD_OPTDATA(u32buf.b, 4);
> > ADD_OPTDATA(config_data, config_len);
> > + if (config_in_json)
>
> Should this not always be true when saving? Even if the original was in
> xl syntax we've by now converted it?
>
This can be done. It's just that I have not coded it to parse user
supplied config file when saving.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-06 15:17 ` Wei Liu
@ 2014-05-06 16:35 ` Wei Liu
2014-05-07 10:11 ` Ian Campbell
2014-05-07 11:16 ` Ian Jackson
2 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 16:35 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 04:17:53PM +0100, Wei Liu wrote:
[...]
> > >
> > > +static void nic_update_default(libxl_domain_config *d_config)
> > > +{
> > > + int i;
> > > + libxl_device_nic *nic;
> > > +
> > > + for (i = 0; i < d_config->num_nics; i++) {
> > > + nic = &d_config->nics[i];
> > > +
> > > + if (default_vifscript) {
> > > + free(nic->script);
> > > + nic->script = strdup(default_vifscript);
> > > + }
> >
> > You can use the replace_string() helper for all these I think.
> >
> > Except -- what about any explicitly configured script (ie. vif =
> > ['script=blargle']), does this not replace it with the default? Should
> > this be if !nic->script ?
> >
>
> Haven't thought about this. This is actually just code motion. I think
> you just discovered a bug. One more patch added to queue!
>
Actually I was wrong, this code motion is buggy and I will fix it.
There's no bug in the original code.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-06 15:17 ` Wei Liu
2014-05-06 16:35 ` Wei Liu
@ 2014-05-07 10:11 ` Ian Campbell
2014-05-07 11:18 ` Ian Jackson
2014-05-07 11:20 ` Wei Liu
2014-05-07 11:16 ` Ian Jackson
2 siblings, 2 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:11 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Tue, 2014-05-06 at 16:17 +0100, Wei Liu wrote:
> > > diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> > > index 30bd4bf..f1a1b9b 100644
> > > --- a/docs/man/xl.pod.1
> > > +++ b/docs/man/xl.pod.1
> > > @@ -146,6 +146,11 @@ useful for determining issues with crashing domains and just as a
> > > general convenience since you often want to watch the
> > > domain boot.
> > >
> > > +=item B<-j>
> > > +
> > > +The provided I<configfile> is in JSON format. Cannot be used with "key=value"
> > > +at the same time.
> >
> > Do I understand correctly that the intention here is to introduce and
> > support JSON format configuration files generally, rather than only as
> > part of the migration protocol?
> >
>
> Yes. You understanding is correct.
>
> > I don't necessarily object to this but we should we wary of adding new
> > things to support just because they appear to be easy to do as a side
> > effect of some other work, there is a maintenance burden associated with
> > adding this feature, which might potentially be substantial in the long
> > run.
> >
> > Do we think there is much call for this feature?
> >
> > (I'm not vetoing this, I just want to make sure we've thought it
> > through...)
> >
>
> IanJ seemed to fond of this so I added it here. But we can wait, I
> think. Next version will be much shorter without this. :-)
It certainly would be. IanJ -- thoughts?
> > (having read through the patch, I'm not sure how "free" this is -- the
> > use of config_in_json seems to have got its tendrils into a lot of
> > places)
> >
> > > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > > index c6a9a0d..970eba2 100644
> > > --- a/tools/libxl/libxl.h
> > > +++ b/tools/libxl/libxl.h
> > > @@ -1081,6 +1081,7 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
> > > *
> > > * userid Data contents
> > > * "xl" domain config file in xl format, Unix line endings
> > > + * "xl-json" domain config in JSON format generated by xl
> >
> > "generated by libxl" I think. Could even call it "libxl-json".
> >
>
> In my original design this file is actually a JSON representation of xl
> config file, but it's not the case anymore. So I think calling it
> libxl-json is sufficient.
>
> On the flip side this file is produced by xl (for now) and only used by
> xl only, so I think "xl-json" is also approriate.
>
> Not sure which one is better.
Me neither. I suppose the question is would any other toolstack which
wanted to save a libxl_domain_config in this way want to use the same
userid or a different one?
By using a common one could we perhaps fix the issue of not being able
to poke at e.g. libvirt domain using xl? (specifically things like xl
list -v on a system with a mix of domains does odd things IIRC).
Perhaps storing (and all the updating) this thing is something which
libxl rather than xl should take care of? With a function
libxl_domain_fetch_config(ctx, domid, &libxl_domain_config)?
> > > +
> > > + nic_update_default(d_config);
> > > +}
> > > +
> > > static void parse_config_data(const char *config_source,
> > > const char *config_data,
> > > int config_len,
> > > @@ -930,6 +1000,8 @@ static void parse_config_data(const char *config_source,
> > > b_info->rtc_timeoffset = l;
> > >
> > > if (dom_info && !xlu_cfg_get_long(config, "vncviewer", &l, 0)) {
> > > + fprintf(stderr, "WARNING: \"vncviewer\" option found. It might be removed in future release. "
> > > + "Use \"-V\" option of \"xl create\" to automatically spawn vncviewer.\n");
> >
> > We should either decide this is deprecated or not "might be removed in
> > the future" is not a helpful thing to say I don't think. It either will
> > or it won't be removed. Are we feeling brave enough to just remove it?
> >
>
> I think it's quite safe to remove it. This functional regression is in
> no way critical and can be easily worked around.
Lets do that then, in a separate patch with an obvious title so it isn't
going in "under the radar".
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-07 10:11 ` Ian Campbell
@ 2014-05-07 11:18 ` Ian Jackson
2014-05-07 11:20 ` Wei Liu
1 sibling, 0 replies; 120+ messages in thread
From: Ian Jackson @ 2014-05-07 11:18 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
Ian Campbell writes ("Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format"):
> By using a common one could we perhaps fix the issue of not being able
> to poke at e.g. libvirt domain using xl? (specifically things like xl
> list -v on a system with a mix of domains does odd things IIRC).
>
> Perhaps storing (and all the updating) this thing is something which
> libxl rather than xl should take care of? With a function
> libxl_domain_fetch_config(ctx, domid, &libxl_domain_config)?
In that case libxl_domain_create would have to transform the supplied
domain config back into json before saving it.
But that might well be a better approach.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-07 10:11 ` Ian Campbell
2014-05-07 11:18 ` Ian Jackson
@ 2014-05-07 11:20 ` Wei Liu
2014-05-07 11:30 ` Ian Campbell
1 sibling, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-07 11:20 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Wed, May 07, 2014 at 11:11:09AM +0100, Ian Campbell wrote:
> On Tue, 2014-05-06 at 16:17 +0100, Wei Liu wrote:
> > > > diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> > > > index 30bd4bf..f1a1b9b 100644
> > > > --- a/docs/man/xl.pod.1
> > > > +++ b/docs/man/xl.pod.1
> > > > @@ -146,6 +146,11 @@ useful for determining issues with crashing domains and just as a
> > > > general convenience since you often want to watch the
> > > > domain boot.
> > > >
> > > > +=item B<-j>
> > > > +
> > > > +The provided I<configfile> is in JSON format. Cannot be used with "key=value"
> > > > +at the same time.
> > >
> > > Do I understand correctly that the intention here is to introduce and
> > > support JSON format configuration files generally, rather than only as
> > > part of the migration protocol?
> > >
> >
> > Yes. You understanding is correct.
> >
> > > I don't necessarily object to this but we should we wary of adding new
> > > things to support just because they appear to be easy to do as a side
> > > effect of some other work, there is a maintenance burden associated with
> > > adding this feature, which might potentially be substantial in the long
> > > run.
> > >
> > > Do we think there is much call for this feature?
> > >
> > > (I'm not vetoing this, I just want to make sure we've thought it
> > > through...)
> > >
> >
> > IanJ seemed to fond of this so I added it here. But we can wait, I
> > think. Next version will be much shorter without this. :-)
>
> It certainly would be. IanJ -- thoughts?
>
> > > (having read through the patch, I'm not sure how "free" this is -- the
> > > use of config_in_json seems to have got its tendrils into a lot of
> > > places)
> > >
> > > > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > > > index c6a9a0d..970eba2 100644
> > > > --- a/tools/libxl/libxl.h
> > > > +++ b/tools/libxl/libxl.h
> > > > @@ -1081,6 +1081,7 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
> > > > *
> > > > * userid Data contents
> > > > * "xl" domain config file in xl format, Unix line endings
> > > > + * "xl-json" domain config in JSON format generated by xl
> > >
> > > "generated by libxl" I think. Could even call it "libxl-json".
> > >
> >
> > In my original design this file is actually a JSON representation of xl
> > config file, but it's not the case anymore. So I think calling it
> > libxl-json is sufficient.
> >
> > On the flip side this file is produced by xl (for now) and only used by
> > xl only, so I think "xl-json" is also approriate.
> >
> > Not sure which one is better.
>
> Me neither. I suppose the question is would any other toolstack which
> wanted to save a libxl_domain_config in this way want to use the same
> userid or a different one?
>
I am not sure if any other toolstack will use the same format. libvirt
has its own entry called "libvirt-xml".
> By using a common one could we perhaps fix the issue of not being able
> to poke at e.g. libvirt domain using xl? (specifically things like xl
> list -v on a system with a mix of domains does odd things IIRC).
>
Not really, at least not without making libvirt use the same stored
data, or not without xl being able to consume libvirt-xml.
> Perhaps storing (and all the updating) this thing is something which
> libxl rather than xl should take care of? With a function
> libxl_domain_fetch_config(ctx, domid, &libxl_domain_config)?
>
As I stated in 00 I wasn't sure about this either. It's a design
decision which needs more input.
One thing that makes me feel weird is that having libxl to manage domain
state -- that means a library is managing domain state? My gut feeling
is that it should be a "tool" not a "library" that manages domain
states.
Wei.
> > > > +
> > > > + nic_update_default(d_config);
> > > > +}
> > > > +
> > > > static void parse_config_data(const char *config_source,
> > > > const char *config_data,
> > > > int config_len,
> > > > @@ -930,6 +1000,8 @@ static void parse_config_data(const char *config_source,
> > > > b_info->rtc_timeoffset = l;
> > > >
> > > > if (dom_info && !xlu_cfg_get_long(config, "vncviewer", &l, 0)) {
> > > > + fprintf(stderr, "WARNING: \"vncviewer\" option found. It might be removed in future release. "
> > > > + "Use \"-V\" option of \"xl create\" to automatically spawn vncviewer.\n");
> > >
> > > We should either decide this is deprecated or not "might be removed in
> > > the future" is not a helpful thing to say I don't think. It either will
> > > or it won't be removed. Are we feeling brave enough to just remove it?
> > >
> >
> > I think it's quite safe to remove it. This functional regression is in
> > no way critical and can be easily worked around.
>
> Lets do that then, in a separate patch with an obvious title so it isn't
> going in "under the radar".
>
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-07 11:20 ` Wei Liu
@ 2014-05-07 11:30 ` Ian Campbell
2014-05-07 13:00 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 11:30 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
> > I suppose the question is would any other toolstack which
> > wanted to save a libxl_domain_config in this way want to use the same
> > userid or a different one?
> >
>
> I am not sure if any other toolstack will use the same format. libvirt
> has its own entry called "libvirt-xml".
Does it seem implausible that some hypothetical toolstack would also
want to save the pickled content of libxl_domain_config?
(I'm not sure what libvirt uses libvirt-xml for, it seems mostly to be
just to detect if this is a libvirt managed domain).
> > By using a common one could we perhaps fix the issue of not being able
> > to poke at e.g. libvirt domain using xl? (specifically things like xl
> > list -v on a system with a mix of domains does odd things IIRC).
> >
>
> Not really, at least not without making libvirt use the same stored
> data, or not without xl being able to consume libvirt-xml.
What I meant to suggest was that if libxl took care of saving a
libxl_domain_config for each domain then it could work for any toolstack
which cared to opt into using it.
> > Perhaps storing (and all the updating) this thing is something which
> > libxl rather than xl should take care of? With a function
> > libxl_domain_fetch_config(ctx, domid, &libxl_domain_config)?
> >
>
> As I stated in 00 I wasn't sure about this either. It's a design
> decision which needs more input.
>
> One thing that makes me feel weird is that having libxl to manage domain
> state -- that means a library is managing domain state? My gut feeling
> is that it should be a "tool" not a "library" that manages domain
> states.
libxl manages the "state" in xenstore too I suppose, and libxl is
responsible for modifying the state so it isn't that weird to consider
that it might also be able to answer the question "what is the current
state". You've implemented that by saving the domain config as JSON, I
had previously thought that libxl could reconstitute the domain config
from the other state (I think your way is better, but in some sense this
could be considered an internal implementation detail of a libxl
interface).
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-07 11:30 ` Ian Campbell
@ 2014-05-07 13:00 ` Wei Liu
2014-05-07 13:08 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-07 13:00 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Wed, May 07, 2014 at 12:30:30PM +0100, Ian Campbell wrote:
> > > I suppose the question is would any other toolstack which
> > > wanted to save a libxl_domain_config in this way want to use the same
> > > userid or a different one?
> > >
> >
> > I am not sure if any other toolstack will use the same format. libvirt
> > has its own entry called "libvirt-xml".
>
> Does it seem implausible that some hypothetical toolstack would also
> want to save the pickled content of libxl_domain_config?
>
> (I'm not sure what libvirt uses libvirt-xml for, it seems mostly to be
> just to detect if this is a libvirt managed domain).
>
> > > By using a common one could we perhaps fix the issue of not being able
> > > to poke at e.g. libvirt domain using xl? (specifically things like xl
> > > list -v on a system with a mix of domains does odd things IIRC).
> > >
> >
> > Not really, at least not without making libvirt use the same stored
> > data, or not without xl being able to consume libvirt-xml.
>
> What I meant to suggest was that if libxl took care of saving a
> libxl_domain_config for each domain then it could work for any toolstack
> which cared to opt into using it.
>
OK, it makes sense.
> > > Perhaps storing (and all the updating) this thing is something which
> > > libxl rather than xl should take care of? With a function
> > > libxl_domain_fetch_config(ctx, domid, &libxl_domain_config)?
> > >
> >
> > As I stated in 00 I wasn't sure about this either. It's a design
> > decision which needs more input.
> >
> > One thing that makes me feel weird is that having libxl to manage domain
> > state -- that means a library is managing domain state? My gut feeling
> > is that it should be a "tool" not a "library" that manages domain
> > states.
>
> libxl manages the "state" in xenstore too I suppose, and libxl is
> responsible for modifying the state so it isn't that weird to consider
> that it might also be able to answer the question "what is the current
> state". You've implemented that by saving the domain config as JSON, I
> had previously thought that libxl could reconstitute the domain config
> from the other state (I think your way is better, but in some sense this
> could be considered an internal implementation detail of a libxl
> interface).
>
OK. Then I shall push the configuration synchronization down to libxl
level.
Wei.
> Ian.
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-06 15:17 ` Wei Liu
2014-05-06 16:35 ` Wei Liu
2014-05-07 10:11 ` Ian Campbell
@ 2014-05-07 11:16 ` Ian Jackson
2014-05-07 11:20 ` Ian Campbell
2014-05-07 12:56 ` Wei Liu
2 siblings, 2 replies; 120+ messages in thread
From: Ian Jackson @ 2014-05-07 11:16 UTC (permalink / raw)
To: Wei Liu; +Cc: Ian Campbell, xen-devel
Wei Liu writes ("Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format"):
> On Tue, May 06, 2014 at 03:26:42PM +0100, Ian Campbell wrote:
> > I don't necessarily object to this but we should we wary of adding new
> > things to support just because they appear to be easy to do as a side
> > effect of some other work, there is a maintenance burden associated with
> > adding this feature, which might potentially be substantial in the long
> > run.
> >
> > Do we think there is much call for this feature?
I think this is an essential feature. The official interface to libxl
is the IDL, and the xl config file format is a very indirect (but more
convenient for humans) way of specifying that. We should certainly
provide a way to use xl with a programmatically-generated config file.
> IanJ seemed to fond of this so I added it here. But we can wait, I
> think. Next version will be much shorter without this. :-)
OTOH it might profitably be in a separate patch.
> > (having read through the patch, I'm not sure how "free" this is -- the
> > use of config_in_json seems to have got its tendrils into a lot of
> > places)
Perhaps the code could be simplified by changing the types of
"config_file" et al to some struct containing the char* and the
format indicator.
> In my original design this file is actually a JSON representation of xl
> config file, but it's not the case anymore. So I think calling it
> libxl-json is sufficient.
>
> On the flip side this file is produced by xl (for now) and only used by
> xl only, so I think "xl-json" is also approriate.
>
> Not sure which one is better.
I don't think it matters very much.
> > You can use the replace_string() helper for all these I think.
> >
> > Except -- what about any explicitly configured script (ie. vif =
> > ['script=blargle']), does this not replace it with the default? Should
> > this be if !nic->script ?
>
> Haven't thought about this. This is actually just code motion. I think
> you just discovered a bug. One more patch added to queue!
I think these defaults belong in libxl, actually. IMO there shouldn't
be any validation or modification of the input json structure before
it is passed to libxl.
> > We should either decide this is deprecated or not "might be removed in
> > the future" is not a helpful thing to say I don't think. It either will
> > or it won't be removed. Are we feeling brave enough to just remove it?
>
> I think it's quite safe to remove it. This functional regression is in
> no way critical and can be easily worked around.
Yes. Probably it would be best to remove it in a pre-patch to make
this patch smaller.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-07 11:16 ` Ian Jackson
@ 2014-05-07 11:20 ` Ian Campbell
2014-05-07 12:56 ` Wei Liu
1 sibling, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 11:20 UTC (permalink / raw)
To: Ian Jackson; +Cc: Wei Liu, xen-devel
On Wed, 2014-05-07 at 12:16 +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format"):
> > On Tue, May 06, 2014 at 03:26:42PM +0100, Ian Campbell wrote:
> > > I don't necessarily object to this but we should we wary of adding new
> > > things to support just because they appear to be easy to do as a side
> > > effect of some other work, there is a maintenance burden associated with
> > > adding this feature, which might potentially be substantial in the long
> > > run.
> > >
> > > Do we think there is much call for this feature?
>
> I think this is an essential feature. The official interface to libxl
> is the IDL, and the xl config file format is a very indirect (but more
> convenient for humans) way of specifying that. We should certainly
> provide a way to use xl with a programmatically-generated config file.
OK. We really ought to start testing that around the same time then.
> > IanJ seemed to fond of this so I added it here. But we can wait, I
> > think. Next version will be much shorter without this. :-)
>
> OTOH it might profitably be in a separate patch.
Probable.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format
2014-05-07 11:16 ` Ian Jackson
2014-05-07 11:20 ` Ian Campbell
@ 2014-05-07 12:56 ` Wei Liu
1 sibling, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-07 12:56 UTC (permalink / raw)
To: Ian Jackson; +Cc: Wei Liu, Ian Campbell, xen-devel
On Wed, May 07, 2014 at 12:16:11PM +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [PATCH V4 19/24] xl: introduce and use "xl-json" format"):
> > On Tue, May 06, 2014 at 03:26:42PM +0100, Ian Campbell wrote:
> > > I don't necessarily object to this but we should we wary of adding new
> > > things to support just because they appear to be easy to do as a side
> > > effect of some other work, there is a maintenance burden associated with
> > > adding this feature, which might potentially be substantial in the long
> > > run.
> > >
> > > Do we think there is much call for this feature?
>
> I think this is an essential feature. The official interface to libxl
> is the IDL, and the xl config file format is a very indirect (but more
> convenient for humans) way of specifying that. We should certainly
> provide a way to use xl with a programmatically-generated config file.
>
> > IanJ seemed to fond of this so I added it here. But we can wait, I
> > think. Next version will be much shorter without this. :-)
>
> OTOH it might profitably be in a separate patch.
>
Let's do it in later patch then. This change alone is complex enough.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 20/24] xl: introduce load/save_domain_config
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (18 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 19/24] xl: introduce and use "xl-json" format Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 14:28 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 21/24] xl: update domain configuration when running mem-set and mem-max Wei Liu
` (4 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
They are used to serialize / deserialize domain configuration to / from
user data store.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/xl_cmdimpl.c | 119 ++++++++++++++++++++++++++++++----------------
1 file changed, 77 insertions(+), 42 deletions(-)
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 7443d86..f891ab5 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -160,6 +160,73 @@ struct domain_create {
char **migration_domname_r; /* from malloc */
};
+static int _parse_config_data_json(char *config_data, int config_len,
+ libxl_domain_config *d_config)
+{
+ int ret;
+
+ /* Make sure this string ends with \0 -- the parser expects a NULL
+ * terminated string.
+ */
+ if (config_data[config_len-1] != '\0') {
+ config_data = realloc(config_data, config_len + 1);
+ if (!config_data) {
+ fprintf(stderr, "Failed to realloc config_data\n");
+ ret = ERROR_NOMEM;
+ goto out;
+ }
+ config_data[config_len] = '\0';
+ }
+
+ ret = libxl_domain_config_from_json(ctx, d_config, config_data);
+
+out:
+ return ret;
+}
+/* Load domain configuration from userdata store */
+static int load_domain_config(uint32_t domid, libxl_domain_config *d_config)
+{
+ uint8_t *data;
+ int ret, len;
+
+ ret = libxl_userdata_retrieve(ctx, domid, "xl-json", &data, &len);
+ if (ret) {
+ fprintf(stderr, "failed to retrieve guest configuration (rc=%d).", ret);
+ ret = ERROR_FAIL;
+ goto out;
+ }
+
+ ret = _parse_config_data_json((char *)data, len, d_config);
+ if (ret)
+ goto out;
+
+ free(data);
+
+out:
+ return ret;
+}
+
+/* Store domain configuration to userdata store */
+static int store_domain_config(uint32_t domid, libxl_domain_config *d_config)
+{
+ char *d_config_json;
+ int ret;
+
+ d_config_json = libxl_domain_config_to_json(ctx, d_config);
+
+ ret = libxl_userdata_store(ctx, domid, "xl-json",
+ (const uint8_t *)d_config_json,
+ strlen(d_config_json));
+
+ free(d_config_json);
+
+ if (ret) {
+ fprintf(stderr, "failed to store guest configuration (rc=%d). ", ret);
+ return ERROR_FAIL;
+ }
+
+ return 0;
+}
static uint32_t find_domain(const char *p) __attribute__((warn_unused_result));
static uint32_t find_domain(const char *p)
@@ -737,19 +804,7 @@ static void parse_config_data_json(char *config_data,
exit(1);
}
- /* Make sure this string ends with \0 -- the parser expects a NULL
- * terminated string.
- */
- if (config_data[config_len-1] != '\0') {
- config_data = realloc(config_data, config_len + 1);
- if (!config_data) {
- fprintf(stderr, "Failed to realloc config_data\n");
- exit(1);
- }
- config_data[config_len] = '\0';
- }
-
- ret = libxl_domain_config_from_json(ctx, d_config, config_data);
+ ret = _parse_config_data_json(config_data, config_len, d_config);
if (ret) {
fprintf(stderr, "Failed to parse config\n");
exit(1);
@@ -2060,7 +2115,6 @@ static uint32_t create_domain(struct domain_create *dom_info)
uint32_t domid = INVALID_DOMID;
libxl_domain_config d_config, d_config_saved;
- char *d_config_json = NULL;
int debug = dom_info->debug;
int daemonize = dom_info->daemonize;
@@ -2318,15 +2372,8 @@ start:
}
update_domain_config(&d_config_saved, &d_config);
- d_config_json = libxl_domain_config_to_json(ctx, &d_config_saved);
- if (d_config_json == NULL) {
- perror("cannot generate JSON object of domain configuration");
- ret = ERROR_FAIL;
- goto error_out;
- }
- ret = libxl_userdata_store(ctx, domid, "xl-json",
- (const uint8_t *)d_config_json,
- strlen(d_config_json));
+
+ ret = store_domain_config(domid, &d_config_saved);
if (ret) {
perror("cannot save config file");
ret = ERROR_FAIL;
@@ -2492,8 +2539,6 @@ out:
free(config_data);
- free(d_config_json);
-
console_child_report(child_console);
if (deathw)
@@ -3218,9 +3263,7 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain)
{
libxl_domain_config d_config;
- char *config_source;
- uint8_t *data;
- int i, len, rc;
+ int i, rc;
yajl_gen hand = NULL;
yajl_gen_status s;
@@ -3244,20 +3287,17 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain)
/* no detailed info available on dom0 */
if (info[i].domid == 0)
continue;
- rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl-json",
- &data, &len);
- if (rc)
- continue;
- CHK_SYSCALL(asprintf(&config_source, "<domid %d data>", info[i].domid));
libxl_domain_config_init(&d_config);
- parse_config_data_json((char *)data, len, &d_config);
+ rc = load_domain_config(info[i].domid, &d_config);
+ if (rc) {
+ libxl_domain_config_dispose(&d_config);
+ continue;
+ }
if (default_output_format == OUTPUT_FORMAT_JSON)
s = printf_info_one_json(hand, info[i].domid, &d_config);
else
printf_info_sexp(info[i].domid, &d_config);
libxl_domain_config_dispose(&d_config);
- free(data);
- free(config_source);
if (s != yajl_gen_status_ok)
goto out;
}
@@ -4622,17 +4662,12 @@ int main_config_update(int argc, char **argv)
printf_info(default_output_format, -1, &d_config);
if (!dryrun_only) {
- char *d_config_json = NULL;
fprintf(stderr, "setting dom%d configuration\n", domid);
- d_config_json = libxl_domain_config_to_json(ctx, &d_config);
- rc = libxl_userdata_store(ctx, domid, "xl-json",
- (const uint8_t*)d_config_json,
- strlen(d_config_json));
+ rc = store_domain_config(domid, &d_config);
if (rc) {
fprintf(stderr, "failed to update configuration\n");
exit(1);
}
- free(d_config_json);
}
libxl_domain_config_dispose(&d_config);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 20/24] xl: introduce load/save_domain_config
2014-05-01 12:58 ` [PATCH V4 20/24] xl: introduce load/save_domain_config Wei Liu
@ 2014-05-06 14:28 ` Ian Campbell
2014-05-06 15:03 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 14:28 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> They are used to serialize / deserialize domain configuration to / from
> user data store.
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/xl_cmdimpl.c | 119 ++++++++++++++++++++++++++++++----------------
> 1 file changed, 77 insertions(+), 42 deletions(-)
>
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 7443d86..f891ab5 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -160,6 +160,73 @@ struct domain_create {
> char **migration_domname_r; /* from malloc */
> };
>
> +static int _parse_config_data_json(char *config_data, int config_len,
> + libxl_domain_config *d_config)
The _ namespace is reserved for something (either POSIX or the compiler,
but not us, for sure)
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 20/24] xl: introduce load/save_domain_config
2014-05-06 14:28 ` Ian Campbell
@ 2014-05-06 15:03 ` Wei Liu
2014-05-07 10:12 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 15:03 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 03:28:12PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > They are used to serialize / deserialize domain configuration to / from
> > user data store.
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/xl_cmdimpl.c | 119 ++++++++++++++++++++++++++++++----------------
> > 1 file changed, 77 insertions(+), 42 deletions(-)
> >
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 7443d86..f891ab5 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -160,6 +160,73 @@ struct domain_create {
> > char **migration_domname_r; /* from malloc */
> > };
> >
> > +static int _parse_config_data_json(char *config_data, int config_len,
> > + libxl_domain_config *d_config)
>
> The _ namespace is reserved for something (either POSIX or the compiler,
> but not us, for sure)
>
I will use _xl namespace instead.
Wei.
>
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 20/24] xl: introduce load/save_domain_config
2014-05-06 15:03 ` Wei Liu
@ 2014-05-07 10:12 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:12 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Tue, 2014-05-06 at 16:03 +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 03:28:12PM +0100, Ian Campbell wrote:
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > They are used to serialize / deserialize domain configuration to / from
> > > user data store.
> > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > > ---
> > > tools/libxl/xl_cmdimpl.c | 119 ++++++++++++++++++++++++++++++----------------
> > > 1 file changed, 77 insertions(+), 42 deletions(-)
> > >
> > > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > > index 7443d86..f891ab5 100644
> > > --- a/tools/libxl/xl_cmdimpl.c
> > > +++ b/tools/libxl/xl_cmdimpl.c
> > > @@ -160,6 +160,73 @@ struct domain_create {
> > > char **migration_domname_r; /* from malloc */
> > > };
> > >
> > > +static int _parse_config_data_json(char *config_data, int config_len,
> > > + libxl_domain_config *d_config)
> >
> > The _ namespace is reserved for something (either POSIX or the compiler,
> > but not us, for sure)
> >
>
> I will use _xl namespace instead.
Anything starting with _ is reserved. xl_foo would be fine, in the
context of a static function parse_config_data_json would also be fine.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 21/24] xl: update domain configuration when running mem-set and mem-max
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (19 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 20/24] xl: introduce load/save_domain_config Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 14:30 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 22/24] xl: update domain configuration when we hotplug a device Wei Liu
` (3 subsequent siblings)
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/xl_cmdimpl.c | 55 +++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 52 insertions(+), 3 deletions(-)
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index f891ab5..83f058e 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2745,6 +2745,7 @@ int main_memmax(int argc, char **argv)
int opt = 0;
char *mem;
int rc;
+ libxl_domain_config d_config;
SWITCH_FOREACH_OPT(opt, "", NULL, "mem-max", 2) {
/* No options */
@@ -2756,10 +2757,34 @@ int main_memmax(int argc, char **argv)
rc = set_memory_max(domid, mem);
if (rc) {
fprintf(stderr, "cannot set domid %d static max memory to : %s\n", domid, mem);
- return 1;
+ rc = 1;
+ goto out;
}
- return 0;
+ libxl_domain_config_init(&d_config);
+ if (load_domain_config(domid, &d_config)) {
+ fprintf(stderr, "cannot load domain configuration from userdata store, "
+ "not updating configuration\n");
+ rc = 1;
+ goto out_dispose;
+ }
+
+ /* parse_mem_size_kb cannot fail, otherwise we won't be here. */
+ d_config.b_info.max_memkb = parse_mem_size_kb(mem);
+
+ if (store_domain_config(domid, &d_config)) {
+ fprintf(stderr, "cannot store domain configuration from userdata store, "
+ "not updating configuration\n");
+ rc = 1;
+ goto out_dispose;
+ }
+
+ rc = 0;
+
+out_dispose:
+ libxl_domain_config_dispose(&d_config);
+out:
+ return rc;
}
static void set_memory_target(uint32_t domid, const char *mem)
@@ -2780,6 +2805,8 @@ int main_memset(int argc, char **argv)
uint32_t domid;
int opt = 0;
const char *mem;
+ libxl_domain_config d_config;
+ int rc = 1;
SWITCH_FOREACH_OPT(opt, "", NULL, "mem-set", 2) {
/* No options */
@@ -2789,7 +2816,29 @@ int main_memset(int argc, char **argv)
mem = argv[optind + 1];
set_memory_target(domid, mem);
- return 0;
+
+ libxl_domain_config_init(&d_config);
+ if (load_domain_config(domid, &d_config)) {
+ fprintf(stderr, "cannot load domain configuration from userdata store, "
+ "not updating configuration\n");
+ rc = 1;
+ goto out;
+ }
+
+ /* parse_mem_size_kb cannot fail, otherwise we won't be here. */
+ d_config.b_info.target_memkb = parse_mem_size_kb(mem);
+
+ if (store_domain_config(domid, &d_config)) {
+ fprintf(stderr, "cannot store domain configuration from userdata store, "
+ "not updating configuration\n");
+ rc = 1;
+ goto out;
+ }
+
+ rc = 0;
+out:
+ libxl_domain_config_dispose(&d_config);
+ return rc;
}
static int cd_insert(uint32_t domid, const char *virtdev, char *phys)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 21/24] xl: update domain configuration when running mem-set and mem-max
2014-05-01 12:58 ` [PATCH V4 21/24] xl: update domain configuration when running mem-set and mem-max Wei Liu
@ 2014-05-06 14:30 ` Ian Campbell
2014-05-06 14:56 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 14:30 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
Wouldn't doing this in set_memory_max mean you need to patch less places
and keep the places to change to a minimum by keeping this code "near
the action"?
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/xl_cmdimpl.c | 55 +++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 52 insertions(+), 3 deletions(-)
>
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index f891ab5..83f058e 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -2745,6 +2745,7 @@ int main_memmax(int argc, char **argv)
> int opt = 0;
> char *mem;
> int rc;
> + libxl_domain_config d_config;
>
> SWITCH_FOREACH_OPT(opt, "", NULL, "mem-max", 2) {
> /* No options */
> @@ -2756,10 +2757,34 @@ int main_memmax(int argc, char **argv)
> rc = set_memory_max(domid, mem);
> if (rc) {
> fprintf(stderr, "cannot set domid %d static max memory to : %s\n", domid, mem);
> - return 1;
> + rc = 1;
> + goto out;
> }
>
> - return 0;
> + libxl_domain_config_init(&d_config);
> + if (load_domain_config(domid, &d_config)) {
> + fprintf(stderr, "cannot load domain configuration from userdata store, "
> + "not updating configuration\n");
I expect this is going to get repeated a lot for the different loaders
-- so move it into the helper? Possibly make it exit() itself to avoid
needing to propagate too many errors.
Perhaps even a helper to load, call a callback and save?
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 21/24] xl: update domain configuration when running mem-set and mem-max
2014-05-06 14:30 ` Ian Campbell
@ 2014-05-06 14:56 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 14:56 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 03:30:56PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
>
> Wouldn't doing this in set_memory_max mean you need to patch less places
> and keep the places to change to a minimum by keeping this code "near
> the action"?
>
No, main_memset calls a different function.
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/xl_cmdimpl.c | 55 +++++++++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 52 insertions(+), 3 deletions(-)
> >
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index f891ab5..83f058e 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -2745,6 +2745,7 @@ int main_memmax(int argc, char **argv)
> > int opt = 0;
> > char *mem;
> > int rc;
> > + libxl_domain_config d_config;
> >
> > SWITCH_FOREACH_OPT(opt, "", NULL, "mem-max", 2) {
> > /* No options */
> > @@ -2756,10 +2757,34 @@ int main_memmax(int argc, char **argv)
> > rc = set_memory_max(domid, mem);
> > if (rc) {
> > fprintf(stderr, "cannot set domid %d static max memory to : %s\n", domid, mem);
> > - return 1;
> > + rc = 1;
> > + goto out;
> > }
> >
> > - return 0;
> > + libxl_domain_config_init(&d_config);
> > + if (load_domain_config(domid, &d_config)) {
> > + fprintf(stderr, "cannot load domain configuration from userdata store, "
> > + "not updating configuration\n");
>
> I expect this is going to get repeated a lot for the different loaders
> -- so move it into the helper? Possibly make it exit() itself to avoid
> needing to propagate too many errors.
>
> Perhaps even a helper to load, call a callback and save?
>
I've done this for devices with two macros, but forgot to do so for
memory targets. I plan to wrap the repeative in two macros in my next
version.
Wei.
> Ian.
>
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (20 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 21/24] xl: update domain configuration when running mem-set and mem-max Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 15:44 ` Ian Campbell
2014-05-06 16:57 ` David Vrabel
2014-05-01 12:58 ` [PATCH V4 23/24] libxl: consider force removal of device successful Wei Liu
` (2 subsequent siblings)
24 siblings, 2 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
A convenient macro is written to accomplish following tasks:
1. load domain configuration
2. allocate a new device structure NEW
3. copy the parsed device strcture PARSED to NEW
4. call libxl_device_TYPE_add(PARSED)
5. pull from PARSED any fields that might be touched by libxl to NEW
6. store domain configuration
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
1 file changed, 69 insertions(+), 16 deletions(-)
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 83f058e..91172c5 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
&(array)[array_extend_old_count]; \
})
+#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
+ ({ \
+ typeof((count)) array_extend_old_count = (count); \
+ (count)++; \
+ (array) = xrealloc((array), sizeof(*array) * (count)); \
+ (initfn)(&(array)[array_extend_old_count]); \
+ &(array)[array_extend_old_count]; \
+ })
+
+/* Add a device and update the stored configuration */
+#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
+ do { \
+ libxl_domain_config_init((d_config)); \
+ load_domain_config((domid), (d_config)); \
+ \
+ allocate; \
+ \
+ libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
+ \
+ if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
+ fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
+ exit(1); \
+ } \
+ \
+ maybe_fixup; \
+ \
+ store_domain_config((domid), (d_config)); \
+ libxl_domain_config_dispose((d_config)); \
+ } while (0)
+
#define LOG(_f, _a...) dolog(__FILE__, __LINE__, __func__, _f "\n", ##_a)
static void dolog(const char *file, int line, const char *func, char *fmt, ...)
@@ -3047,7 +3077,8 @@ int main_pcidetach(int argc, char **argv)
}
static void pciattach(uint32_t domid, const char *bdf, const char *vs)
{
- libxl_device_pci pcidev;
+ libxl_domain_config d_config;
+ libxl_device_pci pcidev, *ppcidev;
XLU_Config *config;
libxl_device_pci_init(&pcidev);
@@ -3059,7 +3090,12 @@ static void pciattach(uint32_t domid, const char *bdf, const char *vs)
fprintf(stderr, "pci-attach: malformed BDF specification \"%s\"\n", bdf);
exit(2);
}
- libxl_device_pci_add(ctx, domid, &pcidev, 0);
+
+ ADD_DEVICE(pci, domid, pcidev, &d_config, ppcidev, {
+ ppcidev = ARRAY_EXTEND_INIT_NODEVID(d_config.pcidevs,
+ d_config.num_pcidevs,
+ libxl_device_pci_init);
+ }, {});
libxl_device_pci_dispose(&pcidev);
xlu_cfg_destroy(config);
@@ -5959,7 +5995,8 @@ int main_networkattach(int argc, char **argv)
{
uint32_t domid;
int opt;
- libxl_device_nic nic;
+ libxl_domain_config d_config;
+ libxl_device_nic nic, *pnic;
XLU_Config *config = 0;
char *endptr, *oparg;
const char *tok;
@@ -6040,10 +6077,16 @@ int main_networkattach(int argc, char **argv)
return 0;
}
- if (libxl_device_nic_add(ctx, domid, &nic, 0)) {
- fprintf(stderr, "libxl_device_nic_add failed.\n");
- return 1;
- }
+ /* libxl allocates devid and may generate mac */
+ ADD_DEVICE(nic, domid, nic, &d_config, pnic, {
+ pnic = ARRAY_EXTEND_INIT(d_config.nics,
+ d_config.num_nics,
+ libxl_device_nic_init);
+ }, {
+ pnic->devid = nic.devid;
+ libxl_mac_copy(ctx, &pnic->mac, &nic.mac);
+ });
+
libxl_device_nic_dispose(&nic);
xlu_cfg_destroy(config);
return 0;
@@ -6123,7 +6166,8 @@ int main_blockattach(int argc, char **argv)
{
int opt;
uint32_t fe_domid;
- libxl_device_disk disk;
+ libxl_domain_config d_config;
+ libxl_device_disk disk, *pdisk;
XLU_Config *config = 0;
SWITCH_FOREACH_OPT(opt, "", NULL, "block-attach", 2) {
@@ -6147,9 +6191,12 @@ int main_blockattach(int argc, char **argv)
return 0;
}
- if (libxl_device_disk_add(ctx, fe_domid, &disk, 0)) {
- fprintf(stderr, "libxl_device_disk_add failed.\n");
- }
+ ADD_DEVICE(disk, fe_domid, disk, &d_config, pdisk, {
+ pdisk = ARRAY_EXTEND_INIT_NODEVID(d_config.disks,
+ d_config.num_disks,
+ libxl_device_disk_init);
+ }, {});
+
return 0;
}
@@ -6218,7 +6265,8 @@ int main_blockdetach(int argc, char **argv)
int main_vtpmattach(int argc, char **argv)
{
int opt;
- libxl_device_vtpm vtpm;
+ libxl_domain_config d_config;
+ libxl_device_vtpm vtpm, *pvtpm;
char *oparg;
uint32_t domid;
@@ -6256,10 +6304,15 @@ int main_vtpmattach(int argc, char **argv)
return 0;
}
- if (libxl_device_vtpm_add(ctx, domid, &vtpm, 0)) {
- fprintf(stderr, "libxl_device_vtpm_add failed.\n");
- return 1;
- }
+ /* libxl may generates UUID for vtpm */
+ ADD_DEVICE(vtpm, domid, vtpm, &d_config, pvtpm, {
+ pvtpm = ARRAY_EXTEND_INIT(d_config.vtpms,
+ d_config.num_vtpms,
+ libxl_device_vtpm_init);
+ }, {
+ libxl_uuid_copy(&pvtpm->uuid, &vtpm.uuid);
+ });
+
libxl_device_vtpm_dispose(&vtpm);
return 0;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-01 12:58 ` [PATCH V4 22/24] xl: update domain configuration when we hotplug a device Wei Liu
@ 2014-05-06 15:44 ` Ian Campbell
2014-05-06 15:58 ` Wei Liu
2014-05-06 16:57 ` David Vrabel
1 sibling, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 15:44 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> A convenient macro is written to accomplish following tasks:
"convenience macro" (a convenient macro isn't wrong, it's just not what
we usually say).
>
> 1. load domain configuration
> 2. allocate a new device structure NEW
> 3. copy the parsed device strcture PARSED to NEW
typo strcture.
> 4. call libxl_device_TYPE_add(PARSED)
> 5. pull from PARSED any fields that might be touched by libxl to NEW
> 6. store domain configuration
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
> 1 file changed, 69 insertions(+), 16 deletions(-)
>
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 83f058e..91172c5 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
> &(array)[array_extend_old_count]; \
> })
>
> +#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
> + ({ \
> + typeof((count)) array_extend_old_count = (count); \
> + (count)++; \
> + (array) = xrealloc((array), sizeof(*array) * (count)); \
> + (initfn)(&(array)[array_extend_old_count]); \
> + &(array)[array_extend_old_count]; \
> + })
> +
> +/* Add a device and update the stored configuration */
> +#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
> + do { \
> + libxl_domain_config_init((d_config)); \
> + load_domain_config((domid), (d_config)); \
> + \
> + allocate; \
> + \
> + libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
> + \
> + if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
> + fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
> + exit(1); \
> + } \
> + \
> + maybe_fixup; \
> + \
> + store_domain_config((domid), (d_config)); \
> + libxl_domain_config_dispose((d_config)); \
> + } while (0)
> +
> #define LOG(_f, _a...) dolog(__FILE__, __LINE__, __func__, _f "\n", ##_a)
>
> static void dolog(const char *file, int line, const char *func, char *fmt, ...)
> @@ -3047,7 +3077,8 @@ int main_pcidetach(int argc, char **argv)
> }
> static void pciattach(uint32_t domid, const char *bdf, const char *vs)
> {
> - libxl_device_pci pcidev;
> + libxl_domain_config d_config;
Is d_config needed outside the scope of the helper macros? If not then
perhaps push down?
> + libxl_device_pci pcidev, *ppcidev;
Likewise ppcidev here, perhaps meaning you would need to pass the type
rather then an instance to the variable.
> XLU_Config *config;
>
> libxl_device_pci_init(&pcidev);
> @@ -3059,7 +3090,12 @@ static void pciattach(uint32_t domid, const char *bdf, const char *vs)
> fprintf(stderr, "pci-attach: malformed BDF specification \"%s\"\n", bdf);
> exit(2);
> }
> - libxl_device_pci_add(ctx, domid, &pcidev, 0);
> +
> + ADD_DEVICE(pci, domid, pcidev, &d_config, ppcidev, {
> + ppcidev = ARRAY_EXTEND_INIT_NODEVID(d_config.pcidevs,
> + d_config.num_pcidevs,
> + libxl_device_pci_init);
Do you really need to do a deep copy of pcidev into the new array slot?
If you just memcpy the thing over and then do not dispose the original
then the copy will simply take over those allocations, which is fine I
think.
> + }, {});
>
> libxl_device_pci_dispose(&pcidev);
> xlu_cfg_destroy(config);
> @@ -5959,7 +5995,8 @@ int main_networkattach(int argc, char **argv)
> {
> uint32_t domid;
> int opt;
> - libxl_device_nic nic;
> + libxl_domain_config d_config;
> + libxl_device_nic nic, *pnic;
> XLU_Config *config = 0;
> char *endptr, *oparg;
> const char *tok;
> @@ -6040,10 +6077,16 @@ int main_networkattach(int argc, char **argv)
> return 0;
> }
>
> - if (libxl_device_nic_add(ctx, domid, &nic, 0)) {
> - fprintf(stderr, "libxl_device_nic_add failed.\n");
> - return 1;
> - }
> + /* libxl allocates devid and may generate mac */
> + ADD_DEVICE(nic, domid, nic, &d_config, pnic, {
> + pnic = ARRAY_EXTEND_INIT(d_config.nics,
> + d_config.num_nics,
> + libxl_device_nic_init);
> + }, {
> + pnic->devid = nic.devid;
> + libxl_mac_copy(ctx, &pnic->mac, &nic.mac);
Doesn't the copy handle this already? If not then why not?
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-06 15:44 ` Ian Campbell
@ 2014-05-06 15:58 ` Wei Liu
2014-05-07 10:16 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 15:58 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 04:44:56PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > A convenient macro is written to accomplish following tasks:
>
> "convenience macro" (a convenient macro isn't wrong, it's just not what
> we usually say).
>
Ack.
> >
> > 1. load domain configuration
> > 2. allocate a new device structure NEW
> > 3. copy the parsed device strcture PARSED to NEW
>
> typo strcture.
>
Thanks.
> > 4. call libxl_device_TYPE_add(PARSED)
> > 5. pull from PARSED any fields that might be touched by libxl to NEW
> > 6. store domain configuration
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
> > 1 file changed, 69 insertions(+), 16 deletions(-)
> >
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 83f058e..91172c5 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
> > &(array)[array_extend_old_count]; \
> > })
> >
> > +#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
> > + ({ \
> > + typeof((count)) array_extend_old_count = (count); \
> > + (count)++; \
> > + (array) = xrealloc((array), sizeof(*array) * (count)); \
> > + (initfn)(&(array)[array_extend_old_count]); \
> > + &(array)[array_extend_old_count]; \
> > + })
> > +
> > +/* Add a device and update the stored configuration */
> > +#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
> > + do { \
> > + libxl_domain_config_init((d_config)); \
> > + load_domain_config((domid), (d_config)); \
> > + \
> > + allocate; \
> > + \
> > + libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
> > + \
> > + if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
> > + fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
> > + exit(1); \
> > + } \
> > + \
> > + maybe_fixup; \
> > + \
> > + store_domain_config((domid), (d_config)); \
> > + libxl_domain_config_dispose((d_config)); \
> > + } while (0)
> > +
> > #define LOG(_f, _a...) dolog(__FILE__, __LINE__, __func__, _f "\n", ##_a)
> >
> > static void dolog(const char *file, int line, const char *func, char *fmt, ...)
> > @@ -3047,7 +3077,8 @@ int main_pcidetach(int argc, char **argv)
> > }
> > static void pciattach(uint32_t domid, const char *bdf, const char *vs)
> > {
> > - libxl_device_pci pcidev;
> > + libxl_domain_config d_config;
>
> Is d_config needed outside the scope of the helper macros? If not then
> perhaps push down?
>
The allocation and fixup part in the macro need to access fields in
d_config. It can be pushed down to the macro, but it's less obvious when
you access d_config in allocation and fixup code. I chose to be explicit
about this.
> > + libxl_device_pci pcidev, *ppcidev;
>
> Likewise ppcidev here, perhaps meaning you would need to pass the type
> rather then an instance to the variable.
>
> > XLU_Config *config;
> >
> > libxl_device_pci_init(&pcidev);
> > @@ -3059,7 +3090,12 @@ static void pciattach(uint32_t domid, const char *bdf, const char *vs)
> > fprintf(stderr, "pci-attach: malformed BDF specification \"%s\"\n", bdf);
> > exit(2);
> > }
> > - libxl_device_pci_add(ctx, domid, &pcidev, 0);
> > +
> > + ADD_DEVICE(pci, domid, pcidev, &d_config, ppcidev, {
> > + ppcidev = ARRAY_EXTEND_INIT_NODEVID(d_config.pcidevs,
> > + d_config.num_pcidevs,
> > + libxl_device_pci_init);
>
> Do you really need to do a deep copy of pcidev into the new array slot?
> If you just memcpy the thing over and then do not dispose the original
> then the copy will simply take over those allocations, which is fine I
> think.
>
The copy is done so that we know what user provides at the beginning,
i.e. this is the template that we need to keep. After the structure is
handed back by libxl the content might be altered already, say, all
"default" values replaced by libxl with some values that it deems
sensible. Then fixup will run to copy those bits we care about back to
the template.
> > + }, {});
> >
> > libxl_device_pci_dispose(&pcidev);
> > xlu_cfg_destroy(config);
> > @@ -5959,7 +5995,8 @@ int main_networkattach(int argc, char **argv)
> > {
> > uint32_t domid;
> > int opt;
> > - libxl_device_nic nic;
> > + libxl_domain_config d_config;
> > + libxl_device_nic nic, *pnic;
> > XLU_Config *config = 0;
> > char *endptr, *oparg;
> > const char *tok;
> > @@ -6040,10 +6077,16 @@ int main_networkattach(int argc, char **argv)
> > return 0;
> > }
> >
> > - if (libxl_device_nic_add(ctx, domid, &nic, 0)) {
> > - fprintf(stderr, "libxl_device_nic_add failed.\n");
> > - return 1;
> > - }
> > + /* libxl allocates devid and may generate mac */
> > + ADD_DEVICE(nic, domid, nic, &d_config, pnic, {
> > + pnic = ARRAY_EXTEND_INIT(d_config.nics,
> > + d_config.num_nics,
> > + libxl_device_nic_init);
> > + }, {
> > + pnic->devid = nic.devid;
> > + libxl_mac_copy(ctx, &pnic->mac, &nic.mac);
>
> Doesn't the copy handle this already? If not then why not?
>
This is the fixup. If user doesn't supply uuid, libxl generates one for
this device and we really want to save it in our state. If user supplies
before hand, it just copies the original value back.
The fixup is device dependent. You may find nic has different fixup and
disk has no fixup at all.
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-06 15:58 ` Wei Liu
@ 2014-05-07 10:16 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:16 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Tue, 2014-05-06 at 16:58 +0100, Wei Liu wrote:
> > > + libxl_device_pci pcidev, *ppcidev;
> >
> > Likewise ppcidev here, perhaps meaning you would need to pass the type
> > rather then an instance to the variable.
> >
> > > XLU_Config *config;
> > >
> > > libxl_device_pci_init(&pcidev);
> > > @@ -3059,7 +3090,12 @@ static void pciattach(uint32_t domid, const char *bdf, const char *vs)
> > > fprintf(stderr, "pci-attach: malformed BDF specification \"%s\"\n", bdf);
> > > exit(2);
> > > }
> > > - libxl_device_pci_add(ctx, domid, &pcidev, 0);
> > > +
> > > + ADD_DEVICE(pci, domid, pcidev, &d_config, ppcidev, {
> > > + ppcidev = ARRAY_EXTEND_INIT_NODEVID(d_config.pcidevs,
> > > + d_config.num_pcidevs,
> > > + libxl_device_pci_init);
> >
> > Do you really need to do a deep copy of pcidev into the new array slot?
> > If you just memcpy the thing over and then do not dispose the original
> > then the copy will simply take over those allocations, which is fine I
> > think.
> >
>
> The copy is done so that we know what user provides at the beginning,
> i.e. this is the template that we need to keep. After the structure is
> handed back by libxl the content might be altered already, say, all
> "default" values replaced by libxl with some values that it deems
> sensible. Then fixup will run to copy those bits we care about back to
> the template.
I see. (A shame, avoiding this deep copy stuff would have simplified all
sorts of things!)
BTW I think it's not so much the replacement of fields but the freeing
of the old values which prevents the shallow copy.
> > > + libxl_mac_copy(ctx, &pnic->mac, &nic.mac);
> >
> > Doesn't the copy handle this already? If not then why not?
> >
>
> This is the fixup. If user doesn't supply uuid, libxl generates one for
> this device and we really want to save it in our state. If user supplies
> before hand, it just copies the original value back.
>
> The fixup is device dependent. You may find nic has different fixup and
> disk has no fixup at all.
I see. eventually we might want to provide this fixup as a libxl helper
function, but having it here for now will let us bed it in first.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-01 12:58 ` [PATCH V4 22/24] xl: update domain configuration when we hotplug a device Wei Liu
2014-05-06 15:44 ` Ian Campbell
@ 2014-05-06 16:57 ` David Vrabel
2014-05-06 17:13 ` Wei Liu
1 sibling, 1 reply; 120+ messages in thread
From: David Vrabel @ 2014-05-06 16:57 UTC (permalink / raw)
To: Wei Liu, xen-devel; +Cc: ian.jackson, ian.campbell
On 01/05/14 13:58, Wei Liu wrote:
> A convenient macro is written to accomplish following tasks:
>
> 1. load domain configuration
> 2. allocate a new device structure NEW
> 3. copy the parsed device strcture PARSED to NEW
> 4. call libxl_device_TYPE_add(PARSED)
> 5. pull from PARSED any fields that might be touched by libxl to NEW
> 6. store domain configuration
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
> 1 file changed, 69 insertions(+), 16 deletions(-)
>
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 83f058e..91172c5 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
> &(array)[array_extend_old_count]; \
> })
>
> +#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
> + ({ \
> + typeof((count)) array_extend_old_count = (count); \
> + (count)++; \
> + (array) = xrealloc((array), sizeof(*array) * (count)); \
> + (initfn)(&(array)[array_extend_old_count]); \
> + &(array)[array_extend_old_count]; \
> + })
> +
> +/* Add a device and update the stored configuration */
> +#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
> + do { \
> + libxl_domain_config_init((d_config)); \
> + load_domain_config((domid), (d_config)); \
> + \
> + allocate; \
> + \
> + libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
> + \
> + if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
> + fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
> + exit(1); \
> + } \
> + \
> + maybe_fixup; \
> + \
> + store_domain_config((domid), (d_config)); \
> + libxl_domain_config_dispose((d_config)); \
> + } while (0)
> +
Is there a better way of doing this without these obtuse macros?
Is libxl lacking is a proper device abstraction layer?
David
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-06 16:57 ` David Vrabel
@ 2014-05-06 17:13 ` Wei Liu
2014-05-07 10:21 ` David Vrabel
2014-05-07 10:59 ` Andrew Cooper
0 siblings, 2 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-06 17:13 UTC (permalink / raw)
To: David Vrabel; +Cc: ian.jackson, Wei Liu, ian.campbell, xen-devel
On Tue, May 06, 2014 at 05:57:34PM +0100, David Vrabel wrote:
> On 01/05/14 13:58, Wei Liu wrote:
> > A convenient macro is written to accomplish following tasks:
> >
> > 1. load domain configuration
> > 2. allocate a new device structure NEW
> > 3. copy the parsed device strcture PARSED to NEW
> > 4. call libxl_device_TYPE_add(PARSED)
> > 5. pull from PARSED any fields that might be touched by libxl to NEW
> > 6. store domain configuration
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
> > 1 file changed, 69 insertions(+), 16 deletions(-)
> >
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 83f058e..91172c5 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
> > &(array)[array_extend_old_count]; \
> > })
> >
> > +#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
> > + ({ \
> > + typeof((count)) array_extend_old_count = (count); \
> > + (count)++; \
> > + (array) = xrealloc((array), sizeof(*array) * (count)); \
> > + (initfn)(&(array)[array_extend_old_count]); \
> > + &(array)[array_extend_old_count]; \
> > + })
> > +
> > +/* Add a device and update the stored configuration */
> > +#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
> > + do { \
> > + libxl_domain_config_init((d_config)); \
> > + load_domain_config((domid), (d_config)); \
> > + \
> > + allocate; \
> > + \
> > + libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
> > + \
> > + if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
> > + fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
> > + exit(1); \
> > + } \
> > + \
> > + maybe_fixup; \
> > + \
> > + store_domain_config((domid), (d_config)); \
> > + libxl_domain_config_dispose((d_config)); \
> > + } while (0)
> > +
>
> Is there a better way of doing this without these obtuse macros?
>
Do you not like macro? Or do you not like this be done in xl?
> Is libxl lacking is a proper device abstraction layer?
>
libxl does have a device abstraction layer, but it cannot help much
here. Not quite sure what "proper" means.
It's done like this (regardless putting this macro in xl or libxl)
because the library device abstraction layer (for now) by itself doesn't
know what to preserve, what information is critical to restore a domain.
Wei.
> David
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-06 17:13 ` Wei Liu
@ 2014-05-07 10:21 ` David Vrabel
2014-05-07 10:42 ` Ian Campbell
2014-05-07 10:59 ` Andrew Cooper
1 sibling, 1 reply; 120+ messages in thread
From: David Vrabel @ 2014-05-07 10:21 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, ian.campbell, xen-devel
On 06/05/14 18:13, Wei Liu wrote:
> On Tue, May 06, 2014 at 05:57:34PM +0100, David Vrabel wrote:
>> On 01/05/14 13:58, Wei Liu wrote:
>>> A convenient macro is written to accomplish following tasks:
>>>
>>> 1. load domain configuration
>>> 2. allocate a new device structure NEW
>>> 3. copy the parsed device strcture PARSED to NEW
>>> 4. call libxl_device_TYPE_add(PARSED)
>>> 5. pull from PARSED any fields that might be touched by libxl to NEW
>>> 6. store domain configuration
>>>
>>> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>>> ---
>>> tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
>>> 1 file changed, 69 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
>>> index 83f058e..91172c5 100644
>>> --- a/tools/libxl/xl_cmdimpl.c
>>> +++ b/tools/libxl/xl_cmdimpl.c
>>> @@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
>>> &(array)[array_extend_old_count]; \
>>> })
>>>
>>> +#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
>>> + ({ \
>>> + typeof((count)) array_extend_old_count = (count); \
>>> + (count)++; \
>>> + (array) = xrealloc((array), sizeof(*array) * (count)); \
>>> + (initfn)(&(array)[array_extend_old_count]); \
>>> + &(array)[array_extend_old_count]; \
>>> + })
>>> +
>>> +/* Add a device and update the stored configuration */
>>> +#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
>>> + do { \
>>> + libxl_domain_config_init((d_config)); \
>>> + load_domain_config((domid), (d_config)); \
>>> + \
>>> + allocate; \
>>> + \
>>> + libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
>>> + \
>>> + if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
>>> + fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
>>> + exit(1); \
>>> + } \
>>> + \
>>> + maybe_fixup; \
>>> + \
>>> + store_domain_config((domid), (d_config)); \
>>> + libxl_domain_config_dispose((d_config)); \
>>> + } while (0)
>>> +
>>
>> Is there a better way of doing this without these obtuse macros?
>>
>
> Do you not like macro? Or do you not like this be done in xl?
I don't like macros like this. They make the code difficult to read and
debug.
>> Is libxl lacking is a proper device abstraction layer?
>
> libxl does have a device abstraction layer, but it cannot help much
> here. Not quite sure what "proper" means.
At a minimum it looks like a libxl__device needs copy and add methods.
This level of refactoring is probably out-of-scope of this patch series
though.
David
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-07 10:21 ` David Vrabel
@ 2014-05-07 10:42 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:42 UTC (permalink / raw)
To: David Vrabel; +Cc: ian.jackson, Wei Liu, xen-devel
On Wed, 2014-05-07 at 11:21 +0100, David Vrabel wrote:
> >> Is libxl lacking is a proper device abstraction layer?
> >
> > libxl does have a device abstraction layer, but it cannot help much
> > here. Not quite sure what "proper" means.
>
> At a minimum it looks like a libxl__device needs copy and add methods.
It already has both, and this macro is using them. The real need for the
macro here is C's lack of polymorphism (and/or libxl's failure to
implement such a thing on top of C).
Some of this might be alleviated by pushing the saving and updating of
the stored config down into libxl as I suggested in anther reply this
morning.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 22/24] xl: update domain configuration when we hotplug a device
2014-05-06 17:13 ` Wei Liu
2014-05-07 10:21 ` David Vrabel
@ 2014-05-07 10:59 ` Andrew Cooper
1 sibling, 0 replies; 120+ messages in thread
From: Andrew Cooper @ 2014-05-07 10:59 UTC (permalink / raw)
To: Wei Liu; +Cc: xen-devel, ian.jackson, David Vrabel, ian.campbell
On 06/05/14 18:13, Wei Liu wrote:
> On Tue, May 06, 2014 at 05:57:34PM +0100, David Vrabel wrote:
>> On 01/05/14 13:58, Wei Liu wrote:
>>> A convenient macro is written to accomplish following tasks:
>>>
>>> 1. load domain configuration
>>> 2. allocate a new device structure NEW
>>> 3. copy the parsed device strcture PARSED to NEW
>>> 4. call libxl_device_TYPE_add(PARSED)
>>> 5. pull from PARSED any fields that might be touched by libxl to NEW
>>> 6. store domain configuration
>>>
>>> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
>>> ---
>>> tools/libxl/xl_cmdimpl.c | 85 +++++++++++++++++++++++++++++++++++++---------
>>> 1 file changed, 69 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
>>> index 83f058e..91172c5 100644
>>> --- a/tools/libxl/xl_cmdimpl.c
>>> +++ b/tools/libxl/xl_cmdimpl.c
>>> @@ -381,6 +381,36 @@ static void *xrealloc(void *ptr, size_t sz) {
>>> &(array)[array_extend_old_count]; \
>>> })
>>>
>>> +#define ARRAY_EXTEND_INIT_NODEVID(array,count,initfn) \
>>> + ({ \
>>> + typeof((count)) array_extend_old_count = (count); \
>>> + (count)++; \
>>> + (array) = xrealloc((array), sizeof(*array) * (count)); \
>>> + (initfn)(&(array)[array_extend_old_count]); \
>>> + &(array)[array_extend_old_count]; \
>>> + })
>>> +
>>> +/* Add a device and update the stored configuration */
>>> +#define ADD_DEVICE(devtype,domid,dev,d_config,ptr,allocate,maybe_fixup) \
>>> + do { \
>>> + libxl_domain_config_init((d_config)); \
>>> + load_domain_config((domid), (d_config)); \
>>> + \
>>> + allocate; \
>>> + \
>>> + libxl_device_ ## devtype ## _copy(ctx,(ptr),&(dev)); \
>>> + \
>>> + if (libxl_device_ ## devtype ## _add(ctx,(domid),&(dev), 0)) { \
>>> + fprintf(stderr, "libxl_device_%s_add failed.\n", #devtype); \
>>> + exit(1); \
>>> + } \
>>> + \
>>> + maybe_fixup; \
>>> + \
>>> + store_domain_config((domid), (d_config)); \
>>> + libxl_domain_config_dispose((d_config)); \
>>> + } while (0)
>>> +
>> Is there a better way of doing this without these obtuse macros?
>>
> Do you not like macro?
Macros like this mean that you are no longer programming C. You are
emulating something akin to m4, but without a sane preprocessor to help you.
~Andrew
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 23/24] libxl: consider force removal of device successful
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (21 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 22/24] xl: update domain configuration when we hotplug a device Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 15:48 ` Ian Campbell
2014-05-01 12:58 ` [PATCH V4 24/24] xl: update configuration when we unplug a device Wei Liu
2014-05-01 13:02 ` [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
If a device is force removed and the force removal successes, from
guest's PoV this device is gone. So the toolstack should consider this a
successful case as well.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_device.c | 12 ++++++++++--
tools/libxl/libxl_internal.h | 4 ++++
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
index fa99f77..7a37778 100644
--- a/tools/libxl/libxl_device.c
+++ b/tools/libxl/libxl_device.c
@@ -845,6 +845,11 @@ void libxl__initiate_device_remove(libxl__egc *egc,
if (rc < 0) goto out;
}
+ /* At this point the XS transaction is commited. So check if we
+ * force removal of the device.
+ */
+ aodev->force_removed = aodev->force;
+
rc = libxl__ev_devstate_wait(gc, &aodev->backend_ds,
device_backend_callback,
state_path, XenbusStateClosed,
@@ -928,7 +933,7 @@ static void device_backend_callback(libxl__egc *egc, libxl__ev_devstate *ds,
return;
}
- if (rc) {
+ if (rc && !aodev->force_removed) {
LOG(ERROR, "unable to %s device with path %s",
libxl__device_action_to_string(aodev->action),
libxl__device_backend_path(gc, aodev->dev));
@@ -939,7 +944,10 @@ static void device_backend_callback(libxl__egc *egc, libxl__ev_devstate *ds,
return;
out:
- aodev->rc = rc;
+ /* If force removal is successful, the device is gone from guest's
+ * PoV. The toolstack should consider it successful too.
+ */
+ aodev->rc = aodev->force_removed ? 0 : rc;
device_hotplug_done(egc, aodev);
return;
}
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 6e3d19d..cf9af6b 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -2078,6 +2078,10 @@ struct libxl__ao_device {
libxl__device_action action;
libxl__device *dev;
int force;
+ /* If force removal is successful, force_removed=1. The device is
+ * gone from guest's PoV
+ */
+ int force_removed;
libxl__device_callback *callback;
/* return value, zeroed by user on entry, is valid on callback */
int rc;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 23/24] libxl: consider force removal of device successful
2014-05-01 12:58 ` [PATCH V4 23/24] libxl: consider force removal of device successful Wei Liu
@ 2014-05-06 15:48 ` Ian Campbell
2014-05-06 16:04 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 15:48 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> If a device is force removed and the force removal successes, from
"succeeds".
> guest's PoV this device is gone. So the toolstack should consider this a
> successful case as well.
This seems rather self evident (consider success as success), were you
perhaps trying to say something opposite, which is that if a force
remove fails for some reason that we should (e.g. for the purposes of a
later migration etc) consider the device to have actually been removed?
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/libxl_device.c | 12 ++++++++++--
> tools/libxl/libxl_internal.h | 4 ++++
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> index fa99f77..7a37778 100644
> --- a/tools/libxl/libxl_device.c
> +++ b/tools/libxl/libxl_device.c
> @@ -845,6 +845,11 @@ void libxl__initiate_device_remove(libxl__egc *egc,
> if (rc < 0) goto out;
> }
>
> + /* At this point the XS transaction is commited. So check if we
> + * force removal of the device.
"check if we were force removing the device". I think committed has two
t-s too (although I confess I'm not 100% sure...).
> + */
> + aodev->force_removed = aodev->force;
How and when can aodev->force_removed be distinct from aodev->force?
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 23/24] libxl: consider force removal of device successful
2014-05-06 15:48 ` Ian Campbell
@ 2014-05-06 16:04 ` Wei Liu
2014-05-07 10:18 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 16:04 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 04:48:40PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > If a device is force removed and the force removal successes, from
>
> "succeeds".
>
> > guest's PoV this device is gone. So the toolstack should consider this a
> > successful case as well.
>
> This seems rather self evident (consider success as success), were you
> perhaps trying to say something opposite, which is that if a force
> remove fails for some reason that we should (e.g. for the purposes of a
> later migration etc) consider the device to have actually been removed?
>
No.
The commit message is written like this due to the current behavior.
If you remove a device, and the backend times out, libxl will try to
force-remove (destroy) this device. Then it returns "fail to remove a
device" even if the force removal is successful.
This is rather confusing because libxl tells you the removal fails, but
actually the device is long gone from guest's PoV.
>
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/libxl_device.c | 12 ++++++++++--
> > tools/libxl/libxl_internal.h | 4 ++++
> > 2 files changed, 14 insertions(+), 2 deletions(-)
> >
> > diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> > index fa99f77..7a37778 100644
> > --- a/tools/libxl/libxl_device.c
> > +++ b/tools/libxl/libxl_device.c
> > @@ -845,6 +845,11 @@ void libxl__initiate_device_remove(libxl__egc *egc,
> > if (rc < 0) goto out;
> > }
> >
> > + /* At this point the XS transaction is commited. So check if we
> > + * force removal of the device.
>
> "check if we were force removing the device". I think committed has two
> t-s too (although I confess I'm not 100% sure...).
>
You're right. I have lots of typos in this series. :-(
> > + */
> > + aodev->force_removed = aodev->force;
>
> How and when can aodev->force_removed be distinct from aodev->force?
>
The removal of xenstore entries can fail. So we are only sure if the
device is force removed if the xenstore transaction is committed.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 23/24] libxl: consider force removal of device successful
2014-05-06 16:04 ` Wei Liu
@ 2014-05-07 10:18 ` Ian Campbell
2014-05-07 13:09 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:18 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Tue, 2014-05-06 at 17:04 +0100, Wei Liu wrote:
> On Tue, May 06, 2014 at 04:48:40PM +0100, Ian Campbell wrote:
> > On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > > If a device is force removed and the force removal successes, from
> >
> > "succeeds".
> >
> > > guest's PoV this device is gone. So the toolstack should consider this a
> > > successful case as well.
> >
> > This seems rather self evident (consider success as success), were you
> > perhaps trying to say something opposite, which is that if a force
> > remove fails for some reason that we should (e.g. for the purposes of a
> > later migration etc) consider the device to have actually been removed?
> >
>
> No.
>
> The commit message is written like this due to the current behavior.
>
> If you remove a device, and the backend times out, libxl will try to
> force-remove (destroy) this device. Then it returns "fail to remove a
> device" even if the force removal is successful.
>
> This is rather confusing because libxl tells you the removal fails, but
> actually the device is long gone from guest's PoV.
I see. I'm not sure how sensible I think this automatic fallback to
forcing is, but given that is the behaviour I think your change does
make sense. I'd be in favour of pasting most of your explanation above
into the commit message though and/or comments.
> > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > > ---
> > > tools/libxl/libxl_device.c | 12 ++++++++++--
> > > tools/libxl/libxl_internal.h | 4 ++++
> > > 2 files changed, 14 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> > > index fa99f77..7a37778 100644
> > > --- a/tools/libxl/libxl_device.c
> > > +++ b/tools/libxl/libxl_device.c
> > > @@ -845,6 +845,11 @@ void libxl__initiate_device_remove(libxl__egc *egc,
> > > if (rc < 0) goto out;
> > > }
> > >
> > > + /* At this point the XS transaction is commited. So check if we
> > > + * force removal of the device.
> >
> > "check if we were force removing the device". I think committed has two
> > t-s too (although I confess I'm not 100% sure...).
> >
>
> You're right. I have lots of typos in this series. :-(
FWIW I only spot other people's, not my own ;-)
>
> > > + */
> > > + aodev->force_removed = aodev->force;
> >
> > How and when can aodev->force_removed be distinct from aodev->force?
> >
>
> The removal of xenstore entries can fail. So we are only sure if the
> device is force removed if the xenstore transaction is committed.
If the xenstore commit fails the force_removed and force can differ?
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 23/24] libxl: consider force removal of device successful
2014-05-07 10:18 ` Ian Campbell
@ 2014-05-07 13:09 ` Wei Liu
0 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-07 13:09 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Wed, May 07, 2014 at 11:18:51AM +0100, Ian Campbell wrote:
[...]
> > The commit message is written like this due to the current behavior.
> >
> > If you remove a device, and the backend times out, libxl will try to
> > force-remove (destroy) this device. Then it returns "fail to remove a
> > device" even if the force removal is successful.
> >
> > This is rather confusing because libxl tells you the removal fails, but
> > actually the device is long gone from guest's PoV.
>
> I see. I'm not sure how sensible I think this automatic fallback to
> forcing is, but given that is the behaviour I think your change does
> make sense. I'd be in favour of pasting most of your explanation above
> into the commit message though and/or comments.
>
NP.
> > > > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > > > ---
> > > > tools/libxl/libxl_device.c | 12 ++++++++++--
> > > > tools/libxl/libxl_internal.h | 4 ++++
> > > > 2 files changed, 14 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> > > > index fa99f77..7a37778 100644
> > > > --- a/tools/libxl/libxl_device.c
> > > > +++ b/tools/libxl/libxl_device.c
> > > > @@ -845,6 +845,11 @@ void libxl__initiate_device_remove(libxl__egc *egc,
> > > > if (rc < 0) goto out;
> > > > }
> > > >
> > > > + /* At this point the XS transaction is commited. So check if we
> > > > + * force removal of the device.
> > >
> > > "check if we were force removing the device". I think committed has two
> > > t-s too (although I confess I'm not 100% sure...).
> > >
> >
> > You're right. I have lots of typos in this series. :-(
>
> FWIW I only spot other people's, not my own ;-)
>
> >
> > > > + */
> > > > + aodev->force_removed = aodev->force;
> > >
> > > How and when can aodev->force_removed be distinct from aodev->force?
> > >
> >
> > The removal of xenstore entries can fail. So we are only sure if the
> > device is force removed if the xenstore transaction is committed.
>
> If the xenstore commit fails the force_removed and force can differ?
>
If xenstore transaction fails (commit failure being one of them), AO
ends with error code from the failed operation. In that case the
change is not committed and device entries are still there.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* [PATCH V4 24/24] xl: update configuration when we unplug a device
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (22 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 23/24] libxl: consider force removal of device successful Wei Liu
@ 2014-05-01 12:58 ` Wei Liu
2014-05-06 15:55 ` Ian Campbell
2014-05-01 13:02 ` [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
24 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-01 12:58 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Some macros are written to accomplish following tasks:
1. load domain configuration
2. allocate a new array of devices
3. copy the devices that are to be remain in configuration
4. replace pointer in libxl_domain_config
5. store domain configuration
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/xl_cmdimpl.c | 83 ++++++++++++++++++++++++++++++++++++++--------
1 file changed, 69 insertions(+), 14 deletions(-)
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 91172c5..ae3df6e 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -411,6 +411,59 @@ static void *xrealloc(void *ptr, size_t sz) {
libxl_domain_config_dispose((d_config)); \
} while (0)
+#define COMPARE_DEVID(a, b) ((a)->devid == (b)->devid)
+#define COMPARE_DISK(a, b) (!strcmp((a)->vdev, (b)->vdev))
+#define COMPARE_PCI(a, b) ((a)->bus == (b)->bus && \
+ (a)->dev == (b)->dev && \
+ (a)->func == (b)->func)
+
+/* Remove / destroy a device and update the stored configuration */
+#define REMOVEDESTROY_DEVICE(devtype,domid,dev,ptr,cnt, \
+ compare,removedestroy) \
+ do { \
+ libxl_domain_config d_config; \
+ libxl_device_ ## devtype *p = NULL, *x; \
+ int num; \
+ int j, k; \
+ \
+ libxl_domain_config_init(&d_config); \
+ load_domain_config((domid), &d_config); \
+ \
+ k = 0; \
+ for (j = 0; j < d_config.cnt; j++) { \
+ x = d_config.ptr + j; \
+ if (compare(x, &(dev))) \
+ k++; \
+ } \
+ \
+ num = d_config.cnt - k; \
+ \
+ p = xrealloc(p, sizeof(*p) * num); \
+ \
+ k = 0; \
+ for (j = 0; j < d_config.cnt; j++) { \
+ x = d_config.ptr + j; \
+ if (compare(x, &(dev))) \
+ continue; \
+ libxl_device_ ## devtype ## _copy(ctx, &p[k], \
+ d_config.ptr + j); \
+ k++; \
+ } \
+ \
+ if (libxl_device_ ## devtype ## _ ## removedestroy(ctx,(domid), \
+ &(dev), 0)) { \
+ fprintf(stderr, "libxl_device_%s_remove failed.\n", #devtype); \
+ exit(1); \
+ } \
+ \
+ free(d_config.ptr); \
+ d_config.ptr = p; \
+ d_config.cnt = num; \
+ \
+ store_domain_config((domid), &d_config); \
+ libxl_domain_config_dispose(&d_config); \
+ } while (0)
+
#define LOG(_f, _a...) dolog(__FILE__, __LINE__, __func__, _f "\n", ##_a)
static void dolog(const char *file, int line, const char *func, char *fmt, ...)
@@ -3048,9 +3101,11 @@ static void pcidetach(uint32_t domid, const char *bdf, int force)
exit(2);
}
if (force)
- libxl_device_pci_destroy(ctx, domid, &pcidev, 0);
+ REMOVEDESTROY_DEVICE(pci, domid, pcidev, pcidevs,
+ num_pcidevs, COMPARE_PCI, destroy);
else
- libxl_device_pci_remove(ctx, domid, &pcidev, 0);
+ REMOVEDESTROY_DEVICE(pci, domid, pcidev, pcidevs,
+ num_pcidevs, COMPARE_PCI, remove);
libxl_device_pci_dispose(&pcidev);
xlu_cfg_destroy(config);
@@ -6154,10 +6209,10 @@ int main_networkdetach(int argc, char **argv)
return 1;
}
}
- if (libxl_device_nic_remove(ctx, domid, &nic, 0)) {
- fprintf(stderr, "libxl_device_nic_del failed.\n");
- return 1;
- }
+
+ REMOVEDESTROY_DEVICE(nic, domid, nic, nics, num_nics,
+ COMPARE_DEVID, remove);
+
libxl_device_nic_dispose(&nic);
return 0;
}
@@ -6254,10 +6309,10 @@ int main_blockdetach(int argc, char **argv)
fprintf(stderr, "Error: Device %s not connected.\n", argv[optind+1]);
return 1;
}
- rc = libxl_device_disk_remove(ctx, domid, &disk, 0);
- if (rc) {
- fprintf(stderr, "libxl_device_disk_remove failed.\n");
- }
+
+ REMOVEDESTROY_DEVICE(disk, domid, disk, disks, num_disks,
+ COMPARE_DISK, remove);
+
libxl_device_disk_dispose(&disk);
return rc;
}
@@ -6382,10 +6437,10 @@ int main_vtpmdetach(int argc, char **argv)
return 1;
}
}
- rc = libxl_device_vtpm_remove(ctx, domid, &vtpm, 0);
- if (rc) {
- fprintf(stderr, "libxl_device_vtpm_remove failed.\n");
- }
+
+ REMOVEDESTROY_DEVICE(vtpm, domid, vtpm, vtpms, num_vtpms,
+ COMPARE_DEVID, remove);
+
libxl_device_vtpm_dispose(&vtpm);
return rc;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 120+ messages in thread* Re: [PATCH V4 24/24] xl: update configuration when we unplug a device
2014-05-01 12:58 ` [PATCH V4 24/24] xl: update configuration when we unplug a device Wei Liu
@ 2014-05-06 15:55 ` Ian Campbell
2014-05-06 16:12 ` Wei Liu
0 siblings, 1 reply; 120+ messages in thread
From: Ian Campbell @ 2014-05-06 15:55 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> Some macros are written to accomplish following tasks:
> 1. load domain configuration
> 2. allocate a new array of devices
> 3. copy the devices that are to be remain in configuration
s/are to be// and s/in/into/.
> 4. replace pointer in libxl_domain_config
> 5. store domain configuration
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/xl_cmdimpl.c | 83 ++++++++++++++++++++++++++++++++++++++--------
> 1 file changed, 69 insertions(+), 14 deletions(-)
>
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 91172c5..ae3df6e 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -411,6 +411,59 @@ static void *xrealloc(void *ptr, size_t sz) {
> libxl_domain_config_dispose((d_config)); \
> } while (0)
>
> +#define COMPARE_DEVID(a, b) ((a)->devid == (b)->devid)
> +#define COMPARE_DISK(a, b) (!strcmp((a)->vdev, (b)->vdev))
> +#define COMPARE_PCI(a, b) ((a)->bus == (b)->bus && \
> + (a)->dev == (b)->dev && \
> + (a)->func == (b)->func)
> +
> +/* Remove / destroy a device and update the stored configuration */
> +#define REMOVEDESTROY_DEVICE(devtype,domid,dev,ptr,cnt, \
> + compare,removedestroy) \
You've been talking to Ian J haven't you :-P
> + do { \
> + libxl_domain_config d_config; \
> + libxl_device_ ## devtype *p = NULL, *x; \
> + int num; \
> + int j, k; \
> + \
> + libxl_domain_config_init(&d_config); \
> + load_domain_config((domid), &d_config); \
> + \
> + k = 0; \
> + for (j = 0; j < d_config.cnt; j++) { \
> + x = d_config.ptr + j; \
> + if (compare(x, &(dev))) \
Are you concerned with the possibility that two entries in the array
might match dev? Wouldn't that equate to e.g. two xvda devices? Should
we not reject such things elsewhere?
> + k++; \
> + } \
> + \
> + num = d_config.cnt - k; \
> + \
> + p = xrealloc(p, sizeof(*p) * num); \
Why realloc, isn't the input p always NULL? (turning it into xmalloc).
But couldn't this be done in place, with a little care?
for(i = j = 0; i < d_config.cnt; i++)
if (!compare(d_config.ptr[i], dev)
d_config.ptr[j] = d_config.ptr[i] /* XXX optimise/skip if j == i */
j++
(note that j <= i is always true)
then realloc the array down to only j entries.
> + \
> + k = 0; \
> + for (j = 0; j < d_config.cnt; j++) { \
> + x = d_config.ptr + j; \
> + if (compare(x, &(dev))) \
> + continue; \
> + libxl_device_ ## devtype ## _copy(ctx, &p[k], \
> + d_config.ptr + j); \
Same comment as the previous patch re the actual need for a deep copy.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 24/24] xl: update configuration when we unplug a device
2014-05-06 15:55 ` Ian Campbell
@ 2014-05-06 16:12 ` Wei Liu
2014-05-07 10:20 ` Ian Campbell
0 siblings, 1 reply; 120+ messages in thread
From: Wei Liu @ 2014-05-06 16:12 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, Wei Liu, xen-devel
On Tue, May 06, 2014 at 04:55:33PM +0100, Ian Campbell wrote:
> On Thu, 2014-05-01 at 13:58 +0100, Wei Liu wrote:
> > Some macros are written to accomplish following tasks:
> > 1. load domain configuration
> > 2. allocate a new array of devices
> > 3. copy the devices that are to be remain in configuration
>
> s/are to be// and s/in/into/.
>
> > 4. replace pointer in libxl_domain_config
> > 5. store domain configuration
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/xl_cmdimpl.c | 83 ++++++++++++++++++++++++++++++++++++++--------
> > 1 file changed, 69 insertions(+), 14 deletions(-)
> >
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> > index 91172c5..ae3df6e 100644
> > --- a/tools/libxl/xl_cmdimpl.c
> > +++ b/tools/libxl/xl_cmdimpl.c
> > @@ -411,6 +411,59 @@ static void *xrealloc(void *ptr, size_t sz) {
> > libxl_domain_config_dispose((d_config)); \
> > } while (0)
> >
> > +#define COMPARE_DEVID(a, b) ((a)->devid == (b)->devid)
> > +#define COMPARE_DISK(a, b) (!strcmp((a)->vdev, (b)->vdev))
> > +#define COMPARE_PCI(a, b) ((a)->bus == (b)->bus && \
> > + (a)->dev == (b)->dev && \
> > + (a)->func == (b)->func)
> > +
> > +/* Remove / destroy a device and update the stored configuration */
> > +#define REMOVEDESTROY_DEVICE(devtype,domid,dev,ptr,cnt, \
> > + compare,removedestroy) \
>
> You've been talking to Ian J haven't you :-P
>
Many times. :-)
> > + do { \
> > + libxl_domain_config d_config; \
> > + libxl_device_ ## devtype *p = NULL, *x; \
> > + int num; \
> > + int j, k; \
> > + \
> > + libxl_domain_config_init(&d_config); \
> > + load_domain_config((domid), &d_config); \
> > + \
> > + k = 0; \
> > + for (j = 0; j < d_config.cnt; j++) { \
> > + x = d_config.ptr + j; \
> > + if (compare(x, &(dev))) \
>
> Are you concerned with the possibility that two entries in the array
> might match dev? Wouldn't that equate to e.g. two xvda devices? Should
> we not reject such things elsewhere?
>
Yes. And it's not just concern, I've seen this already!
The current xl block-attach doesn't complain if you add same CD images
twice.
> > + k++; \
> > + } \
> > + \
> > + num = d_config.cnt - k; \
> > + \
> > + p = xrealloc(p, sizeof(*p) * num); \
>
> Why realloc, isn't the input p always NULL? (turning it into xmalloc).
>
Oh I missed xmalloc.
> But couldn't this be done in place, with a little care?
>
> for(i = j = 0; i < d_config.cnt; i++)
> if (!compare(d_config.ptr[i], dev)
> d_config.ptr[j] = d_config.ptr[i] /* XXX optimise/skip if j == i */
> j++
> (note that j <= i is always true)
>
> then realloc the array down to only j entries.
>
This can be done. Will use this approach.
> > + \
> > + k = 0; \
> > + for (j = 0; j < d_config.cnt; j++) { \
> > + x = d_config.ptr + j; \
> > + if (compare(x, &(dev))) \
> > + continue; \
> > + libxl_device_ ## devtype ## _copy(ctx, &p[k], \
> > + d_config.ptr + j); \
>
> Same comment as the previous patch re the actual need for a deep copy.
>
This is a bit different because we don't need to fix up when removing a
device. And with your approach this deep copy is not necessary.
Wei.
> Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread* Re: [PATCH V4 24/24] xl: update configuration when we unplug a device
2014-05-06 16:12 ` Wei Liu
@ 2014-05-07 10:20 ` Ian Campbell
0 siblings, 0 replies; 120+ messages in thread
From: Ian Campbell @ 2014-05-07 10:20 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, xen-devel
On Tue, 2014-05-06 at 17:12 +0100, Wei Liu wrote:
> > > + do { \
> > > + libxl_domain_config d_config; \
> > > + libxl_device_ ## devtype *p = NULL, *x; \
> > > + int num; \
> > > + int j, k; \
> > > + \
> > > + libxl_domain_config_init(&d_config); \
> > > + load_domain_config((domid), &d_config); \
> > > + \
> > > + k = 0; \
> > > + for (j = 0; j < d_config.cnt; j++) { \
> > > + x = d_config.ptr + j; \
> > > + if (compare(x, &(dev))) \
> >
> > Are you concerned with the possibility that two entries in the array
> > might match dev? Wouldn't that equate to e.g. two xvda devices? Should
> > we not reject such things elsewhere?
> >
>
> Yes. And it's not just concern, I've seen this already!
> The current xl block-attach doesn't complain if you add same CD images
> twice.
Urk. That sounds like a bug, probably in libxl_device_disk_add rather
than xl. Perhaps that is one yak too many for now though.
Ian.
^ permalink raw reply [flat|nested] 120+ messages in thread
* Re: [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization
2014-05-01 12:57 [PATCH V4 00/24] JSON infrastructure, new "xl-json" format and domain configuration synchronization Wei Liu
` (23 preceding siblings ...)
2014-05-01 12:58 ` [PATCH V4 24/24] xl: update configuration when we unplug a device Wei Liu
@ 2014-05-01 13:02 ` Wei Liu
24 siblings, 0 replies; 120+ messages in thread
From: Wei Liu @ 2014-05-01 13:02 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
This series can be found at
git://xenbits.xen.org/people/liuw/xen.git wip.save-restore-overhaul7
Wei.
^ permalink raw reply [flat|nested] 120+ messages in thread