* [PATCH V3 01/13] libxl: fix memory leak in libxl_cpuid_dispose
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 02/13] libxl.h: document the paradigm of using libxl types Wei Liu
` (11 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
libxl_cpuid_policy_list is not allocated with GC-aware allocation so it
needs to be freed manually, just like what libxl_string_list_dispose and
libxl_key_value_list_dispose do.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_cpuid.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index dd21b78..8a59c4d 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -26,6 +26,7 @@ void libxl_cpuid_dispose(libxl_cpuid_policy_list *p_cpuid_list)
if (cpuid_list[i].policy[j] != NULL)
free(cpuid_list[i].policy[j]);
}
+ free(cpuid_list);
return;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 02/13] libxl.h: document the paradigm of using libxl types
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
2014-04-23 16:59 ` [PATCH V3 01/13] libxl: fix memory leak in libxl_cpuid_dispose Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 03/13] libxl.h: move / add some libxl defbool #define here Wei Liu
` (10 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.h | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index b2c3015..47ddd65 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -233,6 +233,11 @@
* libxl_types.idl). The library provides a common set of methods for
* initialising and freeing these types.
*
+ * The paradigm of using libxl types is that user always calls "init"
+ * function before using a type, and always calls "dispose" after using
+ * a type to clean up, regardless of the operations on this type
+ * succeed or fail. See xl code for examples.
+ *
* void libxl_<type>_init(<type> *p):
*
* Initialises the members of "p" to all defaults. These may either
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 03/13] libxl.h: move / add some libxl defbool #define here
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
2014-04-23 16:59 ` [PATCH V3 01/13] libxl: fix memory leak in libxl_cpuid_dispose Wei Liu
2014-04-23 16:59 ` [PATCH V3 02/13] libxl.h: document the paradigm of using libxl types Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 04/13] libxl IDL: rename json_fn to json_gen_fn Wei Liu
` (9 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
They will be used by both JSON generator and parser so they should be in
header file.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl.c | 10 +++-------
tools/libxl/libxl.h | 6 ++++++
2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 30b0b06..34de26b 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -232,10 +232,6 @@ void libxl_key_value_list_dispose(libxl_key_value_list *pkvl)
free(kvl);
}
-#define LIBXL__DEFBOOL_DEFAULT (0)
-#define LIBXL__DEFBOOL_FALSE (-1)
-#define LIBXL__DEFBOOL_TRUE (1)
-
void libxl_defbool_set(libxl_defbool *db, bool b)
{
db->val = b ? LIBXL__DEFBOOL_TRUE : LIBXL__DEFBOOL_FALSE;
@@ -266,11 +262,11 @@ bool libxl_defbool_val(libxl_defbool db)
const char *libxl_defbool_to_string(libxl_defbool b)
{
if (b.val < 0)
- return "False";
+ return LIBXL__DEFBOOL_STR_FALSE;
else if (b.val > 0)
- return "True";
+ return LIBXL__DEFBOOL_STR_TRUE;
else
- return "<default>";
+ return LIBXL__DEFBOOL_STR_DEFAULT;
}
/******************************************************************************/
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 47ddd65..7bface3 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -526,6 +526,12 @@ typedef LIBXL_TAILQ_ENTRY(struct libxl_event) libxl_ev_link;
* To allow users of the library to naively select all defaults this
* state is represented as 0. False is < 0 and True is > 0.
*/
+#define LIBXL__DEFBOOL_DEFAULT (0)
+#define LIBXL__DEFBOOL_FALSE (-1)
+#define LIBXL__DEFBOOL_TRUE (1)
+#define LIBXL__DEFBOOL_STR_DEFAULT "<default>"
+#define LIBXL__DEFBOOL_STR_FALSE "False"
+#define LIBXL__DEFBOOL_STR_TRUE "True"
typedef struct {
int val;
} libxl_defbool;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 04/13] libxl IDL: rename json_fn to json_gen_fn
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (2 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 03/13] libxl.h: move / add some libxl defbool #define here Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 05/13] libxl_json: introduce libx__object_from_json Wei Liu
` (8 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
This json_fn is in fact used to generate string representation of a json
data structure. We will introduce another json function to parse json
data structure in later changeset, so rename json_fn to json_gen_fn to
clarify.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxl/gentest.py | 4 ++--
tools/libxl/gentypes.py | 12 ++++++------
tools/libxl/idl.py | 10 +++++-----
tools/libxl/idl.txt | 4 ++--
tools/libxl/libxl_types.idl | 4 ++--
tools/libxl/libxl_types_internal.idl | 2 +-
6 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index 0ae55b1..5d22e3b 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -52,7 +52,7 @@ def gen_rand_init(ty, v, indent = " ", parent = None):
s += " break;\n"
s += "}\n"
elif isinstance(ty, idl.Struct) \
- and (parent is None or ty.json_fn is None):
+ and (parent is None or ty.json_gen_fn is None):
for f in [f for f in ty.fields if not f.const]:
(nparent,fexpr) = ty.member(v, f, parent is None)
s += gen_rand_init(f.type, fexpr, "", nparent)
@@ -243,7 +243,7 @@ int main(int argc, char **argv)
f.write(" printf(\"Testing TYPE_to_json()\\n\");\n")
f.write(" printf(\"----------------------\\n\");\n")
f.write(" printf(\"\\n\");\n")
- for ty in [t for t in types if t.json_fn is not None]:
+ for ty in [t for t in types if t.json_gen_fn is not None]:
arg = ty.typename + "_val"
f.write(" %s_rand_init(%s);\n" % (ty.typename, \
ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 917e2c2..61a2b3d 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -229,7 +229,7 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
s += " goto out;\n"
s += " break;\n"
s += "}\n"
- elif isinstance(ty, idl.Struct) and (parent is None or ty.json_fn is None):
+ elif isinstance(ty, idl.Struct) and (parent is None or ty.json_gen_fn is None):
s += "s = yajl_gen_map_open(hand);\n"
s += "if (s != yajl_gen_status_ok)\n"
s += " goto out;\n"
@@ -243,8 +243,8 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
s += "if (s != yajl_gen_status_ok)\n"
s += " goto out;\n"
else:
- if ty.json_fn is not None:
- s += "s = %s(hand, %s);\n" % (ty.json_fn, ty.pass_arg(v, parent is None))
+ if ty.json_gen_fn is not None:
+ s += "s = %s(hand, %s);\n" % (ty.json_gen_fn, ty.pass_arg(v, parent is None))
s += "if (s != yajl_gen_status_ok)\n"
s += " goto out;\n"
@@ -341,7 +341,7 @@ if __name__ == '__main__':
f.write("%svoid %s(%s, %s);\n" % (ty.hidden(), ty.init_fn + "_" + ku.keyvar.name,
ty.make_arg("p"),
ku.keyvar.type.make_arg(ku.keyvar.name)))
- if ty.json_fn is not None:
+ if ty.json_gen_fn is not None:
f.write("%schar *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
if isinstance(ty, idl.Enumeration):
f.write("%sconst char *%s_to_string(%s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
@@ -369,7 +369,7 @@ if __name__ == '__main__':
""" % (header_json_define, header_json_define, " ".join(sys.argv)))
- for ty in [ty for ty in types if ty.json_fn is not None]:
+ for ty in [ty for ty in types if ty.json_gen_fn is not None]:
f.write("%syajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
f.write("\n")
@@ -426,7 +426,7 @@ if __name__ == '__main__':
f.write("}\n")
f.write("\n")
- for ty in [t for t in types if t.json_fn is not None]:
+ for ty in [t for t in types if t.json_gen_fn is not None]:
f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s)\n" % (ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
f.write("{\n")
f.write(libxl_C_type_gen_json(ty, "p"))
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index e4dc79b..92133a3 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -65,9 +65,9 @@ class Type(object):
self.autogenerate_init_fn = kwargs.setdefault('autogenerate_init_fn', False)
if self.typename is not None and not self.private:
- self.json_fn = kwargs.setdefault('json_fn', self.typename + "_gen_json")
+ self.json_gen_fn = kwargs.setdefault('json_gen_fn', self.typename + "_gen_json")
else:
- self.json_fn = kwargs.setdefault('json_fn', None)
+ self.json_gen_fn = kwargs.setdefault('json_gen_fn', None)
self.autogenerate_json = kwargs.setdefault('autogenerate_json', True)
@@ -118,7 +118,7 @@ class Number(Builtin):
kwargs.setdefault('namespace', None)
kwargs.setdefault('dispose_fn', None)
kwargs.setdefault('signed', False)
- kwargs.setdefault('json_fn', "yajl_gen_integer")
+ kwargs.setdefault('json_gen_fn', "yajl_gen_integer")
self.signed = kwargs['signed']
Builtin.__init__(self, ctype, **kwargs)
@@ -256,7 +256,7 @@ class KeyedUnion(Aggregate):
void = Builtin("void *", namespace = None)
bool = Builtin("bool", namespace = None,
- json_fn = "yajl_gen_bool",
+ json_gen_fn = "yajl_gen_bool",
autogenerate_json = False)
size_t = Number("size_t", namespace = None)
@@ -269,7 +269,7 @@ uint32 = UInt(32)
uint64 = UInt(64)
string = Builtin("char *", namespace = None, dispose_fn = "free",
- json_fn = "libxl__string_gen_json",
+ json_gen_fn = "libxl__string_gen_json",
autogenerate_json = False)
class Array(Type):
diff --git a/tools/libxl/idl.txt b/tools/libxl/idl.txt
index 439aede..6a53dd8 100644
--- a/tools/libxl/idl.txt
+++ b/tools/libxl/idl.txt
@@ -60,14 +60,14 @@ Type.autogenerate_init_fn: (default: True if dir in [IN, BOTH])
Indicates if the above named Type.init_fn should be
autogenerated.
-Type.json_fn: (default: typename + "_gen_json" or None if type == None)
+Type.json_gen_fn: (default: typename + "_gen_json" or None if type == None)
The name of the C function which will generate a YAJL data structure
representing this type.
Type.autogenerate_json: (default: True)
- Indicates if the above named Type.json_fn should be autogenerated.
+ Indicates if the above named Type.json_gen_fn should be autogenerated.
Other simple type-Classes
-------------------------
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 0f7bbf8..755c918 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -7,8 +7,8 @@ namespace("libxl_")
libxl_defbool = Builtin("defbool", passby=PASS_BY_REFERENCE)
-libxl_domid = Builtin("domid", json_fn = "yajl_gen_integer", autogenerate_json = False)
-libxl_devid = Builtin("devid", json_fn = "yajl_gen_integer", autogenerate_json = False, signed = True, init_val="-1")
+libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False)
+libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False, signed = True, init_val="-1")
libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE)
libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE)
libxl_bitmap = Builtin("bitmap", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index cb9444f..a964851 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -1,7 +1,7 @@
namespace("libxl__")
hidden(True)
-libxl_domid = Builtin("domid", namespace="libxl_", json_fn = "yajl_gen_integer")
+libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer")
libxl__qmp_message_type = Enumeration("qmp_message_type", [
(1, "QMP"),
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 05/13] libxl_json: introduce libx__object_from_json
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (3 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 04/13] libxl IDL: rename json_fn to json_gen_fn Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 06/13] libxl_internal: make JSON_* types a bit-field Wei Liu
` (7 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Given a JSON string, we need to convert it to libxl_FOO struct.
The approach is:
JSON string -> libxl__json_object -> libxl_FOO struct
With this approach we can make use of libxl's infrastructure to do the
first half (JSON string -> libxl__json_object).
Second half is done by auto-generated code by libxl's IDL
infrastructure. IDL patch(es) will come later.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_internal.h | 8 ++++++++
tools/libxl/libxl_json.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 38 insertions(+)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index c2b73c4..0c10bf7 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1640,6 +1640,14 @@ typedef struct libxl__json_object {
struct libxl__json_object *parent;
} libxl__json_object;
+typedef int (*libxl__json_parse_callback)(libxl__gc *gc,
+ libxl__json_object *o,
+ void *p);
+_hidden int libxl__object_from_json(libxl_ctx *ctx, const char *type,
+ libxl__json_parse_callback parse,
+ void *p,
+ const char *s);
+
typedef struct {
char *map_key;
libxl__json_object *obj;
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 3ea56a4..b11ad4b 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -794,6 +794,36 @@ out:
return ret;
}
+int libxl__object_from_json(libxl_ctx *ctx, const char *type,
+ libxl__json_parse_callback parse,
+ void *p, const char *s)
+{
+ GC_INIT(ctx);
+ libxl__json_object *o;
+ int rc;
+
+ o = libxl__json_parse(gc, s);
+ if (!o) {
+ LOG(ERROR,
+ "unable to generate libxl__json_object from JSON representation of %s.",
+ type);
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ rc = parse(gc, o, p);
+ if (rc) {
+ LOG(ERROR, "unable to convert libxl__json_object to %s.", type);
+ rc = ERROR_FAIL;
+ goto out;
+ }
+
+ rc = 0;
+out:
+ GC_FREE;
+ return rc;
+}
+
/*
* Local variables:
* mode: C
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 06/13] libxl_internal: make JSON_* types a bit-field
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (4 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 05/13] libxl_json: introduce libx__object_from_json Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 07/13] libxl_internal: introduce libxl__json_object_is_{null, number, double} Wei Liu
` (6 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Libxl can generate number as type JSON_INTEGER, JSON_DOUBLE or
JSON_NUMBER, string as type JSON_STRING or JSON_NULL (if string is
null).
So make JSON_* type a bit-field and use it in libxl__json_map_get. This is
useful when parsing a libxl__json_object to libxl_FOO struct. We can
enforce type checking on libxl__json_object in an easy way.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxl/libxl_internal.h | 18 +++++++++---------
tools/libxl/libxl_json.c | 2 +-
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 0c10bf7..f37a102 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1613,16 +1613,16 @@ _hidden yajl_gen_status libxl__yajl_gen_asciiz(yajl_gen hand, const char *str);
_hidden yajl_gen_status libxl__yajl_gen_enum(yajl_gen hand, const char *str);
typedef enum {
- JSON_NULL,
- JSON_BOOL,
- JSON_INTEGER,
- JSON_DOUBLE,
+ JSON_NULL = (1 << 0),
+ JSON_BOOL = (1 << 1),
+ JSON_INTEGER = (1 << 2),
+ JSON_DOUBLE = (1 << 3),
/* number is store in string, it's too big to be a long long or a double */
- JSON_NUMBER,
- JSON_STRING,
- JSON_MAP,
- JSON_ARRAY,
- JSON_ANY
+ JSON_NUMBER = (1 << 4),
+ JSON_STRING = (1 << 5),
+ JSON_MAP = (1 << 6),
+ JSON_ARRAY = (1 << 7),
+ JSON_ANY = 255 /* this is a mask of all values above, adjust as needed */
} libxl__json_node_type;
typedef struct libxl__json_object {
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index b11ad4b..27cce9c 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -363,7 +363,7 @@ const libxl__json_object *libxl__json_map_get(const char *key,
return NULL;
if (strcmp(key, node->map_key) == 0) {
if (expected_type == JSON_ANY
- || (node->obj && node->obj->type == expected_type)) {
+ || (node->obj && (node->obj->type & expected_type))) {
return node->obj;
} else {
return NULL;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 07/13] libxl_internal: introduce libxl__json_object_is_{null, number, double}
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (5 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 06/13] libxl_internal: make JSON_* types a bit-field Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 08/13] libxl_json: introduce parser functions for builtin types Wei Liu
` (5 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
... which return true if json object is valid and of type
JSON_{NULL,NUMBER,DOUBLE}.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
tools/libxl/libxl_internal.h | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index f37a102..9074d40 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1655,6 +1655,10 @@ typedef struct {
typedef struct libxl__yajl_ctx libxl__yajl_ctx;
+static inline bool libxl__json_object_is_null(const libxl__json_object *o)
+{
+ return o != NULL && o->type == JSON_NULL;
+}
static inline bool libxl__json_object_is_bool(const libxl__json_object *o)
{
return o != NULL && o->type == JSON_BOOL;
@@ -1667,6 +1671,14 @@ static inline bool libxl__json_object_is_integer(const libxl__json_object *o)
{
return o != NULL && o->type == JSON_INTEGER;
}
+static inline bool libxl__json_object_is_double(const libxl__json_object *o)
+{
+ return o != NULL && o->type == JSON_DOUBLE;
+}
+static inline bool libxl__json_object_is_number(const libxl__json_object *o)
+{
+ return o != NULL && o->type == JSON_NUMBER;
+}
static inline bool libxl__json_object_is_map(const libxl__json_object *o)
{
return o != NULL && o->type == JSON_MAP;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 08/13] libxl_json: introduce parser functions for builtin types
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (6 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 07/13] libxl_internal: introduce libxl__json_object_is_{null, number, double} Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-05-01 14:28 ` Anthony PERARD
2014-04-23 16:59 ` [PATCH V3 09/13] libxl_json: allow basic JSON type objects generation Wei Liu
` (4 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
This changeset introduces following functions:
* libxl_defbool_parse_json
* libxl__bool_parse_json
* libxl_uuid_parse_json
* libxl_mac_parse_json
* libxl_bitmap_parse_json
* libxl_cpuid_policy_list_parse_json
* libxl_string_list_parse_json
* libxl_key_value_list_parse_json
* libxl_hwcap_parse_json
* libxl__int_parse_json
* libxl__uint{8,16,32,64}_parse_json
* libxl__string_parse_json
They will be used in later patch to convert the libxl__json_object
tree of a builtin type to libxl_FOO struct.
Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
yajl_gen_integer to generate JSON object.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_cpuid.c | 92 +++++++++++++---
tools/libxl/libxl_json.c | 245 +++++++++++++++++++++++++++++++++++++++++++
tools/libxl/libxl_json.h | 36 ++++++-
tools/libxl/libxl_nocpuid.c | 7 ++
4 files changed, 363 insertions(+), 17 deletions(-)
diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
index 8a59c4d..895f5fd 100644
--- a/tools/libxl/libxl_cpuid.c
+++ b/tools/libxl/libxl_cpuid.c
@@ -337,29 +337,29 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
(const char**)(cpuid[i].policy), cpuid_res);
}
+static const char *input_names[2] = { "leaf", "subleaf" };
+static const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
+/*
+ * Aiming for:
+ * [
+ * { 'leaf': 'val-eax',
+ * 'subleaf': 'val-ecx',
+ * 'eax': 'filter',
+ * 'ebx': 'filter',
+ * 'ecx': 'filter',
+ * 'edx': 'filter' },
+ * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
+ * ... etc ...
+ * ]
+ */
+
yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
libxl_cpuid_policy_list *pcpuid)
{
libxl_cpuid_policy_list cpuid = *pcpuid;
yajl_gen_status s;
- const char *input_names[2] = { "leaf", "subleaf" };
- const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
int i, j;
- /*
- * Aiming for:
- * [
- * { 'leaf': 'val-eax',
- * 'subleaf': 'val-ecx',
- * 'eax': 'filter',
- * 'ebx': 'filter',
- * 'ecx': 'filter',
- * 'edx': 'filter' },
- * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
- * ... etc ...
- * ]
- */
-
s = yajl_gen_array_open(hand);
if (s != yajl_gen_status_ok) goto out;
@@ -397,6 +397,66 @@ out:
return s;
}
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p)
+{
+ int i, size;
+ libxl_cpuid_policy_list l;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ if (!o->u.array->count)
+ return 0;
+
+ size = o->u.array->count;
+ /* need one extra slot as sentinel */
+ l = *p = calloc(size + 1, sizeof(libxl_cpuid_policy));
+
+ if (!l)
+ return ERROR_NOMEM;
+
+ l[size].input[0] = XEN_CPUID_INPUT_UNUSED;
+ l[size].input[1] = XEN_CPUID_INPUT_UNUSED;
+
+ for (i = 0; i < size; i++) {
+ const libxl__json_object *t;
+ int j;
+
+ if (flexarray_get(o->u.array, i, (void**)&t) != 0)
+ return ERROR_FAIL;
+
+ if (!libxl__json_object_is_map(t))
+ return ERROR_FAIL;
+
+ for (j = 0; j < ARRAY_SIZE(l[0].input); j++) {
+ const libxl__json_object *r;
+
+ r = libxl__json_map_get(input_names[j], t, JSON_INTEGER);
+ if (!r)
+ l[i].input[j] = XEN_CPUID_INPUT_UNUSED;
+ else
+ l[i].input[j] = r->u.i;
+ }
+
+ for (j = 0; j < ARRAY_SIZE(l[0].policy); j++) {
+ const libxl__json_object *r;
+
+ r = libxl__json_map_get(policy_names[j], t, JSON_STRING);
+ if (!r) {
+ l[i].policy[j] = NULL;
+ } else {
+ l[i].policy[j] = strdup(r->u.string);
+ if (!l[i].policy[j])
+ return ERROR_NOMEM;
+ }
+ }
+ }
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 27cce9c..3e7eb95 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -100,6 +100,38 @@ yajl_gen_status libxl_defbool_gen_json(yajl_gen hand,
return libxl__yajl_gen_asciiz(hand, libxl_defbool_to_string(*db));
}
+int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_defbool *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_DEFAULT,
+ strlen(LIBXL__DEFBOOL_STR_DEFAULT)))
+ p->val = LIBXL__DEFBOOL_DEFAULT;
+ else if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_TRUE,
+ strlen(LIBXL__DEFBOOL_STR_TRUE)))
+ p->val = LIBXL__DEFBOOL_TRUE;
+ else if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_FALSE,
+ strlen(LIBXL__DEFBOOL_STR_FALSE)))
+ p->val = LIBXL__DEFBOOL_FALSE;
+ else
+ return ERROR_FAIL;
+
+ return 0;
+}
+
+int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ bool *p)
+{
+ if (!libxl__json_object_is_bool(o))
+ return ERROR_FAIL;
+
+ *p = o->u.b;
+
+ return 0;
+}
+
yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
libxl_uuid *uuid)
{
@@ -108,6 +140,15 @@ yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN);
}
+int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_uuid *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ return libxl_uuid_from_string(p, o->u.string);
+}
+
yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand,
libxl_bitmap *bitmap)
{
@@ -128,6 +169,36 @@ out:
return s;
}
+int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_bitmap *p)
+{
+ int i;
+ int size;
+ const libxl__json_object *t;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ if (!o->u.array->count) {
+ libxl_bitmap_init(p);
+ return 0;
+ }
+
+ t = libxl__json_array_get(o, o->u.array->count - 1);
+ size = t->u.i + 1;
+
+ libxl_bitmap_alloc(CTX, p, size);
+
+ for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
+ if (!libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+
+ libxl_bitmap_set(p, t->u.i);
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
libxl_key_value_list *pkvl)
{
@@ -155,6 +226,47 @@ out:
return s;
}
+int libxl_key_value_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_key_value_list *p)
+{
+ libxl__json_map_node *node = NULL;
+ flexarray_t *maps = NULL;
+ int i, size;
+ libxl_key_value_list kvl;
+
+ if (!libxl__json_object_is_map(o))
+ return ERROR_FAIL;
+
+ maps = o->u.map;
+ size = maps->count * 2;
+ kvl = *p = calloc(size, sizeof(char *));
+ if (!kvl)
+ return ERROR_NOMEM;
+
+ for (i = 0; i < maps->count; i++) {
+ int idx = i * 2;
+ if (flexarray_get(maps, i, (void**)&node) != 0)
+ return ERROR_FAIL;
+
+ if (!libxl__json_object_is_string(node->obj) &&
+ !libxl__json_object_is_null(node->obj))
+ return ERROR_FAIL;
+
+ kvl[idx] = strdup(node->map_key);
+ if (!kvl[idx])
+ return ERROR_NOMEM;
+ if (libxl__json_object_is_string(node->obj)) {
+ kvl[idx+1] = strdup(node->obj->u.string);
+ if (!kvl[idx+1])
+ return ERROR_NOMEM;
+ } else {
+ kvl[idx+1] = NULL;
+ }
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl)
{
libxl_string_list l = *pl;
@@ -176,6 +288,37 @@ out:
return s;
}
+int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_string_list *p)
+{
+ const libxl__json_object *t;
+ libxl_string_list l;
+ flexarray_t *array = NULL;
+ int i, size;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ array = o->u.array;
+ size = array->count;
+ /* need one extra slot as sentinel */
+ l = *p = calloc((size + 1), sizeof(char *));
+
+ if (!l)
+ return ERROR_NOMEM;
+
+ for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
+ if (!libxl__json_object_is_string(t))
+ return ERROR_FAIL;
+
+ l[i] = strdup(t->u.string);
+ if (!l[i])
+ return ERROR_NOMEM;
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
{
char buf[LIBXL_MAC_FMTLEN+1];
@@ -183,6 +326,15 @@ yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN);
}
+int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_mac *p)
+{
+ if (!libxl__json_object_is_string(o))
+ return ERROR_FAIL;
+
+ return libxl__parse_mac(o->u.string, *p);
+}
+
yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand,
libxl_hwcap *p)
{
@@ -201,6 +353,27 @@ out:
return s;
}
+int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_hwcap *p)
+{
+ int i;
+
+ if (!libxl__json_object_is_array(o))
+ return ERROR_FAIL;
+
+ for (i = 0; i<4; i++) {
+ const libxl__json_object *t;
+
+ t = libxl__json_array_get(o, i);
+ if (!t || !libxl__json_object_is_integer(t))
+ return ERROR_FAIL;
+
+ (*p)[i] = t->u.i;
+ }
+
+ return 0;
+}
+
yajl_gen_status libxl__string_gen_json(yajl_gen hand,
const char *p)
{
@@ -210,6 +383,23 @@ yajl_gen_status libxl__string_gen_json(yajl_gen hand,
return yajl_gen_null(hand);
}
+int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ char **p)
+{
+ if (!libxl__json_object_is_string(o) && !libxl__json_object_is_null(o))
+ return ERROR_FAIL;
+
+ if (libxl__json_object_is_null(o)) {
+ *p = NULL;
+ } else {
+ *p = strdup(o->u.string);
+ if (!*p)
+ return ERROR_NOMEM;
+ }
+
+ return 0;
+}
+
/*
* libxl__json_object helper functions
*/
@@ -824,6 +1014,61 @@ out:
return rc;
}
+int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((int *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint8_t *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint16_t *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint32_t *)p) = o->u.i;
+
+ return 0;
+}
+
+int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p)
+{
+ if (!libxl__json_object_is_integer(o))
+ return ERROR_FAIL;
+
+ *((uint64_t *)p) = o->u.i;
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_json.h b/tools/libxl/libxl_json.h
index a4dd8fc..924b2aa 100644
--- a/tools/libxl/libxl_json.h
+++ b/tools/libxl/libxl_json.h
@@ -22,17 +22,51 @@
# include <yajl/yajl_version.h>
#endif
+typedef struct libxl__gc libxl__gc;
+typedef struct libxl__json_object libxl__json_object;
+
yajl_gen_status libxl_defbool_gen_json(yajl_gen hand, libxl_defbool *p);
-yajl_gen_status libxl_domid_gen_json(yajl_gen hand, libxl_domid *p);
+int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_defbool *p);
+int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ bool *p);
yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, libxl_uuid *p);
+int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_uuid *p);
yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *p);
+int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_mac *p);
yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand, libxl_bitmap *p);
+int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_bitmap *p);
yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
libxl_cpuid_policy_list *p);
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p);
yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *p);
+int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_string_list *p);
yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
libxl_key_value_list *p);
+int libxl_key_value_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_key_value_list *p);
yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, libxl_hwcap *p);
+int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ libxl_hwcap *p);
+int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ void *p);
+int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
+ char **p);
#include <_libxl_types_json.h>
diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
index 5f7cb6a..eb525fc 100644
--- a/tools/libxl/libxl_nocpuid.c
+++ b/tools/libxl/libxl_nocpuid.c
@@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
return 0;
}
+int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
+ const libxl__json_object *o,
+ libxl_cpuid_policy_list *p)
+{
+ return 0;
+}
+
/*
* Local variables:
* mode: C
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH V3 08/13] libxl_json: introduce parser functions for builtin types
2014-04-23 16:59 ` [PATCH V3 08/13] libxl_json: introduce parser functions for builtin types Wei Liu
@ 2014-05-01 14:28 ` Anthony PERARD
2014-05-01 14:38 ` Wei Liu
0 siblings, 1 reply; 20+ messages in thread
From: Anthony PERARD @ 2014-05-01 14:28 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.jackson, ian.campbell, xen-devel
On Wed, Apr 23, 2014 at 05:59:18PM +0100, Wei Liu wrote:
> This changeset introduces following functions:
> * libxl_defbool_parse_json
> * libxl__bool_parse_json
> * libxl_uuid_parse_json
> * libxl_mac_parse_json
> * libxl_bitmap_parse_json
> * libxl_cpuid_policy_list_parse_json
> * libxl_string_list_parse_json
> * libxl_key_value_list_parse_json
> * libxl_hwcap_parse_json
> * libxl__int_parse_json
> * libxl__uint{8,16,32,64}_parse_json
> * libxl__string_parse_json
>
> They will be used in later patch to convert the libxl__json_object
> tree of a builtin type to libxl_FOO struct.
>
> Also remove delcaration of libxl_domid_gen_json as libxl_domid uses
> yajl_gen_integer to generate JSON object.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/libxl_cpuid.c | 92 +++++++++++++---
> tools/libxl/libxl_json.c | 245 +++++++++++++++++++++++++++++++++++++++++++
> tools/libxl/libxl_json.h | 36 ++++++-
> tools/libxl/libxl_nocpuid.c | 7 ++
> 4 files changed, 363 insertions(+), 17 deletions(-)
>
> diff --git a/tools/libxl/libxl_cpuid.c b/tools/libxl/libxl_cpuid.c
> index 8a59c4d..895f5fd 100644
> --- a/tools/libxl/libxl_cpuid.c
> +++ b/tools/libxl/libxl_cpuid.c
> @@ -337,29 +337,29 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
> (const char**)(cpuid[i].policy), cpuid_res);
> }
>
> +static const char *input_names[2] = { "leaf", "subleaf" };
> +static const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
> +/*
> + * Aiming for:
> + * [
> + * { 'leaf': 'val-eax',
> + * 'subleaf': 'val-ecx',
> + * 'eax': 'filter',
> + * 'ebx': 'filter',
> + * 'ecx': 'filter',
> + * 'edx': 'filter' },
> + * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
> + * ... etc ...
> + * ]
> + */
> +
> yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> libxl_cpuid_policy_list *pcpuid)
> {
> libxl_cpuid_policy_list cpuid = *pcpuid;
> yajl_gen_status s;
> - const char *input_names[2] = { "leaf", "subleaf" };
> - const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" };
> int i, j;
>
> - /*
> - * Aiming for:
> - * [
> - * { 'leaf': 'val-eax',
> - * 'subleaf': 'val-ecx',
> - * 'eax': 'filter',
> - * 'ebx': 'filter',
> - * 'ecx': 'filter',
> - * 'edx': 'filter' },
> - * { 'leaf': 'val-eax', ..., 'eax': 'filter', ... },
> - * ... etc ...
> - * ]
> - */
> -
> s = yajl_gen_array_open(hand);
> if (s != yajl_gen_status_ok) goto out;
>
> @@ -397,6 +397,66 @@ out:
> return s;
> }
>
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p)
> +{
> + int i, size;
> + libxl_cpuid_policy_list l;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + if (!o->u.array->count)
> + return 0;
There is a function called libxl__json_object_get_array which return a
flexarray_t. It might be better to use it than accessing it through
o->u.array.
> +
> + size = o->u.array->count;
> + /* need one extra slot as sentinel */
> + l = *p = calloc(size + 1, sizeof(libxl_cpuid_policy));
> +
> + if (!l)
> + return ERROR_NOMEM;
> +
> + l[size].input[0] = XEN_CPUID_INPUT_UNUSED;
> + l[size].input[1] = XEN_CPUID_INPUT_UNUSED;
> +
> + for (i = 0; i < size; i++) {
> + const libxl__json_object *t;
> + int j;
> +
> + if (flexarray_get(o->u.array, i, (void**)&t) != 0)
> + return ERROR_FAIL;
> +
> + if (!libxl__json_object_is_map(t))
> + return ERROR_FAIL;
> +
> + for (j = 0; j < ARRAY_SIZE(l[0].input); j++) {
> + const libxl__json_object *r;
> +
> + r = libxl__json_map_get(input_names[j], t, JSON_INTEGER);
> + if (!r)
> + l[i].input[j] = XEN_CPUID_INPUT_UNUSED;
> + else
> + l[i].input[j] = r->u.i;
What about libxl__json_object_get_integer here? Same thing about
different types through the patch.
> + }
> +
> + for (j = 0; j < ARRAY_SIZE(l[0].policy); j++) {
> + const libxl__json_object *r;
> +
> + r = libxl__json_map_get(policy_names[j], t, JSON_STRING);
> + if (!r) {
> + l[i].policy[j] = NULL;
> + } else {
> + l[i].policy[j] = strdup(r->u.string);
libxl__json_object_get_string?
> + if (!l[i].policy[j])
> + return ERROR_NOMEM;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
> index 27cce9c..3e7eb95 100644
> --- a/tools/libxl/libxl_json.c
> +++ b/tools/libxl/libxl_json.c
> @@ -100,6 +100,38 @@ yajl_gen_status libxl_defbool_gen_json(yajl_gen hand,
> return libxl__yajl_gen_asciiz(hand, libxl_defbool_to_string(*db));
> }
>
> +int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_defbool *p)
> +{
> + if (!libxl__json_object_is_string(o))
> + return ERROR_FAIL;
> +
> + if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_DEFAULT,
> + strlen(LIBXL__DEFBOOL_STR_DEFAULT)))
> + p->val = LIBXL__DEFBOOL_DEFAULT;
> + else if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_TRUE,
> + strlen(LIBXL__DEFBOOL_STR_TRUE)))
> + p->val = LIBXL__DEFBOOL_TRUE;
> + else if (!strncmp(o->u.string, LIBXL__DEFBOOL_STR_FALSE,
> + strlen(LIBXL__DEFBOOL_STR_FALSE)))
> + p->val = LIBXL__DEFBOOL_FALSE;
> + else
> + return ERROR_FAIL;
> +
> + return 0;
> +}
> +
> +int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + bool *p)
> +{
> + if (!libxl__json_object_is_bool(o))
> + return ERROR_FAIL;
> +
> + *p = o->u.b;
libxl__json_object_get_bool?
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
> libxl_uuid *uuid)
> {
> @@ -108,6 +140,15 @@ yajl_gen_status libxl_uuid_gen_json(yajl_gen hand,
> return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN);
> }
>
> +int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_uuid *p)
> +{
> + if (!libxl__json_object_is_string(o))
> + return ERROR_FAIL;
> +
> + return libxl_uuid_from_string(p, o->u.string);
> +}
> +
> yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand,
> libxl_bitmap *bitmap)
> {
> @@ -128,6 +169,36 @@ out:
> return s;
> }
>
> +int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_bitmap *p)
> +{
> + int i;
> + int size;
> + const libxl__json_object *t;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + if (!o->u.array->count) {
> + libxl_bitmap_init(p);
> + return 0;
> + }
> +
> + t = libxl__json_array_get(o, o->u.array->count - 1);
> + size = t->u.i + 1;
t is not an integer, yet. You should check first with
libxl__json_object_is_integer and get the value with
libxl__json_object_get_integer.
> +
> + libxl_bitmap_alloc(CTX, p, size);
> +
> + for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
> + if (!libxl__json_object_is_integer(t))
> + return ERROR_FAIL;
> +
> + libxl_bitmap_set(p, t->u.i);
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
> libxl_key_value_list *pkvl)
> {
> @@ -155,6 +226,47 @@ out:
> return s;
> }
>
> +int libxl_key_value_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_key_value_list *p)
> +{
> + libxl__json_map_node *node = NULL;
> + flexarray_t *maps = NULL;
> + int i, size;
> + libxl_key_value_list kvl;
> +
> + if (!libxl__json_object_is_map(o))
> + return ERROR_FAIL;
> +
> + maps = o->u.map;
libxl__json_object_get_map?
> + size = maps->count * 2;
> + kvl = *p = calloc(size, sizeof(char *));
> + if (!kvl)
> + return ERROR_NOMEM;
> +
> + for (i = 0; i < maps->count; i++) {
> + int idx = i * 2;
> + if (flexarray_get(maps, i, (void**)&node) != 0)
> + return ERROR_FAIL;
> +
> + if (!libxl__json_object_is_string(node->obj) &&
> + !libxl__json_object_is_null(node->obj))
> + return ERROR_FAIL;
> +
> + kvl[idx] = strdup(node->map_key);
> + if (!kvl[idx])
> + return ERROR_NOMEM;
> + if (libxl__json_object_is_string(node->obj)) {
> + kvl[idx+1] = strdup(node->obj->u.string);
> + if (!kvl[idx+1])
> + return ERROR_NOMEM;
> + } else {
> + kvl[idx+1] = NULL;
> + }
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl)
> {
> libxl_string_list l = *pl;
> @@ -176,6 +288,37 @@ out:
> return s;
> }
>
> +int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_string_list *p)
> +{
> + const libxl__json_object *t;
> + libxl_string_list l;
> + flexarray_t *array = NULL;
> + int i, size;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + array = o->u.array;
> + size = array->count;
> + /* need one extra slot as sentinel */
> + l = *p = calloc((size + 1), sizeof(char *));
> +
> + if (!l)
> + return ERROR_NOMEM;
> +
> + for (i = 0; (t = libxl__json_array_get(o, i)); i++) {
> + if (!libxl__json_object_is_string(t))
> + return ERROR_FAIL;
> +
> + l[i] = strdup(t->u.string);
> + if (!l[i])
> + return ERROR_NOMEM;
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
> {
> char buf[LIBXL_MAC_FMTLEN+1];
> @@ -183,6 +326,15 @@ yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac)
> return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN);
> }
>
> +int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_mac *p)
> +{
> + if (!libxl__json_object_is_string(o))
> + return ERROR_FAIL;
> +
> + return libxl__parse_mac(o->u.string, *p);
> +}
> +
> yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand,
> libxl_hwcap *p)
> {
> @@ -201,6 +353,27 @@ out:
> return s;
> }
>
> +int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_hwcap *p)
> +{
> + int i;
> +
> + if (!libxl__json_object_is_array(o))
> + return ERROR_FAIL;
> +
> + for (i = 0; i<4; i++) {
> + const libxl__json_object *t;
> +
> + t = libxl__json_array_get(o, i);
> + if (!t || !libxl__json_object_is_integer(t))
> + return ERROR_FAIL;
> +
> + (*p)[i] = t->u.i;
> + }
> +
> + return 0;
> +}
> +
> yajl_gen_status libxl__string_gen_json(yajl_gen hand,
> const char *p)
> {
> @@ -210,6 +383,23 @@ yajl_gen_status libxl__string_gen_json(yajl_gen hand,
> return yajl_gen_null(hand);
> }
>
> +int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + char **p)
> +{
> + if (!libxl__json_object_is_string(o) && !libxl__json_object_is_null(o))
> + return ERROR_FAIL;
> +
> + if (libxl__json_object_is_null(o)) {
> + *p = NULL;
> + } else {
> + *p = strdup(o->u.string);
> + if (!*p)
> + return ERROR_NOMEM;
> + }
> +
> + return 0;
> +}
> +
> /*
> * libxl__json_object helper functions
> */
> @@ -824,6 +1014,61 @@ out:
> return rc;
> }
>
> +int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o))
> + return ERROR_FAIL;
> +
> + *((int *)p) = o->u.i;
> +
> + return 0;
> +}
> +
> +int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o))
> + return ERROR_FAIL;
> +
> + *((uint8_t *)p) = o->u.i;
Maybe we should check the value of the int. If it those not fit in
uint8, then there is an error in the json input. Same thing for the
other functions below.
> +
> + return 0;
> +}
> +
> +int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o))
> + return ERROR_FAIL;
> +
> + *((uint16_t *)p) = o->u.i;
> +
> + return 0;
> +}
> +
> +int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o))
> + return ERROR_FAIL;
> +
> + *((uint32_t *)p) = o->u.i;
> +
> + return 0;
> +}
> +
> +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p)
> +{
> + if (!libxl__json_object_is_integer(o))
> + return ERROR_FAIL;
> +
> + *((uint64_t *)p) = o->u.i;
> +
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
> diff --git a/tools/libxl/libxl_json.h b/tools/libxl/libxl_json.h
> index a4dd8fc..924b2aa 100644
> --- a/tools/libxl/libxl_json.h
> +++ b/tools/libxl/libxl_json.h
> @@ -22,17 +22,51 @@
> # include <yajl/yajl_version.h>
> #endif
>
> +typedef struct libxl__gc libxl__gc;
> +typedef struct libxl__json_object libxl__json_object;
> +
> yajl_gen_status libxl_defbool_gen_json(yajl_gen hand, libxl_defbool *p);
> -yajl_gen_status libxl_domid_gen_json(yajl_gen hand, libxl_domid *p);
> +int libxl_defbool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_defbool *p);
> +int libxl__bool_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + bool *p);
> yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, libxl_uuid *p);
> +int libxl_uuid_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_uuid *p);
> yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *p);
> +int libxl_mac_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_mac *p);
> yajl_gen_status libxl_bitmap_gen_json(yajl_gen hand, libxl_bitmap *p);
> +int libxl_bitmap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_bitmap *p);
> yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> libxl_cpuid_policy_list *p);
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p);
> yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *p);
> +int libxl_string_list_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_string_list *p);
> yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand,
> libxl_key_value_list *p);
> +int libxl_key_value_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_key_value_list *p);
> yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, libxl_hwcap *p);
> +int libxl_hwcap_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + libxl_hwcap *p);
> +int libxl__int_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint8_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint16_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint32_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__uint64_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + void *p);
> +int libxl__string_parse_json(libxl__gc *gc, const libxl__json_object *o,
> + char **p);
>
> #include <_libxl_types_json.h>
>
> diff --git a/tools/libxl/libxl_nocpuid.c b/tools/libxl/libxl_nocpuid.c
> index 5f7cb6a..eb525fc 100644
> --- a/tools/libxl/libxl_nocpuid.c
> +++ b/tools/libxl/libxl_nocpuid.c
> @@ -44,6 +44,13 @@ yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand,
> return 0;
> }
>
> +int libxl_cpuid_policy_list_parse_json(libxl__gc *gc,
> + const libxl__json_object *o,
> + libxl_cpuid_policy_list *p)
> +{
> + return 0;
> +}
> +
> /*
> * Local variables:
> * mode: C
Regards,
--
Anthony PERARD
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH V3 09/13] libxl_json: allow basic JSON type objects generation
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (7 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 08/13] libxl_json: introduce parser functions for builtin types Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 10/13] libxl/gentypes.py: include discriminator in JSON output Wei Liu
` (3 subsequent siblings)
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
The original logic is that basic JSON types (number, string and null)
must be an element of JSON map or array. This assumption doesn't hold
true anymore when we need to return basic JSON types.
Returning basic JSON types is required for parsing number, string and
null objects back into libxl__json_object.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/libxl_internal.h | 2 +-
tools/libxl/libxl_json.c | 87 ++++++++++++++++++------------------------
2 files changed, 38 insertions(+), 51 deletions(-)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 9074d40..6e3d19d 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1735,7 +1735,7 @@ _hidden libxl__json_object *libxl__json_object_alloc(libxl__gc *gc_opt,
libxl__json_node_type type);
_hidden int libxl__json_object_append_to(libxl__gc *gc_opt,
libxl__json_object *obj,
- libxl__json_object *dst);
+ libxl__yajl_ctx *ctx);
_hidden libxl__json_object *libxl__json_array_get(const libxl__json_object *o,
int i);
_hidden
diff --git a/tools/libxl/libxl_json.c b/tools/libxl/libxl_json.c
index 3e7eb95..44efbc0 100644
--- a/tools/libxl/libxl_json.c
+++ b/tools/libxl/libxl_json.c
@@ -424,36 +424,43 @@ libxl__json_object *libxl__json_object_alloc(libxl__gc *gc,
return obj;
}
-int libxl__json_object_append_to(libxl__gc *gc,
- libxl__json_object *obj,
- libxl__json_object *dst)
+int libxl__json_object_append_to(libxl__gc *gc, libxl__json_object *obj,
+ libxl__yajl_ctx *ctx)
{
- assert(dst != NULL);
+ libxl__json_object *dst = ctx->current;
- switch (dst->type) {
- case JSON_MAP: {
- libxl__json_map_node *last;
+ if (dst) {
+ switch (dst->type) {
+ case JSON_MAP: {
+ libxl__json_map_node *last;
- if (dst->u.map->count == 0) {
+ if (dst->u.map->count == 0) {
+ LIBXL__LOG(libxl__gc_owner(gc), LIBXL__LOG_ERROR,
+ "Try to add a value to an empty map (with no key)");
+ return ERROR_FAIL;
+ }
+ flexarray_get(dst->u.map, dst->u.map->count - 1, (void**)&last);
+ last->obj = obj;
+ break;
+ }
+ case JSON_ARRAY:
+ flexarray_append(dst->u.array, obj);
+ break;
+ default:
LIBXL__LOG(libxl__gc_owner(gc), LIBXL__LOG_ERROR,
- "Try to add a value to an empty map (with no key)");
- return -1;
+ "Try append an object is not a map/array (%i)\n",
+ dst->type);
+ return ERROR_FAIL;
}
- flexarray_get(dst->u.map, dst->u.map->count - 1, (void**)&last);
- last->obj = obj;
- break;
- }
- case JSON_ARRAY:
- flexarray_append(dst->u.array, obj);
- break;
- default:
- LIBXL__LOG(libxl__gc_owner(gc), LIBXL__LOG_ERROR,
- "Try append an object is not a map/array (%i)\n",
- dst->type);
- return -1;
}
obj->parent = dst;
+
+ if (libxl__json_object_is_map(obj) || libxl__json_object_is_array(obj))
+ ctx->current = obj;
+ if (ctx->head == NULL)
+ ctx->head = obj;
+
return 0;
}
@@ -639,9 +646,8 @@ static int json_callback_null(void *opaque)
obj = libxl__json_object_alloc(ctx->gc, JSON_NULL);
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -656,9 +662,8 @@ static int json_callback_boolean(void *opaque, int boolean)
obj = libxl__json_object_alloc(ctx->gc, JSON_BOOL);
obj->u.b = boolean;
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -713,9 +718,8 @@ error:
obj->u.string = t;
out:
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -737,9 +741,8 @@ static int json_callback_string(void *opaque, const unsigned char *str,
obj = libxl__json_object_alloc(ctx->gc, JSON_STRING);
obj->u.string = t;
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
return 0;
- }
return 1;
}
@@ -785,16 +788,8 @@ static int json_callback_start_map(void *opaque)
obj = libxl__json_object_alloc(ctx->gc, JSON_MAP);
- if (ctx->current) {
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
- return 0;
- }
- }
-
- ctx->current = obj;
- if (ctx->head == NULL) {
- ctx->head = obj;
- }
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
+ return 0;
return 1;
}
@@ -825,16 +820,8 @@ static int json_callback_start_array(void *opaque)
obj = libxl__json_object_alloc(ctx->gc, JSON_ARRAY);
- if (ctx->current) {
- if (libxl__json_object_append_to(ctx->gc, obj, ctx->current) == -1) {
- return 0;
- }
- }
-
- ctx->current = obj;
- if (ctx->head == NULL) {
- ctx->head = obj;
- }
+ if (libxl__json_object_append_to(ctx->gc, obj, ctx))
+ return 0;
return 1;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 10/13] libxl/gentypes.py: include discriminator in JSON output
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (8 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 09/13] libxl_json: allow basic JSON type objects generation Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 22:40 ` Andrew Cooper
2014-04-23 16:59 ` [PATCH V3 11/13] libxl IDL: generate code to parse libxl__json_object to libxl_FOO struct Wei Liu
` (2 subsequent siblings)
12 siblings, 1 reply; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Parser relies on the discriminator to go to correct branch.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentypes.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 61a2b3d..8d7183a 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -229,6 +229,11 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
s += " goto out;\n"
s += " break;\n"
s += "}\n"
+ s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" \
+ % (ty.keyvar.name, ty.keyvar.name)
+ s += "if (s != yajl_gen_status_ok)\n"
+ s += " goto out;\n"
+ s += libxl_C_type_gen_json(ty.keyvar.type, (parent + ty.keyvar.name), indent, parent)
elif isinstance(ty, idl.Struct) and (parent is None or ty.json_gen_fn is None):
s += "s = yajl_gen_map_open(hand);\n"
s += "if (s != yajl_gen_status_ok)\n"
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH V3 10/13] libxl/gentypes.py: include discriminator in JSON output
2014-04-23 16:59 ` [PATCH V3 10/13] libxl/gentypes.py: include discriminator in JSON output Wei Liu
@ 2014-04-23 22:40 ` Andrew Cooper
2014-04-24 8:12 ` Wei Liu
0 siblings, 1 reply; 20+ messages in thread
From: Andrew Cooper @ 2014-04-23 22:40 UTC (permalink / raw)
To: Wei Liu, xen-devel; +Cc: ian.jackson, ian.campbell
On 23/04/2014 17:59, Wei Liu wrote:
> Parser relies on the discriminator to go to correct branch.
>
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> tools/libxl/gentypes.py | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
> index 61a2b3d..8d7183a 100644
> --- a/tools/libxl/gentypes.py
> +++ b/tools/libxl/gentypes.py
> @@ -229,6 +229,11 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
> s += " goto out;\n"
> s += " break;\n"
> s += "}\n"
> + s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" \
> + % (ty.keyvar.name, ty.keyvar.name)
In python, all forms of string delimiter are equal, precisely to fix the
problem demonstrated here. This line can be made more legible using
single quotes for the main string, at which point the double quotes
around %s don't need escaping.
~Andrew
> + s += "if (s != yajl_gen_status_ok)\n"
> + s += " goto out;\n"
> + s += libxl_C_type_gen_json(ty.keyvar.type, (parent + ty.keyvar.name), indent, parent)
> elif isinstance(ty, idl.Struct) and (parent is None or ty.json_gen_fn is None):
> s += "s = yajl_gen_map_open(hand);\n"
> s += "if (s != yajl_gen_status_ok)\n"
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH V3 10/13] libxl/gentypes.py: include discriminator in JSON output
2014-04-23 22:40 ` Andrew Cooper
@ 2014-04-24 8:12 ` Wei Liu
0 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-24 8:12 UTC (permalink / raw)
To: Andrew Cooper; +Cc: ian.jackson, Wei Liu, ian.campbell, xen-devel
On Wed, Apr 23, 2014 at 11:40:24PM +0100, Andrew Cooper wrote:
> On 23/04/2014 17:59, Wei Liu wrote:
> > Parser relies on the discriminator to go to correct branch.
> >
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> > tools/libxl/gentypes.py | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
> > index 61a2b3d..8d7183a 100644
> > --- a/tools/libxl/gentypes.py
> > +++ b/tools/libxl/gentypes.py
> > @@ -229,6 +229,11 @@ def libxl_C_type_gen_json(ty, v, indent = " ", parent = None):
> > s += " goto out;\n"
> > s += " break;\n"
> > s += "}\n"
> > + s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" \
> > + % (ty.keyvar.name, ty.keyvar.name)
>
> In python, all forms of string delimiter are equal, precisely to fix the
> problem demonstrated here. This line can be made more legible using
> single quotes for the main string, at which point the double quotes
> around %s don't need escaping.
>
Oh, I just wanted to be consistent with the reset of the file. :-)
Wei.
> ~Andrew
>
> > + s += "if (s != yajl_gen_status_ok)\n"
> > + s += " goto out;\n"
> > + s += libxl_C_type_gen_json(ty.keyvar.type, (parent + ty.keyvar.name), indent, parent)
> > elif isinstance(ty, idl.Struct) and (parent is None or ty.json_gen_fn is None):
> > s += "s = yajl_gen_map_open(hand);\n"
> > s += "if (s != yajl_gen_status_ok)\n"
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH V3 11/13] libxl IDL: generate code to parse libxl__json_object to libxl_FOO struct
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (9 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 10/13] libxl/gentypes.py: include discriminator in JSON output Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 12/13] libxl/gentest.py: test JSON parser Wei Liu
2014-04-23 16:59 ` [PATCH V3 13/13] xl: introduce "xl-json" format Wei Liu
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
libxl_FOO_parse_json functions are generated.
Note that these functions are used to parse libxl__json_object to
libxl__FOO struct. They don't consume JSON string.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentypes.py | 114 ++++++++++++++++++++++++++++++++++
tools/libxl/idl.py | 15 +++++
tools/libxl/idl.txt | 7 ++-
tools/libxl/libxl_types.idl | 31 +++++----
tools/libxl/libxl_types_internal.idl | 4 +-
5 files changed, 156 insertions(+), 15 deletions(-)
diff --git a/tools/libxl/gentypes.py b/tools/libxl/gentypes.py
index 8d7183a..6aba75f 100644
--- a/tools/libxl/gentypes.py
+++ b/tools/libxl/gentypes.py
@@ -270,6 +270,99 @@ def libxl_C_type_to_json(ty, v, indent = " "):
s = indent + s
return s.replace("\n", "\n%s" % indent).rstrip(indent)
+def libxl_C_type_parse_json(ty, w, v, indent = " ", parent = None):
+ s = ""
+ if parent is None:
+ s += "int rc = 0;\n"
+ s += "const libxl__json_object *x = o;\n"
+
+ if isinstance(ty, idl.Array):
+ if parent is None:
+ raise Exception("Array type must have a parent")
+ lenvar = parent + ty.lenvar.name
+ s += "{\n"
+ s += " libxl__json_object *t;\n"
+ s += " int i;\n"
+ s += " if (!libxl__json_object_is_array(x)) {\n"
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += " %s = x->u.array->count;\n" % lenvar
+ s += " %s = calloc(%s, sizeof(*%s));\n" % (v, lenvar, v)
+ s += " if (!%s && %s != 0) {\n" % (v, lenvar)
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += " for (i=0; (t=libxl__json_array_get(x,i)); i++) {\n"
+ s += libxl_C_type_parse_json(ty.elem_type, "t", v+"[i]",
+ indent + " ", parent)
+ s += " }\n"
+ s += " if (i != %s) {\n" % lenvar
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += "}\n"
+ elif isinstance(ty, idl.Enumeration):
+ s += "{\n"
+ s += " const char *enum_str;\n"
+ s += " if (!libxl__json_object_is_string(x)) {\n"
+ s += " rc = -1;\n"
+ s += " goto out;\n"
+ s += " }\n"
+ s += " enum_str = libxl__json_object_get_string(x);\n"
+ s += " rc = %s_from_string(enum_str, %s);\n" % (ty.typename, ty.pass_arg(v, parent is None, idl.PASS_BY_REFERENCE))
+ s += " if (rc)\n"
+ s += " goto out;\n"
+ s += "}\n"
+ elif isinstance(ty, idl.KeyedUnion):
+ if parent is None:
+ raise Exception("KeyedUnion type must have a parent")
+ s += "switch (%s) {\n" % (parent + ty.keyvar.name)
+ for f in ty.fields:
+ (nparent,fexpr) = ty.member(v, f, parent is None)
+ s += "case %s:\n" % f.enumname
+ if f.type is not None:
+ s += libxl_C_type_parse_json(f.type, w, fexpr, indent + " ", nparent)
+ s += " break;\n"
+ s += "}\n"
+ elif isinstance(ty, idl.Struct) and (parent is None or ty.json_parse_fn is None):
+ for f in [f for f in ty.fields if not f.const and not f.type.private]:
+ if isinstance(f.type, idl.KeyedUnion):
+ s += "x = libxl__json_map_get(\"%s\", %s, %s);\n" % \
+ (f.type.keyvar.name, w, f.type.keyvar.type.json_parse_type)
+ s += "if (x) {\n"
+ (nparent,fexpr) = ty.member(v, f.type.keyvar, parent is None)
+ s += libxl_C_type_parse_json(f.type.keyvar.type, "x", fexpr, " ", nparent)
+ s += "}\n"
+ s += "x = libxl__json_map_get(\"%s\", %s, %s);\n" % (f.name, w, f.type.json_parse_type)
+ s += "if (x) {\n"
+ (nparent,fexpr) = ty.member(v, f, parent is None)
+ s += libxl_C_type_parse_json(f.type, "x", fexpr, " ", nparent)
+ s += " x = x->parent;\n"
+ s += "}\n"
+ else:
+ if ty.json_parse_fn is not None:
+ s += "rc = %s(gc, %s, &%s);\n" % (ty.json_parse_fn, w, v)
+ s += "if (rc)\n"
+ s += " goto out;\n"
+
+ if parent is None:
+ s += "out:\n"
+ s += "return rc;\n"
+
+ if s != "":
+ s = indent +s
+ return s.replace("\n", "\n%s" % indent).rstrip(indent)
+
+def libxl_C_type_from_json(ty, v, w, indent = " "):
+ s = ""
+ parse = "(libxl__json_parse_callback)&%s_parse_json" % ty.typename
+ s += "return libxl__object_from_json(ctx, \"%s\", %s, %s, %s);\n" % (ty.typename, parse, v, w)
+
+ if s != "":
+ s = indent + s
+ return s.replace("\n", "\n%s" % indent).rstrip(indent)
+
def libxl_C_enum_to_string(ty, e, indent = " "):
s = ""
s += "switch(%s) {\n" % e
@@ -348,6 +441,8 @@ if __name__ == '__main__':
ku.keyvar.type.make_arg(ku.keyvar.name)))
if ty.json_gen_fn is not None:
f.write("%schar *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
+ if ty.json_parse_fn is not None:
+ f.write("%sint %s_from_json(libxl_ctx *ctx, %s, const char *s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
if isinstance(ty, idl.Enumeration):
f.write("%sconst char *%s_to_string(%s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p")))
f.write("%sint %s_from_string(const char *s, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("e", passby=idl.PASS_BY_REFERENCE)))
@@ -377,6 +472,10 @@ if __name__ == '__main__':
for ty in [ty for ty in types if ty.json_gen_fn is not None]:
f.write("%syajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+ for ty in [ty for ty in types if ty.json_parse_fn is not None]:
+ f.write("%sint %s_parse_json(libxl__gc *gc, const libxl__json_object *o, %s);\n" % \
+ (ty.hidden(), ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+
f.write("\n")
f.write("""#endif /* %s */\n""" % header_json_define)
f.close()
@@ -444,4 +543,19 @@ if __name__ == '__main__':
f.write("}\n")
f.write("\n")
+ for ty in [t for t in types if t.json_parse_fn is not None]:
+ f.write("int %s_parse_json(libxl__gc *gc, const libxl__json_object *%s, %s)\n" % (ty.typename,"o",ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+ f.write("{\n")
+ f.write(libxl_C_type_parse_json(ty, "o", "p"))
+ f.write("}\n")
+ f.write("\n")
+
+ f.write("int %s_from_json(libxl_ctx *ctx, %s, const char *s)\n" % (ty.typename, ty.make_arg("p", passby=idl.PASS_BY_REFERENCE)))
+ f.write("{\n")
+ if not isinstance(ty, idl.Enumeration):
+ f.write(" %s_init(p);\n" % ty.typename)
+ f.write(libxl_C_type_from_json(ty, "p", "s"))
+ f.write("}\n")
+ f.write("\n")
+
f.close()
diff --git a/tools/libxl/idl.py b/tools/libxl/idl.py
index 92133a3..9ce1e33 100644
--- a/tools/libxl/idl.py
+++ b/tools/libxl/idl.py
@@ -66,8 +66,12 @@ class Type(object):
if self.typename is not None and not self.private:
self.json_gen_fn = kwargs.setdefault('json_gen_fn', self.typename + "_gen_json")
+ self.json_parse_type = kwargs.setdefault('json_parse_type', "JSON_ANY")
+ self.json_parse_fn = kwargs.setdefault('json_parse_fn', self.typename + "_parse_json")
else:
self.json_gen_fn = kwargs.setdefault('json_gen_fn', None)
+ self.json_parse_type = kwargs.setdefault('json_parse_type', None)
+ self.json_parse_fn = kwargs.setdefault('json_parse_fn', None)
self.autogenerate_json = kwargs.setdefault('autogenerate_json', True)
@@ -119,6 +123,9 @@ class Number(Builtin):
kwargs.setdefault('dispose_fn', None)
kwargs.setdefault('signed', False)
kwargs.setdefault('json_gen_fn', "yajl_gen_integer")
+ kwargs.setdefault('json_parse_type', "JSON_INTEGER")
+ # json_parse_fn might be overriden on specific type
+ kwargs.setdefault('json_parse_fn', "libxl__int_parse_json")
self.signed = kwargs['signed']
Builtin.__init__(self, ctype, **kwargs)
@@ -126,6 +133,7 @@ class UInt(Number):
def __init__(self, w, **kwargs):
kwargs.setdefault('namespace', None)
kwargs.setdefault('dispose_fn', None)
+ kwargs.setdefault('json_parse_fn', "libxl__uint%d_parse_json" % w)
Number.__init__(self, "uint%d_t" % w, **kwargs)
self.width = w
@@ -142,6 +150,7 @@ class EnumerationValue(object):
class Enumeration(Type):
def __init__(self, typename, values, **kwargs):
kwargs.setdefault('dispose_fn', None)
+ kwargs.setdefault('json_parse_type', "JSON_STRING")
Type.__init__(self, typename, **kwargs)
self.value_namespace = kwargs.setdefault('value_namespace',
@@ -171,6 +180,7 @@ class Field(object):
class Aggregate(Type):
"""A type containing a collection of other types"""
def __init__(self, kind, typename, fields, **kwargs):
+ kwargs.setdefault('json_parse_type', "JSON_MAP")
Type.__init__(self, typename, **kwargs)
if self.typename is not None:
@@ -257,6 +267,8 @@ class KeyedUnion(Aggregate):
void = Builtin("void *", namespace = None)
bool = Builtin("bool", namespace = None,
json_gen_fn = "yajl_gen_bool",
+ json_parse_type = "JSON_BOOL",
+ json_parse_fn = "libxl__bool_parse_json",
autogenerate_json = False)
size_t = Number("size_t", namespace = None)
@@ -270,12 +282,15 @@ uint64 = UInt(64)
string = Builtin("char *", namespace = None, dispose_fn = "free",
json_gen_fn = "libxl__string_gen_json",
+ json_parse_type = "JSON_STRING | JSON_NULL",
+ json_parse_fn = "libxl__string_parse_json",
autogenerate_json = False)
class Array(Type):
"""An array of the same type"""
def __init__(self, elem_type, lenvar_name, **kwargs):
kwargs.setdefault('dispose_fn', 'free')
+ kwargs.setdefault('json_parse_type', 'JSON_ARRAY')
Type.__init__(self, namespace=elem_type.namespace, typename=elem_type.rawname + " *", **kwargs)
lv_kwargs = dict([(x.lstrip('lenvar_'),y) for (x,y) in kwargs.items() if x.startswith('lenvar_')])
diff --git a/tools/libxl/idl.txt b/tools/libxl/idl.txt
index 6a53dd8..484d5d7 100644
--- a/tools/libxl/idl.txt
+++ b/tools/libxl/idl.txt
@@ -65,9 +65,14 @@ Type.json_gen_fn: (default: typename + "_gen_json" or None if type == None)
The name of the C function which will generate a YAJL data structure
representing this type.
+Type.json_parse_fn: (default: typename + "_parse_json" or None if type == None)
+
+ The name of the C function which will parse a libxl JSON structure
+ representing this type to C type.
+
Type.autogenerate_json: (default: True)
- Indicates if the above named Type.json_gen_fn should be autogenerated.
+ Indicates if the above named Type.json_*_fn should be autogenerated.
Other simple type-Classes
-------------------------
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 755c918..e064ce8 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -5,18 +5,23 @@
namespace("libxl_")
-libxl_defbool = Builtin("defbool", passby=PASS_BY_REFERENCE)
-
-libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False)
-libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", autogenerate_json = False, signed = True, init_val="-1")
-libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE)
-libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE)
-libxl_bitmap = Builtin("bitmap", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
-libxl_cpuid_policy_list = Builtin("cpuid_policy_list", dispose_fn="libxl_cpuid_dispose", passby=PASS_BY_REFERENCE)
-
-libxl_string_list = Builtin("string_list", dispose_fn="libxl_string_list_dispose", passby=PASS_BY_REFERENCE)
-libxl_key_value_list = Builtin("key_value_list", dispose_fn="libxl_key_value_list_dispose", passby=PASS_BY_REFERENCE)
-libxl_hwcap = Builtin("hwcap", passby=PASS_BY_REFERENCE)
+libxl_defbool = Builtin("defbool", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+
+libxl_domid = Builtin("domid", json_gen_fn = "yajl_gen_integer", json_parse_fn = "libxl__uint32_parse_json",
+ json_parse_type = "JSON_INTEGER", autogenerate_json = False)
+libxl_devid = Builtin("devid", json_gen_fn = "yajl_gen_integer", json_parse_fn = "libxl__int_parse_json",
+ json_parse_type = "JSON_INTEGER", autogenerate_json = False, signed = True, init_val="-1")
+libxl_uuid = Builtin("uuid", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+libxl_mac = Builtin("mac", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE)
+libxl_bitmap = Builtin("bitmap", json_parse_type="JSON_ARRAY", dispose_fn="libxl_bitmap_dispose", passby=PASS_BY_REFERENCE)
+libxl_cpuid_policy_list = Builtin("cpuid_policy_list", json_parse_type="JSON_ARRAY",
+ dispose_fn="libxl_cpuid_dispose", passby=PASS_BY_REFERENCE)
+
+libxl_string_list = Builtin("string_list", json_parse_type="JSON_ARRAY",
+ dispose_fn="libxl_string_list_dispose", passby=PASS_BY_REFERENCE)
+libxl_key_value_list = Builtin("key_value_list", json_parse_type="JSON_MAP",
+ dispose_fn="libxl_key_value_list_dispose", passby=PASS_BY_REFERENCE)
+libxl_hwcap = Builtin("hwcap", json_parse_type="JSON_ARRAY", passby=PASS_BY_REFERENCE)
#
# Specific integer types
@@ -575,7 +580,7 @@ libxl_event_type = Enumeration("event_type", [
libxl_ev_user = UInt(64)
-libxl_ev_link = Builtin("ev_link", passby=PASS_BY_REFERENCE, private=True)
+libxl_ev_link = Builtin("ev_link", json_parse_type="JSON_STRING", passby=PASS_BY_REFERENCE, private=True)
libxl_event = Struct("event",[
("link", libxl_ev_link),
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index a964851..17533f1 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -1,7 +1,9 @@
namespace("libxl__")
hidden(True)
-libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer")
+libxl_domid = Builtin("domid", namespace="libxl_", json_gen_fn = "yajl_gen_integer",
+ json_parse_fn = "libxl__uint32_parse_json", json_parse_type = "JSON_INTEGER",
+ autogenerate_json = False)
libxl__qmp_message_type = Enumeration("qmp_message_type", [
(1, "QMP"),
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 12/13] libxl/gentest.py: test JSON parser
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (10 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 11/13] libxl IDL: generate code to parse libxl__json_object to libxl_FOO struct Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-23 16:59 ` [PATCH V3 13/13] xl: introduce "xl-json" format Wei Liu
12 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
The test is done in following steps:
1. initialise libxl_FOO struct
2. generate JSON string A for libxl_FOO struct FOO1
3. convert JSON string A to libxl_FOO struct FOO2
4. generate JSON string B for libxl_FOO struct FOO2
5. compare A and B
If A and B are identical then we are good.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
tools/libxl/gentest.py | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/tools/libxl/gentest.py b/tools/libxl/gentest.py
index 5d22e3b..67972b7 100644
--- a/tools/libxl/gentest.py
+++ b/tools/libxl/gentest.py
@@ -225,10 +225,11 @@ int main(int argc, char **argv)
""")
for ty in types:
- f.write(" %s %s_val;\n" % (ty.typename, ty.typename))
+ f.write(" %s %s_val, %s_val_new;\n" % \
+ (ty.typename, ty.typename, ty.typename))
f.write("""
int rc;
- char *s;
+ char *s, *new_s;
xentoollog_logger_stdiostream *logger;
libxl_ctx *ctx;
@@ -240,20 +241,36 @@ int main(int argc, char **argv)
exit(1);
}
""")
- f.write(" printf(\"Testing TYPE_to_json()\\n\");\n")
+ f.write(" printf(\"Testing TYPE_to/from_json()\\n\");\n")
f.write(" printf(\"----------------------\\n\");\n")
f.write(" printf(\"\\n\");\n")
for ty in [t for t in types if t.json_gen_fn is not None]:
arg = ty.typename + "_val"
f.write(" %s_rand_init(%s);\n" % (ty.typename, \
ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
+ if not isinstance(ty, idl.Enumeration):
+ f.write(" %s_init(%s_new);\n" % (ty.typename, \
+ ty.pass_arg(arg, isref=False, passby=idl.PASS_BY_REFERENCE)))
f.write(" s = %s_to_json(ctx, %s);\n" % \
(ty.typename, ty.pass_arg(arg, isref=False)))
f.write(" printf(\"%%s: %%s\\n\", \"%s\", s);\n" % ty.typename)
f.write(" if (s == NULL) abort();\n")
+ f.write(" rc = %s_from_json(ctx, &%s_val_new, s);\n" % \
+ (ty.typename, ty.typename))
+ f.write(" if (rc) abort();\n")
+ f.write(" new_s = %s_to_json(ctx, %s_new);\n" % \
+ (ty.typename, ty.pass_arg(arg, isref=False)))
+ f.write(" if (new_s == NULL) abort();\n")
+ f.write(" if (strcmp(s, new_s)) {\n")
+ f.write(" printf(\"Huh? Regenerated string different from original string.\\n\");\n")
+ f.write(" printf(\"Regenerated string: %s\\n\", new_s);\n")
+ f.write(" abort();\n")
+ f.write(" }\n")
f.write(" free(s);\n")
+ f.write(" free(new_s);\n")
if ty.dispose_fn is not None:
f.write(" %s(&%s_val);\n" % (ty.dispose_fn, ty.typename))
+ f.write(" %s(&%s_val_new);\n" % (ty.dispose_fn, ty.typename))
f.write("\n")
f.write(" printf(\"Testing Enumerations\\n\");\n")
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* [PATCH V3 13/13] xl: introduce "xl-json" format
2014-04-23 16:59 [PATCH V3 00/13] JSON infrastructure and new "xl-json" format Wei Liu
` (11 preceding siblings ...)
2014-04-23 16:59 ` [PATCH V3 12/13] libxl/gentest.py: test JSON parser Wei Liu
@ 2014-04-23 16:59 ` Wei Liu
2014-04-24 16:11 ` Ian Jackson
12 siblings, 1 reply; 20+ messages in thread
From: Wei Liu @ 2014-04-23 16:59 UTC (permalink / raw)
To: xen-devel; +Cc: Wei Liu, ian.jackson, ian.campbell
Originally xl verbatimly copies the domain config file in its user data
store. This patch adds the functionality to transform text domain config
file to JSON object and save that object in user data store.
What this patch does:
1. add a mandatory flag to save protocol to indicate whether the saved
config is a JSON object
2. register a new private data type "xl-json" in libxl.h
3. modify xl to save / load "xl-json" file where necessary
After this change xl supports both "xl" format and "xl-json" format.
The user-supplied config file is still restricted to normal text config
file format ("xl"), as xl has more sanity checks when parsing text
config file. "xl-json" format is only used internally. The saved config
file is always in "xl-json" format.
Tests done so far (xl.{new,old} denotes xl with{,out} "xl_json"
support):
1. xl.new create then hexdump saved file: domain config saved in JSON format
2. xl.new create then xl.old restore: failed on mandatory flag check
3. xl.new create then xl.new restore: succeeded
4. xl.old create then xl.new restore: succeeded
5. xl.new create then local migrate, receiving end xl.new: succeeded
6. xl.old create then local migrate, receiving end xl.new: succeeded
The only drawback is that when restoring a domain, xl cannot
automatically spawn a vncviewer anymore. That's because that option is
part of domain_create info not a domain configuration thus it's not
saved in the JSON config. A warning is printed out and documentation
is updated.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
docs/man/xl.cfg.pod.5 | 3 +-
tools/libxl/libxl.h | 1 +
tools/libxl/xl_cmdimpl.c | 112 ++++++++++++++++++++++++++++++++++++----------
3 files changed, 92 insertions(+), 24 deletions(-)
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a6663b9..9e11cdb 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1106,7 +1106,8 @@ other VNC-related settings. The default is to enable this.
=item B<vncviewer=BOOLEAN>
-Automatically spawn a vncviewer when creating/restoring a guest.
+Automatically spawn a vncviewer when creating a guest. This option has
+no effect on restoring a guest and it might be removed in the future.
=item B<vnclisten="ADDRESS[:DISPLAYNUM]">
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 7bface3..44f8897 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1057,6 +1057,7 @@ void libxl_cpuid_set(libxl_ctx *ctx, uint32_t domid,
*
* userid Data contents
* "xl" domain config file in xl format, Unix line endings
+ * "xl-json" domain config in JSON format generated by xl
* "libvirt-xml" domain config file in libvirt XML format. See
* http://libvirt.org/formatdomain.html
*
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 0b38b32..d72388b 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -110,6 +110,8 @@ static const char migrate_report[]=
* from target to source
*/
+#define XL_MANDATORY_FLAG_JSON (1U << 0) /* config data is in JSON format */
+#define XL_MANDATORY_FLAG_ALL (XL_MANDATORY_FLAG_JSON)
struct save_file_header {
char magic[32]; /* savefileheader_magic */
/* All uint32_ts are in domain's byte order. */
@@ -725,6 +727,24 @@ static void parse_top_level_sdl_options(XLU_Config *config,
xlu_cfg_replace_string (config, "xauthority", &sdl->xauthority, 0);
}
+static void parse_config_data_json(const char *config_data,
+ int config_len,
+ libxl_domain_config *d_config)
+{
+ int ret;
+
+ if (!config_len) {
+ fprintf(stderr, "Config data stream empty\n");
+ exit(1);
+ }
+
+ ret = libxl_domain_config_from_json(ctx, d_config, config_data);
+ if (ret) {
+ fprintf(stderr, "Failed to parse config\n");
+ exit(1);
+ }
+}
+
static void parse_config_data(const char *config_source,
const char *config_data,
int config_len,
@@ -989,6 +1009,8 @@ static void parse_config_data(const char *config_source,
b_info->rtc_timeoffset = l;
if (dom_info && !xlu_cfg_get_long(config, "vncviewer", &l, 0)) {
+ fprintf(stderr, "WARNING: \"vncviewer\" option found. It might be removed in future release. "
+ "Use \"-V\" option of \"xl create\" to automatically spawn vncviewer.\n");
/* Command line arguments must take precedence over what's
* specified in the configuration file. */
if (!dom_info->vnc)
@@ -1769,12 +1791,13 @@ skip_vfb:
}
static void reload_domain_config(uint32_t domid,
- uint8_t **config_data, int *config_len)
+ uint8_t **config_data, int *config_len,
+ bool *config_in_json)
{
uint8_t *t_data;
int ret, t_len;
- ret = libxl_userdata_retrieve(ctx, domid, "xl", &t_data, &t_len);
+ ret = libxl_userdata_retrieve(ctx, domid, "xl-json", &t_data, &t_len);
if (ret) {
LOG("failed to retrieve guest configuration (rc=%d). "
"reusing old configuration", ret);
@@ -1784,6 +1807,7 @@ static void reload_domain_config(uint32_t domid,
free(*config_data);
*config_data = t_data;
*config_len = t_len;
+ *config_in_json = true;
}
/* Returns 1 if domain should be restarted,
@@ -1792,6 +1816,7 @@ static void reload_domain_config(uint32_t domid,
static int handle_domain_death(uint32_t *r_domid,
libxl_event *event,
uint8_t **config_data, int *config_len,
+ bool *config_in_json,
libxl_domain_config *d_config)
{
@@ -1849,12 +1874,14 @@ static int handle_domain_death(uint32_t *r_domid,
break;
case LIBXL_ACTION_ON_SHUTDOWN_RESTART_RENAME:
- reload_domain_config(*r_domid, config_data, config_len);
+ reload_domain_config(*r_domid, config_data, config_len,
+ config_in_json);
restart = 2;
break;
case LIBXL_ACTION_ON_SHUTDOWN_RESTART:
- reload_domain_config(*r_domid, config_data, config_len);
+ reload_domain_config(*r_domid, config_data, config_len,
+ config_in_json);
restart = 1;
/* fall-through */
@@ -2040,6 +2067,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
uint32_t domid = INVALID_DOMID;
libxl_domain_config d_config;
+ char *d_config_json = NULL;
int debug = dom_info->debug;
int daemonize = dom_info->daemonize;
@@ -2060,6 +2088,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
libxl_evgen_disk_eject **diskws = NULL; /* one per disk */
void *config_data = 0;
int config_len = 0;
+ bool config_in_json = false;
int restore_fd = -1;
const libxl_asyncprogress_how *autoconnect_console_how;
struct save_file_header hdr;
@@ -2106,7 +2135,7 @@ static uint32_t create_domain(struct domain_create *dom_info)
restore_source, hdr.mandatory_flags, hdr.optional_flags,
hdr.optional_data_len);
- badflags = hdr.mandatory_flags & ~( 0 /* none understood yet */ );
+ badflags = hdr.mandatory_flags & ~XL_MANDATORY_FLAG_ALL;
if (badflags) {
fprintf(stderr, "Savefile has mandatory flag(s) 0x%"PRIx32" "
"which are not supported; need newer xl\n",
@@ -2134,7 +2163,10 @@ static uint32_t create_domain(struct domain_create *dom_info)
optdata_here = optdata_begin;
if (OPTDATA_LEFT) {
- fprintf(stderr, " Savefile contains xl domain config\n");
+ config_in_json =
+ !!(hdr.mandatory_flags & XL_MANDATORY_FLAG_JSON);
+ fprintf(stderr, " Savefile contains xl domain config%s\n",
+ config_in_json ? " in JSON format" : "");
WITH_OPTDATA(4, {
memcpy(u32buf.b, optdata_here, 4);
config_len = u32buf.u32;
@@ -2174,6 +2206,8 @@ static uint32_t create_domain(struct domain_create *dom_info)
extra_config);
}
config_source=config_file;
+ /* User supplied file is just a text config file */
+ config_in_json = false;
} else {
if (!config_data) {
fprintf(stderr, "Config file not specified and"
@@ -2186,7 +2220,11 @@ static uint32_t create_domain(struct domain_create *dom_info)
if (!dom_info->quiet)
printf("Parsing config from %s\n", config_source);
- parse_config_data(config_source, config_data, config_len, &d_config, dom_info);
+ if (config_in_json)
+ parse_config_data_json(config_data, config_len, &d_config);
+ else
+ parse_config_data(config_source, config_data, config_len,
+ &d_config, dom_info);
if (migrate_fd >= 0) {
if (d_config.c_info.name) {
@@ -2213,6 +2251,10 @@ static uint32_t create_domain(struct domain_create *dom_info)
if (dom_info->dryrun)
goto out;
+ d_config_json = libxl_domain_config_to_json(ctx, &d_config);
+ if (!d_config_json)
+ goto out;
+
start:
assert(domid == INVALID_DOMID);
@@ -2281,8 +2323,9 @@ start:
free(vcpu_to_pcpu); vcpu_to_pcpu = NULL;
}
- ret = libxl_userdata_store(ctx, domid, "xl",
- config_data, config_len);
+ ret = libxl_userdata_store(ctx, domid, "xl-json",
+ (const uint8_t *)d_config_json,
+ strlen(d_config_json));
if (ret) {
perror("cannot save config file");
ret = ERROR_FAIL;
@@ -2347,7 +2390,7 @@ start:
event->u.domain_shutdown.shutdown_reason);
switch (handle_domain_death(&domid, event,
(uint8_t **)&config_data, &config_len,
- &d_config)) {
+ &config_in_json, &d_config)) {
case 2:
if (!preserve_domain(&domid, event, &d_config)) {
/* If we fail then exit leaving the old domain in place. */
@@ -2387,8 +2430,11 @@ start:
/* Reparse the configuration in case it has changed */
libxl_domain_config_dispose(&d_config);
libxl_domain_config_init(&d_config);
- parse_config_data(config_source, config_data, config_len,
- &d_config, dom_info);
+ if (config_in_json)
+ parse_config_data_json(config_data, config_len, &d_config);
+ else
+ parse_config_data(config_source, config_data, config_len,
+ &d_config, dom_info);
/*
* XXX FIXME: If this sleep is not there then domain
@@ -2444,6 +2490,8 @@ out:
free(config_data);
+ free(d_config_json);
+
console_child_report(child_console);
if (deathw)
@@ -3194,12 +3242,14 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain)
/* no detailed info available on dom0 */
if (info[i].domid == 0)
continue;
- rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len);
+ rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl-json",
+ &data, &len);
if (rc)
continue;
CHK_SYSCALL(asprintf(&config_source, "<domid %d data>", info[i].domid));
libxl_domain_config_init(&d_config);
- parse_config_data(config_source, (char *)data, len, &d_config, NULL);
+ /* Saved config file is in JSON format */
+ parse_config_data_json((const char *)data, len, &d_config);
if (default_output_format == OUTPUT_FORMAT_JSON)
s = printf_info_one_json(hand, info[i].domid, &d_config);
else
@@ -3407,7 +3457,7 @@ static void save_domain_core_begin(uint32_t domid,
&config_v, config_len_r);
*config_data_r = config_v;
} else {
- rc = libxl_userdata_retrieve(ctx, domid, "xl",
+ rc = libxl_userdata_retrieve(ctx, domid, "xl-json",
config_data_r, config_len_r);
}
if (rc) {
@@ -3417,7 +3467,8 @@ static void save_domain_core_begin(uint32_t domid,
}
static void save_domain_core_writeconfig(int fd, const char *source,
- const uint8_t *config_data, int config_len)
+ const uint8_t *config_data, int config_len,
+ bool config_in_json)
{
struct save_file_header hdr;
uint8_t *optdata_begin;
@@ -3441,6 +3492,8 @@ static void save_domain_core_writeconfig(int fd, const char *source,
u32buf.u32 = config_len;
ADD_OPTDATA(u32buf.b, 4);
ADD_OPTDATA(config_data, config_len);
+ if (config_in_json)
+ hdr.mandatory_flags |= XL_MANDATORY_FLAG_JSON;
/* that's the optional data */
@@ -3464,12 +3517,17 @@ static int save_domain(uint32_t domid, const char *filename, int checkpoint,
int fd;
uint8_t *config_data;
int config_len;
+ /* If user doesn't provide override_config_file, we use saved
+ * config file which is in JSON format.
+ */
+ bool config_in_json = (override_config_file == NULL);
save_domain_core_begin(domid, override_config_file,
&config_data, &config_len);
if (!config_len) {
fputs(" Savefile will not contain xl domain config\n", stderr);
+ config_in_json = false;
}
fd = open(filename, O_WRONLY|O_CREAT|O_TRUNC, 0644);
@@ -3478,7 +3536,8 @@ static int save_domain(uint32_t domid, const char *filename, int checkpoint,
exit(2);
}
- save_domain_core_writeconfig(fd, filename, config_data, config_len);
+ save_domain_core_writeconfig(fd, filename, config_data, config_len,
+ config_in_json);
int rc = libxl_domain_suspend(ctx, domid, fd, 0, NULL);
close(fd);
@@ -3619,7 +3678,7 @@ static void migration_child_report(int recv_fd) {
static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child,
uint8_t *config_data, int config_len,
- const char *rune)
+ bool config_in_json, const char *rune)
{
int rc = 0;
@@ -3638,7 +3697,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child,
}
save_domain_core_writeconfig(send_fd, "migration stream",
- config_data, config_len);
+ config_data, config_len, config_in_json);
}
@@ -3652,6 +3711,7 @@ static void migrate_domain(uint32_t domid, const char *rune, int debug,
char rc_buf;
uint8_t *config_data;
int config_len, flags = LIBXL_SUSPEND_LIVE;
+ bool config_in_json = (override_config_file == NULL);
save_domain_core_begin(domid, override_config_file,
&config_data, &config_len);
@@ -3665,7 +3725,7 @@ static void migrate_domain(uint32_t domid, const char *rune, int debug,
child = create_migration_child(rune, &send_fd, &recv_fd);
migrate_do_preamble(send_fd, recv_fd, child, config_data, config_len,
- rune);
+ config_in_json, rune);
xtl_stdiostream_adjust_flags(logger, XTL_STDIOSTREAM_HIDE_PROGRESS, 0);
@@ -4507,19 +4567,24 @@ int main_config_update(int argc, char **argv)
libxl_domain_config_init(&d_config);
+ /* User supplied config is just text config file */
parse_config_data(filename, config_data, config_len, &d_config, NULL);
if (debug || dryrun_only)
printf_info(default_output_format, -1, &d_config);
if (!dryrun_only) {
+ char *d_config_json = NULL;
fprintf(stderr, "setting dom%d configuration\n", domid);
- rc = libxl_userdata_store(ctx, domid, "xl",
- config_data, config_len);
+ d_config_json = libxl_domain_config_to_json(ctx, &d_config);
+ rc = libxl_userdata_store(ctx, domid, "xl-json",
+ (const uint8_t*)d_config_json,
+ strlen(d_config_json));
if (rc) {
fprintf(stderr, "failed to update configuration\n");
exit(1);
}
+ free(d_config_json);
}
libxl_domain_config_dispose(&d_config);
@@ -7333,7 +7398,8 @@ int main_remus(int argc, char **argv)
child = create_migration_child(rune, &send_fd, &recv_fd);
- migrate_do_preamble(send_fd, recv_fd, child, config_data, config_len,
+ migrate_do_preamble(send_fd, recv_fd, child,
+ config_data, config_len, true /* config_in_json */,
rune);
if (ssh_command[0])
--
1.7.10.4
^ permalink raw reply related [flat|nested] 20+ messages in thread* Re: [PATCH V3 13/13] xl: introduce "xl-json" format
2014-04-23 16:59 ` [PATCH V3 13/13] xl: introduce "xl-json" format Wei Liu
@ 2014-04-24 16:11 ` Ian Jackson
2014-04-25 15:53 ` Wei Liu
0 siblings, 1 reply; 20+ messages in thread
From: Ian Jackson @ 2014-04-24 16:11 UTC (permalink / raw)
To: Wei Liu; +Cc: ian.campbell, xen-devel
Wei Liu writes ("[PATCH V3 13/13] xl: introduce "xl-json" format"):
> Originally xl verbatimly copies the domain config file in its user data
> store. This patch adds the functionality to transform text domain config
> file to JSON object and save that object in user data store.
As I said on IRC, I think you should explain your plans for this. I
approve of your plans :-).
> What this patch does:
> 1. add a mandatory flag to save protocol to indicate whether the saved
> config is a JSON object
> 2. register a new private data type "xl-json" in libxl.h
> 3. modify xl to save / load "xl-json" file where necessary
>
> After this change xl supports both "xl" format and "xl-json" format.
> The user-supplied config file is still restricted to normal text config
> file format ("xl"), as xl has more sanity checks when parsing text
> config file. "xl-json" format is only used internally. The saved config
> file is always in "xl-json" format.
I think it would be very useful to allow people to
- retreive the domain's json config
- start a new domain with a json config
That would let you write a program to adjust a domain's config, for
example.
If there is something important error that xl checks for, which can
also be represented as an error in the domain config IDL struct, but
which isn't checked for by libxl, I think that's a bug in libxl.
> Tests done so far (xl.{new,old} denotes xl with{,out} "xl_json"
> support):
> 1. xl.new create then hexdump saved file: domain config saved in JSON format
> 2. xl.new create then xl.old restore: failed on mandatory flag check
> 3. xl.new create then xl.new restore: succeeded
> 4. xl.old create then xl.new restore: succeeded
> 5. xl.new create then local migrate, receiving end xl.new: succeeded
> 6. xl.old create then local migrate, receiving end xl.new: succeeded
Great, thanks.
Ian.
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: [PATCH V3 13/13] xl: introduce "xl-json" format
2014-04-24 16:11 ` Ian Jackson
@ 2014-04-25 15:53 ` Wei Liu
0 siblings, 0 replies; 20+ messages in thread
From: Wei Liu @ 2014-04-25 15:53 UTC (permalink / raw)
To: Ian Jackson; +Cc: Wei Liu, ian.campbell, xen-devel
On Thu, Apr 24, 2014 at 05:11:11PM +0100, Ian Jackson wrote:
> Wei Liu writes ("[PATCH V3 13/13] xl: introduce "xl-json" format"):
> > Originally xl verbatimly copies the domain config file in its user data
> > store. This patch adds the functionality to transform text domain config
> > file to JSON object and save that object in user data store.
>
> As I said on IRC, I think you should explain your plans for this. I
> approve of your plans :-).
>
> > What this patch does:
> > 1. add a mandatory flag to save protocol to indicate whether the saved
> > config is a JSON object
> > 2. register a new private data type "xl-json" in libxl.h
> > 3. modify xl to save / load "xl-json" file where necessary
> >
> > After this change xl supports both "xl" format and "xl-json" format.
> > The user-supplied config file is still restricted to normal text config
> > file format ("xl"), as xl has more sanity checks when parsing text
> > config file. "xl-json" format is only used internally. The saved config
> > file is always in "xl-json" format.
>
> I think it would be very useful to allow people to
> - retreive the domain's json config
This can already be done. Just that the JSON config is parsed from
stored config file and not up to date.
> - start a new domain with a json config
This is doable.
> That would let you write a program to adjust a domain's config, for
> example.
>
> If there is something important error that xl checks for, which can
> also be represented as an error in the domain config IDL struct, but
> which isn't checked for by libxl, I think that's a bug in libxl.
>
OK, got it. The parser should be able to catch illegal input and aborts.
The concern is that the input is legal but not meaningful to libxl, say,
some fileds require a string and that's the only thing parser checks.
But since you point out any breakage caused by illegal input beyond
parser's scope is classfied as libxl bug, then I think my concern on
sanity check can go away.
Wei.
> > Tests done so far (xl.{new,old} denotes xl with{,out} "xl_json"
> > support):
> > 1. xl.new create then hexdump saved file: domain config saved in JSON format
> > 2. xl.new create then xl.old restore: failed on mandatory flag check
> > 3. xl.new create then xl.new restore: succeeded
> > 4. xl.old create then xl.new restore: succeeded
> > 5. xl.new create then local migrate, receiving end xl.new: succeeded
> > 6. xl.old create then local migrate, receiving end xl.new: succeeded
>
> Great, thanks.
>
> Ian.
^ permalink raw reply [flat|nested] 20+ messages in thread