* [Qemu-devel] Add infrastructure for supporting QIDL annotations
@ 2012-08-14 16:27 Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 01/20] qapi: qapi-visit.py -> qapi_visit.py so we can import Michael Roth
` (19 more replies)
0 siblings, 20 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
These patches are based are origin/master, and can also be obtained from:
git://github.com/mdroth/qemu.git qidl-base
This is a cleanup of the infrastructure bits from
"[RFC v2] Use QEMU IDL for device serialization/introspection".
I know this is pretty late into the release cycle, but the patches in
this series have gotten decent review, and I have a seperate branch (qidl-conv1
in my github repo) that uses these bits to serialize i440fx, piix3/piix ide,
usb, cirrus_vga, and a few others via QIDL. So there shouldn't be too much
churn going forward.
Changes since rfc v2:
- Parser/Codegen fix-ups for cases encountered converting piix ide and usb.
- Fixed license headers.
- Stricter arg-checking for QIDL macros when passing to codegen.
- Added asserts to detect mis-use of QIDL property/visitor interfaces.
- Renamed QAPI visit_*_array interfaces to visit_*_carray to clarify that
these are serialization routines for single-dimension C arrays.
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 01/20] qapi: qapi-visit.py -> qapi_visit.py so we can import
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 02/20] qapi: qapi-types.py -> qapi_types.py Michael Roth
` (18 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Python doesn't allow "-" in module names, so we need to rename the file
so we can re-use bits of the codegen
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
Makefile | 8 ++++----
scripts/{qapi-visit.py => qapi_visit.py} | 0
tests/Makefile | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
rename scripts/{qapi-visit.py => qapi_visit.py} (100%)
diff --git a/Makefile b/Makefile
index d736ea5..08c7d0e 100644
--- a/Makefile
+++ b/Makefile
@@ -187,8 +187,8 @@ qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
-$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi_visit.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
@@ -197,8 +197,8 @@ qapi-types.c qapi-types.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o "." < $<, " GEN $@")
qapi-visit.c qapi-visit.h :\
-$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi_visit.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o "." < $<, " GEN $@")
qmp-commands.h qmp-marshal.c :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -m -o "." < $<, " GEN $@")
diff --git a/scripts/qapi-visit.py b/scripts/qapi_visit.py
similarity index 100%
rename from scripts/qapi-visit.py
rename to scripts/qapi_visit.py
diff --git a/tests/Makefile b/tests/Makefile
index f3f4159..3c05bdf 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -55,8 +55,8 @@ tests/test-qapi-types.c tests/test-qapi-types.h :\
$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
tests/test-qapi-visit.c tests/test-qapi-visit.h :\
-$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-visit.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi_visit.py
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
tests/test-qmp-commands.h tests/test-qmp-marshal.c :\
$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 02/20] qapi: qapi-types.py -> qapi_types.py
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 01/20] qapi: qapi-visit.py -> qapi_visit.py so we can import Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 03/20] qapi: qapi-commands.py -> qapi_commands.py Michael Roth
` (17 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Python doesn't allow "-" in module names, so we need to rename the file
so we can re-use bits of the codegen
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
Makefile | 8 ++++----
scripts/{qapi-types.py => qapi_types.py} | 0
tests/Makefile | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
rename scripts/{qapi-types.py => qapi_types.py} (100%)
diff --git a/Makefile b/Makefile
index 08c7d0e..e92ea6b 100644
--- a/Makefile
+++ b/Makefile
@@ -184,8 +184,8 @@ endif
qapi-py = $(SRC_PATH)/scripts/qapi.py $(SRC_PATH)/scripts/ordereddict.py
qga/qapi-generated/qga-qapi-types.c qga/qapi-generated/qga-qapi-types.h :\
-$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi_types.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_types.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi_visit.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
@@ -194,8 +194,8 @@ $(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-p
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
qapi-types.c qapi-types.h :\
-$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o "." < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi_types.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_types.py $(gen-out-type) -o "." < $<, " GEN $@")
qapi-visit.c qapi-visit.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi_visit.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o "." < $<, " GEN $@")
diff --git a/scripts/qapi-types.py b/scripts/qapi_types.py
similarity index 100%
rename from scripts/qapi-types.py
rename to scripts/qapi_types.py
diff --git a/tests/Makefile b/tests/Makefile
index 3c05bdf..156105c 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -52,8 +52,8 @@ tests/test-coroutine$(EXESUF): tests/test-coroutine.o $(coroutine-obj-y) $(tools
tests/test-iov$(EXESUF): tests/test-iov.o iov.o
tests/test-qapi-types.c tests/test-qapi-types.h :\
-$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi_types.py
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_types.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
tests/test-qapi-visit.c tests/test-qapi-visit.h :\
$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi_visit.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 03/20] qapi: qapi-commands.py -> qapi_commands.py
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 01/20] qapi: qapi-visit.py -> qapi_visit.py so we can import Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 02/20] qapi: qapi-types.py -> qapi_types.py Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 04/20] qapi: qapi_visit.py, make code useable as module Michael Roth
` (16 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Python doesn't allow "-" in module names, so we need to rename the file
so we can re-use bits of the codegen.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
Makefile | 8 ++++----
scripts/{qapi-commands.py => qapi_commands.py} | 0
tests/Makefile | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
rename scripts/{qapi-commands.py => qapi_commands.py} (100%)
diff --git a/Makefile b/Makefile
index e92ea6b..7733e4c 100644
--- a/Makefile
+++ b/Makefile
@@ -190,8 +190,8 @@ qga/qapi-generated/qga-qapi-visit.c qga/qapi-generated/qga-qapi-visit.h :\
$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi_visit.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
qga/qapi-generated/qga-qmp-commands.h qga/qapi-generated/qga-qmp-marshal.c :\
-$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema-guest.json $(SRC_PATH)/scripts/qapi_commands.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_commands.py $(gen-out-type) -o qga/qapi-generated -p "qga-" < $<, " GEN $@")
qapi-types.c qapi-types.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi_types.py $(qapi-py)
@@ -200,8 +200,8 @@ qapi-visit.c qapi-visit.h :\
$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi_visit.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o "." < $<, " GEN $@")
qmp-commands.h qmp-marshal.c :\
-$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -m -o "." < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi_commands.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_commands.py $(gen-out-type) -m -o "." < $<, " GEN $@")
QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
$(qga-obj-y) qemu-ga.o: $(QGALIB_GEN)
diff --git a/scripts/qapi-commands.py b/scripts/qapi_commands.py
similarity index 100%
rename from scripts/qapi-commands.py
rename to scripts/qapi_commands.py
diff --git a/tests/Makefile b/tests/Makefile
index 156105c..ec6a050 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -58,8 +58,8 @@ tests/test-qapi-visit.c tests/test-qapi-visit.h :\
$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi_visit.py
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_visit.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
tests/test-qmp-commands.h tests/test-qmp-marshal.c :\
-$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-commands.py
- $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
+$(SRC_PATH)/qapi-schema-test.json $(SRC_PATH)/scripts/qapi_commands.py
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi_commands.py $(gen-out-type) -o tests -p "test-" < $<, " GEN $@")
tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 04/20] qapi: qapi_visit.py, make code useable as module
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (2 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 03/20] qapi: qapi-commands.py -> qapi_commands.py Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 05/20] qapi: qapi_visit.py, support arrays and complex qapi definitions Michael Roth
` (15 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qapi_visit.py | 143 +++++++++++++++++++++++++------------------------
1 file changed, 74 insertions(+), 69 deletions(-)
diff --git a/scripts/qapi_visit.py b/scripts/qapi_visit.py
index 04ef7c4..25707f5 100644
--- a/scripts/qapi_visit.py
+++ b/scripts/qapi_visit.py
@@ -224,55 +224,57 @@ void visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **e
''',
name=name)
-try:
- opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:o:",
- ["source", "header", "prefix=", "output-dir="])
-except getopt.GetoptError, err:
- print str(err)
- sys.exit(1)
-
-output_dir = ""
-prefix = ""
-c_file = 'qapi-visit.c'
-h_file = 'qapi-visit.h'
-
-do_c = False
-do_h = False
-
-for o, a in opts:
- if o in ("-p", "--prefix"):
- prefix = a
- elif o in ("-o", "--output-dir"):
- output_dir = a + "/"
- elif o in ("-c", "--source"):
+def main(argv=[]):
+ try:
+ opts, args = getopt.gnu_getopt(sys.argv[1:], "chp:o:",
+ ["source", "header", "prefix=",
+ "output-dir="])
+ except getopt.GetoptError, err:
+ print str(err)
+ sys.exit(1)
+
+ output_dir = ""
+ prefix = ""
+ c_file = 'qapi-visit.c'
+ h_file = 'qapi-visit.h'
+
+ do_c = False
+ do_h = False
+
+ for o, a in opts:
+ if o in ("-p", "--prefix"):
+ prefix = a
+ elif o in ("-o", "--output-dir"):
+ output_dir = a + "/"
+ elif o in ("-c", "--source"):
+ do_c = True
+ elif o in ("-h", "--header"):
+ do_h = True
+
+ if not do_c and not do_h:
do_c = True
- elif o in ("-h", "--header"):
do_h = True
-if not do_c and not do_h:
- do_c = True
- do_h = True
+ c_file = output_dir + prefix + c_file
+ h_file = output_dir + prefix + h_file
-c_file = output_dir + prefix + c_file
-h_file = output_dir + prefix + h_file
+ try:
+ os.makedirs(output_dir)
+ except os.error, e:
+ if e.errno != errno.EEXIST:
+ raise
-try:
- os.makedirs(output_dir)
-except os.error, e:
- if e.errno != errno.EEXIST:
- raise
-
-def maybe_open(really, name, opt):
- if really:
- return open(name, opt)
- else:
- import StringIO
- return StringIO.StringIO()
+ def maybe_open(really, name, opt):
+ if really:
+ return open(name, opt)
+ else:
+ import StringIO
+ return StringIO.StringIO()
-fdef = maybe_open(do_c, c_file, 'w')
-fdecl = maybe_open(do_h, h_file, 'w')
+ fdef = maybe_open(do_c, c_file, 'w')
+ fdecl = maybe_open(do_h, h_file, 'w')
-fdef.write(mcgen('''
+ fdef.write(mcgen('''
/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */
/*
@@ -292,7 +294,7 @@ fdef.write(mcgen('''
''',
header=basename(h_file)))
-fdecl.write(mcgen('''
+ fdecl.write(mcgen('''
/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */
/*
@@ -316,37 +318,40 @@ fdecl.write(mcgen('''
''',
prefix=prefix, guard=guardname(h_file)))
-exprs = parse_schema(sys.stdin)
+ exprs = parse_schema(sys.stdin)
-for expr in exprs:
- if expr.has_key('type'):
- ret = generate_visit_struct(expr['type'], expr['data'])
- ret += generate_visit_list(expr['type'], expr['data'])
- fdef.write(ret)
+ for expr in exprs:
+ if expr.has_key('type'):
+ ret = generate_visit_struct(expr['type'], expr['data'])
+ ret += generate_visit_list(expr['type'], expr['data'])
+ fdef.write(ret)
- ret = generate_declaration(expr['type'], expr['data'])
- fdecl.write(ret)
- elif expr.has_key('union'):
- ret = generate_visit_union(expr['union'], expr['data'])
- ret += generate_visit_list(expr['union'], expr['data'])
- fdef.write(ret)
+ ret = generate_declaration(expr['type'], expr['data'])
+ fdecl.write(ret)
+ elif expr.has_key('union'):
+ ret = generate_visit_union(expr['union'], expr['data'])
+ ret += generate_visit_list(expr['union'], expr['data'])
+ fdef.write(ret)
- ret = generate_decl_enum('%sKind' % expr['union'], expr['data'].keys())
- ret += generate_declaration(expr['union'], expr['data'])
- fdecl.write(ret)
- elif expr.has_key('enum'):
- ret = generate_visit_enum(expr['enum'], expr['data'])
- fdef.write(ret)
+ ret = generate_decl_enum('%sKind' % expr['union'], expr['data'].keys())
+ ret += generate_declaration(expr['union'], expr['data'])
+ fdecl.write(ret)
+ elif expr.has_key('enum'):
+ ret = generate_visit_enum(expr['enum'], expr['data'])
+ fdef.write(ret)
- ret = generate_decl_enum(expr['enum'], expr['data'])
- fdecl.write(ret)
+ ret = generate_decl_enum(expr['enum'], expr['data'])
+ fdecl.write(ret)
-fdecl.write('''
+ fdecl.write('''
#endif
-''')
+ ''')
+
+ fdecl.flush()
+ fdecl.close()
-fdecl.flush()
-fdecl.close()
+ fdef.flush()
+ fdef.close()
-fdef.flush()
-fdef.close()
+if __name__ == '__main__':
+ sys.exit(main(sys.argv))
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 05/20] qapi: qapi_visit.py, support arrays and complex qapi definitions
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (3 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 04/20] qapi: qapi_visit.py, make code useable as module Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 06/20] qapi: add visitor interfaces for C arrays Michael Roth
` (14 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Add support for arrays in the code generators.
Complex field descriptions can now be used to provide additional
information to the visitor generators, such as the max size of an array,
or the field within a struct to use to determine how many elements are
present in the array to avoid serializing uninitialized elements.
Add handling for these in the code generators as well.
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qapi.py | 8 ++++--
scripts/qapi_commands.py | 8 +++---
scripts/qapi_types.py | 2 +-
scripts/qapi_visit.py | 64 +++++++++++++++++++++++++++++++++++++++++-----
4 files changed, 68 insertions(+), 14 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 122b4cb..a347203 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -110,12 +110,16 @@ def parse_args(typeinfo):
argentry = typeinfo[member]
optional = False
structured = False
+ annotated = False
if member.startswith('*'):
argname = member[1:]
optional = True
if isinstance(argentry, OrderedDict):
- structured = True
- yield (argname, argentry, optional, structured)
+ if argentry.has_key('<annotated>'):
+ annotated = True
+ else:
+ structured = True
+ yield (argname, argentry, optional, structured, annotated)
def de_camel_case(name):
new_name = ''
diff --git a/scripts/qapi_commands.py b/scripts/qapi_commands.py
index 3c4678d..a2e0c23 100644
--- a/scripts/qapi_commands.py
+++ b/scripts/qapi_commands.py
@@ -32,7 +32,7 @@ void %(visitor)s(Visitor *m, %(name)s * obj, const char *name, Error **errp);
def generate_command_decl(name, args, ret_type):
arglist=""
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional, structured, annotated in parse_args(args):
argtype = c_type(argtype)
if argtype == "char *":
argtype = "const char *"
@@ -50,7 +50,7 @@ def gen_sync_call(name, args, ret_type, indent=0):
retval=""
if ret_type:
retval = "retval = "
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional, structured, annotated in parse_args(args):
if optional:
arglist += "has_%s, " % c_var(argname)
arglist += "%s, " % (c_var(argname))
@@ -106,7 +106,7 @@ Visitor *v;
def gen_visitor_input_vars_decl(args):
ret = ""
push_indent()
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional, structured, annotated in parse_args(args):
if optional:
ret += mcgen('''
bool has_%(argname)s = false;
@@ -145,7 +145,7 @@ v = qmp_input_get_visitor(mi);
''',
obj=obj)
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional, structured, annotated in parse_args(args):
if optional:
ret += mcgen('''
visit_start_optional(v, &has_%(c_name)s, "%(name)s", errp);
diff --git a/scripts/qapi_types.py b/scripts/qapi_types.py
index cf601ae..8babfe1 100644
--- a/scripts/qapi_types.py
+++ b/scripts/qapi_types.py
@@ -35,7 +35,7 @@ struct %(name)s
''',
name=structname)
- for argname, argentry, optional, structured in parse_args(members):
+ for argname, argentry, optional, structured, annotated in parse_args(members):
if optional:
ret += mcgen('''
bool has_%(c_name)s;
diff --git a/scripts/qapi_visit.py b/scripts/qapi_visit.py
index 25707f5..aac2172 100644
--- a/scripts/qapi_visit.py
+++ b/scripts/qapi_visit.py
@@ -16,6 +16,52 @@ import sys
import os
import getopt
import errno
+import types
+
+def generate_visit_carray_body(name, info):
+ if info['array_size'][0].isdigit():
+ array_size = info['array_size']
+ elif info['array_size'][0] == '(' and info['array_size'][-1] == ')':
+ array_size = info['array_size']
+ else:
+ array_size = "(*obj)->%s" % info['array_size']
+
+ if info.has_key('array_capacity'):
+ array_capacity = info['array_capacity']
+ else:
+ array_capacity = array_size
+
+ if camel_case(de_camel_case(info['type'][0])) == info['type'][0]:
+ ret = mcgen('''
+visit_start_carray(m, (void **)obj, "%(name)s", %(array_capacity)s, sizeof(%(type)s), errp);
+int %(name)s_i;
+for (%(name)s_i = 0; %(name)s_i < %(array_size)s; %(name)s_i++) {
+ %(type)s %(name)s_ptr = &(*obj)->%(name)s[%(name)s_i];
+ visit_type_%(type_short)s(m, &%(name)s_ptr, NULL, errp);
+ visit_next_carray(m, errp);
+}
+visit_end_carray(m, errp);
+''',
+ name=name, type=c_type(info['type'][0]),
+ type_short=info['type'][0],
+ array_size=array_size,
+ array_capacity=array_capacity)
+ else:
+ ret = mcgen('''
+visit_start_carray(m, (void **)obj, "%(name)s", %(array_capacity)s, sizeof(%(type)s), errp);
+int %(name)s_i;
+for (%(name)s_i = 0; %(name)s_i < %(array_size)s; %(name)s_i++) {
+ %(type)s *%(name)s_ptr = (%(type)s *)&(*obj)->%(name)s[%(name)s_i];
+ visit_next_carray(m, errp);
+ visit_type_%(type_short)s(m, %(name)s_ptr, NULL, errp);
+}
+visit_end_carray(m, errp);
+''',
+ name=name, type=c_type(info['type'][0]),
+ type_short=info['type'][0],
+ array_size=array_size,
+ array_capacity=array_capacity)
+ return ret
def generate_visit_struct_body(field_prefix, name, members):
ret = mcgen('''
@@ -45,10 +91,10 @@ if (!err) {
push_indent()
push_indent()
- for argname, argentry, optional, structured in parse_args(members):
+ for argname, argentry, optional, structured, annotated in parse_args(members):
if optional:
ret += mcgen('''
-visit_start_optional(m, obj ? &(*obj)->%(c_prefix)shas_%(c_name)s : NULL, "%(name)s", &err);
+visit_start_optional(m, obj ? &(*obj)->%(c_prefix)shas_%(c_name)s : NULL, "%(name)s", errp);
if (obj && (*obj)->%(prefix)shas_%(c_name)s) {
''',
c_prefix=c_var(field_prefix), prefix=field_prefix,
@@ -58,12 +104,16 @@ if (obj && (*obj)->%(prefix)shas_%(c_name)s) {
if structured:
ret += generate_visit_struct_body(field_prefix + argname, argname, argentry)
else:
- ret += mcgen('''
-visit_type_%(type)s(m, obj ? &(*obj)->%(c_prefix)s%(c_name)s : NULL, "%(name)s", &err);
+ if annotated:
+ if isinstance(argentry['type'], types.ListType):
+ ret += generate_visit_carray_body(argname, argentry)
+ else:
+ ret += mcgen('''
+visit_type_%(type)s(m, obj ? &(*obj)->%(c_prefix)s%(c_name)s : NULL, "%(name)s", errp);
''',
- c_prefix=c_var(field_prefix), prefix=field_prefix,
- type=type_name(argentry), c_name=c_var(argname),
- name=argname)
+ c_prefix=c_var(field_prefix), prefix=field_prefix,
+ type=type_name(argentry), c_name=c_var(argname),
+ name=argname)
if optional:
pop_indent()
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 06/20] qapi: add visitor interfaces for C arrays
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (4 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 05/20] qapi: qapi_visit.py, support arrays and complex qapi definitions Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 07/20] qapi: qapi_visit.py, support generating static functions Michael Roth
` (13 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Generally these will be serialized into lists, but the
representation can be of any form so long as it can
be deserialized into a single-dimension C array.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qapi/qapi-visit-core.c | 25 +++++++++++++++++++++++++
qapi/qapi-visit-core.h | 8 ++++++++
2 files changed, 33 insertions(+)
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 7a82b63..9a74ed0 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -311,3 +311,28 @@ void input_type_enum(Visitor *v, int *obj, const char *strings[],
g_free(enum_str);
*obj = value;
}
+
+void visit_start_carray(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp)
+{
+ g_assert(v->start_carray);
+ if (!error_is_set(errp)) {
+ v->start_carray(v, obj, name, elem_count, elem_size, errp);
+ }
+}
+
+void visit_next_carray(Visitor *v, Error **errp)
+{
+ g_assert(v->next_carray);
+ if (!error_is_set(errp)) {
+ v->next_carray(v, errp);
+ }
+}
+
+void visit_end_carray(Visitor *v, Error **errp)
+{
+ g_assert(v->end_carray);
+ if (!error_is_set(errp)) {
+ v->end_carray(v, errp);
+ }
+}
diff --git a/qapi/qapi-visit-core.h b/qapi/qapi-visit-core.h
index 60aceda..5eb1616 100644
--- a/qapi/qapi-visit-core.h
+++ b/qapi/qapi-visit-core.h
@@ -43,6 +43,10 @@ struct Visitor
void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
void (*type_number)(Visitor *v, double *obj, const char *name,
Error **errp);
+ void (*start_carray)(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp);
+ void (*next_carray)(Visitor *v, Error **errp);
+ void (*end_carray)(Visitor *v, Error **errp);
/* May be NULL */
void (*start_optional)(Visitor *v, bool *present, const char *name,
@@ -91,5 +95,9 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
+void visit_start_carray(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp);
+void visit_next_carray(Visitor *v, Error **errp);
+void visit_end_carray(Visitor *v, Error **errp);
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 07/20] qapi: qapi_visit.py, support generating static functions
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (5 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 06/20] qapi: add visitor interfaces for C arrays Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 08/20] qapi: qapi_visit.py, support for visiting non-pointer/embedded structs Michael Roth
` (12 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
qidl embeds visitor code into object files rather than linking against
seperate files, so allow for static declarations when we're using
qapi_visit.py as a library as we do with qidl.py
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qapi_visit.py | 51 ++++++++++++++++++++++++++++++++-----------------
1 file changed, 33 insertions(+), 18 deletions(-)
diff --git a/scripts/qapi_visit.py b/scripts/qapi_visit.py
index aac2172..d146013 100644
--- a/scripts/qapi_visit.py
+++ b/scripts/qapi_visit.py
@@ -141,13 +141,16 @@ visit_end_optional(m, &err);
''')
return ret
-def generate_visit_struct(name, members):
+def generate_visit_struct(name, members, static=False):
+ ret_type = "void"
+ if static:
+ ret_type = "static " + ret_type
ret = mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s ** obj, const char *name, Error **errp)
+%(ret_type)s visit_type_%(name)s(Visitor *m, %(name)s ** obj, const char *name, Error **errp)
{
''',
- name=name)
+ name=name, ret_type=ret_type)
push_indent()
ret += generate_visit_struct_body("", name, members)
@@ -158,10 +161,13 @@ void visit_type_%(name)s(Visitor *m, %(name)s ** obj, const char *name, Error **
''')
return ret
-def generate_visit_list(name, members):
+def generate_visit_list(name, members, static=False):
+ ret_type = "void"
+ if static:
+ ret_type = "static " + ret_type
return mcgen('''
-void visit_type_%(name)sList(Visitor *m, %(name)sList ** obj, const char *name, Error **errp)
+%(ret_type)s visit_type_%(name)sList(Visitor *m, %(name)sList ** obj, const char *name, Error **errp)
{
GenericList *i, **prev = (GenericList **)obj;
Error *err = NULL;
@@ -183,19 +189,22 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList ** obj, const char *name,
}
}
''',
- name=name)
+ name=name, ret_type=ret_type)
-def generate_visit_enum(name, members):
+def generate_visit_enum(name, members, static=False):
+ ret_type = "void"
+ if static:
+ ret_type = "static " + ret_type
return mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **errp)
+%(ret_type)s visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **errp)
{
visit_type_enum(m, (int *)obj, %(name)s_lookup, "%(name)s", name, errp);
}
''',
- name=name)
+ name=name, ret_type=ret_type)
-def generate_visit_union(name, members):
+def generate_visit_union(name, members, static=False):
ret = generate_visit_enum('%sKind' % name, members.keys())
ret += mcgen('''
@@ -252,27 +261,33 @@ void visit_type_%(name)s(Visitor *m, %(name)s ** obj, const char *name, Error **
return ret
-def generate_declaration(name, members, genlist=True):
+def generate_declaration(name, members, genlist=True, static=False):
+ ret_type = "void"
+ if static:
+ ret_type = "static " + ret_type
ret = mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s ** obj, const char *name, Error **errp);
+%(ret_type)s visit_type_%(name)s(Visitor *m, %(name)s ** obj, const char *name, Error **errp);
''',
- name=name)
+ name=name, ret_type=ret_type)
if genlist:
ret += mcgen('''
-void visit_type_%(name)sList(Visitor *m, %(name)sList ** obj, const char *name, Error **errp);
+%(ret_type)s visit_type_%(name)sList(Visitor *m, %(name)sList ** obj, const char *name, Error **errp);
''',
- name=name)
+ name=name, ret_type=ret_type)
return ret
-def generate_decl_enum(name, members, genlist=True):
+def generate_decl_enum(name, members, genlist=True, static=False):
+ ret_type = "void"
+ if static:
+ ret_type = "static " + ret_type
return mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **errp);
+%(ret_type)s visit_type_%(name)s(Visitor *m, %(name)s * obj, const char *name, Error **errp);
''',
- name=name)
+ name=name, ret_type=ret_type)
def main(argv=[]):
try:
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 08/20] qapi: qapi_visit.py, support for visiting non-pointer/embedded structs
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (6 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 07/20] qapi: qapi_visit.py, support generating static functions Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 09/20] qapi: QmpOutputVisitor, implement array handling Michael Roth
` (11 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qapi_visit.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/scripts/qapi_visit.py b/scripts/qapi_visit.py
index d146013..ab44f11 100644
--- a/scripts/qapi_visit.py
+++ b/scripts/qapi_visit.py
@@ -107,6 +107,15 @@ if (obj && (*obj)->%(prefix)shas_%(c_name)s) {
if annotated:
if isinstance(argentry['type'], types.ListType):
ret += generate_visit_carray_body(argname, argentry)
+ elif argentry.has_key('<embedded struct>'):
+ tmp_ptr_name = "%s_%s_ptr" % (c_var(field_prefix).replace(".", ""), c_var(argname))
+ ret += mcgen('''
+%(type)s *%(tmp_ptr)s = &(*obj)->%(c_prefix)s%(c_name)s;
+visit_type_%(type)s(m, (obj && *obj) ? &%(tmp_ptr)s : NULL, "%(name)s", errp);
+''',
+ c_prefix=c_var(field_prefix), prefix=field_prefix,
+ type=type_name(argentry['type']), c_name=c_var(argname),
+ name=argname, tmp_ptr=tmp_ptr_name)
else:
ret += mcgen('''
visit_type_%(type)s(m, obj ? &(*obj)->%(c_prefix)s%(c_name)s : NULL, "%(name)s", errp);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 09/20] qapi: QmpOutputVisitor, implement array handling
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (7 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 08/20] qapi: qapi_visit.py, support for visiting non-pointer/embedded structs Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 10/20] qapi: QmpInputVisitor, " Michael Roth
` (10 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qapi/qmp-output-visitor.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index 2bce9d5..ea08795 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -181,6 +181,23 @@ static void qmp_output_type_number(Visitor *v, double *obj, const char *name,
qmp_output_add(qov, name, qfloat_from_double(*obj));
}
+static void qmp_output_start_carray(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size,
+ Error **errp)
+{
+ qmp_output_start_list(v, name, errp);
+}
+
+static void qmp_output_next_carray(Visitor *v, Error **errp)
+{
+}
+
+static void qmp_output_end_carray(Visitor *v, Error **errp)
+{
+ QmpOutputVisitor *qov = to_qov(v);
+ qmp_output_pop(qov);
+}
+
QObject *qmp_output_get_qobject(QmpOutputVisitor *qov)
{
QObject *obj = qmp_output_first(qov);
@@ -228,6 +245,9 @@ QmpOutputVisitor *qmp_output_visitor_new(void)
v->visitor.type_bool = qmp_output_type_bool;
v->visitor.type_str = qmp_output_type_str;
v->visitor.type_number = qmp_output_type_number;
+ v->visitor.start_carray = qmp_output_start_carray;
+ v->visitor.next_carray = qmp_output_next_carray;
+ v->visitor.end_carray = qmp_output_end_carray;
QTAILQ_INIT(&v->stack);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 10/20] qapi: QmpInputVisitor, implement array handling
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (8 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 09/20] qapi: QmpOutputVisitor, implement array handling Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 11/20] qapi: qapi.py, make json parser more robust Michael Roth
` (9 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qapi/qmp-input-visitor.c | 32 +++++++++++++++++++++++++++++++-
1 file changed, 31 insertions(+), 1 deletion(-)
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 107d8d3..c4388f3 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -132,7 +132,7 @@ static void qmp_input_start_struct(Visitor *v, void **obj, const char *kind,
return;
}
- if (obj) {
+ if (obj && *obj == NULL) {
*obj = g_malloc0(size);
}
}
@@ -274,6 +274,33 @@ static void qmp_input_start_optional(Visitor *v, bool *present,
*present = true;
}
+static void qmp_input_start_carray(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size,
+ Error **errp)
+{
+ if (obj && *obj == NULL) {
+ *obj = g_malloc0(elem_count * elem_size);
+ }
+ qmp_input_start_list(v, name, errp);
+}
+
+static void qmp_input_next_carray(Visitor *v, Error **errp)
+{
+ QmpInputVisitor *qiv = to_qiv(v);
+ StackObject *so = &qiv->stack[qiv->nb_stack - 1];
+
+ if (so->entry == NULL) {
+ so->entry = qlist_first(qobject_to_qlist(so->obj));
+ } else {
+ so->entry = qlist_next(so->entry);
+ }
+}
+
+static void qmp_input_end_carray(Visitor *v, Error **errp)
+{
+ qmp_input_end_list(v, errp);
+}
+
Visitor *qmp_input_get_visitor(QmpInputVisitor *v)
{
return &v->visitor;
@@ -302,6 +329,9 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)
v->visitor.type_str = qmp_input_type_str;
v->visitor.type_number = qmp_input_type_number;
v->visitor.start_optional = qmp_input_start_optional;
+ v->visitor.start_carray = qmp_input_start_carray;
+ v->visitor.next_carray = qmp_input_next_carray;
+ v->visitor.end_carray = qmp_input_end_carray;
qmp_input_push(v, obj, NULL);
qobject_incref(obj);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 11/20] qapi: qapi.py, make json parser more robust
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (9 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 10/20] qapi: QmpInputVisitor, " Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 12/20] qapi: add open-coded visitor for struct tm types Michael Roth
` (8 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Currently the QAPI JSON parser expects a very particular style of code
indentation, the major one being that terminating curly/square brackets are
not on placed on a seperate line. This is incompatible with most
pretty-print formats, so make it a little more robust by supporting
these cases.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qapi.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a347203..47cd672 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -81,7 +81,7 @@ def parse_schema(fp):
if line.startswith('#') or line == '\n':
continue
- if line.startswith(' '):
+ if line[0] in ['}', ']', ' ', '\t']:
expr += line
elif expr:
expr_eval = evaluate(expr)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 12/20] qapi: add open-coded visitor for struct tm types
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (10 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 11/20] qapi: qapi.py, make json parser more robust Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 13/20] qom-fuse: workaround for truncated properties > 4096 Michael Roth
` (7 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qapi/Makefile.objs | 1 +
qapi/misc-qapi-visit.c | 14 ++++++++++++++
qapi/qapi-visit-core.h | 3 +++
3 files changed, 18 insertions(+)
create mode 100644 qapi/misc-qapi-visit.c
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 5f5846e..7604b52 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,3 +1,4 @@
qapi-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
qapi-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
qapi-obj-y += string-input-visitor.o string-output-visitor.o opts-visitor.o
+qapi-obj-y += misc-qapi-visit.o
diff --git a/qapi/misc-qapi-visit.c b/qapi/misc-qapi-visit.c
new file mode 100644
index 0000000..a44773d
--- /dev/null
+++ b/qapi/misc-qapi-visit.c
@@ -0,0 +1,14 @@
+#include <time.h>
+#include "qidl.h"
+
+void visit_type_tm(Visitor *v, struct tm *obj, const char *name, Error **errp)
+{
+ visit_start_struct(v, NULL, "struct tm", name, 0, errp);
+ visit_type_int32(v, &obj->tm_year, "tm_year", errp);
+ visit_type_int32(v, &obj->tm_mon, "tm_mon", errp);
+ visit_type_int32(v, &obj->tm_mday, "tm_mday", errp);
+ visit_type_int32(v, &obj->tm_hour, "tm_hour", errp);
+ visit_type_int32(v, &obj->tm_min, "tm_min", errp);
+ visit_type_int32(v, &obj->tm_sec, "tm_sec", errp);
+ visit_end_struct(v, errp);
+}
diff --git a/qapi/qapi-visit-core.h b/qapi/qapi-visit-core.h
index 5eb1616..10ec044 100644
--- a/qapi/qapi-visit-core.h
+++ b/qapi/qapi-visit-core.h
@@ -100,4 +100,7 @@ void visit_start_carray(Visitor *v, void **obj, const char *name,
void visit_next_carray(Visitor *v, Error **errp);
void visit_end_carray(Visitor *v, Error **errp);
+/* misc. visitors */
+void visit_type_tm(Visitor *m, struct tm *obj, const char *name, Error **errp);
+
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 13/20] qom-fuse: workaround for truncated properties > 4096
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (11 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 12/20] qapi: add open-coded visitor for struct tm types Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 14/20] module additions for schema registration Michael Roth
` (6 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
We currently hard-code property size at 4096 for the purposes of
getattr()/stat()/etc. For 'state' properties we can exceed this easily,
leading to truncated responses.
Instead, for a particular property, make it
max(4096, most_recent_property_size * 2). This allows some
head-room for properties that change size periodically (numbers,
strings, state properties containing arrays, etc)
Also, implement a simple property cache to avoid spinning on qom-get
if an application reads beyond the actual size. This also allows us
to use a snapshot of a single qom-get that persists across read()'s.
Old Cache entries are evicted as soon as we attempt to read() from
offset 0 again.
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
QMP/qom-fuse | 24 +++++++++++++++++-------
1 file changed, 17 insertions(+), 7 deletions(-)
diff --git a/QMP/qom-fuse b/QMP/qom-fuse
index 5c6754a..bd43f29 100755
--- a/QMP/qom-fuse
+++ b/QMP/qom-fuse
@@ -26,6 +26,7 @@ class QOMFS(Fuse):
self.qmp.connect()
self.ino_map = {}
self.ino_count = 1
+ self.prop_cache = {}
def get_ino(self, path):
if self.ino_map.has_key(path):
@@ -67,12 +68,16 @@ class QOMFS(Fuse):
if not self.is_property(path):
return -ENOENT
- path, prop = path.rsplit('/', 1)
- try:
- data = str(self.qmp.command('qom-get', path=path, property=prop))
- data += '\n' # make values shell friendly
- except:
- return -EPERM
+ # avoid extra calls to qom-get by using cached value when offset > 0
+ if offset == 0 or not self.prop_cache.has_key(path):
+ directory, prop = path.rsplit('/', 1)
+ try:
+ resp = str(self.qmp.command('qom-get', path=directory, property=prop))
+ self.prop_cache[path] = resp + '\n' # make values shell friendly
+ except:
+ return -EPERM
+
+ data = self.prop_cache[path]
if offset > len(data):
return ''
@@ -111,13 +116,18 @@ class QOMFS(Fuse):
0,
0))
elif self.is_property(path):
+ directory, prop = path.rsplit('/', 1)
+ try:
+ resp = str(self.qmp.command('qom-get', path=directory, property=prop))
+ except:
+ return -ENOENT
value = posix.stat_result((0644 | stat.S_IFREG,
self.get_ino(path),
0,
1,
1000,
1000,
- 4096,
+ max(len(resp) * 2, 4096),
0,
0,
0))
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 14/20] module additions for schema registration
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (12 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 13/20] qom-fuse: workaround for truncated properties > 4096 Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 15/20] qdev: move Property-related declarations to qdev-properties.h Michael Roth
` (5 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
module.h | 2 ++
vl.c | 1 +
2 files changed, 3 insertions(+)
diff --git a/module.h b/module.h
index c4ccd57..cb81aa2 100644
--- a/module.h
+++ b/module.h
@@ -25,6 +25,7 @@ typedef enum {
MODULE_INIT_MACHINE,
MODULE_INIT_QAPI,
MODULE_INIT_QOM,
+ MODULE_INIT_QIDL,
MODULE_INIT_MAX
} module_init_type;
@@ -32,6 +33,7 @@ typedef enum {
#define machine_init(function) module_init(function, MODULE_INIT_MACHINE)
#define qapi_init(function) module_init(function, MODULE_INIT_QAPI)
#define type_init(function) module_init(function, MODULE_INIT_QOM)
+#define qidl_init(function) module_init(function, MODULE_INIT_QIDL)
void register_module_init(void (*fn)(void), module_init_type type);
diff --git a/vl.c b/vl.c
index d01256a..7895e9d 100644
--- a/vl.c
+++ b/vl.c
@@ -2358,6 +2358,7 @@ int main(int argc, char **argv, char **envp)
}
module_call_init(MODULE_INIT_QOM);
+ module_call_init(MODULE_INIT_QIDL);
runstate_init();
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 15/20] qdev: move Property-related declarations to qdev-properties.h
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (13 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 14/20] module additions for schema registration Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 16/20] qidl: Add documentation Michael Roth
` (4 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
hw/qdev-properties.h | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++
hw/qdev.h | 126 +----------------------------------------
2 files changed, 152 insertions(+), 125 deletions(-)
create mode 100644 hw/qdev-properties.h
diff --git a/hw/qdev-properties.h b/hw/qdev-properties.h
new file mode 100644
index 0000000..ce79a79
--- /dev/null
+++ b/hw/qdev-properties.h
@@ -0,0 +1,151 @@
+/*
+ * Property definitions for qdev
+ *
+ * Copyright (c) 2009 CodeSourcery
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QDEV_PROPERTIES_H
+#define QDEV_PROPERTIES_H
+
+#include "qemu/object.h"
+#include "qemu-queue.h"
+
+typedef struct Property Property;
+
+typedef struct PropertyInfo PropertyInfo;
+
+typedef struct CompatProperty CompatProperty;
+
+struct Property {
+ const char *name;
+ PropertyInfo *info;
+ int offset;
+ uint8_t bitnr;
+ uint8_t qtype;
+ int64_t defval;
+};
+
+struct PropertyInfo {
+ const char *name;
+ const char *legacy_name;
+ const char **enum_table;
+ int (*parse)(DeviceState *dev, Property *prop, const char *str);
+ int (*print)(DeviceState *dev, Property *prop, char *dest, size_t len);
+ ObjectPropertyAccessor *get;
+ ObjectPropertyAccessor *set;
+ ObjectPropertyRelease *release;
+};
+
+typedef struct GlobalProperty {
+ const char *driver;
+ const char *property;
+ const char *value;
+ QTAILQ_ENTRY(GlobalProperty) next;
+} GlobalProperty;
+
+extern PropertyInfo qdev_prop_bit;
+extern PropertyInfo qdev_prop_uint8;
+extern PropertyInfo qdev_prop_uint16;
+extern PropertyInfo qdev_prop_uint32;
+extern PropertyInfo qdev_prop_int32;
+extern PropertyInfo qdev_prop_uint64;
+extern PropertyInfo qdev_prop_hex8;
+extern PropertyInfo qdev_prop_hex32;
+extern PropertyInfo qdev_prop_hex64;
+extern PropertyInfo qdev_prop_string;
+extern PropertyInfo qdev_prop_chr;
+extern PropertyInfo qdev_prop_ptr;
+extern PropertyInfo qdev_prop_macaddr;
+extern PropertyInfo qdev_prop_losttickpolicy;
+extern PropertyInfo qdev_prop_bios_chs_trans;
+extern PropertyInfo qdev_prop_drive;
+extern PropertyInfo qdev_prop_netdev;
+extern PropertyInfo qdev_prop_vlan;
+extern PropertyInfo qdev_prop_pci_devfn;
+extern PropertyInfo qdev_prop_blocksize;
+extern PropertyInfo qdev_prop_pci_host_devaddr;
+
+#define DEFINE_PROP(_name, _state, _field, _prop, _type) { \
+ .name = (_name), \
+ .info = &(_prop), \
+ .offset = offsetof(_state, _field) \
+ + type_check(_type,typeof_field(_state, _field)), \
+ }
+#define DEFINE_PROP_DEFAULT(_name, _state, _field, _defval, _prop, _type) { \
+ .name = (_name), \
+ .info = &(_prop), \
+ .offset = offsetof(_state, _field) \
+ + type_check(_type,typeof_field(_state, _field)), \
+ .qtype = QTYPE_QINT, \
+ .defval = (_type)_defval, \
+ }
+#define DEFINE_PROP_BIT(_name, _state, _field, _bit, _defval) { \
+ .name = (_name), \
+ .info = &(qdev_prop_bit), \
+ .bitnr = (_bit), \
+ .offset = offsetof(_state, _field) \
+ + type_check(uint32_t,typeof_field(_state, _field)), \
+ .qtype = QTYPE_QBOOL, \
+ .defval = (bool)_defval, \
+ }
+
+#define DEFINE_PROP_UINT8(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint8, uint8_t)
+#define DEFINE_PROP_UINT16(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint16, uint16_t)
+#define DEFINE_PROP_UINT32(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint32, uint32_t)
+#define DEFINE_PROP_INT32(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_int32, int32_t)
+#define DEFINE_PROP_UINT64(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint64, uint64_t)
+#define DEFINE_PROP_HEX8(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_hex8, uint8_t)
+#define DEFINE_PROP_HEX32(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_hex32, uint32_t)
+#define DEFINE_PROP_HEX64(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_hex64, uint64_t)
+#define DEFINE_PROP_PCI_DEVFN(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
+
+#define DEFINE_PROP_PTR(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_ptr, void*)
+#define DEFINE_PROP_CHR(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_chr, CharDriverState*)
+#define DEFINE_PROP_STRING(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_string, char*)
+#define DEFINE_PROP_NETDEV(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_netdev, NetClientState*)
+#define DEFINE_PROP_VLAN(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_vlan, NetClientState*)
+#define DEFINE_PROP_DRIVE(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_drive, BlockDriverState *)
+#define DEFINE_PROP_MACADDR(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_macaddr, MACAddr)
+#define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_losttickpolicy, \
+ LostTickPolicy)
+#define DEFINE_PROP_BIOS_CHS_TRANS(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_bios_chs_trans, int)
+#define DEFINE_PROP_BLOCKSIZE(_n, _s, _f, _d) \
+ DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_blocksize, uint16_t)
+#define DEFINE_PROP_PCI_HOST_DEVADDR(_n, _s, _f) \
+ DEFINE_PROP(_n, _s, _f, qdev_prop_pci_host_devaddr, PCIHostDeviceAddress)
+
+#define DEFINE_PROP_END_OF_LIST() \
+ {}
+
+#endif
diff --git a/hw/qdev.h b/hw/qdev.h
index d699194..5e8fb21 100644
--- a/hw/qdev.h
+++ b/hw/qdev.h
@@ -8,10 +8,7 @@
#include "qapi/qapi-visit-core.h"
#include "qemu/object.h"
#include "error.h"
-
-typedef struct Property Property;
-
-typedef struct PropertyInfo PropertyInfo;
+#include "qdev-properties.h"
typedef struct CompatProperty CompatProperty;
@@ -122,33 +119,6 @@ struct BusState {
QLIST_ENTRY(BusState) sibling;
};
-struct Property {
- const char *name;
- PropertyInfo *info;
- int offset;
- uint8_t bitnr;
- uint8_t qtype;
- int64_t defval;
-};
-
-struct PropertyInfo {
- const char *name;
- const char *legacy_name;
- const char **enum_table;
- int (*parse)(DeviceState *dev, Property *prop, const char *str);
- int (*print)(DeviceState *dev, Property *prop, char *dest, size_t len);
- ObjectPropertyAccessor *get;
- ObjectPropertyAccessor *set;
- ObjectPropertyRelease *release;
-};
-
-typedef struct GlobalProperty {
- const char *driver;
- const char *property;
- const char *value;
- QTAILQ_ENTRY(GlobalProperty) next;
-} GlobalProperty;
-
/*** Board API. This should go away once we have a machine config file. ***/
DeviceState *qdev_create(BusState *bus, const char *name);
@@ -215,100 +185,6 @@ void do_info_qdm(Monitor *mon);
int do_device_add(Monitor *mon, const QDict *qdict, QObject **ret_data);
int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data);
-/*** qdev-properties.c ***/
-
-extern PropertyInfo qdev_prop_bit;
-extern PropertyInfo qdev_prop_uint8;
-extern PropertyInfo qdev_prop_uint16;
-extern PropertyInfo qdev_prop_uint32;
-extern PropertyInfo qdev_prop_int32;
-extern PropertyInfo qdev_prop_uint64;
-extern PropertyInfo qdev_prop_hex8;
-extern PropertyInfo qdev_prop_hex32;
-extern PropertyInfo qdev_prop_hex64;
-extern PropertyInfo qdev_prop_string;
-extern PropertyInfo qdev_prop_chr;
-extern PropertyInfo qdev_prop_ptr;
-extern PropertyInfo qdev_prop_macaddr;
-extern PropertyInfo qdev_prop_losttickpolicy;
-extern PropertyInfo qdev_prop_bios_chs_trans;
-extern PropertyInfo qdev_prop_drive;
-extern PropertyInfo qdev_prop_netdev;
-extern PropertyInfo qdev_prop_vlan;
-extern PropertyInfo qdev_prop_pci_devfn;
-extern PropertyInfo qdev_prop_blocksize;
-extern PropertyInfo qdev_prop_pci_host_devaddr;
-
-#define DEFINE_PROP(_name, _state, _field, _prop, _type) { \
- .name = (_name), \
- .info = &(_prop), \
- .offset = offsetof(_state, _field) \
- + type_check(_type,typeof_field(_state, _field)), \
- }
-#define DEFINE_PROP_DEFAULT(_name, _state, _field, _defval, _prop, _type) { \
- .name = (_name), \
- .info = &(_prop), \
- .offset = offsetof(_state, _field) \
- + type_check(_type,typeof_field(_state, _field)), \
- .qtype = QTYPE_QINT, \
- .defval = (_type)_defval, \
- }
-#define DEFINE_PROP_BIT(_name, _state, _field, _bit, _defval) { \
- .name = (_name), \
- .info = &(qdev_prop_bit), \
- .bitnr = (_bit), \
- .offset = offsetof(_state, _field) \
- + type_check(uint32_t,typeof_field(_state, _field)), \
- .qtype = QTYPE_QBOOL, \
- .defval = (bool)_defval, \
- }
-
-#define DEFINE_PROP_UINT8(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint8, uint8_t)
-#define DEFINE_PROP_UINT16(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint16, uint16_t)
-#define DEFINE_PROP_UINT32(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint32, uint32_t)
-#define DEFINE_PROP_INT32(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_int32, int32_t)
-#define DEFINE_PROP_UINT64(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_uint64, uint64_t)
-#define DEFINE_PROP_HEX8(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_hex8, uint8_t)
-#define DEFINE_PROP_HEX32(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_hex32, uint32_t)
-#define DEFINE_PROP_HEX64(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_hex64, uint64_t)
-#define DEFINE_PROP_PCI_DEVFN(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
-
-#define DEFINE_PROP_PTR(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_ptr, void*)
-#define DEFINE_PROP_CHR(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_chr, CharDriverState*)
-#define DEFINE_PROP_STRING(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_string, char*)
-#define DEFINE_PROP_NETDEV(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_netdev, NetClientState*)
-#define DEFINE_PROP_VLAN(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_vlan, NetClientState*)
-#define DEFINE_PROP_DRIVE(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_drive, BlockDriverState *)
-#define DEFINE_PROP_MACADDR(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_macaddr, MACAddr)
-#define DEFINE_PROP_LOSTTICKPOLICY(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_losttickpolicy, \
- LostTickPolicy)
-#define DEFINE_PROP_BIOS_CHS_TRANS(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_bios_chs_trans, int)
-#define DEFINE_PROP_BLOCKSIZE(_n, _s, _f, _d) \
- DEFINE_PROP_DEFAULT(_n, _s, _f, _d, qdev_prop_blocksize, uint16_t)
-#define DEFINE_PROP_PCI_HOST_DEVADDR(_n, _s, _f) \
- DEFINE_PROP(_n, _s, _f, qdev_prop_pci_host_devaddr, PCIHostDeviceAddress)
-
-#define DEFINE_PROP_END_OF_LIST() \
- {}
-
/* Set properties between creation and init. */
void *qdev_get_prop_ptr(DeviceState *dev, Property *prop);
int qdev_prop_parse(DeviceState *dev, const char *name, const char *value);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 16/20] qidl: Add documentation
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (14 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 15/20] qdev: move Property-related declarations to qdev-properties.h Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 19:01 ` Eric Blake
2012-08-14 19:41 ` Peter Maydell
2012-08-14 16:27 ` [Qemu-devel] [PATCH 17/20] qidl: parser, initial import from qc.git Michael Roth
` (3 subsequent siblings)
19 siblings, 2 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
docs/qidl.txt | 343 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 343 insertions(+)
create mode 100644 docs/qidl.txt
diff --git a/docs/qidl.txt b/docs/qidl.txt
new file mode 100644
index 0000000..19976d9
--- /dev/null
+++ b/docs/qidl.txt
@@ -0,0 +1,343 @@
+How to Serialize Device State with QIDL
+======================================
+
+This document describes how to implement save/restore of a device in QEMU using
+the QIDL compiler. The QIDL compiler makes it easier to support live
+migration in devices by converging the serialization description with the
+device type declaration. It has the following features:
+
+ 1. Single description of device state and how to serialize
+
+ 2. Fully inclusive serialization description--fields that aren't serialized
+ are explicitly marked as such including the reason why.
+
+ 3. Optimized for the common case. Even without any special annotations,
+ many devices will Just Work out of the box.
+
+ 4. Build time schema definition. Since QIDL runs at build time, we have full
+ access to the schema during the build which means we can fail the build if
+ the schema breaks.
+
+For the rest, of the document, the following simple device will be used as an
+example.
+
+ typedef struct SerialDevice {
+ SysBusDevice parent;
+
+ uint8_t thr; // transmit holding register
+ uint8_t lsr; // line status register
+ uint8_t ier; // interrupt enable register
+
+ int int_pending; // whether we have a pending queued interrupt
+ CharDriverState *chr; // backend
+ } SerialDevice;
+
+Getting Started
+---------------
+
+The first step is to move your device struct definition to a header file. This
+header file should only contain the struct definition and any preprocessor
+declarations you need to define the structure. This header file will act as
+the source for the QIDL compiler.
+
+Do not include any function declarations in this header file as QIDL does not
+understand function declarations.
+
+Determining What State Gets Saved
+---------------------------------
+
+By default, QIDL saves every field in a structure it sees. This provides maximum
+correctness by default. However, device structures generally contain state
+that reflects state that is in someway duplicated or not guest visible. This
+more often that not reflects design implementation details.
+
+Since design implementation details change over time, saving this state makes
+compatibility hard to maintain since it would effectively lock down a device's
+implementation.
+
+QIDL allows a device author to suppress certain fields from being saved although
+there are very strict rules about when this is allowed and what needs to be done
+to ensure that this does not impact correctness.
+
+There are three cases where state can be suppressed: when it is **immutable**,
+**derived**, or **broken**. In addition, QIDL can decide at run time whether to
+suppress a field by assigning it a **default** value.
+
+## Immutable Fields
+
+If a field is only set during device construction, based on parameters passed to
+the device's constructor, then there is no need to send save and restore this
+value. We call these fields immutable and we tell QIDL about this fact by using
+a **immutable** marker.
+
+In our *SerialDevice* example, the *CharDriverState* pointer reflects the host
+backend that we use to send serial output to the user. This is only assigned
+during device construction and never changes. This means we can add an
+**immutable** marker to it:
+
+ QIDL_START(SerialDevice, state)
+ typedef struct SerialDevice {
+ SysBusDevice parent;
+
+ uint8_t thr; // transmit holding register
+ uint8_t lsr; // line status register
+ uint8_t ier; // interrupt enable register
+
+ int int_pending; // whether we have a pending queued interrupt
+ CharDriverState *chr QIDL(immutable);
+ } SerialDevice;
+ QIDL_END(SerialDevice)
+
+When reviewing patches that make use of the **immutable** marker, the following
+guidelines should be followed to determine if the marker is being used
+correctly.
+
+ 1. Check to see if the field is assigned anywhere other than the device
+ initialization function.
+
+ 2. Check to see if any function is being called that modifies the state of the
+ field outside of the initialization function.
+
+It can be subtle whether a field is truly immutable. A good example is a
+*QEMUTimer*. Timer's will usually have their timeout modified with a call to
+*qemu_mod_timer()* even though they are only assigned in the device
+initialization function.
+
+If the timer is always modified with a fixed value that is not dependent on
+guest state, then the timer is immutable since it's unaffected by the state of
+the guest.
+
+On the other hand, if the timer is modified based on guest state (such as a
+guest programmed time out), then the timer carries state. It may be necessary
+to save/restore the timer or mark it as **derived** and work with it
+accordingly.
+
+### Derived Fields
+
+If a field is set based on some other field in the device's structure, then its
+value is derived. Since this is effectively duplicate state, we can avoid
+sending it and then recompute it when we need to. Derived state requires a bit
+more handling that immutable state.
+
+In our *SerialDevice* example, our *int_pending* flag is really derived from
+two pieces of state. It is set based on whether interrupts are enabled in the
+*ier* register and whether there is *THRE* flag is not set in the *lsr*
+register.
+
+To mark a field as derived, use the **derived** marker. To update our
+example, we would do:
+
+ QIDL_START(SerialDevice, state)
+ typedef struct SerialDevice {
+ SysBusDevice parent;
+
+ uint8_t thr; // transmit holding register
+ uint8_t lsr; // line status register
+ uint8_t ier; // interrupt enable register
+
+ int _derived int_pending; // whether we have a pending queued interrupt
+ CharDriverState *chr QIDL(immutable);
+ } SerialDevice;
+ QIDL_END(SerialDevice)
+
+There is one other critical step needed when marking a field as derived. A
+*post_load* function must be added that updates this field after loading the
+rest of the device state. This function is implemented in the device's source
+file, not in the QIDL header. Below is an example of what this function may do:
+
+ static void serial_post_load(SerialDevice *s)
+ {
+ s->int_pending = !(s->lsr & THRE) && (s->ier & INTE);
+ }
+
+When reviewing a patch that marks a field as *derived*, the following criteria
+should be used:
+
+ 1. Does the device have a post load function?
+
+ 2. Does the post load function assign a value to all of the derived fields?
+
+ 3. Are there any obvious places where a derived field is holding unique state?
+
+### Broken State
+
+QEMU does migration with a lot of devices today. When applying this methodology
+to these devices, one will quickly discover that there are a lot of fields that
+are not being saved today that are not derived or immutable state.
+
+These are all bugs. It just so happens that these bugs are usually not very
+serious. In many cases, they cause small functionality glitches that so far
+have not created any problems.
+
+Consider our *SerialDevice* example. In QEMU's real *SerialState* device, the
+*thr* register is not saved yet we have not marked it immutable or derived.
+
+The *thr* register is a temporary holding register that the next character to
+transmit is placed in while we wait for the next baud cycle. In QEMU, we
+emulate a very fast baud rate regardless of what guest programs. This means
+that the contents of the *thr* register only matter for a very small period of
+time (measured in microseconds).
+
+The likelihood of a migration converging in that very small period of time when
+the *thr* register has a meaningful value is very small. Moreover, the worst
+thing that can happen by not saving this register is that we lose a byte in the
+data stream. Even if this has happened in practice, the chances of someone
+noticing this as a bug is pretty small.
+
+Nonetheless, this is a bug and needs to be eventually fixed. However, it would
+be very inconvenient to constantly break migration by fixing all of these bugs
+one-by-one. Instead, QIDL has a **broken** marker. This indicates that a field
+is not currently saved, but should be in the future.
+
+The idea behind the broken marker is that we can convert a large number of
+devices without breaking migration compatibility, and then institute a flag day
+where we go through and remove broken markers en-mass.
+
+Below is an update of our example to reflect our real life serial device:
+
+ QIDL_START(SerialDevice, state)
+ typedef struct SerialDevice {
+ SysBusDevice parent;
+
+ uint8_t thr QIDL(broken); // transmit holding register
+ uint8_t lsr; // line status register
+ uint8_t ier; // interrupt enable register
+
+ int _derived int_pending; // whether we have a pending queued interrupt
+ CharDriverState _immutable *chr;
+ } SerialDevice;
+ QIDL_END(SerialDevice)
+
+When reviewing the use of the broken marker, the following things should be
+considered:
+
+ 1. What are the ramifications of not sending this data field?
+
+ 2. If the not sending this data field can cause data corruption or very poor
+ behavior within the guest, the broken marker is not appropriate to use.
+
+ 3. Assigning a default value to a field can also be used to fix a broken field
+ without significantly impacting live migration compatibility.
+
+### Default Values
+
+In many cases, a field that gets marked broken was not originally saved because
+in the vast majority of the time, the field does not contain a meaningful value.
+
+In the case of our *thr* example, the field usually does not have a meaningful
+value.
+
+Instead of always saving the field, QIDL has another mechanism that allows the
+field to be saved only when it has a meaningful value. This is done using the
+**default** marker. The default marker tells QIDL that if the field currently
+has a specific value, do not save the value as part of serialization.
+
+When loading a field, QIDL will assign the default value to the field before it
+tries to load the field. If the field cannot be loaded, QIDL will ignore the
+error and rely on the default value.
+
+Using default values, we can fix broken fields while also minimizing the cases
+where we break live migration compatibility. The **default** marker can be
+used in conjunction with the **broken** marker. We can extend our example as
+follows:
+
+ QIDL_START(SerialDevice, state)
+ typedef struct SerialDevice {
+ SysBusDevice parent;
+
+
+ uint8_t thr QIDL(default, 0); // transmit holding register
+ uint8_t lsr; // line status register
+ uint8_t ier; // interrupt enable register
+
+ int _derived int_pending; // whether we have a pending queued interrupt
+ CharDriverState _immutable *chr;
+ } SerialDevice;
+ QIDL_END(SerialFIFO)
+
+The following guidelines should be followed when using a default marker:
+
+ 1. Is the field set to the default value both during device initialization and
+ whenever the field is no longer in use?
+
+ 2. If the non-default value is expected to occur often, then consider using the
+ **broken** marker along with the default marker and using a flag day to
+ remove the **broken** marker.
+
+ 3. In general, setting default values as the value during device initialization
+ is a good idea even if the field was never broken. This gives us maximum
+ flexibility in the long term.
+
+ 4. Never change a default value without renaming a field. The default value is
+ part of the device's ABI.
+
+The first guideline is particularly important. In the case of QEMU's real
+*SerialDevice*, it would be necessary to add code to set the *thr* register to
+zero after the byte has been successfully transmitted. Otherwise, it is
+unlikely that it would ever contain the default value.
+
+Arrays
+------
+
+QIDL has support for multiple types of arrays. The following sections describe
+the different rules for arrays.
+
+Fixed Sized Arrays
+------------------
+
+A fixed sized array has a size that is known at build time. A typical example
+would be:
+
+ QIDL_START(SerialFIFO, state)
+ struct SerialFIFO {
+ uint8_t data[UART_FIFO_LENGTH];
+ uint8_t count;
+ uint8_t itl;
+ uint8_t tail;
+ uint8_t head;
+ };
+ QIDL_END(SerialFIFO)
+
+In this example, *data* is a fixed sized array. No special annotation is needed
+for QIDL to marshal this area correctly. The following guidelines apply to
+fixed sized arrays:
+
+ 1. The size of the array is part of the device ABI. It should not change
+ without renaming the field.
+
+Variable Sized, Fixed Capacity Arrays
+-------------------------------------
+
+Sometimes it's desirable to have a variable sized array. QIDL currently supported
+variable sized arrays provided that the maximum capacity is fixed and part of
+the device structure memory.
+
+A typical example would be a slightly modified version of our above example:
+
+ QIDL_START(SerialFIFO, state)
+ struct SerialFIFO {
+ uint8_t count;
+ uint8_t data[UART_FIFO_LENGTH] QIDL(size_is, count);
+ uint8_t itl;
+ uint8_t tail;
+ uint8_t head;
+ };
+ QIDL_END(SerialFIFO)
+
+In this example, *data* is a variable sized array with a fixed capacity of
+*UART_FIFO_LENGTH*. When we serialize, we want only want to serialize *count*
+members.
+
+The ABI implications of capacity are a bit more relaxed with variable sized
+arrays. In general, you can increase or decrease the capacity without breaking
+the ABI although you may cause some instances of migration to fail between
+versions of QEMU with different capacities.
+
+When reviewing variable sized, fixed capacity arrays, keep the following things
+in mind:
+
+ 1. The variable size must occur before the array element in the state
+ structure.
+
+ 2. The capacity can change without breaking the ABI, but care should be used
+ when making these types of changes.
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 17/20] qidl: parser, initial import from qc.git
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (15 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 16/20] qidl: Add documentation Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 18/20] qidl: codegen, initial commit Michael Roth
` (2 subsequent siblings)
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qidl_parser.py | 512 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 512 insertions(+)
create mode 100644 scripts/qidl_parser.py
diff --git a/scripts/qidl_parser.py b/scripts/qidl_parser.py
new file mode 100644
index 0000000..831b3f5
--- /dev/null
+++ b/scripts/qidl_parser.py
@@ -0,0 +1,512 @@
+#
+# QEMU IDL Parser
+#
+# Copyright IBM, Corp. 2012
+#
+# Authors:
+# Anthony Liguori <aliguori@us.ibm.com>
+# Michael Roth <mdroth@linux.vnet.ibm.com>
+#
+# This work is licensed under the terms of the GNU GPLv2.
+# See the COPYING file in the top-level directory.
+#
+# The lexer code is based off of:
+# http://www.lysator.liu.se/c/ANSI-C-grammar-l.html
+
+import sys, json
+
+class Input(object):
+ def __init__(self, text):
+ self.fp = text
+ self.buf = text
+ self.eof = False
+
+ def pop(self):
+ if len(self.buf) == 0:
+ self.eof = True
+ return ''
+ ch = self.buf[0]
+ self.buf = self.buf[1:]
+ return ch
+
+def in_range(ch, start, end):
+ if ch >= start and ch <= end:
+ return True
+ return False
+
+# D [0-9]
+# L [a-zA-Z_]
+# H [a-fA-F0-9]
+# E [Ee][+-]?{D}+
+# FS (f|F|l|L)
+# IS (u|U|l|L)*
+
+def is_D(ch):
+ return in_range(ch, '0', '9')
+
+def is_L(ch):
+ return in_range(ch, 'a', 'z') or in_range(ch, 'A', 'Z') or ch == '_'
+
+def is_H(ch):
+ return in_range(ch, 'a', 'f') or in_range(ch, 'A', 'F') or is_D(ch)
+
+def is_FS(ch):
+ return ch in 'fFlL'
+
+def is_IS(ch):
+ return ch in 'uUlL'
+
+def lexer(fp):
+ ch = fp.pop()
+
+ while not fp.eof:
+ token = ''
+
+ if is_L(ch):
+ token += ch
+
+ ch = fp.pop()
+ while is_L(ch) or is_D(ch):
+ token += ch
+ ch = fp.pop()
+ if token in [ 'auto', 'break', 'case', 'const', 'continue',
+ 'default', 'do', 'else', 'enum', 'extern',
+ 'for', 'goto', 'if', 'register', 'return', 'signed',
+ 'sizeof',
+ 'static', 'struct', 'typedef', 'union', 'unsigned',
+ 'volatile', 'while' ]:
+ yield (token, token)
+ else:
+ yield ('symbol', token)
+ elif ch == "'":
+ token += ch
+
+ ch = fp.pop()
+ if ch == '\\':
+ token += ch
+ token += fp.pop()
+ else:
+ token += ch
+ token += fp.pop()
+ ch = fp.pop()
+ yield ('literal', token)
+ elif ch == '"':
+ token += ch
+
+ ch = fp.pop()
+ while ch not in ['', '"']:
+ token += ch
+ if ch == '\\':
+ token += fp.pop()
+ ch = fp.pop()
+ token += ch
+ yield ('literal', token)
+ ch = fp.pop()
+ elif ch in '.><+-*/%&^|!;{},:=()[]~?':
+ token += ch
+ ch = fp.pop()
+ tmp_token = token + ch
+ if tmp_token in ['<:']:
+ yield ('operator', '[')
+ ch = fp.pop()
+ elif tmp_token in [':>']:
+ yield ('operator', ']')
+ ch = fp.pop()
+ elif tmp_token in ['<%']:
+ yield ('operator', '{')
+ ch = fp.pop()
+ elif tmp_token in ['%>']:
+ yield ('operator', '}')
+ ch = fp.pop()
+ elif tmp_token == '//':
+ token = tmp_token
+ ch = fp.pop()
+ while ch != '\n' and ch != '':
+ token += ch
+ ch = fp.pop()
+ yield ('comment', token)
+ elif tmp_token == '/*':
+ token = tmp_token
+
+ ch = fp.pop()
+ while True:
+ while ch != '*':
+ token += ch
+ ch = fp.pop()
+ token += ch
+ ch = fp.pop()
+ if ch == '/':
+ token += ch
+ break
+ yield ('comment', token)
+ ch = fp.pop()
+ elif tmp_token in [ '+=', '-=', '*=', '/=', '%=', '&=', '^=',
+ '|=', '>>', '<<', '++', '--', '->', '&&',
+ '||', '<=', '>=', '==', '!=' ]:
+ yield ('operator', tmp_token)
+ ch = fp.pop()
+ else:
+ yield ('operator', token)
+ elif ch == '0':
+ token += ch
+ ch = fp.pop()
+ if ch in 'xX':
+ token += ch
+ ch = fp.pop()
+ while is_H(ch):
+ token += ch
+ ch = fp.pop()
+ while is_IS(ch):
+ token += ch
+ ch = fp.pop()
+ elif is_D(ch):
+ token += ch
+ ch = fp.pop()
+ while is_D(ch):
+ token += ch
+ ch = fp.pop()
+ yield ('literal', token)
+ elif is_D(ch):
+ token += ch
+ ch = fp.pop()
+ while is_D(ch):
+ token += ch
+ ch = fp.pop()
+ yield ('literal', token)
+ elif ch in ' \t\v\n\f':
+ token += ch
+ ch = fp.pop()
+ while len(ch) and ch in ' \t\v\n\f':
+ token += ch
+ ch = fp.pop()
+ yield ('whitespace', token)
+ elif ch in '#':
+ token += ch
+ ch = fp.pop()
+ while len(ch) and ch != '\n':
+ token += ch
+ ch = fp.pop()
+ yield ('directive', token)
+ else:
+ yield ('unknown', ch)
+ ch = fp.pop()
+
+class LookAhead(object):
+ def __init__(self, container):
+ self.i = container.__iter__()
+ self.la = []
+ self.full = False
+
+ def at(self, i):
+ if i >= len(self.la):
+ if self.full:
+ raise StopIteration()
+ else:
+ try:
+ self.la.append(self.i.next())
+ except StopIteration, e:
+ self.full = True
+ raise StopIteration()
+
+ return self.la[i]
+
+ def eof(self):
+ try:
+ self.at(len(self.la))
+ except StopIteration, e:
+ return True
+
+ return False
+
+def skip(c):
+ for token, value in c:
+ if token in ['whitespace', 'comment', 'directive']:
+ continue
+ yield (token, value)
+
+def expect(la, index, first, second=None):
+ if la.at(index)[0] != first:
+ raise Exception("expected '%s', got %s %s" % (first, la.at(index)[0], la.at(index)[1]))
+ if second != None:
+ if la.at(index)[1] != second:
+ raise Exception("expected '%s', got %s" % (second, la.at(index)[1]))
+ return index + 1, la.at(index)[1]
+
+def choice(la, index, first, second=None):
+ if la.at(index)[0] != first:
+ return False
+ if second != None:
+ if la.at(index)[1] != second:
+ return False
+ return True
+
+def process_marker(ret, params):
+ marker_type = params[0]
+ if marker_type == "derived":
+ ret['is_derived'] = True
+ elif marker_type == 'immutable':
+ ret['is_immutable'] = True
+ elif marker_type == 'elsewhere':
+ ret['is_elsewhere'] = True
+ elif marker_type == 'broken':
+ ret['is_broken'] = True
+ elif marker_type == 'size_is':
+ ret['is_array'] = True
+ ret['array_size'] = params[1]
+ elif marker_type == 'optional':
+ ret['is_optional'] = True
+ elif marker_type == 'property':
+ ret['is_property'] = True
+ if ret.has_key('property_fields'):
+ ret['property_fields'].append(params[1:])
+ else:
+ ret['property_fields'] = [params[1:]]
+
+ return ret
+
+def parse_markers(la, index, ret):
+ next = index
+
+ while choice(la, next, 'symbol', 'QIDL'):
+ params = []
+ next += 1
+
+ next, _ = expect(la, next, 'operator', '(')
+ open_parens = 1
+ param = ""
+ while open_parens:
+ if choice(la, next, 'operator', ','):
+ params.append(param)
+ param = ""
+ next += 1
+ continue
+
+ if choice(la, next, 'operator', '('):
+ open_parens += 1
+ elif choice(la, next, 'operator', ')'):
+ open_parens -= 1
+
+ if open_parens > 0:
+ param += la.at(next)[1]
+ next += 1
+ if param != "":
+ params.append(param)
+
+ ret = process_marker(ret, params)
+
+ return (next - index), ret
+
+def parse_type(la, index):
+ next = index
+ ret = {}
+
+ typename = ''
+ if choice(la, next, 'const', 'const'):
+ typename += 'const '
+ next += 1
+
+ if choice(la, next, 'struct', 'struct'):
+ typename += 'struct '
+ next += 1
+
+ if choice(la, next, 'unsigned', 'unsigned'):
+ typename += 'unsigned '
+ next += 1
+
+ if choice(la, next, 'union', 'union'):
+ typename += 'union '
+ next += 1
+
+ if choice(la, next, 'enum', 'enum'):
+ typename += 'enum '
+ next += 1
+
+ # we don't currently handle embedded struct declarations, skip them for now
+ if choice(la, next, 'operator', '{'):
+ open_braces = 1
+ while open_braces:
+ next += 1
+ if choice(la, next, 'operator', '{'):
+ open_braces += 1
+ elif choice(la, next, 'operator', '}'):
+ open_braces -= 1
+ next += 1
+ typename += "<anon>"
+ ret['is_nested_decl'] = True
+ else:
+ if choice(la, next, 'operator', '*'):
+ next += 1
+ ret['is_pointer'] = True
+ else:
+ next, rest = expect(la, next, 'symbol')
+ typename += rest
+
+ ret['type'] = typename
+
+ off, ret = parse_markers(la, next, ret)
+ next += off
+
+ if choice(la, next, 'operator', '*'):
+ next += 1
+ ret['is_pointer'] = True
+
+ return (next - index), ret
+
+def parse_var_decl(la, index, repeating_type=None):
+ next = index
+
+ if repeating_type == None:
+ off, ret = parse_type(la, next)
+ next += off
+ else:
+ ret = { 'type': repeating_type }
+
+ if choice(la, next, 'operator', '('):
+ ret['is_function'] = True
+ next += 1
+ next, _ = expect(la, next, 'operator', '*')
+ next, variable = expect(la, next, 'symbol')
+ next, _ = expect(la, next, 'operator', ')')
+
+ # skip the param list since we don't use it currently
+ next, _ = expect(la, next, 'operator', '(')
+ open_parens = 1
+ while open_parens:
+ if choice(la, next, 'operator', '('):
+ open_parens += 1
+ elif choice(la, next, 'operator', ')'):
+ open_parens -= 1
+ next += 1
+ else:
+ next, variable = expect(la, next, 'symbol')
+ ret['variable'] = variable
+
+ if choice(la, next, 'operator', '['):
+ next += 1
+ expression = ""
+ while not choice(la, next, 'operator', ']'):
+ expression += la.at(next)[1]
+ next += 1
+ next, _ = expect(la, next, 'operator', ']')
+
+ if not ret.has_key('is_array'):
+ ret['is_array'] = True
+ ret['array_size'] = expression
+ else:
+ ret['array_capacity'] = expression
+
+ off, ret = parse_markers(la, next, ret)
+ next += off
+
+ return (next - index), ret
+
+def parse_struct(la, index):
+ next = index
+
+ next, _ = expect(la, next, 'struct', 'struct')
+
+ name = None
+ if choice(la, next, 'symbol'):
+ name = la.at(next)[1]
+ next += 1
+
+ next, _ = expect(la, next, 'operator', '{')
+
+ nodes = []
+
+ while not choice(la, next, 'operator', '}'):
+ offset, node = parse_var_decl(la, next)
+ next += offset
+ nodes.append(node)
+ while choice(la, next, 'operator', ','):
+ next += 1
+ offset, node = parse_var_decl(la, next, node['type'])
+ next += offset
+ nodes.append(node)
+
+ next, _ = expect(la, next, 'operator', ';')
+
+ next += 1
+
+ return (next - index), { 'struct': name, 'fields': nodes }
+
+def parse_typedef(la, index):
+ next = index
+
+ next, _ = expect(la, next, 'typedef', 'typedef')
+
+ offset, node = parse_struct(la, next)
+ next += offset
+
+ next, typename = expect(la, next, 'symbol')
+
+ return (next - index), { 'typedef': typename, 'type': node }
+
+def parse(la, index=0):
+ next = index
+
+ if choice(la, next, 'typedef'):
+ offset, node = parse_typedef(la, next)
+ elif choice(la, next, 'struct'):
+ offset, node = parse_struct(la, next)
+ else:
+ raise Exception("unsupported QIDL declaration")
+
+ next, _ = expect(la, next + offset, 'operator', ';')
+
+ return (next - index), node
+
+def process_declaration_params(params, declaration={}):
+ declaration['id'] = params[0]
+ declaration['do_state'] = False
+ declaration['do_properties'] = False
+ if "state" in params:
+ declaration['do_state'] = True
+ if "properties" in params:
+ declaration['do_properties'] = True
+ return declaration
+
+def get_declaration_params(line):
+ params = []
+ for param in line.split("(")[1][:-2].split(","):
+ params.append(param.strip())
+ return params
+
+def get_declarations(f):
+ in_declaration = False
+ declaration = {}
+ while True:
+ line = f.readline()
+ if line == '':
+ raise StopIteration()
+ elif line.startswith("QIDL_START("):
+ params = get_declaration_params(line)
+ declaration = process_declaration_params(params, declaration)
+ declaration['text'] = ""
+ in_declaration = True
+ elif line.startswith("QIDL_END("):
+ params = get_declaration_params(line)
+ if declaration['id'] != params[0]:
+ raise Exception("unterminated QIDL declaration: %s" % declaration['id'])
+ in_declaration = False
+ yield declaration
+ elif in_declaration:
+ declaration['text'] += line
+
+def parse_file(f):
+ nodes = []
+ for declaration in get_declarations(f):
+ la = LookAhead(skip(lexer(Input(declaration['text']))))
+ _, node = parse(la)
+ node['id'] = declaration['id']
+ node['do_state'] = declaration['do_state']
+ node['do_properties'] = declaration['do_properties']
+ nodes.append(node)
+ return nodes
+
+def main():
+ nodes = parse_file(sys.stdin)
+ print json.dumps(nodes, sort_keys=True, indent=2)
+
+if __name__ == '__main__':
+ main()
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 18/20] qidl: codegen, initial commit
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (16 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 17/20] qidl: parser, initial import from qc.git Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 19/20] qidl: qidl.h, definitions for qidl annotations Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 20/20] qidl: unit tests Michael Roth
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
scripts/qidl.py | 282 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 282 insertions(+)
create mode 100644 scripts/qidl.py
diff --git a/scripts/qidl.py b/scripts/qidl.py
new file mode 100644
index 0000000..71c89bc
--- /dev/null
+++ b/scripts/qidl.py
@@ -0,0 +1,282 @@
+#
+# QIDL Code Generator
+#
+# Copyright IBM, Corp. 2012
+#
+# Authors:
+# Anthony Liguori <aliguori@us.ibm.com>
+# Michael Roth <mdroth@linux.vnet.ibm.com>
+#
+# This work is licensed under the terms of the GNU GPLv2.
+# See the COPYING file in the top-level directory.
+
+from ordereddict import OrderedDict
+from qidl_parser import parse_file
+from qapi import parse_schema, mcgen, camel_case, de_camel_case
+from qapi_visit import generate_visit_struct, push_indent, pop_indent
+import sys
+import json
+import getopt
+import os
+import errno
+
+def qapi_schema(node):
+ schema = OrderedDict()
+ data = OrderedDict()
+ fields = None
+ if node.has_key('typedef'):
+ schema['type'] = node['typedef']
+ fields = node['type']['fields']
+ elif node.has_key('struct'):
+ schema['type'] = node['struct']
+ fields = node['fields']
+ else:
+ raise Exception("top-level neither typedef nor struct")
+
+ for field in fields:
+ if filter(lambda x: field.has_key(x), \
+ ['is_derived', 'is_immutable', 'is_broken', 'is_function', \
+ 'is_nested_decl', 'is_elsewhere']):
+ continue
+
+ description = {}
+
+ if field['type'].endswith('_t'):
+ typename = field['type'][:-2]
+ elif field['type'].startswith('struct '):
+ typename = field['type'].split(" ")[1]
+ elif field['type'].startswith('enum '):
+ typename = 'int'
+ elif field['type'] == "_Bool":
+ typename = 'bool'
+ elif field['type'].endswith("char") and field.has_key('is_pointer'):
+ typename = 'str';
+ elif field['type'] == 'int':
+ typename = 'int32'
+ elif field['type'] == 'unsigned int':
+ typename = 'uint32'
+ elif field['type'] == 'char':
+ typename = 'uint8'
+ else:
+ typename = field['type']
+
+ if field.has_key('is_array') and field['is_array']:
+ description['type'] = [typename]
+ description['<annotated>'] = 'true'
+ if field.has_key('array_size'):
+ description['array_size'] = field['array_size']
+ if field.has_key('array_capacity'):
+ description['array_capacity'] = field['array_capacity']
+ elif camel_case(de_camel_case(typename)) == typename and \
+ (not field.has_key('is_pointer') or not field['is_pointer']):
+ description['<annotated>'] = 'true'
+ description['<embedded struct>'] = 'true'
+ description['type'] = typename
+ else:
+ description = typename
+
+ if field.has_key('is_optional') and field['is_optional']:
+ data["*" + field['variable']] = description
+ else:
+ data[field['variable']] = description
+
+ schema['data'] = data
+ return schema
+
+
+def parse_schema_file(filename):
+ return parse_schema(open(filename, 'r'))
+
+def write_file(output, filename):
+ if filename:
+ output_file = open(filename, 'w')
+ else:
+ output_file = sys.stdout
+ output_file.write(output)
+ if filename:
+ output_file.close()
+
+def property_list(node):
+ prop_list = []
+ fields = None
+ if node.has_key('typedef'):
+ state = node['typedef']
+ fields = node['type']['fields']
+ elif node.has_key('struct'):
+ state = node['struct']
+ fields = node['fields']
+ else:
+ raise Exception("top-level neither typedef nor struct")
+
+ for field in fields:
+ if not field.has_key('is_property'):
+ continue
+
+ for arglist in field['property_fields']:
+ if field['variable'] == 'devfn':
+ typename = 'pci_devfn'
+ elif field['type'].endswith('_t'):
+ typename = field['type'][:-2]
+ elif field['type'] == "_Bool":
+ typename = 'bool'
+ elif field.has_key('is_pointer'):
+ if field['type'] in ("char", "const char"):
+ typename = "string"
+ elif field['type'] == "void":
+ typename = "ptr"
+ else:
+ typename = field['type']
+
+ prop = {}
+ prop['name'] = arglist[0]
+ prop['state'] = state
+ prop['type'] = typename
+ prop['field'] = field['variable']
+ if len(arglist) == 2:
+ prop['default'] = arglist[1]
+ elif len(arglist) == 3:
+ prop['type'] = 'bit'
+ prop['bit'] = arglist[1]
+ prop['default'] = arglist[2]
+
+ prop_list.append(prop)
+
+ return prop_list
+
+def generate_include(include_path):
+ return mcgen('''
+#include "%(include)s"
+''',
+ include=include_path)
+
+def generate_property_bit(type_name, prop):
+ if prop.has_key('default'):
+ return mcgen('''
+ DEFINE_PROP_BIT(%(name)s, %(state)s, %(field)s, %(bit)s, %(default)s),
+''',
+ name=prop['name'], state=prop['state'],
+ field=prop['field'], bit=prop['bit'],
+ default=prop['default'])
+ return mcgen('''
+ DEFINE_PROP_BIT(%(name)s, %(state)s, %(field)s, %(bit)s),
+''',
+ name=prop['name'], state=prop['state'],
+ field=prop['field'], bit=prop['bit'])
+
+def generate_property(type_name, prop):
+ if prop.has_key('default'):
+ return mcgen('''
+ DEFINE_PROP_%(type)s(%(name)s, %(state)s, %(field)s, %(default)s),
+''',
+ type=prop['type'].upper(), name=prop['name'],
+ state=prop['state'], field=prop['field'],
+ default=prop['default'])
+ return mcgen('''
+ DEFINE_PROP_%(type)s(%(name)s, %(state)s, %(field)s),
+''',
+ type=prop['type'].upper(), name=prop['name'],
+ state=prop['state'], field=prop['field'])
+
+def generate_properties(type_name, prop_list=[]):
+ output = ""
+
+ for prop in prop_list:
+ if prop['type'] == 'bit':
+ output += generate_property_bit(type_name, prop)
+ else:
+ output += generate_property(type_name, prop)
+
+ output += mcgen('''
+ DEFINE_PROP_END_OF_LIST()
+''')
+
+ return output
+
+def generate_qidl_registration(type_name, schema, do_state, prop_list=[]):
+ schema_text = json.dumps("%s" % json.dumps(schema))
+ visitor = "NULL"
+ if do_state:
+ visitor = "visit_type_%s" % type_name
+
+ return mcgen('''
+static char *%(type_name)s_get_schema(Object *obj, Error **errp)
+{
+ return g_strdup(qidl_data_%(type_name)s.schema_json_text);
+}
+
+static void %(type_name)s_register_qidl(void)
+{
+ static Property properties[] = {
+%(properties)s
+ };
+ ObjectProperty *schema_link;
+
+ qidl_data_%(type_name)s.properties = properties;
+ qidl_data_%(type_name)s.visitor = %(visitor)s;
+ qidl_data_%(type_name)s.schema_json_text = %(schema_text)s;
+
+ schema_link = object_property_find(container_get(object_get_root(), "/qidl/schemas"),
+ "%(type_name)s", NULL);
+ qidl_data_%(type_name)s.schema_obj = container_get(object_get_root(), "/qidl/schemas/%(type_name)s");
+ if (!schema_link) {
+ object_property_add_str(qidl_data_%(type_name)s.schema_obj, "json_text",
+ %(type_name)s_get_schema, NULL, NULL);
+ }
+}
+
+qidl_init(%(type_name)s_register_qidl)
+''',
+ type_name=type_name, schema_text=schema_text, visitor=visitor,
+ properties=generate_properties(type_name, prop_list))
+
+def main(argv=[]):
+ try:
+ opts, args = getopt.gnu_getopt(argv[1:], "o:cd:I:",
+ ["output-filepath=", "include="])
+ except getopt.GetoptError, err:
+ print >> sys.stderr, err
+ return 1
+
+ output_filepath = None
+ includes = []
+ for o, a in opts:
+ if o in ("-f", "--output-filepath"):
+ output_filepath = a
+ elif o in ("-I", "--include"):
+ includes.append(a)
+
+ nodes = parse_file(sys.stdin)
+ if not nodes:
+ return 2
+
+ if os.path.dirname(output_filepath) != "":
+ try:
+ os.makedirs(os.path.dirname(output_filepath))
+ except os.error, e:
+ if e.errno != errno.EEXIST:
+ raise
+ output = ""
+ for include in includes:
+ output += generate_include(include)
+ for node in nodes:
+ do_state = False
+ schema = qapi_schema(node)
+ prop_list = []
+ # qapi parser expects iteration to be line-by-line
+ schema_text = json.dumps(schema, indent=4).replace("\"", "'").split("\n")
+ expr = parse_schema(schema_text)[0]
+
+ if node.has_key('do_state') and node['do_state']:
+ do_state = True
+ output += generate_visit_struct(expr['type'], expr['data'], True)
+ if node.has_key('do_properties') and node['do_properties']:
+ prop_list = property_list(node)
+
+ output += generate_qidl_registration(expr['type'], schema, do_state, prop_list)
+
+ write_file(output, output_filepath)
+
+ return 0
+
+if __name__ == '__main__':
+ sys.exit(main(sys.argv))
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 19/20] qidl: qidl.h, definitions for qidl annotations
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (17 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 18/20] qidl: codegen, initial commit Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 20/20] qidl: unit tests Michael Roth
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
qidl.h | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)
create mode 100644 qidl.h
diff --git a/qidl.h b/qidl.h
new file mode 100644
index 0000000..210f4c6
--- /dev/null
+++ b/qidl.h
@@ -0,0 +1,63 @@
+/*
+ * QEMU IDL Macros/stubs
+ *
+ * See docs/qidl.txt for usage information.
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPLv2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QIDL_H
+#define QIDL_H
+
+#include <glib.h>
+#include "qapi/qapi-visit-core.h"
+#include "qemu/object.h"
+#include "hw/qdev-properties.h"
+
+#ifdef QIDL_GEN
+
+/* we pass the code through the preprocessor with QIDL_GEN defined to parse
+ * structures as they'd appear after preprocessing, and use the following
+ * definitions mostly to re-insert the initial macros/annotations so they
+ * stick around for the parser to process
+ */
+#define QIDL(...) QIDL(__VA_ARGS__)
+#define QIDL_START(name, ...) QIDL_START(name, ##__VA_ARGS__)
+#define QIDL_END(name) QIDL_END(name)
+
+#define QIDL_VISIT_TYPE(name, v, s, f, e)
+#define QIDL_SCHEMA_ADD_LINK(name, obj, path, errp)
+#define QIDL_PROPERTIES(name)
+
+#else /* !QIDL_GEN */
+
+#define QIDL(...)
+#define QIDL_START(name, ...)
+#define QIDL_END(name) \
+ static struct { \
+ void (*visitor)(Visitor *, struct name **, const char *, Error **); \
+ const char *schema_json_text; \
+ Object *schema_obj; \
+ Property *properties; \
+ } qidl_data_##name;
+
+#define QIDL_VISIT_TYPE(name, v, s, f, e) \
+ g_assert(qidl_data_##name.visitor); \
+ qidl_data_##name.visitor(v, s, f, e)
+#define QIDL_SCHEMA_ADD_LINK(name, obj, path, errp) \
+ g_assert(qidl_data_##name.schema_obj); \
+ object_property_add_link(obj, path, "container", \
+ &qidl_data_##name.schema_obj, errp)
+#define QIDL_PROPERTIES(name) \
+ qidl_data_##name.properties
+
+#endif /* QIDL_GEN */
+
+#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 20/20] qidl: unit tests
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
` (18 preceding siblings ...)
2012-08-14 16:27 ` [Qemu-devel] [PATCH 19/20] qidl: qidl.h, definitions for qidl annotations Michael Roth
@ 2012-08-14 16:27 ` Michael Roth
19 siblings, 0 replies; 26+ messages in thread
From: Michael Roth @ 2012-08-14 16:27 UTC (permalink / raw)
To: qemu-devel; +Cc: blauwirbel, aliguori
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
Makefile | 2 +
rules.mak | 15 +++-
tests/Makefile | 8 +-
tests/test-qidl-included.h | 31 ++++++++
tests/test-qidl-linked.c | 91 +++++++++++++++++++++++
tests/test-qidl-linked.h | 18 +++++
tests/test-qidl.c | 177 ++++++++++++++++++++++++++++++++++++++++++++
7 files changed, 339 insertions(+), 3 deletions(-)
create mode 100644 tests/test-qidl-included.h
create mode 100644 tests/test-qidl-linked.c
create mode 100644 tests/test-qidl-linked.h
create mode 100644 tests/test-qidl.c
diff --git a/Makefile b/Makefile
index 7733e4c..f02251f 100644
--- a/Makefile
+++ b/Makefile
@@ -233,6 +233,7 @@ clean:
if test -d $$d; then $(MAKE) -C $$d $@ || exit 1; fi; \
rm -f $$d/qemu-options.def; \
done
+ find -depth -name qidl-generated -type d -exec rm -rf {} \;
VERSION ?= $(shell cat VERSION)
@@ -403,6 +404,7 @@ qemu-doc.dvi qemu-doc.html qemu-doc.info qemu-doc.pdf: \
# rebuilt before other object files
Makefile: $(GENERATED_HEADERS)
+
# Include automatically generated dependency files
# Dependencies in Makefile.objs files come from our recursive subdir rules
-include $(wildcard *.d tests/*.d)
diff --git a/rules.mak b/rules.mak
index a284946..48e7a95 100644
--- a/rules.mak
+++ b/rules.mak
@@ -15,7 +15,20 @@ MAKEFLAGS += -rR
QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
%.o: %.c
- $(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<," CC $(TARGET_DIR)$@")
+
+%.qidl.c: %.c $(SRC_PATH)/qidl.h $(addprefix $(SRC_PATH)/scripts/,qidl.py qidl_parser.py qapi.py qapi_visit.py)
+ $(call rm -f $(*D)/qidl-generated/$(*F).qidl.c)
+ $(call quiet-command, $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(CFLAGS) -E -c -DQIDL_GEN $< | \
+ $(PYTHON) $(SRC_PATH)/scripts/qidl.py --output-filepath=$(*D)/qidl-generated/$(*F).qidl.c || [ "$$?" -eq 2 ])
+
+%.o: %.c %.qidl.c
+ $(if $(strip $(shell test -f $(*D)/qidl-generated/$(*F).qidl.c && echo "true")), \
+ $(call quiet-command, \
+ $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c \
+ -include $< -o $@ $(*D)/qidl-generated/$(*F).qidl.c,"qidl CC $@"), \
+ $(call quiet-command, \
+ $(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c \
+ -o $@ $<," CC $@"))
ifeq ($(LIBTOOL),)
%.lo: %.c
diff --git a/tests/Makefile b/tests/Makefile
index ec6a050..a387643 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -15,6 +15,7 @@ check-unit-y += tests/test-string-output-visitor$(EXESUF)
check-unit-y += tests/test-coroutine$(EXESUF)
check-unit-y += tests/test-visitor-serialization$(EXESUF)
check-unit-y += tests/test-iov$(EXESUF)
+check-unit-y += tests/test-qidl$(EXESUF)
check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
@@ -34,11 +35,12 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/test-coroutine.o tests/test-string-output-visitor.o \
tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
- tests/test-qmp-commands.o tests/test-visitor-serialization.o
+ tests/test-qmp-commands.o tests/test-visitor-serialization.o \
+ tests/test-qidl.o
test-qapi-obj-y = $(qobject-obj-y) $(qapi-obj-y) $(tools-obj-y)
test-qapi-obj-y += tests/test-qapi-visit.o tests/test-qapi-types.o
-test-qapi-obj-y += module.o
+test-qapi-obj-y += module.o $(qom-obj-y)
$(test-obj-y): QEMU_INCLUDES += -Itests
@@ -84,6 +86,8 @@ check-qtest-$(CONFIG_POSIX)=$(foreach TARGET,$(TARGETS), $(check-qtest-$(TARGET)
qtest-obj-y = tests/libqtest.o $(oslib-obj-y)
$(check-qtest-y): $(qtest-obj-y)
+tests/test-qidl$(EXESUF): tests/test-qidl.o tests/test-qidl-linked.o $(test-qapi-obj-y) qapi/misc-qapi-visit.o
+
.PHONY: check-help
check-help:
@echo "Regression testing targets:"
diff --git a/tests/test-qidl-included.h b/tests/test-qidl-included.h
new file mode 100644
index 0000000..2aae51e
--- /dev/null
+++ b/tests/test-qidl-included.h
@@ -0,0 +1,31 @@
+/*
+ * Unit-tests for QIDL-generated visitors/code
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef TEST_QIDL_INCLUDED_H
+#define TEST_QIDL_INCLUDED_H
+
+#include "qidl.h"
+
+QIDL_START(TestStructIncluded, state, properties)
+typedef struct TestStructIncluded {
+ int32_t a QIDL(immutable);
+ int32_t b;
+ uint32_t c QIDL(immutable);
+ uint32_t d;
+ uint64_t e QIDL(immutable);
+ uint64_t f QIDL(property, "f", 42);
+ char *g QIDL(property, "g");
+ char *h QIDL(immutable) QIDL(property, "h");
+} TestStructIncluded;
+QIDL_END(TestStructIncluded)
+
+#endif
diff --git a/tests/test-qidl-linked.c b/tests/test-qidl-linked.c
new file mode 100644
index 0000000..b0d9896
--- /dev/null
+++ b/tests/test-qidl-linked.c
@@ -0,0 +1,91 @@
+/*
+ * Unit-tests for QIDL-generated visitors/code
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qidl.h"
+#include "test-qidl-linked.h"
+#include "hw/qdev-properties.h"
+#include "qapi/qmp-input-visitor.h"
+#include "qapi/qmp-output-visitor.h"
+#include "qapi/qapi-dealloc-visitor.h"
+
+QIDL_START(TestStructLinked, state, properties)
+typedef struct TestStructLinked {
+ int32_t a QIDL(immutable);
+ int32_t b;
+ uint32_t c QIDL(immutable);
+ uint32_t d;
+ uint64_t e QIDL(immutable);
+ uint64_t f QIDL(property, "f", 42);
+ char *g QIDL(property, "g");
+ char *h QIDL(immutable) QIDL(property, "h");
+} TestStructLinked;
+QIDL_END(TestStructLinked)
+
+/* exercise generated code from annotations in objects we link against */
+void test_linked_object_annotations(gconstpointer opaque)
+{
+ TestStructLinked *s1, *s2 = NULL;
+ Property *props;
+ QmpInputVisitor *qiv;
+ QmpOutputVisitor *qov;
+ QObject *s1_obj;
+ Error *err = NULL;
+
+ s1 = g_malloc0(sizeof(TestStructLinked));
+ s1->a = 42;
+ s1->b = INT32_MAX;
+ s1->c = 43;
+ s1->d = UINT32_MAX;
+ s1->e = 44;
+ s1->f = UINT64_MAX;
+ s1->g = g_strdup("test string g");
+ s1->h = g_strdup("test string h");
+
+ qov = qmp_output_visitor_new();
+ QIDL_VISIT_TYPE(TestStructLinked, qmp_output_get_visitor(qov), &s1, NULL, &err);
+ g_assert(err == NULL);
+
+ s1_obj = qmp_output_get_qobject(qov);
+ qiv = qmp_input_visitor_new(s1_obj);
+
+ qobject_decref(s1_obj);
+ qmp_output_visitor_cleanup(qov);
+ g_free(s1->g);
+ g_free(s1->h);
+ g_free(s1);
+
+ s2 = g_malloc0(sizeof(TestStructLinked));
+ QIDL_VISIT_TYPE(TestStructLinked, qmp_input_get_visitor(qiv), &s2, NULL, &err);
+ g_assert(err == NULL);
+
+ g_assert_cmpint(s2->a, ==, 0);
+ g_assert_cmpint(s2->b, ==, INT32_MAX);
+ g_assert_cmpint(s2->c, ==, 0);
+ g_assert_cmpint(s2->d, ==, UINT32_MAX);
+ g_assert_cmpint(s2->e, ==, 0);
+ g_assert_cmpint(s2->f, ==, UINT64_MAX);
+ g_assert_cmpstr(s2->g, ==, "test string g");
+ g_assert(s2->h == NULL);
+
+ qmp_input_visitor_cleanup(qiv);
+ g_free(s2->g);
+ g_free(s2);
+
+ props = QIDL_PROPERTIES(TestStructLinked);
+ g_assert_cmpstr(props[0].name, ==, "f");
+ g_assert_cmpint(props[0].defval, ==, 42);
+ g_assert_cmpstr(props[1].name, ==, "g");
+ g_assert_cmpint(props[1].defval, ==, 0);
+ g_assert_cmpstr(props[2].name, ==, "h");
+ g_assert_cmpint(props[2].defval, ==, 0);
+ g_assert(props[3].name == NULL);
+}
diff --git a/tests/test-qidl-linked.h b/tests/test-qidl-linked.h
new file mode 100644
index 0000000..1b100a2
--- /dev/null
+++ b/tests/test-qidl-linked.h
@@ -0,0 +1,18 @@
+/*
+ * Unit-tests for QIDL-generated visitors/code
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef TEST_QIDL_LINKED_H
+#define TEST_QIDL_LINKED_H
+
+void test_linked_object_annotations(gconstpointer opaque);
+
+#endif
diff --git a/tests/test-qidl.c b/tests/test-qidl.c
new file mode 100644
index 0000000..a1d27a2
--- /dev/null
+++ b/tests/test-qidl.c
@@ -0,0 +1,177 @@
+/*
+ * Unit-tests for QIDL-generated visitors/code
+ *
+ * Copyright IBM, Corp. 2012
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include <glib.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include "qidl.h"
+#include "test-qidl-included.h"
+#include "test-qidl-linked.h"
+#include "hw/qdev-properties.h"
+#include "qapi/qmp-input-visitor.h"
+#include "qapi/qmp-output-visitor.h"
+#include "qapi/qapi-dealloc-visitor.h"
+
+PropertyInfo qdev_prop_uint64;
+PropertyInfo qdev_prop_string;
+
+QIDL_START(TestStructMain, state, properties)
+typedef struct TestStructMain {
+ int32_t a QIDL(immutable);
+ int32_t b;
+ uint32_t c QIDL(immutable);
+ uint32_t d;
+ uint64_t e QIDL(immutable);
+ uint64_t f QIDL(property, "f", 42);
+ char *g QIDL(property, "g");
+ char *h QIDL(immutable) QIDL(property, "h");
+} TestStructMain;
+QIDL_END(TestStructMain)
+
+/* exercise generated code from annotations in main() object file */
+static void test_main_object_annotations(gconstpointer opaque)
+{
+ TestStructMain *s1, *s2 = NULL;
+ Property *props;
+ QmpInputVisitor *qiv;
+ QmpOutputVisitor *qov;
+ QObject *s1_obj;
+ Error *err = NULL;
+
+ s1 = g_malloc0(sizeof(TestStructMain));
+ s1->a = 42;
+ s1->b = INT32_MAX;
+ s1->c = 43;
+ s1->d = UINT32_MAX;
+ s1->e = 44;
+ s1->f = UINT64_MAX;
+ s1->g = g_strdup("test string g");
+ s1->h = g_strdup("test string h");
+
+ qov = qmp_output_visitor_new();
+ QIDL_VISIT_TYPE(TestStructMain, qmp_output_get_visitor(qov), &s1, NULL, &err);
+ g_assert(err == NULL);
+
+ s1_obj = qmp_output_get_qobject(qov);
+ qiv = qmp_input_visitor_new(s1_obj);
+
+ qobject_decref(s1_obj);
+ qmp_output_visitor_cleanup(qov);
+ g_free(s1->g);
+ g_free(s1->h);
+ g_free(s1);
+
+ s2 = g_malloc0(sizeof(TestStructMain));
+ QIDL_VISIT_TYPE(TestStructMain, qmp_input_get_visitor(qiv), &s2, NULL, &err);
+ g_assert(err == NULL);
+
+ g_assert_cmpint(s2->a, ==, 0);
+ g_assert_cmpint(s2->b, ==, INT32_MAX);
+ g_assert_cmpint(s2->c, ==, 0);
+ g_assert_cmpint(s2->d, ==, UINT32_MAX);
+ g_assert_cmpint(s2->e, ==, 0);
+ g_assert_cmpint(s2->f, ==, UINT64_MAX);
+ g_assert_cmpstr(s2->g, ==, "test string g");
+ g_assert(s2->h == NULL);
+
+ qmp_input_visitor_cleanup(qiv);
+ g_free(s2->g);
+ g_free(s2);
+
+ props = QIDL_PROPERTIES(TestStructMain);
+ g_assert_cmpstr(props[0].name, ==, "f");
+ g_assert_cmpint(props[0].defval, ==, 42);
+ g_assert_cmpstr(props[1].name, ==, "g");
+ g_assert_cmpint(props[1].defval, ==, 0);
+ g_assert_cmpstr(props[2].name, ==, "h");
+ g_assert_cmpint(props[2].defval, ==, 0);
+ g_assert(props[3].name == NULL);
+}
+
+/* exercise generated code from annotations in included header files */
+static void test_header_file_annotations(gconstpointer opaque)
+{
+ TestStructIncluded *s1, *s2 = NULL;
+ Property *props;
+ QmpInputVisitor *qiv;
+ QmpOutputVisitor *qov;
+ QObject *s1_obj;
+ Error *err = NULL;
+
+ s1 = g_malloc0(sizeof(TestStructIncluded));
+ s1->a = 42;
+ s1->b = INT32_MAX;
+ s1->c = 43;
+ s1->d = UINT32_MAX;
+ s1->e = 44;
+ s1->f = UINT64_MAX;
+ s1->g = g_strdup("test string g");
+ s1->h = g_strdup("test string h");
+
+ qov = qmp_output_visitor_new();
+ QIDL_VISIT_TYPE(TestStructIncluded, qmp_output_get_visitor(qov), &s1, NULL, &err);
+ g_assert(err == NULL);
+
+ s1_obj = qmp_output_get_qobject(qov);
+ qiv = qmp_input_visitor_new(s1_obj);
+
+ qobject_decref(s1_obj);
+ qmp_output_visitor_cleanup(qov);
+ g_free(s1->g);
+ g_free(s1->h);
+ g_free(s1);
+
+ s2 = g_malloc0(sizeof(TestStructIncluded));
+ QIDL_VISIT_TYPE(TestStructIncluded, qmp_input_get_visitor(qiv), &s2, NULL, &err);
+ g_assert(err == NULL);
+
+ g_assert_cmpint(s2->a, ==, 0);
+ g_assert_cmpint(s2->b, ==, INT32_MAX);
+ g_assert_cmpint(s2->c, ==, 0);
+ g_assert_cmpint(s2->d, ==, UINT32_MAX);
+ g_assert_cmpint(s2->e, ==, 0);
+ g_assert_cmpint(s2->f, ==, UINT64_MAX);
+ g_assert_cmpstr(s2->g, ==, "test string g");
+ g_assert(s2->h == NULL);
+
+ qmp_input_visitor_cleanup(qiv);
+ g_free(s2->g);
+ g_free(s2);
+
+ props = QIDL_PROPERTIES(TestStructIncluded);
+ g_assert_cmpstr(props[0].name, ==, "f");
+ g_assert_cmpint(props[0].defval, ==, 42);
+ g_assert_cmpstr(props[1].name, ==, "g");
+ g_assert_cmpint(props[1].defval, ==, 0);
+ g_assert_cmpstr(props[2].name, ==, "h");
+ g_assert_cmpint(props[2].defval, ==, 0);
+ g_assert(props[3].name == NULL);
+}
+
+int main(int argc, char **argv)
+{
+ module_call_init(MODULE_INIT_QOM);
+ module_call_init(MODULE_INIT_QIDL);
+
+ g_test_init(&argc, &argv, NULL);
+
+ g_test_add_data_func("/qidl/build_test/main_object_annotations", NULL,
+ test_main_object_annotations);
+ g_test_add_data_func("/qidl/build_test/linked_object_annotations", NULL,
+ test_linked_object_annotations);
+ g_test_add_data_func("/qidl/build_test/header_file_annotations", NULL,
+ test_header_file_annotations);
+
+ g_test_run();
+
+ return 0;
+}
--
1.7.9.5
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 16/20] qidl: Add documentation
2012-08-14 16:27 ` [Qemu-devel] [PATCH 16/20] qidl: Add documentation Michael Roth
@ 2012-08-14 19:01 ` Eric Blake
2012-08-14 19:41 ` Peter Maydell
1 sibling, 0 replies; 26+ messages in thread
From: Eric Blake @ 2012-08-14 19:01 UTC (permalink / raw)
To: Michael Roth; +Cc: blauwirbel, aliguori, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 3720 bytes --]
On 08/14/2012 10:27 AM, Michael Roth wrote:
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
> docs/qidl.txt | 343 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 343 insertions(+)
> create mode 100644 docs/qidl.txt
>
> +
> +For the rest, of the document, the following simple device will be used as an
s/rest,/rest/
> +
> +Do not include any function declarations in this header file as QIDL does not
> +understand function declarations.
Can QIDL be taught to ignore portions of a file, and stick function
declarations in those portions?
> +### Derived Fields
> +
> +If a field is set based on some other field in the device's structure, then its
> +value is derived. Since this is effectively duplicate state, we can avoid
> +sending it and then recompute it when we need to. Derived state requires a bit
> +more handling that immutable state.
s/that/than/
> +
> +In our *SerialDevice* example, our *int_pending* flag is really derived from
> +two pieces of state. It is set based on whether interrupts are enabled in the
> +*ier* register and whether there is *THRE* flag is not set in the *lsr*
s/there is/the/
> +register.
> +
> +To mark a field as derived, use the **derived** marker. To update our
> +example, we would do:
> +
> + QIDL_START(SerialDevice, state)
> + typedef struct SerialDevice {
> + SysBusDevice parent;
> +
> + uint8_t thr; // transmit holding register
> + uint8_t lsr; // line status register
> + uint8_t ier; // interrupt enable register
> +
> + int _derived int_pending; // whether we have a pending queued interrupt
> + CharDriverState *chr QIDL(immutable);
Why is it marked QIDL(immutable) but only _derived? Wouldn't
QIDL(derived) be more consistent?
> +### Broken State
> +
> +QEMU does migration with a lot of devices today. When applying this methodology
> +to these devices, one will quickly discover that there are a lot of fields that
> +are not being saved today that are not derived or immutable state.
> +
> +These are all bugs. It just so happens that these bugs are usually not very
> +serious. In many cases, they cause small functionality glitches that so far
> +have not created any problems.
> +
> +Consider our *SerialDevice* example. In QEMU's real *SerialState* device, the
> +*thr* register is not saved yet we have not marked it immutable or derived.
s/saved/saved,/
> +
> +The idea behind the broken marker is that we can convert a large number of
> +devices without breaking migration compatibility, and then institute a flag day
> +where we go through and remove broken markers en-mass.
s/en-mass/en masse/
> +
> +Below is an update of our example to reflect our real life serial device:
> +
> + QIDL_START(SerialDevice, state)
> + typedef struct SerialDevice {
> + SysBusDevice parent;
> +
> + uint8_t thr QIDL(broken); // transmit holding register
> + uint8_t lsr; // line status register
> + uint8_t ier; // interrupt enable register
> +
> + int _derived int_pending; // whether we have a pending queued interrupt
> + CharDriverState _immutable *chr;
Now you've changed the example; before it used QIDL(immutable).
> +Variable Sized, Fixed Capacity Arrays
> +-------------------------------------
> +
> +Sometimes it's desirable to have a variable sized array. QIDL currently supported
s/supported/supports/
--
Eric Blake eblake@redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 620 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 16/20] qidl: Add documentation
2012-08-14 16:27 ` [Qemu-devel] [PATCH 16/20] qidl: Add documentation Michael Roth
2012-08-14 19:01 ` Eric Blake
@ 2012-08-14 19:41 ` Peter Maydell
2012-08-14 22:15 ` Michael Roth
1 sibling, 1 reply; 26+ messages in thread
From: Peter Maydell @ 2012-08-14 19:41 UTC (permalink / raw)
To: Michael Roth; +Cc: blauwirbel, aliguori, qemu-devel
On 14 August 2012 17:27, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> ---
> docs/qidl.txt | 343 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 343 insertions(+)
> create mode 100644 docs/qidl.txt
>
> diff --git a/docs/qidl.txt b/docs/qidl.txt
> new file mode 100644
> index 0000000..19976d9
> --- /dev/null
> +++ b/docs/qidl.txt
> @@ -0,0 +1,343 @@
> +How to Serialize Device State with QIDL
> +======================================
> +
> +This document describes how to implement save/restore of a device in QEMU using
> +the QIDL compiler. The QIDL compiler makes it easier to support live
> +migration in devices by converging the serialization description with the
> +device type declaration. It has the following features:
> +
> + 1. Single description of device state and how to serialize
> +
> + 2. Fully inclusive serialization description--fields that aren't serialized
> + are explicitly marked as such including the reason why.
> +
> + 3. Optimized for the common case. Even without any special annotations,
> + many devices will Just Work out of the box.
> +
> + 4. Build time schema definition. Since QIDL runs at build time, we have full
> + access to the schema during the build which means we can fail the build if
> + the schema breaks.
> +
> +For the rest, of the document, the following simple device will be used as an
> +example.
> +
> + typedef struct SerialDevice {
> + SysBusDevice parent;
> +
> + uint8_t thr; // transmit holding register
> + uint8_t lsr; // line status register
> + uint8_t ier; // interrupt enable register
> +
> + int int_pending; // whether we have a pending queued interrupt
> + CharDriverState *chr; // backend
> + } SerialDevice;
"//" style comments are a breach of the QEMU coding style so we shouldn't
be using them in documentation examples...
> +
> +In our *SerialDevice* example, the *CharDriverState* pointer reflects the host
> +backend that we use to send serial output to the user. This is only assigned
> +during device construction and never changes. This means we can add an
> +**immutable** marker to it:
> +
> + QIDL_START(SerialDevice, state)
> + typedef struct SerialDevice {
> + SysBusDevice parent;
> +
> + uint8_t thr; // transmit holding register
> + uint8_t lsr; // line status register
> + uint8_t ier; // interrupt enable register
> +
> + int int_pending; // whether we have a pending queued interrupt
> + CharDriverState *chr QIDL(immutable);
> + } SerialDevice;
> + QIDL_END(SerialDevice)
I think it would be nicer to have a QIDL input format from which the structure
is generated as one of the outputs; that would avoid having to have some of
this ugly QIDL() markup.
We could also use this to autogenerate a lot of the QOM required boilerplate,
qdev Property arrays, etc etc.
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 16/20] qidl: Add documentation
2012-08-14 19:41 ` Peter Maydell
@ 2012-08-14 22:15 ` Michael Roth
2012-08-14 22:20 ` Peter Maydell
0 siblings, 1 reply; 26+ messages in thread
From: Michael Roth @ 2012-08-14 22:15 UTC (permalink / raw)
To: Peter Maydell; +Cc: blauwirbel, aliguori, qemu-devel
On Tue, Aug 14, 2012 at 08:41:56PM +0100, Peter Maydell wrote:
> On 14 August 2012 17:27, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
> >
> > Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> > ---
> > docs/qidl.txt | 343 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 343 insertions(+)
> > create mode 100644 docs/qidl.txt
> >
> > diff --git a/docs/qidl.txt b/docs/qidl.txt
> > new file mode 100644
> > index 0000000..19976d9
> > --- /dev/null
> > +++ b/docs/qidl.txt
> > @@ -0,0 +1,343 @@
> > +How to Serialize Device State with QIDL
> > +======================================
> > +
> > +This document describes how to implement save/restore of a device in QEMU using
> > +the QIDL compiler. The QIDL compiler makes it easier to support live
> > +migration in devices by converging the serialization description with the
> > +device type declaration. It has the following features:
> > +
> > + 1. Single description of device state and how to serialize
> > +
> > + 2. Fully inclusive serialization description--fields that aren't serialized
> > + are explicitly marked as such including the reason why.
> > +
> > + 3. Optimized for the common case. Even without any special annotations,
> > + many devices will Just Work out of the box.
> > +
> > + 4. Build time schema definition. Since QIDL runs at build time, we have full
> > + access to the schema during the build which means we can fail the build if
> > + the schema breaks.
> > +
> > +For the rest, of the document, the following simple device will be used as an
> > +example.
> > +
> > + typedef struct SerialDevice {
> > + SysBusDevice parent;
> > +
> > + uint8_t thr; // transmit holding register
> > + uint8_t lsr; // line status register
> > + uint8_t ier; // interrupt enable register
> > +
> > + int int_pending; // whether we have a pending queued interrupt
> > + CharDriverState *chr; // backend
> > + } SerialDevice;
>
> "//" style comments are a breach of the QEMU coding style so we shouldn't
> be using them in documentation examples...
>
> > +
> > +In our *SerialDevice* example, the *CharDriverState* pointer reflects the host
> > +backend that we use to send serial output to the user. This is only assigned
> > +during device construction and never changes. This means we can add an
> > +**immutable** marker to it:
> > +
> > + QIDL_START(SerialDevice, state)
> > + typedef struct SerialDevice {
> > + SysBusDevice parent;
> > +
> > + uint8_t thr; // transmit holding register
> > + uint8_t lsr; // line status register
> > + uint8_t ier; // interrupt enable register
> > +
> > + int int_pending; // whether we have a pending queued interrupt
> > + CharDriverState *chr QIDL(immutable);
> > + } SerialDevice;
> > + QIDL_END(SerialDevice)
>
> I think it would be nicer to have a QIDL input format from which the structure
> is generated as one of the outputs; that would avoid having to have some of
> this ugly QIDL() markup.
Some kind of inline/embedded input format, or external (like QAPI
schemas)?
In terms of the ugliness of things, Anthony suggested we re-introduce
the simpler form of the annotations by doing something like:
#define _immutable QIDL(immutable)
as syntactic sugar for the more commonly used annotations/macros. This
can all be done in qidl.h, transparently to the parser (thanks to the
fact that we pass the code through the preprocessor prior to parsing).
There's also a few things we can do to replace the QIDL_START/QIDL_END
pairs with a single indicator.
>
> We could also use this to autogenerate a lot of the QOM required boilerplate,
> qdev Property arrays, etc etc.
QIDL handles properties, and could possibly be used to generate some QOM
boilerplate. We can get as creative as need be really, it's just that
properties/serialization were the primary/initial use-cases.
>
> -- PMM
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 16/20] qidl: Add documentation
2012-08-14 22:15 ` Michael Roth
@ 2012-08-14 22:20 ` Peter Maydell
2012-08-18 14:19 ` Blue Swirl
0 siblings, 1 reply; 26+ messages in thread
From: Peter Maydell @ 2012-08-14 22:20 UTC (permalink / raw)
To: Michael Roth; +Cc: blauwirbel, aliguori, qemu-devel
On 14 August 2012 23:15, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
> On Tue, Aug 14, 2012 at 08:41:56PM +0100, Peter Maydell wrote:
>> On 14 August 2012 17:27, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
>> > +In our *SerialDevice* example, the *CharDriverState* pointer reflects the host
>> > +backend that we use to send serial output to the user. This is only assigned
>> > +during device construction and never changes. This means we can add an
>> > +**immutable** marker to it:
>> > +
>> > + QIDL_START(SerialDevice, state)
>> > + typedef struct SerialDevice {
>> > + SysBusDevice parent;
>> > +
>> > + uint8_t thr; // transmit holding register
>> > + uint8_t lsr; // line status register
>> > + uint8_t ier; // interrupt enable register
>> > +
>> > + int int_pending; // whether we have a pending queued interrupt
>> > + CharDriverState *chr QIDL(immutable);
>> > + } SerialDevice;
>> > + QIDL_END(SerialDevice)
>>
>> I think it would be nicer to have a QIDL input format from which the structure
>> is generated as one of the outputs; that would avoid having to have some of
>> this ugly QIDL() markup.
>
> Some kind of inline/embedded input format, or external (like QAPI
> schemas)?
External, I think. I just kinda feel in the long run it's going to be cleaner
and more flexible to have a defined syntax we generate things from
rather than trying to mark up C.
-- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 16/20] qidl: Add documentation
2012-08-14 22:20 ` Peter Maydell
@ 2012-08-18 14:19 ` Blue Swirl
0 siblings, 0 replies; 26+ messages in thread
From: Blue Swirl @ 2012-08-18 14:19 UTC (permalink / raw)
To: Peter Maydell; +Cc: aliguori, Michael Roth, qemu-devel
On Tue, Aug 14, 2012 at 10:20 PM, Peter Maydell
<peter.maydell@linaro.org> wrote:
> On 14 August 2012 23:15, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
>> On Tue, Aug 14, 2012 at 08:41:56PM +0100, Peter Maydell wrote:
>>> On 14 August 2012 17:27, Michael Roth <mdroth@linux.vnet.ibm.com> wrote:
>>> > +In our *SerialDevice* example, the *CharDriverState* pointer reflects the host
>>> > +backend that we use to send serial output to the user. This is only assigned
>>> > +during device construction and never changes. This means we can add an
>>> > +**immutable** marker to it:
>>> > +
>>> > + QIDL_START(SerialDevice, state)
>>> > + typedef struct SerialDevice {
>>> > + SysBusDevice parent;
>>> > +
>>> > + uint8_t thr; // transmit holding register
>>> > + uint8_t lsr; // line status register
>>> > + uint8_t ier; // interrupt enable register
>>> > +
>>> > + int int_pending; // whether we have a pending queued interrupt
>>> > + CharDriverState *chr QIDL(immutable);
>>> > + } SerialDevice;
>>> > + QIDL_END(SerialDevice)
>>>
>>> I think it would be nicer to have a QIDL input format from which the structure
>>> is generated as one of the outputs; that would avoid having to have some of
>>> this ugly QIDL() markup.
>>
>> Some kind of inline/embedded input format, or external (like QAPI
>> schemas)?
>
> External, I think. I just kinda feel in the long run it's going to be cleaner
> and more flexible to have a defined syntax we generate things from
> rather than trying to mark up C.
Inline syntax could be reused for other markup, like sparse or clang.
>
> -- PMM
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2012-08-18 14:19 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-08-14 16:27 [Qemu-devel] Add infrastructure for supporting QIDL annotations Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 01/20] qapi: qapi-visit.py -> qapi_visit.py so we can import Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 02/20] qapi: qapi-types.py -> qapi_types.py Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 03/20] qapi: qapi-commands.py -> qapi_commands.py Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 04/20] qapi: qapi_visit.py, make code useable as module Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 05/20] qapi: qapi_visit.py, support arrays and complex qapi definitions Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 06/20] qapi: add visitor interfaces for C arrays Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 07/20] qapi: qapi_visit.py, support generating static functions Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 08/20] qapi: qapi_visit.py, support for visiting non-pointer/embedded structs Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 09/20] qapi: QmpOutputVisitor, implement array handling Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 10/20] qapi: QmpInputVisitor, " Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 11/20] qapi: qapi.py, make json parser more robust Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 12/20] qapi: add open-coded visitor for struct tm types Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 13/20] qom-fuse: workaround for truncated properties > 4096 Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 14/20] module additions for schema registration Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 15/20] qdev: move Property-related declarations to qdev-properties.h Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 16/20] qidl: Add documentation Michael Roth
2012-08-14 19:01 ` Eric Blake
2012-08-14 19:41 ` Peter Maydell
2012-08-14 22:15 ` Michael Roth
2012-08-14 22:20 ` Peter Maydell
2012-08-18 14:19 ` Blue Swirl
2012-08-14 16:27 ` [Qemu-devel] [PATCH 17/20] qidl: parser, initial import from qc.git Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 18/20] qidl: codegen, initial commit Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 19/20] qidl: qidl.h, definitions for qidl annotations Michael Roth
2012-08-14 16:27 ` [Qemu-devel] [PATCH 20/20] qidl: unit tests Michael Roth
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.