* [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection
@ 2015-08-04 15:57 Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 01/32] qapi: Rename class QAPISchema to QAPISchemaParser Markus Armbruster
` (31 more replies)
0 siblings, 32 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Still RFC, because it has a couple of FIXMEs and TODOs.
* PATCH 01-24 basically replace the interemediate representation. The
replacement isn't complete, but fully functional. See PATCH 02 for
rationale and future work.
PATCH 06-09 and 16-17 are temporary scaffolding to ease review. I
don't intend to include them in future non-RFC revisions.
* PATCH 25-29 replace the '**' type bypass by a proper top type called
'any'.
* PATCH 30 is the introspection RFC.
* PATCH 31-32 are minor variations to hide a bit more detail from
clients.
I'm afraid the size of my series will strain reviewers. I did my
level best to split things up into reviewable pieces.
Series is based on "[PATCH 00/26] qapi: Another round of fixes and
cleanups". It also needs my "[PATCH for-2.4 0/2] qom: Fix misuse of
Error API" to pass tests.
You can fetch it from my git://repo.or.cz/qemu/armbru.git branch
qapi-introspect.
v3:
* Straightforward rebase, R-bys retained unless noted otherwise. Due
to my widespread changes, you may prefer to review from scratch
anyway.
* Old PATCH 01-19 split off into separate non-RFC series "[PATCH
00/26] qapi: Another round of fixes and cleanups"
* PATCH 02:
- Commit message spelling fix [Eric] TODO
- Clarify patch's non-effect in commit message.
- Assert enumeration constants are unique.
- Assert object's base is an object without variants [Eric].
- Drop a few superfluous check() recursions.
- Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data for events.
- Lower simple variants to flat ones as described on
qapi-code-gen.txt. Replace QAPISchemaObjectType's .flat by
.simple_union_type(), for use by pre-existing code-generation
warts.
- _make_implicit_object_type() idempotent (previous item needs
that).
- Clean up how empty command argument dictionaries work [Eric].
- Same for event.
- No R-by to drop.
* PATCH 03:
- Fix QAPISchemaEnumType.c_null() for empty enums [Eric].
- R-by dropped.
* PATCH 04:
- Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data for events.
* PATCH 05:
- Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data for events.
- .flat is gone, drop code to print it.
- Expected test output changed due to revision of PATCH 02.
* PATCH 06-09,16-17: New, replacing old PATCH 25 "qapi: Make
generators work on sorted schema expressions".
* PATCH 10:
- Update qapi-code-gen.txt with the patch's changes.
- Earlier patches ripple into gen_union().
- Nicer comment on -b [Eric].
- Objects with both members and unions aren't implemented. Add an
assertion [Eric].
- No R-by to drop.
* PATCH 11:
- Earlier patches ripple into gen_visit_union().
- Rename guard QAPI_VISIT_BUILTIN_VISITOR_DECL to
QAPI_VISIT_BUILTIN.
- Nicer comment on -b [Eric].
- Objects with both members and unions aren't implemented. Add an
assertion [Eric].
- No R-by to drop.
* PATCH 12:
- Use arg_type, ret_type instead of args, ret [Eric].
- Zap .regy unconditionally in visit_end (squashed from PATCH 30).
* PATCH 13:
- Commit message fixed and improved [Eric].
- Update qapi-code-gen.txt with the patch's changes.
* PATCH 15
- Copyright notice pasto fixed [Eric].
- Minimize patch by delaying more renames to PATCH 19.
- Use arg_type instead of data.
- Free .event_names in visit_end(), moved here from PATCH 30.
* PATCH 18:
- Moved; was right after PATCH 12 before.
- One hunk dropped due to "[PATCH 25/26] qapi-commands: Drop useless
initialization".
* PATCH 19:
- Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data or params for events.
- Consistently use single quotes for strings holding generated code
[Eric].
- Generate just 'FOO' instead of 'struct FOO' when possible [Eric].
- Earlier patches ripple into gen_union() and gen_visit_union().
- R-by dropped (death by a thousand cuts).
* PATCH 20:
- Straightforward conflicts due to new "[PATCH 14/26] qapi: Document
that input visitor semantics are prone to leaks" and "[PATCH
15/26] qapi: Document shortcoming with union 'data' branch".
* PATCH 21:
- Use arg_type, ret_type instead of args, ret [Eric].
- Straightforward conflict due to "[PATCH 25/26] qapi-commands: Drop
useless initialization".
* PATCH 22:
- Rename in qmp.c's comment, too [Eric].
* PATCH 23:
- Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data for events.
- Consistently use single quotes for strings holding generated code
[Eric].
* PATCH 24:
- Update qapi-code-gen.txt with the patch's changes.
- Use arg_type, ret_type instead of args, ret [Eric].
- Free .visited_ret_types in visit_end(), moved here from PATCH 30.
* PATCH 26:
- Indentation fix [Eric].
- Some test case updates moved here from PATCH 29: move positive
test case from args-returns-any.json to qapi-schema-test.json
(with new qmp_guest_sync() dummy in test-qmp-commands.c), move
negative test case to args-any.json, drop bogus test case.
- Rename flat-union-base-star.* to flat-union-base-any.*.
- Test QMP input visitor's visit_type_any() doesn't add extra
members [Eric].
* PATCH 27:
- Drop a TODO obsoleted by the patch [Eric].
* PATCH 28:
- Old PATCH 42+43 squashed together [Eric].
* PATCH 29:
- Documentation update on 'gen' was botched, rephrase [Eric].
- Some test case updates move to PATCH 26.
* PATCH 30:
- success-response isn't implementation detail, add TODO and fix
commit message [Eric].
- Cover new qapi-introspect.py in qapi-code-gen.txt [Eric].
- Clarify qmp_query_schema()'s comment.
- Fix introspect.json's comment on default values [Eric].
- Replace SchemaInfoObjectVariant member 'members' by 'type' [Eric].
- New SchemaInfoAlternateMember [Eric].
- Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data for events.
- Rewrite actual code generation: instead of generating a list of
JSON strings directly, first generate a list of dicts, then format
them as JSON.
- Fix to use " instead of ' for JSON strings [Eric].
- Make the generated array const [Eric].
- Fix accidental drop of a line of code [Eric].
- Fix file name in tests/.gitignore [Eric].
- Revision of PATCH 02 ripples into test output diffs.
- Move new test from test-qmp-input-visitor.c to
test-qmp-input-strict.c to make it catch extra members.
- R-by dropped.
* PATCH 32:
- Update qapi-code-gen.txt with the patch's changes.
- Rewrite actual code generation: instead of generating a list of
JSON strings directly, first generate a list of dicts, then format
them as JSON.
- Use plain integers as type names [Eric].
- Don't hide names of builtin types [Eric].
- Put a TODO on type name comments in commit message.
Markus Armbruster (32):
qapi: Rename class QAPISchema to QAPISchemaParser
qapi: New QAPISchema intermediate reperesentation
qapi: QAPISchema code generation helper methods
qapi: New QAPISchemaVisitor
tests/qapi-schema: Convert test harness to QAPISchemaVisitor
qapi: Split up some typedefs to ease review
qapi: Generate comments to simplify splitting for review
Revert "qapi: Generate comments to simplify splitting for review"
Revert "qapi: Split up some typedefs to ease review"
qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
qapi-commands: Convert to QAPISchemaVisitor
qapi: De-duplicate enum code generation
qapi-event: Eliminate global variable event_enum_value
qapi-event: Convert to QAPISchemaVisitor, fixing data with base
qapi: Generate comments to simplify splitting for review
Revert "qapi: Generate comments to simplify splitting for review"
qapi: Replace dirty is_c_ptr() by method c_null()
qapi: Clean up after recent conversions to QAPISchemaVisitor
qapi-visit: Rearrange code a bit
qapi-commands: Rearrange code
qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()
qapi: De-duplicate parameter list generation
qapi-commands: De-duplicate output marshaling functions
qapi: Improve built-in type documentation
qapi: Introduce a first class 'any' type
qom: Don't use 'gen': false for qom-get, qom-set, object-add
qapi-schema: Fix up misleading specification of netdev_add
qapi: Pseudo-type '**' is now unused, drop it
qapi: New QMP command query-schema for QMP schema introspection
qapi-introspect: Map all integer types to 'int'
qapi-introspect: Hide type names
.gitignore | 1 +
Makefile | 9 +-
Makefile.objs | 4 +-
docs/qapi-code-gen.txt | 154 +++--
docs/writing-qmp-commands.txt | 8 +-
include/monitor/monitor.h | 3 -
include/qapi/visitor-impl.h | 2 +
include/qapi/visitor.h | 1 +
monitor.c | 17 +-
qapi-schema.json | 25 +-
qapi/introspect.json | 71 +++
qapi/qapi-dealloc-visitor.c | 9 +
qapi/qapi-visit-core.c | 6 +
qapi/qmp-input-visitor.c | 11 +
qapi/qmp-output-visitor.c | 9 +
qmp-commands.hx | 266 +++++----
qmp.c | 27 +-
scripts/qapi-commands.py | 272 ++++-----
scripts/qapi-event.py | 239 +++-----
scripts/qapi-introspect.py | 197 +++++++
scripts/qapi-types.py | 355 +++++-------
scripts/qapi-visit.py | 350 +++++-------
scripts/qapi.py | 634 +++++++++++++++++----
tests/.gitignore | 1 +
tests/Makefile | 15 +-
tests/qapi-schema/alternate-good.out | 16 +-
tests/qapi-schema/args-any.err | 1 +
.../{type-bypass-no-gen.exit => args-any.exit} | 0
tests/qapi-schema/args-any.json | 2 +
.../qapi-schema/{type-bypass.err => args-any.out} | 0
tests/qapi-schema/args-member-array.out | 14 +-
tests/qapi-schema/comments.out | 5 +-
tests/qapi-schema/empty.out | 4 +-
tests/qapi-schema/enum-empty.out | 5 +-
tests/qapi-schema/event-case.out | 5 +-
tests/qapi-schema/flat-union-base-any.err | 1 +
...ion-base-star.exit => flat-union-base-any.exit} | 0
...ion-base-star.json => flat-union-base-any.json} | 2 +-
...e-bypass-no-gen.out => flat-union-base-any.out} | 0
tests/qapi-schema/flat-union-base-star.err | 1 -
tests/qapi-schema/flat-union-base-star.out | 0
tests/qapi-schema/flat-union-reverse-define.out | 22 +-
tests/qapi-schema/ident-with-escape.out | 8 +-
tests/qapi-schema/include-relpath.out | 5 +-
tests/qapi-schema/include-repetition.out | 5 +-
tests/qapi-schema/include-simple.out | 5 +-
tests/qapi-schema/indented-expr.out | 8 +-
tests/qapi-schema/qapi-schema-test.json | 8 +-
tests/qapi-schema/qapi-schema-test.out | 219 +++++--
tests/qapi-schema/returns-int.out | 6 +-
tests/qapi-schema/test-qapi.py | 38 +-
tests/qapi-schema/type-bypass-no-gen.err | 1 -
tests/qapi-schema/type-bypass-no-gen.json | 2 -
tests/qapi-schema/type-bypass.exit | 1 -
tests/qapi-schema/type-bypass.json | 2 -
tests/qapi-schema/type-bypass.out | 3 -
tests/test-qmp-commands.c | 5 +
tests/test-qmp-input-strict.c | 57 +-
tests/test-qmp-input-visitor.c | 49 +-
tests/test-qmp-output-visitor.c | 53 ++
60 files changed, 2054 insertions(+), 1185 deletions(-)
create mode 100644 qapi/introspect.json
create mode 100644 scripts/qapi-introspect.py
create mode 100644 tests/qapi-schema/args-any.err
rename tests/qapi-schema/{type-bypass-no-gen.exit => args-any.exit} (100%)
create mode 100644 tests/qapi-schema/args-any.json
rename tests/qapi-schema/{type-bypass.err => args-any.out} (100%)
create mode 100644 tests/qapi-schema/flat-union-base-any.err
rename tests/qapi-schema/{flat-union-base-star.exit => flat-union-base-any.exit} (100%)
rename tests/qapi-schema/{flat-union-base-star.json => flat-union-base-any.json} (95%)
rename tests/qapi-schema/{type-bypass-no-gen.out => flat-union-base-any.out} (100%)
delete mode 100644 tests/qapi-schema/flat-union-base-star.err
delete mode 100644 tests/qapi-schema/flat-union-base-star.out
delete mode 100644 tests/qapi-schema/type-bypass-no-gen.err
delete mode 100644 tests/qapi-schema/type-bypass-no-gen.json
delete mode 100644 tests/qapi-schema/type-bypass.exit
delete mode 100644 tests/qapi-schema/type-bypass.json
delete mode 100644 tests/qapi-schema/type-bypass.out
--
2.4.3
^ permalink raw reply [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 01/32] qapi: Rename class QAPISchema to QAPISchemaParser
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation Markus Armbruster
` (30 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
I want to name a new class QAPISchema.
While there, make it a new-style class.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 56efb18..6297656 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -103,7 +103,7 @@ class QAPIExprError(Exception):
return error_path(self.info['parent']) + \
"%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
-class QAPISchema:
+class QAPISchemaParser(object):
def __init__(self, fp, previously_included = [], incl_info = None):
abs_fname = os.path.abspath(fp.name)
@@ -149,8 +149,8 @@ class QAPISchema:
except IOError, e:
raise QAPIExprError(expr_info,
'%s: %s' % (e.strerror, include))
- exprs_include = QAPISchema(fobj, previously_included,
- expr_info)
+ exprs_include = QAPISchemaParser(fobj, previously_included,
+ expr_info)
self.exprs.extend(exprs_include.exprs)
else:
expr_elem = {'expr': expr,
@@ -759,7 +759,7 @@ def check_exprs(exprs):
def parse_schema(fname):
try:
- schema = QAPISchema(open(fname, "r"))
+ schema = QAPISchemaParser(open(fname, "r"))
return check_exprs(schema.exprs)
except (QAPISchemaError, QAPIExprError), e:
print >>sys.stderr, e
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 01/32] qapi: Rename class QAPISchema to QAPISchemaParser Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 21:53 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 03/32] qapi: QAPISchema code generation helper methods Markus Armbruster
` (29 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
The QAPI code generators work with a syntax tree (nested dictionaries)
plus a few symbol tables (also dictionaries) on the side.
They have clearly outgrown these simple data structures. There's lots
of rummaging around in dictionaries, and information is recomputed on
the fly. For the work I'm going to do, I want more clearly defined
and more convenient interfaces.
Going forward, I also want less coupling between the back-ends and the
syntax tree, to make messing with the syntax easier.
Create a bunch of classes to represent QAPI schemata.
Have the QAPISchema initializer call the parser, then walk the syntax
tree to create the new internal representation, and finally perform
semantic analysis.
Shortcut: the semantic analysis still relies on existing check_exprs()
to do the actual semantic checking. All this code needs to move into
the classes. Mark as TODO.
We generate array types eagerly, even though most of them aren't used.
Mark as TODO.
Nothing uses the new intermediate representation just yet, thus no
change to generated files.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 2 +-
scripts/qapi-event.py | 2 +-
scripts/qapi-types.py | 2 +-
scripts/qapi-visit.py | 2 +-
scripts/qapi.py | 361 ++++++++++++++++++++++++++++++++++++++++-
tests/qapi-schema/test-qapi.py | 2 +-
6 files changed, 357 insertions(+), 14 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 890ce5d..12bdc4c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -272,7 +272,7 @@ for o, a in opts:
if o in ("-m", "--middle"):
middle_mode = True
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
commands = filter(lambda expr: expr.has_key('command'), exprs)
commands = filter(lambda expr: not expr.has_key('gen'), commands)
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 7f238df..aec2d32 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -263,7 +263,7 @@ fdecl.write(mcgen('''
''',
prefix=prefix))
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
event_enum_values = []
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index f2428f3..d162ca2 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -336,7 +336,7 @@ fdecl.write(mcgen('''
#include <stdint.h>
'''))
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
for typename in builtin_types.keys():
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3cd662b..c493964 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -446,7 +446,7 @@ fdecl.write(mcgen('''
''',
prefix=prefix))
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
# to avoid header dependency hell, we always generate declarations
# for built-in types in our header files and simply guard them
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6297656..34e52c5 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -302,6 +302,7 @@ class QAPISchemaParser(object):
#
# Semantic analysis of schema expressions
+# TODO fold into QAPISchema
#
def find_base_fields(base):
@@ -755,15 +756,357 @@ def check_exprs(exprs):
else:
assert False, 'unexpected meta type'
- return map(lambda expr_elem: expr_elem['expr'], exprs)
-
-def parse_schema(fname):
- try:
- schema = QAPISchemaParser(open(fname, "r"))
- return check_exprs(schema.exprs)
- except (QAPISchemaError, QAPIExprError), e:
- print >>sys.stderr, e
- exit(1)
+ return exprs
+
+#
+# Schema compiler frontend
+#
+
+class QAPISchemaEntity(object):
+ def __init__(self, name, info):
+ assert isinstance(name, str)
+ self.name = name
+ self.info = info
+ def check(self, schema):
+ pass
+
+class QAPISchemaType(QAPISchemaEntity):
+ pass
+
+class QAPISchemaBuiltinType(QAPISchemaType):
+ def __init__(self, name):
+ QAPISchemaType.__init__(self, name, None)
+
+class QAPISchemaEnumType(QAPISchemaType):
+ def __init__(self, name, info, values):
+ QAPISchemaType.__init__(self, name, info)
+ for v in values:
+ assert isinstance(v, str)
+ self.values = values
+ def check(self, schema):
+ assert len(set(self.values)) == len(self.values)
+
+class QAPISchemaArrayType(QAPISchemaType):
+ def __init__(self, name, info, element_type):
+ QAPISchemaType.__init__(self, name, info)
+ assert isinstance(element_type, str)
+ self.element_type_name = element_type
+ self.element_type = None
+ def check(self, schema):
+ self.element_type = schema.lookup_type(self.element_type_name)
+ assert self.element_type
+
+class QAPISchemaObjectType(QAPISchemaType):
+ def __init__(self, name, info, base, local_members, variants):
+ QAPISchemaType.__init__(self, name, info)
+ assert base == None or isinstance(base, str)
+ for m in local_members:
+ assert isinstance(m, QAPISchemaObjectTypeMember)
+ if variants != None:
+ assert isinstance(variants, QAPISchemaObjectTypeVariants)
+ self.base_name = base
+ self.base = None
+ self.local_members = local_members
+ self.variants = variants
+ self.members = None
+ def check(self, schema):
+ assert self.members != False # not running in cycles
+ if self.members:
+ return
+ self.members = False # mark as being checked
+ if self.base_name:
+ self.base = schema.lookup_type(self.base_name)
+ assert isinstance(self.base, QAPISchemaObjectType)
+ assert not self.base.variants # not implemented
+ self.base.check(schema)
+ members = list(self.base.members)
+ else:
+ members = []
+ seen = {}
+ for m in members:
+ seen[m.name] = m
+ for m in self.local_members:
+ m.check(schema, members, seen)
+ if self.variants:
+ self.variants.check(schema, members, seen)
+ self.members = members
+
+class QAPISchemaObjectTypeMember(object):
+ def __init__(self, name, typ, optional):
+ assert isinstance(name, str)
+ assert isinstance(typ, str)
+ assert isinstance(optional, bool)
+ self.name = name
+ self.type_name = typ
+ self.type = None
+ self.optional = optional
+ def check(self, schema, all_members, seen):
+ assert self.name not in seen
+ self.type = schema.lookup_type(self.type_name)
+ assert self.type
+ all_members.append(self)
+ seen[self.name] = self
+
+class QAPISchemaObjectTypeVariants(object):
+ def __init__(self, tag_name, tag_enum, variants):
+ assert tag_name == None or isinstance(tag_name, str)
+ assert tag_enum == None or isinstance(tag_enum, str)
+ for v in variants:
+ assert isinstance(v, QAPISchemaObjectTypeVariant)
+ self.tag_name = tag_name
+ if tag_name:
+ assert not tag_enum
+ self.tag_member = None
+ else:
+ self.tag_member = QAPISchemaObjectTypeMember('kind', tag_enum,
+ False)
+ self.variants = variants
+ def check(self, schema, members, seen):
+ if self.tag_name:
+ self.tag_member = seen[self.tag_name]
+ else:
+ self.tag_member.check(schema, members, seen)
+ assert isinstance(self.tag_member.type, QAPISchemaEnumType)
+ for v in self.variants:
+ vseen = dict(seen)
+ v.check(schema, self.tag_member.type, vseen)
+
+class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
+ def __init__(self, name, typ):
+ QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
+ def check(self, schema, tag_type, seen):
+ QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ assert self.name in tag_type.values
+ # TODO try to get rid of .simple_union_type()
+ def simple_union_type(self):
+ if isinstance(self.type, QAPISchemaObjectType) and not self.type.info:
+ assert len(self.type.members) == 1
+ assert not self.type.variants # not implemented
+ return self.type.members[0].type
+ return None
+
+class QAPISchemaAlternateType(QAPISchemaType):
+ def __init__(self, name, info, variants):
+ QAPISchemaType.__init__(self, name, info)
+ assert isinstance(variants, QAPISchemaObjectTypeVariants)
+ assert not variants.tag_name
+ self.variants = variants
+ def check(self, schema):
+ self.variants.check(schema, [], {})
+
+class QAPISchemaCommand(QAPISchemaEntity):
+ def __init__(self, name, info, arg_type, ret_type, gen, success_response):
+ QAPISchemaEntity.__init__(self, name, info)
+ assert not arg_type or isinstance(arg_type, str)
+ assert not ret_type or isinstance(ret_type, str)
+ self.arg_type_name = arg_type
+ self.arg_type = None
+ self.ret_type_name = ret_type
+ self.ret_type = None
+ self.gen = gen
+ self.success_response = success_response
+ def check(self, schema):
+ if self.arg_type_name:
+ self.arg_type = schema.lookup_type(self.arg_type_name)
+ assert isinstance(self.arg_type, QAPISchemaObjectType)
+ assert not self.arg_type.variants # not implemented
+ if self.ret_type_name:
+ self.ret_type = schema.lookup_type(self.ret_type_name)
+ assert isinstance(self.ret_type, QAPISchemaType)
+
+class QAPISchemaEvent(QAPISchemaEntity):
+ def __init__(self, name, info, arg_type):
+ QAPISchemaEntity.__init__(self, name, info)
+ assert not arg_type or isinstance(arg_type, str)
+ self.arg_type_name = arg_type
+ self.arg_type = None
+ def check(self, schema):
+ if self.arg_type_name:
+ self.arg_type = schema.lookup_type(self.arg_type_name)
+ assert isinstance(self.arg_type, QAPISchemaObjectType)
+ assert not self.arg_type.variants # not implemented
+
+class QAPISchema(object):
+ def __init__(self, fname):
+ try:
+ self.exprs = check_exprs(QAPISchemaParser(open(fname, "r")).exprs)
+ except (QAPISchemaError, QAPIExprError), err:
+ print >>sys.stderr, err
+ exit(1)
+ self.entity_dict = {}
+ self._def_predefineds()
+ self._def_exprs()
+ self.check()
+
+ def get_exprs(self):
+ return [expr_elem['expr'] for expr_elem in self.exprs]
+
+ def _def_entity(self, ent):
+ assert ent.name not in self.entity_dict
+ self.entity_dict[ent.name] = ent
+
+ def lookup_entity(self, name, typ=None):
+ ent = self.entity_dict.get(name)
+ if typ and not isinstance(ent, typ):
+ return None
+ return ent
+
+ def lookup_type(self, name):
+ return self.lookup_entity(name, QAPISchemaType)
+
+ def _def_builtin_type(self, name):
+ self._def_entity(QAPISchemaBuiltinType(name))
+ if name != '**':
+ self._make_array_type(name) # TODO really needed?
+
+ def _def_predefineds(self):
+ for t in ['str', 'number', 'int', 'int8', 'int16', 'int32', 'int64',
+ 'uint8', 'uint16', 'uint32', 'uint64', 'size', 'bool', '**']:
+ self._def_builtin_type(t)
+
+ def _make_implicit_enum_type(self, name, values):
+ name = name + 'Kind'
+ self._def_entity(QAPISchemaEnumType(name, None, values))
+ return name
+
+ def _make_array_type(self, element_type):
+ name = element_type + 'List'
+ if not self.lookup_type(name):
+ self._def_entity(QAPISchemaArrayType(name, None, element_type))
+ return name
+
+ def _make_implicit_object_type(self, name, role, members):
+ if not members:
+ return None
+ name = ':obj-%s-%s' % (name, role)
+ if not self.lookup_entity(name, QAPISchemaObjectType):
+ self._def_entity(QAPISchemaObjectType(name, None, None,
+ members, None))
+ return name
+
+ def _def_enum_type(self, expr, info):
+ name = expr['enum']
+ data = expr['data']
+ self._def_entity(QAPISchemaEnumType(name, info, data))
+ self._make_array_type(name) # TODO really needed?
+
+ def _make_member(self, name, typ):
+ optional = False
+ if name.startswith('*'):
+ name = name[1:]
+ optional = True
+ if isinstance(typ, list):
+ assert len(typ) == 1
+ typ = self._make_array_type(typ[0])
+ return QAPISchemaObjectTypeMember(name, typ, optional)
+
+ def _make_members(self, data):
+ return [self._make_member(key, data[key]) for key in data.keys()]
+
+ def _def_struct_type(self, expr, info):
+ name = expr['struct']
+ base = expr.get('base')
+ data = expr['data']
+ self._def_entity(QAPISchemaObjectType(name, info, base,
+ self._make_members(data),
+ None))
+ self._make_array_type(name) # TODO really needed?
+
+ def _make_variant(self, case, typ):
+ return QAPISchemaObjectTypeVariant(case, typ)
+
+ def _make_simple_variant(self, case, typ):
+ if isinstance(typ, list):
+ assert len(typ) == 1
+ typ = self._make_array_type(typ[0])
+ typ = self._make_implicit_object_type(typ, 'wrapper',
+ [self._make_member('data', typ)])
+ return QAPISchemaObjectTypeVariant(case, typ)
+
+ def _make_tag_enum(self, type_name, variants):
+ return self._make_implicit_enum_type(type_name,
+ [v.name for v in variants])
+
+ def _def_union_type(self, expr, info):
+ name = expr['union']
+ data = expr['data']
+ base = expr.get('base')
+ tag_name = expr.get('discriminator')
+ tag_enum = None
+ if tag_name:
+ variants = [self._make_variant(key, data[key])
+ for key in data.keys()]
+ else:
+ variants = [self._make_simple_variant(key, data[key])
+ for key in data.keys()]
+ tag_enum = self._make_tag_enum(name, variants)
+ self._def_entity(QAPISchemaObjectType(name, info, base,
+ self._make_members(OrderedDict()),
+ QAPISchemaObjectTypeVariants(tag_name,
+ tag_enum,
+ variants)))
+ self._make_array_type(name) # TODO really needed?
+
+ def _def_alternate_type(self, expr, info):
+ name = expr['alternate']
+ data = expr['data']
+ variants = [self._make_variant(key, data[key])
+ for key in data.keys()]
+ tag_enum = self._make_tag_enum(name, variants)
+ self._def_entity(QAPISchemaAlternateType(name, info,
+ QAPISchemaObjectTypeVariants(None,
+ tag_enum,
+ variants)))
+ self._make_array_type(name) # TODO really needed?
+
+ def _def_command(self, expr, info):
+ name = expr['command']
+ data = expr.get('data')
+ rets = expr.get('returns')
+ gen = expr.get('gen', True)
+ success_response = expr.get('success-response', True)
+ if isinstance(data, OrderedDict):
+ data = self._make_implicit_object_type(name, 'arg',
+ self._make_members(data))
+ if isinstance(rets, list):
+ assert len(rets) == 1
+ rets = self._make_array_type(rets[0])
+ elif isinstance(rets, OrderedDict):
+ rets = self._make_implicit_object_type(name, 'ret',
+ self._make_members(rets))
+ self._def_entity(QAPISchemaCommand(name, info, data, rets, gen,
+ success_response))
+
+ def _def_event(self, expr, info):
+ name = expr['event']
+ data = expr.get('data')
+ if isinstance(data, OrderedDict):
+ data = self._make_implicit_object_type(name, 'arg',
+ self._make_members(data))
+ self._def_entity(QAPISchemaEvent(name, info, data))
+
+ def _def_exprs(self):
+ for expr_elem in self.exprs:
+ expr = expr_elem['expr']
+ info = expr_elem['info']
+ if 'enum' in expr:
+ self._def_enum_type(expr, info)
+ elif 'struct' in expr:
+ self._def_struct_type(expr, info)
+ elif 'union' in expr:
+ self._def_union_type(expr, info)
+ elif 'alternate' in expr:
+ self._def_alternate_type(expr, info)
+ elif 'command' in expr:
+ self._def_command(expr, info)
+ elif 'event' in expr:
+ self._def_event(expr, info)
+ else:
+ assert False
+
+ def check(self):
+ for ent in self.entity_dict.values():
+ ent.check(self)
#
# Code generation helpers
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 634ef2d..461c713 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -16,7 +16,7 @@ import os
import sys
try:
- exprs = parse_schema(sys.argv[1])
+ exprs = QAPISchema(sys.argv[1]).get_exprs()
except SystemExit:
raise
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 03/32] qapi: QAPISchema code generation helper methods
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 01/32] qapi: Rename class QAPISchema to QAPISchemaParser Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 22:18 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor Markus Armbruster
` (28 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
New methods c_name(), c_type(), c_null(), json_type(),
alternate_qtype().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 65 insertions(+), 7 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 34e52c5..3c596c3 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -767,15 +767,47 @@ class QAPISchemaEntity(object):
assert isinstance(name, str)
self.name = name
self.info = info
+ def c_name(self):
+ return c_name(self.name)
def check(self, schema):
pass
class QAPISchemaType(QAPISchemaEntity):
- pass
+ def c_type(self, is_param=False):
+ return c_name(self.name) + pointer_suffix
+ def c_null(self):
+ return 'NULL'
+ def json_type(self):
+ return None
+ def alternate_qtype(self):
+ json2qtype = {
+ 'string': 'QTYPE_QSTRING',
+ 'number': 'QTYPE_QFLOAT',
+ 'int': 'QTYPE_QINT',
+ 'boolean': 'QTYPE_QBOOL',
+ 'object': 'QTYPE_QDICT'
+ }
+ return json2qtype.get(self.json_type())
class QAPISchemaBuiltinType(QAPISchemaType):
- def __init__(self, name):
+ def __init__(self, name, json_type, c_type, c_null):
QAPISchemaType.__init__(self, name, None)
+ assert not c_type or isinstance(c_type, str)
+ assert json_type in ('string', 'number', 'int', 'boolean', 'null',
+ 'value')
+ self.json_type_name = json_type
+ self.c_type_name = c_type
+ self.c_null_val = c_null
+ def c_name(self):
+ return self.name
+ def c_type(self, is_param=False):
+ if is_param and self.name == 'str':
+ return 'const ' + self.c_type_name
+ return self.c_type_name
+ def c_null(self):
+ return self.c_null_val
+ def json_type(self):
+ return self.json_type_name
class QAPISchemaEnumType(QAPISchemaType):
def __init__(self, name, info, values):
@@ -785,6 +817,12 @@ class QAPISchemaEnumType(QAPISchemaType):
self.values = values
def check(self, schema):
assert len(set(self.values)) == len(self.values)
+ def c_type(self, is_param=False):
+ return c_name(self.name)
+ def c_null(self):
+ return c_enum_const(self.name, (self.values + ['MAX'])[0])
+ def json_type(self):
+ return 'string'
class QAPISchemaArrayType(QAPISchemaType):
def __init__(self, name, info, element_type):
@@ -830,6 +868,14 @@ class QAPISchemaObjectType(QAPISchemaType):
if self.variants:
self.variants.check(schema, members, seen)
self.members = members
+ def c_name(self):
+ assert self.info
+ return QAPISchemaType.c_name(self)
+ def c_type(self, is_param=False):
+ assert self.info
+ return QAPISchemaType.c_type(self)
+ def json_type(self):
+ return 'object'
class QAPISchemaObjectTypeMember(object):
def __init__(self, name, typ, optional):
@@ -954,15 +1000,27 @@ class QAPISchema(object):
def lookup_type(self, name):
return self.lookup_entity(name, QAPISchemaType)
- def _def_builtin_type(self, name):
- self._def_entity(QAPISchemaBuiltinType(name))
+ def _def_builtin_type(self, name, json_type, c_type, c_null):
+ self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type, c_null))
if name != '**':
self._make_array_type(name) # TODO really needed?
def _def_predefineds(self):
- for t in ['str', 'number', 'int', 'int8', 'int16', 'int32', 'int64',
- 'uint8', 'uint16', 'uint32', 'uint64', 'size', 'bool', '**']:
- self._def_builtin_type(t)
+ for t in [('str', 'string', 'char' + pointer_suffix, 'NULL'),
+ ('number', 'number', 'double', '0'),
+ ('int', 'int', 'int64_t', '0'),
+ ('int8', 'int', 'int8_t', '0'),
+ ('int16', 'int', 'int16_t', '0'),
+ ('int32', 'int', 'int32_t', '0'),
+ ('int64', 'int', 'int64_t', '0'),
+ ('uint8', 'int', 'uint8_t', '0'),
+ ('uint16', 'int', 'uint16_t', '0'),
+ ('uint32', 'int', 'uint32_t', '0'),
+ ('uint64', 'int', 'uint64_t', '0'),
+ ('size', 'int', 'uint64_t', '0'),
+ ('bool', 'boolean', 'bool', 'false'),
+ ('**', 'value', None, None)]:
+ self._def_builtin_type(*t)
def _make_implicit_enum_type(self, name, values):
name = name + 'Kind'
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (2 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 03/32] qapi: QAPISchema code generation helper methods Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 22:26 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor Markus Armbruster
` (27 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
The visitor will help keeping the code generation code simple and
reasonably separated from QAPISchema details.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 46 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 3c596c3..019d22c 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -771,6 +771,29 @@ class QAPISchemaEntity(object):
return c_name(self.name)
def check(self, schema):
pass
+ def visit(self, visitor):
+ pass
+
+class QAPISchemaVisitor(object):
+ def visit_begin(self):
+ pass
+ def visit_end(self):
+ pass
+ def visit_builtin_type(self, name, info, json_type):
+ pass
+ def visit_enum_type(self, name, info, values):
+ pass
+ def visit_array_type(self, name, info, element_type):
+ pass
+ def visit_object_type(self, name, info, base, members, variants):
+ pass
+ def visit_alternate_type(self, name, info, variants):
+ pass
+ def visit_command(self, name, info, arg_type, ret_type,
+ gen, success_response):
+ pass
+ def visit_event(self, name, info, arg_type):
+ pass
class QAPISchemaType(QAPISchemaEntity):
def c_type(self, is_param=False):
@@ -808,6 +831,8 @@ class QAPISchemaBuiltinType(QAPISchemaType):
return self.c_null_val
def json_type(self):
return self.json_type_name
+ def visit(self, visitor):
+ visitor.visit_builtin_type(self.name, self.info, self.json_type())
class QAPISchemaEnumType(QAPISchemaType):
def __init__(self, name, info, values):
@@ -823,6 +848,8 @@ class QAPISchemaEnumType(QAPISchemaType):
return c_enum_const(self.name, (self.values + ['MAX'])[0])
def json_type(self):
return 'string'
+ def visit(self, visitor):
+ visitor.visit_enum_type(self.name, self.info, self.values)
class QAPISchemaArrayType(QAPISchemaType):
def __init__(self, name, info, element_type):
@@ -833,6 +860,8 @@ class QAPISchemaArrayType(QAPISchemaType):
def check(self, schema):
self.element_type = schema.lookup_type(self.element_type_name)
assert self.element_type
+ def visit(self, visitor):
+ visitor.visit_array_type(self.name, self.info, self.element_type)
class QAPISchemaObjectType(QAPISchemaType):
def __init__(self, name, info, base, local_members, variants):
@@ -876,6 +905,9 @@ class QAPISchemaObjectType(QAPISchemaType):
return QAPISchemaType.c_type(self)
def json_type(self):
return 'object'
+ def visit(self, visitor):
+ visitor.visit_object_type(self.name, self.info,
+ self.base, self.local_members, self.variants)
class QAPISchemaObjectTypeMember(object):
def __init__(self, name, typ, optional):
@@ -939,6 +971,8 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
self.variants.check(schema, [], {})
+ def visit(self, visitor):
+ visitor.visit_alternate_type(self.name, self.info, self.variants)
class QAPISchemaCommand(QAPISchemaEntity):
def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -959,6 +993,10 @@ class QAPISchemaCommand(QAPISchemaEntity):
if self.ret_type_name:
self.ret_type = schema.lookup_type(self.ret_type_name)
assert isinstance(self.ret_type, QAPISchemaType)
+ def visit(self, visitor):
+ visitor.visit_command(self.name, self.info,
+ self.arg_type, self.ret_type,
+ self.gen, self.success_response)
class QAPISchemaEvent(QAPISchemaEntity):
def __init__(self, name, info, arg_type):
@@ -971,6 +1009,8 @@ class QAPISchemaEvent(QAPISchemaEntity):
self.arg_type = schema.lookup_type(self.arg_type_name)
assert isinstance(self.arg_type, QAPISchemaObjectType)
assert not self.arg_type.variants # not implemented
+ def visit(self, visitor):
+ visitor.visit_event(self.name, self.info, self.arg_type)
class QAPISchema(object):
def __init__(self, fname):
@@ -1166,6 +1206,12 @@ class QAPISchema(object):
for ent in self.entity_dict.values():
ent.check(self)
+ def visit(self, visitor):
+ visitor.visit_begin()
+ for name in sorted(self.entity_dict.keys()):
+ self.entity_dict[name].visit(visitor)
+ visitor.visit_end()
+
#
# Code generation helpers
#
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (3 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 22:35 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 06/32] qapi: Split up some typedefs to ease review Markus Armbruster
` (26 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
The old code prints the result of parsing (list of expression
dictionaries), and partial results of semantic analysis (list of enum
dictionaries, list of struct dictionaries).
The new code prints a trace of a schema visit, i.e. what the back-ends
are going to use. Built-in and array types are omitted, because
they're boring.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
tests/qapi-schema/alternate-good.out | 15 +-
tests/qapi-schema/args-member-array.out | 13 +-
| 4 +-
tests/qapi-schema/empty.out | 3 -
tests/qapi-schema/enum-empty.out | 4 +-
tests/qapi-schema/event-case.out | 4 +-
tests/qapi-schema/flat-union-reverse-define.out | 21 ++-
tests/qapi-schema/ident-with-escape.out | 7 +-
tests/qapi-schema/include-relpath.out | 4 +-
tests/qapi-schema/include-repetition.out | 4 +-
tests/qapi-schema/include-simple.out | 4 +-
tests/qapi-schema/indented-expr.out | 7 +-
tests/qapi-schema/qapi-schema-test.out | 214 ++++++++++++++++++------
tests/qapi-schema/returns-int.out | 5 +-
tests/qapi-schema/test-qapi.py | 38 ++++-
tests/qapi-schema/type-bypass.out | 7 +-
16 files changed, 239 insertions(+), 115 deletions(-)
diff --git a/tests/qapi-schema/alternate-good.out b/tests/qapi-schema/alternate-good.out
index 99848ee..3d765ff 100644
--- a/tests/qapi-schema/alternate-good.out
+++ b/tests/qapi-schema/alternate-good.out
@@ -1,6 +1,9 @@
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))]),
- OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
- OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('value', 'int'), ('string', 'Enum'), ('struct', 'Data')]))])]
-[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
- {'enum_name': 'AltKind', 'enum_values': None}]
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))])]
+alternate Alt
+ case value: int
+ case string: Enum
+ case struct: Data
+enum AltKind ['value', 'string', 'struct']
+object Data
+ member number: int optional=True
+ member name: str optional=True
+enum Enum ['hello', 'world']
diff --git a/tests/qapi-schema/args-member-array.out b/tests/qapi-schema/args-member-array.out
index c39fa25..b67384c 100644
--- a/tests/qapi-schema/args-member-array.out
+++ b/tests/qapi-schema/args-member-array.out
@@ -1,5 +1,8 @@
-[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
- OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
- OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), ('member2', ['def'])]))])]
-[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
-[OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
+object :obj-okay-arg
+ member member1: intList optional=False
+ member member2: defList optional=False
+enum abc ['a', 'b', 'c']
+object def
+ member array: abcList optional=False
+command okay :obj-okay-arg -> None
+ gen=True success_response=True
--git a/tests/qapi-schema/comments.out b/tests/qapi-schema/comments.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/comments.out
+++ b/tests/qapi-schema/comments.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/empty.out b/tests/qapi-schema/empty.out
index b7f89a4..e69de29 100644
--- a/tests/qapi-schema/empty.out
+++ b/tests/qapi-schema/empty.out
@@ -1,3 +0,0 @@
-[]
-[]
-[]
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
index 3b75c16..e09b00f 100644
--- a/tests/qapi-schema/enum-empty.out
+++ b/tests/qapi-schema/enum-empty.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', [])])]
-[{'enum_name': 'MyEnum', 'enum_values': []}]
-[]
+enum MyEnum []
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index 3764bc7..b5ae4c2 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1,3 +1 @@
-[OrderedDict([('event', 'oops')])]
-[]
-[]
+event oops None
diff --git a/tests/qapi-schema/flat-union-reverse-define.out b/tests/qapi-schema/flat-union-reverse-define.out
index 1ed7b8a..477fb31 100644
--- a/tests/qapi-schema/flat-union-reverse-define.out
+++ b/tests/qapi-schema/flat-union-reverse-define.out
@@ -1,9 +1,12 @@
-[OrderedDict([('union', 'TestUnion'), ('base', 'TestBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 'TestTypeB')]))]),
- OrderedDict([('struct', 'TestBase'), ('data', OrderedDict([('enum1', 'TestEnum')]))]),
- OrderedDict([('enum', 'TestEnum'), ('data', ['value1', 'value2'])]),
- OrderedDict([('struct', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('struct', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))])]
-[{'enum_name': 'TestEnum', 'enum_values': ['value1', 'value2']}]
-[OrderedDict([('struct', 'TestBase'), ('data', OrderedDict([('enum1', 'TestEnum')]))]),
- OrderedDict([('struct', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('struct', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))])]
+object TestBase
+ member enum1: TestEnum optional=False
+enum TestEnum ['value1', 'value2']
+object TestTypeA
+ member string: str optional=False
+object TestTypeB
+ member integer: int optional=False
+object TestUnion
+ base TestBase
+ tag enum1
+ case value1: TestTypeA
+ case value2: TestTypeB
diff --git a/tests/qapi-schema/ident-with-escape.out b/tests/qapi-schema/ident-with-escape.out
index 4028430..9577d1b 100644
--- a/tests/qapi-schema/ident-with-escape.out
+++ b/tests/qapi-schema/ident-with-escape.out
@@ -1,3 +1,4 @@
-[OrderedDict([('command', 'fooA'), ('data', OrderedDict([('bar1', 'str')]))])]
-[]
-[]
+object :obj-fooA-arg
+ member bar1: str optional=False
+command fooA :obj-fooA-arg -> None
+ gen=True success_response=True
diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/include-relpath.out
+++ b/tests/qapi-schema/include-relpath.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/include-repetition.out b/tests/qapi-schema/include-repetition.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/include-repetition.out
+++ b/tests/qapi-schema/include-repetition.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/include-simple.out
+++ b/tests/qapi-schema/include-simple.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/indented-expr.out b/tests/qapi-schema/indented-expr.out
index b5ce915..c5af55a 100644
--- a/tests/qapi-schema/indented-expr.out
+++ b/tests/qapi-schema/indented-expr.out
@@ -1,3 +1,4 @@
-[OrderedDict([('command', 'eins')]), OrderedDict([('command', 'zwei')])]
-[]
-[]
+command eins None -> None
+ gen=True success_response=True
+command zwei None -> None
+ gen=True success_response=True
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index b0b7187..b30bccc 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -1,55 +1,159 @@
-[OrderedDict([('enum', 'EnumOne'), ('data', ['value1', 'value2', 'value3'])]),
- OrderedDict([('struct', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
- OrderedDict([('struct', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('struct', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
- OrderedDict([('struct', 'UserDefTwoDictDict'), ('data', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]),
- OrderedDict([('struct', 'UserDefTwoDict'), ('data', OrderedDict([('string1', 'str'), ('dict2', 'UserDefTwoDictDict'), ('*dict3', 'UserDefTwoDictDict')]))]),
- OrderedDict([('struct', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', 'UserDefTwoDict')]))]),
- OrderedDict([('struct', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
- OrderedDict([('struct', 'UserDefB'), ('data', OrderedDict([('intb', 'int')]))]),
- OrderedDict([('union', 'UserDefFlatUnion'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefA'), ('value2', 'UserDefB'), ('value3', 'UserDefB')]))]),
- OrderedDict([('struct', 'UserDefUnionBase'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
- OrderedDict([('union', 'UserDefFlatUnion2'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefC'), ('value2', 'UserDefB'), ('value3', 'UserDefA')]))]),
- OrderedDict([('alternate', 'UserDefAlternate'), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
- OrderedDict([('struct', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
- OrderedDict([('union', 'UserDefNativeListUnion'), ('data', OrderedDict([('integer', ['int']), ('s8', ['int8']), ('s16', ['int16']), ('s32', ['int32']), ('s64', ['int64']), ('u8', ['uint8']), ('u16', ['uint16']), ('u32', ['uint32']), ('u64', ['uint64']), ('number', ['number']), ('boolean', ['bool']), ('string', ['str']), ('sizes', ['size'])]))]),
- OrderedDict([('command', 'user_def_cmd'), ('data', OrderedDict())]),
- OrderedDict([('command', 'user_def_cmd1'), ('data', OrderedDict([('ud1a', 'UserDefOne')]))]),
- OrderedDict([('command', 'user_def_cmd2'), ('data', OrderedDict([('ud1a', 'UserDefOne'), ('*ud1b', 'UserDefOne')])), ('returns', 'UserDefTwo')]),
- OrderedDict([('command', 'user_def_cmd3'), ('data', OrderedDict([('a', 'int'), ('*b', 'int')])), ('returns', 'int')]),
- OrderedDict([('struct', 'UserDefOptions'), ('data', OrderedDict([('*i64', ['int']), ('*u64', ['uint64']), ('*u16', ['uint16']), ('*i64x', 'int'), ('*u64x', 'uint64')]))]),
- OrderedDict([('struct', 'EventStructOne'), ('data', OrderedDict([('struct1', 'UserDefOne'), ('string', 'str'), ('*enum2', 'EnumOne')]))]),
- OrderedDict([('event', 'EVENT_A')]),
- OrderedDict([('event', 'EVENT_B'), ('data', OrderedDict())]),
- OrderedDict([('event', 'EVENT_C'), ('data', OrderedDict([('*a', 'int'), ('*b', 'UserDefOne'), ('c', 'str')]))]),
- OrderedDict([('event', 'EVENT_D'), ('data', OrderedDict([('a', 'EventStructOne'), ('b', 'str'), ('*c', 'str'), ('*enum3', 'EnumOne')]))]),
- OrderedDict([('enum', '__org.qemu_x-Enum'), ('data', ['__org.qemu_x-value'])]),
- OrderedDict([('struct', '__org.qemu_x-Base'), ('data', OrderedDict([('__org.qemu_x-member1', '__org.qemu_x-Enum')]))]),
- OrderedDict([('struct', '__org.qemu_x-Struct'), ('base', '__org.qemu_x-Base'), ('data', OrderedDict([('__org.qemu_x-member2', 'str')]))]),
- OrderedDict([('union', '__org.qemu_x-Union1'), ('data', OrderedDict([('__org.qemu_x-branch', 'str')]))]),
- OrderedDict([('struct', '__org.qemu_x-Struct2'), ('data', OrderedDict([('array', ['__org.qemu_x-Union1'])]))]),
- OrderedDict([('union', '__org.qemu_x-Union2'), ('base', '__org.qemu_x-Base'), ('discriminator', '__org.qemu_x-member1'), ('data', OrderedDict([('__org.qemu_x-value', '__org.qemu_x-Struct2')]))]),
- OrderedDict([('alternate', '__org.qemu_x-Alt'), ('data', OrderedDict([('__org.qemu_x-branch', 'str'), ('b', '__org.qemu_x-Base')]))]),
- OrderedDict([('event', '__ORG.QEMU_X-EVENT'), ('data', '__org.qemu_x-Struct')]),
- OrderedDict([('command', '__org.qemu_x-command'), ('data', OrderedDict([('a', ['__org.qemu_x-Enum']), ('b', ['__org.qemu_x-Struct']), ('c', '__org.qemu_x-Union2'), ('d', '__org.qemu_x-Alt')])), ('returns', '__org.qemu_x-Union1')])]
-[{'enum_name': 'EnumOne', 'enum_values': ['value1', 'value2', 'value3']},
- {'enum_name': '__org.qemu_x-Enum', 'enum_values': ['__org.qemu_x-value']},
- {'enum_name': 'UserDefAlternateKind', 'enum_values': None},
- {'enum_name': 'UserDefNativeListUnionKind', 'enum_values': None},
- {'enum_name': '__org.qemu_x-Union1Kind', 'enum_values': None},
- {'enum_name': '__org.qemu_x-AltKind', 'enum_values': None}]
-[OrderedDict([('struct', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
- OrderedDict([('struct', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('struct', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
- OrderedDict([('struct', 'UserDefTwoDictDict'), ('data', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]),
- OrderedDict([('struct', 'UserDefTwoDict'), ('data', OrderedDict([('string1', 'str'), ('dict2', 'UserDefTwoDictDict'), ('*dict3', 'UserDefTwoDictDict')]))]),
- OrderedDict([('struct', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', 'UserDefTwoDict')]))]),
- OrderedDict([('struct', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
- OrderedDict([('struct', 'UserDefB'), ('data', OrderedDict([('intb', 'int')]))]),
- OrderedDict([('struct', 'UserDefUnionBase'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
- OrderedDict([('struct', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
- OrderedDict([('struct', 'UserDefOptions'), ('data', OrderedDict([('*i64', ['int']), ('*u64', ['uint64']), ('*u16', ['uint16']), ('*i64x', 'int'), ('*u64x', 'uint64')]))]),
- OrderedDict([('struct', 'EventStructOne'), ('data', OrderedDict([('struct1', 'UserDefOne'), ('string', 'str'), ('*enum2', 'EnumOne')]))]),
- OrderedDict([('struct', '__org.qemu_x-Base'), ('data', OrderedDict([('__org.qemu_x-member1', '__org.qemu_x-Enum')]))]),
- OrderedDict([('struct', '__org.qemu_x-Struct'), ('base', '__org.qemu_x-Base'), ('data', OrderedDict([('__org.qemu_x-member2', 'str')]))]),
- OrderedDict([('struct', '__org.qemu_x-Struct2'), ('data', OrderedDict([('array', ['__org.qemu_x-Union1'])]))])]
+object :obj-EVENT_C-arg
+ member a: int optional=True
+ member b: UserDefOne optional=True
+ member c: str optional=False
+object :obj-EVENT_D-arg
+ member a: EventStructOne optional=False
+ member b: str optional=False
+ member c: str optional=True
+ member enum3: EnumOne optional=True
+object :obj-__org.qemu_x-command-arg
+ member a: __org.qemu_x-EnumList optional=False
+ member b: __org.qemu_x-StructList optional=False
+ member c: __org.qemu_x-Union2 optional=False
+ member d: __org.qemu_x-Alt optional=False
+object :obj-boolList-wrapper
+ member data: boolList optional=False
+object :obj-int16List-wrapper
+ member data: int16List optional=False
+object :obj-int32List-wrapper
+ member data: int32List optional=False
+object :obj-int64List-wrapper
+ member data: int64List optional=False
+object :obj-int8List-wrapper
+ member data: int8List optional=False
+object :obj-intList-wrapper
+ member data: intList optional=False
+object :obj-numberList-wrapper
+ member data: numberList optional=False
+object :obj-sizeList-wrapper
+ member data: sizeList optional=False
+object :obj-str-wrapper
+ member data: str optional=False
+object :obj-strList-wrapper
+ member data: strList optional=False
+object :obj-uint16List-wrapper
+ member data: uint16List optional=False
+object :obj-uint32List-wrapper
+ member data: uint32List optional=False
+object :obj-uint64List-wrapper
+ member data: uint64List optional=False
+object :obj-uint8List-wrapper
+ member data: uint8List optional=False
+object :obj-user_def_cmd1-arg
+ member ud1a: UserDefOne optional=False
+object :obj-user_def_cmd2-arg
+ member ud1a: UserDefOne optional=False
+ member ud1b: UserDefOne optional=True
+object :obj-user_def_cmd3-arg
+ member a: int optional=False
+ member b: int optional=True
+event EVENT_A None
+event EVENT_B None
+event EVENT_C :obj-EVENT_C-arg
+event EVENT_D :obj-EVENT_D-arg
+enum EnumOne ['value1', 'value2', 'value3']
+object EventStructOne
+ member struct1: UserDefOne optional=False
+ member string: str optional=False
+ member enum2: EnumOne optional=True
+object NestedEnumsOne
+ member enum1: EnumOne optional=False
+ member enum2: EnumOne optional=True
+ member enum3: EnumOne optional=False
+ member enum4: EnumOne optional=True
+object UserDefA
+ member boolean: bool optional=False
+alternate UserDefAlternate
+ case uda: UserDefA
+ case s: str
+ case i: int
+enum UserDefAlternateKind ['uda', 's', 'i']
+object UserDefB
+ member intb: int optional=False
+object UserDefC
+ member string1: str optional=False
+ member string2: str optional=False
+object UserDefFlatUnion
+ base UserDefUnionBase
+ tag enum1
+ case value1: UserDefA
+ case value2: UserDefB
+ case value3: UserDefB
+object UserDefFlatUnion2
+ base UserDefUnionBase
+ tag enum1
+ case value1: UserDefC
+ case value2: UserDefB
+ case value3: UserDefA
+object UserDefNativeListUnion
+ case integer: :obj-intList-wrapper
+ case s8: :obj-int8List-wrapper
+ case s16: :obj-int16List-wrapper
+ case s32: :obj-int32List-wrapper
+ case s64: :obj-int64List-wrapper
+ case u8: :obj-uint8List-wrapper
+ case u16: :obj-uint16List-wrapper
+ case u32: :obj-uint32List-wrapper
+ case u64: :obj-uint64List-wrapper
+ case number: :obj-numberList-wrapper
+ case boolean: :obj-boolList-wrapper
+ case string: :obj-strList-wrapper
+ case sizes: :obj-sizeList-wrapper
+enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes']
+object UserDefOne
+ base UserDefZero
+ member string: str optional=False
+ member enum1: EnumOne optional=True
+object UserDefOptions
+ member i64: intList optional=True
+ member u64: uint64List optional=True
+ member u16: uint16List optional=True
+ member i64x: int optional=True
+ member u64x: uint64 optional=True
+object UserDefTwo
+ member string0: str optional=False
+ member dict1: UserDefTwoDict optional=False
+object UserDefTwoDict
+ member string1: str optional=False
+ member dict2: UserDefTwoDictDict optional=False
+ member dict3: UserDefTwoDictDict optional=True
+object UserDefTwoDictDict
+ member userdef: UserDefOne optional=False
+ member string: str optional=False
+object UserDefUnionBase
+ base UserDefZero
+ member string: str optional=False
+ member enum1: EnumOne optional=False
+object UserDefZero
+ member integer: int optional=False
+event __ORG.QEMU_X-EVENT __org.qemu_x-Struct
+alternate __org.qemu_x-Alt
+ case __org.qemu_x-branch: str
+ case b: __org.qemu_x-Base
+enum __org.qemu_x-AltKind ['__org.qemu_x-branch', 'b']
+object __org.qemu_x-Base
+ member __org.qemu_x-member1: __org.qemu_x-Enum optional=False
+enum __org.qemu_x-Enum ['__org.qemu_x-value']
+object __org.qemu_x-Struct
+ base __org.qemu_x-Base
+ member __org.qemu_x-member2: str optional=False
+object __org.qemu_x-Struct2
+ member array: __org.qemu_x-Union1List optional=False
+object __org.qemu_x-Union1
+ case __org.qemu_x-branch: :obj-str-wrapper
+enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch']
+object __org.qemu_x-Union2
+ base __org.qemu_x-Base
+ tag __org.qemu_x-member1
+ case __org.qemu_x-value: __org.qemu_x-Struct2
+command __org.qemu_x-command :obj-__org.qemu_x-command-arg -> __org.qemu_x-Union1
+ gen=True success_response=True
+command user_def_cmd None -> None
+ gen=True success_response=True
+command user_def_cmd1 :obj-user_def_cmd1-arg -> None
+ gen=True success_response=True
+command user_def_cmd2 :obj-user_def_cmd2-arg -> UserDefTwo
+ gen=True success_response=True
+command user_def_cmd3 :obj-user_def_cmd3-arg -> int
+ gen=True success_response=True
diff --git a/tests/qapi-schema/returns-int.out b/tests/qapi-schema/returns-int.out
index 70b3ac5..1ac3e1e 100644
--- a/tests/qapi-schema/returns-int.out
+++ b/tests/qapi-schema/returns-int.out
@@ -1,3 +1,2 @@
-[OrderedDict([('command', 'guest-get-time'), ('returns', 'int')])]
-[]
-[]
+command guest-get-time None -> int
+ gen=True success_response=True
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index 461c713..471f8e1 100644
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -15,11 +15,35 @@ from pprint import pprint
import os
import sys
-try:
- exprs = QAPISchema(sys.argv[1]).get_exprs()
-except SystemExit:
- raise
+class QAPISchemaTestVisitor(QAPISchemaVisitor):
+ def visit_enum_type(self, name, info, values):
+ print 'enum %s %s' % (name, values)
+ def visit_object_type(self, name, info, base, members, variants):
+ print 'object %s' % name
+ if base:
+ print ' base %s' % base.name
+ for m in members:
+ print ' member %s: %s optional=%s' % \
+ (m.name, m.type.name, m.optional)
+ self._print_variants(variants)
+ def visit_alternate_type(self, name, info, variants):
+ print 'alternate %s' % name
+ self._print_variants(variants)
+ def visit_command(self, name, info, arg_type, ret_type,
+ gen, success_response):
+ print 'command %s %s -> %s' % \
+ (name, arg_type and arg_type.name, ret_type and ret_type.name)
+ print ' gen=%s success_response=%s' % (gen, success_response)
+ def visit_event(self, name, info, arg_type):
+ print 'event %s %s' % (name, arg_type and arg_type.name)
-pprint(exprs)
-pprint(enum_types)
-pprint(struct_types)
+ @staticmethod
+ def _print_variants(variants):
+ if variants:
+ if variants.tag_name:
+ print ' tag %s' % variants.tag_name
+ for v in variants.variants:
+ print ' case %s: %s' % (v.name, v.type.name)
+
+schema = QAPISchema(sys.argv[1])
+schema.visit(QAPISchemaTestVisitor())
diff --git a/tests/qapi-schema/type-bypass.out b/tests/qapi-schema/type-bypass.out
index eaf20f8..0070d4b 100644
--- a/tests/qapi-schema/type-bypass.out
+++ b/tests/qapi-schema/type-bypass.out
@@ -1,3 +1,4 @@
-[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**'), ('gen', False)])]
-[]
-[]
+object :obj-unsafe-arg
+ member arg: ** optional=False
+command unsafe :obj-unsafe-arg -> **
+ gen=False success_response=True
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 06/32] qapi: Split up some typedefs to ease review
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (4 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 22:37 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 07/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
` (25 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
One of the next patches will among other things generate a separate
typedef for some struct types, i.e.
typedef struct FOO FOO;
struct FOO {
...
};
instead of
typedef struct FOO {
...
} FOO;
To make the generated files easier to diff, anticipate the change.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-types.py | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index d162ca2..124a788 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -15,13 +15,15 @@ from qapi import *
def generate_fwd_builtin(name):
return mcgen('''
-typedef struct %(name)sList {
+typedef struct %(name)sList %(name)sList;
+
+struct %(name)sList {
union {
%(type)s value;
uint64_t padding;
};
struct %(name)sList *next;
-} %(name)sList;
+};
''',
type=c_type(name),
name=name)
@@ -31,26 +33,30 @@ def generate_fwd_struct(name):
typedef struct %(name)s %(name)s;
-typedef struct %(name)sList {
+typedef struct %(name)sList %(name)sList;
+
+struct %(name)sList {
union {
%(name)s *value;
uint64_t padding;
};
struct %(name)sList *next;
-} %(name)sList;
+};
''',
name=c_name(name))
def generate_fwd_enum_struct(name):
return mcgen('''
-typedef struct %(name)sList {
+typedef struct %(name)sList %(name)sList;
+
+struct %(name)sList {
union {
%(name)s value;
uint64_t padding;
};
struct %(name)sList *next;
-} %(name)sList;
+};
''',
name=c_name(name))
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 07/32] qapi: Generate comments to simplify splitting for review
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (5 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 06/32] qapi: Split up some typedefs to ease review Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 22:54 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 08/32] Revert "qapi: Generate comments to simplify splitting for review" Markus Armbruster
` (24 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
The effect of the next few patches on generated files is hard to
review, because stuff gets generated in different order, which renders
diffs of the generated files useless.
To get reviewable diffs, we need to split the generated files into
suitable parts: put every top-level declaration in a file named like
the thing declared, so we can diff top-level declarations regardless
of their order in the generated files.
Since I don't feel like parsing C, simply generate a // comment
identifying the declaration right before each top-level declaration.
This lets us split with a simple shell loop:
for i in q*-{commands,marshal,event,types,visit}.[ch]
do
csplit -n 4 -s "$i" '/^\/\//' '{*}'
for j in xx*
do
read h t <$j
[ "$h" == "//" ] || t=""
mv $j "$i-${j#xx}-${t/ /-}"
done
done
Splits each file F into F-NUMB-ID, where NUMB counts up from 0001, and
ID comes from the // comment.
To check the split's sane, we can run
for i in q*-{commands,marshal,event,types,visit}.[ch]
do cat o/$i-* | diff $i -
done
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 7 +++++++
scripts/qapi-event.py | 12 ++++++++++++
scripts/qapi-types.py | 18 ++++++++++++++++++
scripts/qapi-visit.py | 12 ++++++++++++
scripts/qapi.py | 5 +++++
5 files changed, 54 insertions(+)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 12bdc4c..c3afb64 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -24,6 +24,7 @@ def generate_command_decl(name, args, ret_type):
arglist += "bool has_%s, " % c_name(argname)
arglist += "%s %s, " % (argtype, c_name(argname))
return mcgen('''
+// decl qmp_%(name)s
%(ret_type)s qmp_%(name)s(%(args)sError **errp);
''',
ret_type=c_type(ret_type), name=c_name(name),
@@ -160,6 +161,7 @@ def gen_marshal_output(name, ret_type):
return ""
ret = mcgen('''
+// def qmp_marshal_output_%(c_name)s
static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
@@ -198,10 +200,12 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
hdr = gen_marshal_input_decl(name, middle_mode)
ret = mcgen('''
+// def qmp_marshal_input_%(name)s
%(header)s
{
Error *local_err = NULL;
''',
+ name=name,
header=hdr)
if ret_type:
@@ -252,6 +256,7 @@ qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
opts=options)
pop_indent()
ret = mcgen('''
+// def qmp_init_marshal
static void qmp_init_marshal(void)
{
''')
@@ -259,6 +264,7 @@ static void qmp_init_marshal(void)
ret += mcgen('''
}
+// decl qapi_init
qapi_init(qmp_init_marshal);
''')
return ret
@@ -347,6 +353,7 @@ for cmd in commands:
fdef.write(ret)
if middle_mode:
+ fdecl.write('// decl gen_marshal_input_%s\n' % cmd['command'])
fdecl.write('%s;\n' % gen_marshal_input_decl(cmd['command'], middle_mode))
ret = gen_marshal_input(cmd['command'], arglist, ret_type, middle_mode) + "\n"
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aec2d32..e511437 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -9,6 +9,7 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
+import re;
from ordereddict import OrderedDict
from qapi import *
@@ -33,22 +34,30 @@ def _generate_event_api_name(event_name, params):
# Following are the core functions that generate C APIs to emit event.
def generate_event_declaration(api_name):
+ match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
+ name = match.group(1)
return mcgen('''
+// decl %(name)s
%(api_name)s;
''',
+ name=name,
api_name = api_name)
def generate_event_implement(api_name, event_name, params):
+ match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
+ name = match.group(1)
# step 1: declare any variables
ret = mcgen("""
+// def %(name)s
%(api_name)s
{
QDict *qmp;
Error *local_err = NULL;
QMPEventFuncEmit emit;
""",
+ name=name,
api_name = api_name)
if params:
@@ -162,11 +171,13 @@ def generate_event_implement(api_name, event_name, params):
def generate_event_enum_decl(event_enum_name, event_enum_values):
lookup_decl = mcgen('''
+// decl %(event_enum_name)s_lookup
extern const char *%(event_enum_name)s_lookup[];
''',
event_enum_name = event_enum_name)
enum_decl = mcgen('''
+// typedef %(event_enum_name)s
typedef enum %(event_enum_name)s {
''',
event_enum_name = event_enum_name)
@@ -194,6 +205,7 @@ typedef enum %(event_enum_name)s {
def generate_event_enum_lookup(event_enum_name, event_enum_strings):
ret = mcgen('''
+// def %(event_enum_name)s_lookup
const char *%(event_enum_name)s_lookup[] = {
''',
event_enum_name = event_enum_name)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 124a788..7fabcaa 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -15,8 +15,10 @@ from qapi import *
def generate_fwd_builtin(name):
return mcgen('''
+// typedef %(name)sList
typedef struct %(name)sList %(name)sList;
+// struct %(name)sList
struct %(name)sList {
union {
%(type)s value;
@@ -31,10 +33,13 @@ struct %(name)sList {
def generate_fwd_struct(name):
return mcgen('''
+// typedef %(name)s
typedef struct %(name)s %(name)s;
+// typedef %(name)sList
typedef struct %(name)sList %(name)sList;
+// struct %(name)sList
struct %(name)sList {
union {
%(name)s *value;
@@ -48,8 +53,10 @@ struct %(name)sList {
def generate_fwd_enum_struct(name):
return mcgen('''
+// typedef %(name)sList
typedef struct %(name)sList %(name)sList;
+// struct %(name)sList
struct %(name)sList {
union {
%(name)s value;
@@ -84,6 +91,7 @@ def generate_struct(expr):
ret = mcgen('''
+// struct %(name)s
struct %(name)s {
''',
name=c_name(structname))
@@ -110,6 +118,7 @@ struct %(name)s {
def generate_enum_lookup(name, values):
ret = mcgen('''
+// def %(name)s_lookup
const char *const %(name)s_lookup[] = {
''',
name=c_name(name))
@@ -132,12 +141,14 @@ def generate_enum(name, values):
name = c_name(name)
lookup_decl = mcgen('''
+// decl %(name)s_lookup
extern const char *const %(name)s_lookup[];
''',
name=name)
enum_decl = mcgen('''
+// typedef %(name)s
typedef enum %(name)s {
''',
name=name)
@@ -169,6 +180,7 @@ def generate_alternate_qtypes(expr):
ret = mcgen('''
+// def %(name)s_qtypes
const int %(name)s_qtypes[QTYPE_MAX] = {
''',
name=c_name(name))
@@ -205,6 +217,7 @@ def generate_union(expr, meta):
ret = mcgen('''
+// struct %(name)s
struct %(name)s {
''',
name=name)
@@ -252,6 +265,8 @@ struct %(name)s {
''')
if meta == 'alternate':
ret += mcgen('''
+
+// decl %(name)s_qtypes
extern const int %(name)s_qtypes[];
''',
name=name)
@@ -261,6 +276,8 @@ extern const int %(name)s_qtypes[];
def generate_type_cleanup_decl(name):
ret = mcgen('''
+
+// decl qapi_free_%(name)s
void qapi_free_%(name)s(%(c_type)s obj);
''',
c_type=c_type(name), name=c_name(name))
@@ -269,6 +286,7 @@ void qapi_free_%(name)s(%(c_type)s obj);
def generate_type_cleanup(name):
ret = mcgen('''
+// def qapi_free_%(name)s
void qapi_free_%(name)s(%(c_type)s obj)
{
QapiDeallocVisitor *md;
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index c493964..f0fef55 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -28,12 +28,14 @@ def generate_visit_implicit_struct(type):
# Need a forward declaration
ret += mcgen('''
+// def visit_type_%(c_type)s_fields
static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error **errp);
''',
c_type=type_name(type))
ret += mcgen('''
+// def visit_type_implicit_%(c_type)s
static void visit_type_implicit_%(c_type)s(Visitor *m, %(c_type)s **obj, Error **errp)
{
Error *err = NULL;
@@ -59,6 +61,7 @@ def generate_visit_struct_fields(name, members, base = None):
ret += mcgen('''
+// def visit_type_%(name)s_fields
static void visit_type_%(name)s_fields(Visitor *m, %(name)s **obj, Error **errp)
{
Error *err = NULL;
@@ -146,6 +149,7 @@ def generate_visit_struct(expr):
ret += mcgen('''
+// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
''',
@@ -161,6 +165,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
def generate_visit_list(name):
return mcgen('''
+// def visit_type_%(name)sList
void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -190,6 +195,7 @@ out:
def generate_visit_enum(name):
return mcgen('''
+// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error **errp)
{
visit_type_enum(m, (int *)obj, %(c_name)s_lookup, "%(name)s", name, errp);
@@ -200,6 +206,7 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error
def generate_visit_alternate(name, members):
ret = mcgen('''
+// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -283,6 +290,7 @@ def generate_visit_union(expr):
ret += mcgen('''
+// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -365,11 +373,13 @@ def generate_declaration(name, builtin_type=False):
name = c_name(name)
ret += mcgen('''
+// decl visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp);
''',
name=name)
ret += mcgen('''
+// decl visit_type_%(name)sList
void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp);
''',
name=name)
@@ -378,6 +388,7 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, E
def generate_enum_declaration(name):
ret = mcgen('''
+// decl visit_type_%(name)sList
void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp);
''',
name=c_name(name))
@@ -387,6 +398,7 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, E
def generate_decl_enum(name):
return mcgen('''
+// decl visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s *obj, const char *name, Error **errp);
''',
name=c_name(name))
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 019d22c..8a94b10 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1415,6 +1415,7 @@ def guardname(filename):
def guardstart(name):
return mcgen('''
+// guardstart %(name)s
#ifndef %(name)s
#define %(name)s
@@ -1424,6 +1425,7 @@ def guardstart(name):
def guardend(name):
return mcgen('''
+// guardend %(name)s
#endif /* %(name)s */
''',
@@ -1507,12 +1509,14 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
fdecl = maybe_open(do_h, h_file, 'w')
fdef.write(mcgen('''
+// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
''',
comment = c_comment))
fdecl.write(mcgen('''
+// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
#ifndef %(guard)s
@@ -1525,6 +1529,7 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
def close_output(fdef, fdecl):
fdecl.write('''
+// epilogue
#endif
''')
fdecl.close()
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 08/32] Revert "qapi: Generate comments to simplify splitting for review"
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (6 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 07/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 09/32] Revert "qapi: Split up some typedefs to ease review" Markus Armbruster
` (23 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
This reverts commit 7aa8d05011251e777ad72241ec8919f73d8d8273.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 7 -------
scripts/qapi-event.py | 12 ------------
scripts/qapi-types.py | 18 ------------------
scripts/qapi-visit.py | 12 ------------
scripts/qapi.py | 5 -----
5 files changed, 54 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index c3afb64..12bdc4c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -24,7 +24,6 @@ def generate_command_decl(name, args, ret_type):
arglist += "bool has_%s, " % c_name(argname)
arglist += "%s %s, " % (argtype, c_name(argname))
return mcgen('''
-// decl qmp_%(name)s
%(ret_type)s qmp_%(name)s(%(args)sError **errp);
''',
ret_type=c_type(ret_type), name=c_name(name),
@@ -161,7 +160,6 @@ def gen_marshal_output(name, ret_type):
return ""
ret = mcgen('''
-// def qmp_marshal_output_%(c_name)s
static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
@@ -200,12 +198,10 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
hdr = gen_marshal_input_decl(name, middle_mode)
ret = mcgen('''
-// def qmp_marshal_input_%(name)s
%(header)s
{
Error *local_err = NULL;
''',
- name=name,
header=hdr)
if ret_type:
@@ -256,7 +252,6 @@ qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
opts=options)
pop_indent()
ret = mcgen('''
-// def qmp_init_marshal
static void qmp_init_marshal(void)
{
''')
@@ -264,7 +259,6 @@ static void qmp_init_marshal(void)
ret += mcgen('''
}
-// decl qapi_init
qapi_init(qmp_init_marshal);
''')
return ret
@@ -353,7 +347,6 @@ for cmd in commands:
fdef.write(ret)
if middle_mode:
- fdecl.write('// decl gen_marshal_input_%s\n' % cmd['command'])
fdecl.write('%s;\n' % gen_marshal_input_decl(cmd['command'], middle_mode))
ret = gen_marshal_input(cmd['command'], arglist, ret_type, middle_mode) + "\n"
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index e511437..aec2d32 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -9,7 +9,6 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
-import re;
from ordereddict import OrderedDict
from qapi import *
@@ -34,30 +33,22 @@ def _generate_event_api_name(event_name, params):
# Following are the core functions that generate C APIs to emit event.
def generate_event_declaration(api_name):
- match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
- name = match.group(1)
return mcgen('''
-// decl %(name)s
%(api_name)s;
''',
- name=name,
api_name = api_name)
def generate_event_implement(api_name, event_name, params):
- match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
- name = match.group(1)
# step 1: declare any variables
ret = mcgen("""
-// def %(name)s
%(api_name)s
{
QDict *qmp;
Error *local_err = NULL;
QMPEventFuncEmit emit;
""",
- name=name,
api_name = api_name)
if params:
@@ -171,13 +162,11 @@ def generate_event_implement(api_name, event_name, params):
def generate_event_enum_decl(event_enum_name, event_enum_values):
lookup_decl = mcgen('''
-// decl %(event_enum_name)s_lookup
extern const char *%(event_enum_name)s_lookup[];
''',
event_enum_name = event_enum_name)
enum_decl = mcgen('''
-// typedef %(event_enum_name)s
typedef enum %(event_enum_name)s {
''',
event_enum_name = event_enum_name)
@@ -205,7 +194,6 @@ typedef enum %(event_enum_name)s {
def generate_event_enum_lookup(event_enum_name, event_enum_strings):
ret = mcgen('''
-// def %(event_enum_name)s_lookup
const char *%(event_enum_name)s_lookup[] = {
''',
event_enum_name = event_enum_name)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 7fabcaa..124a788 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -15,10 +15,8 @@ from qapi import *
def generate_fwd_builtin(name):
return mcgen('''
-// typedef %(name)sList
typedef struct %(name)sList %(name)sList;
-// struct %(name)sList
struct %(name)sList {
union {
%(type)s value;
@@ -33,13 +31,10 @@ struct %(name)sList {
def generate_fwd_struct(name):
return mcgen('''
-// typedef %(name)s
typedef struct %(name)s %(name)s;
-// typedef %(name)sList
typedef struct %(name)sList %(name)sList;
-// struct %(name)sList
struct %(name)sList {
union {
%(name)s *value;
@@ -53,10 +48,8 @@ struct %(name)sList {
def generate_fwd_enum_struct(name):
return mcgen('''
-// typedef %(name)sList
typedef struct %(name)sList %(name)sList;
-// struct %(name)sList
struct %(name)sList {
union {
%(name)s value;
@@ -91,7 +84,6 @@ def generate_struct(expr):
ret = mcgen('''
-// struct %(name)s
struct %(name)s {
''',
name=c_name(structname))
@@ -118,7 +110,6 @@ struct %(name)s {
def generate_enum_lookup(name, values):
ret = mcgen('''
-// def %(name)s_lookup
const char *const %(name)s_lookup[] = {
''',
name=c_name(name))
@@ -141,14 +132,12 @@ def generate_enum(name, values):
name = c_name(name)
lookup_decl = mcgen('''
-// decl %(name)s_lookup
extern const char *const %(name)s_lookup[];
''',
name=name)
enum_decl = mcgen('''
-// typedef %(name)s
typedef enum %(name)s {
''',
name=name)
@@ -180,7 +169,6 @@ def generate_alternate_qtypes(expr):
ret = mcgen('''
-// def %(name)s_qtypes
const int %(name)s_qtypes[QTYPE_MAX] = {
''',
name=c_name(name))
@@ -217,7 +205,6 @@ def generate_union(expr, meta):
ret = mcgen('''
-// struct %(name)s
struct %(name)s {
''',
name=name)
@@ -265,8 +252,6 @@ struct %(name)s {
''')
if meta == 'alternate':
ret += mcgen('''
-
-// decl %(name)s_qtypes
extern const int %(name)s_qtypes[];
''',
name=name)
@@ -276,8 +261,6 @@ extern const int %(name)s_qtypes[];
def generate_type_cleanup_decl(name):
ret = mcgen('''
-
-// decl qapi_free_%(name)s
void qapi_free_%(name)s(%(c_type)s obj);
''',
c_type=c_type(name), name=c_name(name))
@@ -286,7 +269,6 @@ void qapi_free_%(name)s(%(c_type)s obj);
def generate_type_cleanup(name):
ret = mcgen('''
-// def qapi_free_%(name)s
void qapi_free_%(name)s(%(c_type)s obj)
{
QapiDeallocVisitor *md;
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index f0fef55..c493964 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -28,14 +28,12 @@ def generate_visit_implicit_struct(type):
# Need a forward declaration
ret += mcgen('''
-// def visit_type_%(c_type)s_fields
static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error **errp);
''',
c_type=type_name(type))
ret += mcgen('''
-// def visit_type_implicit_%(c_type)s
static void visit_type_implicit_%(c_type)s(Visitor *m, %(c_type)s **obj, Error **errp)
{
Error *err = NULL;
@@ -61,7 +59,6 @@ def generate_visit_struct_fields(name, members, base = None):
ret += mcgen('''
-// def visit_type_%(name)s_fields
static void visit_type_%(name)s_fields(Visitor *m, %(name)s **obj, Error **errp)
{
Error *err = NULL;
@@ -149,7 +146,6 @@ def generate_visit_struct(expr):
ret += mcgen('''
-// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
''',
@@ -165,7 +161,6 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
def generate_visit_list(name):
return mcgen('''
-// def visit_type_%(name)sList
void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -195,7 +190,6 @@ out:
def generate_visit_enum(name):
return mcgen('''
-// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error **errp)
{
visit_type_enum(m, (int *)obj, %(c_name)s_lookup, "%(name)s", name, errp);
@@ -206,7 +200,6 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error
def generate_visit_alternate(name, members):
ret = mcgen('''
-// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -290,7 +283,6 @@ def generate_visit_union(expr):
ret += mcgen('''
-// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -373,13 +365,11 @@ def generate_declaration(name, builtin_type=False):
name = c_name(name)
ret += mcgen('''
-// decl visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp);
''',
name=name)
ret += mcgen('''
-// decl visit_type_%(name)sList
void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp);
''',
name=name)
@@ -388,7 +378,6 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, E
def generate_enum_declaration(name):
ret = mcgen('''
-// decl visit_type_%(name)sList
void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp);
''',
name=c_name(name))
@@ -398,7 +387,6 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, E
def generate_decl_enum(name):
return mcgen('''
-// decl visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s *obj, const char *name, Error **errp);
''',
name=c_name(name))
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8a94b10..019d22c 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1415,7 +1415,6 @@ def guardname(filename):
def guardstart(name):
return mcgen('''
-// guardstart %(name)s
#ifndef %(name)s
#define %(name)s
@@ -1425,7 +1424,6 @@ def guardstart(name):
def guardend(name):
return mcgen('''
-// guardend %(name)s
#endif /* %(name)s */
''',
@@ -1509,14 +1507,12 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
fdecl = maybe_open(do_h, h_file, 'w')
fdef.write(mcgen('''
-// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
''',
comment = c_comment))
fdecl.write(mcgen('''
-// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
#ifndef %(guard)s
@@ -1529,7 +1525,6 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
def close_output(fdef, fdecl):
fdecl.write('''
-// epilogue
#endif
''')
fdecl.close()
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 09/32] Revert "qapi: Split up some typedefs to ease review"
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (7 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 08/32] Revert "qapi: Generate comments to simplify splitting for review" Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions Markus Armbruster
` (22 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
This reverts commit c5790fa910530f82cb605778bcd74318c9290279.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-types.py | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 124a788..d162ca2 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -15,15 +15,13 @@ from qapi import *
def generate_fwd_builtin(name):
return mcgen('''
-typedef struct %(name)sList %(name)sList;
-
-struct %(name)sList {
+typedef struct %(name)sList {
union {
%(type)s value;
uint64_t padding;
};
struct %(name)sList *next;
-};
+} %(name)sList;
''',
type=c_type(name),
name=name)
@@ -33,30 +31,26 @@ def generate_fwd_struct(name):
typedef struct %(name)s %(name)s;
-typedef struct %(name)sList %(name)sList;
-
-struct %(name)sList {
+typedef struct %(name)sList {
union {
%(name)s *value;
uint64_t padding;
};
struct %(name)sList *next;
-};
+} %(name)sList;
''',
name=c_name(name))
def generate_fwd_enum_struct(name):
return mcgen('''
-typedef struct %(name)sList %(name)sList;
-
-struct %(name)sList {
+typedef struct %(name)sList {
union {
%(name)s value;
uint64_t padding;
};
struct %(name)sList *next;
-};
+} %(name)sList;
''',
name=c_name(name))
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (8 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 09/32] Revert "qapi: Split up some typedefs to ease review" Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-05 15:15 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs Markus Armbruster
` (21 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Fixes flat unions to get the base's base members. Test case is from
commit 2fc0043, in qapi-schema-test.json:
{ 'union': 'UserDefFlatUnion',
'base': 'UserDefUnionBase',
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
{ 'struct': 'UserDefZero',
'data': { 'integer': 'int' } }
Patch's effect on UserDefFlatUnion:
struct UserDefFlatUnion {
/* Members inherited from UserDefUnionBase: */
+ int64_t integer;
char *string;
EnumOne enum1;
/* Own members: */
union { /* union tag is @enum1 */
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
};
Flat union visitors remain broken. They'll be fixed next.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
docs/qapi-code-gen.txt | 51 +++---
scripts/qapi-types.py | 273 ++++++++++++++------------------
tests/qapi-schema/qapi-schema-test.json | 4 +-
3 files changed, 144 insertions(+), 184 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index da6c6b7..9553d3a 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -539,22 +539,6 @@ Example:
$ cat qapi-generated/example-qapi-types.c
[Uninteresting stuff omitted...]
- void qapi_free_UserDefOneList(UserDefOneList *obj)
- {
- QapiDeallocVisitor *md;
- Visitor *v;
-
- if (!obj) {
- return;
- }
-
- md = qapi_dealloc_visitor_new();
- v = qapi_dealloc_get_visitor(md);
- visit_type_UserDefOneList(v, &obj, NULL, NULL);
- qapi_dealloc_visitor_cleanup(md);
- }
-
-
void qapi_free_UserDefOne(UserDefOne *obj)
{
QapiDeallocVisitor *md;
@@ -569,6 +553,21 @@ Example:
visit_type_UserDefOne(v, &obj, NULL, NULL);
qapi_dealloc_visitor_cleanup(md);
}
+
+ void qapi_free_UserDefOneList(UserDefOneList *obj)
+ {
+ QapiDeallocVisitor *md;
+ Visitor *v;
+
+ if (!obj) {
+ return;
+ }
+
+ md = qapi_dealloc_visitor_new();
+ v = qapi_dealloc_get_visitor(md);
+ visit_type_UserDefOneList(v, &obj, NULL, NULL);
+ qapi_dealloc_visitor_cleanup(md);
+ }
$ cat qapi-generated/example-qapi-types.h
[Uninteresting stuff omitted...]
@@ -579,24 +578,24 @@ Example:
typedef struct UserDefOne UserDefOne;
- typedef struct UserDefOneList {
+ typedef struct UserDefOneList UserDefOneList;
+
+ struct UserDefOne {
+ int64_t integer;
+ char *string;
+ };
+
+ void qapi_free_UserDefOne(UserDefOne *obj);
+
+ struct UserDefOneList {
union {
UserDefOne *value;
uint64_t padding;
};
struct UserDefOneList *next;
- } UserDefOneList;
-
-
-[Functions on built-in types omitted...]
-
- struct UserDefOne {
- int64_t integer;
- char *string;
};
void qapi_free_UserDefOneList(UserDefOneList *obj);
- void qapi_free_UserDefOne(UserDefOne *obj);
#endif
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index d162ca2..181bf4b 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -2,88 +2,67 @@
# QAPI types generator
#
# Copyright IBM, Corp. 2011
+# Copyright (c) 2013-2015 Red Hat Inc.
#
# Authors:
# Anthony Liguori <aliguori@us.ibm.com>
+# Markus Armbruster <armbru@redhat.com>
#
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
-from ordereddict import OrderedDict
from qapi import *
-def generate_fwd_builtin(name):
- return mcgen('''
-
-typedef struct %(name)sList {
- union {
- %(type)s value;
- uint64_t padding;
- };
- struct %(name)sList *next;
-} %(name)sList;
-''',
- type=c_type(name),
- name=name)
-
-def generate_fwd_struct(name):
+def gen_fwd_object_or_array(name):
return mcgen('''
typedef struct %(name)s %(name)s;
-
-typedef struct %(name)sList {
- union {
- %(name)s *value;
- uint64_t padding;
- };
- struct %(name)sList *next;
-} %(name)sList;
''',
name=c_name(name))
-def generate_fwd_enum_struct(name):
+def gen_array(name, element_type):
return mcgen('''
-typedef struct %(name)sList {
+struct %(name)s {
union {
- %(name)s value;
+ %(c_type)s value;
uint64_t padding;
};
- struct %(name)sList *next;
-} %(name)sList;
+ struct %(name)s *next;
+};
''',
- name=c_name(name))
+ name=c_name(name), c_type=element_type.c_type())
-def generate_struct_fields(members):
+def gen_struct_field(name, typ, optional):
ret = ''
- for argname, argentry, optional in parse_args(members):
- if optional:
- ret += mcgen('''
+ if optional:
+ ret += mcgen('''
bool has_%(c_name)s;
''',
- c_name=c_name(argname))
- ret += mcgen('''
+ c_name=c_name(name))
+ ret += mcgen('''
%(c_type)s %(c_name)s;
''',
- c_type=c_type(argentry), c_name=c_name(argname))
-
+ c_type=typ.c_type(), c_name=c_name(name))
return ret
-def generate_struct(expr):
+def generate_struct_fields(members):
+ ret = ''
- structname = expr.get('struct', "")
- members = expr['data']
- base = expr.get('base')
+ for memb in members:
+ ret += gen_struct_field(memb.name, memb.type, memb.optional)
+ return ret
+def gen_struct(name, base, members):
ret = mcgen('''
struct %(name)s {
''',
- name=c_name(structname))
+ name=c_name(name))
if base:
- ret += generate_struct_fields({'base': base})
+ ret += gen_struct_field('base', base, False)
ret += generate_struct_fields(members)
@@ -156,46 +135,38 @@ typedef enum %(name)s {
return enum_decl + lookup_decl
-def generate_alternate_qtypes(expr):
+def gen_alternate_qtypes_decl(name):
+ return mcgen('''
- name = expr['alternate']
- members = expr['data']
+extern const int %(c_name)s_qtypes[];
+''',
+ c_name=c_name(name))
+def gen_alternate_qtypes(name, variants):
ret = mcgen('''
const int %(name)s_qtypes[QTYPE_MAX] = {
''',
name=c_name(name))
- for key in members:
- qtype = find_alternate_member_qtype(members[key])
- assert qtype, "Invalid alternate member"
+ for var in variants.variants:
+ qtype = var.type.alternate_qtype()
+ assert qtype
ret += mcgen('''
[%(qtype)s] = %(enum_const)s,
''',
qtype = qtype,
- enum_const = c_enum_const(name + 'Kind', key))
+ enum_const=c_enum_const(variants.tag_member.type.name,
+ var.name))
ret += mcgen('''
};
''')
return ret
-
-def generate_union(expr, meta):
-
- name = c_name(expr[meta])
- typeinfo = expr['data']
-
- base = expr.get('base')
- discriminator = expr.get('discriminator')
-
- enum_define = discriminator_find_enum_define(expr)
- if enum_define:
- discriminator_type_name = enum_define['enum_name']
- else:
- discriminator_type_name = '%sKind' % (name)
+def gen_union(name, base, variants):
+ name = c_name(name)
ret = mcgen('''
@@ -206,18 +177,16 @@ struct %(name)s {
ret += mcgen('''
/* Members inherited from %(c_name)s: */
''',
- c_name=c_name(base))
- base_fields = find_struct(base)['data']
- ret += generate_struct_fields(base_fields)
+ c_name=c_name(base.name))
+ ret += generate_struct_fields(base.members)
ret += mcgen('''
/* Own members: */
''')
else:
- assert not discriminator
ret += mcgen('''
%(discriminator_type_name)s kind;
''',
- discriminator_type_name=c_name(discriminator_type_name))
+ discriminator_type_name=c_name(variants.tag_member.type.name))
# FIXME: What purpose does data serve, besides preventing a union that
# has a branch named 'data'? We use it in qapi-visit.py to decide
@@ -231,39 +200,36 @@ struct %(name)s {
union { /* union tag is @%(c_name)s */
void *data;
''',
- c_name=c_name(discriminator or 'kind'))
+ c_name=c_name(variants.tag_member.name))
- for key in typeinfo:
+ for var in variants.variants:
+ # TODO ugly special case for simple union
+ typ = var.simple_union_type() or var.type
ret += mcgen('''
%(c_type)s %(c_name)s;
''',
- c_type=c_type(typeinfo[key]),
- c_name=c_name(key))
+ c_type=typ.c_type(),
+ c_name=c_name(var.name))
ret += mcgen('''
};
};
''')
- if meta == 'alternate':
- ret += mcgen('''
-extern const int %(name)s_qtypes[];
-''',
- name=name)
-
return ret
def generate_type_cleanup_decl(name):
ret = mcgen('''
-void qapi_free_%(name)s(%(c_type)s obj);
+
+void qapi_free_%(name)s(%(name)s *obj);
''',
- c_type=c_type(name), name=c_name(name))
+ name=c_name(name))
return ret
def generate_type_cleanup(name):
ret = mcgen('''
-void qapi_free_%(name)s(%(c_type)s obj)
+void qapi_free_%(name)s(%(name)s *obj)
{
QapiDeallocVisitor *md;
Visitor *v;
@@ -278,9 +244,71 @@ void qapi_free_%(name)s(%(c_type)s obj)
qapi_dealloc_visitor_cleanup(md);
}
''',
- c_type=c_type(name), name=c_name(name))
+ name=c_name(name))
return ret
+class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
+ def __init__(self):
+ self.decl = None
+ self.defn = None
+ self.fwdecl = None
+ self.fwdefn = None
+ self.btin = None
+ def visit_begin(self):
+ self.decl = ''
+ self.defn = ''
+ self.fwdecl = ''
+ self.fwdefn = ''
+ self.btin = guardstart('QAPI_TYPES_BUILTIN')
+ def visit_end(self):
+ self.decl = self.fwdecl + self.decl
+ self.fwdecl = None
+ self.defn = self.fwdefn + self.defn
+ self.fwdefn = None
+ # To avoid header dependency hell, we always generate
+ # declarations for built-in types in our header files and
+ # simply guard them. See also do_builtins (command line
+ # option -b).
+ self.btin += guardend('QAPI_TYPES_BUILTIN')
+ self.decl = self.btin + self.decl
+ self.btin = None
+ def _gen_type_cleanup(self, name):
+ self.decl += generate_type_cleanup_decl(name)
+ self.defn += generate_type_cleanup(name)
+ def visit_enum_type(self, name, info, values):
+ self.fwdecl += generate_enum(name, values)
+ self.fwdefn += generate_enum_lookup(name, values)
+ def visit_array_type(self, name, info, element_type):
+ if isinstance(element_type, QAPISchemaBuiltinType):
+ self.btin += gen_fwd_object_or_array(name)
+ self.btin += gen_array(name, element_type)
+ self.btin += generate_type_cleanup_decl(name)
+ if do_builtins:
+ self.defn += generate_type_cleanup(name)
+ else:
+ self.fwdecl += gen_fwd_object_or_array(name)
+ self.decl += gen_array(name, element_type)
+ self._gen_type_cleanup(name)
+ def visit_object_type(self, name, info, base, members, variants):
+ if info:
+ self.fwdecl += gen_fwd_object_or_array(name)
+ if variants:
+ assert not members # not implemented
+ self.decl += gen_union(name, base, variants)
+ else:
+ self.decl += gen_struct(name, base, members)
+ self._gen_type_cleanup(name)
+ def visit_alternate_type(self, name, info, variants):
+ self.fwdecl += gen_fwd_object_or_array(name)
+ self.fwdefn += gen_alternate_qtypes(name, variants)
+ self.decl += gen_union(name, None, variants)
+ self.decl += gen_alternate_qtypes_decl(name)
+ self._gen_type_cleanup(name)
+
+# If you link code generated from multiple schemata, you want only one
+# instance include the code generated for built-in types. Generate it
+# only when do_builtins, enabled by command line option -b. See also
+# QAPISchemaGenTypeVisitor.visit_end().
do_builtins = False
(input_file, output_dir, do_c, do_h, prefix, opts) = \
@@ -336,77 +364,10 @@ fdecl.write(mcgen('''
#include <stdint.h>
'''))
-exprs = QAPISchema(input_file).get_exprs()
-
-fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
-for typename in builtin_types.keys():
- fdecl.write(generate_fwd_builtin(typename))
-fdecl.write(guardend("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
-
-for expr in exprs:
- ret = ""
- if expr.has_key('struct'):
- ret += generate_fwd_struct(expr['struct'])
- elif expr.has_key('enum'):
- ret += generate_enum(expr['enum'], expr['data'])
- ret += generate_fwd_enum_struct(expr['enum'])
- fdef.write(generate_enum_lookup(expr['enum'], expr['data']))
- elif expr.has_key('union'):
- ret += generate_fwd_struct(expr['union'])
- enum_define = discriminator_find_enum_define(expr)
- if not enum_define:
- ret += generate_enum('%sKind' % expr['union'], expr['data'].keys())
- fdef.write(generate_enum_lookup('%sKind' % expr['union'],
- expr['data'].keys()))
- elif expr.has_key('alternate'):
- ret += generate_fwd_struct(expr['alternate'])
- ret += generate_enum('%sKind' % expr['alternate'], expr['data'].keys())
- fdef.write(generate_enum_lookup('%sKind' % expr['alternate'],
- expr['data'].keys()))
- fdef.write(generate_alternate_qtypes(expr))
- else:
- continue
- fdecl.write(ret)
-
-# to avoid header dependency hell, we always generate declarations
-# for built-in types in our header files and simply guard them
-fdecl.write(guardstart("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
-for typename in builtin_types.keys():
- fdecl.write(generate_type_cleanup_decl(typename + "List"))
-fdecl.write(guardend("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
-
-# ...this doesn't work for cases where we link in multiple objects that
-# have the functions defined, so we use -b option to provide control
-# over these cases
-if do_builtins:
- for typename in builtin_types.keys():
- fdef.write(generate_type_cleanup(typename + "List"))
-
-for expr in exprs:
- ret = ""
- if expr.has_key('struct'):
- ret += generate_struct(expr) + "\n"
- ret += generate_type_cleanup_decl(expr['struct'] + "List")
- fdef.write(generate_type_cleanup(expr['struct'] + "List"))
- ret += generate_type_cleanup_decl(expr['struct'])
- fdef.write(generate_type_cleanup(expr['struct']))
- elif expr.has_key('union'):
- ret += generate_union(expr, 'union') + "\n"
- ret += generate_type_cleanup_decl(expr['union'] + "List")
- fdef.write(generate_type_cleanup(expr['union'] + "List"))
- ret += generate_type_cleanup_decl(expr['union'])
- fdef.write(generate_type_cleanup(expr['union']))
- elif expr.has_key('alternate'):
- ret += generate_union(expr, 'alternate') + "\n"
- ret += generate_type_cleanup_decl(expr['alternate'] + "List")
- fdef.write(generate_type_cleanup(expr['alternate'] + "List"))
- ret += generate_type_cleanup_decl(expr['alternate'])
- fdef.write(generate_type_cleanup(expr['alternate']))
- elif expr.has_key('enum'):
- ret += "\n" + generate_type_cleanup_decl(expr['enum'] + "List")
- fdef.write(generate_type_cleanup(expr['enum'] + "List"))
- else:
- continue
- fdecl.write(ret)
+schema = QAPISchema(input_file)
+gen = QAPISchemaGenTypeVisitor()
+schema.visit(gen)
+fdef.write(gen.defn)
+fdecl.write(gen.decl)
close_output(fdef, fdecl)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index a9e5aab..257b4d4 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -39,8 +39,8 @@
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
-# FIXME generated struct UserDefFlatUnion has members for direct base
-# UserDefUnionBase, but lacks members for indirect base UserDefZero
+# FIXME generated visit_type_UserDefFlatUnion_fields() fails to visit
+# members of indirect base UserDefZero
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (9 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-05 16:03 ` Eric Blake
2015-08-06 22:53 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 12/32] qapi-commands: Convert to QAPISchemaVisitor Markus Armbruster
` (20 subsequent siblings)
31 siblings, 2 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Fixes flat unions to visit the base's base members (the previous
commit merely added them to the struct). Same test case.
Patch's effect on visit_type_UserDefFlatUnion():
static void visit_type_UserDefFlatUnion_fields(Visitor *m, UserDefFlatUnion **obj, Error **errp)
{
Error *err = NULL;
+ visit_type_int(m, &(*obj)->integer, "integer", &err);
+ if (err) {
+ goto out;
+ }
visit_type_str(m, &(*obj)->string, "string", &err);
if (err) {
goto out;
Test cases updated for the bug fix.
Fixes alternates to generate a visitor for their implicit enumeration
type. None of them are currently used, obviously. Example:
block-core.json's BlockdevRef now generates
visit_type_BlockdevRefKind().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-visit.py | 260 +++++++++++++-------------------
tests/qapi-schema/qapi-schema-test.json | 3 -
tests/test-qmp-input-strict.c | 2 +-
tests/test-qmp-input-visitor.c | 4 +-
4 files changed, 106 insertions(+), 163 deletions(-)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index c493964..38857f7 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -12,7 +12,6 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
-from ordereddict import OrderedDict
from qapi import *
import re
@@ -24,13 +23,13 @@ def generate_visit_implicit_struct(type):
return ''
implicit_structs_seen.add(type)
ret = ''
- if type not in struct_fields_seen:
+ if type.name not in struct_fields_seen:
# Need a forward declaration
ret += mcgen('''
static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error **errp);
''',
- c_type=type_name(type))
+ c_type=type.c_name())
ret += mcgen('''
@@ -46,7 +45,7 @@ static void visit_type_implicit_%(c_type)s(Visitor *m, %(c_type)s **obj, Error *
error_propagate(errp, err);
}
''',
- c_type=type_name(type))
+ c_type=type.c_name())
return ret
def generate_visit_struct_fields(name, members, base = None):
@@ -74,24 +73,24 @@ if (err) {
goto out;
}
''',
- type=type_name(base), c_name=c_name('base'))
+ type=base.c_name(), c_name=c_name('base'))
- for argname, argentry, optional in parse_args(members):
- if optional:
+ for memb in members:
+ if memb.optional:
ret += mcgen('''
visit_optional(m, &(*obj)->has_%(c_name)s, "%(name)s", &err);
if (!err && (*obj)->has_%(c_name)s) {
''',
- c_name=c_name(argname), name=argname)
+ c_name=c_name(memb.name), name=memb.name)
push_indent()
ret += mcgen('''
visit_type_%(type)s(m, &(*obj)->%(c_name)s, "%(name)s", &err);
''',
- type=type_name(argentry), c_name=c_name(argname),
- name=argname)
+ type=memb.type.c_name(), c_name=c_name(memb.name),
+ name=memb.name)
- if optional:
+ if memb.optional:
pop_indent()
ret += mcgen('''
}
@@ -136,12 +135,7 @@ def generate_visit_struct_body(name):
return ret
-def generate_visit_struct(expr):
-
- name = expr['struct']
- members = expr['data']
- base = expr.get('base')
-
+def gen_visit_struct(name, base, members):
ret = generate_visit_struct_fields(name, members, base)
ret += mcgen('''
@@ -158,10 +152,10 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
''')
return ret
-def generate_visit_list(name):
+def gen_visit_list(name, element_type):
return mcgen('''
-void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp)
+void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
GenericList *i, **prev;
@@ -174,8 +168,8 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, E
for (prev = (GenericList **)obj;
!err && (i = visit_next_list(m, prev, &err)) != NULL;
prev = &i) {
- %(name)sList *native_i = (%(name)sList *)i;
- visit_type_%(name)s(m, &native_i->value, NULL, &err);
+ %(name)s *native_i = (%(name)s *)i;
+ visit_type_%(c_elt_type)s(m, &native_i->value, NULL, &err);
}
error_propagate(errp, err);
@@ -185,7 +179,8 @@ out:
error_propagate(errp, err);
}
''',
- name=type_name(name))
+ name=c_name(name),
+ c_elt_type=element_type.c_name())
def generate_visit_enum(name):
return mcgen('''
@@ -197,7 +192,7 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error
''',
c_name=c_name(name), name=name)
-def generate_visit_alternate(name, members):
+def gen_visit_alternate(name, variants):
ret = mcgen('''
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
@@ -216,25 +211,17 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
''',
name=c_name(name))
- # For alternate, always use the default enum type automatically generated
- # as name + 'Kind'
- disc_type = c_name(name) + 'Kind'
-
- for key in members:
- assert (members[key] in builtin_types.keys()
- or find_struct(members[key])
- or find_union(members[key])
- or find_enum(members[key])), "Invalid alternate member"
-
- enum_full_value = c_enum_const(disc_type, key)
+ for var in variants.variants:
+ enum_full_value = c_enum_const(variants.tag_member.type.name,
+ var.name)
ret += mcgen('''
case %(enum_full_value)s:
visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, name, &err);
break;
''',
enum_full_value = enum_full_value,
- c_type = type_name(members[key]),
- c_name = c_name(key))
+ c_type=var.type.c_name(),
+ c_name=c_name(var.name))
ret += mcgen('''
default:
@@ -251,35 +238,17 @@ out:
return ret
-
-def generate_visit_union(expr):
-
- name = expr['union']
- members = expr['data']
-
- base = expr.get('base')
- discriminator = expr.get('discriminator')
-
- enum_define = discriminator_find_enum_define(expr)
- if enum_define:
- # Use the enum type as discriminator
- ret = ""
- disc_type = c_name(enum_define['enum_name'])
- else:
- # There will always be a discriminator in the C switch code, by default
- # it is an enum type generated silently
- ret = generate_visit_enum(name + 'Kind')
- disc_type = c_name(name) + 'Kind'
+def gen_visit_union(name, base, variants):
+ ret = ''
if base:
- assert discriminator
- base_fields = find_struct(base)['data'].copy()
- del base_fields[discriminator]
- ret += generate_visit_struct_fields(name, base_fields)
+ members = [m for m in base.members if m != variants.tag_member]
+ ret += generate_visit_struct_fields(name, members)
- if discriminator:
- for key in members:
- ret += generate_visit_implicit_struct(members[key])
+ for var in variants.variants:
+ # TODO ugly special case for simple union
+ if not var.simple_union_type():
+ ret += generate_visit_implicit_struct(var.type)
ret += mcgen('''
@@ -304,41 +273,41 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error
''',
name=c_name(name))
- if not discriminator:
- tag = 'kind'
- disc_key = "type"
- else:
- tag = discriminator
- disc_key = discriminator
+ disc_key = variants.tag_member.name
+ if not variants.tag_name:
+ # we pointlessly use a different key for simple unions
+ disc_key = 'type'
ret += mcgen('''
- visit_type_%(disc_type)s(m, &(*obj)->%(c_tag)s, "%(disc_key)s", &err);
+ visit_type_%(disc_type)s(m, &(*obj)->%(c_name)s, "%(disc_key)s", &err);
if (err) {
goto out_obj;
}
if (!visit_start_union(m, !!(*obj)->data, &err) || err) {
goto out_obj;
}
- switch ((*obj)->%(c_tag)s) {
+ switch ((*obj)->%(c_name)s) {
''',
- disc_type = disc_type,
- c_tag=c_name(tag),
+ disc_type=variants.tag_member.type.c_name(),
+ c_name=c_name(variants.tag_member.name),
disc_key = disc_key)
- for key in members:
- if not discriminator:
+ for var in variants.variants:
+ # TODO ugly special case for simple union
+ simple_union_type = var.simple_union_type()
+ if simple_union_type:
fmt = 'visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, "data", &err);'
else:
fmt = 'visit_type_implicit_%(c_type)s(m, &(*obj)->%(c_name)s, &err);'
- enum_full_value = c_enum_const(disc_type, key)
+ enum_full_value = c_enum_const(variants.tag_member.type.name, var.name)
ret += mcgen('''
case %(enum_full_value)s:
''' + fmt + '''
break;
''',
enum_full_value = enum_full_value,
- c_type=type_name(members[key]),
- c_name=c_name(key))
+ c_type=(simple_union_type or var.type).c_name(),
+ c_name=c_name(var.name))
ret += mcgen('''
default:
@@ -359,38 +328,61 @@ out:
return ret
-def generate_declaration(name, builtin_type=False):
- ret = ""
- if not builtin_type:
- name = c_name(name)
- ret += mcgen('''
-
-void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp);
-''',
- name=name)
-
- ret += mcgen('''
-void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp);
-''',
- name=name)
-
- return ret
-
-def generate_enum_declaration(name):
- ret = mcgen('''
-void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, Error **errp);
-''',
- name=c_name(name))
-
- return ret
-
-def generate_decl_enum(name):
+def gen_visit_decl(name, scalar=False):
+ c_type = c_name(name) + ' *'
+ if not scalar:
+ c_type += '*'
return mcgen('''
-
-void visit_type_%(name)s(Visitor *m, %(name)s *obj, const char *name, Error **errp);
+void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error **errp);
''',
- name=c_name(name))
+ c_name=c_name(name), c_type=c_type)
+class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
+ def __init__(self):
+ self.decl = None
+ self.defn = None
+ self.btin = None
+ def visit_begin(self):
+ self.decl = ''
+ self.defn = ''
+ self.btin = guardstart('QAPI_VISIT_BUILTIN')
+ def visit_end(self):
+ # To avoid header dependency hell, we always generate
+ # declarations for built-in types in our header files and
+ # simply guard them. See also do_builtins (command line
+ # option -b).
+ self.btin += guardend('QAPI_VISIT_BUILTIN')
+ self.decl = self.btin + self.decl
+ self.btin = None
+ def visit_enum_type(self, name, info, values):
+ self.decl += gen_visit_decl(name, scalar=True)
+ self.defn += generate_visit_enum(name)
+ def visit_array_type(self, name, info, element_type):
+ decl = gen_visit_decl(name)
+ defn = gen_visit_list(name, element_type)
+ if isinstance(element_type, QAPISchemaBuiltinType):
+ self.btin += decl
+ if do_builtins:
+ self.defn += defn
+ else:
+ self.decl += decl
+ self.defn += defn
+ def visit_object_type(self, name, info, base, members, variants):
+ if info:
+ self.decl += gen_visit_decl(name)
+ if variants:
+ assert not members # not implemented
+ self.defn += gen_visit_union(name, base, variants)
+ else:
+ self.defn += gen_visit_struct(name, base, members)
+ def visit_alternate_type(self, name, info, variants):
+ self.decl += gen_visit_decl(name)
+ self.defn += gen_visit_alternate(name, variants)
+
+# If you link code generated from multiple schemata, you want only one
+# instance include the code generated for built-in types. Generate it
+# only when do_builtins, enabled by command line option -b. See also
+# QAPISchemaGenVisitVisitor.visit_end().
do_builtins = False
(input_file, output_dir, do_c, do_h, prefix, opts) = \
@@ -446,56 +438,10 @@ fdecl.write(mcgen('''
''',
prefix=prefix))
-exprs = QAPISchema(input_file).get_exprs()
-
-# to avoid header dependency hell, we always generate declarations
-# for built-in types in our header files and simply guard them
-fdecl.write(guardstart("QAPI_VISIT_BUILTIN_VISITOR_DECL"))
-for typename in builtin_types.keys():
- fdecl.write(generate_declaration(typename, builtin_type=True))
-fdecl.write(guardend("QAPI_VISIT_BUILTIN_VISITOR_DECL"))
-
-# ...this doesn't work for cases where we link in multiple objects that
-# have the functions defined, so we use -b option to provide control
-# over these cases
-if do_builtins:
- for typename in builtin_types.keys():
- fdef.write(generate_visit_list(typename))
-
-for expr in exprs:
- if expr.has_key('struct'):
- ret = generate_visit_struct(expr)
- ret += generate_visit_list(expr['struct'])
- fdef.write(ret)
-
- ret = generate_declaration(expr['struct'])
- fdecl.write(ret)
- elif expr.has_key('union'):
- ret = generate_visit_union(expr)
- ret += generate_visit_list(expr['union'])
- fdef.write(ret)
-
- enum_define = discriminator_find_enum_define(expr)
- ret = ""
- if not enum_define:
- ret = generate_decl_enum('%sKind' % expr['union'])
- ret += generate_declaration(expr['union'])
- fdecl.write(ret)
- elif expr.has_key('alternate'):
- ret = generate_visit_alternate(expr['alternate'], expr['data'])
- ret += generate_visit_list(expr['alternate'])
- fdef.write(ret)
-
- ret = generate_decl_enum('%sKind' % expr['alternate'])
- ret += generate_declaration(expr['alternate'])
- fdecl.write(ret)
- elif expr.has_key('enum'):
- ret = generate_visit_list(expr['enum'])
- ret += generate_visit_enum(expr['enum'])
- fdef.write(ret)
-
- ret = generate_decl_enum(expr['enum'])
- ret += generate_enum_declaration(expr['enum'])
- fdecl.write(ret)
+schema = QAPISchema(input_file)
+gen = QAPISchemaGenVisitVisitor()
+schema.visit(gen)
+fdef.write(gen.defn)
+fdecl.write(gen.decl)
close_output(fdef, fdecl)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 257b4d4..b182174 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -39,8 +39,6 @@
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
-# FIXME generated visit_type_UserDefFlatUnion_fields() fails to visit
-# members of indirect base UserDefZero
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
@@ -57,7 +55,6 @@
{ 'alternate': 'UserDefAlternate',
'data': { 'uda': 'UserDefA', 's': 'str', 'i': 'int' } }
-# FIXME only a declaration of visit_type_UserDefAlternateKind() generated
{ 'struct': 'UserDefC',
'data': { 'string1': 'str', 'string2': 'str' } }
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 68f855b..00c3e29 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -167,9 +167,9 @@ static void test_validate_union_flat(TestInputVisitorData *data,
v = validate_test_init(data,
"{ 'enum1': 'value1', "
+ "'integer': 41, "
"'string': 'str', "
"'boolean': true }");
- /* TODO when generator bug is fixed, add 'integer': 41 */
visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
g_assert(!err);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index a5cfefa..508c11a 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -307,15 +307,15 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,
v = visitor_input_test_init(data,
"{ 'enum1': 'value1', "
+ "'integer': 41, "
"'string': 'str', "
"'boolean': true }");
- /* TODO when generator bug is fixed, add 'integer': 41 */
visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
g_assert(err == NULL);
g_assert_cmpint(tmp->enum1, ==, ENUM_ONE_VALUE1);
g_assert_cmpstr(tmp->string, ==, "str");
- /* TODO g_assert_cmpint(tmp->integer, ==, 41); */
+ g_assert_cmpint(tmp->integer, ==, 41);
g_assert_cmpint(tmp->value1->boolean, ==, true);
qapi_free_UserDefFlatUnion(tmp);
}
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 12/32] qapi-commands: Convert to QAPISchemaVisitor
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (10 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 13/32] qapi: De-duplicate enum code generation Markus Armbruster
` (19 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Output unchanged apart from reordering and white-space.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-commands.py | 154 ++++++++++++++++++++++++++---------------------
scripts/qapi.py | 2 +-
2 files changed, 86 insertions(+), 70 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 12bdc4c..1c363c2 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -12,21 +12,22 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
-from ordereddict import OrderedDict
from qapi import *
import re
def generate_command_decl(name, args, ret_type):
arglist=""
- for argname, argtype, optional in parse_args(args):
- argtype = c_type(argtype, is_param=True)
- if optional:
- arglist += "bool has_%s, " % c_name(argname)
- arglist += "%s %s, " % (argtype, c_name(argname))
+ if args:
+ for memb in args.members:
+ argtype = memb.type.c_type(is_param=True)
+ if memb.optional:
+ arglist += "bool has_%s, " % c_name(memb.name)
+ arglist += "%s %s, " % (argtype, c_name(memb.name))
return mcgen('''
%(ret_type)s qmp_%(name)s(%(args)sError **errp);
''',
- ret_type=c_type(ret_type), name=c_name(name),
+ ret_type=(ret_type and ret_type.c_type()) or 'void',
+ name=c_name(name),
args=arglist)
def gen_err_check(err):
@@ -45,10 +46,11 @@ def gen_sync_call(name, args, ret_type):
retval=""
if ret_type:
retval = "retval = "
- for argname, argtype, optional in parse_args(args):
- if optional:
- arglist += "has_%s, " % c_name(argname)
- arglist += "%s, " % (c_name(argname))
+ if args:
+ for memb in args.members:
+ if memb.optional:
+ arglist += "has_%s, " % c_name(memb.name)
+ arglist += "%s, " % c_name(memb.name)
push_indent()
ret = mcgen('''
%(retval)sqmp_%(name)s(%(args)s&local_err);
@@ -68,7 +70,7 @@ def gen_visitor_input_containers_decl(args):
ret = ""
push_indent()
- if len(args) > 0:
+ if args:
ret += mcgen('''
QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
QapiDeallocVisitor *md;
@@ -81,22 +83,26 @@ Visitor *v;
def gen_visitor_input_vars_decl(args):
ret = ""
push_indent()
- for argname, argtype, optional in parse_args(args):
- if optional:
- ret += mcgen('''
+
+ if args:
+ for memb in args.members:
+ if memb.optional:
+ ret += mcgen('''
bool has_%(argname)s = false;
''',
- argname=c_name(argname))
- if is_c_ptr(argtype):
- ret += mcgen('''
+ argname=c_name(memb.name))
+ if is_c_ptr(memb.type.c_type()):
+ ret += mcgen('''
%(argtype)s %(argname)s = NULL;
''',
- argname=c_name(argname), argtype=c_type(argtype))
- else:
- ret += mcgen('''
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
+ else:
+ ret += mcgen('''
%(argtype)s %(argname)s = {0};
''',
- argname=c_name(argname), argtype=c_type(argtype))
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
pop_indent()
return ret
@@ -106,7 +112,7 @@ def gen_visitor_input_block(args, dealloc=False):
errparg = '&local_err'
errarg = 'local_err'
- if len(args) == 0:
+ if not args:
return ret
push_indent()
@@ -124,25 +130,26 @@ v = qapi_dealloc_get_visitor(md);
v = qmp_input_get_visitor(mi);
''')
- for argname, argtype, optional in parse_args(args):
- if optional:
+ for memb in args.members:
+ if memb.optional:
ret += mcgen('''
visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
''',
- c_name=c_name(argname), name=argname, errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ errp=errparg)
ret += gen_err_check(errarg)
ret += mcgen('''
if (has_%(c_name)s) {
''',
- c_name=c_name(argname))
+ c_name=c_name(memb.name))
push_indent()
ret += mcgen('''
visit_type_%(visitor)s(v, &%(c_name)s, "%(name)s", %(errp)s);
''',
- c_name=c_name(argname), name=argname, argtype=argtype,
- visitor=type_name(argtype), errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ visitor=memb.type.c_name(), errp=errparg)
ret += gen_err_check(errarg)
- if optional:
+ if memb.optional:
pop_indent()
ret += mcgen('''
}
@@ -160,6 +167,7 @@ def gen_marshal_output(name, ret_type):
return ""
ret = mcgen('''
+
static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
@@ -183,8 +191,8 @@ out:
qapi_dealloc_visitor_cleanup(md);
}
''',
- c_ret_type=c_type(ret_type), c_name=c_name(name),
- visitor=type_name(ret_type))
+ c_ret_type=ret_type.c_type(), c_name=c_name(name),
+ visitor=ret_type.c_name())
return ret
@@ -198,6 +206,7 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
hdr = gen_marshal_input_decl(name, middle_mode)
ret = mcgen('''
+
%(header)s
{
Error *local_err = NULL;
@@ -208,9 +217,9 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
ret += mcgen('''
%(c_type)s retval;
''',
- c_type=c_type(ret_type))
+ c_type=ret_type.c_type())
- if len(args) > 0:
+ if args:
ret += gen_visitor_input_containers_decl(args)
ret += gen_visitor_input_vars_decl(args) + '\n'
ret += gen_visitor_input_block(args) + '\n'
@@ -237,21 +246,23 @@ out:
''')
return ret
-def gen_registry(commands):
- registry=""
+def gen_register_command(name, success_response):
push_indent()
- for cmd in commands:
- options = 'QCO_NO_OPTIONS'
- if not cmd.get('success-response', True):
- options = 'QCO_NO_SUCCESS_RESP'
+ options = 'QCO_NO_OPTIONS'
+ if not success_response:
+ options = 'QCO_NO_SUCCESS_RESP'
- registry += mcgen('''
+ ret = mcgen('''
qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
''',
- name=cmd['command'], c_name=c_name(cmd['command']),
+ name=name, c_name=c_name(name),
opts=options)
pop_indent()
+ return ret
+
+def gen_registry(registry):
ret = mcgen('''
+
static void qmp_init_marshal(void)
{
''')
@@ -263,6 +274,32 @@ qapi_init(qmp_init_marshal);
''')
return ret
+class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
+ def __init__(self):
+ self.decl = None
+ self.defn = None
+ self.regy = None
+ def visit_begin(self):
+ self.decl = ''
+ self.defn = ''
+ self.regy = ''
+ def visit_end(self):
+ if not middle_mode:
+ self.defn += gen_registry(self.regy)
+ self.regy = None
+ def visit_command(self, name, info, arg_type, ret_type,
+ gen, success_response):
+ if not gen:
+ return
+ self.decl += generate_command_decl(name, arg_type, ret_type)
+ if ret_type:
+ self.defn += gen_marshal_output(name, ret_type)
+ if middle_mode:
+ self.decl += gen_marshal_input_decl(name, middle_mode) + ';\n'
+ self.defn += gen_marshal_input(name, arg_type, ret_type, middle_mode)
+ if not middle_mode:
+ self.regy += gen_register_command(name, success_response)
+
middle_mode = False
(input_file, output_dir, do_c, do_h, prefix, opts) = \
@@ -272,10 +309,6 @@ for o, a in opts:
if o in ("-m", "--middle"):
middle_mode = True
-exprs = QAPISchema(input_file).get_exprs()
-commands = filter(lambda expr: expr.has_key('command'), exprs)
-commands = filter(lambda expr: not expr.has_key('gen'), commands)
-
c_comment = '''
/*
* schema-defined QMP->QAPI command dispatch
@@ -331,29 +364,12 @@ fdecl.write(mcgen('''
#include "qapi/error.h"
''',
- prefix=prefix))
+ prefix=prefix))
-for cmd in commands:
- arglist = []
- ret_type = None
- if cmd.has_key('data'):
- arglist = cmd['data']
- if cmd.has_key('returns'):
- ret_type = cmd['returns']
- ret = generate_command_decl(cmd['command'], arglist, ret_type)
- fdecl.write(ret)
- if ret_type:
- ret = gen_marshal_output(cmd['command'], ret_type) + "\n"
- fdef.write(ret)
-
- if middle_mode:
- fdecl.write('%s;\n' % gen_marshal_input_decl(cmd['command'], middle_mode))
-
- ret = gen_marshal_input(cmd['command'], arglist, ret_type, middle_mode) + "\n"
- fdef.write(ret)
-
-if not middle_mode:
- ret = gen_registry(commands)
- fdef.write(ret)
+schema = QAPISchema(input_file)
+gen = QAPISchemaGenCommandVisitor()
+schema.visit(gen)
+fdef.write(gen.defn)
+fdecl.write(gen.decl)
close_output(fdef, fdecl)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 019d22c..bc0e5a2 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1375,7 +1375,7 @@ def c_type(value, is_param=False):
return c_name(value) + pointer_suffix
def is_c_ptr(value):
- return c_type(value).endswith(pointer_suffix)
+ return value.endswith(pointer_suffix)
def genindent(count):
ret = ""
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 13/32] qapi: De-duplicate enum code generation
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (11 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 12/32] qapi-commands: Convert to QAPISchemaVisitor Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 14/32] qapi-event: Eliminate global variable event_enum_value Markus Armbruster
` (18 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Duplicated in commit 21cd70d. Yes, we can't import qapi-types, but
that's no excuse. Move the helpers from qapi-types.py to qapi.py, and
replace the duplicates in qapi-event.py.
The generated event enumeration type's lookup table becomes
const-correct (see commit 2e4450f), and uses explicit indexes instead
of relying on order (see commit 912ae9c).
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 9 ++++---
scripts/qapi-event.py | 67 +++-----------------------------------------------
scripts/qapi-types.py | 55 -----------------------------------------
scripts/qapi.py | 55 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 64 insertions(+), 122 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 9553d3a..beba9c0 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -823,9 +823,9 @@ Example:
QDECREF(qmp);
}
- const char *example_QAPIEvent_lookup[] = {
- "MY_EVENT",
- NULL,
+ const char *const example_QAPIEvent_lookup[] = {
+ [EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
+ [EXAMPLE_QAPI_EVENT_MAX] = NULL,
};
$ cat qapi-generated/example-qapi-event.h
[Uninteresting stuff omitted...]
@@ -840,10 +840,11 @@ Example:
void qapi_event_send_my_event(Error **errp);
- extern const char *example_QAPIEvent_lookup[];
typedef enum example_QAPIEvent {
EXAMPLE_QAPI_EVENT_MY_EVENT = 0,
EXAMPLE_QAPI_EVENT_MAX = 1,
} example_QAPIEvent;
+ extern const char *const example_QAPIEvent_lookup[];
+
#endif
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aec2d32..aed45d6 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -153,63 +153,6 @@ def generate_event_implement(api_name, event_name, params):
return ret
-
-# Following are the functions that generate an enum type for all defined
-# events, similar to qapi-types.py. Here we already have enum name and
-# values which were generated before and recorded in event_enum_*. It also
-# works around the issue that "import qapi-types" can't work.
-
-def generate_event_enum_decl(event_enum_name, event_enum_values):
- lookup_decl = mcgen('''
-
-extern const char *%(event_enum_name)s_lookup[];
-''',
- event_enum_name = event_enum_name)
-
- enum_decl = mcgen('''
-typedef enum %(event_enum_name)s {
-''',
- event_enum_name = event_enum_name)
-
- # append automatically generated _MAX value
- enum_max_value = c_enum_const(event_enum_name, "MAX")
- enum_values = event_enum_values + [ enum_max_value ]
-
- i = 0
- for value in enum_values:
- enum_decl += mcgen('''
- %(value)s = %(i)d,
-''',
- value = value,
- i = i)
- i += 1
-
- enum_decl += mcgen('''
-} %(event_enum_name)s;
-''',
- event_enum_name = event_enum_name)
-
- return lookup_decl + enum_decl
-
-def generate_event_enum_lookup(event_enum_name, event_enum_strings):
- ret = mcgen('''
-
-const char *%(event_enum_name)s_lookup[] = {
-''',
- event_enum_name = event_enum_name)
-
- for string in event_enum_strings:
- ret += mcgen('''
- "%(string)s",
-''',
- string = string)
-
- ret += mcgen('''
- NULL,
-};
-''')
- return ret
-
(input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
c_comment = '''
@@ -266,8 +209,7 @@ fdecl.write(mcgen('''
exprs = QAPISchema(input_file).get_exprs()
event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
-event_enum_values = []
-event_enum_strings = []
+event_names = []
for expr in exprs:
if expr.has_key('event'):
@@ -286,12 +228,11 @@ for expr in exprs:
fdef.write(ret)
# Record it, and generate enum later
- event_enum_values.append(event_enum_value)
- event_enum_strings.append(event_name)
+ event_names.append(event_name)
-ret = generate_event_enum_decl(event_enum_name, event_enum_values)
+ret = generate_enum(event_enum_name, event_names)
fdecl.write(ret)
-ret = generate_event_enum_lookup(event_enum_name, event_enum_strings)
+ret = generate_enum_lookup(event_enum_name, event_names)
fdef.write(ret)
close_output(fdef, fdecl)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 181bf4b..203051c 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -80,61 +80,6 @@ struct %(name)s {
return ret
-def generate_enum_lookup(name, values):
- ret = mcgen('''
-
-const char *const %(name)s_lookup[] = {
-''',
- name=c_name(name))
- for value in values:
- index = c_enum_const(name, value)
- ret += mcgen('''
- [%(index)s] = "%(value)s",
-''',
- index = index, value = value)
-
- max_index = c_enum_const(name, 'MAX')
- ret += mcgen('''
- [%(max_index)s] = NULL,
-};
-''',
- max_index=max_index)
- return ret
-
-def generate_enum(name, values):
- name = c_name(name)
- lookup_decl = mcgen('''
-
-extern const char *const %(name)s_lookup[];
-''',
- name=name)
-
- enum_decl = mcgen('''
-
-typedef enum %(name)s {
-''',
- name=name)
-
- # append automatically generated _MAX value
- enum_values = values + [ 'MAX' ]
-
- i = 0
- for value in enum_values:
- enum_full_value = c_enum_const(name, value)
- enum_decl += mcgen('''
- %(enum_full_value)s = %(i)d,
-''',
- enum_full_value = enum_full_value,
- i=i)
- i += 1
-
- enum_decl += mcgen('''
-} %(name)s;
-''',
- name=name)
-
- return enum_decl + lookup_decl
-
def gen_alternate_qtypes_decl(name):
return mcgen('''
diff --git a/scripts/qapi.py b/scripts/qapi.py
index bc0e5a2..8f1fb66 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1429,6 +1429,61 @@ def guardend(name):
''',
name=guardname(name))
+def generate_enum_lookup(name, values):
+ ret = mcgen('''
+
+const char *const %(name)s_lookup[] = {
+''',
+ name=c_name(name))
+ for value in values:
+ index = c_enum_const(name, value)
+ ret += mcgen('''
+ [%(index)s] = "%(value)s",
+''',
+ index = index, value = value)
+
+ max_index = c_enum_const(name, 'MAX')
+ ret += mcgen('''
+ [%(max_index)s] = NULL,
+};
+''',
+ max_index=max_index)
+ return ret
+
+def generate_enum(name, values):
+ name = c_name(name)
+ lookup_decl = mcgen('''
+
+extern const char *const %(name)s_lookup[];
+''',
+ name=name)
+
+ enum_decl = mcgen('''
+
+typedef enum %(name)s {
+''',
+ name=name)
+
+ # append automatically generated _MAX value
+ enum_values = values + [ 'MAX' ]
+
+ i = 0
+ for value in enum_values:
+ enum_full_value = c_enum_const(name, value)
+ enum_decl += mcgen('''
+ %(enum_full_value)s = %(i)d,
+''',
+ enum_full_value = enum_full_value,
+ i=i)
+ i += 1
+
+ enum_decl += mcgen('''
+} %(name)s;
+''',
+ name=name)
+
+ return enum_decl + lookup_decl
+
#
# Common command line parsing
#
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 14/32] qapi-event: Eliminate global variable event_enum_value
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (12 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 13/32] qapi: De-duplicate enum code generation Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 15/32] qapi-event: Convert to QAPISchemaVisitor, fixing data with base Markus Armbruster
` (17 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-event.py | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aed45d6..537da17 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -137,7 +137,7 @@ def generate_event_implement(api_name, event_name, params):
emit(%(event_enum_value)s, qmp, &local_err);
""",
- event_enum_value = event_enum_value)
+ event_enum_value = c_enum_const(event_enum_name, event_name))
# step 5: clean up
if params:
@@ -223,7 +223,6 @@ for expr in exprs:
fdecl.write(ret)
# We need an enum value per event
- event_enum_value = c_enum_const(event_enum_name, event_name)
ret = generate_event_implement(api_name, event_name, params)
fdef.write(ret)
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 15/32] qapi-event: Convert to QAPISchemaVisitor, fixing data with base
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (13 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 14/32] qapi-event: Eliminate global variable event_enum_value Markus Armbruster
@ 2015-08-04 15:57 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 16/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
` (16 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:57 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Fixes events whose data is struct with base to include the struct's
base members. Test case is qapi-schema-test.json's event
__org.qemu_x-command:
{ 'event': '__ORG.QEMU_X-EVENT', 'data': '__org.qemu_x-Struct' }
{ 'struct': '__org.qemu_x-Struct', 'base': '__org.qemu_x-Base',
'data': { '__org.qemu_x-member2': 'str' } }
{ 'struct': '__org.qemu_x-Base',
'data': { '__org.qemu_x-member1': '__org.qemu_x-Enum' } }
Patch's effect on generated qapi_event_send___org_qemu_x_event():
-void qapi_event_send___org_qemu_x_event(const char *__org_qemu_x_member2,
+void qapi_event_send___org_qemu_x_event(__org_qemu_x_Enum __org_qemu_x_member1,
+ const char *__org_qemu_x_member2,
Error **errp)
{
QDict *qmp;
@@ -224,6 +225,10 @@ void qapi_event_send___org_qemu_x_event(
goto clean;
}
+ visit_type___org_qemu_x_Enum(v, &__org_qemu_x_member1, "__org.qemu_x-member1", &local_err);
+ if (local_err) {
+ goto clean;
+ }
visit_type_str(v, (char **)&__org_qemu_x_member2, "__org.qemu_x-member2", &local_err);
if (local_err) {
goto clean;
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-event.py | 84 ++++++++++++++++-----------------
tests/qapi-schema/qapi-schema-test.json | 3 --
2 files changed, 42 insertions(+), 45 deletions(-)
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 537da17..316f369 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -2,14 +2,15 @@
# QAPI event generator
#
# Copyright (c) 2014 Wenchao Xia
+# Copyright (c) 2015 Red Hat Inc.
#
# Authors:
# Wenchao Xia <wenchaoqemu@gmail.com>
+# Markus Armbruster <armbru@redhat.com>
#
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
-from ordereddict import OrderedDict
from qapi import *
def _generate_event_api_name(event_name, params):
@@ -17,13 +18,13 @@ def _generate_event_api_name(event_name, params):
l = len(api_name)
if params:
- for argname, argentry, optional in parse_args(params):
- if optional:
- api_name += "bool has_%s,\n" % c_name(argname)
+ for m in params.members:
+ if m.optional:
+ api_name += "bool has_%s,\n" % c_name(m.name)
api_name += "".ljust(l)
- api_name += "%s %s,\n" % (c_type(argentry, is_param=True),
- c_name(argname))
+ api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
+ c_name(m.name))
api_name += "".ljust(l)
api_name += "Error **errp)"
@@ -51,7 +52,7 @@ def generate_event_implement(api_name, event_name, params):
""",
api_name = api_name)
- if params:
+ if params and params.members:
ret += mcgen("""
QmpOutputVisitor *qov;
Visitor *v;
@@ -72,7 +73,7 @@ def generate_event_implement(api_name, event_name, params):
event_name = event_name)
# step 3: visit the params if params != None
- if params:
+ if params and params.members:
ret += mcgen("""
qov = qmp_output_visitor_new();
g_assert(qov);
@@ -89,15 +90,15 @@ def generate_event_implement(api_name, event_name, params):
""",
event_name = event_name)
- for argname, argentry, optional in parse_args(params):
- if optional:
+ for memb in params.members:
+ if memb.optional:
ret += mcgen("""
if (has_%(var)s) {
""",
- var = c_name(argname))
+ var=c_name(memb.name))
push_indent()
- if argentry == "str":
+ if memb.type.name == "str":
var_type = "(char **)"
else:
var_type = ""
@@ -109,11 +110,11 @@ def generate_event_implement(api_name, event_name, params):
}
""",
var_type = var_type,
- var = c_name(argname),
- type = type_name(argentry),
- name = argname)
+ var=c_name(memb.name),
+ type=memb.type.c_name(),
+ name=memb.name)
- if optional:
+ if memb.optional:
pop_indent()
ret += mcgen("""
}
@@ -140,7 +141,7 @@ def generate_event_implement(api_name, event_name, params):
event_enum_value = c_enum_const(event_enum_name, event_name))
# step 5: clean up
- if params:
+ if params and params.members:
ret += mcgen("""
clean:
qmp_output_visitor_cleanup(qov);
@@ -153,6 +154,25 @@ def generate_event_implement(api_name, event_name, params):
return ret
+class QAPISchemaGenEventVisitor(QAPISchemaVisitor):
+ def __init__(self):
+ self.decl = None
+ self.defn = None
+ self.event_names = None
+ def visit_begin(self):
+ self.decl = ''
+ self.defn = ''
+ self.event_names = []
+ def visit_end(self):
+ self.decl += generate_enum(event_enum_name, self.event_names)
+ self.defn += generate_enum_lookup(event_enum_name, self.event_names)
+ self.event_names = None
+ def visit_event(self, name, info, arg_type):
+ api_name = _generate_event_api_name(name, arg_type)
+ self.decl += generate_event_declaration(api_name)
+ self.defn += generate_event_implement(api_name, name, arg_type)
+ self.event_names.append(name)
+
(input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
c_comment = '''
@@ -206,32 +226,12 @@ fdecl.write(mcgen('''
''',
prefix=prefix))
-exprs = QAPISchema(input_file).get_exprs()
-
event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
-event_names = []
-for expr in exprs:
- if expr.has_key('event'):
- event_name = expr['event']
- params = expr.get('data')
- if params and len(params) == 0:
- params = None
-
- api_name = _generate_event_api_name(event_name, params)
- ret = generate_event_declaration(api_name)
- fdecl.write(ret)
-
- # We need an enum value per event
- ret = generate_event_implement(api_name, event_name, params)
- fdef.write(ret)
-
- # Record it, and generate enum later
- event_names.append(event_name)
-
-ret = generate_enum(event_enum_name, event_names)
-fdecl.write(ret)
-ret = generate_enum_lookup(event_enum_name, event_names)
-fdef.write(ret)
+schema = QAPISchema(input_file)
+gen = QAPISchemaGenEventVisitor()
+schema.visit(gen)
+fdef.write(gen.defn)
+fdecl.write(gen.decl)
close_output(fdef, fdecl)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index b182174..90b4740 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -126,9 +126,6 @@
{ 'alternate': '__org.qemu_x-Alt',
'data': { '__org.qemu_x-branch': 'str', 'b': '__org.qemu_x-Base' } }
{ 'event': '__ORG.QEMU_X-EVENT', 'data': '__org.qemu_x-Struct' }
-# FIXME generated qapi_event_send___org_qemu_x_event() has only a
-# parameter for data's member __org_qemu_x_member2, none for its base
-# __org.qemu_x-Base's member __org_qemu_x_member1
{ 'command': '__org.qemu_x-command',
'data': { 'a': ['__org.qemu_x-Enum'], 'b': ['__org.qemu_x-Struct'],
'c': '__org.qemu_x-Union2', 'd': '__org.qemu_x-Alt' },
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 16/32] qapi: Generate comments to simplify splitting for review
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (14 preceding siblings ...)
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 15/32] qapi-event: Convert to QAPISchemaVisitor, fixing data with base Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 23:02 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 17/32] Revert "qapi: Generate comments to simplify splitting for review" Markus Armbruster
` (15 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
The effect of the previous few patches on generated files is hard to
review, because stuff gets generated in different order, which renders
diffs of the generated files useless.
To get reviewable diffs, we need to split the generated files into
suitable parts: put every top-level declaration in a file named like
the thing declared, so we can diff top-level declarations regardless
of their order in the generated files.
Since I don't feel like parsing C, simply generate a // comment
identifying the declaration right before each top-level declaration.
This lets us split with a simple shell loop:
for i in q*-{commands,marshal,event,types,visit}.[ch]
do
csplit -n 4 -s "$i" '/^\/\//' '{*}'
for j in xx*
do
read h t <$j
[ "$h" == "//" ] || t=""
mv $j "$i-${j#xx}-${t/ /-}"
done
done
Splits each file F into F-NUMB-ID, where NUMB counts up from 0001, and
ID comes from the // comment.
To check the split's sane, we can run
for i in q*-{commands,marshal,event,types,visit}.[ch]
do cat $i-* | diff $i -
done
We got a commit similar to this one right before the patches we want
to check. If you have that commit's split files in directory o, and
this commit's split files in directory n, you can diff them with this
loop:
for i in `(ls o; ls n) | sed -n 's/-[0-9][0-9][0-9][0-9]-/-*-/p' | sort | uniq`
do
diff -uBp o/$i n/$i
done
Complete annotated output:
* QAPIEvent_lookup[] and enum QAPIEvent
Innocent changes due to "De-duplicate enum code generation".
--- o/qapi-event.c-0037-def-QAPIEvent_lookup 2015-08-04 12:47:16.479980075 +0200
+++ n/qapi-event.c-0037-def-QAPIEvent_lookup 2015-08-04 13:02:46.336626925 +0200
@@ -1,39 +1,39 @@
// def QAPIEvent_lookup
-const char *QAPIEvent_lookup[] = {
- "BLOCK_IMAGE_CORRUPTED",
- "BLOCK_IO_ERROR",
- "BLOCK_JOB_COMPLETED",
- "BLOCK_JOB_CANCELLED",
- "BLOCK_JOB_ERROR",
- "BLOCK_JOB_READY",
- "BLOCK_WRITE_THRESHOLD",
- "DEVICE_TRAY_MOVED",
- "SHUTDOWN",
- "POWERDOWN",
- "RESET",
- "STOP",
- "RESUME",
- "SUSPEND",
- "SUSPEND_DISK",
- "WAKEUP",
- "RTC_CHANGE",
- "WATCHDOG",
- "DEVICE_DELETED",
- "NIC_RX_FILTER_CHANGED",
- "VNC_CONNECTED",
- "VNC_INITIALIZED",
- "VNC_DISCONNECTED",
- "SPICE_CONNECTED",
- "SPICE_INITIALIZED",
- "SPICE_DISCONNECTED",
- "SPICE_MIGRATE_COMPLETED",
- "MIGRATION",
- "ACPI_DEVICE_OST",
- "BALLOON_CHANGE",
- "GUEST_PANICKED",
- "QUORUM_FAILURE",
- "QUORUM_REPORT_BAD",
- "VSERPORT_CHANGE",
- "MEM_UNPLUG_ERROR",
- NULL,
+const char *const QAPIEvent_lookup[] = {
+ [QAPI_EVENT_ACPI_DEVICE_OST] = "ACPI_DEVICE_OST",
+ [QAPI_EVENT_BALLOON_CHANGE] = "BALLOON_CHANGE",
+ [QAPI_EVENT_BLOCK_IMAGE_CORRUPTED] = "BLOCK_IMAGE_CORRUPTED",
+ [QAPI_EVENT_BLOCK_IO_ERROR] = "BLOCK_IO_ERROR",
+ [QAPI_EVENT_BLOCK_JOB_CANCELLED] = "BLOCK_JOB_CANCELLED",
+ [QAPI_EVENT_BLOCK_JOB_COMPLETED] = "BLOCK_JOB_COMPLETED",
+ [QAPI_EVENT_BLOCK_JOB_ERROR] = "BLOCK_JOB_ERROR",
+ [QAPI_EVENT_BLOCK_JOB_READY] = "BLOCK_JOB_READY",
+ [QAPI_EVENT_BLOCK_WRITE_THRESHOLD] = "BLOCK_WRITE_THRESHOLD",
+ [QAPI_EVENT_DEVICE_DELETED] = "DEVICE_DELETED",
+ [QAPI_EVENT_DEVICE_TRAY_MOVED] = "DEVICE_TRAY_MOVED",
+ [QAPI_EVENT_GUEST_PANICKED] = "GUEST_PANICKED",
+ [QAPI_EVENT_MEM_UNPLUG_ERROR] = "MEM_UNPLUG_ERROR",
+ [QAPI_EVENT_MIGRATION] = "MIGRATION",
+ [QAPI_EVENT_NIC_RX_FILTER_CHANGED] = "NIC_RX_FILTER_CHANGED",
+ [QAPI_EVENT_POWERDOWN] = "POWERDOWN",
+ [QAPI_EVENT_QUORUM_FAILURE] = "QUORUM_FAILURE",
+ [QAPI_EVENT_QUORUM_REPORT_BAD] = "QUORUM_REPORT_BAD",
+ [QAPI_EVENT_RESET] = "RESET",
+ [QAPI_EVENT_RESUME] = "RESUME",
+ [QAPI_EVENT_RTC_CHANGE] = "RTC_CHANGE",
+ [QAPI_EVENT_SHUTDOWN] = "SHUTDOWN",
+ [QAPI_EVENT_SPICE_CONNECTED] = "SPICE_CONNECTED",
+ [QAPI_EVENT_SPICE_DISCONNECTED] = "SPICE_DISCONNECTED",
+ [QAPI_EVENT_SPICE_INITIALIZED] = "SPICE_INITIALIZED",
+ [QAPI_EVENT_SPICE_MIGRATE_COMPLETED] = "SPICE_MIGRATE_COMPLETED",
+ [QAPI_EVENT_STOP] = "STOP",
+ [QAPI_EVENT_SUSPEND] = "SUSPEND",
+ [QAPI_EVENT_SUSPEND_DISK] = "SUSPEND_DISK",
+ [QAPI_EVENT_VNC_CONNECTED] = "VNC_CONNECTED",
+ [QAPI_EVENT_VNC_DISCONNECTED] = "VNC_DISCONNECTED",
+ [QAPI_EVENT_VNC_INITIALIZED] = "VNC_INITIALIZED",
+ [QAPI_EVENT_VSERPORT_CHANGE] = "VSERPORT_CHANGE",
+ [QAPI_EVENT_WAKEUP] = "WAKEUP",
+ [QAPI_EVENT_WATCHDOG] = "WATCHDOG",
+ [QAPI_EVENT_MAX] = NULL,
};
--- o/qapi-event.h-0037-decl-QAPIEvent_lookup 2015-08-04 12:47:16.480980103 +0200
+++ n/qapi-event.h-0038-decl-QAPIEvent_lookup 2015-08-04 13:02:46.387628338 +0200
@@ -1,2 +1,3 @@
// decl QAPIEvent_lookup
-extern const char *QAPIEvent_lookup[];
+extern const char *const QAPIEvent_lookup[];
+
--- o/qapi-event.h-0038-typedef-QAPIEvent 2015-08-04 12:47:16.480980103 +0200
+++ n/qapi-event.h-0037-typedef-QAPIEvent 2015-08-04 13:02:46.387628338 +0200
@@ -1,40 +1,40 @@
// typedef QAPIEvent
typedef enum QAPIEvent {
- QAPI_EVENT_BLOCK_IMAGE_CORRUPTED = 0,
- QAPI_EVENT_BLOCK_IO_ERROR = 1,
- QAPI_EVENT_BLOCK_JOB_COMPLETED = 2,
- QAPI_EVENT_BLOCK_JOB_CANCELLED = 3,
- QAPI_EVENT_BLOCK_JOB_ERROR = 4,
- QAPI_EVENT_BLOCK_JOB_READY = 5,
- QAPI_EVENT_BLOCK_WRITE_THRESHOLD = 6,
- QAPI_EVENT_DEVICE_TRAY_MOVED = 7,
- QAPI_EVENT_SHUTDOWN = 8,
- QAPI_EVENT_POWERDOWN = 9,
- QAPI_EVENT_RESET = 10,
- QAPI_EVENT_STOP = 11,
- QAPI_EVENT_RESUME = 12,
- QAPI_EVENT_SUSPEND = 13,
- QAPI_EVENT_SUSPEND_DISK = 14,
- QAPI_EVENT_WAKEUP = 15,
- QAPI_EVENT_RTC_CHANGE = 16,
- QAPI_EVENT_WATCHDOG = 17,
- QAPI_EVENT_DEVICE_DELETED = 18,
- QAPI_EVENT_NIC_RX_FILTER_CHANGED = 19,
- QAPI_EVENT_VNC_CONNECTED = 20,
- QAPI_EVENT_VNC_INITIALIZED = 21,
- QAPI_EVENT_VNC_DISCONNECTED = 22,
- QAPI_EVENT_SPICE_CONNECTED = 23,
+ QAPI_EVENT_ACPI_DEVICE_OST = 0,
+ QAPI_EVENT_BALLOON_CHANGE = 1,
+ QAPI_EVENT_BLOCK_IMAGE_CORRUPTED = 2,
+ QAPI_EVENT_BLOCK_IO_ERROR = 3,
+ QAPI_EVENT_BLOCK_JOB_CANCELLED = 4,
+ QAPI_EVENT_BLOCK_JOB_COMPLETED = 5,
+ QAPI_EVENT_BLOCK_JOB_ERROR = 6,
+ QAPI_EVENT_BLOCK_JOB_READY = 7,
+ QAPI_EVENT_BLOCK_WRITE_THRESHOLD = 8,
+ QAPI_EVENT_DEVICE_DELETED = 9,
+ QAPI_EVENT_DEVICE_TRAY_MOVED = 10,
+ QAPI_EVENT_GUEST_PANICKED = 11,
+ QAPI_EVENT_MEM_UNPLUG_ERROR = 12,
+ QAPI_EVENT_MIGRATION = 13,
+ QAPI_EVENT_NIC_RX_FILTER_CHANGED = 14,
+ QAPI_EVENT_POWERDOWN = 15,
+ QAPI_EVENT_QUORUM_FAILURE = 16,
+ QAPI_EVENT_QUORUM_REPORT_BAD = 17,
+ QAPI_EVENT_RESET = 18,
+ QAPI_EVENT_RESUME = 19,
+ QAPI_EVENT_RTC_CHANGE = 20,
+ QAPI_EVENT_SHUTDOWN = 21,
+ QAPI_EVENT_SPICE_CONNECTED = 22,
+ QAPI_EVENT_SPICE_DISCONNECTED = 23,
QAPI_EVENT_SPICE_INITIALIZED = 24,
- QAPI_EVENT_SPICE_DISCONNECTED = 25,
- QAPI_EVENT_SPICE_MIGRATE_COMPLETED = 26,
- QAPI_EVENT_MIGRATION = 27,
- QAPI_EVENT_ACPI_DEVICE_OST = 28,
- QAPI_EVENT_BALLOON_CHANGE = 29,
- QAPI_EVENT_GUEST_PANICKED = 30,
- QAPI_EVENT_QUORUM_FAILURE = 31,
- QAPI_EVENT_QUORUM_REPORT_BAD = 32,
- QAPI_EVENT_VSERPORT_CHANGE = 33,
- QAPI_EVENT_MEM_UNPLUG_ERROR = 34,
+ QAPI_EVENT_SPICE_MIGRATE_COMPLETED = 25,
+ QAPI_EVENT_STOP = 26,
+ QAPI_EVENT_SUSPEND = 27,
+ QAPI_EVENT_SUSPEND_DISK = 28,
+ QAPI_EVENT_VNC_CONNECTED = 29,
+ QAPI_EVENT_VNC_DISCONNECTED = 30,
+ QAPI_EVENT_VNC_INITIALIZED = 31,
+ QAPI_EVENT_VSERPORT_CHANGE = 32,
+ QAPI_EVENT_WAKEUP = 33,
+ QAPI_EVENT_WATCHDOG = 34,
QAPI_EVENT_MAX = 35,
} QAPIEvent;
* Guards QAPI_TYPES_BUILTIN_STRUCT_DECL and
QAPI_TYPES_BUILTIN_CLEANUP_DECL replaced by just QAPI_TYPES_BUILTIN
diff: o/qapi-types.h-*-guardend-QAPI_TYPES_BUILTIN: No such file or directory
diff: n/qapi-types.h-*-guardend-QAPI_TYPES_BUILTIN_CLEANUP_DECL: No such file or directory
diff: n/qapi-types.h-*-guardend-QAPI_TYPES_BUILTIN_STRUCT_DECL: No such file or directory
diff: o/qapi-types.h-*-guardstart-QAPI_TYPES_BUILTIN: No such file or directory
diff: n/qapi-types.h-*-guardstart-QAPI_TYPES_BUILTIN_CLEANUP_DECL: No such file or directory
diff: n/qapi-types.h-*-guardstart-QAPI_TYPES_BUILTIN_STRUCT_DECL: No such file or directory
* A bunch of new forward declarations due to different ordering
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsArchipelago_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsBlkdebug_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsBlkverify_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsFile_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsGenericCOWFormat_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsGenericFormat_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsNull_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsQcow2_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsQuorum_fields: No such file or directory
diff: o/qapi-visit.c-*-decl-visit_type_BlockdevOptionsVVFAT_fields: No such file or directory
* New visitors due to "qapi-visit: Convert to QAPISchemaVisitor, fixing bugs"
diff: o/qapi-visit.c-*-def-visit_type_BlockdevRefKind: No such file or directory
diff: o/qapi-visit.c-*-def-visit_type_Qcow2OverlapChecksKind: No such file or directory
* Guard QAPI_VISIT_BUILTIN_VISITOR_DECL replaced by QAPI_VISIT_BUILTIN
diff: o/qapi-visit.h-*-guardend-QAPI_VISIT_BUILTIN: No such file or directory
diff: n/qapi-visit.h-*-guardend-QAPI_VISIT_BUILTIN_VISITOR_DECL: No such file or directory
diff: o/qapi-visit.h-*-guardstart-QAPI_VISIT_BUILTIN: No such file or directory
diff: n/qapi-visit.h-*-guardstart-QAPI_VISIT_BUILTIN_VISITOR_DECL: No such file or directory
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 7 +++++++
scripts/qapi-event.py | 9 +++++++++
scripts/qapi-types.py | 8 ++++++++
scripts/qapi-visit.py | 9 +++++++++
scripts/qapi.py | 8 ++++++++
5 files changed, 41 insertions(+)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 1c363c2..7e1072a 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -24,6 +24,7 @@ def generate_command_decl(name, args, ret_type):
arglist += "bool has_%s, " % c_name(memb.name)
arglist += "%s %s, " % (argtype, c_name(memb.name))
return mcgen('''
+// decl qmp_%(name)s
%(ret_type)s qmp_%(name)s(%(args)sError **errp);
''',
ret_type=(ret_type and ret_type.c_type()) or 'void',
@@ -168,6 +169,7 @@ def gen_marshal_output(name, ret_type):
ret = mcgen('''
+// def qmp_marshal_output_%(c_name)s
static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
@@ -207,10 +209,12 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
ret = mcgen('''
+// def qmp_marshal_input_%(name)s
%(header)s
{
Error *local_err = NULL;
''',
+ name=name,
header=hdr)
if ret_type:
@@ -263,6 +267,7 @@ qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
def gen_registry(registry):
ret = mcgen('''
+// def qmp_init_marshal
static void qmp_init_marshal(void)
{
''')
@@ -270,6 +275,7 @@ static void qmp_init_marshal(void)
ret += mcgen('''
}
+// decl qapi_init
qapi_init(qmp_init_marshal);
''')
return ret
@@ -295,6 +301,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
if ret_type:
self.defn += gen_marshal_output(name, ret_type)
if middle_mode:
+ self.decl += '// decl gen_marshal_input_%s\n' % name
self.decl += gen_marshal_input_decl(name, middle_mode) + ';\n'
self.defn += gen_marshal_input(name, arg_type, ret_type, middle_mode)
if not middle_mode:
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 316f369..e1cfab9 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -11,6 +11,7 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
+import re;
from qapi import *
def _generate_event_api_name(event_name, params):
@@ -34,22 +35,30 @@ def _generate_event_api_name(event_name, params):
# Following are the core functions that generate C APIs to emit event.
def generate_event_declaration(api_name):
+ match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
+ name = match.group(1)
return mcgen('''
+// decl %(name)s
%(api_name)s;
''',
+ name=name,
api_name = api_name)
def generate_event_implement(api_name, event_name, params):
+ match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
+ name = match.group(1)
# step 1: declare any variables
ret = mcgen("""
+// def %(name)s
%(api_name)s
{
QDict *qmp;
Error *local_err = NULL;
QMPEventFuncEmit emit;
""",
+ name=name,
api_name = api_name)
if params and params.members:
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 203051c..1ab5008 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -16,6 +16,7 @@ from qapi import *
def gen_fwd_object_or_array(name):
return mcgen('''
+// typedef %(name)s
typedef struct %(name)s %(name)s;
''',
name=c_name(name))
@@ -23,6 +24,7 @@ typedef struct %(name)s %(name)s;
def gen_array(name, element_type):
return mcgen('''
+// struct %(name)s
struct %(name)s {
union {
%(c_type)s value;
@@ -57,6 +59,7 @@ def generate_struct_fields(members):
def gen_struct(name, base, members):
ret = mcgen('''
+// struct %(name)s
struct %(name)s {
''',
name=c_name(name))
@@ -83,6 +86,7 @@ struct %(name)s {
def gen_alternate_qtypes_decl(name):
return mcgen('''
+// decl %(c_name)s_qtypes
extern const int %(c_name)s_qtypes[];
''',
c_name=c_name(name))
@@ -90,6 +94,7 @@ extern const int %(c_name)s_qtypes[];
def gen_alternate_qtypes(name, variants):
ret = mcgen('''
+// def %(name)s_qtypes
const int %(name)s_qtypes[QTYPE_MAX] = {
''',
name=c_name(name))
@@ -115,6 +120,7 @@ def gen_union(name, base, variants):
ret = mcgen('''
+// struct %(name)s
struct %(name)s {
''',
name=name)
@@ -166,6 +172,7 @@ struct %(name)s {
def generate_type_cleanup_decl(name):
ret = mcgen('''
+// decl qapi_free_%(name)s
void qapi_free_%(name)s(%(name)s *obj);
''',
name=c_name(name))
@@ -174,6 +181,7 @@ void qapi_free_%(name)s(%(name)s *obj);
def generate_type_cleanup(name):
ret = mcgen('''
+// def qapi_free_%(name)s
void qapi_free_%(name)s(%(name)s *obj)
{
QapiDeallocVisitor *md;
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 38857f7..5814d8f 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -27,12 +27,14 @@ def generate_visit_implicit_struct(type):
# Need a forward declaration
ret += mcgen('''
+// decl visit_type_%(c_type)s_fields
static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error **errp);
''',
c_type=type.c_name())
ret += mcgen('''
+// def visit_type_implicit_%(c_type)s
static void visit_type_implicit_%(c_type)s(Visitor *m, %(c_type)s **obj, Error **errp)
{
Error *err = NULL;
@@ -58,6 +60,7 @@ def generate_visit_struct_fields(name, members, base = None):
ret += mcgen('''
+// def visit_type_%(name)s_fields
static void visit_type_%(name)s_fields(Visitor *m, %(name)s **obj, Error **errp)
{
Error *err = NULL;
@@ -140,6 +143,7 @@ def gen_visit_struct(name, base, members):
ret += mcgen('''
+// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
''',
@@ -155,6 +159,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
def gen_visit_list(name, element_type):
return mcgen('''
+// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -185,6 +190,7 @@ out:
def generate_visit_enum(name):
return mcgen('''
+// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error **errp)
{
visit_type_enum(m, (int *)obj, %(c_name)s_lookup, "%(name)s", name, errp);
@@ -195,6 +201,7 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error
def gen_visit_alternate(name, variants):
ret = mcgen('''
+// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -252,6 +259,7 @@ def gen_visit_union(name, base, variants):
ret += mcgen('''
+// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -333,6 +341,7 @@ def gen_visit_decl(name, scalar=False):
if not scalar:
c_type += '*'
return mcgen('''
+// decl visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error **errp);
''',
c_name=c_name(name), c_type=c_type)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8f1fb66..e462b87 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1415,6 +1415,7 @@ def guardname(filename):
def guardstart(name):
return mcgen('''
+// guardstart %(name)s
#ifndef %(name)s
#define %(name)s
@@ -1424,6 +1425,7 @@ def guardstart(name):
def guardend(name):
return mcgen('''
+// guardend %(name)s
#endif /* %(name)s */
''',
@@ -1432,6 +1434,7 @@ def guardend(name):
def generate_enum_lookup(name, values):
ret = mcgen('''
+// def %(name)s_lookup
const char *const %(name)s_lookup[] = {
''',
name=c_name(name))
@@ -1454,12 +1457,14 @@ def generate_enum(name, values):
name = c_name(name)
lookup_decl = mcgen('''
+// decl %(name)s_lookup
extern const char *const %(name)s_lookup[];
''',
name=name)
enum_decl = mcgen('''
+// typedef %(name)s
typedef enum %(name)s {
''',
name=name)
@@ -1562,12 +1567,14 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
fdecl = maybe_open(do_h, h_file, 'w')
fdef.write(mcgen('''
+// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
''',
comment = c_comment))
fdecl.write(mcgen('''
+// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
#ifndef %(guard)s
@@ -1580,6 +1587,7 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
def close_output(fdef, fdecl):
fdecl.write('''
+// epilogue
#endif
''')
fdecl.close()
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 17/32] Revert "qapi: Generate comments to simplify splitting for review"
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (15 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 16/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null() Markus Armbruster
` (14 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
This reverts commit 33048fffa49175099f8a93056884f27cc8845ea3.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 7 -------
scripts/qapi-event.py | 9 ---------
scripts/qapi-types.py | 8 --------
scripts/qapi-visit.py | 9 ---------
scripts/qapi.py | 8 --------
5 files changed, 41 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 7e1072a..1c363c2 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -24,7 +24,6 @@ def generate_command_decl(name, args, ret_type):
arglist += "bool has_%s, " % c_name(memb.name)
arglist += "%s %s, " % (argtype, c_name(memb.name))
return mcgen('''
-// decl qmp_%(name)s
%(ret_type)s qmp_%(name)s(%(args)sError **errp);
''',
ret_type=(ret_type and ret_type.c_type()) or 'void',
@@ -169,7 +168,6 @@ def gen_marshal_output(name, ret_type):
ret = mcgen('''
-// def qmp_marshal_output_%(c_name)s
static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
@@ -209,12 +207,10 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
ret = mcgen('''
-// def qmp_marshal_input_%(name)s
%(header)s
{
Error *local_err = NULL;
''',
- name=name,
header=hdr)
if ret_type:
@@ -267,7 +263,6 @@ qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
def gen_registry(registry):
ret = mcgen('''
-// def qmp_init_marshal
static void qmp_init_marshal(void)
{
''')
@@ -275,7 +270,6 @@ static void qmp_init_marshal(void)
ret += mcgen('''
}
-// decl qapi_init
qapi_init(qmp_init_marshal);
''')
return ret
@@ -301,7 +295,6 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
if ret_type:
self.defn += gen_marshal_output(name, ret_type)
if middle_mode:
- self.decl += '// decl gen_marshal_input_%s\n' % name
self.decl += gen_marshal_input_decl(name, middle_mode) + ';\n'
self.defn += gen_marshal_input(name, arg_type, ret_type, middle_mode)
if not middle_mode:
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index e1cfab9..316f369 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -11,7 +11,6 @@
# This work is licensed under the terms of the GNU GPL, version 2.
# See the COPYING file in the top-level directory.
-import re;
from qapi import *
def _generate_event_api_name(event_name, params):
@@ -35,30 +34,22 @@ def _generate_event_api_name(event_name, params):
# Following are the core functions that generate C APIs to emit event.
def generate_event_declaration(api_name):
- match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
- name = match.group(1)
return mcgen('''
-// decl %(name)s
%(api_name)s;
''',
- name=name,
api_name = api_name)
def generate_event_implement(api_name, event_name, params):
- match = re.match(r'void ([A-Za-z0-9_]+)', api_name)
- name = match.group(1)
# step 1: declare any variables
ret = mcgen("""
-// def %(name)s
%(api_name)s
{
QDict *qmp;
Error *local_err = NULL;
QMPEventFuncEmit emit;
""",
- name=name,
api_name = api_name)
if params and params.members:
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 1ab5008..203051c 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -16,7 +16,6 @@ from qapi import *
def gen_fwd_object_or_array(name):
return mcgen('''
-// typedef %(name)s
typedef struct %(name)s %(name)s;
''',
name=c_name(name))
@@ -24,7 +23,6 @@ typedef struct %(name)s %(name)s;
def gen_array(name, element_type):
return mcgen('''
-// struct %(name)s
struct %(name)s {
union {
%(c_type)s value;
@@ -59,7 +57,6 @@ def generate_struct_fields(members):
def gen_struct(name, base, members):
ret = mcgen('''
-// struct %(name)s
struct %(name)s {
''',
name=c_name(name))
@@ -86,7 +83,6 @@ struct %(name)s {
def gen_alternate_qtypes_decl(name):
return mcgen('''
-// decl %(c_name)s_qtypes
extern const int %(c_name)s_qtypes[];
''',
c_name=c_name(name))
@@ -94,7 +90,6 @@ extern const int %(c_name)s_qtypes[];
def gen_alternate_qtypes(name, variants):
ret = mcgen('''
-// def %(name)s_qtypes
const int %(name)s_qtypes[QTYPE_MAX] = {
''',
name=c_name(name))
@@ -120,7 +115,6 @@ def gen_union(name, base, variants):
ret = mcgen('''
-// struct %(name)s
struct %(name)s {
''',
name=name)
@@ -172,7 +166,6 @@ struct %(name)s {
def generate_type_cleanup_decl(name):
ret = mcgen('''
-// decl qapi_free_%(name)s
void qapi_free_%(name)s(%(name)s *obj);
''',
name=c_name(name))
@@ -181,7 +174,6 @@ void qapi_free_%(name)s(%(name)s *obj);
def generate_type_cleanup(name):
ret = mcgen('''
-// def qapi_free_%(name)s
void qapi_free_%(name)s(%(name)s *obj)
{
QapiDeallocVisitor *md;
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 5814d8f..38857f7 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -27,14 +27,12 @@ def generate_visit_implicit_struct(type):
# Need a forward declaration
ret += mcgen('''
-// decl visit_type_%(c_type)s_fields
static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error **errp);
''',
c_type=type.c_name())
ret += mcgen('''
-// def visit_type_implicit_%(c_type)s
static void visit_type_implicit_%(c_type)s(Visitor *m, %(c_type)s **obj, Error **errp)
{
Error *err = NULL;
@@ -60,7 +58,6 @@ def generate_visit_struct_fields(name, members, base = None):
ret += mcgen('''
-// def visit_type_%(name)s_fields
static void visit_type_%(name)s_fields(Visitor *m, %(name)s **obj, Error **errp)
{
Error *err = NULL;
@@ -143,7 +140,6 @@ def gen_visit_struct(name, base, members):
ret += mcgen('''
-// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
''',
@@ -159,7 +155,6 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
def gen_visit_list(name, element_type):
return mcgen('''
-// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -190,7 +185,6 @@ out:
def generate_visit_enum(name):
return mcgen('''
-// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error **errp)
{
visit_type_enum(m, (int *)obj, %(c_name)s_lookup, "%(name)s", name, errp);
@@ -201,7 +195,6 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error
def gen_visit_alternate(name, variants):
ret = mcgen('''
-// def visit_type_%(name)s
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -259,7 +252,6 @@ def gen_visit_union(name, base, variants):
ret += mcgen('''
-// def visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -341,7 +333,6 @@ def gen_visit_decl(name, scalar=False):
if not scalar:
c_type += '*'
return mcgen('''
-// decl visit_type_%(c_name)s
void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error **errp);
''',
c_name=c_name(name), c_type=c_type)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index e462b87..8f1fb66 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1415,7 +1415,6 @@ def guardname(filename):
def guardstart(name):
return mcgen('''
-// guardstart %(name)s
#ifndef %(name)s
#define %(name)s
@@ -1425,7 +1424,6 @@ def guardstart(name):
def guardend(name):
return mcgen('''
-// guardend %(name)s
#endif /* %(name)s */
''',
@@ -1434,7 +1432,6 @@ def guardend(name):
def generate_enum_lookup(name, values):
ret = mcgen('''
-// def %(name)s_lookup
const char *const %(name)s_lookup[] = {
''',
name=c_name(name))
@@ -1457,14 +1454,12 @@ def generate_enum(name, values):
name = c_name(name)
lookup_decl = mcgen('''
-// decl %(name)s_lookup
extern const char *const %(name)s_lookup[];
''',
name=name)
enum_decl = mcgen('''
-// typedef %(name)s
typedef enum %(name)s {
''',
name=name)
@@ -1567,14 +1562,12 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
fdecl = maybe_open(do_h, h_file, 'w')
fdef.write(mcgen('''
-// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
''',
comment = c_comment))
fdecl.write(mcgen('''
-// prologue
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
%(comment)s
#ifndef %(guard)s
@@ -1587,7 +1580,6 @@ def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
def close_output(fdef, fdecl):
fdecl.write('''
-// epilogue
#endif
''')
fdecl.close()
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null()
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (16 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 17/32] Revert "qapi: Generate comments to simplify splitting for review" Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-05 16:13 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 19/32] qapi: Clean up after recent conversions to QAPISchemaVisitor Markus Armbruster
` (13 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
is_c_ptr() looks whether the end of the C text for the type looks like
a pointer. Works, but is fragile.
We now have a better tool: use QAPISchemaType method c_null(). The
initializers for non-pointers become prettier: 0, false or the
enumeration constant with the value 0 instead of {0}.
One place looks suspicious: we initialize pointers, but not
non-pointers. Either the initialization is superfluous and should be
deleted, or the non-pointers need it as well, or something subtle is
going on and needs a comment. Since I lack the time to figure it out
now, mark it FIXME.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 16 +++++-----------
scripts/qapi.py | 3 ---
2 files changed, 5 insertions(+), 14 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 1c363c2..577b514 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -91,18 +91,12 @@ def gen_visitor_input_vars_decl(args):
bool has_%(argname)s = false;
''',
argname=c_name(memb.name))
- if is_c_ptr(memb.type.c_type()):
- ret += mcgen('''
-%(argtype)s %(argname)s = NULL;
+ ret += mcgen('''
+%(c_type)s %(c_name)s = %(c_null)s;
''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
- else:
- ret += mcgen('''
-%(argtype)s %(argname)s = {0};
-''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
+ c_name=c_name(memb.name),
+ c_type=memb.type.c_type(),
+ c_null=memb.type.c_null())
pop_indent()
return ret
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8f1fb66..855ca97 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1374,9 +1374,6 @@ def c_type(value, is_param=False):
assert isinstance(value, str) and value != ""
return c_name(value) + pointer_suffix
-def is_c_ptr(value):
- return value.endswith(pointer_suffix)
-
def genindent(count):
ret = ""
for i in range(count):
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 19/32] qapi: Clean up after recent conversions to QAPISchemaVisitor
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (17 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null() Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-05 16:29 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 20/32] qapi-visit: Rearrange code a bit Markus Armbruster
` (12 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Generate just 'FOO' instead of 'struct FOO' when possible.
Drop helper functions that are now unused.
Make pylint reasonably happy.
Rename generate_FOO() functions to gen_FOO() for consistency.
Use more consistent and sensible variable names.
Consistently use c_ for mapping keys when their value is a C
identifier or type.
Simplify gen_enum() and gen_visit_union()
Consistently use single quotes for C text string literals.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
docs/qapi-code-gen.txt | 2 +-
scripts/qapi-commands.py | 129 +++++++++++++++++++++++-----------------------
scripts/qapi-event.py | 121 ++++++++++++++++++++-----------------------
scripts/qapi-types.py | 64 +++++++++++------------
scripts/qapi-visit.py | 119 +++++++++++++++++++++---------------------
scripts/qapi.py | 131 +++++++++--------------------------------------
6 files changed, 239 insertions(+), 327 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index beba9c0..5cf401f 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -592,7 +592,7 @@ Example:
UserDefOne *value;
uint64_t padding;
};
- struct UserDefOneList *next;
+ UserDefOneList *next;
};
void qapi_free_UserDefOneList(UserDefOneList *obj);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 577b514..c165c63 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -15,20 +15,20 @@
from qapi import *
import re
-def generate_command_decl(name, args, ret_type):
- arglist=""
- if args:
- for memb in args.members:
- argtype = memb.type.c_type(is_param=True)
+def gen_command_decl(name, arg_type, ret_type):
+ argstr = ''
+ if arg_type:
+ for memb in arg_type.members:
if memb.optional:
- arglist += "bool has_%s, " % c_name(memb.name)
- arglist += "%s %s, " % (argtype, c_name(memb.name))
+ argstr += 'bool has_%s, ' % c_name(memb.name)
+ argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
+ c_name(memb.name))
return mcgen('''
-%(ret_type)s qmp_%(name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
''',
- ret_type=(ret_type and ret_type.c_type()) or 'void',
- name=c_name(name),
- args=arglist)
+ c_type=(ret_type and ret_type.c_type()) or 'void',
+ c_name=c_name(name),
+ args=argstr)
def gen_err_check(err):
if not err:
@@ -40,37 +40,40 @@ if (%(err)s) {
''',
err=err)
-def gen_sync_call(name, args, ret_type):
- ret = ""
- arglist=""
- retval=""
- if ret_type:
- retval = "retval = "
- if args:
- for memb in args.members:
+def gen_call(name, arg_type, ret_type):
+ ret = ''
+
+ argstr = ''
+ if arg_type:
+ for memb in arg_type.members:
if memb.optional:
- arglist += "has_%s, " % c_name(memb.name)
- arglist += "%s, " % c_name(memb.name)
+ argstr += 'has_%s, ' % c_name(memb.name)
+ argstr += '%s, ' % c_name(memb.name)
+
+ lhs = ''
+ if ret_type:
+ lhs = 'retval = '
+
push_indent()
ret = mcgen('''
-%(retval)sqmp_%(name)s(%(args)s&local_err);
+%(lhs)sqmp_%(c_name)s(%(args)s&local_err);
''',
- name=c_name(name), args=arglist, retval=retval)
+ c_name=c_name(name), args=argstr, lhs=lhs)
if ret_type:
ret += gen_err_check('local_err')
ret += mcgen('''
qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
''',
- c_name=c_name(name))
+ c_name=c_name(name))
pop_indent()
return ret
-def gen_visitor_input_containers_decl(args):
- ret = ""
+def gen_visitor_input_containers_decl(arg_type):
+ ret = ''
push_indent()
- if args:
+ if arg_type:
ret += mcgen('''
QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
QapiDeallocVisitor *md;
@@ -80,17 +83,17 @@ Visitor *v;
return ret
-def gen_visitor_input_vars_decl(args):
- ret = ""
+def gen_visitor_input_vars_decl(arg_type):
+ ret = ''
push_indent()
- if args:
- for memb in args.members:
+ if arg_type:
+ for memb in arg_type.members:
if memb.optional:
ret += mcgen('''
-bool has_%(argname)s = false;
+bool has_%(c_name)s = false;
''',
- argname=c_name(memb.name))
+ c_name=c_name(memb.name))
ret += mcgen('''
%(c_type)s %(c_name)s = %(c_null)s;
''',
@@ -101,19 +104,19 @@ bool has_%(argname)s = false;
pop_indent()
return ret
-def gen_visitor_input_block(args, dealloc=False):
- ret = ""
+def gen_visitor_input_block(arg_type, dealloc=False):
+ ret = ''
errparg = '&local_err'
errarg = 'local_err'
- if not args:
+ if not arg_type:
return ret
push_indent()
if dealloc:
errparg = 'NULL'
- errarg = None;
+ errarg = None
ret += mcgen('''
qmp_input_visitor_cleanup(mi);
md = qapi_dealloc_visitor_new();
@@ -124,7 +127,7 @@ v = qapi_dealloc_get_visitor(md);
v = qmp_input_get_visitor(mi);
''')
- for memb in args.members:
+ for memb in arg_type.members:
if memb.optional:
ret += mcgen('''
visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
@@ -138,10 +141,10 @@ if (has_%(c_name)s) {
c_name=c_name(memb.name))
push_indent()
ret += mcgen('''
-visit_type_%(visitor)s(v, &%(c_name)s, "%(name)s", %(errp)s);
+visit_type_%(c_type)s(v, &%(c_name)s, "%(name)s", %(errp)s);
''',
c_name=c_name(memb.name), name=memb.name,
- visitor=memb.type.c_name(), errp=errparg)
+ c_type=memb.type.c_name(), errp=errparg)
ret += gen_err_check(errarg)
if memb.optional:
pop_indent()
@@ -158,11 +161,11 @@ qapi_dealloc_visitor_cleanup(md);
def gen_marshal_output(name, ret_type):
if not ret_type:
- return ""
+ return ''
ret = mcgen('''
-static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_out, Error **errp)
+static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -170,7 +173,7 @@ static void qmp_marshal_output_%(c_name)s(%(c_ret_type)s ret_in, QObject **ret_o
Visitor *v;
v = qmp_output_get_visitor(mo);
- visit_type_%(visitor)s(v, &ret_in, "unused", &local_err);
+ visit_type_%(c_name)s(v, &ret_in, "unused", &local_err);
if (local_err) {
goto out;
}
@@ -181,23 +184,23 @@ out:
qmp_output_visitor_cleanup(mo);
md = qapi_dealloc_visitor_new();
v = qapi_dealloc_get_visitor(md);
- visit_type_%(visitor)s(v, &ret_in, "unused", NULL);
+ visit_type_%(c_name)s(v, &ret_in, "unused", NULL);
qapi_dealloc_visitor_cleanup(md);
}
''',
- c_ret_type=ret_type.c_type(), c_name=c_name(name),
- visitor=ret_type.c_name())
+ c_type=ret_type.c_type(), c_cmd_name=c_name(name),
+ c_name=ret_type.c_name())
return ret
-def gen_marshal_input_decl(name, middle_mode):
+def gen_marshal_input_decl(name):
ret = 'void qmp_marshal_input_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
if not middle_mode:
- ret = "static " + ret
+ ret = 'static ' + ret
return ret
-def gen_marshal_input(name, args, ret_type, middle_mode):
- hdr = gen_marshal_input_decl(name, middle_mode)
+def gen_marshal_input(name, arg_type, ret_type):
+ hdr = gen_marshal_input_decl(name)
ret = mcgen('''
@@ -213,10 +216,10 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
''',
c_type=ret_type.c_type())
- if args:
- ret += gen_visitor_input_containers_decl(args)
- ret += gen_visitor_input_vars_decl(args) + '\n'
- ret += gen_visitor_input_block(args) + '\n'
+ if arg_type:
+ ret += gen_visitor_input_containers_decl(arg_type)
+ ret += gen_visitor_input_vars_decl(arg_type) + '\n'
+ ret += gen_visitor_input_block(arg_type) + '\n'
else:
ret += mcgen('''
@@ -224,9 +227,9 @@ def gen_marshal_input(name, args, ret_type, middle_mode):
''')
- ret += gen_sync_call(name, args, ret_type)
+ ret += gen_call(name, arg_type, ret_type)
- if re.search('^ *goto out\\;', ret, re.MULTILINE):
+ if re.search('^ *goto out;', ret, re.MULTILINE):
ret += mcgen('''
out:
@@ -234,7 +237,7 @@ out:
ret += mcgen('''
error_propagate(errp, local_err);
''')
- ret += gen_visitor_input_block(args, dealloc=True)
+ ret += gen_visitor_input_block(arg_type, dealloc=True)
ret += mcgen('''
}
''')
@@ -249,8 +252,8 @@ def gen_register_command(name, success_response):
ret = mcgen('''
qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
''',
- name=name, c_name=c_name(name),
- opts=options)
+ name=name, c_name=c_name(name),
+ opts=options)
pop_indent()
return ret
@@ -285,12 +288,12 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
gen, success_response):
if not gen:
return
- self.decl += generate_command_decl(name, arg_type, ret_type)
+ self.decl += gen_command_decl(name, arg_type, ret_type)
if ret_type:
self.defn += gen_marshal_output(name, ret_type)
if middle_mode:
- self.decl += gen_marshal_input_decl(name, middle_mode) + ';\n'
- self.defn += gen_marshal_input(name, arg_type, ret_type, middle_mode)
+ self.decl += gen_marshal_input_decl(name) + ';\n'
+ self.defn += gen_marshal_input(name, arg_type, ret_type)
if not middle_mode:
self.regy += gen_register_command(name, success_response)
@@ -350,7 +353,7 @@ fdef.write(mcgen('''
#include "%(prefix)sqmp-commands.h"
''',
- prefix=prefix))
+ prefix=prefix))
fdecl.write(mcgen('''
#include "%(prefix)sqapi-types.h"
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 316f369..24ca01a 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -13,12 +13,12 @@
from qapi import *
-def _generate_event_api_name(event_name, params):
- api_name = "void qapi_event_send_%s(" % c_name(event_name).lower();
+def gen_event_send_proto(name, arg_type):
+ api_name = "void qapi_event_send_%s(" % c_name(name).lower()
l = len(api_name)
- if params:
- for m in params.members:
+ if arg_type:
+ for m in arg_type.members:
if m.optional:
api_name += "bool has_%s,\n" % c_name(m.name)
api_name += "".ljust(l)
@@ -28,53 +28,47 @@ def _generate_event_api_name(event_name, params):
api_name += "".ljust(l)
api_name += "Error **errp)"
- return api_name;
+ return api_name
-
-# Following are the core functions that generate C APIs to emit event.
-
-def generate_event_declaration(api_name):
+def gen_event_send_decl(name, arg_type):
return mcgen('''
-%(api_name)s;
+%(proto)s;
''',
- api_name = api_name)
+ proto=gen_event_send_proto(name, arg_type))
-def generate_event_implement(api_name, event_name, params):
- # step 1: declare any variables
- ret = mcgen("""
+def gen_event_send(name, arg_type):
+ ret = mcgen('''
-%(api_name)s
+%(proto)s
{
QDict *qmp;
Error *local_err = NULL;
QMPEventFuncEmit emit;
-""",
- api_name = api_name)
+''',
+ proto=gen_event_send_proto(name, arg_type))
- if params and params.members:
- ret += mcgen("""
+ if arg_type and arg_type.members:
+ ret += mcgen('''
QmpOutputVisitor *qov;
Visitor *v;
QObject *obj;
-""")
+''')
- # step 2: check emit function, create a dict
- ret += mcgen("""
+ ret += mcgen('''
emit = qmp_event_get_func_emit();
if (!emit) {
return;
}
- qmp = qmp_event_build_dict("%(event_name)s");
+ qmp = qmp_event_build_dict("%(name)s");
-""",
- event_name = event_name)
+''',
+ name=name)
- # step 3: visit the params if params != None
- if params and params.members:
- ret += mcgen("""
+ if arg_type and arg_type.members:
+ ret += mcgen('''
qov = qmp_output_visitor_new();
g_assert(qov);
@@ -82,45 +76,46 @@ def generate_event_implement(api_name, event_name, params):
g_assert(v);
/* Fake visit, as if all members are under a structure */
- visit_start_struct(v, NULL, "", "%(event_name)s", 0, &local_err);
+ visit_start_struct(v, NULL, "", "%(name)s", 0, &local_err);
if (local_err) {
goto clean;
}
-""",
- event_name = event_name)
+''',
+ name=name)
- for memb in params.members:
+ for memb in arg_type.members:
if memb.optional:
- ret += mcgen("""
- if (has_%(var)s) {
-""",
- var=c_name(memb.name))
+ ret += mcgen('''
+ if (has_%(c_name)s) {
+''',
+ c_name=c_name(memb.name))
push_indent()
+ # Ugly: need to cast away the const
if memb.type.name == "str":
- var_type = "(char **)"
+ cast = '(char **)'
else:
- var_type = ""
+ cast = ''
- ret += mcgen("""
- visit_type_%(type)s(v, %(var_type)s&%(var)s, "%(name)s", &local_err);
+ ret += mcgen('''
+ visit_type_%(c_type)s(v, %(cast)s&%(c_name)s, "%(name)s", &local_err);
if (local_err) {
goto clean;
}
-""",
- var_type = var_type,
- var=c_name(memb.name),
- type=memb.type.c_name(),
+''',
+ cast=cast,
+ c_name=c_name(memb.name),
+ c_type=memb.type.c_name(),
name=memb.name)
if memb.optional:
pop_indent()
- ret += mcgen("""
+ ret += mcgen('''
}
-""")
+''')
- ret += mcgen("""
+ ret += mcgen('''
visit_end_struct(v, &local_err);
if (local_err) {
@@ -131,27 +126,24 @@ def generate_event_implement(api_name, event_name, params):
g_assert(obj != NULL);
qdict_put_obj(qmp, "data", obj);
-""")
+''')
- # step 4: call qmp event api
- ret += mcgen("""
- emit(%(event_enum_value)s, qmp, &local_err);
+ ret += mcgen('''
+ emit(%(c_enum)s, qmp, &local_err);
-""",
- event_enum_value = c_enum_const(event_enum_name, event_name))
+''',
+ c_enum=c_enum_const(event_enum_name, name))
- # step 5: clean up
- if params and params.members:
- ret += mcgen("""
+ if arg_type and arg_type.members:
+ ret += mcgen('''
clean:
qmp_output_visitor_cleanup(qov);
-""")
- ret += mcgen("""
+''')
+ ret += mcgen('''
error_propagate(errp, local_err);
QDECREF(qmp);
}
-""")
-
+''')
return ret
class QAPISchemaGenEventVisitor(QAPISchemaVisitor):
@@ -164,13 +156,12 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor):
self.defn = ''
self.event_names = []
def visit_end(self):
- self.decl += generate_enum(event_enum_name, self.event_names)
- self.defn += generate_enum_lookup(event_enum_name, self.event_names)
+ self.decl += gen_enum(event_enum_name, self.event_names)
+ self.defn += gen_enum_lookup(event_enum_name, self.event_names)
self.event_names = None
def visit_event(self, name, info, arg_type):
- api_name = _generate_event_api_name(name, arg_type)
- self.decl += generate_event_declaration(api_name)
- self.defn += generate_event_implement(api_name, name, arg_type)
+ self.decl += gen_event_send_decl(name, arg_type)
+ self.defn += gen_event_send(name, arg_type)
self.event_names.append(name)
(input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 203051c..d7f21b3 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -16,22 +16,22 @@ from qapi import *
def gen_fwd_object_or_array(name):
return mcgen('''
-typedef struct %(name)s %(name)s;
+typedef struct %(c_name)s %(c_name)s;
''',
- name=c_name(name))
+ c_name=c_name(name))
def gen_array(name, element_type):
return mcgen('''
-struct %(name)s {
+struct %(c_name)s {
union {
%(c_type)s value;
uint64_t padding;
};
- struct %(name)s *next;
+ %(c_name)s *next;
};
''',
- name=c_name(name), c_type=element_type.c_type())
+ c_name=c_name(name), c_type=element_type.c_type())
def gen_struct_field(name, typ, optional):
ret = ''
@@ -47,7 +47,7 @@ def gen_struct_field(name, typ, optional):
c_type=typ.c_type(), c_name=c_name(name))
return ret
-def generate_struct_fields(members):
+def gen_struct_fields(members):
ret = ''
for memb in members:
@@ -57,20 +57,20 @@ def generate_struct_fields(members):
def gen_struct(name, base, members):
ret = mcgen('''
-struct %(name)s {
+struct %(c_name)s {
''',
- name=c_name(name))
+ c_name=c_name(name))
if base:
ret += gen_struct_field('base', base, False)
- ret += generate_struct_fields(members)
+ ret += gen_struct_fields(members)
# Make sure that all structs have at least one field; this avoids
# potential issues with attempting to malloc space for zero-length structs
# in C, and also incompatibility with C++ (where an empty struct is size 1).
if not base and not members:
- ret += mcgen('''
+ ret += mcgen('''
char qapi_dummy_field_for_empty_struct;
''')
@@ -90,9 +90,9 @@ extern const int %(c_name)s_qtypes[];
def gen_alternate_qtypes(name, variants):
ret = mcgen('''
-const int %(name)s_qtypes[QTYPE_MAX] = {
+const int %(c_name)s_qtypes[QTYPE_MAX] = {
''',
- name=c_name(name))
+ c_name=c_name(name))
for var in variants.variants:
qtype = var.type.alternate_qtype()
@@ -101,7 +101,7 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
ret += mcgen('''
[%(qtype)s] = %(enum_const)s,
''',
- qtype = qtype,
+ qtype=qtype,
enum_const=c_enum_const(variants.tag_member.type.name,
var.name))
@@ -111,27 +111,25 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
return ret
def gen_union(name, base, variants):
- name = c_name(name)
-
ret = mcgen('''
-struct %(name)s {
+struct %(c_name)s {
''',
- name=name)
+ c_name=c_name(name))
if base:
ret += mcgen('''
/* Members inherited from %(c_name)s: */
''',
c_name=c_name(base.name))
- ret += generate_struct_fields(base.members)
+ ret += gen_struct_fields(base.members)
ret += mcgen('''
/* Own members: */
''')
else:
ret += mcgen('''
- %(discriminator_type_name)s kind;
+ %(c_type)s kind;
''',
- discriminator_type_name=c_name(variants.tag_member.type.name))
+ c_type=c_name(variants.tag_member.type.name))
# FIXME: What purpose does data serve, besides preventing a union that
# has a branch named 'data'? We use it in qapi-visit.py to decide
@@ -163,18 +161,18 @@ struct %(name)s {
return ret
-def generate_type_cleanup_decl(name):
+def gen_type_cleanup_decl(name):
ret = mcgen('''
-void qapi_free_%(name)s(%(name)s *obj);
+void qapi_free_%(c_name)s(%(c_name)s *obj);
''',
- name=c_name(name))
+ c_name=c_name(name))
return ret
-def generate_type_cleanup(name):
+def gen_type_cleanup(name):
ret = mcgen('''
-void qapi_free_%(name)s(%(name)s *obj)
+void qapi_free_%(c_name)s(%(c_name)s *obj)
{
QapiDeallocVisitor *md;
Visitor *v;
@@ -185,11 +183,11 @@ void qapi_free_%(name)s(%(name)s *obj)
md = qapi_dealloc_visitor_new();
v = qapi_dealloc_get_visitor(md);
- visit_type_%(name)s(v, &obj, NULL, NULL);
+ visit_type_%(c_name)s(v, &obj, NULL, NULL);
qapi_dealloc_visitor_cleanup(md);
}
''',
- name=c_name(name))
+ c_name=c_name(name))
return ret
class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
@@ -218,18 +216,18 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
self.decl = self.btin + self.decl
self.btin = None
def _gen_type_cleanup(self, name):
- self.decl += generate_type_cleanup_decl(name)
- self.defn += generate_type_cleanup(name)
+ self.decl += gen_type_cleanup_decl(name)
+ self.defn += gen_type_cleanup(name)
def visit_enum_type(self, name, info, values):
- self.fwdecl += generate_enum(name, values)
- self.fwdefn += generate_enum_lookup(name, values)
+ self.fwdecl += gen_enum(name, values)
+ self.fwdefn += gen_enum_lookup(name, values)
def visit_array_type(self, name, info, element_type):
if isinstance(element_type, QAPISchemaBuiltinType):
self.btin += gen_fwd_object_or_array(name)
self.btin += gen_array(name, element_type)
- self.btin += generate_type_cleanup_decl(name)
+ self.btin += gen_type_cleanup_decl(name)
if do_builtins:
- self.defn += generate_type_cleanup(name)
+ self.defn += gen_type_cleanup(name)
else:
self.fwdecl += gen_fwd_object_or_array(name)
self.decl += gen_array(name, element_type)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 38857f7..ab7da81 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -18,18 +18,19 @@ import re
implicit_structs_seen = set()
struct_fields_seen = set()
-def generate_visit_implicit_struct(type):
- if type in implicit_structs_seen:
+def gen_visit_implicit_struct(typ):
+ if typ in implicit_structs_seen:
return ''
- implicit_structs_seen.add(type)
+ implicit_structs_seen.add(typ)
+
ret = ''
- if type.name not in struct_fields_seen:
+ if typ.name not in struct_fields_seen:
# Need a forward declaration
ret += mcgen('''
static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error **errp);
''',
- c_type=type.c_name())
+ c_type=typ.c_name())
ret += mcgen('''
@@ -45,35 +46,35 @@ static void visit_type_implicit_%(c_type)s(Visitor *m, %(c_type)s **obj, Error *
error_propagate(errp, err);
}
''',
- c_type=type.c_name())
+ c_type=typ.c_name())
return ret
-def generate_visit_struct_fields(name, members, base = None):
+def gen_visit_struct_fields(name, base, members):
struct_fields_seen.add(name)
ret = ''
if base:
- ret += generate_visit_implicit_struct(base)
+ ret += gen_visit_implicit_struct(base)
ret += mcgen('''
-static void visit_type_%(name)s_fields(Visitor *m, %(name)s **obj, Error **errp)
+static void visit_type_%(c_name)s_fields(Visitor *m, %(c_name)s **obj, Error **errp)
{
Error *err = NULL;
''',
- name=c_name(name))
+ c_name=c_name(name))
push_indent()
if base:
ret += mcgen('''
-visit_type_implicit_%(type)s(m, &(*obj)->%(c_name)s, &err);
+visit_type_implicit_%(c_type)s(m, &(*obj)->%(c_name)s, &err);
if (err) {
goto out;
}
''',
- type=base.c_name(), c_name=c_name('base'))
+ c_type=base.c_name(), c_name=c_name('base'))
for memb in members:
if memb.optional:
@@ -85,9 +86,9 @@ if (!err && (*obj)->has_%(c_name)s) {
push_indent()
ret += mcgen('''
-visit_type_%(type)s(m, &(*obj)->%(c_name)s, "%(name)s", &err);
+visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, "%(name)s", &err);
''',
- type=memb.type.c_name(), c_name=c_name(memb.name),
+ c_type=memb.type.c_name(), c_name=c_name(memb.name),
name=memb.name)
if memb.optional:
@@ -102,7 +103,7 @@ if (err) {
''')
pop_indent()
- if re.search('^ *goto out\\;', ret, re.MULTILINE):
+ if re.search('^ *goto out;', ret, re.MULTILINE):
ret += mcgen('''
out:
@@ -113,8 +114,7 @@ out:
''')
return ret
-
-def generate_visit_struct_body(name):
+def gen_visit_struct_body(name):
# FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
# *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
# rather than leaving it non-NULL. As currently written, the caller must
@@ -132,20 +132,18 @@ def generate_visit_struct_body(name):
error_propagate(errp, err);
''',
name=name, c_name=c_name(name))
-
return ret
def gen_visit_struct(name, base, members):
- ret = generate_visit_struct_fields(name, members, base)
-
+ ret = gen_visit_struct_fields(name, base, members)
ret += mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
''',
- name=c_name(name))
+ c_name=c_name(name))
- ret += generate_visit_struct_body(name)
+ ret += gen_visit_struct_body(name)
ret += mcgen('''
}
@@ -155,7 +153,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
def gen_visit_list(name, element_type):
return mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
GenericList *i, **prev;
@@ -168,7 +166,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
for (prev = (GenericList **)obj;
!err && (i = visit_next_list(m, prev, &err)) != NULL;
prev = &i) {
- %(name)s *native_i = (%(name)s *)i;
+ %(c_name)s *native_i = (%(c_name)s *)i;
visit_type_%(c_elt_type)s(m, &native_i->value, NULL, &err);
}
@@ -179,10 +177,9 @@ out:
error_propagate(errp, err);
}
''',
- name=c_name(name),
- c_elt_type=element_type.c_name())
+ c_name=c_name(name), c_elt_type=element_type.c_name())
-def generate_visit_enum(name):
+def gen_visit_enum(name):
return mcgen('''
void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error **errp)
@@ -195,33 +192,32 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s *obj, const char *name, Error
def gen_visit_alternate(name, variants):
ret = mcgen('''
-void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
{
Error *err = NULL;
- visit_start_implicit_struct(m, (void**) obj, sizeof(%(name)s), &err);
+ visit_start_implicit_struct(m, (void**) obj, sizeof(%(c_name)s), &err);
if (err) {
goto out;
}
- visit_get_next_type(m, (int*) &(*obj)->kind, %(name)s_qtypes, name, &err);
+ visit_get_next_type(m, (int*) &(*obj)->kind, %(c_name)s_qtypes, name, &err);
if (err) {
goto out_end;
}
switch ((*obj)->kind) {
''',
- name=c_name(name))
+ c_name=c_name(name))
for var in variants.variants:
- enum_full_value = c_enum_const(variants.tag_member.type.name,
- var.name)
ret += mcgen('''
- case %(enum_full_value)s:
+ case %(case)s:
visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, name, &err);
break;
''',
- enum_full_value = enum_full_value,
- c_type=var.type.c_name(),
- c_name=c_name(var.name))
+ case=c_enum_const(variants.tag_member.type.name,
+ var.name),
+ c_type=var.type.c_name(),
+ c_name=c_name(var.name))
ret += mcgen('''
default:
@@ -243,12 +239,12 @@ def gen_visit_union(name, base, variants):
if base:
members = [m for m in base.members if m != variants.tag_member]
- ret += generate_visit_struct_fields(name, members)
+ ret += gen_visit_struct_fields(name, None, members)
for var in variants.variants:
# TODO ugly special case for simple union
if not var.simple_union_type():
- ret += generate_visit_implicit_struct(var.type)
+ ret += gen_visit_implicit_struct(var.type)
ret += mcgen('''
@@ -266,19 +262,19 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error
if base:
ret += mcgen('''
- visit_type_%(name)s_fields(m, obj, &err);
+ visit_type_%(c_name)s_fields(m, obj, &err);
if (err) {
goto out_obj;
}
''',
- name=c_name(name))
+ c_name=c_name(name))
- disc_key = variants.tag_member.name
+ tag_key = variants.tag_member.name
if not variants.tag_name:
# we pointlessly use a different key for simple unions
- disc_key = 'type'
+ tag_key = 'type'
ret += mcgen('''
- visit_type_%(disc_type)s(m, &(*obj)->%(c_name)s, "%(disc_key)s", &err);
+ visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, "%(name)s", &err);
if (err) {
goto out_obj;
}
@@ -287,27 +283,32 @@ void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error
}
switch ((*obj)->%(c_name)s) {
''',
- disc_type=variants.tag_member.type.c_name(),
+ c_type=variants.tag_member.type.c_name(),
c_name=c_name(variants.tag_member.name),
- disc_key = disc_key)
+ name=tag_key)
for var in variants.variants:
# TODO ugly special case for simple union
simple_union_type = var.simple_union_type()
+ ret += mcgen('''
+ case %(case)s:
+''',
+ case=c_enum_const(variants.tag_member.type.name, var.name))
if simple_union_type:
- fmt = 'visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, "data", &err);'
+ ret += mcgen('''
+ visit_type_%(c_type)s(m, &(*obj)->%(c_name)s, "data", &err);
+''',
+ c_type=simple_union_type.c_name(),
+ c_name=c_name(var.name))
else:
- fmt = 'visit_type_implicit_%(c_type)s(m, &(*obj)->%(c_name)s, &err);'
-
- enum_full_value = c_enum_const(variants.tag_member.type.name, var.name)
+ ret += mcgen('''
+ visit_type_implicit_%(c_type)s(m, &(*obj)->%(c_name)s, &err);
+''',
+ c_type=var.type.c_name(),
+ c_name=c_name(var.name))
ret += mcgen('''
- case %(enum_full_value)s:
- ''' + fmt + '''
break;
-''',
- enum_full_value = enum_full_value,
- c_type=(simple_union_type or var.type).c_name(),
- c_name=c_name(var.name))
+''')
ret += mcgen('''
default:
@@ -356,7 +357,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
self.btin = None
def visit_enum_type(self, name, info, values):
self.decl += gen_visit_decl(name, scalar=True)
- self.defn += generate_visit_enum(name)
+ self.defn += gen_visit_enum(name)
def visit_array_type(self, name, info, element_type):
decl = gen_visit_decl(name)
defn = gen_visit_list(name, element_type)
@@ -429,7 +430,7 @@ fdef.write(mcgen('''
#include "qemu-common.h"
#include "%(prefix)sqapi-visit.h"
''',
- prefix = prefix))
+ prefix=prefix))
fdecl.write(mcgen('''
#include "qapi/visitor.h"
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 855ca97..137ee15 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1024,9 +1024,6 @@ class QAPISchema(object):
self._def_exprs()
self.check()
- def get_exprs(self):
- return [expr_elem['expr'] for expr_elem in self.exprs]
-
def _def_entity(self, ent):
assert ent.name not in self.entity_dict
self.entity_dict[ent.name] = ent
@@ -1216,23 +1213,6 @@ class QAPISchema(object):
# Code generation helpers
#
-def parse_args(typeinfo):
- if isinstance(typeinfo, str):
- struct = find_struct(typeinfo)
- assert struct != None
- typeinfo = struct['data']
-
- for member in typeinfo:
- argname = member
- argentry = typeinfo[member]
- optional = False
- if member.startswith('*'):
- argname = member[1:]
- optional = True
- # Todo: allow argentry to be OrderedDict, for providing the
- # value of an optional argument.
- yield (argname, argentry, optional)
-
def camel_case(name):
new_name = ''
first = True
@@ -1313,67 +1293,9 @@ def c_name(name, protect=True):
return "q_" + name
return name.translate(c_name_trans)
-# Map type @name to the C typedef name for the list form.
-#
-# ['Name'] -> 'NameList', ['x-Foo'] -> 'x_FooList', ['int'] -> 'intList'
-def c_list_type(name):
- return type_name(name) + 'List'
-
-# Map type @value to the C typedef form.
-#
-# Used for converting 'type' from a 'member':'type' qapi definition
-# into the alphanumeric portion of the type for a generated C parameter,
-# as well as generated C function names. See c_type() for the rest of
-# the conversion such as adding '*' on pointer types.
-# 'int' -> 'int', '[x-Foo]' -> 'x_FooList', '__a.b_c' -> '__a_b_c'
-def type_name(value):
- if type(value) == list:
- return c_list_type(value[0])
- if value in builtin_types.keys():
- return value
- return c_name(value)
-
eatspace = '\033EATSPACE.'
pointer_suffix = ' *' + eatspace
-# Map type @name to its C type expression.
-# If @is_param, const-qualify the string type.
-#
-# This function is used for computing the full C type of 'member':'name'.
-# A special suffix is added in c_type() for pointer types, and it's
-# stripped in mcgen(). So please notice this when you check the return
-# value of c_type() outside mcgen().
-def c_type(value, is_param=False):
- if value == 'str':
- if is_param:
- return 'const char' + pointer_suffix
- return 'char' + pointer_suffix
-
- elif value == 'int':
- return 'int64_t'
- elif (value == 'int8' or value == 'int16' or value == 'int32' or
- value == 'int64' or value == 'uint8' or value == 'uint16' or
- value == 'uint32' or value == 'uint64'):
- return value + '_t'
- elif value == 'size':
- return 'uint64_t'
- elif value == 'bool':
- return 'bool'
- elif value == 'number':
- return 'double'
- elif type(value) == list:
- return c_list_type(value[0]) + pointer_suffix
- elif is_enum(value):
- return c_name(value)
- elif value == None:
- return 'void'
- elif value in events:
- return camel_case(value) + 'Event' + pointer_suffix
- else:
- # complex type name
- assert isinstance(value, str) and value != ""
- return c_name(value) + pointer_suffix
-
def genindent(count):
ret = ""
for i in range(count):
@@ -1426,60 +1348,57 @@ def guardend(name):
''',
name=guardname(name))
-def generate_enum_lookup(name, values):
+def gen_enum_lookup(name, values):
ret = mcgen('''
-const char *const %(name)s_lookup[] = {
+const char *const %(c_name)s_lookup[] = {
''',
- name=c_name(name))
+ c_name=c_name(name))
for value in values:
index = c_enum_const(name, value)
ret += mcgen('''
[%(index)s] = "%(value)s",
''',
- index = index, value = value)
+ index=index, value=value)
max_index = c_enum_const(name, 'MAX')
ret += mcgen('''
[%(max_index)s] = NULL,
};
''',
- max_index=max_index)
+ max_index=max_index)
return ret
-def generate_enum(name, values):
- name = c_name(name)
- lookup_decl = mcgen('''
-
-extern const char *const %(name)s_lookup[];
-''',
- name=name)
-
- enum_decl = mcgen('''
-
-typedef enum %(name)s {
-''',
- name=name)
-
+def gen_enum(name, values):
# append automatically generated _MAX value
- enum_values = values + [ 'MAX' ]
+ enum_values = values + ['MAX']
+
+ ret = mcgen('''
+
+typedef enum %(c_name)s {
+''',
+ c_name=c_name(name))
i = 0
for value in enum_values:
- enum_full_value = c_enum_const(name, value)
- enum_decl += mcgen('''
- %(enum_full_value)s = %(i)d,
+ ret += mcgen('''
+ %(c_enum)s = %(i)d,
''',
- enum_full_value = enum_full_value,
+ c_enum=c_enum_const(name, value),
i=i)
i += 1
- enum_decl += mcgen('''
-} %(name)s;
+ ret += mcgen('''
+} %(c_name)s;
''',
- name=name)
+ c_name=c_name(name))
- return enum_decl + lookup_decl
+ ret += mcgen('''
+
+extern const char *const %(c_name)s_lookup[];
+''',
+ c_name=c_name(name))
+ return ret
#
# Common command line parsing
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 20/32] qapi-visit: Rearrange code a bit
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (18 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 19/32] qapi: Clean up after recent conversions to QAPISchemaVisitor Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 21/32] qapi-commands: Rearrange code Markus Armbruster
` (11 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Move gen_visit_decl() to a better place. Inline
generate_visit_struct_body().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-visit.py | 43 +++++++++++++++++--------------------------
1 file changed, 17 insertions(+), 26 deletions(-)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index ab7da81..9cfd2ff 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -18,6 +18,15 @@ import re
implicit_structs_seen = set()
struct_fields_seen = set()
+def gen_visit_decl(name, scalar=False):
+ c_type = c_name(name) + ' *'
+ if not scalar:
+ c_type += '*'
+ return mcgen('''
+void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error **errp);
+''',
+ c_name=c_name(name), c_type=c_type)
+
def gen_visit_implicit_struct(typ):
if typ in implicit_structs_seen:
return ''
@@ -114,12 +123,17 @@ out:
''')
return ret
-def gen_visit_struct_body(name):
+def gen_visit_struct(name, base, members):
+ ret = gen_visit_struct_fields(name, base, members)
+
# FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
# *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
# rather than leaving it non-NULL. As currently written, the caller must
# call qapi_free_FOO() to avoid a memory leak of the partial FOO.
- ret = mcgen('''
+ ret += mcgen('''
+
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
+{
Error *err = NULL;
visit_start_struct(m, (void **)obj, "%(name)s", name, sizeof(%(c_name)s), &err);
@@ -130,24 +144,10 @@ def gen_visit_struct_body(name):
visit_end_struct(m, &err);
}
error_propagate(errp, err);
+}
''',
name=name, c_name=c_name(name))
- return ret
-def gen_visit_struct(name, base, members):
- ret = gen_visit_struct_fields(name, base, members)
- ret += mcgen('''
-
-void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, Error **errp)
-{
-''',
- c_name=c_name(name))
-
- ret += gen_visit_struct_body(name)
-
- ret += mcgen('''
-}
-''')
return ret
def gen_visit_list(name, element_type):
@@ -329,15 +329,6 @@ out:
return ret
-def gen_visit_decl(name, scalar=False):
- c_type = c_name(name) + ' *'
- if not scalar:
- c_type += '*'
- return mcgen('''
-void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error **errp);
-''',
- c_name=c_name(name), c_type=c_type)
-
class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
def __init__(self):
self.decl = None
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 21/32] qapi-commands: Rearrange code
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (19 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 20/32] qapi-visit: Rearrange code a bit Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 22/32] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO() Markus Armbruster
` (10 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Rename gen_marshal_input() to gen_marshal(), because the generated
function marshals both arguments and results.
Rename gen_visitor_input_containers_decl() to gen_marshal_vars(), and
move the other variable declarations there, too.
Rename gen_visitor_input_block() to gen_marshal_input_visit(), and
rearrange its code slightly.
Rename gen_marshal_input_decl() to gen_marshal_proto(), because the
result isn't a full declaration, unlike gen_command_decl()'s.
New gen_marshal_decl() actually returns a full declaration.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-commands.py | 85 ++++++++++++++++++++++--------------------------
1 file changed, 38 insertions(+), 47 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index c165c63..7eb3544 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -56,6 +56,7 @@ def gen_call(name, arg_type, ret_type):
push_indent()
ret = mcgen('''
+
%(lhs)sqmp_%(c_name)s(%(args)s&local_err);
''',
c_name=c_name(name), args=argstr, lhs=lhs)
@@ -69,25 +70,26 @@ qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
pop_indent()
return ret
-def gen_visitor_input_containers_decl(arg_type):
- ret = ''
+def gen_marshal_vars(arg_type, ret_type):
+ ret = mcgen('''
+ Error *local_err = NULL;
+''')
push_indent()
+
+ if ret_type:
+ ret += mcgen('''
+%(c_type)s retval;
+''',
+ c_type=ret_type.c_type())
+
if arg_type:
ret += mcgen('''
QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
QapiDeallocVisitor *md;
Visitor *v;
''')
- pop_indent()
- return ret
-
-def gen_visitor_input_vars_decl(arg_type):
- ret = ''
- push_indent()
-
- if arg_type:
for memb in arg_type.members:
if memb.optional:
ret += mcgen('''
@@ -100,14 +102,18 @@ bool has_%(c_name)s = false;
c_name=c_name(memb.name),
c_type=memb.type.c_type(),
c_null=memb.type.c_null())
+ ret += '\n'
+ else:
+ ret += mcgen('''
+
+(void)args;
+''')
pop_indent()
return ret
-def gen_visitor_input_block(arg_type, dealloc=False):
+def gen_marshal_input_visit(arg_type, dealloc=False):
ret = ''
- errparg = '&local_err'
- errarg = 'local_err'
if not arg_type:
return ret
@@ -123,6 +129,8 @@ md = qapi_dealloc_visitor_new();
v = qapi_dealloc_get_visitor(md);
''')
else:
+ errparg = '&local_err'
+ errarg = 'local_err'
ret += mcgen('''
v = qmp_input_get_visitor(mi);
''')
@@ -160,10 +168,7 @@ qapi_dealloc_visitor_cleanup(md);
return ret
def gen_marshal_output(name, ret_type):
- if not ret_type:
- return ''
-
- ret = mcgen('''
+ return mcgen('''
static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject **ret_out, Error **errp)
{
@@ -188,45 +193,31 @@ out:
qapi_dealloc_visitor_cleanup(md);
}
''',
- c_type=ret_type.c_type(), c_cmd_name=c_name(name),
- c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_cmd_name=c_name(name),
+ c_name=ret_type.c_name())
- return ret
-
-def gen_marshal_input_decl(name):
+def gen_marshal_proto(name):
ret = 'void qmp_marshal_input_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
if not middle_mode:
ret = 'static ' + ret
return ret
-def gen_marshal_input(name, arg_type, ret_type):
- hdr = gen_marshal_input_decl(name)
+def gen_marshal_decl(name):
+ return mcgen('''
+%(proto)s;
+''',
+ proto=gen_marshal_proto(name))
+def gen_marshal(name, arg_type, ret_type):
ret = mcgen('''
-%(header)s
+%(proto)s
{
- Error *local_err = NULL;
''',
- header=hdr)
-
- if ret_type:
- ret += mcgen('''
- %(c_type)s retval;
-''',
- c_type=ret_type.c_type())
-
- if arg_type:
- ret += gen_visitor_input_containers_decl(arg_type)
- ret += gen_visitor_input_vars_decl(arg_type) + '\n'
- ret += gen_visitor_input_block(arg_type) + '\n'
- else:
- ret += mcgen('''
-
- (void)args;
-
-''')
+ proto=gen_marshal_proto(name))
+ ret += gen_marshal_vars(arg_type, ret_type)
+ ret += gen_marshal_input_visit(arg_type)
ret += gen_call(name, arg_type, ret_type)
if re.search('^ *goto out;', ret, re.MULTILINE):
@@ -237,7 +228,7 @@ out:
ret += mcgen('''
error_propagate(errp, local_err);
''')
- ret += gen_visitor_input_block(arg_type, dealloc=True)
+ ret += gen_marshal_input_visit(arg_type, dealloc=True)
ret += mcgen('''
}
''')
@@ -292,8 +283,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
if ret_type:
self.defn += gen_marshal_output(name, ret_type)
if middle_mode:
- self.decl += gen_marshal_input_decl(name) + ';\n'
- self.defn += gen_marshal_input(name, arg_type, ret_type)
+ self.decl += gen_marshal_decl(name)
+ self.defn += gen_marshal(name, arg_type, ret_type)
if not middle_mode:
self.regy += gen_register_command(name, success_response)
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 22/32] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (20 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 21/32] qapi-commands: Rearrange code Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 23/32] qapi: De-duplicate parameter list generation Markus Armbruster
` (9 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
These functions marshal both input and output.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 4 +-
docs/writing-qmp-commands.txt | 8 +-
monitor.c | 2 +-
qmp-commands.hx | 240 +++++++++++++++++++++---------------------
qmp.c | 6 +-
scripts/qapi-commands.py | 4 +-
6 files changed, 132 insertions(+), 132 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 5cf401f..7eafcde 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -738,7 +738,7 @@ Example:
qapi_dealloc_visitor_cleanup(md);
}
- static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error **errp)
+ static void qmp_marshal_my_command(QDict *args, QObject **ret, Error **errp)
{
Error *local_err = NULL;
UserDefOne *retval;
@@ -771,7 +771,7 @@ Example:
static void qmp_init_marshal(void)
{
- qmp_register_command("my-command", qmp_marshal_input_my_command, QCO_NO_OPTIONS);
+ qmp_register_command("my-command", qmp_marshal_my_command, QCO_NO_OPTIONS);
}
qapi_init(qmp_init_marshal);
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index ab1fdd3..c65bdc6 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -127,7 +127,7 @@ following in the botton:
{
.name = "hello-world",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_hello_world,
+ .mhandler.cmd_new = qmp_marshal_hello_world,
},
You're done. Now build qemu, run it as suggested in the "Testing" section,
@@ -179,7 +179,7 @@ The last step is to update the qmp-commands.hx file:
{
.name = "hello-world",
.args_type = "message:s?",
- .mhandler.cmd_new = qmp_marshal_input_hello_world,
+ .mhandler.cmd_new = qmp_marshal_hello_world,
},
Notice that the "args_type" member got our "message" argument. The character
@@ -461,7 +461,7 @@ The last step is to add the correspoding entry in the qmp-commands.hx file:
{
.name = "query-alarm-clock",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_alarm_clock,
+ .mhandler.cmd_new = qmp_marshal_query_alarm_clock,
},
Time to test the new command. Build qemu, run it as described in the "Testing"
@@ -607,7 +607,7 @@ To test this you have to add the corresponding qmp-commands.hx entry:
{
.name = "query-alarm-methods",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_alarm_methods,
+ .mhandler.cmd_new = qmp_marshal_query_alarm_methods,
},
Now Build qemu, run it as explained in the "Testing" section and try our new
diff --git a/monitor.c b/monitor.c
index aeea2b5..16672f1 100644
--- a/monitor.c
+++ b/monitor.c
@@ -5102,7 +5102,7 @@ static QObject *get_qmp_greeting(void)
{
QObject *ver = NULL;
- qmp_marshal_input_query_version(NULL, &ver, NULL);
+ qmp_marshal_query_version(NULL, &ver, NULL);
return qobject_from_jsonf("{'QMP':{'version': %p,'capabilities': []}}",ver);
}
diff --git a/qmp-commands.hx b/qmp-commands.hx
index ba630b1..41a39fc 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -63,7 +63,7 @@ EQMP
{
.name = "quit",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_quit,
+ .mhandler.cmd_new = qmp_marshal_quit,
},
SQMP
@@ -84,7 +84,7 @@ EQMP
{
.name = "eject",
.args_type = "force:-f,device:B",
- .mhandler.cmd_new = qmp_marshal_input_eject,
+ .mhandler.cmd_new = qmp_marshal_eject,
},
SQMP
@@ -110,7 +110,7 @@ EQMP
{
.name = "change",
.args_type = "device:B,target:F,arg:s?",
- .mhandler.cmd_new = qmp_marshal_input_change,
+ .mhandler.cmd_new = qmp_marshal_change,
},
SQMP
@@ -146,7 +146,7 @@ EQMP
{
.name = "screendump",
.args_type = "filename:F",
- .mhandler.cmd_new = qmp_marshal_input_screendump,
+ .mhandler.cmd_new = qmp_marshal_screendump,
},
SQMP
@@ -169,7 +169,7 @@ EQMP
{
.name = "stop",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_stop,
+ .mhandler.cmd_new = qmp_marshal_stop,
},
SQMP
@@ -190,7 +190,7 @@ EQMP
{
.name = "cont",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_cont,
+ .mhandler.cmd_new = qmp_marshal_cont,
},
SQMP
@@ -211,7 +211,7 @@ EQMP
{
.name = "system_wakeup",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_system_wakeup,
+ .mhandler.cmd_new = qmp_marshal_system_wakeup,
},
SQMP
@@ -232,7 +232,7 @@ EQMP
{
.name = "system_reset",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_system_reset,
+ .mhandler.cmd_new = qmp_marshal_system_reset,
},
SQMP
@@ -253,7 +253,7 @@ EQMP
{
.name = "system_powerdown",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_system_powerdown,
+ .mhandler.cmd_new = qmp_marshal_system_powerdown,
},
SQMP
@@ -310,7 +310,7 @@ EQMP
{
.name = "device_del",
.args_type = "id:s",
- .mhandler.cmd_new = qmp_marshal_input_device_del,
+ .mhandler.cmd_new = qmp_marshal_device_del,
},
SQMP
@@ -333,7 +333,7 @@ EQMP
{
.name = "send-key",
.args_type = "keys:q,hold-time:i?",
- .mhandler.cmd_new = qmp_marshal_input_send_key,
+ .mhandler.cmd_new = qmp_marshal_send_key,
},
SQMP
@@ -364,7 +364,7 @@ EQMP
{
.name = "cpu",
.args_type = "index:i",
- .mhandler.cmd_new = qmp_marshal_input_cpu,
+ .mhandler.cmd_new = qmp_marshal_cpu,
},
SQMP
@@ -389,7 +389,7 @@ EQMP
{
.name = "cpu-add",
.args_type = "id:i",
- .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+ .mhandler.cmd_new = qmp_marshal_cpu_add,
},
SQMP
@@ -412,7 +412,7 @@ EQMP
{
.name = "memsave",
.args_type = "val:l,size:i,filename:s,cpu:i?",
- .mhandler.cmd_new = qmp_marshal_input_memsave,
+ .mhandler.cmd_new = qmp_marshal_memsave,
},
SQMP
@@ -441,7 +441,7 @@ EQMP
{
.name = "pmemsave",
.args_type = "val:l,size:i,filename:s",
- .mhandler.cmd_new = qmp_marshal_input_pmemsave,
+ .mhandler.cmd_new = qmp_marshal_pmemsave,
},
SQMP
@@ -469,7 +469,7 @@ EQMP
{
.name = "inject-nmi",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_inject_nmi,
+ .mhandler.cmd_new = qmp_marshal_inject_nmi,
},
SQMP
@@ -492,7 +492,7 @@ EQMP
{
.name = "ringbuf-write",
.args_type = "device:s,data:s,format:s?",
- .mhandler.cmd_new = qmp_marshal_input_ringbuf_write,
+ .mhandler.cmd_new = qmp_marshal_ringbuf_write,
},
SQMP
@@ -523,7 +523,7 @@ EQMP
{
.name = "ringbuf-read",
.args_type = "device:s,size:i,format:s?",
- .mhandler.cmd_new = qmp_marshal_input_ringbuf_read,
+ .mhandler.cmd_new = qmp_marshal_ringbuf_read,
},
SQMP
@@ -559,7 +559,7 @@ EQMP
{
.name = "xen-save-devices-state",
.args_type = "filename:F",
- .mhandler.cmd_new = qmp_marshal_input_xen_save_devices_state,
+ .mhandler.cmd_new = qmp_marshal_xen_save_devices_state,
},
SQMP
@@ -586,7 +586,7 @@ EQMP
{
.name = "xen-set-global-dirty-log",
.args_type = "enable:b",
- .mhandler.cmd_new = qmp_marshal_input_xen_set_global_dirty_log,
+ .mhandler.cmd_new = qmp_marshal_xen_set_global_dirty_log,
},
SQMP
@@ -610,7 +610,7 @@ EQMP
{
.name = "migrate",
.args_type = "detach:-d,blk:-b,inc:-i,uri:s",
- .mhandler.cmd_new = qmp_marshal_input_migrate,
+ .mhandler.cmd_new = qmp_marshal_migrate,
},
SQMP
@@ -643,7 +643,7 @@ EQMP
{
.name = "migrate_cancel",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_migrate_cancel,
+ .mhandler.cmd_new = qmp_marshal_migrate_cancel,
},
SQMP
@@ -664,7 +664,7 @@ EQMP
{
.name = "migrate-incoming",
.args_type = "uri:s",
- .mhandler.cmd_new = qmp_marshal_input_migrate_incoming,
+ .mhandler.cmd_new = qmp_marshal_migrate_incoming,
},
SQMP
@@ -692,7 +692,7 @@ EQMP
{
.name = "migrate-set-cache-size",
.args_type = "value:o",
- .mhandler.cmd_new = qmp_marshal_input_migrate_set_cache_size,
+ .mhandler.cmd_new = qmp_marshal_migrate_set_cache_size,
},
SQMP
@@ -715,7 +715,7 @@ EQMP
{
.name = "query-migrate-cache-size",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_migrate_cache_size,
+ .mhandler.cmd_new = qmp_marshal_query_migrate_cache_size,
},
SQMP
@@ -737,7 +737,7 @@ EQMP
{
.name = "migrate_set_speed",
.args_type = "value:o",
- .mhandler.cmd_new = qmp_marshal_input_migrate_set_speed,
+ .mhandler.cmd_new = qmp_marshal_migrate_set_speed,
},
SQMP
@@ -760,7 +760,7 @@ EQMP
{
.name = "migrate_set_downtime",
.args_type = "value:T",
- .mhandler.cmd_new = qmp_marshal_input_migrate_set_downtime,
+ .mhandler.cmd_new = qmp_marshal_migrate_set_downtime,
},
SQMP
@@ -785,7 +785,7 @@ EQMP
.args_type = "protocol:s,hostname:s,port:i?,tls-port:i?,cert-subject:s?",
.params = "protocol hostname port tls-port cert-subject",
.help = "set migration information for remote display",
- .mhandler.cmd_new = qmp_marshal_input_client_migrate_info,
+ .mhandler.cmd_new = qmp_marshal_client_migrate_info,
},
SQMP
@@ -819,7 +819,7 @@ EQMP
.args_type = "paging:b,protocol:s,begin:i?,end:i?,format:s?",
.params = "-p protocol [begin] [length] [format]",
.help = "dump guest memory to file",
- .mhandler.cmd_new = qmp_marshal_input_dump_guest_memory,
+ .mhandler.cmd_new = qmp_marshal_dump_guest_memory,
},
SQMP
@@ -855,7 +855,7 @@ EQMP
{
.name = "query-dump-guest-memory-capability",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_dump_guest_memory_capability,
+ .mhandler.cmd_new = qmp_marshal_query_dump_guest_memory_capability,
},
SQMP
@@ -904,7 +904,7 @@ EQMP
{
.name = "netdev_del",
.args_type = "id:s",
- .mhandler.cmd_new = qmp_marshal_input_netdev_del,
+ .mhandler.cmd_new = qmp_marshal_netdev_del,
},
SQMP
@@ -954,7 +954,7 @@ EQMP
{
.name = "object-del",
.args_type = "id:s",
- .mhandler.cmd_new = qmp_marshal_input_object_del,
+ .mhandler.cmd_new = qmp_marshal_object_del,
},
SQMP
@@ -979,7 +979,7 @@ EQMP
{
.name = "block_resize",
.args_type = "device:s?,node-name:s?,size:o",
- .mhandler.cmd_new = qmp_marshal_input_block_resize,
+ .mhandler.cmd_new = qmp_marshal_block_resize,
},
SQMP
@@ -1004,7 +1004,7 @@ EQMP
{
.name = "block-stream",
.args_type = "device:B,base:s?,speed:o?,backing-file:s?,on-error:s?",
- .mhandler.cmd_new = qmp_marshal_input_block_stream,
+ .mhandler.cmd_new = qmp_marshal_block_stream,
},
SQMP
@@ -1047,7 +1047,7 @@ EQMP
{
.name = "block-commit",
.args_type = "device:B,base:s?,top:s?,backing-file:s?,speed:o?",
- .mhandler.cmd_new = qmp_marshal_input_block_commit,
+ .mhandler.cmd_new = qmp_marshal_block_commit,
},
SQMP
@@ -1111,7 +1111,7 @@ EQMP
.name = "drive-backup",
.args_type = "sync:s,device:B,target:s,speed:i?,mode:s?,format:s?,"
"bitmap:s?,on-source-error:s?,on-target-error:s?",
- .mhandler.cmd_new = qmp_marshal_input_drive_backup,
+ .mhandler.cmd_new = qmp_marshal_drive_backup,
},
SQMP
@@ -1165,7 +1165,7 @@ EQMP
.name = "blockdev-backup",
.args_type = "sync:s,device:B,target:B,speed:i?,"
"on-source-error:s?,on-target-error:s?",
- .mhandler.cmd_new = qmp_marshal_input_blockdev_backup,
+ .mhandler.cmd_new = qmp_marshal_blockdev_backup,
},
SQMP
@@ -1205,33 +1205,33 @@ EQMP
{
.name = "block-job-set-speed",
.args_type = "device:B,speed:o",
- .mhandler.cmd_new = qmp_marshal_input_block_job_set_speed,
+ .mhandler.cmd_new = qmp_marshal_block_job_set_speed,
},
{
.name = "block-job-cancel",
.args_type = "device:B,force:b?",
- .mhandler.cmd_new = qmp_marshal_input_block_job_cancel,
+ .mhandler.cmd_new = qmp_marshal_block_job_cancel,
},
{
.name = "block-job-pause",
.args_type = "device:B",
- .mhandler.cmd_new = qmp_marshal_input_block_job_pause,
+ .mhandler.cmd_new = qmp_marshal_block_job_pause,
},
{
.name = "block-job-resume",
.args_type = "device:B",
- .mhandler.cmd_new = qmp_marshal_input_block_job_resume,
+ .mhandler.cmd_new = qmp_marshal_block_job_resume,
},
{
.name = "block-job-complete",
.args_type = "device:B",
- .mhandler.cmd_new = qmp_marshal_input_block_job_complete,
+ .mhandler.cmd_new = qmp_marshal_block_job_complete,
},
{
.name = "transaction",
.args_type = "actions:q",
- .mhandler.cmd_new = qmp_marshal_input_transaction,
+ .mhandler.cmd_new = qmp_marshal_transaction,
},
SQMP
@@ -1310,7 +1310,7 @@ EQMP
{
.name = "block-dirty-bitmap-add",
.args_type = "node:B,name:s,granularity:i?",
- .mhandler.cmd_new = qmp_marshal_input_block_dirty_bitmap_add,
+ .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_add,
},
SQMP
@@ -1338,7 +1338,7 @@ EQMP
{
.name = "block-dirty-bitmap-remove",
.args_type = "node:B,name:s",
- .mhandler.cmd_new = qmp_marshal_input_block_dirty_bitmap_remove,
+ .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_remove,
},
SQMP
@@ -1366,7 +1366,7 @@ EQMP
{
.name = "block-dirty-bitmap-clear",
.args_type = "node:B,name:s",
- .mhandler.cmd_new = qmp_marshal_input_block_dirty_bitmap_clear,
+ .mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_clear,
},
SQMP
@@ -1395,7 +1395,7 @@ EQMP
{
.name = "blockdev-snapshot-sync",
.args_type = "device:s?,node-name:s?,snapshot-file:s,snapshot-node-name:s?,format:s?,mode:s?",
- .mhandler.cmd_new = qmp_marshal_input_blockdev_snapshot_sync,
+ .mhandler.cmd_new = qmp_marshal_blockdev_snapshot_sync,
},
SQMP
@@ -1431,7 +1431,7 @@ EQMP
{
.name = "blockdev-snapshot-internal-sync",
.args_type = "device:B,name:s",
- .mhandler.cmd_new = qmp_marshal_input_blockdev_snapshot_internal_sync,
+ .mhandler.cmd_new = qmp_marshal_blockdev_snapshot_internal_sync,
},
SQMP
@@ -1461,7 +1461,7 @@ EQMP
.name = "blockdev-snapshot-delete-internal-sync",
.args_type = "device:B,id:s?,name:s?",
.mhandler.cmd_new =
- qmp_marshal_input_blockdev_snapshot_delete_internal_sync,
+ qmp_marshal_blockdev_snapshot_delete_internal_sync,
},
SQMP
@@ -1505,7 +1505,7 @@ EQMP
"on-source-error:s?,on-target-error:s?,"
"unmap:b?,"
"granularity:i?,buf-size:i?",
- .mhandler.cmd_new = qmp_marshal_input_drive_mirror,
+ .mhandler.cmd_new = qmp_marshal_drive_mirror,
},
SQMP
@@ -1565,7 +1565,7 @@ EQMP
{
.name = "change-backing-file",
.args_type = "device:s,image-node-name:s,backing-file:s",
- .mhandler.cmd_new = qmp_marshal_input_change_backing_file,
+ .mhandler.cmd_new = qmp_marshal_change_backing_file,
},
SQMP
@@ -1604,7 +1604,7 @@ EQMP
{
.name = "balloon",
.args_type = "value:M",
- .mhandler.cmd_new = qmp_marshal_input_balloon,
+ .mhandler.cmd_new = qmp_marshal_balloon,
},
SQMP
@@ -1627,7 +1627,7 @@ EQMP
{
.name = "set_link",
.args_type = "name:s,up:b",
- .mhandler.cmd_new = qmp_marshal_input_set_link,
+ .mhandler.cmd_new = qmp_marshal_set_link,
},
SQMP
@@ -1653,7 +1653,7 @@ EQMP
.args_type = "fdname:s",
.params = "getfd name",
.help = "receive a file descriptor via SCM rights and assign it a name",
- .mhandler.cmd_new = qmp_marshal_input_getfd,
+ .mhandler.cmd_new = qmp_marshal_getfd,
},
SQMP
@@ -1686,7 +1686,7 @@ EQMP
.args_type = "fdname:s",
.params = "closefd name",
.help = "close a file descriptor previously passed via SCM rights",
- .mhandler.cmd_new = qmp_marshal_input_closefd,
+ .mhandler.cmd_new = qmp_marshal_closefd,
},
SQMP
@@ -1711,7 +1711,7 @@ EQMP
.args_type = "fdset-id:i?,opaque:s?",
.params = "add-fd fdset-id opaque",
.help = "Add a file descriptor, that was passed via SCM rights, to an fd set",
- .mhandler.cmd_new = qmp_marshal_input_add_fd,
+ .mhandler.cmd_new = qmp_marshal_add_fd,
},
SQMP
@@ -1750,7 +1750,7 @@ EQMP
.args_type = "fdset-id:i,fd:i?",
.params = "remove-fd fdset-id fd",
.help = "Remove a file descriptor from an fd set",
- .mhandler.cmd_new = qmp_marshal_input_remove_fd,
+ .mhandler.cmd_new = qmp_marshal_remove_fd,
},
SQMP
@@ -1782,7 +1782,7 @@ EQMP
.name = "query-fdsets",
.args_type = "",
.help = "Return information describing all fd sets",
- .mhandler.cmd_new = qmp_marshal_input_query_fdsets,
+ .mhandler.cmd_new = qmp_marshal_query_fdsets,
},
SQMP
@@ -1831,7 +1831,7 @@ EQMP
{
.name = "block_passwd",
.args_type = "device:s?,node-name:s?,password:s",
- .mhandler.cmd_new = qmp_marshal_input_block_passwd,
+ .mhandler.cmd_new = qmp_marshal_block_passwd,
},
SQMP
@@ -1857,7 +1857,7 @@ EQMP
{
.name = "block_set_io_throttle",
.args_type = "device:B,bps:l,bps_rd:l,bps_wr:l,iops:l,iops_rd:l,iops_wr:l,bps_max:l?,bps_rd_max:l?,bps_wr_max:l?,iops_max:l?,iops_rd_max:l?,iops_wr_max:l?,iops_size:l?,group:s?",
- .mhandler.cmd_new = qmp_marshal_input_block_set_io_throttle,
+ .mhandler.cmd_new = qmp_marshal_block_set_io_throttle,
},
SQMP
@@ -1907,7 +1907,7 @@ EQMP
{
.name = "set_password",
.args_type = "protocol:s,password:s,connected:s?",
- .mhandler.cmd_new = qmp_marshal_input_set_password,
+ .mhandler.cmd_new = qmp_marshal_set_password,
},
SQMP
@@ -1933,7 +1933,7 @@ EQMP
{
.name = "expire_password",
.args_type = "protocol:s,time:s",
- .mhandler.cmd_new = qmp_marshal_input_expire_password,
+ .mhandler.cmd_new = qmp_marshal_expire_password,
},
SQMP
@@ -1958,7 +1958,7 @@ EQMP
{
.name = "add_client",
.args_type = "protocol:s,fdname:s,skipauth:b?,tls:b?",
- .mhandler.cmd_new = qmp_marshal_input_add_client,
+ .mhandler.cmd_new = qmp_marshal_add_client,
},
SQMP
@@ -2009,7 +2009,7 @@ EQMP
{
.name = "human-monitor-command",
.args_type = "command-line:s,cpu-index:i?",
- .mhandler.cmd_new = qmp_marshal_input_human_monitor_command,
+ .mhandler.cmd_new = qmp_marshal_human_monitor_command,
},
SQMP
@@ -2088,7 +2088,7 @@ EQMP
{
.name = "query-version",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_version,
+ .mhandler.cmd_new = qmp_marshal_query_version,
},
SQMP
@@ -2125,7 +2125,7 @@ EQMP
{
.name = "query-commands",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_commands,
+ .mhandler.cmd_new = qmp_marshal_query_commands,
},
SQMP
@@ -2162,7 +2162,7 @@ EQMP
{
.name = "query-events",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_events,
+ .mhandler.cmd_new = qmp_marshal_query_events,
},
SQMP
@@ -2207,7 +2207,7 @@ EQMP
{
.name = "query-chardev",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_chardev,
+ .mhandler.cmd_new = qmp_marshal_query_chardev,
},
SQMP
@@ -2248,7 +2248,7 @@ EQMP
{
.name = "query-chardev-backends",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_chardev_backends,
+ .mhandler.cmd_new = qmp_marshal_query_chardev_backends,
},
SQMP
@@ -2432,7 +2432,7 @@ EQMP
{
.name = "query-block",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_block,
+ .mhandler.cmd_new = qmp_marshal_query_block,
},
SQMP
@@ -2559,7 +2559,7 @@ EQMP
{
.name = "query-blockstats",
.args_type = "query-nodes:b?",
- .mhandler.cmd_new = qmp_marshal_input_query_blockstats,
+ .mhandler.cmd_new = qmp_marshal_query_blockstats,
},
SQMP
@@ -2610,7 +2610,7 @@ EQMP
{
.name = "query-cpus",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_cpus,
+ .mhandler.cmd_new = qmp_marshal_query_cpus,
},
SQMP
@@ -2649,7 +2649,7 @@ EQMP
{
.name = "query-iothreads",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_iothreads,
+ .mhandler.cmd_new = qmp_marshal_query_iothreads,
},
SQMP
@@ -2866,7 +2866,7 @@ EQMP
{
.name = "query-pci",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_pci,
+ .mhandler.cmd_new = qmp_marshal_query_pci,
},
SQMP
@@ -2890,7 +2890,7 @@ EQMP
{
.name = "query-kvm",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_kvm,
+ .mhandler.cmd_new = qmp_marshal_query_kvm,
},
SQMP
@@ -2930,7 +2930,7 @@ EQMP
{
.name = "query-status",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_status,
+ .mhandler.cmd_new = qmp_marshal_query_status,
},
SQMP
@@ -2974,7 +2974,7 @@ EQMP
{
.name = "query-mice",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_mice,
+ .mhandler.cmd_new = qmp_marshal_query_mice,
},
SQMP
@@ -3037,12 +3037,12 @@ EQMP
{
.name = "query-vnc",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_vnc,
+ .mhandler.cmd_new = qmp_marshal_query_vnc,
},
{
.name = "query-vnc-servers",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_vnc_servers,
+ .mhandler.cmd_new = qmp_marshal_query_vnc_servers,
},
SQMP
@@ -3119,7 +3119,7 @@ EQMP
{
.name = "query-spice",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_spice,
+ .mhandler.cmd_new = qmp_marshal_query_spice,
},
#endif
@@ -3143,7 +3143,7 @@ EQMP
{
.name = "query-name",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_name,
+ .mhandler.cmd_new = qmp_marshal_query_name,
},
SQMP
@@ -3166,7 +3166,7 @@ EQMP
{
.name = "query-uuid",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_uuid,
+ .mhandler.cmd_new = qmp_marshal_query_uuid,
},
SQMP
@@ -3215,7 +3215,7 @@ EQMP
{
.name = "query-command-line-options",
.args_type = "option:s?",
- .mhandler.cmd_new = qmp_marshal_input_query_command_line_options,
+ .mhandler.cmd_new = qmp_marshal_query_command_line_options,
},
SQMP
@@ -3393,7 +3393,7 @@ EQMP
{
.name = "query-migrate",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_migrate,
+ .mhandler.cmd_new = qmp_marshal_query_migrate,
},
SQMP
@@ -3421,7 +3421,7 @@ EQMP
.name = "migrate-set-capabilities",
.args_type = "capabilities:q",
.params = "capability:s,state:b",
- .mhandler.cmd_new = qmp_marshal_input_migrate_set_capabilities,
+ .mhandler.cmd_new = qmp_marshal_migrate_set_capabilities,
},
SQMP
query-migrate-capabilities
@@ -3447,7 +3447,7 @@ EQMP
{
.name = "query-migrate-capabilities",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_migrate_capabilities,
+ .mhandler.cmd_new = qmp_marshal_query_migrate_capabilities,
},
SQMP
@@ -3473,7 +3473,7 @@ EQMP
.name = "migrate-set-parameters",
.args_type =
"compress-level:i?,compress-threads:i?,decompress-threads:i?",
- .mhandler.cmd_new = qmp_marshal_input_migrate_set_parameters,
+ .mhandler.cmd_new = qmp_marshal_migrate_set_parameters,
},
SQMP
query-migrate-parameters
@@ -3504,7 +3504,7 @@ EQMP
{
.name = "query-migrate-parameters",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_migrate_parameters,
+ .mhandler.cmd_new = qmp_marshal_query_migrate_parameters,
},
SQMP
@@ -3532,19 +3532,19 @@ EQMP
{
.name = "query-balloon",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_balloon,
+ .mhandler.cmd_new = qmp_marshal_query_balloon,
},
{
.name = "query-block-jobs",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_block_jobs,
+ .mhandler.cmd_new = qmp_marshal_query_block_jobs,
},
{
.name = "qom-list",
.args_type = "path:s",
- .mhandler.cmd_new = qmp_marshal_input_qom_list,
+ .mhandler.cmd_new = qmp_marshal_qom_list,
},
{
@@ -3562,58 +3562,58 @@ EQMP
{
.name = "nbd-server-start",
.args_type = "addr:q",
- .mhandler.cmd_new = qmp_marshal_input_nbd_server_start,
+ .mhandler.cmd_new = qmp_marshal_nbd_server_start,
},
{
.name = "nbd-server-add",
.args_type = "device:B,writable:b?",
- .mhandler.cmd_new = qmp_marshal_input_nbd_server_add,
+ .mhandler.cmd_new = qmp_marshal_nbd_server_add,
},
{
.name = "nbd-server-stop",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_nbd_server_stop,
+ .mhandler.cmd_new = qmp_marshal_nbd_server_stop,
},
{
.name = "change-vnc-password",
.args_type = "password:s",
- .mhandler.cmd_new = qmp_marshal_input_change_vnc_password,
+ .mhandler.cmd_new = qmp_marshal_change_vnc_password,
},
{
.name = "qom-list-types",
.args_type = "implements:s?,abstract:b?",
- .mhandler.cmd_new = qmp_marshal_input_qom_list_types,
+ .mhandler.cmd_new = qmp_marshal_qom_list_types,
},
{
.name = "device-list-properties",
.args_type = "typename:s",
- .mhandler.cmd_new = qmp_marshal_input_device_list_properties,
+ .mhandler.cmd_new = qmp_marshal_device_list_properties,
},
{
.name = "query-machines",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_machines,
+ .mhandler.cmd_new = qmp_marshal_query_machines,
},
{
.name = "query-cpu-definitions",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_cpu_definitions,
+ .mhandler.cmd_new = qmp_marshal_query_cpu_definitions,
},
{
.name = "query-target",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_target,
+ .mhandler.cmd_new = qmp_marshal_query_target,
},
{
.name = "query-tpm",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_tpm,
+ .mhandler.cmd_new = qmp_marshal_query_tpm,
},
SQMP
@@ -3647,7 +3647,7 @@ EQMP
{
.name = "query-tpm-models",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_tpm_models,
+ .mhandler.cmd_new = qmp_marshal_query_tpm_models,
},
SQMP
@@ -3668,7 +3668,7 @@ EQMP
{
.name = "query-tpm-types",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_tpm_types,
+ .mhandler.cmd_new = qmp_marshal_query_tpm_types,
},
SQMP
@@ -3689,7 +3689,7 @@ EQMP
{
.name = "chardev-add",
.args_type = "id:s,backend:q",
- .mhandler.cmd_new = qmp_marshal_input_chardev_add,
+ .mhandler.cmd_new = qmp_marshal_chardev_add,
},
SQMP
@@ -3726,7 +3726,7 @@ EQMP
{
.name = "chardev-remove",
.args_type = "id:s",
- .mhandler.cmd_new = qmp_marshal_input_chardev_remove,
+ .mhandler.cmd_new = qmp_marshal_chardev_remove,
},
@@ -3749,7 +3749,7 @@ EQMP
{
.name = "query-rx-filter",
.args_type = "name:s?",
- .mhandler.cmd_new = qmp_marshal_input_query_rx_filter,
+ .mhandler.cmd_new = qmp_marshal_query_rx_filter,
},
SQMP
@@ -3815,7 +3815,7 @@ EQMP
{
.name = "blockdev-add",
.args_type = "options:q",
- .mhandler.cmd_new = qmp_marshal_input_blockdev_add,
+ .mhandler.cmd_new = qmp_marshal_blockdev_add,
},
SQMP
@@ -3874,7 +3874,7 @@ EQMP
{
.name = "query-named-block-nodes",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_named_block_nodes,
+ .mhandler.cmd_new = qmp_marshal_query_named_block_nodes,
},
SQMP
@@ -3936,7 +3936,7 @@ EQMP
{
.name = "query-memdev",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_memdev,
+ .mhandler.cmd_new = qmp_marshal_query_memdev,
},
SQMP
@@ -3974,7 +3974,7 @@ EQMP
{
.name = "query-memory-devices",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_memory_devices,
+ .mhandler.cmd_new = qmp_marshal_query_memory_devices,
},
SQMP
@@ -4001,7 +4001,7 @@ EQMP
{
.name = "query-acpi-ospm-status",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_query_acpi_ospm_status,
+ .mhandler.cmd_new = qmp_marshal_query_acpi_ospm_status,
},
SQMP
@@ -4024,7 +4024,7 @@ EQMP
{
.name = "rtc-reset-reinjection",
.args_type = "",
- .mhandler.cmd_new = qmp_marshal_input_rtc_reset_reinjection,
+ .mhandler.cmd_new = qmp_marshal_rtc_reset_reinjection,
},
#endif
@@ -4045,7 +4045,7 @@ EQMP
{
.name = "trace-event-get-state",
.args_type = "name:s",
- .mhandler.cmd_new = qmp_marshal_input_trace_event_get_state,
+ .mhandler.cmd_new = qmp_marshal_trace_event_get_state,
},
SQMP
@@ -4063,7 +4063,7 @@ EQMP
{
.name = "trace-event-set-state",
.args_type = "name:s,enable:b,ignore-unavailable:b?",
- .mhandler.cmd_new = qmp_marshal_input_trace_event_set_state,
+ .mhandler.cmd_new = qmp_marshal_trace_event_set_state,
},
SQMP
@@ -4081,7 +4081,7 @@ EQMP
{
.name = "x-input-send-event",
.args_type = "console:i?,events:q",
- .mhandler.cmd_new = qmp_marshal_input_x_input_send_event,
+ .mhandler.cmd_new = qmp_marshal_x_input_send_event,
},
SQMP
@@ -4146,7 +4146,7 @@ EQMP
{
.name = "block-set-write-threshold",
.args_type = "node-name:s,write-threshold:l",
- .mhandler.cmd_new = qmp_marshal_input_block_set_write_threshold,
+ .mhandler.cmd_new = qmp_marshal_block_set_write_threshold,
},
SQMP
@@ -4174,7 +4174,7 @@ EQMP
{
.name = "query-rocker",
.args_type = "name:s",
- .mhandler.cmd_new = qmp_marshal_input_query_rocker,
+ .mhandler.cmd_new = qmp_marshal_query_rocker,
},
SQMP
@@ -4195,7 +4195,7 @@ EQMP
{
.name = "query-rocker-ports",
.args_type = "name:s",
- .mhandler.cmd_new = qmp_marshal_input_query_rocker_ports,
+ .mhandler.cmd_new = qmp_marshal_query_rocker_ports,
},
SQMP
@@ -4220,7 +4220,7 @@ EQMP
{
.name = "query-rocker-of-dpa-flows",
.args_type = "name:s,tbl-id:i?",
- .mhandler.cmd_new = qmp_marshal_input_query_rocker_of_dpa_flows,
+ .mhandler.cmd_new = qmp_marshal_query_rocker_of_dpa_flows,
},
SQMP
@@ -4249,7 +4249,7 @@ EQMP
{
.name = "query-rocker-of-dpa-groups",
.args_type = "name:s,type:i?",
- .mhandler.cmd_new = qmp_marshal_input_query_rocker_of_dpa_groups,
+ .mhandler.cmd_new = qmp_marshal_query_rocker_of_dpa_groups,
},
SQMP
diff --git a/qmp.c b/qmp.c
index 403805a..f31c6dc 100644
--- a/qmp.c
+++ b/qmp.c
@@ -151,9 +151,9 @@ VncInfo2List *qmp_query_vnc_servers(Error **errp)
* #ifdef CONFIG_SPICE. Necessary for an accurate query-commands
* result. However, the QAPI schema is blissfully unaware of that,
* and the QAPI code generator happily generates a dead
- * qmp_marshal_input_query_spice() that calls qmp_query_spice().
- * Provide it one, or else linking fails.
- * FIXME Educate the QAPI schema on CONFIG_SPICE.
+ * qmp_marshal_query_spice() that calls qmp_query_spice(). Provide it
+ * one, or else linking fails. FIXME Educate the QAPI schema on
+ * CONFIG_SPICE.
*/
SpiceInfo *qmp_query_spice(Error **errp)
{
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 7eb3544..ab96b4a 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -197,7 +197,7 @@ out:
c_name=ret_type.c_name())
def gen_marshal_proto(name):
- ret = 'void qmp_marshal_input_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
+ ret = 'void qmp_marshal_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
if not middle_mode:
ret = 'static ' + ret
return ret
@@ -241,7 +241,7 @@ def gen_register_command(name, success_response):
options = 'QCO_NO_SUCCESS_RESP'
ret = mcgen('''
-qmp_register_command("%(name)s", qmp_marshal_input_%(c_name)s, %(opts)s);
+qmp_register_command("%(name)s", qmp_marshal_%(c_name)s, %(opts)s);
''',
name=name, c_name=c_name(name),
opts=options)
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 23/32] qapi: De-duplicate parameter list generation
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (21 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 22/32] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO() Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-05 17:00 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 24/32] qapi-commands: De-duplicate output marshaling functions Markus Armbruster
` (8 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Generated qapi-event.[ch] lose line breaks. No change otherwise.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi-commands.py | 11 ++---------
scripts/qapi-event.py | 18 +++---------------
scripts/qapi.py | 16 ++++++++++++++++
3 files changed, 21 insertions(+), 24 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index ab96b4a..0658abd 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -16,19 +16,12 @@ from qapi import *
import re
def gen_command_decl(name, arg_type, ret_type):
- argstr = ''
- if arg_type:
- for memb in arg_type.members:
- if memb.optional:
- argstr += 'bool has_%s, ' % c_name(memb.name)
- argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
- c_name(memb.name))
return mcgen('''
-%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(params)s);
''',
c_type=(ret_type and ret_type.c_type()) or 'void',
c_name=c_name(name),
- args=argstr)
+ params=gen_params(arg_type, 'Error **errp'))
def gen_err_check(err):
if not err:
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 24ca01a..10f4bae 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -14,21 +14,9 @@
from qapi import *
def gen_event_send_proto(name, arg_type):
- api_name = "void qapi_event_send_%s(" % c_name(name).lower()
- l = len(api_name)
-
- if arg_type:
- for m in arg_type.members:
- if m.optional:
- api_name += "bool has_%s,\n" % c_name(m.name)
- api_name += "".ljust(l)
-
- api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
- c_name(m.name))
- api_name += "".ljust(l)
-
- api_name += "Error **errp)"
- return api_name
+ return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
+ 'c_name': c_name(name.lower()),
+ 'param': gen_params(arg_type, 'Error **errp')}
def gen_event_send_decl(name, arg_type):
return mcgen('''
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 137ee15..ab4b792 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1400,6 +1400,22 @@ extern const char *const %(c_name)s_lookup[];
c_name=c_name(name))
return ret
+def gen_params(arg_type, extra):
+ if not arg_type:
+ return extra
+ assert not arg_type.variants
+ ret = ''
+ sep = ''
+ for memb in arg_type.members:
+ ret += sep
+ sep = ', '
+ if memb.optional:
+ ret += 'bool has_%s, ' % c_name(memb.name)
+ ret += '%s %s' % (memb.type.c_type(is_param=True), c_name(memb.name))
+ if extra:
+ ret += sep + extra
+ return ret
+
#
# Common command line parsing
#
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 24/32] qapi-commands: De-duplicate output marshaling functions
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (22 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 23/32] qapi: De-duplicate parameter list generation Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 25/32] qapi: Improve built-in type documentation Markus Armbruster
` (7 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
gen_marshal_output() uses its parameter name only for name of the
generated function. Name it after the type being marshaled instead of
its caller, and drop duplicates.
Saves 7 copies of qmp_marshal_output_int() in qemu-ga, and one copy of
qmp_marshal_output_str() in qemu-system-*.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 4 ++--
scripts/qapi-commands.py | 17 ++++++++++-------
2 files changed, 12 insertions(+), 9 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 7eafcde..5daa335 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -715,7 +715,7 @@ Example:
$ cat qapi-generated/example-qmp-marshal.c
[Uninteresting stuff omitted...]
- static void qmp_marshal_output_my_command(UserDefOne *ret_in, QObject **ret_out, Error **errp)
+ static void qmp_marshal_output_UserDefOne(UserDefOne *ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -758,7 +758,7 @@ Example:
goto out;
}
- qmp_marshal_output_my_command(retval, ret, &local_err);
+ qmp_marshal_output_UserDefOne(retval, ret, &local_err);
out:
error_propagate(errp, local_err);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 0658abd..0603cff 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -59,7 +59,7 @@ def gen_call(name, arg_type, ret_type):
qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
''',
- c_name=c_name(name))
+ c_name=ret_type.c_name())
pop_indent()
return ret
@@ -160,10 +160,10 @@ qapi_dealloc_visitor_cleanup(md);
pop_indent()
return ret
-def gen_marshal_output(name, ret_type):
+def gen_marshal_output(ret_type):
return mcgen('''
-static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject **ret_out, Error **errp)
+static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -186,8 +186,7 @@ out:
qapi_dealloc_visitor_cleanup(md);
}
''',
- c_type=ret_type.c_type(), c_cmd_name=c_name(name),
- c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_name=ret_type.c_name())
def gen_marshal_proto(name):
ret = 'void qmp_marshal_%s(QDict *args, QObject **ret, Error **errp)' % c_name(name)
@@ -260,21 +259,25 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
self.decl = None
self.defn = None
self.regy = None
+ self.visited_ret_types = None
def visit_begin(self):
self.decl = ''
self.defn = ''
self.regy = ''
+ self.visited_ret_types = set()
def visit_end(self):
if not middle_mode:
self.defn += gen_registry(self.regy)
self.regy = None
+ self.visited_ret_types = None
def visit_command(self, name, info, arg_type, ret_type,
gen, success_response):
if not gen:
return
self.decl += gen_command_decl(name, arg_type, ret_type)
- if ret_type:
- self.defn += gen_marshal_output(name, ret_type)
+ if ret_type and ret_type not in self.visited_ret_types:
+ self.visited_ret_types.add(ret_type)
+ self.defn += gen_marshal_output(ret_type)
if middle_mode:
self.decl += gen_marshal_decl(name)
self.defn += gen_marshal(name, arg_type, ret_type)
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 25/32] qapi: Improve built-in type documentation
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (23 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 24/32] qapi-commands: De-duplicate output marshaling functions Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type Markus Armbruster
` (6 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Clarify how they map to JSON. Add how they map to C. Fix the
reference to StringInputVisitor.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 29 ++++++++++++++++++-----------
1 file changed, 18 insertions(+), 11 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 5daa335..5fb35d2 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -140,17 +140,24 @@ must have a value that forms a struct name.
=== Built-in Types ===
-The following types are built-in to the parser:
- 'str' - arbitrary UTF-8 string
- 'int' - 64-bit signed integer (although the C code may place further
- restrictions on acceptable range)
- 'number' - floating point number
- 'bool' - JSON value of true or false
- 'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
- bit size
- 'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
- 'size' - like 'uint64', but allows scaled suffix from command line
- visitor
+The following types are predefined, and map to C as follows:
+
+ Schema C JSON
+ str char * any JSON string, UTF-8
+ number double any JSON number
+ int int64_t a JSON number without fractional part
+ that fits into the C integer type
+ int8 int8_t likewise
+ int16 int16_t likewise
+ int32 int32_t likewise
+ int64 int64_t likewise
+ uint8 uint8_t likewise
+ uint16 uint16_t likewise
+ uint32 uint32_t likewise
+ uint64 uint64_t likewise
+ size uint64_t like uint64_t, except StringInputVisitor
+ accepts size suffixes
+ bool bool JSON true or false
=== Includes ===
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (24 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 25/32] qapi: Improve built-in type documentation Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-07 19:30 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 27/32] qom: Don't use 'gen': false for qom-get, qom-set, object-add Markus Armbruster
` (5 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.
'**' will go away next.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 1 +
include/qapi/visitor-impl.h | 2 +
include/qapi/visitor.h | 1 +
qapi/qapi-dealloc-visitor.c | 9 ++++
qapi/qapi-visit-core.c | 6 +++
qapi/qmp-input-visitor.c | 11 +++++
qapi/qmp-output-visitor.c | 9 ++++
scripts/qapi-types.py | 1 +
scripts/qapi.py | 9 ++--
tests/Makefile | 3 +-
tests/qapi-schema/args-any.err | 1 +
.../{flat-union-base-star.exit => args-any.exit} | 0
tests/qapi-schema/args-any.json | 2 +
.../{flat-union-base-star.out => args-any.out} | 0
tests/qapi-schema/flat-union-base-any.err | 1 +
tests/qapi-schema/flat-union-base-any.exit | 1 +
...ion-base-star.json => flat-union-base-any.json} | 2 +-
tests/qapi-schema/flat-union-base-any.out | 0
tests/qapi-schema/flat-union-base-star.err | 1 -
tests/qapi-schema/qapi-schema-test.json | 2 +
tests/qapi-schema/qapi-schema-test.out | 4 ++
tests/qapi-schema/type-bypass.out | 4 +-
tests/test-qmp-commands.c | 5 ++
tests/test-qmp-input-visitor.c | 45 ++++++++++++++++++
tests/test-qmp-output-visitor.c | 53 ++++++++++++++++++++++
25 files changed, 165 insertions(+), 8 deletions(-)
create mode 100644 tests/qapi-schema/args-any.err
rename tests/qapi-schema/{flat-union-base-star.exit => args-any.exit} (100%)
create mode 100644 tests/qapi-schema/args-any.json
rename tests/qapi-schema/{flat-union-base-star.out => args-any.out} (100%)
create mode 100644 tests/qapi-schema/flat-union-base-any.err
create mode 100644 tests/qapi-schema/flat-union-base-any.exit
rename tests/qapi-schema/{flat-union-base-star.json => flat-union-base-any.json} (95%)
create mode 100644 tests/qapi-schema/flat-union-base-any.out
delete mode 100644 tests/qapi-schema/flat-union-base-star.err
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 5fb35d2..96f1dd1 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -158,6 +158,7 @@ The following types are predefined, and map to C as follows:
size uint64_t like uint64_t, except StringInputVisitor
accepts size suffixes
bool bool JSON true or false
+ any QObject * any JSON value
=== Includes ===
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index f4a2f74..8c0ba57 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -40,6 +40,8 @@ struct Visitor
void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
void (*type_number)(Visitor *v, double *obj, const char *name,
Error **errp);
+ void (*type_any)(Visitor *v, QObject **obj, const char *name,
+ Error **errp);
/* May be NULL */
void (*optional)(Visitor *v, bool *present, const char *name,
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 00ba104..cfc19a6 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -58,6 +58,7 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
+void visit_type_any(Visitor *v, QObject **obj, const char *name, Error **errp);
bool visit_start_union(Visitor *v, bool data_present, Error **errp);
void visit_end_union(Visitor *v, bool data_present, Error **errp);
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index d7f92c5..737deab 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -151,6 +151,14 @@ static void qapi_dealloc_type_number(Visitor *v, double *obj, const char *name,
{
}
+static void qapi_dealloc_type_anything(Visitor *v, QObject **obj,
+ const char *name, Error **errp)
+{
+ if (obj) {
+ qobject_decref(*obj);
+ }
+}
+
static void qapi_dealloc_type_size(Visitor *v, uint64_t *obj, const char *name,
Error **errp)
{
@@ -216,6 +224,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
v->visitor.type_bool = qapi_dealloc_type_bool;
v->visitor.type_str = qapi_dealloc_type_str;
v->visitor.type_number = qapi_dealloc_type_number;
+ v->visitor.type_any = qapi_dealloc_type_anything;
v->visitor.type_size = qapi_dealloc_type_size;
v->visitor.start_union = qapi_dealloc_start_union;
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 5a7c900..59ed506 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -260,6 +260,12 @@ void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp)
v->type_number(v, obj, name, errp);
}
+void visit_type_any(Visitor *v, QObject **obj, const char *name,
+ Error **errp)
+{
+ v->type_any(v, obj, name, errp);
+}
+
void output_type_enum(Visitor *v, int *obj, const char * const strings[],
const char *kind, const char *name,
Error **errp)
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index e97b8a4..5dd9ed5 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -286,6 +286,16 @@ static void qmp_input_type_number(Visitor *v, double *obj, const char *name,
}
}
+static void qmp_input_type_any(Visitor *v, QObject **obj, const char *name,
+ Error **errp)
+{
+ QmpInputVisitor *qiv = to_qiv(v);
+ QObject *qobj = qmp_input_get_object(qiv, name, true);
+
+ qobject_incref(qobj);
+ *obj = qobj;
+}
+
static void qmp_input_optional(Visitor *v, bool *present, const char *name,
Error **errp)
{
@@ -329,6 +339,7 @@ QmpInputVisitor *qmp_input_visitor_new(QObject *obj)
v->visitor.type_bool = qmp_input_type_bool;
v->visitor.type_str = qmp_input_type_str;
v->visitor.type_number = qmp_input_type_number;
+ v->visitor.type_any = qmp_input_type_any;
v->visitor.optional = qmp_input_optional;
v->visitor.get_next_type = qmp_input_get_next_type;
diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index efc19d5..1e36a7a 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -186,6 +186,14 @@ static void qmp_output_type_number(Visitor *v, double *obj, const char *name,
qmp_output_add(qov, name, qfloat_from_double(*obj));
}
+static void qmp_output_type_any(Visitor *v, QObject **obj, const char *name,
+ Error **errp)
+{
+ QmpOutputVisitor *qov = to_qov(v);
+ qobject_incref(*obj);
+ qmp_output_add_obj(qov, name, *obj);
+}
+
QObject *qmp_output_get_qobject(QmpOutputVisitor *qov)
{
QObject *obj = qmp_output_first(qov);
@@ -233,6 +241,7 @@ QmpOutputVisitor *qmp_output_visitor_new(void)
v->visitor.type_bool = qmp_output_type_bool;
v->visitor.type_str = qmp_output_type_str;
v->visitor.type_number = qmp_output_type_number;
+ v->visitor.type_any = qmp_output_type_any;
QTAILQ_INIT(&v->stack);
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index d7f21b3..5b6de4c 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -305,6 +305,7 @@ fdef.write(mcgen('''
fdecl.write(mcgen('''
#include <stdbool.h>
#include <stdint.h>
+#include "qapi/qmp/qobject.h"
'''))
schema = QAPISchema(input_file)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index ab4b792..dd534f3 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -33,6 +33,7 @@ builtin_types = {
'uint32': 'QTYPE_QINT',
'uint64': 'QTYPE_QINT',
'size': 'QTYPE_QINT',
+ 'any': None, # any qtype_code possible, actually
}
# Whitelist of commands allowed to return a non-dictionary
@@ -1039,8 +1040,7 @@ class QAPISchema(object):
def _def_builtin_type(self, name, json_type, c_type, c_null):
self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type, c_null))
- if name != '**':
- self._make_array_type(name) # TODO really needed?
+ self._make_array_type(name) # TODO really needed?
def _def_predefineds(self):
for t in [('str', 'string', 'char' + pointer_suffix, 'NULL'),
@@ -1056,8 +1056,9 @@ class QAPISchema(object):
('uint64', 'int', 'uint64_t', '0'),
('size', 'int', 'uint64_t', '0'),
('bool', 'boolean', 'bool', 'false'),
- ('**', 'value', None, None)]:
+ ('any', 'value', 'QObject' + pointer_suffix , 'NULL')]:
self._def_builtin_type(*t)
+ self.entity_dict['**'] = self.lookup_type('any') # TODO drop this alias
def _make_implicit_enum_type(self, name, values):
name = name + 'Kind'
@@ -1206,6 +1207,8 @@ class QAPISchema(object):
def visit(self, visitor):
visitor.visit_begin()
for name in sorted(self.entity_dict.keys()):
+ if self.entity_dict[name].name != name:
+ continue # ignore alias TODO drop alias and remove
self.entity_dict[name].visit(visitor)
visitor.visit_end()
diff --git a/tests/Makefile b/tests/Makefile
index 2d6ca9d..b578d80 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -232,6 +232,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
args-array-empty.json args-array-unknown.json args-int.json \
args-unknown.json args-member-unknown.json args-member-array.json \
args-member-array-bad.json args-alternate.json args-union.json \
+ args-any.json \
returns-array-bad.json returns-int.json returns-dict.json \
returns-unknown.json returns-alternate.json returns-whitelist.json \
missing-colon.json missing-comma-list.json missing-comma-object.json \
@@ -246,7 +247,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
flat-union-invalid-branch-key.json flat-union-reverse-define.json \
flat-union-string-discriminator.json union-base-no-discriminator.json \
flat-union-bad-discriminator.json flat-union-bad-base.json \
- flat-union-base-star.json \
+ flat-union-base-any.json \
flat-union-array-branch.json flat-union-int-branch.json \
flat-union-base-union.json flat-union-branch-clash.json \
alternate-nested.json alternate-unknown.json alternate-clash.json \
diff --git a/tests/qapi-schema/args-any.err b/tests/qapi-schema/args-any.err
new file mode 100644
index 0000000..bf9b5e0
--- /dev/null
+++ b/tests/qapi-schema/args-any.err
@@ -0,0 +1 @@
+tests/qapi-schema/args-any.json:2: 'data' for command 'oops' cannot use built-in type 'any'
diff --git a/tests/qapi-schema/flat-union-base-star.exit b/tests/qapi-schema/args-any.exit
similarity index 100%
rename from tests/qapi-schema/flat-union-base-star.exit
rename to tests/qapi-schema/args-any.exit
diff --git a/tests/qapi-schema/args-any.json b/tests/qapi-schema/args-any.json
new file mode 100644
index 0000000..58fe5e4
--- /dev/null
+++ b/tests/qapi-schema/args-any.json
@@ -0,0 +1,2 @@
+# we do not allow an 'any' argument
+{ 'command': 'oops', 'data': 'any' }
diff --git a/tests/qapi-schema/flat-union-base-star.out b/tests/qapi-schema/args-any.out
similarity index 100%
rename from tests/qapi-schema/flat-union-base-star.out
rename to tests/qapi-schema/args-any.out
diff --git a/tests/qapi-schema/flat-union-base-any.err b/tests/qapi-schema/flat-union-base-any.err
new file mode 100644
index 0000000..ad4d629
--- /dev/null
+++ b/tests/qapi-schema/flat-union-base-any.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-base-any.json:8: Base 'any' is not a valid struct
diff --git a/tests/qapi-schema/flat-union-base-any.exit b/tests/qapi-schema/flat-union-base-any.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/flat-union-base-any.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-base-star.json b/tests/qapi-schema/flat-union-base-any.json
similarity index 95%
rename from tests/qapi-schema/flat-union-base-star.json
rename to tests/qapi-schema/flat-union-base-any.json
index 5099439..fe66b71 100644
--- a/tests/qapi-schema/flat-union-base-star.json
+++ b/tests/qapi-schema/flat-union-base-any.json
@@ -6,7 +6,7 @@
{ 'struct': 'TestTypeB',
'data': { 'integer': 'int' } }
{ 'union': 'TestUnion',
- 'base': '**',
+ 'base': 'any',
'discriminator': 'enum1',
'data': { 'value1': 'TestTypeA',
'value2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-base-any.out b/tests/qapi-schema/flat-union-base-any.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/flat-union-base-star.err b/tests/qapi-schema/flat-union-base-star.err
deleted file mode 100644
index b7748f0..0000000
--- a/tests/qapi-schema/flat-union-base-star.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/flat-union-base-star.json:8: Base '**' is not a valid struct
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 90b4740..6665281 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -83,6 +83,8 @@
'returns': 'UserDefTwo' }
{ 'command': 'user_def_cmd3', 'data': {'a': 'int', '*b': 'int' },
'returns': 'int' }
+# note: command name 'guest-sync' chosen to avoid "cannot use built-in" error
+{ 'command': 'guest-sync', 'data': { 'arg': 'any' }, 'returns': 'any' }
# For testing integer range flattening in opts-visitor. The following schema
# corresponds to the option format:
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index b30bccc..49c1197 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -14,6 +14,8 @@ object :obj-__org.qemu_x-command-arg
member d: __org.qemu_x-Alt optional=False
object :obj-boolList-wrapper
member data: boolList optional=False
+object :obj-guest-sync-arg
+ member arg: any optional=False
object :obj-int16List-wrapper
member data: int16List optional=False
object :obj-int32List-wrapper
@@ -149,6 +151,8 @@ object __org.qemu_x-Union2
case __org.qemu_x-value: __org.qemu_x-Struct2
command __org.qemu_x-command :obj-__org.qemu_x-command-arg -> __org.qemu_x-Union1
gen=True success_response=True
+command guest-sync :obj-guest-sync-arg -> any
+ gen=True success_response=True
command user_def_cmd None -> None
gen=True success_response=True
command user_def_cmd1 :obj-user_def_cmd1-arg -> None
diff --git a/tests/qapi-schema/type-bypass.out b/tests/qapi-schema/type-bypass.out
index 0070d4b..db2a4e6 100644
--- a/tests/qapi-schema/type-bypass.out
+++ b/tests/qapi-schema/type-bypass.out
@@ -1,4 +1,4 @@
object :obj-unsafe-arg
- member arg: ** optional=False
-command unsafe :obj-unsafe-arg -> **
+ member arg: any optional=False
+command unsafe :obj-unsafe-arg -> any
gen=False success_response=True
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 9918f23..8d5249e 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -51,6 +51,11 @@ int64_t qmp_user_def_cmd3(int64_t a, bool has_b, int64_t b, Error **errp)
return a + (has_b ? b : 0);
}
+QObject *qmp_guest_sync(QObject *arg, Error **errp)
+{
+ return arg;
+}
+
__org_qemu_x_Union1 *qmp___org_qemu_x_command(__org_qemu_x_EnumList *a,
__org_qemu_x_StructList *b,
__org_qemu_x_Union2 *c,
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 508c11a..61715b3 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -298,6 +298,49 @@ static void test_visitor_in_list(TestInputVisitorData *data,
qapi_free_UserDefOneList(head);
}
+static void test_visitor_in_any(TestInputVisitorData *data,
+ const void *unused)
+{
+ QObject *res = NULL;
+ Error *err = NULL;
+ Visitor *v;
+ QInt *qint;
+ QBool *qbool;
+ QString *qstring;
+ QDict *qdict;
+ QObject *qobj;
+
+ v = visitor_input_test_init(data, "-42");
+ visit_type_any(v, &res, NULL, &err);
+ g_assert(!err);
+ qint = qobject_to_qint(res);
+ g_assert(qint);
+ g_assert_cmpint(qint_get_int(qint), ==, -42);
+ qobject_decref(res);
+
+ v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
+ visit_type_any(v, &res, NULL, &err);
+ g_assert(!err);
+ qdict = qobject_to_qdict(res);
+ g_assert(qdict && qdict_size(qdict) == 3);
+ qobj = qdict_get(qdict, "integer");
+ g_assert(qobj);
+ qint = qobject_to_qint(qobj);
+ g_assert(qint);
+ g_assert_cmpint(qint_get_int(qint), ==, -42);
+ qobj = qdict_get(qdict, "boolean");
+ g_assert(qobj);
+ qbool = qobject_to_qbool(qobj);
+ g_assert(qbool);
+ g_assert(qbool_get_bool(qbool) == true);
+ qobj = qdict_get(qdict, "string");
+ g_assert(qobj);
+ qstring = qobject_to_qstring(qobj);
+ g_assert(qstring);
+ g_assert_cmpstr(qstring_get_str(qstring), ==, "foo");
+ qobject_decref(res);
+}
+
static void test_visitor_in_union_flat(TestInputVisitorData *data,
const void *unused)
{
@@ -669,6 +712,8 @@ int main(int argc, char **argv)
&in_visitor_data, test_visitor_in_struct_nested);
input_visitor_test_add("/visitor/input/list",
&in_visitor_data, test_visitor_in_list);
+ input_visitor_test_add("/visitor/input/any",
+ &in_visitor_data, test_visitor_in_any);
input_visitor_test_add("/visitor/input/union-flat",
&in_visitor_data, test_visitor_in_union_flat);
input_visitor_test_add("/visitor/input/alternate",
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 338ada0..1a28dc2 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -428,6 +428,57 @@ static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
qapi_free_UserDefTwoList(head);
}
+static void test_visitor_out_any(TestOutputVisitorData *data,
+ const void *unused)
+{
+ QObject *qobj;
+ Error *err = NULL;
+ QInt *qint;
+ QBool *qbool;
+ QString *qstring;
+ QDict *qdict;
+ QObject *obj;
+
+ qobj = QOBJECT(qint_from_int(-42));
+ visit_type_any(data->ov, &qobj, NULL, &err);
+ g_assert(!err);
+ obj = qmp_output_get_qobject(data->qov);
+ g_assert(obj != NULL);
+ g_assert(qobject_type(obj) == QTYPE_QINT);
+ g_assert_cmpint(qint_get_int(qobject_to_qint(obj)), ==, -42);
+ qobject_decref(obj);
+ qobject_decref(qobj);
+
+ qdict = qdict_new();
+ qdict_put(qdict, "integer", qint_from_int(-42));
+ qdict_put(qdict, "boolean", qbool_from_bool(true));
+ qdict_put(qdict, "string", qstring_from_str("foo"));
+ qobj = QOBJECT(qdict);
+ visit_type_any(data->ov, &qobj, NULL, &err);
+ g_assert(!err);
+ obj = qmp_output_get_qobject(data->qov);
+ g_assert(obj != NULL);
+ qdict = qobject_to_qdict(obj);
+ g_assert(qdict);
+ qobj = qdict_get(qdict, "integer");
+ g_assert(qobj);
+ qint = qobject_to_qint(qobj);
+ g_assert(qint);
+ g_assert_cmpint(qint_get_int(qint), ==, -42);
+ qobj = qdict_get(qdict, "boolean");
+ g_assert(qobj);
+ qbool = qobject_to_qbool(qobj);
+ g_assert(qbool);
+ g_assert(qbool_get_bool(qbool) == true);
+ qobj = qdict_get(qdict, "string");
+ g_assert(qobj);
+ qstring = qobject_to_qstring(qobj);
+ g_assert(qstring);
+ g_assert_cmpstr(qstring_get_str(qstring), ==, "foo");
+ qobject_decref(obj);
+ qobject_decref(qobj);
+}
+
static void test_visitor_out_union_flat(TestOutputVisitorData *data,
const void *unused)
{
@@ -832,6 +883,8 @@ int main(int argc, char **argv)
&out_visitor_data, test_visitor_out_struct_errors);
output_visitor_test_add("/visitor/output/list",
&out_visitor_data, test_visitor_out_list);
+ output_visitor_test_add("/visitor/output/any",
+ &out_visitor_data, test_visitor_out_any);
output_visitor_test_add("/visitor/output/list-qapi-free",
&out_visitor_data, test_visitor_out_list_qapi_free);
output_visitor_test_add("/visitor/output/union-flat",
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 27/32] qom: Don't use 'gen': false for qom-get, qom-set, object-add
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (25 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 28/32] qapi-schema: Fix up misleading specification of netdev_add Markus Armbruster
` (4 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
With the previous commit, the generated marshalers just work, and save
us a bit of handwritten code.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
include/monitor/monitor.h | 3 ---
qapi-schema.json | 9 +++------
qmp-commands.hx | 6 +++---
qmp.c | 21 +++++++--------------
scripts/qapi.py | 1 +
5 files changed, 14 insertions(+), 26 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 9aff47e..bc6cb6d 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -42,9 +42,6 @@ void monitor_read_command(Monitor *mon, int show_prompt);
int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
void *opaque);
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp);
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp);
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp);
void object_add(const char *type, const char *id, const QDict *qdict,
Visitor *v, Error **errp);
diff --git a/qapi-schema.json b/qapi-schema.json
index 4342a08..17d2e60 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1698,8 +1698,7 @@
##
{ 'command': 'qom-get',
'data': { 'path': 'str', 'property': 'str' },
- 'returns': '**',
- 'gen': false }
+ 'returns': 'any' }
##
# @qom-set:
@@ -1716,8 +1715,7 @@
# Since: 1.2
##
{ 'command': 'qom-set',
- 'data': { 'path': 'str', 'property': 'str', 'value': '**' },
- 'gen': false }
+ 'data': { 'path': 'str', 'property': 'str', 'value': 'any' } }
##
# @set_password:
@@ -2113,8 +2111,7 @@
# Since: 2.0
##
{ 'command': 'object-add',
- 'data': {'qom-type': 'str', 'id': 'str', '*props': '**'},
- 'gen': false }
+ 'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }
##
# @object-del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 41a39fc..76942a6 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -928,7 +928,7 @@ EQMP
{
.name = "object-add",
.args_type = "qom-type:s,id:s,props:q?",
- .mhandler.cmd_new = qmp_object_add,
+ .mhandler.cmd_new = qmp_marshal_object_add,
},
SQMP
@@ -3550,13 +3550,13 @@ EQMP
{
.name = "qom-set",
.args_type = "path:s,property:s,value:q",
- .mhandler.cmd_new = qmp_qom_set,
+ .mhandler.cmd_new = qmp_marshal_qom_set,
},
{
.name = "qom-get",
.args_type = "path:s,property:s",
- .mhandler.cmd_new = qmp_qom_get,
+ .mhandler.cmd_new = qmp_marshal_qom_get,
},
{
diff --git a/qmp.c b/qmp.c
index f31c6dc..b8b8b5b 100644
--- a/qmp.c
+++ b/qmp.c
@@ -228,12 +228,9 @@ ObjectPropertyInfoList *qmp_qom_list(const char *path, Error **errp)
return props;
}
-/* FIXME: teach qapi about how to pass through Visitors */
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp)
+void qmp_qom_set(const char *path, const char *property, QObject *value,
+ Error **errp)
{
- const char *path = qdict_get_str(qdict, "path");
- const char *property = qdict_get_str(qdict, "property");
- QObject *value = qdict_get(qdict, "value");
Object *obj;
obj = object_resolve_path(path, NULL);
@@ -246,20 +243,18 @@ void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp)
object_property_set_qobject(obj, value, property, errp);
}
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp)
+QObject *qmp_qom_get(const char *path, const char *property, Error **errp)
{
- const char *path = qdict_get_str(qdict, "path");
- const char *property = qdict_get_str(qdict, "property");
Object *obj;
obj = object_resolve_path(path, NULL);
if (!obj) {
error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
"Device '%s' not found", path);
- return;
+ return NULL;
}
- *ret = object_property_get_qobject(obj, property, errp);
+ return object_property_get_qobject(obj, property, errp);
}
void qmp_set_password(const char *protocol, const char *password,
@@ -655,11 +650,9 @@ out:
object_unref(obj);
}
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp)
+void qmp_object_add(const char *type, const char *id,
+ bool has_props, QObject *props, Error **errp)
{
- const char *type = qdict_get_str(qdict, "qom-type");
- const char *id = qdict_get_str(qdict, "id");
- QObject *props = qdict_get(qdict, "props");
const QDict *pdict = NULL;
QmpInputVisitor *qiv;
diff --git a/scripts/qapi.py b/scripts/qapi.py
index dd534f3..8a40912 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -40,6 +40,7 @@ builtin_types = {
returns_whitelist = [
# From QMP:
'human-monitor-command',
+ 'qom-get',
'query-migrate-cache-size',
'query-tpm-models',
'query-tpm-types',
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 28/32] qapi-schema: Fix up misleading specification of netdev_add
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (26 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 27/32] qom: Don't use 'gen': false for qom-get, qom-set, object-add Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 29/32] qapi: Pseudo-type '**' is now unused, drop it Markus Armbruster
` (3 subsequent siblings)
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
It doesn't take a 'props' argument, let alone one in the format
"NAME=VALUE,..."
The bogus arguments specification doesn't matter due to 'gen': false.
Clean it up to be incomplete rather than wrong, and document the
incompleteness.
While there, improve netdev_add usage example in the manual: add a
device option to show how it's done.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 2 +-
qapi-schema.json | 13 +++++++------
qmp-commands.hx | 4 +++-
3 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 96f1dd1..4d3e77a 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -465,7 +465,7 @@ try to avoid adding new commands that rely on this, and instead use
type-safe unions. For an example of bypass usage:
{ 'command': 'netdev_add',
- 'data': {'type': 'str', 'id': 'str', '*props': '**'},
+ 'data': {'type': 'str', 'id': 'str'},
'gen': false }
Normally, the QAPI schema is used to describe synchronous exchanges,
diff --git a/qapi-schema.json b/qapi-schema.json
index 17d2e60..e91e3b9 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2065,11 +2065,12 @@
#
# @id: the name of the new network backend
#
-# @props: #optional a list of properties to be passed to the backend in
-# the format 'name=value', like 'ifname=tap0,script=no'
+# Additional arguments depend on the type.
#
-# Notes: The semantics of @props is not well defined. Future commands will be
-# introduced that provide stronger typing for backend creation.
+# TODO This command effectively bypasses QAPI completely due to its
+# "additional arguments" business. It shouldn't have been added to
+# the schema in this form. It should be qapified properly, or
+# replaced by a properly qapified command.
#
# Since: 0.14.0
#
@@ -2077,8 +2078,8 @@
# If @type is not a valid network backend, DeviceNotFound
##
{ 'command': 'netdev_add',
- 'data': {'type': 'str', 'id': 'str', '*props': '**'},
- 'gen': false }
+ 'data': {'type': 'str', 'id': 'str'},
+ 'gen': false } # so we can get the additional arguments
##
# @netdev_del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 76942a6..b06d74c 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -892,7 +892,9 @@ Arguments:
Example:
--> { "execute": "netdev_add", "arguments": { "type": "user", "id": "netdev1" } }
+-> { "execute": "netdev_add",
+ "arguments": { "type": "user", "id": "netdev1",
+ "dnssearch": "example.org" } }
<- { "return": {} }
Note: The supported device options are the same ones supported by the '-netdev'
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 29/32] qapi: Pseudo-type '**' is now unused, drop it
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (27 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 28/32] qapi-schema: Fix up misleading specification of netdev_add Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-05 17:13 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
` (2 subsequent siblings)
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
'gen': false needs to stay for now, because netdev_add is still using
it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
docs/qapi-code-gen.txt | 18 ++++++------------
scripts/qapi.py | 20 ++++----------------
tests/Makefile | 2 +-
tests/qapi-schema/type-bypass-no-gen.err | 1 -
tests/qapi-schema/type-bypass-no-gen.exit | 1 -
tests/qapi-schema/type-bypass-no-gen.json | 2 --
tests/qapi-schema/type-bypass-no-gen.out | 0
tests/qapi-schema/type-bypass.err | 0
tests/qapi-schema/type-bypass.exit | 1 -
tests/qapi-schema/type-bypass.json | 2 --
tests/qapi-schema/type-bypass.out | 4 ----
11 files changed, 11 insertions(+), 40 deletions(-)
delete mode 100644 tests/qapi-schema/type-bypass-no-gen.err
delete mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
delete mode 100644 tests/qapi-schema/type-bypass-no-gen.json
delete mode 100644 tests/qapi-schema/type-bypass-no-gen.out
delete mode 100644 tests/qapi-schema/type-bypass.err
delete mode 100644 tests/qapi-schema/type-bypass.exit
delete mode 100644 tests/qapi-schema/type-bypass.json
delete mode 100644 tests/qapi-schema/type-bypass.out
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 4d3e77a..53e6d9c 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -111,10 +111,7 @@ and field names within a type, should be all lower case with words
separated by a hyphen. However, some existing older commands and
complex types use underscore; when extending such expressions,
consistency is preferred over blindly avoiding underscore. Event
-names should be ALL_CAPS with words separated by underscore. The
-special string '**' appears for some commands that manually perform
-their own type checking rather than relying on the type-safe code
-produced by the qapi code generators.
+names should be ALL_CAPS with words separated by underscore.
Any name (command, event, type, field, or enum value) beginning with
"x-" is marked experimental, and may be withdrawn or changed
@@ -455,14 +452,11 @@ which would validate this Client JSON Protocol transaction:
<= { "return": [ { "value": "one" }, { } ] }
In rare cases, QAPI cannot express a type-safe representation of a
-corresponding Client JSON Protocol command. In these cases, if the
-command expression includes the key 'gen' with boolean value false,
-then the 'data' or 'returns' member that intends to bypass generated
-type-safety and do its own manual validation should use an inline
-dictionary definition, with a value of '**' rather than a valid type
-name for the keys that the generated code will not validate. Please
-try to avoid adding new commands that rely on this, and instead use
-type-safe unions. For an example of bypass usage:
+corresponding Client JSON Protocol command. You then have to suppress
+generation of a marshalling function by including a key 'gen' with
+boolean value false, and instead write your own function. Please try
+to avoid adding new commands that rely on this, and instead use
+type-safe unions. For an example of this usage:
{ 'command': 'netdev_add',
'data': {'type': 'str', 'id': 'str'},
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8a40912..fd8abad 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -427,16 +427,13 @@ def is_enum(name):
def check_type(expr_info, source, value, allow_array = False,
allow_dict = False, allow_optional = False,
- allow_star = False, allow_metas = []):
+ allow_metas = []):
global all_names
orig_value = value
if value is None:
return
- if allow_star and value == '**':
- return
-
# Check if array type for value is okay
if isinstance(value, list):
if not allow_array:
@@ -451,10 +448,6 @@ def check_type(expr_info, source, value, allow_array = False,
# Check if type name for value is okay
if isinstance(value, str):
- if value == '**':
- raise QAPIExprError(expr_info,
- "%s uses '**' but did not request 'gen':false"
- % source)
if not value in all_names:
raise QAPIExprError(expr_info,
"%s uses unknown type '%s'"
@@ -478,7 +471,7 @@ def check_type(expr_info, source, value, allow_array = False,
# Todo: allow dictionaries to represent default values of
# an optional argument.
check_type(expr_info, "Member '%s' of %s" % (key, source), arg,
- allow_array=True, allow_star=allow_star,
+ allow_array=True,
allow_metas=['built-in', 'union', 'alternate', 'struct',
'enum'])
@@ -498,18 +491,16 @@ def check_member_clash(expr_info, base_name, data, source = ""):
def check_command(expr, expr_info):
name = expr['command']
- allow_star = expr.has_key('gen')
check_type(expr_info, "'data' for command '%s'" % name,
expr.get('data'), allow_dict=True, allow_optional=True,
- allow_metas=['struct'], allow_star=allow_star)
+ allow_metas=['struct'])
returns_meta = ['union', 'struct']
if name in returns_whitelist:
returns_meta += ['built-in', 'alternate', 'enum']
check_type(expr_info, "'returns' for command '%s'" % name,
expr.get('returns'), allow_array=True,
- allow_optional=True, allow_metas=returns_meta,
- allow_star=allow_star)
+ allow_optional=True, allow_metas=returns_meta)
def check_event(expr, expr_info):
global events
@@ -1059,7 +1050,6 @@ class QAPISchema(object):
('bool', 'boolean', 'bool', 'false'),
('any', 'value', 'QObject' + pointer_suffix , 'NULL')]:
self._def_builtin_type(*t)
- self.entity_dict['**'] = self.lookup_type('any') # TODO drop this alias
def _make_implicit_enum_type(self, name, values):
name = name + 'Kind'
@@ -1208,8 +1198,6 @@ class QAPISchema(object):
def visit(self, visitor):
visitor.visit_begin()
for name in sorted(self.entity_dict.keys()):
- if self.entity_dict[name].name != name:
- continue # ignore alias TODO drop alias and remove
self.entity_dict[name].visit(visitor)
visitor.visit_end()
diff --git a/tests/Makefile b/tests/Makefile
index b578d80..badf915 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -228,7 +228,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
bad-type-dict.json double-data.json unknown-expr-key.json \
redefined-type.json redefined-command.json redefined-builtin.json \
redefined-event.json command-int.json bad-data.json event-max.json \
- type-bypass.json type-bypass-no-gen.json type-bypass-bad-gen.json \
+ type-bypass-bad-gen.json \
args-array-empty.json args-array-unknown.json args-int.json \
args-unknown.json args-member-unknown.json args-member-array.json \
args-member-array-bad.json args-alternate.json args-union.json \
diff --git a/tests/qapi-schema/type-bypass-no-gen.err b/tests/qapi-schema/type-bypass-no-gen.err
deleted file mode 100644
index 20cef0a..0000000
--- a/tests/qapi-schema/type-bypass-no-gen.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/type-bypass-no-gen.json:2: Member 'arg' of 'data' for command 'unsafe' uses '**' but did not request 'gen':false
diff --git a/tests/qapi-schema/type-bypass-no-gen.exit b/tests/qapi-schema/type-bypass-no-gen.exit
deleted file mode 100644
index d00491f..0000000
--- a/tests/qapi-schema/type-bypass-no-gen.exit
+++ /dev/null
@@ -1 +0,0 @@
-1
diff --git a/tests/qapi-schema/type-bypass-no-gen.json b/tests/qapi-schema/type-bypass-no-gen.json
deleted file mode 100644
index 4feae37..0000000
--- a/tests/qapi-schema/type-bypass-no-gen.json
+++ /dev/null
@@ -1,2 +0,0 @@
-# type bypass only works with 'gen':false
-{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**' }
diff --git a/tests/qapi-schema/type-bypass-no-gen.out b/tests/qapi-schema/type-bypass-no-gen.out
deleted file mode 100644
index e69de29..0000000
diff --git a/tests/qapi-schema/type-bypass.err b/tests/qapi-schema/type-bypass.err
deleted file mode 100644
index e69de29..0000000
diff --git a/tests/qapi-schema/type-bypass.exit b/tests/qapi-schema/type-bypass.exit
deleted file mode 100644
index 573541a..0000000
--- a/tests/qapi-schema/type-bypass.exit
+++ /dev/null
@@ -1 +0,0 @@
-0
diff --git a/tests/qapi-schema/type-bypass.json b/tests/qapi-schema/type-bypass.json
deleted file mode 100644
index 48b2137..0000000
--- a/tests/qapi-schema/type-bypass.json
+++ /dev/null
@@ -1,2 +0,0 @@
-# Use of 'gen':false allows bypassing type system
-{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**', 'gen': false }
diff --git a/tests/qapi-schema/type-bypass.out b/tests/qapi-schema/type-bypass.out
deleted file mode 100644
index db2a4e6..0000000
--- a/tests/qapi-schema/type-bypass.out
+++ /dev/null
@@ -1,4 +0,0 @@
-object :obj-unsafe-arg
- member arg: any optional=False
-command unsafe :obj-unsafe-arg -> any
- gen=False success_response=True
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (28 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 29/32] qapi: Pseudo-type '**' is now unused, drop it Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-05 20:20 ` Eric Blake
` (3 more replies)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 31/32] qapi-introspect: Map all integer types to 'int' Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names Markus Armbruster
31 siblings, 4 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
Caution, rough edges.
qapi/introspect.json defines the introspection schema. It should do
for uses other than QMP.
FIXME it's almost entirely devoid of comments.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details:
* The built-in types are declared with their JSON type.
TODO Should we map all the integer types to just int?
* Implicit type definitions are made explicit, and given
auto-generated names. These names start with ':' so they don't
clash with the user's names.
Example: a simple union implicitly defines an enumeration type for
its discriminator.
* All type references are by name.
* Base types are flattened.
* The top type (named 'any') can hold any value.
* The struct and union types are generalized into an object type.
* Commands take a single argument and return a single result.
Dictionary argument/result or list result is an implicit type
definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is ommitted as well for now, even
though it's not an implementation detail.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default.
Alternate members can't have defaults, but the introspection schema
doesn't reflect that.
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
TODO much of the above should go into docs.
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
FIXME it can generate awfully long lines
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-schema takes its return value from that
variable. Command documentation is incomplete, and marked FIXME. Its
reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-schema-hash. Clients can have a cache indexed by hash, and
re-query the schema only when they don't have it cached.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
.gitignore | 1 +
Makefile | 9 +-
Makefile.objs | 4 +-
docs/qapi-code-gen.txt | 34 +++++
monitor.c | 15 +++
qapi-schema.json | 3 +
qapi/introspect.json | 71 ++++++++++
qmp-commands.hx | 16 +++
scripts/qapi-commands.py | 2 +-
scripts/qapi-event.py | 2 +-
scripts/qapi-introspect.py | 172 ++++++++++++++++++++++++
scripts/qapi-types.py | 2 +-
scripts/qapi-visit.py | 2 +-
scripts/qapi.py | 14 +-
tests/.gitignore | 1 +
tests/Makefile | 10 +-
tests/qapi-schema/alternate-good.out | 1 +
tests/qapi-schema/args-member-array.out | 1 +
| 1 +
tests/qapi-schema/empty.out | 1 +
tests/qapi-schema/enum-empty.out | 1 +
tests/qapi-schema/event-case.out | 1 +
tests/qapi-schema/flat-union-reverse-define.out | 1 +
tests/qapi-schema/ident-with-escape.out | 1 +
tests/qapi-schema/include-relpath.out | 1 +
tests/qapi-schema/include-repetition.out | 1 +
tests/qapi-schema/include-simple.out | 1 +
tests/qapi-schema/indented-expr.out | 1 +
tests/qapi-schema/qapi-schema-test.out | 1 +
tests/qapi-schema/returns-int.out | 1 +
tests/test-qmp-input-strict.c | 55 ++++++++
31 files changed, 416 insertions(+), 11 deletions(-)
create mode 100644 qapi/introspect.json
create mode 100644 scripts/qapi-introspect.py
diff --git a/.gitignore b/.gitignore
index 61bc492..72c6170 100644
--- a/.gitignore
+++ b/.gitignore
@@ -34,6 +34,7 @@
/qapi-visit.[ch]
/qapi-event.[ch]
/qmp-commands.h
+/qmp-introspect.[ch]
/qmp-marshal.c
/qemu-doc.html
/qemu-tech.html
diff --git a/Makefile b/Makefile
index 340d9c8..6140e06 100644
--- a/Makefile
+++ b/Makefile
@@ -52,6 +52,8 @@ endif
GENERATED_HEADERS = config-host.h qemu-options.def
GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h
GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
+GENERATED_HEADERS += qmp-introspect.h
+GENERATED_SOURCES += qmp-introspect.c
GENERATED_HEADERS += trace/generated-events.h
GENERATED_SOURCES += trace/generated-events.c
@@ -263,7 +265,7 @@ $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
$(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
- $(SRC_PATH)/qapi/event.json
+ $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json
qapi-types.c qapi-types.h :\
$(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
@@ -285,6 +287,11 @@ $(qapi-modules) $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py \
$(gen-out-type) -o "." -m $<, \
" GEN $@")
+qmp-introspect.h qmp-introspect.c :\
+$(qapi-modules) $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \
+ $(gen-out-type) -o "." $<, \
+ " GEN $@")
QGALIB_GEN=$(addprefix qga/qapi-generated/, qga-qapi-types.h qga-qapi-visit.h qga-qmp-commands.h)
$(qga-obj-y) qemu-ga.o: $(QGALIB_GEN)
diff --git a/Makefile.objs b/Makefile.objs
index f094eff..b5a9e99 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -1,7 +1,8 @@
#######################################################################
# Common libraries for tools and emulators
stub-obj-y = stubs/
-util-obj-y = util/ qobject/ qapi/ qapi-types.o qapi-visit.o qapi-event.o
+util-obj-y = util/ qobject/ qapi/
+util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
util-obj-y += crypto/
#######################################################################
@@ -83,6 +84,7 @@ common-obj-$(CONFIG_FDT) += device_tree.o
# qapi
common-obj-y += qmp-marshal.o
+common-obj-y += qmp-introspect.o
common-obj-y += qmp.o hmp.o
endif
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 53e6d9c..ed04770 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -850,3 +850,37 @@ Example:
extern const char *const example_QAPIEvent_lookup[];
#endif
+
+=== scripts/qapi-introspect.py ===
+
+Used to generate the introspection C code for a schema. The following
+files are created:
+
+$(prefix)qmp-introspect.c - Defines a string holding a JSON
+ description of the schema.
+$(prefix)qmp-introspect.h - Declares the above string.
+
+Example:
+
+ $ python scripts/qapi-event.py --output-dir="qapi-generated"
+ --prefix="example-" example-schema.json
+ $ cat qapi-generated/example-qmp-introspect.c
+[Uninteresting stuff omitted...]
+
+ const char example_qmp_schema_json[] = "["
+ "{ \"arg-type\": \":empty\", \"name\": \"MY_EVENT\", \"meta-type\": \"event\" }, "
+ "{ \"meta-type\": \"builtin\", \"name\": \"int\", \"json-type\": \"int\" }, "
+ "{ \"meta-type\": \"builtin\", \"name\": \"str\", \"json-type\": \"string\" }, "
+ "{ \"meta-type\": \"object\", \"name\": \":empty\", \"members\": [ ] }, "
+ "{ \"meta-type\": \"object\", \"name\": \":obj-my-command-arg\", \"members\": [{ \"type\": \"UserDefOne\", \"name\": \"arg1\" } ] }, "
+ "{ \"meta-type\": \"object\", \"name\": \"UserDefOne\", \"members\": [{ \"type\": \"int\", \"name\": \"integer\" }, { \"type\": \"str\", \"name\": \"string\" } ] }, "
+ "{ \"arg-type\": \":obj-my-command-arg\", \"ret-type\": \"UserDefOne\", \"name\": \"my-command\", \"meta-type\": \"command\" } ]";
+ $ cat qapi-generated/example-qmp-introspect.h
+[Uninteresting stuff omitted...]
+
+ #ifndef EXAMPLE_QMP_INTROSPECT_H
+ #define EXAMPLE_QMP_INTROSPECT_H
+
+ extern const char example_qmp_schema_json[];
+
+ #endif
diff --git a/monitor.c b/monitor.c
index 16672f1..f49f292 100644
--- a/monitor.c
+++ b/monitor.c
@@ -74,6 +74,7 @@
#include "block/qapi.h"
#include "qapi/qmp-event.h"
#include "qapi-event.h"
+#include "qmp-introspect.h"
#include "sysemu/block-backend.h"
/* for hmp_info_irq/pic */
@@ -924,6 +925,20 @@ EventInfoList *qmp_query_events(Error **errp)
return ev_list;
}
+/*
+ * Minor hack: generated marshalling suppressed for this command
+ * ('gen': false in the schema) so we can parse the JSON string
+ * directly into QObject instead of first parsing it with
+ * visit_type_SchemaInfoList() into a SchemaInfoList, then marshal it
+ * to QObject with generated output marshallers, every time. Instead,
+ * we do it in test-qmp-input-visitor.c, just to make sure
+ * qapi-introspect.py's output actually conforms to the schema.
+ */
+static void qmp_query_schema(QDict *qdict, QObject **ret_data, Error **errp)
+{
+ *ret_data = qobject_from_json(qmp_schema_json);
+}
+
/* set the current CPU defined by the user */
int monitor_set_cpu(int cpu_index)
{
diff --git a/qapi-schema.json b/qapi-schema.json
index e91e3b9..af19810 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -14,6 +14,9 @@
# Tracing commands
{ 'include': 'qapi/trace.json' }
+# QAPI introspection
+{ 'include': 'qapi/introspect.json' }
+
##
# @LostTickPolicy:
#
diff --git a/qapi/introspect.json b/qapi/introspect.json
new file mode 100644
index 0000000..2002f8f
--- /dev/null
+++ b/qapi/introspect.json
@@ -0,0 +1,71 @@
+# -*- Mode: Python -*-
+#
+# QAPI introspection
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# Authors:
+# Markus Armbruster <armbru@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+{ 'enum': 'SchemaMetaType',
+ 'data': [ 'builtin', 'enum', 'array', 'object', 'alternate',
+ 'command', 'event' ] }
+
+{ 'struct': 'SchemaInfoBase',
+ 'data': { 'name': 'str', 'meta-type': 'SchemaMetaType' } }
+
+{ 'enum': 'JSONType',
+ 'data': [ 'string', 'number', 'int', 'boolean', 'null',
+ 'object', 'array', 'value' ] }
+
+{ 'struct': 'SchemaInfoBuiltin',
+ 'data': { 'json-type': 'JSONType' } }
+
+{ 'struct': 'SchemaInfoEnum',
+ 'data': { 'values': ['str'] } }
+
+{ 'struct': 'SchemaInfoArray',
+ 'data': { 'element-type': 'str' } }
+
+{ 'struct': 'SchemaInfoObjectMember',
+ 'data': { 'name': 'str', 'type': 'str', '*default': 'any' } }
+# @default's type must be null or match @type
+
+{ 'struct': 'SchemaInfoObjectVariant',
+ 'data': { 'case': 'str', 'type': 'str' } }
+
+{ 'struct': 'SchemaInfoObject',
+ 'data': { 'members': [ 'SchemaInfoObjectMember' ],
+ '*tag': 'str',
+ '*variants': [ 'SchemaInfoObjectVariant' ] } }
+
+{ 'struct': 'SchemaInfoAlternateMember',
+ 'data': { 'type': 'str' } }
+
+{ 'struct': 'SchemaInfoAlternate',
+ 'data': { 'members': [ 'SchemaInfoAlternateMember' ] } }
+
+{ 'struct': 'SchemaInfoCommand',
+ 'data': { 'arg-type': 'str', 'ret-type': 'str' } } # TODO success-response
+
+{ 'struct': 'SchemaInfoEvent',
+ 'data': { 'arg-type': 'str' } }
+
+{ 'union': 'SchemaInfo',
+ 'base': 'SchemaInfoBase',
+ 'discriminator': 'meta-type',
+ 'data': {
+ 'builtin': 'SchemaInfoBuiltin',
+ 'enum': 'SchemaInfoEnum',
+ 'array': 'SchemaInfoArray',
+ 'object': 'SchemaInfoObject',
+ 'alternate': 'SchemaInfoAlternate',
+ 'command': 'SchemaInfoCommand',
+ 'event': 'SchemaInfoEvent' } }
+
+{ 'command': 'query-schema',
+ 'returns': [ 'SchemaInfo' ],
+ 'gen': false } # just to simplify qmp_query_json()
diff --git a/qmp-commands.hx b/qmp-commands.hx
index b06d74c..3ad3b65 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -2168,6 +2168,22 @@ EQMP
},
SQMP
+query-schema
+------------
+
+Return the QMP schema.
+
+FIXME explain
+
+EQMP
+
+ {
+ .name = "query-schema",
+ .args_type = "",
+ .mhandler.cmd_new = qmp_query_schema,
+ },
+
+SQMP
query-chardev
-------------
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 0603cff..7ff7c31 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -260,7 +260,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
self.defn = None
self.regy = None
self.visited_ret_types = None
- def visit_begin(self):
+ def visit_begin(self, schema):
self.decl = ''
self.defn = ''
self.regy = ''
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 10f4bae..d100099 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -139,7 +139,7 @@ class QAPISchemaGenEventVisitor(QAPISchemaVisitor):
self.decl = None
self.defn = None
self.event_names = None
- def visit_begin(self):
+ def visit_begin(self, schema):
self.decl = ''
self.defn = ''
self.event_names = []
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
new file mode 100644
index 0000000..8354d06
--- /dev/null
+++ b/scripts/qapi-introspect.py
@@ -0,0 +1,172 @@
+#
+# QAPI introspection generator
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# Authors:
+# Markus Armbruster <armbru@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2.
+# See the COPYING file in the top-level directory.
+
+from qapi import *
+
+# Caveman's json.dumps() replacement (we're stuck at 2.4)
+# TODO try to use json.dumps() once we get unstuck
+def to_json(obj, level=0):
+ if obj == None:
+ ret = 'null'
+ elif isinstance(obj, str):
+ ret = '"' + obj.replace('"', r'\"') + '"'
+ elif isinstance(obj, list):
+ elts = [to_json(elt, level + 1)
+ for elt in obj]
+ ret = '[' + ', '.join(elts) + ' ]'
+ elif isinstance(obj, dict):
+ elts = ['"%s": %s' % (key, to_json(obj[key], level + 1))
+ for key in sorted(obj.keys())]
+ ret = '{ ' + ', '.join(elts) + ' }'
+ else:
+ assert False # not implemented
+ if level == 1:
+ ret = '\n' + ret
+ return ret
+
+def to_c_string(string):
+ return '"' + string.replace('\\', r'\\').replace('"', r'\"') + '"'
+
+class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor):
+ def __init__(self):
+ self.defn = None
+ self.decl = None
+ self.schema = None
+ self.jsons = None
+ self.used_types = None
+
+ def visit_begin(self, schema):
+ self.schema = schema
+ self.jsons = []
+ self.used_types = []
+ return QAPISchemaType # don't visit types for now
+
+ def visit_end(self):
+ # visit the types that are actually used
+ for typ in self.used_types:
+ typ.visit(self)
+ self.jsons.sort()
+ # generate C
+ name = prefix + 'qmp_schema_json'
+ self.decl = mcgen('''
+extern const char %(c_name)s[];
+''',
+ c_name=c_name(name))
+ lines = to_json(self.jsons).split('\n')
+ c_string = '\n '.join([to_c_string(line) for line in lines])
+ self.defn = mcgen('''
+const char %(c_name)s[] = %(c_string)s;
+''',
+ c_name=c_name(name),
+ c_string=c_string)
+ self.schema = None
+ self.jsons = None
+ self.used_types = None
+
+ def _use_type(self, typ):
+ if typ not in self.used_types:
+ self.used_types.append(typ)
+ return typ.name
+
+ def _gen_json(self, name, mtype, obj={}):
+ obj['name'] = name
+ obj['meta-type'] = mtype
+ self.jsons.append(obj)
+
+ def _gen_member(self, member):
+ ret = { 'name': member.name, 'type': self._use_type(member.type) }
+ if member.optional:
+ ret['default'] = None
+ return ret
+
+ def _gen_variants(self, tag_name, variants):
+ return { 'tag': tag_name or 'kind',
+ 'variants': [self._gen_variant(v) for v in variants] }
+
+ def _gen_variant(self, variant):
+ return { 'case': variant.name, 'type': self._use_type(variant.type) }
+
+ def visit_builtin_type(self, name, info, json_type):
+ self._gen_json(name, 'builtin', { 'json-type': json_type })
+
+ def visit_enum_type(self, name, info, values):
+ self._gen_json(name, 'enum', { 'values': values })
+
+ def visit_array_type(self, name, info, element_type):
+ self._gen_json(name, 'array',
+ { 'element-type': self._use_type(element_type) })
+
+ def visit_object_type_flat(self, name, info, members, variants):
+ obj = { 'members': [self._gen_member(m) for m in members] }
+ if variants:
+ obj.update(self._gen_variants(variants.tag_name,
+ variants.variants))
+ self._gen_json(name, 'object', obj)
+
+ def visit_alternate_type(self, name, info, variants):
+ self._gen_json(name, 'alternate',
+ { 'members': [ { 'type': self._use_type(m.type) }
+ for m in variants.variants] })
+
+ def visit_command(self, name, info, arg_type, ret_type,
+ gen, success_response):
+ arg_type = arg_type or self.schema.the_empty_object_type
+ ret_type = ret_type or self.schema.the_empty_object_type
+ self._gen_json(name, 'command',
+ { 'arg-type': self._use_type(arg_type),
+ 'ret-type': self._use_type(ret_type) })
+
+ def visit_event(self, name, info, arg_type):
+ arg_type = arg_type or self.schema.the_empty_object_type
+ self._gen_json(name, 'event', { 'arg-type': self._use_type(arg_type) })
+
+(input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
+
+c_comment = '''
+/*
+ * QAPI/QMP schema introspection
+ *
+ * Copyright (C) 2015 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+'''
+h_comment = '''
+/*
+ * QAPI/QMP schema introspection
+ *
+ * Copyright (C) 2015 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+'''
+
+(fdef, fdecl) = open_output(output_dir, do_c, do_h, prefix,
+ 'qmp-introspect.c', 'qmp-introspect.h',
+ c_comment, h_comment)
+
+fdef.write(mcgen('''
+#include "%(prefix)sqmp-introspect.h"
+
+''',
+ prefix=prefix))
+
+schema = QAPISchema(input_file)
+gen = QAPISchemaGenIntrospectVisitor()
+schema.visit(gen)
+fdef.write(gen.defn)
+fdecl.write(gen.decl)
+
+close_output(fdef, fdecl)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 5b6de4c..9caf7bd 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -197,7 +197,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
self.fwdecl = None
self.fwdefn = None
self.btin = None
- def visit_begin(self):
+ def visit_begin(self, schema):
self.decl = ''
self.defn = ''
self.fwdecl = ''
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 9cfd2ff..6705eaf 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -334,7 +334,7 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
self.decl = None
self.defn = None
self.btin = None
- def visit_begin(self):
+ def visit_begin(self, schema):
self.decl = ''
self.defn = ''
self.btin = guardstart('QAPI_VISIT_BUILTIN')
diff --git a/scripts/qapi.py b/scripts/qapi.py
index fd8abad..34c2884 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -768,7 +768,7 @@ class QAPISchemaEntity(object):
pass
class QAPISchemaVisitor(object):
- def visit_begin(self):
+ def visit_begin(self, schema):
pass
def visit_end(self):
pass
@@ -780,6 +780,8 @@ class QAPISchemaVisitor(object):
pass
def visit_object_type(self, name, info, base, members, variants):
pass
+ def visit_object_type_flat(self, name, info, members, variants):
+ pass
def visit_alternate_type(self, name, info, variants):
pass
def visit_command(self, name, info, arg_type, ret_type,
@@ -901,6 +903,8 @@ class QAPISchemaObjectType(QAPISchemaType):
def visit(self, visitor):
visitor.visit_object_type(self.name, self.info,
self.base, self.local_members, self.variants)
+ visitor.visit_object_type_flat(self.name, self.info,
+ self.members, self.variants)
class QAPISchemaObjectTypeMember(object):
def __init__(self, name, typ, optional):
@@ -1050,6 +1054,9 @@ class QAPISchema(object):
('bool', 'boolean', 'bool', 'false'),
('any', 'value', 'QObject' + pointer_suffix , 'NULL')]:
self._def_builtin_type(*t)
+ self.the_empty_object_type = QAPISchemaObjectType(':empty', None, None,
+ [], None)
+ self._def_entity(self.the_empty_object_type)
def _make_implicit_enum_type(self, name, values):
name = name + 'Kind'
@@ -1196,9 +1203,10 @@ class QAPISchema(object):
ent.check(self)
def visit(self, visitor):
- visitor.visit_begin()
+ ignore = visitor.visit_begin(self)
for name in sorted(self.entity_dict.keys()):
- self.entity_dict[name].visit(visitor)
+ if not ignore or not isinstance(self.entity_dict[name], ignore):
+ self.entity_dict[name].visit(visitor)
visitor.visit_end()
#
diff --git a/tests/.gitignore b/tests/.gitignore
index ccc92e4..e476d52 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -28,6 +28,7 @@ test-qmp-commands.h
test-qmp-event
test-qmp-input-strict
test-qmp-input-visitor
+test-qmp-introspect.[ch]
test-qmp-marshal.c
test-qmp-output-visitor
test-rcu-list
diff --git a/tests/Makefile b/tests/Makefile
index badf915..398dc4a 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -260,7 +260,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
struct-base-clash.json struct-base-clash-deep.json )
GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
- tests/test-qmp-commands.h tests/test-qapi-event.h
+ tests/test-qmp-commands.h tests/test-qapi-event.h \
+ tests/test-qmp-introspect.h
test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qfloat.o tests/check-qjson.o \
@@ -273,7 +274,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/rcutorture.o tests/test-rcu-list.o
test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
- tests/test-qapi-event.o
+ tests/test-qapi-event.o tests/test-qmp-introspect.o
$(test-obj-y): QEMU_INCLUDES += -Itests
QEMU_CFLAGS += -I$(SRC_PATH)/tests
@@ -334,6 +335,11 @@ $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-eve
$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py \
$(gen-out-type) -o tests -p "test-" $<, \
" GEN $@")
+tests/test-qmp-introspect.c tests/test-qmp-introspect.h :\
+$(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-introspect.py $(qapi-py)
+ $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-introspect.py \
+ $(gen-out-type) -o tests -p "test-" $<, \
+ " GEN $@")
tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) libqemuutil.a libqemustub.a
diff --git a/tests/qapi-schema/alternate-good.out b/tests/qapi-schema/alternate-good.out
index 3d765ff..65af727 100644
--- a/tests/qapi-schema/alternate-good.out
+++ b/tests/qapi-schema/alternate-good.out
@@ -1,3 +1,4 @@
+object :empty
alternate Alt
case value: int
case string: Enum
diff --git a/tests/qapi-schema/args-member-array.out b/tests/qapi-schema/args-member-array.out
index b67384c..b3b92df 100644
--- a/tests/qapi-schema/args-member-array.out
+++ b/tests/qapi-schema/args-member-array.out
@@ -1,3 +1,4 @@
+object :empty
object :obj-okay-arg
member member1: intList optional=False
member member2: defList optional=False
--git a/tests/qapi-schema/comments.out b/tests/qapi-schema/comments.out
index 6161b90..9e2c656 100644
--- a/tests/qapi-schema/comments.out
+++ b/tests/qapi-schema/comments.out
@@ -1 +1,2 @@
+object :empty
enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/empty.out b/tests/qapi-schema/empty.out
index e69de29..272b161 100644
--- a/tests/qapi-schema/empty.out
+++ b/tests/qapi-schema/empty.out
@@ -0,0 +1 @@
+object :empty
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
index e09b00f..a449d45 100644
--- a/tests/qapi-schema/enum-empty.out
+++ b/tests/qapi-schema/enum-empty.out
@@ -1 +1,2 @@
+object :empty
enum MyEnum []
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index b5ae4c2..cdfd264 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1 +1,2 @@
+object :empty
event oops None
diff --git a/tests/qapi-schema/flat-union-reverse-define.out b/tests/qapi-schema/flat-union-reverse-define.out
index 477fb31..a5a9134 100644
--- a/tests/qapi-schema/flat-union-reverse-define.out
+++ b/tests/qapi-schema/flat-union-reverse-define.out
@@ -1,3 +1,4 @@
+object :empty
object TestBase
member enum1: TestEnum optional=False
enum TestEnum ['value1', 'value2']
diff --git a/tests/qapi-schema/ident-with-escape.out b/tests/qapi-schema/ident-with-escape.out
index 9577d1b..f4542b1 100644
--- a/tests/qapi-schema/ident-with-escape.out
+++ b/tests/qapi-schema/ident-with-escape.out
@@ -1,3 +1,4 @@
+object :empty
object :obj-fooA-arg
member bar1: str optional=False
command fooA :obj-fooA-arg -> None
diff --git a/tests/qapi-schema/include-relpath.out b/tests/qapi-schema/include-relpath.out
index 6161b90..9e2c656 100644
--- a/tests/qapi-schema/include-relpath.out
+++ b/tests/qapi-schema/include-relpath.out
@@ -1 +1,2 @@
+object :empty
enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/include-repetition.out b/tests/qapi-schema/include-repetition.out
index 6161b90..9e2c656 100644
--- a/tests/qapi-schema/include-repetition.out
+++ b/tests/qapi-schema/include-repetition.out
@@ -1 +1,2 @@
+object :empty
enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/include-simple.out b/tests/qapi-schema/include-simple.out
index 6161b90..9e2c656 100644
--- a/tests/qapi-schema/include-simple.out
+++ b/tests/qapi-schema/include-simple.out
@@ -1 +1,2 @@
+object :empty
enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/indented-expr.out b/tests/qapi-schema/indented-expr.out
index c5af55a..226d300 100644
--- a/tests/qapi-schema/indented-expr.out
+++ b/tests/qapi-schema/indented-expr.out
@@ -1,3 +1,4 @@
+object :empty
command eins None -> None
gen=True success_response=True
command zwei None -> None
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 49c1197..386b057 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -1,3 +1,4 @@
+object :empty
object :obj-EVENT_C-arg
member a: int optional=True
member b: UserDefOne optional=True
diff --git a/tests/qapi-schema/returns-int.out b/tests/qapi-schema/returns-int.out
index 1ac3e1e..a2da259 100644
--- a/tests/qapi-schema/returns-int.out
+++ b/tests/qapi-schema/returns-int.out
@@ -1,2 +1,3 @@
+object :empty
command guest-get-time None -> int
gen=True success_response=True
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 00c3e29..bfd9d04 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -19,6 +19,9 @@
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
#include "qapi/qmp/types.h"
+#include "test-qmp-introspect.h"
+#include "qmp-introspect.h"
+#include "qapi-visit.h"
typedef struct TestInputVisitorData {
QObject *obj;
@@ -62,6 +65,30 @@ Visitor *validate_test_init(TestInputVisitorData *data,
return v;
}
+/* similar to validate_test_init(), but does not expect a string
+ * literal/format json_string argument and so can be used for
+ * programatically generated strings (and we can't pass in programatically
+ * generated strings via %s format parameters since qobject_from_jsonv()
+ * will wrap those in double-quotes and treat the entire object as a
+ * string)
+ */
+static Visitor *validate_test_init_raw(TestInputVisitorData *data,
+ const char *json_string)
+{
+ Visitor *v;
+
+ data->obj = qobject_from_json(json_string);
+ g_assert(data->obj != NULL);
+
+ data->qiv = qmp_input_visitor_new_strict(data->obj);
+ g_assert(data->qiv != NULL);
+
+ v = qmp_input_get_visitor(data->qiv);
+ g_assert(v != NULL);
+
+ return v;
+}
+
typedef struct TestStruct
{
int64_t integer;
@@ -293,6 +320,32 @@ static void test_validate_fail_alternate(TestInputVisitorData *data,
qapi_free_UserDefAlternate(tmp);
}
+static void do_test_validate_qmp_introspect(TestInputVisitorData *data,
+ const char *schema_json)
+{
+ SchemaInfoList *schema = NULL;
+ Error *err = NULL;
+ Visitor *v;
+
+ v = validate_test_init_raw(data, schema_json);
+
+ visit_type_SchemaInfoList(v, &schema, NULL, &err);
+ if (err) {
+ fprintf(stderr, "%s", error_get_pretty(err));
+ }
+ g_assert(!err);
+ g_assert(schema);
+
+ qapi_free_SchemaInfoList(schema);
+}
+
+static void test_validate_qmp_introspect(TestInputVisitorData *data,
+ const void *unused)
+{
+ do_test_validate_qmp_introspect(data, test_qmp_schema_json);
+ do_test_validate_qmp_introspect(data, qmp_schema_json);
+}
+
static void validate_test_add(const char *testpath,
TestInputVisitorData *data,
void (*test_func)(TestInputVisitorData *data, const void *user_data))
@@ -333,6 +386,8 @@ int main(int argc, char **argv)
&testdata, test_validate_fail_alternate);
validate_test_add("/visitor/input-strict/fail/union-native-list",
&testdata, test_validate_fail_union_native_list);
+ validate_test_add("/visitor/input-strict/pass/qmp-introspect",
+ &testdata, test_validate_qmp_introspect);
g_test_run();
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 31/32] qapi-introspect: Map all integer types to 'int'
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (29 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names Markus Armbruster
31 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
How many bits we use internally is an implementation detail. It could
be pressed into external interface service as a very approximate range
information, but that's probably a bad idea. If we need range
information, we better do it properly.
Reduces output of query-schema by a negligible 0.5 out of 85KiB.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-introspect.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
index 8354d06..05d30e5 100644
--- a/scripts/qapi-introspect.py
+++ b/scripts/qapi-introspect.py
@@ -72,6 +72,13 @@ const char %(c_name)s[] = %(c_string)s;
self.used_types = None
def _use_type(self, typ):
+ # Map the various integer types to plain int
+ if typ.json_type() == 'int':
+ typ = self.schema.lookup_type('int')
+ elif isinstance(typ, QAPISchemaArrayType) \
+ and typ.element_type.json_type() == 'int':
+ typ = self.schema.lookup_type('intList')
+ # Add type to work queue if new
if typ not in self.used_types:
self.used_types.append(typ)
return typ.name
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
` (30 preceding siblings ...)
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 31/32] qapi-introspect: Map all integer types to 'int' Markus Armbruster
@ 2015-08-04 15:58 ` Markus Armbruster
2015-08-05 21:06 ` Eric Blake
31 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-04 15:58 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, berto, mdroth
To eliminate the temptation for clients to look up types by name
(which are not ABI), replace all type names by meaningless strings.
Reduces output of query-schema by 13 out of 85KiB.
TODO Either generate comments with the true type names, or provide an
option to generate without type name hiding.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
docs/qapi-code-gen.txt | 14 +++++++-------
scripts/qapi-introspect.py | 24 +++++++++++++++++++++---
2 files changed, 28 insertions(+), 10 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index ed04770..3a78cf4 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -868,13 +868,13 @@ Example:
[Uninteresting stuff omitted...]
const char example_qmp_schema_json[] = "["
- "{ \"arg-type\": \":empty\", \"name\": \"MY_EVENT\", \"meta-type\": \"event\" }, "
- "{ \"meta-type\": \"builtin\", \"name\": \"int\", \"json-type\": \"int\" }, "
- "{ \"meta-type\": \"builtin\", \"name\": \"str\", \"json-type\": \"string\" }, "
- "{ \"meta-type\": \"object\", \"name\": \":empty\", \"members\": [ ] }, "
- "{ \"meta-type\": \"object\", \"name\": \":obj-my-command-arg\", \"members\": [{ \"type\": \"UserDefOne\", \"name\": \"arg1\" } ] }, "
- "{ \"meta-type\": \"object\", \"name\": \"UserDefOne\", \"members\": [{ \"type\": \"int\", \"name\": \"integer\" }, { \"type\": \"str\", \"name\": \"string\" } ] }, "
- "{ \"arg-type\": \":obj-my-command-arg\", \"ret-type\": \"UserDefOne\", \"name\": \"my-command\", \"meta-type\": \"command\" } ]";
+ "{ \"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": \"MY_EVENT\" }, "
+ "{ \"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": \"my-command\", \"ret-type\": \"2\" }, "
+ "{ \"members\": [ ], \"meta-type\": \"object\", \"name\": \"0\" }, "
+ "{ \"members\": [{ \"name\": \"arg1\", \"type\": \"2\" } ], \"meta-type\": \"object\", \"name\": \"1\" }, "
+ "{ \"members\": [{ \"name\": \"integer\", \"type\": \"int\" }, { \"name\": \"string\", \"type\": \"str\" } ], \"meta-type\": \"object\", \"name\": \"2\" }, "
+ "{ \"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": \"int\" }, "
+ "{ \"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": \"str\" } ]";
$ cat qapi-generated/example-qmp-introspect.h
[Uninteresting stuff omitted...]
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
index 05d30e5..5e99d4b 100644
--- a/scripts/qapi-introspect.py
+++ b/scripts/qapi-introspect.py
@@ -42,25 +42,29 @@ class QAPISchemaGenIntrospectVisitor(QAPISchemaVisitor):
self.schema = None
self.jsons = None
self.used_types = None
+ self.name_map = None
def visit_begin(self, schema):
self.schema = schema
self.jsons = []
self.used_types = []
+ self.name_map = {}
return QAPISchemaType # don't visit types for now
def visit_end(self):
# visit the types that are actually used
+ jsons = self.jsons
+ self.jsons = []
for typ in self.used_types:
typ.visit(self)
- self.jsons.sort()
# generate C
+ jsons.extend(self.jsons)
name = prefix + 'qmp_schema_json'
self.decl = mcgen('''
extern const char %(c_name)s[];
''',
c_name=c_name(name))
- lines = to_json(self.jsons).split('\n')
+ lines = to_json(jsons).split('\n')
c_string = '\n '.join([to_c_string(line) for line in lines])
self.defn = mcgen('''
const char %(c_name)s[] = %(c_string)s;
@@ -70,6 +74,13 @@ const char %(c_name)s[] = %(c_string)s;
self.schema = None
self.jsons = None
self.used_types = None
+ self.name_map = None
+
+ def _name(self, name):
+ if name not in self.name_map:
+ n = len(self.name_map)
+ self.name_map[name] = '%s' % n
+ return self.name_map[name]
def _use_type(self, typ):
# Map the various integer types to plain int
@@ -81,9 +92,16 @@ const char %(c_name)s[] = %(c_string)s;
# Add type to work queue if new
if typ not in self.used_types:
self.used_types.append(typ)
- return typ.name
+ # Clients should examine commands and events, not types. Hide
+ # type names to reduce the temptation. Also saves a few
+ # characters.
+ if isinstance(typ, QAPISchemaBuiltinType):
+ return typ.name
+ return self._name(typ.name)
def _gen_json(self, name, mtype, obj={}):
+ if mtype != 'command' and mtype != 'event' and mtype != 'builtin':
+ name = self._name(name)
obj['name'] = name
obj['meta-type'] = mtype
self.jsons.append(obj)
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation Markus Armbruster
@ 2015-08-04 21:53 ` Eric Blake
2015-08-05 6:23 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-04 21:53 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 8332 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> The QAPI code generators work with a syntax tree (nested dictionaries)
> plus a few symbol tables (also dictionaries) on the side.
>
> They have clearly outgrown these simple data structures. There's lots
> of rummaging around in dictionaries, and information is recomputed on
> the fly. For the work I'm going to do, I want more clearly defined
> and more convenient interfaces.
>
> Going forward, I also want less coupling between the back-ends and the
> syntax tree, to make messing with the syntax easier.
>
> Create a bunch of classes to represent QAPI schemata.
>
> Have the QAPISchema initializer call the parser, then walk the syntax
> tree to create the new internal representation, and finally perform
> semantic analysis.
>
> Shortcut: the semantic analysis still relies on existing check_exprs()
> to do the actual semantic checking. All this code needs to move into
> the classes. Mark as TODO.
>
> We generate array types eagerly, even though most of them aren't used.
> Mark as TODO.
I'm not sure if there are any array types that the rest of the code base
uses even though it doesn't appear as a directly used type within the
schemata. Perhaps some of the builtin types (for example, if qom-get
needs to return ['uint8']). Besides builtins, maybe we can add some sort
of 'needs-array':'bool' key to each 'struct'/'union'/'enum'/'alternate',
which defaults to true only if the schema refers to the array type, but
which can be explicitly set to true to force the array type generation
even without a schema reference. But as it is all properly marked TODO,
the idle ramblings in this paragraph don't affect review.
>
> Nothing uses the new intermediate representation just yet, thus no
> change to generated files.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi-commands.py | 2 +-
> scripts/qapi-event.py | 2 +-
> scripts/qapi-types.py | 2 +-
> scripts/qapi-visit.py | 2 +-
> scripts/qapi.py | 361 ++++++++++++++++++++++++++++++++++++++++-
> tests/qapi-schema/test-qapi.py | 2 +-
> 6 files changed, 357 insertions(+), 14 deletions(-)
>
> +class QAPISchemaEnumType(QAPISchemaType):
> + def __init__(self, name, info, values):
> + QAPISchemaType.__init__(self, name, info)
> + for v in values:
> + assert isinstance(v, str)
> + self.values = values
> + def check(self, schema):
> + assert len(set(self.values)) == len(self.values)
Doesn't check whether any of the distinct values map to the same C name.
But not a show-stopper to this patch (the earlier semantic checking in
check_exprs() covers it, and your TODO about moving those checks here at
a later date is the right time to worry about it here).
> +
> +class QAPISchemaArrayType(QAPISchemaType):
> + def __init__(self, name, info, element_type):
> + QAPISchemaType.__init__(self, name, info)
> + assert isinstance(element_type, str)
> + self.element_type_name = element_type
> + self.element_type = None
> + def check(self, schema):
> + self.element_type = schema.lookup_type(self.element_type_name)
> + assert self.element_type
Is it worth adding:
assert not isinstance(self.element_type, QAPISchemaArrayType)
since we don't allow 2D arrays?
> +
> +class QAPISchemaObjectType(QAPISchemaType):
> + def __init__(self, name, info, base, local_members, variants):
> + QAPISchemaType.__init__(self, name, info)
> + assert base == None or isinstance(base, str)
> + for m in local_members:
> + assert isinstance(m, QAPISchemaObjectTypeMember)
> + if variants != None:
> + assert isinstance(variants, QAPISchemaObjectTypeVariants)
Style nit: the 'base' and 'variants' checks are identical patterns
(checking for None or specific type), but only one uses an 'if'.
Possibly because of line-length issues, though, so I can live with it.
> +
> +class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> + def __init__(self, name, typ):
> + QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
> + def check(self, schema, tag_type, seen):
> + QAPISchemaObjectTypeMember.check(self, schema, [], seen)
> + assert self.name in tag_type.values
> + # TODO try to get rid of .simple_union_type()
> + def simple_union_type(self):
> + if isinstance(self.type, QAPISchemaObjectType) and not self.type.info:
> + assert len(self.type.members) == 1
> + assert not self.type.variants # not implemented
and probably never will be (looks like this assert is copy-and-pasted
from other locations where it makes sense that we might implement
support for variants, but I don't see it ever happening for the
generated ':obj-*-wrapper' type for the branch of a simple union)
At any rate, I concur that we have a difference in the generated code
for simple unions compared to flat unions (even if simple unions could
be rewritten in terms of a flat union from the QMP side, the generated C
side is not the same), so I agree that you need this function for now,
as well as the comment for if we later try to clean up the code base to
drop that difference.
> +class QAPISchema(object):
> + def _make_array_type(self, element_type):
> + name = element_type + 'List'
> + if not self.lookup_type(name):
> + self._def_entity(QAPISchemaArrayType(name, None, element_type))
> + return name
Hmm - we probably have collisions if a user tries to explicitly name a
'struct' or other type with a 'List' suffix. Not made worse by this
patch and not an actual problem with any of our existing .json files, so
we can save it for another day.
> +
> + def _make_members(self, data):
> + return [self._make_member(key, data[key]) for key in data.keys()]
You could write this as:
return [self._make_member(key, value) for (key, value) in data.iteritems()]
for fewer copies and lookups (dict.keys() and dict.items() creates
copies, while dict.iteritems() visits key/value pairs in place).
I don't care strongly enough to hold up review, whether or not you
change it.
> +
> + def _def_union_type(self, expr, info):
> + name = expr['union']
> + data = expr['data']
> + base = expr.get('base')
> + tag_name = expr.get('discriminator')
> + tag_enum = None
> + if tag_name:
> + variants = [self._make_variant(key, data[key])
> + for key in data.keys()]
> + else:
> + variants = [self._make_simple_variant(key, data[key])
> + for key in data.keys()]
Same comments about the possibility for writing these list
comprehensions more efficiently. [and "list comprehensions" is my
latest bit of cool python language terminology that I learned the name
of, in order to do this review]
> + def _def_command(self, expr, info):
> + name = expr['command']
> + data = expr.get('data')
> + rets = expr.get('returns')
> + gen = expr.get('gen', True)
> + success_response = expr.get('success-response', True)
> + if isinstance(data, OrderedDict):
> + data = self._make_implicit_object_type(name, 'arg',
> + self._make_members(data))
> + if isinstance(rets, list):
> + assert len(rets) == 1
> + rets = self._make_array_type(rets[0])
> + elif isinstance(rets, OrderedDict):
> + rets = self._make_implicit_object_type(name, 'ret',
> + self._make_members(rets))
Dead 'elif' branch (since you reject dictionaries in "[PATCH 21/26]
qapi: Command returning anonymous type doesn't work, outlaw")
Looks like there will be some minor tweaks when you drop the RFC, but
it's close enough at this point that I'm okay with:
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 03/32] qapi: QAPISchema code generation helper methods
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 03/32] qapi: QAPISchema code generation helper methods Markus Armbruster
@ 2015-08-04 22:18 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-04 22:18 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> New methods c_name(), c_type(), c_null(), json_type(),
> alternate_qtype().
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi.py | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++------
> 1 file changed, 65 insertions(+), 7 deletions(-)
>
> def _def_predefineds(self):
> - for t in ['str', 'number', 'int', 'int8', 'int16', 'int32', 'int64',
> - 'uint8', 'uint16', 'uint32', 'uint64', 'size', 'bool', '**']:
> - self._def_builtin_type(t)
> + for t in [('str', 'string', 'char' + pointer_suffix, 'NULL'),
> + ('number', 'number', 'double', '0'),
So neither of us argued strongly enough for using '0.0' as the preferred
initializer for a double :)
The real fix compared to v2 was the handling of an empty enum, and that
looks correct.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor Markus Armbruster
@ 2015-08-04 22:26 ` Eric Blake
2015-08-05 6:24 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-04 22:26 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> The visitor will help keeping the code generation code simple and
> reasonably separated from QAPISchema details.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 46 insertions(+)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 3c596c3..019d22c 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -771,6 +771,29 @@ class QAPISchemaEntity(object):
> return c_name(self.name)
> def check(self, schema):
> pass
> + def visit(self, visitor):
> + pass
> +
> +class QAPISchemaVisitor(object):
> + def visit_begin(self):
> + pass
Don't know if you wanted to hoist from later patches, and write this as:
def visit_begin(self, schema):
> @@ -1166,6 +1206,12 @@ class QAPISchema(object):
> for ent in self.entity_dict.values():
> ent.check(self)
>
> + def visit(self, visitor):
> + visitor.visit_begin()
and this as
visitor.visit_begin(self)
up front, for less churn later on. Not the end of the world to leave it
as is, so my R-b stands either way.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor Markus Armbruster
@ 2015-08-04 22:35 ` Eric Blake
2015-08-05 6:26 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-04 22:35 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 2347 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> The old code prints the result of parsing (list of expression
> dictionaries), and partial results of semantic analysis (list of enum
> dictionaries, list of struct dictionaries).
>
> The new code prints a trace of a schema visit, i.e. what the back-ends
> are going to use. Built-in and array types are omitted, because
> they're boring.
Array types might be less boring in the future when we fix the TODO
about unconditional array type generation - but that is in the realm of
the future and doesn't affect this patch.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> tests/qapi-schema/alternate-good.out | 15 +-
> tests/qapi-schema/args-member-array.out | 13 +-
> tests/qapi-schema/comments.out | 4 +-
> tests/qapi-schema/empty.out | 3 -
> tests/qapi-schema/enum-empty.out | 4 +-
> tests/qapi-schema/event-case.out | 4 +-
> tests/qapi-schema/flat-union-reverse-define.out | 21 ++-
> tests/qapi-schema/ident-with-escape.out | 7 +-
> tests/qapi-schema/include-relpath.out | 4 +-
> tests/qapi-schema/include-repetition.out | 4 +-
> tests/qapi-schema/include-simple.out | 4 +-
> tests/qapi-schema/indented-expr.out | 7 +-
> tests/qapi-schema/qapi-schema-test.out | 214 ++++++++++++++++++------
> tests/qapi-schema/returns-int.out | 5 +-
> tests/qapi-schema/test-qapi.py | 38 ++++-
> tests/qapi-schema/type-bypass.out | 7 +-
> 16 files changed, 239 insertions(+), 115 deletions(-)
I don't know if you wanted to hoist the creation of the ':empty' object
into this patch, for less churn later on. But if you don't, I'm fine
with leaving this alone.
Reviewed-by: Eric Blake <eblake@redhat.com>
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -1,55 +1,159 @@
> +object :obj-boolList-wrapper
> + member data: boolList optional=False
...
> +object UserDefNativeListUnion
> + case integer: :obj-intList-wrapper
Simple unions turned out nicely with the generated wrapper type.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 06/32] qapi: Split up some typedefs to ease review
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 06/32] qapi: Split up some typedefs to ease review Markus Armbruster
@ 2015-08-04 22:37 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-04 22:37 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 769 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> One of the next patches will among other things generate a separate
> typedef for some struct types, i.e.
>
> typedef struct FOO FOO;
>
> struct FOO {
> ...
> };
>
> instead of
>
> typedef struct FOO {
> ...
> } FOO;
>
> To make the generated files easier to diff, anticipate the change.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi-types.py | 18 ++++++++++++------
> 1 file changed, 12 insertions(+), 6 deletions(-)
No R-b, since you revert it and intend to omit it on v4; but thanks for
doing this! It helped.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 07/32] qapi: Generate comments to simplify splitting for review
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 07/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
@ 2015-08-04 22:54 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-04 22:54 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1883 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> The effect of the next few patches on generated files is hard to
> review, because stuff gets generated in different order, which renders
> diffs of the generated files useless.
>
> To get reviewable diffs, we need to split the generated files into
> suitable parts: put every top-level declaration in a file named like
> the thing declared, so we can diff top-level declarations regardless
> of their order in the generated files.
>
> Since I don't feel like parsing C, simply generate a // comment
> identifying the declaration right before each top-level declaration.
> This lets us split with a simple shell loop:
>
Helpful notes to any other reviewer:
Add 'mkdir -p o' here...
> for i in q*-{commands,marshal,event,types,visit}.[ch]
> do
> csplit -n 4 -s "$i" '/^\/\//' '{*}'
> for j in xx*
> do
> read h t <$j
> [ "$h" == "//" ] || t=""
[ == ] is not portable shell (hi, dash users!); = works fine.
> mv $j "$i-${j#xx}-${t/ /-}"
...and modify this line to mv $j "o/$i-..." if you want...
> done
> done
>
> Splits each file F into F-NUMB-ID, where NUMB counts up from 0001, and
> ID comes from the // comment.
>
> To check the split's sane, we can run
>
> for i in q*-{commands,marshal,event,types,visit}.[ch]
> do cat o/$i-* | diff $i -
...this to work out of the box. (will come in handy later when creating
'n/' for comparing 'o/*' and 'n/*').
> done
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
Since this patch gets reverted and won't be in v4, it doesn't need R-b.
But it definitely helps my review, so thanks for providing it!
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 16/32] qapi: Generate comments to simplify splitting for review
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 16/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
@ 2015-08-04 23:02 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-04 23:02 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 2271 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> The effect of the previous few patches on generated files is hard to
> review, because stuff gets generated in different order, which renders
> diffs of the generated files useless.
>
> To get reviewable diffs, we need to split the generated files into
> suitable parts: put every top-level declaration in a file named like
> the thing declared, so we can diff top-level declarations regardless
> of their order in the generated files.
>
> Since I don't feel like parsing C, simply generate a // comment
> identifying the declaration right before each top-level declaration.
> This lets us split with a simple shell loop:
>
As in 7/32, a tip to other reviewers:
mkdir -p n
> for i in q*-{commands,marshal,event,types,visit}.[ch]
> do
> csplit -n 4 -s "$i" '/^\/\//' '{*}'
> for j in xx*
> do
> read h t <$j
> [ "$h" == "//" ] || t=""
=, not ==
> mv $j "$i-${j#xx}-${t/ /-}"
mv directly into n/$i-... here
> done
> done
>
> Splits each file F into F-NUMB-ID, where NUMB counts up from 0001, and
> ID comes from the // comment.
>
> To check the split's sane, we can run
>
> for i in q*-{commands,marshal,event,types,visit}.[ch]
> do cat $i-* | diff $i -
and test cat n/$i-* here.
> done
>
> We got a commit similar to this one right before the patches we want
> to check. If you have that commit's split files in directory o, and
> this commit's split files in directory n, you can diff them with this
> loop:
>
> for i in `(ls o; ls n) | sed -n 's/-[0-9][0-9][0-9][0-9]-/-*-/p' | sort | uniq`
Then o/* and n/* are populated correctly for this to work.
> do
> diff -uBp o/$i n/$i
> done
>
> Complete annotated output:
Explanations are sound. (I may have more comments later after reviewing
the intermediate patches; this email was just my early reminder hint on
how I made your comparison easier to reproduce)
Thanks again for doing this; it makes it a lot easier to be confident on
the rest of the series.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-04 21:53 ` Eric Blake
@ 2015-08-05 6:23 ` Markus Armbruster
2015-08-05 14:27 ` Eric Blake
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-05 6:23 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>> The QAPI code generators work with a syntax tree (nested dictionaries)
>> plus a few symbol tables (also dictionaries) on the side.
>>
>> They have clearly outgrown these simple data structures. There's lots
>> of rummaging around in dictionaries, and information is recomputed on
>> the fly. For the work I'm going to do, I want more clearly defined
>> and more convenient interfaces.
>>
>> Going forward, I also want less coupling between the back-ends and the
>> syntax tree, to make messing with the syntax easier.
>>
>> Create a bunch of classes to represent QAPI schemata.
>>
>> Have the QAPISchema initializer call the parser, then walk the syntax
>> tree to create the new internal representation, and finally perform
>> semantic analysis.
>>
>> Shortcut: the semantic analysis still relies on existing check_exprs()
>> to do the actual semantic checking. All this code needs to move into
>> the classes. Mark as TODO.
>>
>> We generate array types eagerly, even though most of them aren't used.
>> Mark as TODO.
>
> I'm not sure if there are any array types that the rest of the code base
> uses even though it doesn't appear as a directly used type within the
> schemata. Perhaps some of the builtin types (for example, if qom-get
> needs to return ['uint8']). Besides builtins, maybe we can add some sort
> of 'needs-array':'bool' key to each 'struct'/'union'/'enum'/'alternate',
> which defaults to true only if the schema refers to the array type, but
> which can be explicitly set to true to force the array type generation
> even without a schema reference. But as it is all properly marked TODO,
> the idle ramblings in this paragraph don't affect review.
Because code for built-ins can be shared among schemata (see -b), we
probably have to keep generating code for array of built-in type
unconditionally.
Re needs-array: stupidest solution that could possibly work is to have
an otherwise useless struct mention all the needed arrays.
>> Nothing uses the new intermediate representation just yet, thus no
>> change to generated files.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> scripts/qapi-commands.py | 2 +-
>> scripts/qapi-event.py | 2 +-
>> scripts/qapi-types.py | 2 +-
>> scripts/qapi-visit.py | 2 +-
>> scripts/qapi.py | 361 ++++++++++++++++++++++++++++++++++++++++-
>> tests/qapi-schema/test-qapi.py | 2 +-
>> 6 files changed, 357 insertions(+), 14 deletions(-)
>>
>
>> +class QAPISchemaEnumType(QAPISchemaType):
>> + def __init__(self, name, info, values):
>> + QAPISchemaType.__init__(self, name, info)
>> + for v in values:
>> + assert isinstance(v, str)
>> + self.values = values
>> + def check(self, schema):
>> + assert len(set(self.values)) == len(self.values)
>
> Doesn't check whether any of the distinct values map to the same C name.
Pervasive issue.
> But not a show-stopper to this patch (the earlier semantic checking in
> check_exprs() covers it, and your TODO about moving those checks here at
> a later date is the right time to worry about it here).
>
>> +
>> +class QAPISchemaArrayType(QAPISchemaType):
>> + def __init__(self, name, info, element_type):
>> + QAPISchemaType.__init__(self, name, info)
>> + assert isinstance(element_type, str)
>> + self.element_type_name = element_type
>> + self.element_type = None
>> + def check(self, schema):
>> + self.element_type = schema.lookup_type(self.element_type_name)
>> + assert self.element_type
>
> Is it worth adding:
>
> assert not isinstance(self.element_type, QAPISchemaArrayType)
>
> since we don't allow 2D arrays?
If the generators actually rely on it, yes.
If it's just an arbitrary schema language restriction, probably no.
>> +
>> +class QAPISchemaObjectType(QAPISchemaType):
>> + def __init__(self, name, info, base, local_members, variants):
>> + QAPISchemaType.__init__(self, name, info)
>> + assert base == None or isinstance(base, str)
>> + for m in local_members:
>> + assert isinstance(m, QAPISchemaObjectTypeMember)
>> + if variants != None:
>> + assert isinstance(variants, QAPISchemaObjectTypeVariants)
>
> Style nit: the 'base' and 'variants' checks are identical patterns
> (checking for None or specific type), but only one uses an 'if'.
> Possibly because of line-length issues, though, so I can live with it.
I'll clean it up.
>> +
>> +class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
>> + def __init__(self, name, typ):
>> + QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
>> + def check(self, schema, tag_type, seen):
>> + QAPISchemaObjectTypeMember.check(self, schema, [], seen)
>> + assert self.name in tag_type.values
>> + # TODO try to get rid of .simple_union_type()
>> + def simple_union_type(self):
>> + if isinstance(self.type, QAPISchemaObjectType) and not self.type.info:
>> + assert len(self.type.members) == 1
>> + assert not self.type.variants # not implemented
>
> and probably never will be (looks like this assert is copy-and-pasted
> from other locations where it makes sense that we might implement
> support for variants, but I don't see it ever happening for the
> generated ':obj-*-wrapper' type for the branch of a simple union)
True. I'll drop the comment.
> At any rate, I concur that we have a difference in the generated code
> for simple unions compared to flat unions (even if simple unions could
> be rewritten in terms of a flat union from the QMP side, the generated C
> side is not the same), so I agree that you need this function for now,
> as well as the comment for if we later try to clean up the code base to
> drop that difference.
That's the plan.
>> +class QAPISchema(object):
>
>> + def _make_array_type(self, element_type):
>> + name = element_type + 'List'
>> + if not self.lookup_type(name):
>> + self._def_entity(QAPISchemaArrayType(name, None, element_type))
>> + return name
>
> Hmm - we probably have collisions if a user tries to explicitly name a
> 'struct' or other type with a 'List' suffix. Not made worse by this
> patch and not an actual problem with any of our existing .json files, so
> we can save it for another day.
qapi-code-gen.txt reserves the 'Kind' suffix.
We should either adopt a sane, non-colliding scheme for generated names,
or prevent collisions by rejecting reserved names with a sane error
message (documenting them is then optional), or document reserved names.
The latter two require us to figure out what names we reserve. But as
you say, it's a task for another day.
>> +
>> + def _make_members(self, data):
>> + return [self._make_member(key, data[key]) for key in data.keys()]
>
> You could write this as:
>
> return [self._make_member(key, value) for (key, value) in data.iteritems()]
>
> for fewer copies and lookups (dict.keys() and dict.items() creates
> copies, while dict.iteritems() visits key/value pairs in place).
>
> I don't care strongly enough to hold up review, whether or not you
> change it.
I'll check it out.
>> +
>> + def _def_union_type(self, expr, info):
>> + name = expr['union']
>> + data = expr['data']
>> + base = expr.get('base')
>> + tag_name = expr.get('discriminator')
>> + tag_enum = None
>> + if tag_name:
>> + variants = [self._make_variant(key, data[key])
>> + for key in data.keys()]
>> + else:
>> + variants = [self._make_simple_variant(key, data[key])
>> + for key in data.keys()]
>
> Same comments about the possibility for writing these list
> comprehensions more efficiently.
There are a few more. I'll either change all or none.
> [and "list comprehensions" is my
> latest bit of cool python language terminology that I learned the name
> of, in order to do this review]
If I remember correctly, I first met them in an obscure hybrid of Prolog
and ML that was at the time a Professor's pet project, which naturally
meant every student must experience it, implemented in Lisp by a gifted
grad student, on an underpowered, memory-starved Mac. Let's say not
every student was able to see the beauty within the beast.
>> + def _def_command(self, expr, info):
>> + name = expr['command']
>> + data = expr.get('data')
>> + rets = expr.get('returns')
>> + gen = expr.get('gen', True)
>> + success_response = expr.get('success-response', True)
>> + if isinstance(data, OrderedDict):
>> + data = self._make_implicit_object_type(name, 'arg',
>> + self._make_members(data))
>> + if isinstance(rets, list):
>> + assert len(rets) == 1
>> + rets = self._make_array_type(rets[0])
>> + elif isinstance(rets, OrderedDict):
>> + rets = self._make_implicit_object_type(name, 'ret',
>> + self._make_members(rets))
>
> Dead 'elif' branch (since you reject dictionaries in "[PATCH 21/26]
> qapi: Command returning anonymous type doesn't work, outlaw")
Yup, will drop it.
> Looks like there will be some minor tweaks when you drop the RFC, but
> it's close enough at this point that I'm okay with:
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor
2015-08-04 22:26 ` Eric Blake
@ 2015-08-05 6:24 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-05 6:24 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>> The visitor will help keeping the code generation code simple and
>> reasonably separated from QAPISchema details.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> ---
>> scripts/qapi.py | 46 ++++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 46 insertions(+)
>>
>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>> index 3c596c3..019d22c 100644
>> --- a/scripts/qapi.py
>> +++ b/scripts/qapi.py
>> @@ -771,6 +771,29 @@ class QAPISchemaEntity(object):
>> return c_name(self.name)
>> def check(self, schema):
>> pass
>> + def visit(self, visitor):
>> + pass
>> +
>> +class QAPISchemaVisitor(object):
>> + def visit_begin(self):
>> + pass
>
> Don't know if you wanted to hoist from later patches, and write this as:
>
> def visit_begin(self, schema):
>
>> @@ -1166,6 +1206,12 @@ class QAPISchema(object):
>> for ent in self.entity_dict.values():
>> ent.check(self)
>>
>> + def visit(self, visitor):
>> + visitor.visit_begin()
>
> and this as
>
> visitor.visit_begin(self)
>
> up front, for less churn later on. Not the end of the world to leave it
> as is, so my R-b stands either way.
I ran out of time. Perhaps I can still try in the next iteration.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor
2015-08-04 22:35 ` Eric Blake
@ 2015-08-05 6:26 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-05 6:26 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>> The old code prints the result of parsing (list of expression
>> dictionaries), and partial results of semantic analysis (list of enum
>> dictionaries, list of struct dictionaries).
>>
>> The new code prints a trace of a schema visit, i.e. what the back-ends
>> are going to use. Built-in and array types are omitted, because
>> they're boring.
>
> Array types might be less boring in the future when we fix the TODO
> about unconditional array type generation - but that is in the realm of
> the future and doesn't affect this patch.
>
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> tests/qapi-schema/alternate-good.out | 15 +-
>> tests/qapi-schema/args-member-array.out | 13 +-
>> tests/qapi-schema/comments.out | 4 +-
>> tests/qapi-schema/empty.out | 3 -
>> tests/qapi-schema/enum-empty.out | 4 +-
>> tests/qapi-schema/event-case.out | 4 +-
>> tests/qapi-schema/flat-union-reverse-define.out | 21 ++-
>> tests/qapi-schema/ident-with-escape.out | 7 +-
>> tests/qapi-schema/include-relpath.out | 4 +-
>> tests/qapi-schema/include-repetition.out | 4 +-
>> tests/qapi-schema/include-simple.out | 4 +-
>> tests/qapi-schema/indented-expr.out | 7 +-
>> tests/qapi-schema/qapi-schema-test.out | 214
>> ++++++++++++++++++------
>> tests/qapi-schema/returns-int.out | 5 +-
>> tests/qapi-schema/test-qapi.py | 38 ++++-
>> tests/qapi-schema/type-bypass.out | 7 +-
>> 16 files changed, 239 insertions(+), 115 deletions(-)
>
> I don't know if you wanted to hoist the creation of the ':empty' object
> into this patch, for less churn later on. But if you don't, I'm fine
Again, out of time.
> with leaving this alone.
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
Thanks!
>> +++ b/tests/qapi-schema/qapi-schema-test.out
>> @@ -1,55 +1,159 @@
>
>> +object :obj-boolList-wrapper
>> + member data: boolList optional=False
> ...
>> +object UserDefNativeListUnion
>> + case integer: :obj-intList-wrapper
>
> Simple unions turned out nicely with the generated wrapper type.
Thanks for making me try harder there. It's more churn than I would
like, but the resulting improvement feels right.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-05 6:23 ` Markus Armbruster
@ 2015-08-05 14:27 ` Eric Blake
2015-08-06 5:46 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-05 14:27 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel, mdroth
[-- Attachment #1: Type: text/plain, Size: 2487 bytes --]
On 08/05/2015 12:23 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>>> The QAPI code generators work with a syntax tree (nested dictionaries)
>>> plus a few symbol tables (also dictionaries) on the side.
>>>
>>> +class QAPISchemaArrayType(QAPISchemaType):
>>> + def __init__(self, name, info, element_type):
>>> + QAPISchemaType.__init__(self, name, info)
>>> + assert isinstance(element_type, str)
>>> + self.element_type_name = element_type
>>> + self.element_type = None
>>> + def check(self, schema):
>>> + self.element_type = schema.lookup_type(self.element_type_name)
>>> + assert self.element_type
>>
>> Is it worth adding:
>>
>> assert not isinstance(self.element_type, QAPISchemaArrayType)
>>
>> since we don't allow 2D arrays?
>
> If the generators actually rely on it, yes.
Hmm. What happens if you do
{ 'command': 'Foo', 'returns': [ 'intList' ] }
>
> If it's just an arbitrary schema language restriction, probably no.
That's a tough judgment call. We don't currently allow [ [ 'int' ] ],
and the [ 'intList' ] hack is gross. On the other hand, I'm having a
tough time coming up with technical reasons why we can't do it (arrays
as a parameter or return type should work, and 2D arrays just add
another layer of '*' to the C code).
>>> + def _make_array_type(self, element_type):
>>> + name = element_type + 'List'
>>> + if not self.lookup_type(name):
>>> + self._def_entity(QAPISchemaArrayType(name, None, element_type))
>>> + return name
>>
>> Hmm - we probably have collisions if a user tries to explicitly name a
>> 'struct' or other type with a 'List' suffix. Not made worse by this
>> patch and not an actual problem with any of our existing .json files, so
>> we can save it for another day.
>
> qapi-code-gen.txt reserves the 'Kind' suffix.
>
> We should either adopt a sane, non-colliding scheme for generated names,
> or prevent collisions by rejecting reserved names with a sane error
> message (documenting them is then optional), or document reserved names.
> The latter two require us to figure out what names we reserve. But as
> you say, it's a task for another day.
And that cleanup can worry about [ 'intList' ].
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions Markus Armbruster
@ 2015-08-05 15:15 ` Eric Blake
2015-08-06 5:50 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-05 15:15 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1373 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> Fixes flat unions to get the base's base members. Test case is from
> commit 2fc0043, in qapi-schema-test.json:
>
>
> Flat union visitors remain broken. They'll be fixed next.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> docs/qapi-code-gen.txt | 51 +++---
Thanks for updating the docs. The diffstat of the actual generated code
is large, but the extra commits prove that in the end it's the same
material, just in different ordering. Meanwhile, the doc example is
correct, and being much shorter, makes it easier to see the sort of
reordering that happened.
> scripts/qapi-types.py | 273 ++++++++++++++------------------
> tests/qapi-schema/qapi-schema-test.json | 4 +-
> 3 files changed, 144 insertions(+), 184 deletions(-)
>
> +
> +# If you link code generated from multiple schemata, you want only one
> +# instance include the code generated for built-in types. Generate it
s/include/of/
> +# only when do_builtins, enabled by command line option -b. See also
> +# QAPISchemaGenTypeVisitor.visit_end().
> do_builtins = False
With the grammar fix,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs Markus Armbruster
@ 2015-08-05 16:03 ` Eric Blake
2015-08-06 22:53 ` Eric Blake
1 sibling, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-05 16:03 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1763 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> Fixes flat unions to visit the base's base members (the previous
> commit merely added them to the struct). Same test case.
>
> Patch's effect on visit_type_UserDefFlatUnion():
>
> static void visit_type_UserDefFlatUnion_fields(Visitor *m, UserDefFlatUnion **obj, Error **errp)
> {
> Error *err = NULL;
>
> + visit_type_int(m, &(*obj)->integer, "integer", &err);
> + if (err) {
> + goto out;
> + }
> visit_type_str(m, &(*obj)->string, "string", &err);
> if (err) {
> goto out;
>
> Test cases updated for the bug fix.
>
> Fixes alternates to generate a visitor for their implicit enumeration
> type. None of them are currently used, obviously. Example:
> block-core.json's BlockdevRef now generates
> visit_type_BlockdevRefKind().
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi-visit.py | 260 +++++++++++++-------------------
> tests/qapi-schema/qapi-schema-test.json | 3 -
> tests/test-qmp-input-strict.c | 2 +-
> tests/test-qmp-input-visitor.c | 4 +-
> 4 files changed, 106 insertions(+), 163 deletions(-)
>
> +
> +# If you link code generated from multiple schemata, you want only one
> +# instance include the code generated for built-in types. Generate it
> +# only when do_builtins, enabled by command line option -b. See also
> +# QAPISchemaGenVisitVisitor.visit_end().
> do_builtins = False
again, s/include/of/ and you can add:
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null()
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null() Markus Armbruster
@ 2015-08-05 16:13 ` Eric Blake
2015-08-06 5:52 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-05 16:13 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> is_c_ptr() looks whether the end of the C text for the type looks like
> a pointer. Works, but is fragile.
>
> We now have a better tool: use QAPISchemaType method c_null(). The
> initializers for non-pointers become prettier: 0, false or the
> enumeration constant with the value 0 instead of {0}.
>
> One place looks suspicious: we initialize pointers, but not
> non-pointers. Either the initialization is superfluous and should be
> deleted, or the non-pointers need it as well, or something subtle is
> going on and needs a comment. Since I lack the time to figure it out
> now, mark it FIXME.
Dead paragraph.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi-commands.py | 16 +++++-----------
> scripts/qapi.py | 3 ---
> 2 files changed, 5 insertions(+), 14 deletions(-)
>
With fixed commit message,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 19/32] qapi: Clean up after recent conversions to QAPISchemaVisitor
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 19/32] qapi: Clean up after recent conversions to QAPISchemaVisitor Markus Armbruster
@ 2015-08-05 16:29 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-05 16:29 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> Generate just 'FOO' instead of 'struct FOO' when possible.
>
> Drop helper functions that are now unused.
>
> Make pylint reasonably happy.
>
> Rename generate_FOO() functions to gen_FOO() for consistency.
>
> Use more consistent and sensible variable names.
>
> Consistently use c_ for mapping keys when their value is a C
> identifier or type.
>
> Simplify gen_enum() and gen_visit_union()
>
> Consistently use single quotes for C text string literals.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> docs/qapi-code-gen.txt | 2 +-
> scripts/qapi-commands.py | 129 +++++++++++++++++++++++-----------------------
> scripts/qapi-event.py | 121 ++++++++++++++++++++-----------------------
> scripts/qapi-types.py | 64 +++++++++++------------
> scripts/qapi-visit.py | 119 +++++++++++++++++++++---------------------
> scripts/qapi.py | 131 +++++++++--------------------------------------
> 6 files changed, 239 insertions(+), 327 deletions(-)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 23/32] qapi: De-duplicate parameter list generation
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 23/32] qapi: De-duplicate parameter list generation Markus Armbruster
@ 2015-08-05 17:00 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-05 17:00 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1603 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> Generated qapi-event.[ch] lose line breaks. No change otherwise.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com> ---
> scripts/qapi-commands.py | 11 ++--------- scripts/qapi-event.py |
> 18 +++--------------- scripts/qapi.py | 16 ++++++++++++++++
> 3 files changed, 21 insertions(+), 24 deletions(-)
>
> +++ b/scripts/qapi-event.py @@ -14,21 +14,9 @@ from qapi import *
>
> def gen_event_send_proto(name, arg_type): - api_name = "void
> qapi_event_send_%s(" % c_name(name).lower() - l = len(api_name) -
> - if arg_type: - for m in arg_type.members: - if
> m.optional: - api_name += "bool has_%s,\n" %
> c_name(m.name) - api_name += "".ljust(l) - -
> api_name += "%s %s,\n" % (m.type.c_type(is_param=True), -
> c_name(m.name))
Converting from "" % (list) to...
> - api_name += "".ljust(l) - - api_name += "Error
> **errp)" - return api_name + return 'void
> qapi_event_send_%(c_name)s(%(param)s)' % { + 'c_name':
> c_name(name.lower()), + 'param': gen_params(arg_type, 'Error
> **errp')}
...direct "" % {dict}. I hadn't seen that before in any of the qapi*.py
(I guess I'm too used to seeing mcgen() calls, where the dict is implied
via the **kwds magic). But since you don't want to call mcgen() on the
same input twice, it looks correct.
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 29/32] qapi: Pseudo-type '**' is now unused, drop it
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 29/32] qapi: Pseudo-type '**' is now unused, drop it Markus Armbruster
@ 2015-08-05 17:13 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-05 17:13 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> 'gen': false needs to stay for now, because netdev_add is still using
> it.
And the next patch will use it too.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> docs/qapi-code-gen.txt | 18 ++++++------------
> scripts/qapi.py | 20 ++++----------------
> tests/Makefile | 2 +-
> tests/qapi-schema/type-bypass-no-gen.err | 1 -
> tests/qapi-schema/type-bypass-no-gen.exit | 1 -
> tests/qapi-schema/type-bypass-no-gen.json | 2 --
> tests/qapi-schema/type-bypass-no-gen.out | 0
> tests/qapi-schema/type-bypass.err | 0
> tests/qapi-schema/type-bypass.exit | 1 -
> tests/qapi-schema/type-bypass.json | 2 --
> tests/qapi-schema/type-bypass.out | 4 ----
> 11 files changed, 11 insertions(+), 40 deletions(-)
> delete mode 100644 tests/qapi-schema/type-bypass-no-gen.err
> delete mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
> delete mode 100644 tests/qapi-schema/type-bypass-no-gen.json
> delete mode 100644 tests/qapi-schema/type-bypass-no-gen.out
> delete mode 100644 tests/qapi-schema/type-bypass.err
> delete mode 100644 tests/qapi-schema/type-bypass.exit
> delete mode 100644 tests/qapi-schema/type-bypass.json
> delete mode 100644 tests/qapi-schema/type-bypass.out
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
@ 2015-08-05 20:20 ` Eric Blake
2015-08-06 6:23 ` Markus Armbruster
2015-08-05 20:54 ` Eric Blake
` (2 subsequent siblings)
3 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-05 20:20 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 13172 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> Caution, rough edges.
>
> qapi/introspect.json defines the introspection schema. It should do
> for uses other than QMP.
> FIXME it's almost entirely devoid of comments.
>
That part explains why this is still RFC.
> The introspection schema does not reflect all the rules and
> restrictions that apply to QAPI schemata. A valid QAPI schema has an
> introspection value conforming to the introspection schema, but the
> converse is not true.
>
> Introspection lowers away a number of schema details:
>
> * The built-in types are declared with their JSON type.
>
> TODO Should we map all the integer types to just int?
Is it worth squashing that fix (patch 31) into this one?
>
> * Implicit type definitions are made explicit, and given
> auto-generated names. These names start with ':' so they don't
> clash with the user's names.
>
> Example: a simple union implicitly defines an enumeration type for
> its discriminator.
The names all change if we go with patch 32; but that one should stay
separate.
>
> * All type references are by name.
>
> * Base types are flattened.
>
> * The top type (named 'any') can hold any value.
>
> * The struct and union types are generalized into an object type.
>
> * Commands take a single argument and return a single result.
>
> Dictionary argument/result or list result is an implicit type
> definition.
Dictionary result is not possible, due to earlier changes.
>
> The empty object type is used when a command takes no arguments or
> produces no results.
>
> The argument is always of object type, but the introspection schema
> doesn't reflect that.
>
> The 'gen': false directive is omitted as implementation detail.
>
> The 'success-response' directive is ommitted as well for now, even
s/ommitted/omitted/
> though it's not an implementation detail.
>
> * Events carry a single data value.
>
> Implicit type definition and empty object type use, just like for
> commands.
>
> The value is of object type, but the introspection schema doesn't
> reflect that.
>
> * Types not used by commands or events are omitted.
>
> Indirect use counts as use.
>
> * Optional members have a default, which can only be null right now
>
> Instead of a mandatory "optional" flag, we have an optional default.
> No default means mandatory, default null means optional without
> default value. Non-null is available for optional with default.
>
> Alternate members can't have defaults, but the introspection schema
> doesn't reflect that.
This sentence is no longer necessary, now that alternates have their own
QAPI representation.
>
> * Clients should *not* look up types by name, because type names are
> not ABI. Look up the command or event you're interested in, then
> follow the references.
>
> TODO Should we hide the type names to eliminate the temptation?
>
> TODO much of the above should go into docs.
>
> New generator scripts/qapi-introspect.py computes an introspection
> value for its input, and generates a C variable holding it.
>
> FIXME it can generate awfully long lines
Worth repeating this FIXME in the code?
>
> A new test-qmp-input-visitor test case feeds its result for both
> tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
> QmpInputVisitor to verify it actually conforms to the schema.
>
> New QMP command query-schema takes its return value from that
> variable. Command documentation is incomplete, and marked FIXME. Its
> reply is some 85KiBytes for me right now.
>
> If this turns out to be too much, we have a couple of options:
>
> * We can use shorter names in the JSON. Not the QMP style.
>
> * Optionally return the sub-schema for commands and events given as
> arguments.
>
> Right now qmp_query_schema() sends the string literal computed by
> qmp-introspect.py. To compute sub-schema at run time, we'd have to
> duplicate parts of qapi-introspect.py in C. Unattractive.
>
> * Let clients cache the output of query-schema.
>
> It changes only on QEMU upgrades, i.e. rarely. Provide a command
> query-schema-hash. Clients can have a cache indexed by hash, and
> re-query the schema only when they don't have it cached.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
Lack of in-place documentation is still the biggest reason I'm not
comfortable with R-b yet (although this round is a bit further along
than v2 was); but there's also a bug that you need to fix on the
introspection for simple unions [1].
> +++ b/docs/qapi-code-gen.txt
> +Example:
> +
> + $ python scripts/qapi-event.py --output-dir="qapi-generated"
s/event/introspect/
> + --prefix="example-" example-schema.json
> + $ cat qapi-generated/example-qmp-introspect.c
> +[Uninteresting stuff omitted...]
> +
> + const char example_qmp_schema_json[] = "["
> + "{ \"arg-type\": \":empty\", \"name\": \"MY_EVENT\", \"meta-type\": \"event\" }, "
> + "{ \"meta-type\": \"builtin\", \"name\": \"int\", \"json-type\": \"int\" }, "
Any reason our poor-man's json formatter doesn't output dictionaries in
any particular order? It's particularly odd that 'meta-type' and 'name'
switch places across rows. I'm not sure if an OrderedDict and/or
strategic call to sort() keys would make the introspection nicer for
humans. Also, forcing a particular order means that we will be
guaranteed identical output regardless of python version or other random
hash factors that might otherwise render the generated file differently
across machines. On the other hand, the end result doesn't violate JSON
and shouldn't affect machine parsing, so it's not a reason to hold up
the patch. See also [2]
> + "{ \"meta-type\": \"builtin\", \"name\": \"str\", \"json-type\": \"string\" }, "
> + "{ \"meta-type\": \"object\", \"name\": \":empty\", \"members\": [ ] }, "
> + "{ \"meta-type\": \"object\", \"name\": \":obj-my-command-arg\", \"members\": [{ \"type\": \"UserDefOne\", \"name\": \"arg1\" } ] }, "
Why a space after { but not after [, particularly since there is also a
space before ] and }? Is it so you don't generate "[ ]" for :empty?
But JSON doesn't care about amount of whitespace, so it is not fatal.
> +/*
> + * Minor hack: generated marshalling suppressed for this command
> + * ('gen': false in the schema) so we can parse the JSON string
> + * directly into QObject instead of first parsing it with
> + * visit_type_SchemaInfoList() into a SchemaInfoList, then marshal it
> + * to QObject with generated output marshallers, every time. Instead,
> + * we do it in test-qmp-input-visitor.c, just to make sure
> + * qapi-introspect.py's output actually conforms to the schema.
> + */
> +static void qmp_query_schema(QDict *qdict, QObject **ret_data, Error **errp)
> +{
> + *ret_data = qobject_from_json(qmp_schema_json);
Huh, I just realized: in v2, I complained that your generated string
used 'quotes' instead of "quotes", because I was afraid the 'quote'
would leak onto the QMP wire and cause grief to strict-JSON clients.
But since we are parsing the string with qobject_from_json() (our own
parser, that understands the extension of 'string'), and then generating
the actual JSON output from QObject, what we pass over the QMP wire
_will_ have "quotes", even if qmp_schema_json did not. Furthermore,
what is sent on the wire has different spacing (all one single line,
unless you turn on the pretty qmp option), so it really doesn't matter
that you did all the work to use \" inside the generated string, nor
whether you use spacing for legibility. But at least I don't think it is
worth to try and output the generated string directly by skipping the
round trip into qobject and back out to JSON (it might be more
efficient, but this command will not be frequently called, and doing the
round trip avoids us having to worry about consistency issues).
So if it makes v4 easier, you could output "'string'" rather than
"\"string\"" for a slightly smaller generated file. But since you've
already done the work for escaping ", I'm also fine with the style used
in v3.
> +++ b/qapi/introspect.json
> +{ 'struct': 'SchemaInfoObject',
> + 'data': { 'members': [ 'SchemaInfoObjectMember' ],
> + '*tag': 'str',
> + '*variants': [ 'SchemaInfoObjectVariant' ] } }
I take it that either tag and variants will both be omitted, or both be
present. No clean way to represent that in the schema, though.
> +++ b/scripts/qapi-commands.py
> @@ -260,7 +260,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
> self.defn = None
> self.regy = None
> self.visited_ret_types = None
> - def visit_begin(self):
> + def visit_begin(self, schema):
You already mentioned that you ran out of time in v3 to float this to an
earlier patch; maybe v4 will be better?
> +++ b/scripts/qapi-introspect.py
> @@ -0,0 +1,172 @@
> +#
> +# QAPI introspection generator
> +#
> +# Copyright (C) 2015 Red Hat, Inc.
> +#
> +# Authors:
> +# Markus Armbruster <armbru@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2.
> +# See the COPYING file in the top-level directory.
> +
> +from qapi import *
> +
> +# Caveman's json.dumps() replacement (we're stuck at 2.4)
> +# TODO try to use json.dumps() once we get unstuck
> +def to_json(obj, level=0):
> + if obj == None:
> + ret = 'null'
> + elif isinstance(obj, str):
> + ret = '"' + obj.replace('"', r'\"') + '"'
> + elif isinstance(obj, list):
> + elts = [to_json(elt, level + 1)
> + for elt in obj]
No sorting here makes sense (the json-to-string converter can't know
whether we are using arrays as sets, vs. in the usual JSON semantics of
order-preserving). Thus, if we intend to sort things like enum values
or object members by name, that sorting has to be done prior to feeding
it to this pretty-printer.
> + ret = '[' + ', '.join(elts) + ' ]'
Here's where I was wondering if you want '[ '.
> + elif isinstance(obj, dict):
> + elts = ['"%s": %s' % (key, to_json(obj[key], level + 1))
> + for key in sorted(obj.keys())]
[2] Sorting here makes sense, except that I'm not sure what happened to
the sorting! Oh, I see - you captured your docs/qapi-code-gen.txt
_before_ adding the sort; because when I reran the steps in the docs,
the dictionary keys for "MY_EVENT" correctly list "meta-type" before
"name", and "json-type" before "meta-type", contrary to what you pasted
in the docs.
> + ret = '{ ' + ', '.join(elts) + ' }'
> + else:
> + assert False # not implemented
This would be number and boolean types. We would need to implement it
when we add support for defaults, but that's down the road.
> + def _gen_json(self, name, mtype, obj={}):
> + obj['name'] = name
> + obj['meta-type'] = mtype
> + self.jsons.append(obj)
Why is 'obj' given a default here, when all callers pass a non-empty dict?
> +
> + def _gen_variants(self, tag_name, variants):
> + return { 'tag': tag_name or 'kind',
[1] Bug mentioned above. The QMP wire format for a simple union tag is
'type', not 'kind'.
> + 'variants': [self._gen_variant(v) for v in variants] }
Do you want to add a sort() on variant 'case's here?
> +
> + def visit_enum_type(self, name, info, values):
> + self._gen_json(name, 'enum', { 'values': values })
Do you want to add a sort() on values here?
> +
> + def visit_array_type(self, name, info, element_type):
> + self._gen_json(name, 'array',
> + { 'element-type': self._use_type(element_type) })
> +
> + def visit_object_type_flat(self, name, info, members, variants):
> + obj = { 'members': [self._gen_member(m) for m in members] }
Do you want to add a sort() on member 'name's here?
> + if variants:
> + obj.update(self._gen_variants(variants.tag_name,
> + variants.variants))
> + self._gen_json(name, 'object', obj)
> +
> + def visit_alternate_type(self, name, info, variants):
> + self._gen_json(name, 'alternate',
> + { 'members': [ { 'type': self._use_type(m.type) }
> + for m in variants.variants] })
Do you want to add a sort() on member 'type's here?
> +++ b/tests/test-qmp-input-strict.c
> +
> +static void test_validate_qmp_introspect(TestInputVisitorData *data,
> + const void *unused)
> +{
Indentation is off.
Overall, still looking fairly close to the end product.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
2015-08-05 20:20 ` Eric Blake
@ 2015-08-05 20:54 ` Eric Blake
2015-08-06 6:47 ` Markus Armbruster
2015-08-23 4:17 ` Eric Blake
2015-09-01 18:40 ` Michael Roth
3 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-05 20:54 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 4471 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> * All type references are by name.
>
> Dictionary argument/result or list result is an implicit type
> definition.
>
>
> * Clients should *not* look up types by name, because type names are
> not ABI. Look up the command or event you're interested in, then
> follow the references.
>
> TODO Should we hide the type names to eliminate the temptation?
[not sure this conversation belongs better on 32 than here]
I was just experimenting with things, and noticed that our choice of
array names might not be ideal for clients. We already discussed
earlier in the series that the name 'intList' might collide (the parser
currently rejects collisions with '*Kind', but not '*List'). But in
addition to that, look at what happens here in this patch:
"{ \"element-type\": \"str\", \"meta-type\": \"array\", \"name\":
\"strList\" }, "
"{ \"element-type\": \"BlockStats\", \"meta-type\": \"array\",
\"name\": \"BlockStatsList\" }, "
"{ \"members\": [{ \"name\": \"values\", \"type\": \"strList\" } ],
\"meta-type\": \"object\", \"name\": \"SchemaInfoEnum\" }, "
"{ \"arg-type\": \":obj-query-blockstats-arg\", \"meta-type\":
\"command\", \"name\": \"query-blockstats\", \"ret-type\":
\"BlockStatsList\" }, "
and particularly what happens to it after patch 32 is applied:
"{ \"element-type\": \"str\", \"meta-type\": \"array\", \"name\":
\"270\" }, "
"{ \"element-type\": \"179\", \"meta-type\": \"array\", \"name\":
\"96\" }, "
"{ \"members\": [{ \"name\": \"values\", \"type\": \"270\" } ],
\"meta-type\": \"object\", \"name\": \"273\" }, "
"{ \"arg-type\": \"95\", \"meta-type\": \"command\", \"name\":
\"query-blockstats\", \"ret-type\": \"96\" }, "
In order to learn the return type of query-blockstats (to see if a new
member was added to the struct), I now have to query what type 96 is,
see that it is an array, and then query what members type 179 has. And
when dealing with type 273, I don't know whether to expect an array or
an object for the "type" member until I look up type 270 (but I _do_
know not to expect an int, number, boolean, or null).
What if we instead munged the name of array types in introspection
output to be "[int]" or "[BlockStats]"; or, with type masking in place,
"[int]" or "[179]"? As in the following for the above examples:
"{ \"element-type\": \"str\", \"meta-type\": \"array\", \"name\":
\"[str]\" }, "
"{ \"element-type\": \"179\", \"meta-type\": \"array\", \"name\":
\"[179]\" }, "
"{ \"members\": [{ \"name\": \"values\", \"type\": \"[str]\" } ],
\"meta-type\": \"object\", \"name\": \"273\" }, "
"{ \"arg-type\": \"95\", \"meta-type\": \"command\", \"name\":
\"query-blockstats\", \"ret-type\": \"[179]\" }, "
This way, libvirt or other clients could take shortcuts: I know that the
return of query-blockstats will be an array, and I only have to look up
the single type 179 to learn what members will be in the struct of that
array; likewise, when inspecting struct 273, I know that "type" is an
array of builtin strings and not an array of dicts, without needing to
do another type lookup.
I've always mentioned that I'm not a fan of packing multiple pieces of
information into a single JSON member (often a sign that the schema is
not specific enough), but in this particular case, the names would still
resolve without paying attention to the contents of the name. That is,
we'd be providing extra information (whether a name starts with leading
'[') that clients can optionally use to be more efficient, but which
will not penalize clients that stick with the tried-and-true approach of
finding the array member with the matching name (that is, if I chase
"[179]", I will still learn that I need to look up "179"). And it gives
us the nice property that the first character of a type specifies its
JSON properties (letter=>builtin, [=>array, digit=>object), other than
the magic "any".
I still think we need to munge to '*List' internally (as in our
qapi_free_*() functions, for example), but what I'm trying to propose is
that we don't expose the '*List' naming convention through
introspection. We'd also have to consider what this means for the
technical possibility of any 2D array types.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names Markus Armbruster
@ 2015-08-05 21:06 ` Eric Blake
2015-08-05 21:50 ` Eric Blake
2015-08-06 6:49 ` Markus Armbruster
0 siblings, 2 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-05 21:06 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 3544 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> To eliminate the temptation for clients to look up types by name
> (which are not ABI), replace all type names by meaningless strings.
>
> Reduces output of query-schema by 13 out of 85KiB.
I'm starting to be more in favor of this patch, both for ABI reasons and
for size shavings. It definitely looks better than in v2, where munged
names are just an arbitrary number, and where builtins are not munged.
>
> TODO Either generate comments with the true type names, or provide an
> option to generate without type name hiding.
See also my comments on 30/32 about whether we should mask array types
differently (and yes, the lack of comments made it a bit harder to chase
down types for my commentary in that mail).
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> docs/qapi-code-gen.txt | 14 +++++++-------
> scripts/qapi-introspect.py | 24 +++++++++++++++++++++---
> 2 files changed, 28 insertions(+), 10 deletions(-)
>
If we don't do anything to array names, then:
Reviewed-by: Eric Blake <eblake@redhat.com>
If we DO touch array names, I suspect this will look a lot different,
and have to drop R-b for v4.
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index ed04770..3a78cf4 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -868,13 +868,13 @@ Example:
> [Uninteresting stuff omitted...]
>
> const char example_qmp_schema_json[] = "["
> - "{ \"arg-type\": \":empty\", \"name\": \"MY_EVENT\", \"meta-type\": \"event\" }, "
> - "{ \"meta-type\": \"builtin\", \"name\": \"int\", \"json-type\": \"int\" }, "
> - "{ \"meta-type\": \"builtin\", \"name\": \"str\", \"json-type\": \"string\" }, "
> - "{ \"meta-type\": \"object\", \"name\": \":empty\", \"members\": [ ] }, "
> - "{ \"meta-type\": \"object\", \"name\": \":obj-my-command-arg\", \"members\": [{ \"type\": \"UserDefOne\", \"name\": \"arg1\" } ] }, "
> - "{ \"meta-type\": \"object\", \"name\": \"UserDefOne\", \"members\": [{ \"type\": \"int\", \"name\": \"integer\" }, { \"type\": \"str\", \"name\": \"string\" } ] }, "
> - "{ \"arg-type\": \":obj-my-command-arg\", \"ret-type\": \"UserDefOne\", \"name\": \"my-command\", \"meta-type\": \"command\" } ]";
> + "{ \"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": \"MY_EVENT\" }, "
> + "{ \"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": \"my-command\", \"ret-type\": \"2\" }, "
> + "{ \"members\": [ ], \"meta-type\": \"object\", \"name\": \"0\" }, "
> + "{ \"members\": [{ \"name\": \"arg1\", \"type\": \"2\" } ], \"meta-type\": \"object\", \"name\": \"1\" }, "
> + "{ \"members\": [{ \"name\": \"integer\", \"type\": \"int\" }, { \"name\": \"string\", \"type\": \"str\" } ], \"meta-type\": \"object\", \"name\": \"2\" }, "
> + "{ \"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": \"int\" }, "
> + "{ \"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": \"str\" } ]";
Some churn here once you fix the sorting in 30.
> +
> + def _name(self, name):
> + if name not in self.name_map:
> + n = len(self.name_map)
> + self.name_map[name] = '%s' % n
When string-izing an integer, isn't it better to use:
self.name_map[name] = '%d' % len(self.name_map)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names
2015-08-05 21:06 ` Eric Blake
@ 2015-08-05 21:50 ` Eric Blake
2015-08-06 6:49 ` Markus Armbruster
1 sibling, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-05 21:50 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 2448 bytes --]
On 08/05/2015 03:06 PM, Eric Blake wrote:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> To eliminate the temptation for clients to look up types by name
>> (which are not ABI), replace all type names by meaningless strings.
>>
>> Reduces output of query-schema by 13 out of 85KiB.
>
> I'm starting to be more in favor of this patch, both for ABI reasons and
> for size shavings. It definitely looks better than in v2, where munged
> names are just an arbitrary number, and where builtins are not munged.
>
>>
>> TODO Either generate comments with the true type names, or provide an
>> option to generate without type name hiding.
>
> See also my comments on 30/32 about whether we should mask array types
> differently (and yes, the lack of comments made it a bit harder to chase
> down types for my commentary in that mail).
In fact, squashing this in was all the more I needed to implement the
proposal there:
diff --git i/scripts/qapi-introspect.py w/scripts/qapi-introspect.py
index 5e99d4b..d723ef1 100644
--- i/scripts/qapi-introspect.py
+++ w/scripts/qapi-introspect.py
@@ -97,10 +97,12 @@ const char %(c_name)s[] = %(c_string)s;
# characters.
if isinstance(typ, QAPISchemaBuiltinType):
return typ.name
+ if isinstance(typ, QAPISchemaArrayType):
+ return '[' + self._use_type(typ.element_type) + ']'
return self._name(typ.name)
def _gen_json(self, name, mtype, obj={}):
- if mtype != 'command' and mtype != 'event' and mtype != 'builtin':
+ if mtype not in ('command', 'event', 'builtin', 'array'):
name = self._name(name)
obj['name'] = name
obj['meta-type'] = mtype
@@ -126,8 +128,9 @@ const char %(c_name)s[] = %(c_string)s;
self._gen_json(name, 'enum', { 'values': values })
def visit_array_type(self, name, info, element_type):
- self._gen_json(name, 'array',
- { 'element-type': self._use_type(element_type) })
+ element = self._use_type(element_type)
+ self._gen_json('[' + element + ']', 'array',
+ { 'element-type': element })
def visit_object_type_flat(self, name, info, members, variants):
obj = { 'members': [self._gen_member(m) for m in members] }
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-05 14:27 ` Eric Blake
@ 2015-08-06 5:46 ` Markus Armbruster
2015-08-31 18:09 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-06 5:46 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/05/2015 12:23 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>>>> The QAPI code generators work with a syntax tree (nested dictionaries)
>>>> plus a few symbol tables (also dictionaries) on the side.
>>>>
>
>>>> +class QAPISchemaArrayType(QAPISchemaType):
>>>> + def __init__(self, name, info, element_type):
>>>> + QAPISchemaType.__init__(self, name, info)
>>>> + assert isinstance(element_type, str)
>>>> + self.element_type_name = element_type
>>>> + self.element_type = None
>>>> + def check(self, schema):
>>>> + self.element_type = schema.lookup_type(self.element_type_name)
>>>> + assert self.element_type
>>>
>>> Is it worth adding:
>>>
>>> assert not isinstance(self.element_type, QAPISchemaArrayType)
>>>
>>> since we don't allow 2D arrays?
>>
>> If the generators actually rely on it, yes.
>
> Hmm. What happens if you do
> { 'command': 'Foo', 'returns': [ 'intList' ] }
>
>>
>> If it's just an arbitrary schema language restriction, probably no.
>
> That's a tough judgment call. We don't currently allow [ [ 'int' ] ],
> and the [ 'intList' ] hack is gross. On the other hand, I'm having a
> tough time coming up with technical reasons why we can't do it (arrays
> as a parameter or return type should work, and 2D arrays just add
> another layer of '*' to the C code).
Perhaps a quick experiment can decide the nature of the restriction.
>>>> + def _make_array_type(self, element_type):
>>>> + name = element_type + 'List'
>>>> + if not self.lookup_type(name):
>>>> + self._def_entity(QAPISchemaArrayType(name, None, element_type))
>>>> + return name
>>>
>>> Hmm - we probably have collisions if a user tries to explicitly name a
>>> 'struct' or other type with a 'List' suffix. Not made worse by this
>>> patch and not an actual problem with any of our existing .json files, so
>>> we can save it for another day.
>>
>> qapi-code-gen.txt reserves the 'Kind' suffix.
>>
>> We should either adopt a sane, non-colliding scheme for generated names,
>> or prevent collisions by rejecting reserved names with a sane error
>> message (documenting them is then optional), or document reserved names.
>> The latter two require us to figure out what names we reserve. But as
>> you say, it's a task for another day.
>
> And that cleanup can worry about [ 'intList' ].
Yes.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions
2015-08-05 15:15 ` Eric Blake
@ 2015-08-06 5:50 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-06 5:50 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>> Fixes flat unions to get the base's base members. Test case is from
>> commit 2fc0043, in qapi-schema-test.json:
>>
>
>>
>> Flat union visitors remain broken. They'll be fixed next.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> docs/qapi-code-gen.txt | 51 +++---
>
> Thanks for updating the docs. The diffstat of the actual generated code
> is large, but the extra commits prove that in the end it's the same
> material, just in different ordering. Meanwhile, the doc example is
> correct, and being much shorter, makes it easier to see the sort of
> reordering that happened.
>
>> scripts/qapi-types.py | 273 ++++++++++++++------------------
>> tests/qapi-schema/qapi-schema-test.json | 4 +-
>> 3 files changed, 144 insertions(+), 184 deletions(-)
>>
>
>> +
>> +# If you link code generated from multiple schemata, you want only one
>> +# instance include the code generated for built-in types. Generate it
>
> s/include/of/
Both phrasings parse fine for me :)
Yours: you want it generated just once.
Mine: you want only one generated instance include the code for built-in
types.
I'll go with "you want only one instance of the code for built-in
types."
>> +# only when do_builtins, enabled by command line option -b. See also
>> +# QAPISchemaGenTypeVisitor.visit_end().
>> do_builtins = False
>
> With the grammar fix,
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null()
2015-08-05 16:13 ` Eric Blake
@ 2015-08-06 5:52 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-06 5:52 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> is_c_ptr() looks whether the end of the C text for the type looks like
>> a pointer. Works, but is fragile.
>>
>> We now have a better tool: use QAPISchemaType method c_null(). The
>> initializers for non-pointers become prettier: 0, false or the
>> enumeration constant with the value 0 instead of {0}.
>>
>> One place looks suspicious: we initialize pointers, but not
>> non-pointers. Either the initialization is superfluous and should be
>> deleted, or the non-pointers need it as well, or something subtle is
>> going on and needs a comment. Since I lack the time to figure it out
>> now, mark it FIXME.
>
> Dead paragraph.
The pre KVM Forum time crunch shows %-}
Will drop.
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> scripts/qapi-commands.py | 16 +++++-----------
>> scripts/qapi.py | 3 ---
>> 2 files changed, 5 insertions(+), 14 deletions(-)
>>
>
> With fixed commit message,
> Reviewed-by: Eric Blake <eblake@redhat.com>
Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-05 20:20 ` Eric Blake
@ 2015-08-06 6:23 ` Markus Armbruster
2015-09-01 13:09 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-06 6:23 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> Caution, rough edges.
>>
>> qapi/introspect.json defines the introspection schema. It should do
>> for uses other than QMP.
>> FIXME it's almost entirely devoid of comments.
>>
>
> That part explains why this is still RFC.
>
>> The introspection schema does not reflect all the rules and
>> restrictions that apply to QAPI schemata. A valid QAPI schema has an
>> introspection value conforming to the introspection schema, but the
>> converse is not true.
>>
>> Introspection lowers away a number of schema details:
>>
>> * The built-in types are declared with their JSON type.
>>
>> TODO Should we map all the integer types to just int?
>
> Is it worth squashing that fix (patch 31) into this one?
If we're sufficiently sure the mapping is here to stay.
>>
>> * Implicit type definitions are made explicit, and given
>> auto-generated names. These names start with ':' so they don't
>> clash with the user's names.
>>
>> Example: a simple union implicitly defines an enumeration type for
>> its discriminator.
>
> The names all change if we go with patch 32; but that one should stay
> separate.
Agree.
>>
>> * All type references are by name.
>>
>> * Base types are flattened.
>>
>> * The top type (named 'any') can hold any value.
>>
>> * The struct and union types are generalized into an object type.
>>
>> * Commands take a single argument and return a single result.
>>
>> Dictionary argument/result or list result is an implicit type
>> definition.
>
> Dictionary result is not possible, due to earlier changes.
Will update.
>>
>> The empty object type is used when a command takes no arguments or
>> produces no results.
>>
>> The argument is always of object type, but the introspection schema
>> doesn't reflect that.
>>
>> The 'gen': false directive is omitted as implementation detail.
>>
>> The 'success-response' directive is ommitted as well for now, even
>
> s/ommitted/omitted/
Will fix.
>> though it's not an implementation detail.
>>
>> * Events carry a single data value.
>>
>> Implicit type definition and empty object type use, just like for
>> commands.
>>
>> The value is of object type, but the introspection schema doesn't
>> reflect that.
>>
>> * Types not used by commands or events are omitted.
>>
>> Indirect use counts as use.
>>
>> * Optional members have a default, which can only be null right now
>>
>> Instead of a mandatory "optional" flag, we have an optional default.
>> No default means mandatory, default null means optional without
>> default value. Non-null is available for optional with default.
>>
>> Alternate members can't have defaults, but the introspection schema
>> doesn't reflect that.
>
> This sentence is no longer necessary, now that alternates have their own
> QAPI representation.
Will update.
>>
>> * Clients should *not* look up types by name, because type names are
>> not ABI. Look up the command or event you're interested in, then
>> follow the references.
>>
>> TODO Should we hide the type names to eliminate the temptation?
>>
>> TODO much of the above should go into docs.
>>
>> New generator scripts/qapi-introspect.py computes an introspection
>> value for its input, and generates a C variable holding it.
>>
>> FIXME it can generate awfully long lines
>
> Worth repeating this FIXME in the code?
Yes.
>>
>> A new test-qmp-input-visitor test case feeds its result for both
>> tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
>> QmpInputVisitor to verify it actually conforms to the schema.
>>
>> New QMP command query-schema takes its return value from that
>> variable. Command documentation is incomplete, and marked FIXME. Its
>> reply is some 85KiBytes for me right now.
>>
>> If this turns out to be too much, we have a couple of options:
>>
>> * We can use shorter names in the JSON. Not the QMP style.
>>
>> * Optionally return the sub-schema for commands and events given as
>> arguments.
>>
>> Right now qmp_query_schema() sends the string literal computed by
>> qmp-introspect.py. To compute sub-schema at run time, we'd have to
>> duplicate parts of qapi-introspect.py in C. Unattractive.
>>
>> * Let clients cache the output of query-schema.
>>
>> It changes only on QEMU upgrades, i.e. rarely. Provide a command
>> query-schema-hash. Clients can have a cache indexed by hash, and
>> re-query the schema only when they don't have it cached.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>
> Lack of in-place documentation is still the biggest reason I'm not
> comfortable with R-b yet (although this round is a bit further along
> than v2 was); but there's also a bug that you need to fix on the
> introspection for simple unions [1].
>
>> +++ b/docs/qapi-code-gen.txt
>
>> +Example:
>> +
>> + $ python scripts/qapi-event.py --output-dir="qapi-generated"
>
> s/event/introspect/
Will fix.
>> + --prefix="example-" example-schema.json
>> + $ cat qapi-generated/example-qmp-introspect.c
>> +[Uninteresting stuff omitted...]
>> +
>> + const char example_qmp_schema_json[] = "["
>> + "{ \"arg-type\": \":empty\", \"name\": \"MY_EVENT\",
>> \"meta-type\": \"event\" }, "
>> + "{ \"meta-type\": \"builtin\", \"name\": \"int\", \"json-type\":
>> \"int\" }, "
>
> Any reason our poor-man's json formatter doesn't output dictionaries in
> any particular order? It's particularly odd that 'meta-type' and 'name'
> switch places across rows. I'm not sure if an OrderedDict and/or
> strategic call to sort() keys would make the introspection nicer for
> humans. Also, forcing a particular order means that we will be
> guaranteed identical output regardless of python version or other random
> hash factors that might otherwise render the generated file differently
> across machines. On the other hand, the end result doesn't violate JSON
> and shouldn't affect machine parsing, so it's not a reason to hold up
> the patch. See also [2]
Yes, see [2]. I realized unstable dictionary order could be annoying,
so I added sorting, but forgot to regenerate this example. Will fix.
>> + "{ \"meta-type\": \"builtin\", \"name\": \"str\", \"json-type\":
>> \"string\" }, "
>> + "{ \"meta-type\": \"object\", \"name\": \":empty\", \"members\": [
>> ] }, "
>> + "{ \"meta-type\": \"object\", \"name\": \":obj-my-command-arg\",
>> \"members\": [{ \"type\": \"UserDefOne\", \"name\": \"arg1\" } ] },
>> "
>
> Why a space after { but not after [, particularly since there is also a
> space before ] and }? Is it so you don't generate "[ ]" for :empty?
> But JSON doesn't care about amount of whitespace, so it is not fatal.
Accident. I'll normalize the spacing.
>> +/*
>> + * Minor hack: generated marshalling suppressed for this command
>> + * ('gen': false in the schema) so we can parse the JSON string
>> + * directly into QObject instead of first parsing it with
>> + * visit_type_SchemaInfoList() into a SchemaInfoList, then marshal it
>> + * to QObject with generated output marshallers, every time. Instead,
>> + * we do it in test-qmp-input-visitor.c, just to make sure
>> + * qapi-introspect.py's output actually conforms to the schema.
>> + */
>> +static void qmp_query_schema(QDict *qdict, QObject **ret_data, Error **errp)
>> +{
>> + *ret_data = qobject_from_json(qmp_schema_json);
>
> Huh, I just realized: in v2, I complained that your generated string
> used 'quotes' instead of "quotes", because I was afraid the 'quote'
> would leak onto the QMP wire and cause grief to strict-JSON clients.
> But since we are parsing the string with qobject_from_json() (our own
> parser, that understands the extension of 'string'), and then generating
> the actual JSON output from QObject, what we pass over the QMP wire
> _will_ have "quotes", even if qmp_schema_json did not. Furthermore,
> what is sent on the wire has different spacing (all one single line,
> unless you turn on the pretty qmp option), so it really doesn't matter
> that you did all the work to use \" inside the generated string, nor
> whether you use spacing for legibility.
Correct. I forgot that detail, then bought your "must use double quote
in JSON" argument. It came back later when I wondered why I see a
different order on the wire than in qmp-introspect.c, but by then the
double quote change had been paged out of my working memory.
> But at least I don't think it is
> worth to try and output the generated string directly by skipping the
> round trip into qobject and back out to JSON (it might be more
> efficient, but this command will not be frequently called, and doing the
> round trip avoids us having to worry about consistency issues).
Yes, too invasive to be worthwhile: in addition to the generated
marshaller ('gen': false), we'd have to bypass the QMP wire protocol
emitter.
> So if it makes v4 easier, you could output "'string'" rather than
> "\"string\"" for a slightly smaller generated file. But since you've
> already done the work for escaping ", I'm also fine with the style used
> in v3.
For what it's worth, json.dumps() uses double quotes. I figure it can
be made to use single quotes, but that would be work. I'm leaning
towards keeping the double quotes.
>> +++ b/qapi/introspect.json
>
>> +{ 'struct': 'SchemaInfoObject',
>> + 'data': { 'members': [ 'SchemaInfoObjectMember' ],
>> + '*tag': 'str',
>> + '*variants': [ 'SchemaInfoObjectVariant' ] } }
>
> I take it that either tag and variants will both be omitted, or both be
> present. No clean way to represent that in the schema, though.
Needs a comment.
>> +++ b/scripts/qapi-commands.py
>> @@ -260,7 +260,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
>> self.defn = None
>> self.regy = None
>> self.visited_ret_types = None
>> - def visit_begin(self):
>> + def visit_begin(self, schema):
>
> You already mentioned that you ran out of time in v3 to float this to an
> earlier patch; maybe v4 will be better?
I intend to give it a try to see whether it turns out better.
>> +++ b/scripts/qapi-introspect.py
>> @@ -0,0 +1,172 @@
>> +#
>> +# QAPI introspection generator
>> +#
>> +# Copyright (C) 2015 Red Hat, Inc.
>> +#
>> +# Authors:
>> +# Markus Armbruster <armbru@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2.
>> +# See the COPYING file in the top-level directory.
>> +
>> +from qapi import *
>> +
>> +# Caveman's json.dumps() replacement (we're stuck at 2.4)
>> +# TODO try to use json.dumps() once we get unstuck
>> +def to_json(obj, level=0):
>> + if obj == None:
>> + ret = 'null'
>> + elif isinstance(obj, str):
>> + ret = '"' + obj.replace('"', r'\"') + '"'
>> + elif isinstance(obj, list):
>> + elts = [to_json(elt, level + 1)
>> + for elt in obj]
>
> No sorting here makes sense (the json-to-string converter can't know
> whether we are using arrays as sets, vs. in the usual JSON semantics of
> order-preserving). Thus, if we intend to sort things like enum values
> or object members by name, that sorting has to be done prior to feeding
> it to this pretty-printer.
Correct. I didn't, because I ran out of time.
>> + ret = '[' + ', '.join(elts) + ' ]'
>
> Here's where I was wondering if you want '[ '.
Yes.
>> + elif isinstance(obj, dict):
>> + elts = ['"%s": %s' % (key, to_json(obj[key], level + 1))
>> + for key in sorted(obj.keys())]
>
> [2] Sorting here makes sense, except that I'm not sure what happened to
> the sorting! Oh, I see - you captured your docs/qapi-code-gen.txt
> _before_ adding the sort; because when I reran the steps in the docs,
> the dictionary keys for "MY_EVENT" correctly list "meta-type" before
> "name", and "json-type" before "meta-type", contrary to what you pasted
> in the docs.
You're right.
>> + ret = '{ ' + ', '.join(elts) + ' }'
>> + else:
>> + assert False # not implemented
>
> This would be number and boolean types. We would need to implement it
> when we add support for defaults, but that's down the road.
Yes. I prefer honest assert to untestable code.
>> + def _gen_json(self, name, mtype, obj={}):
>> + obj['name'] = name
>> + obj['meta-type'] = mtype
>> + self.jsons.append(obj)
>
> Why is 'obj' given a default here, when all callers pass a non-empty dict?
I'll drop it. If a caller wants {}, it can pass {}.
>> +
>> + def _gen_variants(self, tag_name, variants):
>> + return { 'tag': tag_name or 'kind',
>
> [1] Bug mentioned above. The QMP wire format for a simple union tag is
> 'type', not 'kind'.
Will fix.
>> + 'variants': [self._gen_variant(v) for v in variants] }
>
> Do you want to add a sort() on variant 'case's here?
>
>> +
>> + def visit_enum_type(self, name, info, values):
>> + self._gen_json(name, 'enum', { 'values': values })
>
> Do you want to add a sort() on values here?
I want to review the introspection schema systematically for sorting
opportunities.
>> +
>> + def visit_array_type(self, name, info, element_type):
>> + self._gen_json(name, 'array',
>> + { 'element-type': self._use_type(element_type) })
>> +
>> + def visit_object_type_flat(self, name, info, members, variants):
>> + obj = { 'members': [self._gen_member(m) for m in members] }
>
> Do you want to add a sort() on member 'name's here?
>
>> + if variants:
>> + obj.update(self._gen_variants(variants.tag_name,
>> + variants.variants))
>> + self._gen_json(name, 'object', obj)
>> +
>> + def visit_alternate_type(self, name, info, variants):
>> + self._gen_json(name, 'alternate',
>> + { 'members': [ { 'type': self._use_type(m.type) }
>> + for m in variants.variants] })
>
> Do you want to add a sort() on member 'type's here?
>
>> +++ b/tests/test-qmp-input-strict.c
>
>> +
>> +static void test_validate_qmp_introspect(TestInputVisitorData *data,
>> + const void *unused)
>> +{
>
> Indentation is off.
Will fix.
> Overall, still looking fairly close to the end product.
Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-05 20:54 ` Eric Blake
@ 2015-08-06 6:47 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-06 6:47 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> * All type references are by name.
>>
>
>> Dictionary argument/result or list result is an implicit type
>> definition.
>>
>
>>
>> * Clients should *not* look up types by name, because type names are
>> not ABI. Look up the command or event you're interested in, then
>> follow the references.
>>
>> TODO Should we hide the type names to eliminate the temptation?
>
> [not sure this conversation belongs better on 32 than here]
>
> I was just experimenting with things, and noticed that our choice of
> array names might not be ideal for clients. We already discussed
> earlier in the series that the name 'intList' might collide (the parser
> currently rejects collisions with '*Kind', but not '*List'). But in
> addition to that, look at what happens here in this patch:
>
> "{ \"element-type\": \"str\", \"meta-type\": \"array\", \"name\":
> \"strList\" }, "
>
> "{ \"element-type\": \"BlockStats\", \"meta-type\": \"array\",
> \"name\": \"BlockStatsList\" }, "
>
> "{ \"members\": [{ \"name\": \"values\", \"type\": \"strList\" } ],
> \"meta-type\": \"object\", \"name\": \"SchemaInfoEnum\" }, "
>
> "{ \"arg-type\": \":obj-query-blockstats-arg\", \"meta-type\":
> \"command\", \"name\": \"query-blockstats\", \"ret-type\":
> \"BlockStatsList\" }, "
>
>
> and particularly what happens to it after patch 32 is applied:
>
> "{ \"element-type\": \"str\", \"meta-type\": \"array\", \"name\":
> \"270\" }, "
>
> "{ \"element-type\": \"179\", \"meta-type\": \"array\", \"name\":
> \"96\" }, "
>
> "{ \"members\": [{ \"name\": \"values\", \"type\": \"270\" } ],
> \"meta-type\": \"object\", \"name\": \"273\" }, "
>
> "{ \"arg-type\": \"95\", \"meta-type\": \"command\", \"name\":
> \"query-blockstats\", \"ret-type\": \"96\" }, "
>
> In order to learn the return type of query-blockstats (to see if a new
> member was added to the struct), I now have to query what type 96 is,
> see that it is an array, and then query what members type 179 has. And
> when dealing with type 273, I don't know whether to expect an array or
> an object for the "type" member until I look up type 270 (but I _do_
> know not to expect an int, number, boolean, or null).
To check for keys returned by query-blockstats:
1. Look up command query-blockstats.
2. Look up its ret-type, it must be array (or else we screwed up).
3. Look up the array's element-type, it must be object (or else we
screwed up).
4. Examine the element-type's member's names.
> What if we instead munged the name of array types in introspection
> output to be "[int]" or "[BlockStats]"; or, with type masking in place,
> "[int]" or "[179]"? As in the following for the above examples:
>
> "{ \"element-type\": \"str\", \"meta-type\": \"array\", \"name\":
> \"[str]\" }, "
>
> "{ \"element-type\": \"179\", \"meta-type\": \"array\", \"name\":
> \"[179]\" }, "
>
> "{ \"members\": [{ \"name\": \"values\", \"type\": \"[str]\" } ],
> \"meta-type\": \"object\", \"name\": \"273\" }, "
>
> "{ \"arg-type\": \"95\", \"meta-type\": \"command\", \"name\":
> \"query-blockstats\", \"ret-type\": \"[179]\" }, "
>
> This way, libvirt or other clients could take shortcuts: I know that the
> return of query-blockstats will be an array, and I only have to look up
> the single type 179 to learn what members will be in the struct of that
> array; likewise, when inspecting struct 273, I know that "type" is an
> array of builtin strings and not an array of dicts, without needing to
> do another type lookup.
To check for keys returned by query-blockstats:
1. Look up command query-blockstats.
2'. Its ret-type must be of the form '[element-type]' (or else we screwed
up). Extract element-type.
3. Look up the array's element-type, it must be object (or else we
screwed up).
4. Examine the element-type's member's names.
Trades a type lookup (code you already have) for a special case
(additional code). I wouldn't do it.
> I've always mentioned that I'm not a fan of packing multiple pieces of
> information into a single JSON member (often a sign that the schema is
> not specific enough), but in this particular case, the names would still
> resolve without paying attention to the contents of the name. That is,
> we'd be providing extra information (whether a name starts with leading
> '[') that clients can optionally use to be more efficient, but which
> will not penalize clients that stick with the tried-and-true approach of
> finding the array member with the matching name (that is, if I chase
> "[179]", I will still learn that I need to look up "179").
Aha, you're *not* proposing to omit the array type definitions. Good,
because I'd object to that :) I intentionally made all types explicit
to keep things as simple as possible.
You're proposing a more innocent change: a special naming convention for
array types.
> And it gives
> us the nice property that the first character of a type specifies its
> JSON properties (letter=>builtin, [=>array, digit=>object), other than
> the magic "any".
Yes, but do we want to make that first letter convention ABI?
If not, then clients can't rely on it.
If we use it, we risk clients relying on it anyway, making it de facto
ABI.
> I still think we need to munge to '*List' internally (as in our
> qapi_free_*() functions, for example), but what I'm trying to propose is
> that we don't expose the '*List' naming convention through
> introspection. We'd also have to consider what this means for the
> technical possibility of any 2D array types.
After this patch, the only names visible in query-schema's result are:
* Built-in type names
* Command and event names
* Object member names
* Enumeration constants
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names
2015-08-05 21:06 ` Eric Blake
2015-08-05 21:50 ` Eric Blake
@ 2015-08-06 6:49 ` Markus Armbruster
1 sibling, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-06 6:49 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> To eliminate the temptation for clients to look up types by name
>> (which are not ABI), replace all type names by meaningless strings.
>>
>> Reduces output of query-schema by 13 out of 85KiB.
>
> I'm starting to be more in favor of this patch, both for ABI reasons and
> for size shavings. It definitely looks better than in v2, where munged
> names are just an arbitrary number, and where builtins are not munged.
Yes, much easier to read, and smaller, too. Review pays :)
>> TODO Either generate comments with the true type names, or provide an
>> option to generate without type name hiding.
>
> See also my comments on 30/32 about whether we should mask array types
> differently (and yes, the lack of comments made it a bit harder to chase
> down types for my commentary in that mail).
>
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> docs/qapi-code-gen.txt | 14 +++++++-------
>> scripts/qapi-introspect.py | 24 +++++++++++++++++++++---
>> 2 files changed, 28 insertions(+), 10 deletions(-)
>>
>
> If we don't do anything to array names, then:
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
> If we DO touch array names, I suspect this will look a lot different,
> and have to drop R-b for v4.
>
>> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
>> index ed04770..3a78cf4 100644
>> --- a/docs/qapi-code-gen.txt
>> +++ b/docs/qapi-code-gen.txt
>> @@ -868,13 +868,13 @@ Example:
>> [Uninteresting stuff omitted...]
>>
>> const char example_qmp_schema_json[] = "["
>> - "{ \"arg-type\": \":empty\", \"name\": \"MY_EVENT\", \"meta-type\": \"event\" }, "
>> - "{ \"meta-type\": \"builtin\", \"name\": \"int\", \"json-type\": \"int\" }, "
>> - "{ \"meta-type\": \"builtin\", \"name\": \"str\", \"json-type\": \"string\" }, "
>> - "{ \"meta-type\": \"object\", \"name\": \":empty\", \"members\": [ ] }, "
>> - "{ \"meta-type\": \"object\", \"name\": \":obj-my-command-arg\", \"members\": [{ \"type\": \"UserDefOne\", \"name\": \"arg1\" } ] }, "
>> - "{ \"meta-type\": \"object\", \"name\": \"UserDefOne\", \"members\": [{ \"type\": \"int\", \"name\": \"integer\" }, { \"type\": \"str\", \"name\": \"string\" } ] }, "
>> - "{ \"arg-type\": \":obj-my-command-arg\", \"ret-type\": \"UserDefOne\", \"name\": \"my-command\", \"meta-type\": \"command\" } ]";
>> + "{ \"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": \"MY_EVENT\" }, "
>> + "{ \"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": \"my-command\", \"ret-type\": \"2\" }, "
>> + "{ \"members\": [ ], \"meta-type\": \"object\", \"name\": \"0\" }, "
>> + "{ \"members\": [{ \"name\": \"arg1\", \"type\": \"2\" } ], \"meta-type\": \"object\", \"name\": \"1\" }, "
>> + "{ \"members\": [{ \"name\": \"integer\", \"type\": \"int\" }, { \"name\": \"string\", \"type\": \"str\" } ], \"meta-type\": \"object\", \"name\": \"2\" }, "
>> + "{ \"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": \"int\" }, "
>> + "{ \"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": \"str\" } ]";
>
> Some churn here once you fix the sorting in 30.
>
>> +
>> + def _name(self, name):
>> + if name not in self.name_map:
>> + n = len(self.name_map)
>> + self.name_map[name] = '%s' % n
>
> When string-izing an integer, isn't it better to use:
>
> self.name_map[name] = '%d' % len(self.name_map)
Yes, will change.
Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs Markus Armbruster
2015-08-05 16:03 ` Eric Blake
@ 2015-08-06 22:53 ` Eric Blake
2015-08-08 6:07 ` Markus Armbruster
1 sibling, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-06 22:53 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 2824 bytes --]
On 08/04/2015 09:57 AM, Markus Armbruster wrote:
> Fixes flat unions to visit the base's base members (the previous
> commit merely added them to the struct). Same test case.
>
> Patch's effect on visit_type_UserDefFlatUnion():
>
> static void visit_type_UserDefFlatUnion_fields(Visitor *m, UserDefFlatUnion **obj, Error **errp)
> {
> Error *err = NULL;
>
> + visit_type_int(m, &(*obj)->integer, "integer", &err);
> + if (err) {
> + goto out;
> + }
> visit_type_str(m, &(*obj)->string, "string", &err);
> if (err) {
> goto out;
>
> Test cases updated for the bug fix.
Not quite right.
>
> Fixes alternates to generate a visitor for their implicit enumeration
> type. None of them are currently used, obviously. Example:
> block-core.json's BlockdevRef now generates
> visit_type_BlockdevRefKind().
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
> scripts/qapi-visit.py | 260 +++++++++++++-------------------
> tests/qapi-schema/qapi-schema-test.json | 3 -
> tests/test-qmp-input-strict.c | 2 +-
> tests/test-qmp-input-visitor.c | 4 +-
> 4 files changed, 106 insertions(+), 163 deletions(-)
> +++ b/tests/test-qmp-input-strict.c
> @@ -167,9 +167,9 @@ static void test_validate_union_flat(TestInputVisitorData *data,
>
> v = validate_test_init(data,
> "{ 'enum1': 'value1', "
> + "'integer': 41, "
> "'string': 'str', "
> "'boolean': true }");
> - /* TODO when generator bug is fixed, add 'integer': 41 */
>
> visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
> g_assert(!err);
Incomplete fix; you need to squash this in to test that cleanup of a
partial struct is still correct (otherwise, the test sets err for the
wrong reason of "missing 'integer'" rather than the intended reason of
"missing 'enum1'"):
diff --git i/tests/test-qmp-input-strict.c w/tests/test-qmp-input-strict.c
index bfd9d04..4c18096 100644
--- i/tests/test-qmp-input-strict.c
+++ w/tests/test-qmp-input-strict.c
@@ -299,7 +299,7 @@ static void
test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
Visitor *v;
/* test situation where discriminator field ('enum1' here) is
missing */
- v = validate_test_init(data, "{ 'string': 'c', 'string1': 'd',
'string2': 'e' }");
+ v = validate_test_init(data, "{ 'integer': 42, 'string': 'c',
'string1': 'd', 'string2': 'e' }");
visit_type_UserDefFlatUnion2(v, &tmp, NULL, &err);
g_assert(err);
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type Markus Armbruster
@ 2015-08-07 19:30 ` Eric Blake
2015-08-08 6:24 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-07 19:30 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 1548 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> It's first class, because unlike '**', it actually works, i.e. doesn't
> require 'gen': false.
>
> '**' will go away next.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
> @@ -1039,8 +1040,7 @@ class QAPISchema(object):
>
> def _def_builtin_type(self, name, json_type, c_type, c_null):
> self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type, c_null))
> - if name != '**':
> - self._make_array_type(name) # TODO really needed?
> + self._make_array_type(name) # TODO really needed?
Do we really want to allow ['any'] in schemata? That would imply the
possibility of a 2D array.
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -83,6 +83,8 @@
> 'returns': 'UserDefTwo' }
> { 'command': 'user_def_cmd3', 'data': {'a': 'int', '*b': 'int' },
> 'returns': 'int' }
> +# note: command name 'guest-sync' chosen to avoid "cannot use built-in" error
> +{ 'command': 'guest-sync', 'data': { 'arg': 'any' }, 'returns': 'any' }
>
In particular, if we DO want to allow it, this file should be enhanced
to include ['any'] in the UserDefNativeListUnion.
As it is, JSON allows mixed-type arrays, but all our uses of QList are
fixed-type (all elements share the same type); allowing an array of any
element may prove to be problematic.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs
2015-08-06 22:53 ` Eric Blake
@ 2015-08-08 6:07 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-08 6:07 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>> Fixes flat unions to visit the base's base members (the previous
>> commit merely added them to the struct). Same test case.
>>
>> Patch's effect on visit_type_UserDefFlatUnion():
>>
>> static void visit_type_UserDefFlatUnion_fields(Visitor *m, UserDefFlatUnion **obj, Error **errp)
>> {
>> Error *err = NULL;
>>
>> + visit_type_int(m, &(*obj)->integer, "integer", &err);
>> + if (err) {
>> + goto out;
>> + }
>> visit_type_str(m, &(*obj)->string, "string", &err);
>> if (err) {
>> goto out;
>>
>> Test cases updated for the bug fix.
>
> Not quite right.
>
>>
>> Fixes alternates to generate a visitor for their implicit enumeration
>> type. None of them are currently used, obviously. Example:
>> block-core.json's BlockdevRef now generates
>> visit_type_BlockdevRefKind().
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> scripts/qapi-visit.py | 260 +++++++++++++-------------------
>> tests/qapi-schema/qapi-schema-test.json | 3 -
>> tests/test-qmp-input-strict.c | 2 +-
>> tests/test-qmp-input-visitor.c | 4 +-
>> 4 files changed, 106 insertions(+), 163 deletions(-)
>
>> +++ b/tests/test-qmp-input-strict.c
>> @@ -167,9 +167,9 @@ static void test_validate_union_flat(TestInputVisitorData *data,
>>
>> v = validate_test_init(data,
>> "{ 'enum1': 'value1', "
>> + "'integer': 41, "
>> "'string': 'str', "
>> "'boolean': true }");
>> - /* TODO when generator bug is fixed, add 'integer': 41 */
>>
>> visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
>> g_assert(!err);
>
> Incomplete fix; you need to squash this in to test that cleanup of a
> partial struct is still correct (otherwise, the test sets err for the
> wrong reason of "missing 'integer'" rather than the intended reason of
> "missing 'enum1'"):
>
> diff --git i/tests/test-qmp-input-strict.c w/tests/test-qmp-input-strict.c
> index bfd9d04..4c18096 100644
> --- i/tests/test-qmp-input-strict.c
> +++ w/tests/test-qmp-input-strict.c
> @@ -299,7 +299,7 @@ static void
> test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
> Visitor *v;
>
> /* test situation where discriminator field ('enum1' here) is
> missing */
> - v = validate_test_init(data, "{ 'string': 'c', 'string1': 'd',
> 'string2': 'e' }");
> + v = validate_test_init(data, "{ 'integer': 42, 'string': 'c',
> 'string1': 'd', 'string2': 'e' }");
>
> visit_type_UserDefFlatUnion2(v, &tmp, NULL, &err);
> g_assert(err);
You're right. I'll squash it in, thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type
2015-08-07 19:30 ` Eric Blake
@ 2015-08-08 6:24 ` Markus Armbruster
2015-08-31 9:07 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-08 6:24 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> It's first class, because unlike '**', it actually works, i.e. doesn't
>> require 'gen': false.
>>
>> '**' will go away next.
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> ---
>
>> @@ -1039,8 +1040,7 @@ class QAPISchema(object):
>>
>> def _def_builtin_type(self, name, json_type, c_type, c_null):
>> self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type, c_null))
>> - if name != '**':
>> - self._make_array_type(name) # TODO really needed?
>> + self._make_array_type(name) # TODO really needed?
>
> Do we really want to allow ['any'] in schemata? That would imply the
> possibility of a 2D array.
It's 2D-by-trickery. Unlike the existing trick of wrapping the inner
array in a struct, this one isn't visible on the wire, though. It
requires giving up some type checking.
>From a backend point of view, array of any type should just work. We're
already debating whether we want them under PATCH 02, so let's continue
there.
>> +++ b/tests/qapi-schema/qapi-schema-test.json
>> @@ -83,6 +83,8 @@
>> 'returns': 'UserDefTwo' }
>> { 'command': 'user_def_cmd3', 'data': {'a': 'int', '*b': 'int' },
>> 'returns': 'int' }
>> +# note: command name 'guest-sync' chosen to avoid "cannot use built-in" error
>> +{ 'command': 'guest-sync', 'data': { 'arg': 'any' }, 'returns': 'any' }
>>
>
> In particular, if we DO want to allow it, this file should be enhanced
> to include ['any'] in the UserDefNativeListUnion.
Yes, to keep the test case complete.
> As it is, JSON allows mixed-type arrays, but all our uses of QList are
> fixed-type (all elements share the same type); allowing an array of any
> element may prove to be problematic.
anyList is fixed-type, too: the fixed type is QObject * ;)
['any'] isn't ABI until we actually use it in an external interface.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
2015-08-05 20:20 ` Eric Blake
2015-08-05 20:54 ` Eric Blake
@ 2015-08-23 4:17 ` Eric Blake
2015-08-24 11:30 ` Markus Armbruster
2015-09-01 18:40 ` Michael Roth
3 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-23 4:17 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto, mdroth
[-- Attachment #1: Type: text/plain, Size: 6146 bytes --]
On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> Caution, rough edges.
>
>
> The empty object type is used when a command takes no arguments or
> produces no results.
> +++ b/scripts/qapi.py
> @@ -1050,6 +1054,9 @@ class QAPISchema(object):
> ('bool', 'boolean', 'bool', 'false'),
> ('any', 'value', 'QObject' + pointer_suffix , 'NULL')]:
> self._def_builtin_type(*t)
> + self.the_empty_object_type = QAPISchemaObjectType(':empty', None, None,
> + [], None)
> + self._def_entity(self.the_empty_object_type)
We mentioned moving this into its own patch. In particular, I looked at
what it would take to allow anonymous structs for flat union types:
{ 'union': 'Flat', 'base': 'Base',
'discriminator': 'type',
'data': { 'branch1': {},
'branch2': { 'integer': 'int' } } }
but having _make_implicit_object_type() return None caused subsequent
failures. Normalizing to always use the special ':empty' might be worth
doing if you hoist this work earlier; here's the diff I was playing with.
And while at it, should types like 'Abort' and 'ChardevDummy' (which are
both intentionally empty) normalize to the same ':empty' type during
introspection, or is it okay if two separate types with the same (lack
of) member contents can result in separate type numbers in the
introspection?
diff --git i/scripts/qapi-commands.py w/scripts/qapi-commands.py
index 7ff7c31..06e6a1e 100644
--- i/scripts/qapi-commands.py
+++ w/scripts/qapi-commands.py
@@ -76,7 +76,7 @@ def gen_marshal_vars(arg_type, ret_type):
''',
c_type=ret_type.c_type())
- if arg_type:
+ if arg_type and arg_type.members:
ret += mcgen('''
QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
QapiDeallocVisitor *md;
@@ -108,7 +108,7 @@ bool has_%(c_name)s = false;
def gen_marshal_input_visit(arg_type, dealloc=False):
ret = ''
- if not arg_type:
+ if not arg_type or not arg_type.members:
return ret
push_indent()
diff --git i/scripts/qapi.py w/scripts/qapi.py
index 34c2884..133b859 100644
--- i/scripts/qapi.py
+++ w/scripts/qapi.py
@@ -1071,7 +1071,7 @@ class QAPISchema(object):
def _make_implicit_object_type(self, name, role, members):
if not members:
- return None
+ return ':empty'
name = ':obj-%s-%s' % (name, role)
if not self.lookup_entity(name, QAPISchemaObjectType):
self._def_entity(QAPISchemaObjectType(name, None, None,
@@ -1155,17 +1155,17 @@ class QAPISchema(object):
def _def_command(self, expr, info):
name = expr['command']
- data = expr.get('data')
+ data = expr.get('data', {})
rets = expr.get('returns')
gen = expr.get('gen', True)
success_response = expr.get('success-response', True)
- if isinstance(data, OrderedDict):
+ if isinstance(data, dict):
data = self._make_implicit_object_type(name, 'arg',
self._make_members(data))
if isinstance(rets, list):
assert len(rets) == 1
rets = self._make_array_type(rets[0])
- elif isinstance(rets, OrderedDict):
+ elif isinstance(rets, dict):
rets = self._make_implicit_object_type(name, 'ret',
self._make_members(rets))
self._def_entity(QAPISchemaCommand(name, info, data, rets, gen,
@@ -1173,8 +1173,8 @@ class QAPISchema(object):
def _def_event(self, expr, info):
name = expr['event']
- data = expr.get('data')
- if isinstance(data, OrderedDict):
+ data = expr.get('data', {})
+ if isinstance(data, dict):
data = self._make_implicit_object_type(name, 'arg',
self._make_members(data))
self._def_entity(QAPISchemaEvent(name, info, data))
diff --git i/tests/qapi-schema/event-case.out
w/tests/qapi-schema/event-case.out
index cdfd264..6a84fdd 100644
--- i/tests/qapi-schema/event-case.out
+++ w/tests/qapi-schema/event-case.out
@@ -1,2 +1,2 @@
object :empty
-event oops None
+event oops :empty
diff --git i/tests/qapi-schema/indented-expr.out
w/tests/qapi-schema/indented-expr.out
index 226d300..8999a6d 100644
--- i/tests/qapi-schema/indented-expr.out
+++ w/tests/qapi-schema/indented-expr.out
@@ -1,5 +1,5 @@
object :empty
-command eins None -> None
+command eins :empty -> None
gen=True success_response=True
-command zwei None -> None
+command zwei :empty -> None
gen=True success_response=True
diff --git i/tests/qapi-schema/qapi-schema-test.out
w/tests/qapi-schema/qapi-schema-test.out
index 386b057..57cae76 100644
--- i/tests/qapi-schema/qapi-schema-test.out
+++ w/tests/qapi-schema/qapi-schema-test.out
@@ -51,8 +51,8 @@ object :obj-user_def_cmd2-arg
object :obj-user_def_cmd3-arg
member a: int optional=False
member b: int optional=True
-event EVENT_A None
-event EVENT_B None
+event EVENT_A :empty
+event EVENT_B :empty
event EVENT_C :obj-EVENT_C-arg
event EVENT_D :obj-EVENT_D-arg
enum EnumOne ['value1', 'value2', 'value3']
@@ -154,7 +154,7 @@ command __org.qemu_x-command
:obj-__org.qemu_x-command-arg -> __org.qemu_x-Union
gen=True success_response=True
command guest-sync :obj-guest-sync-arg -> any
gen=True success_response=True
-command user_def_cmd None -> None
+command user_def_cmd :empty -> None
gen=True success_response=True
command user_def_cmd1 :obj-user_def_cmd1-arg -> None
gen=True success_response=True
diff --git i/tests/qapi-schema/returns-int.out
w/tests/qapi-schema/returns-int.out
index a2da259..29825e8 100644
--- i/tests/qapi-schema/returns-int.out
+++ w/tests/qapi-schema/returns-int.out
@@ -1,3 +1,3 @@
object :empty
-command guest-get-time None -> int
+command guest-get-time :empty -> int
gen=True success_response=True
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-23 4:17 ` Eric Blake
@ 2015-08-24 11:30 ` Markus Armbruster
2015-08-24 13:07 ` Eric Blake
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-24 11:30 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>> Caution, rough edges.
>>
>
>>
>> The empty object type is used when a command takes no arguments or
>> produces no results.
>
>
>> +++ b/scripts/qapi.py
>
>> @@ -1050,6 +1054,9 @@ class QAPISchema(object):
>> ('bool', 'boolean', 'bool', 'false'),
>> ('any', 'value', 'QObject' + pointer_suffix , 'NULL')]:
>> self._def_builtin_type(*t)
>> + self.the_empty_object_type = QAPISchemaObjectType(':empty', None, None,
>> + [], None)
>> + self._def_entity(self.the_empty_object_type)
>
> We mentioned moving this into its own patch. In particular, I looked at
> what it would take to allow anonymous structs for flat union types:
>
> { 'union': 'Flat', 'base': 'Base',
> 'discriminator': 'type',
> 'data': { 'branch1': {},
> 'branch2': { 'integer': 'int' } } }
We used to support inline structs in a few places, but dropped it
(commit 6b5abc7). Do we really want to add them back in other places?
> but having _make_implicit_object_type() return None caused subsequent
> failures. Normalizing to always use the special ':empty' might be worth
> doing if you hoist this work earlier; here's the diff I was playing with.
I experimented in that direction as well, but didn't like my results
much. I'll look at your experimental patch more closely later.
> And while at it, should types like 'Abort' and 'ChardevDummy' (which are
> both intentionally empty) normalize to the same ':empty' type during
> introspection, or is it okay if two separate types with the same (lack
> of) member contents can result in separate type numbers in the
> introspection?
I think it's okay from an introspection point of view. Client's
shouldn't rely on type names anyway, and we don't (promise to) normalize
structurally identical types. We don't promise not to, either.
Both Abort and ChardevDummy exist only because you need a type to
declare a simple union case. I'd like to explore cleaning up the
convoluted union syntax first. If we then still have a need for empty
structs, we can consider optimizing them.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-24 11:30 ` Markus Armbruster
@ 2015-08-24 13:07 ` Eric Blake
2015-08-24 16:51 ` Eric Blake
2015-08-24 16:55 ` Markus Armbruster
0 siblings, 2 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-24 13:07 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel, mdroth
[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]
On 08/24/2015 05:30 AM, Markus Armbruster wrote:
>> We mentioned moving this into its own patch. In particular, I looked at
>> what it would take to allow anonymous structs for flat union types:
>>
>> { 'union': 'Flat', 'base': 'Base',
>> 'discriminator': 'type',
>> 'data': { 'branch1': {},
This one is particularly handy - it would avoid the need to create dummy
types like 'Abort' that exist solely to satisfy the current qapi grammar.
>> 'branch2': { 'integer': 'int' } } }
This one is a bit harder to justify, but may still be useful.
>
> We used to support inline structs in a few places, but dropped it
> (commit 6b5abc7). Do we really want to add them back in other places?
We dropped nested structs, not anonymous structs.
For comparison, here's an anonymous command struct:
{ 'command': 'foo', 'data': { 'i': 'int' } }
vs. the corresponding named version:
{ 'struct': 'Foo', 'data': { 'i': 'int' } }
{ 'command': 'foo', 'data': 'Foo' }
Where things would NOT be allowed due to our prohibition of nested
structs is:
{ 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
'data': { 'okay': 'str', 'bad': { 'i': 'int' } } }
>
> Both Abort and ChardevDummy exist only because you need a type to
> declare a simple union case. I'd like to explore cleaning up the
> convoluted union syntax first. If we then still have a need for empty
> structs, we can consider optimizing them.
And that's where my patches were headed - by allowing a dict instead of
a type name for the branches of a flat union, the syntax for flat unions
becomes simpler, and allows us to sanely represent a
"no-additional-members" variant without needing 'Abort' as an empty type.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-24 13:07 ` Eric Blake
@ 2015-08-24 16:51 ` Eric Blake
2015-08-24 16:55 ` Markus Armbruster
1 sibling, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-08-24 16:51 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel, mdroth
[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]
On 08/24/2015 07:07 AM, Eric Blake wrote:
> Where things would NOT be allowed due to our prohibition of nested
> structs is:
>
> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
> 'data': { 'okay': 'str', 'bad': { 'i': 'int' } } }
I typed that wrong.
{ 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
'data': { 'branch1': 'NamedTypeOkay',
'branch2': { 'anon-type-okay': 'str' },
'branch3': { 'okay': str', 'bad-nesting': { 'i': 'int' } } } }
and where anonymous types are still okay is that if we add long-hand for
expressing optional types, or otherwise use a dict in place of a type,
we can do:
{ 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
'data': { 'branch1': 'NamedTypeOkay',
'branch2': { 'anon-type-okay': { 'type': 'str',
'optional': true } } } }
,
That is, there is a difference between the dictionary expressing an
anonymous type, and the dictionary expressing additional attributes
belonging to a member of a struct.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-24 13:07 ` Eric Blake
2015-08-24 16:51 ` Eric Blake
@ 2015-08-24 16:55 ` Markus Armbruster
2015-08-24 17:14 ` Eric Blake
1 sibling, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-24 16:55 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/24/2015 05:30 AM, Markus Armbruster wrote:
>
>>> We mentioned moving this into its own patch. In particular, I looked at
>>> what it would take to allow anonymous structs for flat union types:
>>>
>>> { 'union': 'Flat', 'base': 'Base',
>>> 'discriminator': 'type',
>>> 'data': { 'branch1': {},
>
> This one is particularly handy - it would avoid the need to create dummy
> types like 'Abort' that exist solely to satisfy the current qapi grammar.
>
>>> 'branch2': { 'integer': 'int' } } }
>
> This one is a bit harder to justify, but may still be useful.
>
>>
>> We used to support inline structs in a few places, but dropped it
>> (commit 6b5abc7). Do we really want to add them back in other places?
>
> We dropped nested structs, not anonymous structs.
>
> For comparison, here's an anonymous command struct:
>
> { 'command': 'foo', 'data': { 'i': 'int' } }
>
> vs. the corresponding named version:
>
> { 'struct': 'Foo', 'data': { 'i': 'int' } }
> { 'command': 'foo', 'data': 'Foo' }
>
> Where things would NOT be allowed due to our prohibition of nested
> structs is:
>
> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
> 'data': { 'okay': 'str', 'bad': { 'i': 'int' } } }
Our motivation for dropping nested structs was to avoid burning the
'name': {} struct member syntax on a trivial and rarely used
convenience, and instead make it available for a way to specify member
attributes beyond name and type.
Is there a chance we want to define simple union cases with attributes
beyond tag value and type?
I think we have a better chance to answer that question after we clean
non-simple unions.
>> Both Abort and ChardevDummy exist only because you need a type to
>> declare a simple union case. I'd like to explore cleaning up the
>> convoluted union syntax first. If we then still have a need for
>> empty structs, we can consider optimizing them.
>
> And that's where my patches were headed - by allowing a dict instead of
> a type name for the branches of a flat union, the syntax for flat unions
> becomes simpler, and allows us to sanely represent a
> "no-additional-members" variant without needing 'Abort' as an empty type.
Empty cases in flat unions are not a problem: simply don't mention the
tag value.
I'd like to explore doing unions in schema syntax the way we represent
them internally and in introspection. Basically get rid of the "need to
inherit the common members from a base" nonsense.
Once that's done, we can decice whether simple unions are still
worthwhile syntactical sugar.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-24 16:55 ` Markus Armbruster
@ 2015-08-24 17:14 ` Eric Blake
2015-08-25 8:33 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-08-24 17:14 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel, mdroth
[-- Attachment #1: Type: text/plain, Size: 4105 bytes --]
On 08/24/2015 10:55 AM, Markus Armbruster wrote:
> Our motivation for dropping nested structs was to avoid burning the
> 'name': {} struct member syntax on a trivial and rarely used
> convenience, and instead make it available for a way to specify member
> attributes beyond name and type.
>
> Is there a chance we want to define simple union cases with attributes
> beyond tag value and type?
You may have a valid point there. It's hard to predict the future, but
leaving dictionary open for future use is the most extensible approach.
But in the patches I'm currently working on, I had only been adding
support for anonymous types for the branches of flat unions; I
intentionally left simple unions to REQUIRE a type name for the branches
(because of the way we create a wrapper type around the single data
member for introspection purposes).
>
> I think we have a better chance to answer that question after we clean
> non-simple unions.
Well, my proposed cleanup was figuring out a way to explicitly specify
that for a given enum value, we add no additional members to the wire
struct. But there is a possible alternative syntax for that:
{ 'union': 'Union', 'base': 'Base', 'discriminator': 'type',
'data': { 'branch1': 'AdditionalMembers',
'branch2': null } }
which uses 'null' in place of '{}' for the explicitly empty case, while
still requiring a type name for all other branches. I still think that
requiring a user to explicitly list all branches of a union is a nice
fail-safe (if the enum is extended, we are then reminded to update the
union to match) that we don't currently have.
>
>>> Both Abort and ChardevDummy exist only because you need a type to
>>> declare a simple union case. I'd like to explore cleaning up the
>>> convoluted union syntax first. If we then still have a need for
>>> empty structs, we can consider optimizing them.
>>
>> And that's where my patches were headed - by allowing a dict instead of
>> a type name for the branches of a flat union, the syntax for flat unions
>> becomes simpler, and allows us to sanely represent a
>> "no-additional-members" variant without needing 'Abort' as an empty type.
>
> Empty cases in flat unions are not a problem: simply don't mention the
> tag value.
But that's opposite of the direction I want to move, where we require
all branches to be listed, but have a clean way to document the branches
that add no additional members to the variant object.
>
> I'd like to explore doing unions in schema syntax the way we represent
> them internally and in introspection. Basically get rid of the "need to
> inherit the common members from a base" nonsense.
I've already posted patches that would allow:
{ 'union': 'Union', 'base': { ... }, 'discriminator': 'type',
'data': { ... } }
that is, allowing the base fields to be specified inline as an anonymous
struct rather than having to create a one-off named struct for that task.
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02346.html
But there's still the question of whether we want to always tie the
union branches to an explicitly named enum, or whether it is nice to
preserve the current simple union semantics that an implicit enum is
created to cover all branches when an explicit enum type is not named.
Conversely, I still want to get to the point that even a simple union
can optionally document that it reuses an existing enum (along with the
corresponding qapi-generator enforced rules that all enum branches are
properly covered), rather than always being forced to use an implicit
enum type (where mismatches between an intended explicit enum are too
likely).
>
> Once that's done, we can decice whether simple unions are still
> worthwhile syntactical sugar.
Agreed - there's still some room to play with things, and flushing our
existing patch queue to have a stable base to play on will make it a bit
nicer.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-24 17:14 ` Eric Blake
@ 2015-08-25 8:33 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-25 8:33 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/24/2015 10:55 AM, Markus Armbruster wrote:
>
>> Our motivation for dropping nested structs was to avoid burning the
>> 'name': {} struct member syntax on a trivial and rarely used
>> convenience, and instead make it available for a way to specify member
>> attributes beyond name and type.
>>
>> Is there a chance we want to define simple union cases with attributes
>> beyond tag value and type?
>
> You may have a valid point there. It's hard to predict the future, but
> leaving dictionary open for future use is the most extensible approach.
>
> But in the patches I'm currently working on, I had only been adding
> support for anonymous types for the branches of flat unions; I
> intentionally left simple unions to REQUIRE a type name for the branches
> (because of the way we create a wrapper type around the single data
> member for introspection purposes).
I asked only about simple unions, but my question actually applies to
any kind of union. In fact, we could decide to reserve the {} syntax
for extensions in the longhand syntactical form, and still burn it on
convenience in shorthand, sugared forms.
>> I think we have a better chance to answer that question after we clean
>> non-simple unions.
>
> Well, my proposed cleanup was figuring out a way to explicitly specify
> that for a given enum value, we add no additional members to the wire
> struct. But there is a possible alternative syntax for that:
>
> { 'union': 'Union', 'base': 'Base', 'discriminator': 'type',
> 'data': { 'branch1': 'AdditionalMembers',
> 'branch2': null } }
>
> which uses 'null' in place of '{}' for the explicitly empty case, while
> still requiring a type name for all other branches.
Let's revisit this once we've figured out how to clean up union syntax.
> I still think that
> requiring a user to explicitly list all branches of a union is a nice
> fail-safe (if the enum is extended, we are then reminded to update the
> union to match) that we don't currently have.
Missing case reminders are obviously useful for code switching over an
enumeration. They're less useful for data. A forgotten case in code
compiles fine, then fails (often catastrophically) at run time. A
forgotten case in data simply won't compile (assuming a statically
checked language).
>>>> Both Abort and ChardevDummy exist only because you need a type to
>>>> declare a simple union case. I'd like to explore cleaning up the
>>>> convoluted union syntax first. If we then still have a need for
>>>> empty structs, we can consider optimizing them.
>>>
>>> And that's where my patches were headed - by allowing a dict instead of
>>> a type name for the branches of a flat union, the syntax for flat unions
>>> becomes simpler, and allows us to sanely represent a
>>> "no-additional-members" variant without needing 'Abort' as an empty type.
>>
>> Empty cases in flat unions are not a problem: simply don't mention the
>> tag value.
>
> But that's opposite of the direction I want to move, where we require
> all branches to be listed, but have a clean way to document the branches
> that add no additional members to the variant object.
Let's figure out how to clean up union syntax first, and how to do empty
cases second.
>> I'd like to explore doing unions in schema syntax the way we represent
>> them internally and in introspection. Basically get rid of the "need to
>> inherit the common members from a base" nonsense.
>
> I've already posted patches that would allow:
>
> { 'union': 'Union', 'base': { ... }, 'discriminator': 'type',
> 'data': { ... } }
>
> that is, allowing the base fields to be specified inline as an anonymous
> struct rather than having to create a one-off named struct for that task.
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02346.html
In my opinion, the whole 'base' business is a hack to inject additional
common members into a union. If I remember correctly, Kevin did that
just to keep his flat union work minimally invasive. Considering what
it took us to do introspection the not minimally invasive way, I can't
fault him for taking a shortcut.
In my recent KVM Forum talk, I showed the QAPI schema and introspection
value for SchemaInfo. The former is a flat union with a struct base,
i.e. two types connected by a (non-trivial) inheritance relation. The
latter is simpler: a single, straightforward variant record. That's
what I'd like to have in the schema, too.
https://events.linuxfoundation.org/sites/events/files/slides/armbru-qemu-introspection.pdf
> But there's still the question of whether we want to always tie the
> union branches to an explicitly named enum, or whether it is nice to
> preserve the current simple union semantics that an implicit enum is
> created to cover all branches when an explicit enum type is not named.
> Conversely, I still want to get to the point that even a simple union
> can optionally document that it reuses an existing enum (along with the
> corresponding qapi-generator enforced rules that all enum branches are
> properly covered), rather than always being forced to use an implicit
> enum type (where mismatches between an intended explicit enum are too
> likely).
I suspect that once we cut the excess baggage from unions, simple unions
will add only marginal convenience. Perhaps enough to keep them for the
truly simple cases. Probably not enough to complicate them so they can
cover a few more cases.
>> Once that's done, we can decice whether simple unions are still
>> worthwhile syntactical sugar.
>
> Agreed - there's still some room to play with things, and flushing our
> existing patch queue to have a stable base to play on will make it a bit
> nicer.
Flushing the patch queue: yes. Next step: respin this series, non-RFC.
Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type
2015-08-08 6:24 ` Markus Armbruster
@ 2015-08-31 9:07 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-08-31 9:07 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
>>> It's first class, because unlike '**', it actually works, i.e. doesn't
>>> require 'gen': false.
>>>
>>> '**' will go away next.
>>>
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>>> ---
>>
>>> @@ -1039,8 +1040,7 @@ class QAPISchema(object):
>>>
>>> def _def_builtin_type(self, name, json_type, c_type, c_null):
>>> self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type, c_null))
>>> - if name != '**':
>>> - self._make_array_type(name) # TODO really needed?
>>> + self._make_array_type(name) # TODO really needed?
>>
>> Do we really want to allow ['any'] in schemata? That would imply the
>> possibility of a 2D array.
>
> It's 2D-by-trickery. Unlike the existing trick of wrapping the inner
> array in a struct, this one isn't visible on the wire, though. It
> requires giving up some type checking.
>
> From a backend point of view, array of any type should just work. We're
> already debating whether we want them under PATCH 02, so let's continue
> there.
>
>>> +++ b/tests/qapi-schema/qapi-schema-test.json
>>> @@ -83,6 +83,8 @@
>>> 'returns': 'UserDefTwo' }
>>> { 'command': 'user_def_cmd3', 'data': {'a': 'int', '*b': 'int' },
>>> 'returns': 'int' }
>>> +# note: command name 'guest-sync' chosen to avoid "cannot use built-in" error
>>> +{ 'command': 'guest-sync', 'data': { 'arg': 'any' }, 'returns': 'any' }
>>>
>>
>> In particular, if we DO want to allow it, this file should be enhanced
>> to include ['any'] in the UserDefNativeListUnion.
>
> Yes, to keep the test case complete.
Squashing in the obvious patch (appended), R-by retained.
>> As it is, JSON allows mixed-type arrays, but all our uses of QList are
>> fixed-type (all elements share the same type); allowing an array of any
>> element may prove to be problematic.
>
> anyList is fixed-type, too: the fixed type is QObject * ;)
>
> ['any'] isn't ABI until we actually use it in an external interface.
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 6665281..e855018 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -73,7 +73,8 @@
'number': ['number'],
'boolean': ['bool'],
'string': ['str'],
- 'sizes': ['size'] } }
+ 'sizes': ['size'],
+ 'any': ['any'] } }
# testing commands
{ 'command': 'user_def_cmd', 'data': {} }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 49c1197..fbb590f 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -12,6 +12,8 @@ object :obj-__org.qemu_x-command-arg
member b: __org.qemu_x-StructList optional=False
member c: __org.qemu_x-Union2 optional=False
member d: __org.qemu_x-Alt optional=False
+object :obj-anyList-wrapper
+ member data: anyList optional=False
object :obj-boolList-wrapper
member data: boolList optional=False
object :obj-guest-sync-arg
@@ -102,7 +104,8 @@ object UserDefNativeListUnion
case boolean: :obj-boolList-wrapper
case string: :obj-strList-wrapper
case sizes: :obj-sizeList-wrapper
-enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes']
+ case any: :obj-anyList-wrapper
+enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes', 'any']
object UserDefOne
base UserDefZero
member string: str optional=False
--
2.4.3
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-06 5:46 ` Markus Armbruster
@ 2015-08-31 18:09 ` Markus Armbruster
2015-09-03 14:52 ` Eric Blake
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-08-31 18:09 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 08/05/2015 12:23 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> On 08/04/2015 09:57 AM, Markus Armbruster wrote:
>>>>> The QAPI code generators work with a syntax tree (nested dictionaries)
>>>>> plus a few symbol tables (also dictionaries) on the side.
>>>>>
>>
>>>>> +class QAPISchemaArrayType(QAPISchemaType):
>>>>> + def __init__(self, name, info, element_type):
>>>>> + QAPISchemaType.__init__(self, name, info)
>>>>> + assert isinstance(element_type, str)
>>>>> + self.element_type_name = element_type
>>>>> + self.element_type = None
>>>>> + def check(self, schema):
>>>>> + self.element_type = schema.lookup_type(self.element_type_name)
>>>>> + assert self.element_type
>>>>
>>>> Is it worth adding:
>>>>
>>>> assert not isinstance(self.element_type, QAPISchemaArrayType)
>>>>
>>>> since we don't allow 2D arrays?
>>>
>>> If the generators actually rely on it, yes.
>>
>> Hmm. What happens if you do
>> { 'command': 'Foo', 'returns': [ 'intList' ] }
>>
>>>
>>> If it's just an arbitrary schema language restriction, probably no.
>>
>> That's a tough judgment call. We don't currently allow [ [ 'int' ] ],
>> and the [ 'intList' ] hack is gross. On the other hand, I'm having a
>> tough time coming up with technical reasons why we can't do it (arrays
>> as a parameter or return type should work, and 2D arrays just add
>> another layer of '*' to the C code).
>
> Perhaps a quick experiment can decide the nature of the restriction.
The appended experimental frontend patch passes "make check". Looks
like the backends are just fine with arrays of arrays. I'll therefore
refrain from adding "element type isn't array" assertions to backends.
Since there's plenty of QAPI work on list already, I'll shelve this
patch for now. We can revisit nested arrays later.
diff --git a/scripts/qapi.py b/scripts/qapi.py
index d1def74..ba51a03 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -434,11 +434,11 @@ def check_type(expr_info, source, value, allow_array = False,
return
# Check if array type for value is okay
- if isinstance(value, list):
+ while isinstance(value, list):
if not allow_array:
raise QAPIExprError(expr_info,
"%s cannot be an array" % source)
- if len(value) != 1 or not isinstance(value[0], str):
+ if len(value) != 1 or not (isinstance(value[0], str) or isinstance(value[0], list)):
raise QAPIExprError(expr_info,
"%s: array type must contain single type name"
% source)
@@ -1056,6 +1056,9 @@ class QAPISchema(object):
return name
def _make_array_type(self, element_type):
+ if isinstance(element_type, list):
+ assert len(element_type) == 1
+ element_type = self._make_array_type(element_type[0])
name = element_type + 'List'
if not self.lookup_type(name):
self._def_entity(QAPISchemaArrayType(name, None, element_type))
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index e855018..8072026 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -74,6 +74,7 @@
'boolean': ['bool'],
'string': ['str'],
'sizes': ['size'],
+ 'array': [['str']],
'any': ['any'] } }
# testing commands
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index a9c87a0..c467d47 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -37,6 +37,8 @@ object :obj-str-wrapper
member data: str optional=False
object :obj-strList-wrapper
member data: strList optional=False
+object :obj-strListList-wrapper
+ member data: strListList optional=False
object :obj-uint16List-wrapper
member data: uint16List optional=False
object :obj-uint32List-wrapper
@@ -105,8 +107,9 @@ object UserDefNativeListUnion
case boolean: :obj-boolList-wrapper
case string: :obj-strList-wrapper
case sizes: :obj-sizeList-wrapper
+ case array: :obj-strListList-wrapper
case any: :obj-anyList-wrapper
-enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes', 'any']
+enum UserDefNativeListUnionKind ['integer', 's8', 's16', 's32', 's64', 'u8', 'u16', 'u32', 'u64', 'number', 'boolean', 'string', 'sizes', 'array', 'any']
object UserDefOne
base UserDefZero
member string: str optional=False
^ permalink raw reply related [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-06 6:23 ` Markus Armbruster
@ 2015-09-01 13:09 ` Markus Armbruster
2015-09-01 15:48 ` Eric Blake
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-09-01 13:09 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
[...]
>>> +++ b/qapi/introspect.json
>>
>>> +{ 'struct': 'SchemaInfoObject',
>>> + 'data': { 'members': [ 'SchemaInfoObjectMember' ],
>>> + '*tag': 'str',
>>> + '*variants': [ 'SchemaInfoObjectVariant' ] } }
>>
>> I take it that either tag and variants will both be omitted, or both be
>> present. No clean way to represent that in the schema, though.
>
> Needs a comment.
Alternatively, wrap both tag and variants in an object, mandatory there,
then make that object optional. Doesn't feel wortwhile to me, but if
you like it better, I'll reconsider.
[...]
>>> +++ b/scripts/qapi-introspect.py
>>> @@ -0,0 +1,172 @@
>>> +#
>>> +# QAPI introspection generator
>>> +#
>>> +# Copyright (C) 2015 Red Hat, Inc.
>>> +#
>>> +# Authors:
>>> +# Markus Armbruster <armbru@redhat.com>
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2.
>>> +# See the COPYING file in the top-level directory.
>>> +
>>> +from qapi import *
>>> +
>>> +# Caveman's json.dumps() replacement (we're stuck at 2.4)
>>> +# TODO try to use json.dumps() once we get unstuck
>>> +def to_json(obj, level=0):
>>> + if obj == None:
>>> + ret = 'null'
>>> + elif isinstance(obj, str):
>>> + ret = '"' + obj.replace('"', r'\"') + '"'
>>> + elif isinstance(obj, list):
>>> + elts = [to_json(elt, level + 1)
>>> + for elt in obj]
>>
>> No sorting here makes sense (the json-to-string converter can't know
>> whether we are using arrays as sets, vs. in the usual JSON semantics of
>> order-preserving). Thus, if we intend to sort things like enum values
>> or object members by name, that sorting has to be done prior to feeding
>> it to this pretty-printer.
>
> Correct. I didn't, because I ran out of time.
>
>>> + ret = '[' + ', '.join(elts) + ' ]'
>>
>> Here's where I was wondering if you want '[ '.
>
> Yes.
I'll consistently drop the spaces, to match Python's conventions.
[...]
>>> + 'variants': [self._gen_variant(v) for v in variants] }
>>
>> Do you want to add a sort() on variant 'case's here?
>>
>>> +
>>> + def visit_enum_type(self, name, info, values):
>>> + self._gen_json(name, 'enum', { 'values': values })
>>
>> Do you want to add a sort() on values here?
>
> I want to review the introspection schema systematically for sorting
> opportunities.
I'm going to leave this job for later, because I feel we should get the
basics in as soon as possible. We then have the rest of the development
cycle to refine stuff.
[...]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-01 13:09 ` Markus Armbruster
@ 2015-09-01 15:48 ` Eric Blake
0 siblings, 0 replies; 88+ messages in thread
From: Eric Blake @ 2015-09-01 15:48 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel, mdroth
[-- Attachment #1: Type: text/plain, Size: 2041 bytes --]
On 09/01/2015 07:09 AM, Markus Armbruster wrote:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> On 08/04/2015 09:58 AM, Markus Armbruster wrote:
> [...]
>>>> +++ b/qapi/introspect.json
>>>
>>>> +{ 'struct': 'SchemaInfoObject',
>>>> + 'data': { 'members': [ 'SchemaInfoObjectMember' ],
>>>> + '*tag': 'str',
>>>> + '*variants': [ 'SchemaInfoObjectVariant' ] } }
>>>
>>> I take it that either tag and variants will both be omitted, or both be
>>> present. No clean way to represent that in the schema, though.
>>
>> Needs a comment.
>
> Alternatively, wrap both tag and variants in an object, mandatory there,
> then make that object optional. Doesn't feel wortwhile to me, but if
> you like it better, I'll reconsider.
No; the extra nesting it would create in QMP is not worth it.
(Well, maybe we want to think about a qapi extension that would allow
unboxed substructs, where you either get all or none of a sub-struct,
but with all of the members of the substruct passed in QMP at the same
nesting level as the rest of the object; but I'm not sure we need it,
and it certainly is too complex to be worrying about right now while the
basics aren't even in place)
>>>> + def visit_enum_type(self, name, info, values):
>>>> + self._gen_json(name, 'enum', { 'values': values })
>>>
>>> Do you want to add a sort() on values here?
>>
>> I want to review the introspection schema systematically for sorting
>> opportunities.
>
> I'm going to leave this job for later, because I feel we should get the
> basics in as soon as possible. We then have the rest of the development
> cycle to refine stuff.
Sure; we have until 2.5 is released to bake in our final assumptions,
and the sooner we can get in a baseline, the easier it will be to play
with what further changes we want to make on top.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
` (2 preceding siblings ...)
2015-08-23 4:17 ` Eric Blake
@ 2015-09-01 18:40 ` Michael Roth
2015-09-01 19:12 ` Eric Blake
3 siblings, 1 reply; 88+ messages in thread
From: Michael Roth @ 2015-09-01 18:40 UTC (permalink / raw)
To: Markus Armbruster, qemu-devel; +Cc: kwolf, berto
Quoting Markus Armbruster (2015-08-04 10:58:14)
> Caution, rough edges.
>
> qapi/introspect.json defines the introspection schema. It should do
> for uses other than QMP.
> FIXME it's almost entirely devoid of comments.
>
> The introspection schema does not reflect all the rules and
> restrictions that apply to QAPI schemata. A valid QAPI schema has an
> introspection value conforming to the introspection schema, but the
> converse is not true.
>
> Introspection lowers away a number of schema details:
>
> * The built-in types are declared with their JSON type.
>
> TODO Should we map all the integer types to just int?
I don't think we should. If management chooses to handle them in this
generic fashion that's their perogative/problem, but treating all ints
as a single generic type can lead to unexpected results when those values
get passed on to functions expecting, for instance, uint8_t. So QEMU
should do its part to convey this information somehow.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-01 18:40 ` Michael Roth
@ 2015-09-01 19:12 ` Eric Blake
2015-09-01 20:23 ` Michael Roth
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-09-01 19:12 UTC (permalink / raw)
To: Michael Roth, Markus Armbruster, qemu-devel; +Cc: kwolf, berto
[-- Attachment #1: Type: text/plain, Size: 2706 bytes --]
On 09/01/2015 12:40 PM, Michael Roth wrote:
> Quoting Markus Armbruster (2015-08-04 10:58:14)
>> Caution, rough edges.
>>
>> qapi/introspect.json defines the introspection schema. It should do
>> for uses other than QMP.
>> FIXME it's almost entirely devoid of comments.
>>
>> The introspection schema does not reflect all the rules and
>> restrictions that apply to QAPI schemata. A valid QAPI schema has an
>> introspection value conforming to the introspection schema, but the
>> converse is not true.
>>
>> Introspection lowers away a number of schema details:
>>
>> * The built-in types are declared with their JSON type.
>>
>> TODO Should we map all the integer types to just int?
>
> I don't think we should. If management chooses to handle them in this
> generic fashion that's their perogative/problem, but treating all ints
> as a single generic type can lead to unexpected results when those values
> get passed on to functions expecting, for instance, uint8_t. So QEMU
> should do its part to convey this information somehow.
The argument here is that it is always more conservative to start with
less information, where we can later add more information (whether in
the form of type constraints such as uint8_t, or in a different form
such as min/max values) if they prove useful. And QMP already
guarantees that it gives sane error messages for parameters that are out
of range, so the worst behavior a client might see is that a parameter
claiming to be 'int' in the introspection gracefully rejects an attempt
to use '256' as a value because the underlying type was uint8_t.
If we advertise full types now, then the choice of integer type becomes
ABI (switching from 'int8_t' to 'uint8_t' has observable impact in the
introspection) that we are stuck exposing in introspection forever. And
in the past, we have successfully retyped a command with no change to
the wire API (see commit 5fba6c0).
At any rate, patch 31/32 in this series gives stronger arguments for
merging the types for at least the initial implementation; we can always
change our minds later and undo the merging, even if it is after 2.5
when we change our minds.
But the mere fact that you are questioning the idea means that patch 30
and 31 should not be merged (previously, I had argued that separating
the patches didn't make sense; I don't know if Markus was planning to
merge the two based on my recommendation), if only so that reverting the
type merging becomes an easier task if we decide down the road that we
don't need the merged types.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-01 19:12 ` Eric Blake
@ 2015-09-01 20:23 ` Michael Roth
2015-09-02 8:56 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Michael Roth @ 2015-09-01 20:23 UTC (permalink / raw)
To: Eric Blake, Markus Armbruster, qemu-devel; +Cc: kwolf, berto
Quoting Eric Blake (2015-09-01 14:12:24)
> On 09/01/2015 12:40 PM, Michael Roth wrote:
> > Quoting Markus Armbruster (2015-08-04 10:58:14)
> >> Caution, rough edges.
> >>
> >> qapi/introspect.json defines the introspection schema. It should do
> >> for uses other than QMP.
> >> FIXME it's almost entirely devoid of comments.
> >>
> >> The introspection schema does not reflect all the rules and
> >> restrictions that apply to QAPI schemata. A valid QAPI schema has an
> >> introspection value conforming to the introspection schema, but the
> >> converse is not true.
> >>
> >> Introspection lowers away a number of schema details:
> >>
> >> * The built-in types are declared with their JSON type.
> >>
> >> TODO Should we map all the integer types to just int?
> >
> > I don't think we should. If management chooses to handle them in this
> > generic fashion that's their perogative/problem, but treating all ints
> > as a single generic type can lead to unexpected results when those values
> > get passed on to functions expecting, for instance, uint8_t. So QEMU
> > should do its part to convey this information somehow.
>
> The argument here is that it is always more conservative to start with
> less information, where we can later add more information (whether in
> the form of type constraints such as uint8_t, or in a different form
> such as min/max values) if they prove useful. And QMP already
> guarantees that it gives sane error messages for parameters that are out
> of range, so the worst behavior a client might see is that a parameter
> claiming to be 'int' in the introspection gracefully rejects an attempt
> to use '256' as a value because the underlying type was uint8_t.
That's true for many cases, but a 255 uint8_t value being passed in as
a int8_t could still cause unexpected results, for instance.
>
> If we advertise full types now, then the choice of integer type becomes
> ABI (switching from 'int8_t' to 'uint8_t' has observable impact in the
> introspection) that we are stuck exposing in introspection forever. And
> in the past, we have successfully retyped a command with no change to
> the wire API (see commit 5fba6c0).
Some changes can be deemed 'compatible' on a case-by-case, such as with
the above, but those changes are were still documented/published through
the schema.
Why should an introspection interface not convey the same specificity of
information as our schema? It seems inconsistent.
If we do things this way it almost seems like we'd need a disclaimer of
the form "please reference QAPI schema for details on acceptable
integer ranges" in the introspection interface documentation. Which
honestly doesn't seem like that bad an approach if we're up front
about it and unsure how
Though, if it happens too often, I could see that being a pain for
management, but the changes are there and in effect as far as QEMU is
concerned, regardless of whether or not the wire protocol is more
generic with how it represents types. We should document QAPI: JSON
RPC is just a transport that could theoretically get swapped
out for different wire protocol if some need arose in the future.
>
> At any rate, patch 31/32 in this series gives stronger arguments for
> merging the types for at least the initial implementation; we can always
> change our minds later and undo the merging, even if it is after 2.5
> when we change our minds.
31/32 makes a good point about fixed-width types not being all that
appropriate for introspection. I suppose you could consider them a QAPI
shorthand for max/min ranges, but I still think that information should
be conveyed at least in that basic form.
If we have some concern that this would end up being a mistake in the
future, we could possibly introduce it as an optional hint for integer
types. Unfortunately, either way that would complicate the format
somewhat...
>
> But the mere fact that you are questioning the idea means that patch 30
> and 31 should not be merged (previously, I had argued that separating
> the patches didn't make sense; I don't know if Markus was planning to
> merge the two based on my recommendation), if only so that reverting the
> type merging becomes an easier task if we decide down the road that we
> don't need the merged types.
I guess another alternative would be to proceed as things are and just
be clear about the lack of some bits of detail like valid integer
ranges. Would still make introspection useful for discovering the
presence and general types of fields, with the schema serving as
the primary reference for how clients should be implemented.
Not sure if that sort of separation of scope was the initial plan or
not.
>
> --
> Eric Blake eblake redhat com +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-01 20:23 ` Michael Roth
@ 2015-09-02 8:56 ` Markus Armbruster
2015-09-02 16:21 ` Michael Roth
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-09-02 8:56 UTC (permalink / raw)
To: Michael Roth; +Cc: kwolf, berto, qemu-devel
Michael Roth <mdroth@linux.vnet.ibm.com> writes:
> Quoting Eric Blake (2015-09-01 14:12:24)
>> On 09/01/2015 12:40 PM, Michael Roth wrote:
>> > Quoting Markus Armbruster (2015-08-04 10:58:14)
>> >> Caution, rough edges.
>> >>
>> >> qapi/introspect.json defines the introspection schema. It should do
>> >> for uses other than QMP.
>> >> FIXME it's almost entirely devoid of comments.
>> >>
>> >> The introspection schema does not reflect all the rules and
>> >> restrictions that apply to QAPI schemata. A valid QAPI schema has an
>> >> introspection value conforming to the introspection schema, but the
>> >> converse is not true.
>> >>
>> >> Introspection lowers away a number of schema details:
>> >>
>> >> * The built-in types are declared with their JSON type.
>> >>
>> >> TODO Should we map all the integer types to just int?
>> >
>> > I don't think we should. If management chooses to handle them in this
>> > generic fashion that's their perogative/problem, but treating all ints
>> > as a single generic type can lead to unexpected results when those values
>> > get passed on to functions expecting, for instance, uint8_t. So QEMU
>> > should do its part to convey this information somehow.
>>
>> The argument here is that it is always more conservative to start with
>> less information, where we can later add more information (whether in
>> the form of type constraints such as uint8_t, or in a different form
>> such as min/max values) if they prove useful. And QMP already
>> guarantees that it gives sane error messages for parameters that are out
>> of range, so the worst behavior a client might see is that a parameter
>> claiming to be 'int' in the introspection gracefully rejects an attempt
>> to use '256' as a value because the underlying type was uint8_t.
>
> That's true for many cases, but a 255 uint8_t value being passed in as
> a int8_t could still cause unexpected results, for instance.
QMP introspection is about the QMP wire ABI.
QMP core translates from the wire to internal C interfaces. It must
error out unless the C interface can take the value read from the wire.
If the C interface takes int8_t, QMP core must reject 255. No different
from 3.14, "hello", or { "answer": 42 }.
>> If we advertise full types now, then the choice of integer type becomes
>> ABI (switching from 'int8_t' to 'uint8_t' has observable impact in the
>> introspection) that we are stuck exposing in introspection forever. And
>> in the past, we have successfully retyped a command with no change to
>> the wire API (see commit 5fba6c0).
>
> Some changes can be deemed 'compatible' on a case-by-case, such as with
> the above, but those changes are were still documented/published through
> the schema.
>
> Why should an introspection interface not convey the same specificity of
> information as our schema? It seems inconsistent.
The QAPI schema defines both the QMP wire ABI and internal C interfaces.
Introspection should expose only the former, not the latter. Thus,
introspection conveying less information than the full QAPI schema is by
design.
However, there's still a perfectly legitimate question, namely whether
the various integer types should be considered wire ABI or not.
The commit message above asks the question, and the next patch proposes
an answer, namely that they aren't ABI.
> If we do things this way it almost seems like we'd need a disclaimer of
> the form "please reference QAPI schema for details on acceptable
> integer ranges" in the introspection interface documentation. Which
> honestly doesn't seem like that bad an approach if we're up front
> about it and unsure how
We need to point to QAPI schema comments for information on domains
anyway, because the integer types are insufficient for specifying
domains that occur in practice.
Consider a command to create an image, similar to "qemu-img create"
(such a command doesn't currently exist, but it has been discussed). If
format is qcow2, the cluster size must be a power of two between 2^9 and
2^21, and size must be a multiple of the cluster size.
In QAPI, the size would be in bytes, and the type would be plain 'int'.
The cluster size could either be given in bytes (plain 'int' again), or
in width bits (any integer type would do, value must be between 9 and 21
inclusive). In both cases, QAPI's types are useless for specifying the
actual domain.
Other schema languages offer richer means for specifying domains. For
instance, JSON Schema has multipleOf, maximum and exclusiveMaximum,
minimum and exclusiveMinimum[*]. Good enough to specify a cluster size
given in bits (minimum 9, maximum 21). Still incapable of specifying
the rest.
> Though, if it happens too often, I could see that being a pain for
> management, but the changes are there and in effect as far as QEMU is
> concerned, regardless of whether or not the wire protocol is more
> generic with how it represents types. We should document QAPI: JSON
> RPC is just a transport that could theoretically get swapped
> out for different wire protocol if some need arose in the future.
Yes, QMP is just one possible RPC transport for interfaces defined in
QAPI.
However, QMP introspection isn't about QAPI, it's about the QMP wire
ABI.
Say we add another transport, and that transport is binary instead of
text. In that transport, the integer encoding (width and signedness) is
part of the wire ABI. We could then chose a tight coupling to the
internal C interface and pick the same encodings. Or we could chose a
loser coupling and go with int64_t throughout, just like QMP. We could
even pick case by case.
My point is: I strongly recommend to distinguish between *encoding* and
*domain*. Encoding is an artifact of the transport, with the boundary
condition that it needs to be able to hold the complete domain.
In the QMP transport, all integers are encoded the same: decimal ASCII.
>> At any rate, patch 31/32 in this series gives stronger arguments for
>> merging the types for at least the initial implementation; we can always
>> change our minds later and undo the merging, even if it is after 2.5
>> when we change our minds.
>
> 31/32 makes a good point about fixed-width types not being all that
> appropriate for introspection. I suppose you could consider them a QAPI
> shorthand for max/min ranges, but I still think that information should
> be conveyed at least in that basic form.
>
> If we have some concern that this would end up being a mistake in the
> future, we could possibly introduce it as an optional hint for integer
> types. Unfortunately, either way that would complicate the format
> somewhat...
I readily concede that range information could be useful. Two remarks:
1. Let's start introspection simple, and add power as needed.
2. If a need for range information emerges, I'd rather follow JSON
Schema's lead and provide for arbitrary ranges, because that way we
can make them tight: we can specify "9 <= cluster-bits <= 21" instead
of "cluster-bits is uint8_t, and therefore 0 <= cluster-bits <= 255".
>> But the mere fact that you are questioning the idea means that patch 30
>> and 31 should not be merged (previously, I had argued that separating
>> the patches didn't make sense; I don't know if Markus was planning to
>> merge the two based on my recommendation), if only so that reverting the
>> type merging becomes an easier task if we decide down the road that we
>> don't need the merged types.
>
> I guess another alternative would be to proceed as things are and just
> be clear about the lack of some bits of detail like valid integer
> ranges. Would still make introspection useful for discovering the
> presence and general types of fields, with the schema serving as
> the primary reference for how clients should be implemented.
Sounds good to me.
Suggest you review the documentation in PATCH v4 once posted to make
sure it's to your taste there.
> Not sure if that sort of separation of scope was the initial plan or
> not.
I separated RFC PATCH 31+32 from RFC PATCH 30 to facilitate discussion.
I envisaged three possible outcomes for each of the two:
1. We're quite sure that's what we want. Squash the patch into 30.
2. We like it well enough, but we're not quite sure about it. Keep the
patch separate for easy revert.
3. Dumb idea. Drop the patch.
My current opinion is to squash PATCH 31 "qapi-introspect: Map all
integer types to 'int'", but keep PATCH 32 "qapi-introspect: Hide type
names" separate. I hasten to add that both are still very much open to
debate.
[*] http://json-schema.org/latest/json-schema-validation.html#anchor13
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-02 8:56 ` Markus Armbruster
@ 2015-09-02 16:21 ` Michael Roth
2015-09-03 9:26 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Michael Roth @ 2015-09-02 16:21 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel
Quoting Markus Armbruster (2015-09-02 03:56:41)
> Michael Roth <mdroth@linux.vnet.ibm.com> writes:
>
> > Quoting Eric Blake (2015-09-01 14:12:24)
> >> On 09/01/2015 12:40 PM, Michael Roth wrote:
> >> > Quoting Markus Armbruster (2015-08-04 10:58:14)
> >> >> Caution, rough edges.
> >> >>
> >> >> qapi/introspect.json defines the introspection schema. It should do
> >> >> for uses other than QMP.
> >> >> FIXME it's almost entirely devoid of comments.
> >> >>
> >> >> The introspection schema does not reflect all the rules and
> >> >> restrictions that apply to QAPI schemata. A valid QAPI schema has an
> >> >> introspection value conforming to the introspection schema, but the
> >> >> converse is not true.
> >> >>
> >> >> Introspection lowers away a number of schema details:
> >> >>
> >> >> * The built-in types are declared with their JSON type.
> >> >>
> >> >> TODO Should we map all the integer types to just int?
> >> >
> >> > I don't think we should. If management chooses to handle them in this
> >> > generic fashion that's their perogative/problem, but treating all ints
> >> > as a single generic type can lead to unexpected results when those values
> >> > get passed on to functions expecting, for instance, uint8_t. So QEMU
> >> > should do its part to convey this information somehow.
> >>
> >> The argument here is that it is always more conservative to start with
> >> less information, where we can later add more information (whether in
> >> the form of type constraints such as uint8_t, or in a different form
> >> such as min/max values) if they prove useful. And QMP already
> >> guarantees that it gives sane error messages for parameters that are out
> >> of range, so the worst behavior a client might see is that a parameter
> >> claiming to be 'int' in the introspection gracefully rejects an attempt
> >> to use '256' as a value because the underlying type was uint8_t.
> >
> > That's true for many cases, but a 255 uint8_t value being passed in as
> > a int8_t could still cause unexpected results, for instance.
>
> QMP introspection is about the QMP wire ABI.
>
> QMP core translates from the wire to internal C interfaces. It must
> error out unless the C interface can take the value read from the wire.
> If the C interface takes int8_t, QMP core must reject 255. No different
> from 3.14, "hello", or { "answer": 42 }.
>
> >> If we advertise full types now, then the choice of integer type becomes
> >> ABI (switching from 'int8_t' to 'uint8_t' has observable impact in the
> >> introspection) that we are stuck exposing in introspection forever. And
> >> in the past, we have successfully retyped a command with no change to
> >> the wire API (see commit 5fba6c0).
> >
> > Some changes can be deemed 'compatible' on a case-by-case, such as with
> > the above, but those changes are were still documented/published through
> > the schema.
> >
> > Why should an introspection interface not convey the same specificity of
> > information as our schema? It seems inconsistent.
>
> The QAPI schema defines both the QMP wire ABI and internal C interfaces.
> Introspection should expose only the former, not the latter. Thus,
> introspection conveying less information than the full QAPI schema is by
> design.
>
> However, there's still a perfectly legitimate question, namely whether
> the various integer types should be considered wire ABI or not.
>
> The commit message above asks the question, and the next patch proposes
> an answer, namely that they aren't ABI.
>
> > If we do things this way it almost seems like we'd need a disclaimer of
> > the form "please reference QAPI schema for details on acceptable
> > integer ranges" in the introspection interface documentation. Which
> > honestly doesn't seem like that bad an approach if we're up front
> > about it and unsure how
>
> We need to point to QAPI schema comments for information on domains
> anyway, because the integer types are insufficient for specifying
> domains that occur in practice.
>
> Consider a command to create an image, similar to "qemu-img create"
> (such a command doesn't currently exist, but it has been discussed). If
> format is qcow2, the cluster size must be a power of two between 2^9 and
> 2^21, and size must be a multiple of the cluster size.
>
> In QAPI, the size would be in bytes, and the type would be plain 'int'.
> The cluster size could either be given in bytes (plain 'int' again), or
> in width bits (any integer type would do, value must be between 9 and 21
> inclusive). In both cases, QAPI's types are useless for specifying the
> actual domain.
>
> Other schema languages offer richer means for specifying domains. For
> instance, JSON Schema has multipleOf, maximum and exclusiveMaximum,
> minimum and exclusiveMinimum[*]. Good enough to specify a cluster size
> given in bits (minimum 9, maximum 21). Still incapable of specifying
> the rest.
That's a good point, it's not feasible to convey all the
constraints through this sort of interface, so I suppose we
have to draw the line somewhere. Generic integer types is a
fairly logical/convenient stopping point.
>
> > Though, if it happens too often, I could see that being a pain for
> > management, but the changes are there and in effect as far as QEMU is
> > concerned, regardless of whether or not the wire protocol is more
> > generic with how it represents types. We should document QAPI: JSON
> > RPC is just a transport that could theoretically get swapped
> > out for different wire protocol if some need arose in the future.
>
> Yes, QMP is just one possible RPC transport for interfaces defined in
> QAPI.
>
> However, QMP introspection isn't about QAPI, it's about the QMP wire
> ABI.
>
> Say we add another transport, and that transport is binary instead of
> text. In that transport, the integer encoding (width and signedness) is
> part of the wire ABI. We could then chose a tight coupling to the
> internal C interface and pick the same encodings. Or we could chose a
> loser coupling and go with int64_t throughout, just like QMP. We could
> even pick case by case.
So I may have missed the point somewhat then: it's not a single
introspection structure for all conceivable wire protocols,
but instead each exposed wire protocol would have it's own
schema visitor implementation to convey how fields are encoded.
So in this case we're essentially documenting how
QMP{Input,Output}Visitor expect/encode the individual parameters,
and anything beyond that happens at the QAPI/C level. QMP*Visitor
does in fact map fixed-width integers to more generic int64_t/QInt,
then on to JSON primitives, so it makes sense to describe them as
such.
I think we need to be careful that these descriptions are not
interpreted by clients as an alternative to the more-specific
constraints in the QAPI schema though. 'query-schema' seems
a bit misleading in that regard, it appears to be more like
'query-schema-encoding' in function. But not sure it's worth
renaming or anything so long as the documentation is clear.
>
> My point is: I strongly recommend to distinguish between *encoding* and
> *domain*. Encoding is an artifact of the transport, with the boundary
> condition that it needs to be able to hold the complete domain.
>
> In the QMP transport, all integers are encoded the same: decimal ASCII.
>
> >> At any rate, patch 31/32 in this series gives stronger arguments for
> >> merging the types for at least the initial implementation; we can always
> >> change our minds later and undo the merging, even if it is after 2.5
> >> when we change our minds.
> >
> > 31/32 makes a good point about fixed-width types not being all that
> > appropriate for introspection. I suppose you could consider them a QAPI
> > shorthand for max/min ranges, but I still think that information should
> > be conveyed at least in that basic form.
> >
> > If we have some concern that this would end up being a mistake in the
> > future, we could possibly introduce it as an optional hint for integer
> > types. Unfortunately, either way that would complicate the format
> > somewhat...
>
> I readily concede that range information could be useful. Two remarks:
>
> 1. Let's start introspection simple, and add power as needed.
>
> 2. If a need for range information emerges, I'd rather follow JSON
> Schema's lead and provide for arbitrary ranges, because that way we
> can make them tight: we can specify "9 <= cluster-bits <= 21" instead
> of "cluster-bits is uint8_t, and therefore 0 <= cluster-bits <= 255".
Both sound reasonable now that I have a better idea of what's going
on. Range isn't necessary to convey the encoding, so it's okay if
that's not a part of the initial implementation. If it starts to seem
useful then we can add that later. If we think this might be the case
we should maybe take some precautions to allow for extensibility, but
it seems like SchemaInfoBuiltin could be extended in that regard, so
we should be okay.
>
> >> But the mere fact that you are questioning the idea means that patch 30
> >> and 31 should not be merged (previously, I had argued that separating
> >> the patches didn't make sense; I don't know if Markus was planning to
> >> merge the two based on my recommendation), if only so that reverting the
> >> type merging becomes an easier task if we decide down the road that we
> >> don't need the merged types.
> >
> > I guess another alternative would be to proceed as things are and just
> > be clear about the lack of some bits of detail like valid integer
> > ranges. Would still make introspection useful for discovering the
> > presence and general types of fields, with the schema serving as
> > the primary reference for how clients should be implemented.
>
> Sounds good to me.
>
> Suggest you review the documentation in PATCH v4 once posted to make
> sure it's to your taste there.
Thanks, will do.
>
> > Not sure if that sort of separation of scope was the initial plan or
> > not.
>
> I separated RFC PATCH 31+32 from RFC PATCH 30 to facilitate discussion.
> I envisaged three possible outcomes for each of the two:
>
> 1. We're quite sure that's what we want. Squash the patch into 30.
>
> 2. We like it well enough, but we're not quite sure about it. Keep the
> patch separate for easy revert.
>
> 3. Dumb idea. Drop the patch.
>
> My current opinion is to squash PATCH 31 "qapi-introspect: Map all
> integer types to 'int'", but keep PATCH 32 "qapi-introspect: Hide type
> names" separate. I hasten to add that both are still very much open to
> debate.
Makes sense. Not sure what to think of 32 atm, but 31 seems reasonable
to me, whether that be approach 1) or 2) I don't have a strong
opinion on anymore.
>
>
> [*] http://json-schema.org/latest/json-schema-validation.html#anchor13
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-02 16:21 ` Michael Roth
@ 2015-09-03 9:26 ` Markus Armbruster
2015-09-03 14:31 ` Eric Blake
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-09-03 9:26 UTC (permalink / raw)
To: Michael Roth; +Cc: kwolf, berto, qemu-devel
Michael Roth <mdroth@linux.vnet.ibm.com> writes:
> Quoting Markus Armbruster (2015-09-02 03:56:41)
>> Michael Roth <mdroth@linux.vnet.ibm.com> writes:
>>
>> > Quoting Eric Blake (2015-09-01 14:12:24)
>> >> On 09/01/2015 12:40 PM, Michael Roth wrote:
>> >> > Quoting Markus Armbruster (2015-08-04 10:58:14)
>> >> >> Caution, rough edges.
>> >> >>
>> >> >> qapi/introspect.json defines the introspection schema. It should do
>> >> >> for uses other than QMP.
>> >> >> FIXME it's almost entirely devoid of comments.
>> >> >>
>> >> >> The introspection schema does not reflect all the rules and
>> >> >> restrictions that apply to QAPI schemata. A valid QAPI schema has an
>> >> >> introspection value conforming to the introspection schema, but the
>> >> >> converse is not true.
>> >> >>
>> >> >> Introspection lowers away a number of schema details:
>> >> >>
>> >> >> * The built-in types are declared with their JSON type.
>> >> >>
>> >> >> TODO Should we map all the integer types to just int?
>> >> >
>> >> > I don't think we should. If management chooses to handle them in this
>> >> > generic fashion that's their perogative/problem, but treating all ints
>> >> > as a single generic type can lead to unexpected results when those values
>> >> > get passed on to functions expecting, for instance, uint8_t. So QEMU
>> >> > should do its part to convey this information somehow.
>> >>
>> >> The argument here is that it is always more conservative to start with
>> >> less information, where we can later add more information (whether in
>> >> the form of type constraints such as uint8_t, or in a different form
>> >> such as min/max values) if they prove useful. And QMP already
>> >> guarantees that it gives sane error messages for parameters that are out
>> >> of range, so the worst behavior a client might see is that a parameter
>> >> claiming to be 'int' in the introspection gracefully rejects an attempt
>> >> to use '256' as a value because the underlying type was uint8_t.
>> >
>> > That's true for many cases, but a 255 uint8_t value being passed in as
>> > a int8_t could still cause unexpected results, for instance.
>>
>> QMP introspection is about the QMP wire ABI.
>>
>> QMP core translates from the wire to internal C interfaces. It must
>> error out unless the C interface can take the value read from the wire.
>> If the C interface takes int8_t, QMP core must reject 255. No different
>> from 3.14, "hello", or { "answer": 42 }.
>>
>> >> If we advertise full types now, then the choice of integer type becomes
>> >> ABI (switching from 'int8_t' to 'uint8_t' has observable impact in the
>> >> introspection) that we are stuck exposing in introspection forever. And
>> >> in the past, we have successfully retyped a command with no change to
>> >> the wire API (see commit 5fba6c0).
>> >
>> > Some changes can be deemed 'compatible' on a case-by-case, such as with
>> > the above, but those changes are were still documented/published through
>> > the schema.
>> >
>> > Why should an introspection interface not convey the same specificity of
>> > information as our schema? It seems inconsistent.
>>
>> The QAPI schema defines both the QMP wire ABI and internal C interfaces.
>> Introspection should expose only the former, not the latter. Thus,
>> introspection conveying less information than the full QAPI schema is by
>> design.
>>
>> However, there's still a perfectly legitimate question, namely whether
>> the various integer types should be considered wire ABI or not.
>>
>> The commit message above asks the question, and the next patch proposes
>> an answer, namely that they aren't ABI.
>>
>> > If we do things this way it almost seems like we'd need a disclaimer of
>> > the form "please reference QAPI schema for details on acceptable
>> > integer ranges" in the introspection interface documentation. Which
>> > honestly doesn't seem like that bad an approach if we're up front
>> > about it and unsure how
>>
>> We need to point to QAPI schema comments for information on domains
>> anyway, because the integer types are insufficient for specifying
>> domains that occur in practice.
>>
>> Consider a command to create an image, similar to "qemu-img create"
>> (such a command doesn't currently exist, but it has been discussed). If
>> format is qcow2, the cluster size must be a power of two between 2^9 and
>> 2^21, and size must be a multiple of the cluster size.
>>
>> In QAPI, the size would be in bytes, and the type would be plain 'int'.
>> The cluster size could either be given in bytes (plain 'int' again), or
>> in width bits (any integer type would do, value must be between 9 and 21
>> inclusive). In both cases, QAPI's types are useless for specifying the
>> actual domain.
>>
>> Other schema languages offer richer means for specifying domains. For
>> instance, JSON Schema has multipleOf, maximum and exclusiveMaximum,
>> minimum and exclusiveMinimum[*]. Good enough to specify a cluster size
>> given in bits (minimum 9, maximum 21). Still incapable of specifying
>> the rest.
>
> That's a good point, it's not feasible to convey all the
> constraints through this sort of interface, so I suppose we
> have to draw the line somewhere. Generic integer types is a
> fairly logical/convenient stopping point.
>
>>
>> > Though, if it happens too often, I could see that being a pain for
>> > management, but the changes are there and in effect as far as QEMU is
>> > concerned, regardless of whether or not the wire protocol is more
>> > generic with how it represents types. We should document QAPI: JSON
>> > RPC is just a transport that could theoretically get swapped
>> > out for different wire protocol if some need arose in the future.
>>
>> Yes, QMP is just one possible RPC transport for interfaces defined in
>> QAPI.
>>
>> However, QMP introspection isn't about QAPI, it's about the QMP wire
>> ABI.
>>
>> Say we add another transport, and that transport is binary instead of
>> text. In that transport, the integer encoding (width and signedness) is
>> part of the wire ABI. We could then chose a tight coupling to the
>> internal C interface and pick the same encodings. Or we could chose a
>> loser coupling and go with int64_t throughout, just like QMP. We could
>> even pick case by case.
>
> So I may have missed the point somewhat then: it's not a single
> introspection structure for all conceivable wire protocols,
> but instead each exposed wire protocol would have it's own
> schema visitor implementation to convey how fields are encoded.
>
> So in this case we're essentially documenting how
> QMP{Input,Output}Visitor expect/encode the individual parameters,
> and anything beyond that happens at the QAPI/C level. QMP*Visitor
> does in fact map fixed-width integers to more generic int64_t/QInt,
> then on to JSON primitives, so it makes sense to describe them as
> such.
>
> I think we need to be careful that these descriptions are not
> interpreted by clients as an alternative to the more-specific
> constraints in the QAPI schema though. 'query-schema' seems
> a bit misleading in that regard, it appears to be more like
> 'query-schema-encoding' in function. But not sure it's worth
> renaming or anything so long as the documentation is clear.
You have a point: "schema" can mean two related, yet different things.
There's the QAPI schema, and there's the QMP (the wire protocol) schema.
QMP introspection is about the latter, not the former.
If we want to avoid the ambiguity, we could call the command
query-qmp-schema or something.
Renaming query-schema now might confuse people coming from my KVM Forum
talk slightly, but if we can agree on a better name, let's do it.
>> My point is: I strongly recommend to distinguish between *encoding* and
>> *domain*. Encoding is an artifact of the transport, with the boundary
>> condition that it needs to be able to hold the complete domain.
>>
>> In the QMP transport, all integers are encoded the same: decimal ASCII.
>>
>> >> At any rate, patch 31/32 in this series gives stronger arguments for
>> >> merging the types for at least the initial implementation; we can always
>> >> change our minds later and undo the merging, even if it is after 2.5
>> >> when we change our minds.
>> >
>> > 31/32 makes a good point about fixed-width types not being all that
>> > appropriate for introspection. I suppose you could consider them a QAPI
>> > shorthand for max/min ranges, but I still think that information should
>> > be conveyed at least in that basic form.
>> >
>> > If we have some concern that this would end up being a mistake in the
>> > future, we could possibly introduce it as an optional hint for integer
>> > types. Unfortunately, either way that would complicate the format
>> > somewhat...
>>
>> I readily concede that range information could be useful. Two remarks:
>>
>> 1. Let's start introspection simple, and add power as needed.
>>
>> 2. If a need for range information emerges, I'd rather follow JSON
>> Schema's lead and provide for arbitrary ranges, because that way we
>> can make them tight: we can specify "9 <= cluster-bits <= 21" instead
>> of "cluster-bits is uint8_t, and therefore 0 <= cluster-bits <= 255".
>
> Both sound reasonable now that I have a better idea of what's going
> on. Range isn't necessary to convey the encoding, so it's okay if
> that's not a part of the initial implementation. If it starts to seem
> useful then we can add that later. If we think this might be the case
> we should maybe take some precautions to allow for extensibility, but
> it seems like SchemaInfoBuiltin could be extended in that regard, so
> we should be okay.
Yes. I can think of two ways:
1. Create new integer subrange types
Example: a type holding integers between 9 and 21
{ "name": ":int-9-21", "meta-type": "builtin", "json-type": "int",
"minimum": 9, "maximum": 21" }
2. Add optional ranges wherever you can use an integer type
These are SchemaInfoArray's element-type, SchemaInfoObjectMember's
type, SchemaInfoAlternate's type, and maybe SchemaInfoCommand's
ret-type.
Example: an object member holding an integer between 9 and 21
{ "name": "cluster-bits", "type": "int",
"minimum": 9, "maximum": 21" }
In both cases, clients that don't know the new stuff can safely ignore
it.
In case (1), a client making the invalid assumption that there is only
one builtin integer type, and it's called "int" would break.
>> >> But the mere fact that you are questioning the idea means that patch 30
>> >> and 31 should not be merged (previously, I had argued that separating
>> >> the patches didn't make sense; I don't know if Markus was planning to
>> >> merge the two based on my recommendation), if only so that reverting the
>> >> type merging becomes an easier task if we decide down the road that we
>> >> don't need the merged types.
>> >
>> > I guess another alternative would be to proceed as things are and just
>> > be clear about the lack of some bits of detail like valid integer
>> > ranges. Would still make introspection useful for discovering the
>> > presence and general types of fields, with the schema serving as
>> > the primary reference for how clients should be implemented.
>>
>> Sounds good to me.
>>
>> Suggest you review the documentation in PATCH v4 once posted to make
>> sure it's to your taste there.
>
> Thanks, will do.
>
>>
>> > Not sure if that sort of separation of scope was the initial plan or
>> > not.
>>
>> I separated RFC PATCH 31+32 from RFC PATCH 30 to facilitate discussion.
>> I envisaged three possible outcomes for each of the two:
>>
>> 1. We're quite sure that's what we want. Squash the patch into 30.
>>
>> 2. We like it well enough, but we're not quite sure about it. Keep the
>> patch separate for easy revert.
>>
>> 3. Dumb idea. Drop the patch.
>>
>> My current opinion is to squash PATCH 31 "qapi-introspect: Map all
>> integer types to 'int'", but keep PATCH 32 "qapi-introspect: Hide type
>> names" separate. I hasten to add that both are still very much open to
>> debate.
>
> Makes sense. Not sure what to think of 32 atm, but 31 seems reasonable
> to me, whether that be approach 1) or 2) I don't have a strong
> opinion on anymore.
Okay. Current plan is to post RFC v4 for easy review, followed
immediately by a non-RFC v5 that gets us to the same end state, but
without the "just to facilitate review" patches (unless you want them in
git), and with whatever squashing we decide to want for 31+32.
>>
>>
>> [*] http://json-schema.org/latest/json-schema-validation.html#anchor13
>>
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-03 9:26 ` Markus Armbruster
@ 2015-09-03 14:31 ` Eric Blake
2015-09-03 15:59 ` Michael Roth
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-09-03 14:31 UTC (permalink / raw)
To: Markus Armbruster, Michael Roth; +Cc: kwolf, berto, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1185 bytes --]
On 09/03/2015 03:26 AM, Markus Armbruster wrote:
>>
>> I think we need to be careful that these descriptions are not
>> interpreted by clients as an alternative to the more-specific
>> constraints in the QAPI schema though. 'query-schema' seems
>> a bit misleading in that regard, it appears to be more like
>> 'query-schema-encoding' in function. But not sure it's worth
>> renaming or anything so long as the documentation is clear.
>
> You have a point: "schema" can mean two related, yet different things.
> There's the QAPI schema, and there's the QMP (the wire protocol) schema.
> QMP introspection is about the latter, not the former.
>
> If we want to avoid the ambiguity, we could call the command
> query-qmp-schema or something.
>
> Renaming query-schema now might confuse people coming from my KVM Forum
> talk slightly, but if we can agree on a better name, let's do it.
I'm okay if you want to use 'query-qmp-schema'; it's a bit longer, but
more precise, and doesn't cause too much grief to change the name at
this point in the game.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-08-31 18:09 ` Markus Armbruster
@ 2015-09-03 14:52 ` Eric Blake
2015-09-03 16:13 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Eric Blake @ 2015-09-03 14:52 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel, mdroth
[-- Attachment #1: Type: text/plain, Size: 883 bytes --]
On 08/31/2015 12:09 PM, Markus Armbruster wrote:
>>>>>
>>>>> since we don't allow 2D arrays?
>>>>
>>>> If the generators actually rely on it, yes.
> The appended experimental frontend patch passes "make check". Looks
> like the backends are just fine with arrays of arrays. I'll therefore
> refrain from adding "element type isn't array" assertions to backends.
>
> Since there's plenty of QAPI work on list already, I'll shelve this
> patch for now. We can revisit nested arrays later.
Another thing to think about if we allow nested arrays: patch 31/32 maps
'uint8' to 'int', and ['uint8'] to ['int'], but leaves [['uint8']]
untouched. We'd have to add a while loop in qapi-introspect.py to
ensure we drill down through all levels of an array.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-03 14:31 ` Eric Blake
@ 2015-09-03 15:59 ` Michael Roth
2015-09-03 17:01 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Michael Roth @ 2015-09-03 15:59 UTC (permalink / raw)
To: Eric Blake, Markus Armbruster; +Cc: kwolf, berto, qemu-devel
Quoting Eric Blake (2015-09-03 09:31:01)
> On 09/03/2015 03:26 AM, Markus Armbruster wrote:
> >>
> >> I think we need to be careful that these descriptions are not
> >> interpreted by clients as an alternative to the more-specific
> >> constraints in the QAPI schema though. 'query-schema' seems
> >> a bit misleading in that regard, it appears to be more like
> >> 'query-schema-encoding' in function. But not sure it's worth
> >> renaming or anything so long as the documentation is clear.
> >
> > You have a point: "schema" can mean two related, yet different things.
> > There's the QAPI schema, and there's the QMP (the wire protocol) schema.
> > QMP introspection is about the latter, not the former.
> >
> > If we want to avoid the ambiguity, we could call the command
> > query-qmp-schema or something.
> >
> > Renaming query-schema now might confuse people coming from my KVM Forum
> > talk slightly, but if we can agree on a better name, let's do it.
>
> I'm okay if you want to use 'query-qmp-schema'; it's a bit longer, but
> more precise, and doesn't cause too much grief to change the name at
> this point in the game.
I'm also okay with this. It avoids confusion down that road if we
ever introduced, say, a query-ber-schema or something.
I have some minor reservations: I think query-qmp-schema-mapping or
query-qmp-schema-encoding, i.e. "give me information on how the
[QAPI] schema maps to the QMP wire protocol", is more correct.
query-qmp-schema reads like "give me the complete schema for the
QMP wire protocol", which makes it more tempting to treat the
result as a complete schema, rather than a component of what's
defined by the QAPI schema.
But in all likelihood trying to cram all that information into the
command name would probably just be confusing to most, so a simpler
name with clear documentation seems reasonable.
>
> --
> Eric Blake eblake redhat com +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation
2015-09-03 14:52 ` Eric Blake
@ 2015-09-03 16:13 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-09-03 16:13 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, berto, qemu-devel, mdroth
Eric Blake <eblake@redhat.com> writes:
> On 08/31/2015 12:09 PM, Markus Armbruster wrote:
>
>>>>>>
>>>>>> since we don't allow 2D arrays?
>>>>>
>>>>> If the generators actually rely on it, yes.
>
>> The appended experimental frontend patch passes "make check". Looks
>> like the backends are just fine with arrays of arrays. I'll therefore
>> refrain from adding "element type isn't array" assertions to backends.
>>
>> Since there's plenty of QAPI work on list already, I'll shelve this
>> patch for now. We can revisit nested arrays later.
>
> Another thing to think about if we allow nested arrays: patch 31/32 maps
> 'uint8' to 'int', and ['uint8'] to ['int'], but leaves [['uint8']]
> untouched. We'd have to add a while loop in qapi-introspect.py to
> ensure we drill down through all levels of an array.
Yes, that's broken in my experimental patch.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-03 15:59 ` Michael Roth
@ 2015-09-03 17:01 ` Markus Armbruster
2015-09-03 19:59 ` Michael Roth
0 siblings, 1 reply; 88+ messages in thread
From: Markus Armbruster @ 2015-09-03 17:01 UTC (permalink / raw)
To: Michael Roth; +Cc: kwolf, berto, qemu-devel
Michael Roth <mdroth@linux.vnet.ibm.com> writes:
> Quoting Eric Blake (2015-09-03 09:31:01)
>> On 09/03/2015 03:26 AM, Markus Armbruster wrote:
>> >>
>> >> I think we need to be careful that these descriptions are not
>> >> interpreted by clients as an alternative to the more-specific
>> >> constraints in the QAPI schema though. 'query-schema' seems
>> >> a bit misleading in that regard, it appears to be more like
>> >> 'query-schema-encoding' in function. But not sure it's worth
>> >> renaming or anything so long as the documentation is clear.
>> >
>> > You have a point: "schema" can mean two related, yet different things.
>> > There's the QAPI schema, and there's the QMP (the wire protocol) schema.
>> > QMP introspection is about the latter, not the former.
>> >
>> > If we want to avoid the ambiguity, we could call the command
>> > query-qmp-schema or something.
>> >
>> > Renaming query-schema now might confuse people coming from my KVM Forum
>> > talk slightly, but if we can agree on a better name, let's do it.
>>
>> I'm okay if you want to use 'query-qmp-schema'; it's a bit longer, but
>> more precise, and doesn't cause too much grief to change the name at
>> this point in the game.
>
> I'm also okay with this. It avoids confusion down that road if we
> ever introduced, say, a query-ber-schema or something.
>
> I have some minor reservations: I think query-qmp-schema-mapping or
> query-qmp-schema-encoding, i.e. "give me information on how the
> [QAPI] schema maps to the QMP wire protocol", is more correct.
> query-qmp-schema reads like "give me the complete schema for the
> QMP wire protocol", which makes it more tempting to treat the
> result as a complete schema, rather than a component of what's
> defined by the QAPI schema.
Well, it *is* the schema for the QMP wire protocol. Which happens to be
mechanically derived from the QAPI schema.
> But in all likelihood trying to cram all that information into the
> command name would probably just be confusing to most, so a simpler
> name with clear documentation seems reasonable.
I think query-qmp-schema hits the sweet spot between "ambiguous because
it's too short" and "too long because we tried to make it unambiguous".
I'll adopt it in v5.
Regarding clear documentation: we don't have to get it perfect in the
initial commit, but I'm of course happy to address review of RFC PATCH
v4 in v5.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-03 17:01 ` Markus Armbruster
@ 2015-09-03 19:59 ` Michael Roth
2015-09-04 6:39 ` Markus Armbruster
0 siblings, 1 reply; 88+ messages in thread
From: Michael Roth @ 2015-09-03 19:59 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, berto, qemu-devel
Quoting Markus Armbruster (2015-09-03 12:01:20)
> Michael Roth <mdroth@linux.vnet.ibm.com> writes:
>
> > Quoting Eric Blake (2015-09-03 09:31:01)
> >> On 09/03/2015 03:26 AM, Markus Armbruster wrote:
> >> >>
> >> >> I think we need to be careful that these descriptions are not
> >> >> interpreted by clients as an alternative to the more-specific
> >> >> constraints in the QAPI schema though. 'query-schema' seems
> >> >> a bit misleading in that regard, it appears to be more like
> >> >> 'query-schema-encoding' in function. But not sure it's worth
> >> >> renaming or anything so long as the documentation is clear.
> >> >
> >> > You have a point: "schema" can mean two related, yet different things.
> >> > There's the QAPI schema, and there's the QMP (the wire protocol) schema.
> >> > QMP introspection is about the latter, not the former.
> >> >
> >> > If we want to avoid the ambiguity, we could call the command
> >> > query-qmp-schema or something.
> >> >
> >> > Renaming query-schema now might confuse people coming from my KVM Forum
> >> > talk slightly, but if we can agree on a better name, let's do it.
> >>
> >> I'm okay if you want to use 'query-qmp-schema'; it's a bit longer, but
> >> more precise, and doesn't cause too much grief to change the name at
> >> this point in the game.
> >
> > I'm also okay with this. It avoids confusion down that road if we
> > ever introduced, say, a query-ber-schema or something.
> >
> > I have some minor reservations: I think query-qmp-schema-mapping or
> > query-qmp-schema-encoding, i.e. "give me information on how the
> > [QAPI] schema maps to the QMP wire protocol", is more correct.
> > query-qmp-schema reads like "give me the complete schema for the
> > QMP wire protocol", which makes it more tempting to treat the
> > result as a complete schema, rather than a component of what's
> > defined by the QAPI schema.
>
> Well, it *is* the schema for the QMP wire protocol. Which happens to be
> mechanically derived from the QAPI schema.
True, but the separation from QMP wire schema and QAPI interface schema
wouldn't be clear to most users. It would be fairly tempting/reasonable
for the author of a QMP client to assume the "QMP schema" was the
definitive source for how a client should interact with QEMU without
realizing or finding the need to reference the QAPI schema for more
specific constraints (like integer ranges).
I'm somewhat harping on this because that's exactly the misconception
I had when I first looked at this series, which is why the loss of
information like integer ranges didn't seem acceptable to me initially.
But I don't really have a problem with query-qmp-schema, the naming
comments are more a suggestion of the sort of confusion we should
try to clear up in the accompanying documentation.
>
> > But in all likelihood trying to cram all that information into the
> > command name would probably just be confusing to most, so a simpler
> > name with clear documentation seems reasonable.
>
> I think query-qmp-schema hits the sweet spot between "ambiguous because
> it's too short" and "too long because we tried to make it unambiguous".
> I'll adopt it in v5.
>
> Regarding clear documentation: we don't have to get it perfect in the
> initial commit, but I'm of course happy to address review of RFC PATCH
> v4 in v5.
>
Thanks, sounds reasonable to me.
^ permalink raw reply [flat|nested] 88+ messages in thread
* Re: [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection
2015-09-03 19:59 ` Michael Roth
@ 2015-09-04 6:39 ` Markus Armbruster
0 siblings, 0 replies; 88+ messages in thread
From: Markus Armbruster @ 2015-09-04 6:39 UTC (permalink / raw)
To: Michael Roth; +Cc: kwolf, berto, qemu-devel
Michael Roth <mdroth@linux.vnet.ibm.com> writes:
> Quoting Markus Armbruster (2015-09-03 12:01:20)
>> Michael Roth <mdroth@linux.vnet.ibm.com> writes:
>>
>> > Quoting Eric Blake (2015-09-03 09:31:01)
>> >> On 09/03/2015 03:26 AM, Markus Armbruster wrote:
>> >> >>
>> >> >> I think we need to be careful that these descriptions are not
>> >> >> interpreted by clients as an alternative to the more-specific
>> >> >> constraints in the QAPI schema though. 'query-schema' seems
>> >> >> a bit misleading in that regard, it appears to be more like
>> >> >> 'query-schema-encoding' in function. But not sure it's worth
>> >> >> renaming or anything so long as the documentation is clear.
>> >> >
>> >> > You have a point: "schema" can mean two related, yet different things.
>> >> > There's the QAPI schema, and there's the QMP (the wire protocol) schema.
>> >> > QMP introspection is about the latter, not the former.
>> >> >
>> >> > If we want to avoid the ambiguity, we could call the command
>> >> > query-qmp-schema or something.
>> >> >
>> >> > Renaming query-schema now might confuse people coming from my KVM Forum
>> >> > talk slightly, but if we can agree on a better name, let's do it.
>> >>
>> >> I'm okay if you want to use 'query-qmp-schema'; it's a bit longer, but
>> >> more precise, and doesn't cause too much grief to change the name at
>> >> this point in the game.
>> >
>> > I'm also okay with this. It avoids confusion down that road if we
>> > ever introduced, say, a query-ber-schema or something.
>> >
>> > I have some minor reservations: I think query-qmp-schema-mapping or
>> > query-qmp-schema-encoding, i.e. "give me information on how the
>> > [QAPI] schema maps to the QMP wire protocol", is more correct.
>> > query-qmp-schema reads like "give me the complete schema for the
>> > QMP wire protocol", which makes it more tempting to treat the
>> > result as a complete schema, rather than a component of what's
>> > defined by the QAPI schema.
>>
>> Well, it *is* the schema for the QMP wire protocol. Which happens to be
>> mechanically derived from the QAPI schema.
>
> True, but the separation from QMP wire schema and QAPI interface schema
> wouldn't be clear to most users. It would be fairly tempting/reasonable
> for the author of a QMP client to assume the "QMP schema" was the
> definitive source for how a client should interact with QEMU without
> realizing or finding the need to reference the QAPI schema for more
> specific constraints (like integer ranges).
The constraints not expressed in the QMP schema generally aren't
expressed in the QAPI schema either, only its comments.
Nevertheless, I quite agree our documentation should refer readers to
the QAPI schema, because it's the source code, and as such it's written
for humans. The value of query-qmp-schema is just for machines.
> I'm somewhat harping on this because that's exactly the misconception
> I had when I first looked at this series, which is why the loss of
> information like integer ranges didn't seem acceptable to me initially.
>
> But I don't really have a problem with query-qmp-schema, the naming
> comments are more a suggestion of the sort of confusion we should
> try to clear up in the accompanying documentation.
>
>>
>> > But in all likelihood trying to cram all that information into the
>> > command name would probably just be confusing to most, so a simpler
>> > name with clear documentation seems reasonable.
>>
>> I think query-qmp-schema hits the sweet spot between "ambiguous because
>> it's too short" and "too long because we tried to make it unambiguous".
>> I'll adopt it in v5.
>>
>> Regarding clear documentation: we don't have to get it perfect in the
>> initial commit, but I'm of course happy to address review of RFC PATCH
>> v4 in v5.
>>
>
> Thanks, sounds reasonable to me.
Okay, we got a plan. Thanks!
^ permalink raw reply [flat|nested] 88+ messages in thread
end of thread, other threads:[~2015-09-04 6:40 UTC | newest]
Thread overview: 88+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-08-04 15:57 [Qemu-devel] [PATCH RFC v3 00/32] qapi: QMP introspection Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 01/32] qapi: Rename class QAPISchema to QAPISchemaParser Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 02/32] qapi: New QAPISchema intermediate reperesentation Markus Armbruster
2015-08-04 21:53 ` Eric Blake
2015-08-05 6:23 ` Markus Armbruster
2015-08-05 14:27 ` Eric Blake
2015-08-06 5:46 ` Markus Armbruster
2015-08-31 18:09 ` Markus Armbruster
2015-09-03 14:52 ` Eric Blake
2015-09-03 16:13 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 03/32] qapi: QAPISchema code generation helper methods Markus Armbruster
2015-08-04 22:18 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 04/32] qapi: New QAPISchemaVisitor Markus Armbruster
2015-08-04 22:26 ` Eric Blake
2015-08-05 6:24 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 05/32] tests/qapi-schema: Convert test harness to QAPISchemaVisitor Markus Armbruster
2015-08-04 22:35 ` Eric Blake
2015-08-05 6:26 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 06/32] qapi: Split up some typedefs to ease review Markus Armbruster
2015-08-04 22:37 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 07/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
2015-08-04 22:54 ` Eric Blake
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 08/32] Revert "qapi: Generate comments to simplify splitting for review" Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 09/32] Revert "qapi: Split up some typedefs to ease review" Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 10/32] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions Markus Armbruster
2015-08-05 15:15 ` Eric Blake
2015-08-06 5:50 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 11/32] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs Markus Armbruster
2015-08-05 16:03 ` Eric Blake
2015-08-06 22:53 ` Eric Blake
2015-08-08 6:07 ` Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 12/32] qapi-commands: Convert to QAPISchemaVisitor Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 13/32] qapi: De-duplicate enum code generation Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 14/32] qapi-event: Eliminate global variable event_enum_value Markus Armbruster
2015-08-04 15:57 ` [Qemu-devel] [PATCH RFC v3 15/32] qapi-event: Convert to QAPISchemaVisitor, fixing data with base Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 16/32] qapi: Generate comments to simplify splitting for review Markus Armbruster
2015-08-04 23:02 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 17/32] Revert "qapi: Generate comments to simplify splitting for review" Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 18/32] qapi: Replace dirty is_c_ptr() by method c_null() Markus Armbruster
2015-08-05 16:13 ` Eric Blake
2015-08-06 5:52 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 19/32] qapi: Clean up after recent conversions to QAPISchemaVisitor Markus Armbruster
2015-08-05 16:29 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 20/32] qapi-visit: Rearrange code a bit Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 21/32] qapi-commands: Rearrange code Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 22/32] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO() Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 23/32] qapi: De-duplicate parameter list generation Markus Armbruster
2015-08-05 17:00 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 24/32] qapi-commands: De-duplicate output marshaling functions Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 25/32] qapi: Improve built-in type documentation Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 26/32] qapi: Introduce a first class 'any' type Markus Armbruster
2015-08-07 19:30 ` Eric Blake
2015-08-08 6:24 ` Markus Armbruster
2015-08-31 9:07 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 27/32] qom: Don't use 'gen': false for qom-get, qom-set, object-add Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 28/32] qapi-schema: Fix up misleading specification of netdev_add Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 29/32] qapi: Pseudo-type '**' is now unused, drop it Markus Armbruster
2015-08-05 17:13 ` Eric Blake
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 30/32] qapi: New QMP command query-schema for QMP schema introspection Markus Armbruster
2015-08-05 20:20 ` Eric Blake
2015-08-06 6:23 ` Markus Armbruster
2015-09-01 13:09 ` Markus Armbruster
2015-09-01 15:48 ` Eric Blake
2015-08-05 20:54 ` Eric Blake
2015-08-06 6:47 ` Markus Armbruster
2015-08-23 4:17 ` Eric Blake
2015-08-24 11:30 ` Markus Armbruster
2015-08-24 13:07 ` Eric Blake
2015-08-24 16:51 ` Eric Blake
2015-08-24 16:55 ` Markus Armbruster
2015-08-24 17:14 ` Eric Blake
2015-08-25 8:33 ` Markus Armbruster
2015-09-01 18:40 ` Michael Roth
2015-09-01 19:12 ` Eric Blake
2015-09-01 20:23 ` Michael Roth
2015-09-02 8:56 ` Markus Armbruster
2015-09-02 16:21 ` Michael Roth
2015-09-03 9:26 ` Markus Armbruster
2015-09-03 14:31 ` Eric Blake
2015-09-03 15:59 ` Michael Roth
2015-09-03 17:01 ` Markus Armbruster
2015-09-03 19:59 ` Michael Roth
2015-09-04 6:39 ` Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 31/32] qapi-introspect: Map all integer types to 'int' Markus Armbruster
2015-08-04 15:58 ` [Qemu-devel] [PATCH RFC v3 32/32] qapi-introspect: Hide type names Markus Armbruster
2015-08-05 21:06 ` Eric Blake
2015-08-05 21:50 ` Eric Blake
2015-08-06 6:49 ` Markus Armbruster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.