* [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs
@ 2015-03-24 20:03 Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
` (29 more replies)
0 siblings, 30 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
After several months of sitting on this, I finally made progress
on it. Let's get it in 2.4, and I promise to kick out a v6
(if needed) with much less delay.
We want to eventually allow qapi defaults, by making:
'data':{'*flag':'bool'}
shorthand for:
'data':{'flag':{'type':'bool', 'optional':true}}
so that the default can be specified:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true}}
This series does not quite get us there, but it DOES do a number
of other things. It gets rid of the three uses of nested inline
structs, changes anonymous unions to use a specific 'alternate'
metatype rather than abusing 'union', and fixes lots of other
parser bugs found while designing the testsuite. The testsuite
changes make the bulk of this series, with a repeating pattern
of writing tests that expose weaknesses in the old parser, then
beefing up the generator to catch the problem during the initial
parse rather than choking with an obscure python message or even
causing a C compilation failure.
v4 was here:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg04022.html
Changes since then: lots of refactoring; 8 new commits; lots of
testsuite additions; address Markus' review comments where I
could. However, so much has changed, and so much time elapsed,
that it is probably easier to just review this series fresh than
to try and compare it to earlier versions.
Eric Blake (27):
qapi: Document type-safety considerations
qapi: Fix generation of 'size' builtin type
qapi: Require ASCII in schema
qapi: Add some enum tests
qapi: Better error messages for bad enums
qapi: Add some union tests
qapi: Simplify tests of simple unions
qapi: Better error messages for bad unions
qapi: Prepare for catching more semantic parse errors
qapi: Segregate anonymous unions into alternates in generator
qapi: Rename anonymous union type in test
qapi: Introduce 'alternate' to replace anonymous union
qapi: Add some expr tests
qapi: Better error messages for bad expressions
qapi: Add tests of redefined expressions
qapi: Better error messages for duplicated expressions
qapi: Unify type bypass and add tests
qapi: Add some type check tests
qapi: More rigourous checking of types
qapi: Require valid names
qapi: Whitelist commands that don't return dictionary
qapi: More rigorous checking for type safety bypass
qapi: Merge UserDefTwo and UserDefNested in tests
qapi: Drop tests for inline nested structs
qapi: Drop inline nested type in query-version
qapi: Drop inline nested types in query-pci
qapi: Drop support for inline nested types
Fam Zheng (1):
qapi: Allow true, false and null in schema json
docs/qapi-code-gen.txt | 347 ++++++++++++---
docs/qmp/qmp-spec.txt | 92 +++-
hmp.c | 28 +-
hw/pci/pci.c | 42 +-
qapi-schema.json | 104 +++--
qapi/block-core.json | 6 +-
qapi/common.json | 26 +-
qga/qapi-schema.json | 8 +-
qmp.c | 9 +-
scripts/qapi-commands.py | 8 +-
scripts/qapi-event.py | 4 +-
scripts/qapi-types.py | 61 ++-
scripts/qapi-visit.py | 75 ++--
scripts/qapi.py | 469 +++++++++++++++++----
tests/Makefile | 32 +-
tests/qapi-schema/alternate-array.err | 1 +
tests/qapi-schema/alternate-array.exit | 1 +
tests/qapi-schema/alternate-array.json | 6 +
tests/qapi-schema/alternate-array.out | 0
tests/qapi-schema/alternate-base.err | 1 +
tests/qapi-schema/alternate-base.exit | 1 +
tests/qapi-schema/alternate-base.json | 6 +
tests/qapi-schema/alternate-base.out | 0
tests/qapi-schema/alternate-clash.err | 1 +
tests/qapi-schema/alternate-clash.exit | 1 +
tests/qapi-schema/alternate-clash.json | 3 +
tests/qapi-schema/alternate-clash.out | 0
tests/qapi-schema/alternate-conflict-dict.err | 1 +
tests/qapi-schema/alternate-conflict-dict.exit | 1 +
tests/qapi-schema/alternate-conflict-dict.json | 8 +
tests/qapi-schema/alternate-conflict-dict.out | 0
tests/qapi-schema/alternate-conflict-string.err | 1 +
tests/qapi-schema/alternate-conflict-string.exit | 1 +
tests/qapi-schema/alternate-conflict-string.json | 7 +
tests/qapi-schema/alternate-conflict-string.out | 0
tests/qapi-schema/alternate-good.err | 0
tests/qapi-schema/alternate-good.exit | 1 +
tests/qapi-schema/alternate-good.json | 9 +
tests/qapi-schema/alternate-good.out | 6 +
tests/qapi-schema/alternate-nested.err | 1 +
tests/qapi-schema/alternate-nested.exit | 1 +
tests/qapi-schema/alternate-nested.json | 5 +
tests/qapi-schema/alternate-nested.out | 0
tests/qapi-schema/alternate-unknown.err | 1 +
tests/qapi-schema/alternate-unknown.exit | 1 +
tests/qapi-schema/alternate-unknown.json | 3 +
tests/qapi-schema/alternate-unknown.out | 0
tests/qapi-schema/bad-base.err | 1 +
tests/qapi-schema/bad-base.exit | 1 +
tests/qapi-schema/bad-base.json | 3 +
tests/qapi-schema/bad-base.out | 0
tests/qapi-schema/bad-ident.err | 1 +
tests/qapi-schema/bad-ident.exit | 1 +
tests/qapi-schema/bad-ident.json | 3 +
tests/qapi-schema/bad-ident.out | 0
tests/qapi-schema/bad-type-bool.err | 1 +
tests/qapi-schema/bad-type-bool.exit | 1 +
tests/qapi-schema/bad-type-bool.json | 2 +
tests/qapi-schema/bad-type-bool.out | 0
tests/qapi-schema/bad-type-dict.err | 1 +
tests/qapi-schema/bad-type-dict.exit | 1 +
tests/qapi-schema/bad-type-dict.json | 2 +
tests/qapi-schema/bad-type-dict.out | 0
tests/qapi-schema/bad-type-int.err | 1 +
tests/qapi-schema/bad-type-int.exit | 1 +
tests/qapi-schema/bad-type-int.json | 3 +
tests/qapi-schema/bad-type-int.out | 0
tests/qapi-schema/command-int.err | 1 +
tests/qapi-schema/command-int.exit | 1 +
tests/qapi-schema/command-int.json | 3 +
tests/qapi-schema/command-int.out | 0
tests/qapi-schema/data-array-empty.err | 1 +
tests/qapi-schema/data-array-empty.exit | 1 +
tests/qapi-schema/data-array-empty.json | 2 +
tests/qapi-schema/data-array-empty.out | 0
tests/qapi-schema/data-array-unknown.err | 1 +
tests/qapi-schema/data-array-unknown.exit | 1 +
tests/qapi-schema/data-array-unknown.json | 2 +
tests/qapi-schema/data-array-unknown.out | 0
tests/qapi-schema/data-int.err | 1 +
tests/qapi-schema/data-int.exit | 1 +
tests/qapi-schema/data-int.json | 2 +
tests/qapi-schema/data-int.out | 0
tests/qapi-schema/data-member-array-bad.err | 1 +
tests/qapi-schema/data-member-array-bad.exit | 1 +
tests/qapi-schema/data-member-array-bad.json | 2 +
tests/qapi-schema/data-member-array-bad.out | 0
tests/qapi-schema/data-member-array.err | 0
tests/qapi-schema/data-member-array.exit | 1 +
tests/qapi-schema/data-member-array.json | 4 +
tests/qapi-schema/data-member-array.out | 5 +
tests/qapi-schema/data-member-unknown.err | 1 +
tests/qapi-schema/data-member-unknown.exit | 1 +
tests/qapi-schema/data-member-unknown.json | 2 +
tests/qapi-schema/data-member-unknown.out | 0
tests/qapi-schema/data-unknown.err | 1 +
tests/qapi-schema/data-unknown.exit | 1 +
tests/qapi-schema/data-unknown.json | 2 +
tests/qapi-schema/data-unknown.out | 0
tests/qapi-schema/double-data.err | 1 +
tests/qapi-schema/double-data.exit | 1 +
tests/qapi-schema/double-data.json | 2 +
tests/qapi-schema/double-data.out | 0
tests/qapi-schema/double-type.err | 1 +
tests/qapi-schema/double-type.exit | 1 +
tests/qapi-schema/double-type.json | 2 +
tests/qapi-schema/double-type.out | 0
tests/qapi-schema/enum-clash-member.err | 1 +
tests/qapi-schema/enum-clash-member.exit | 1 +
tests/qapi-schema/enum-clash-member.json | 2 +
tests/qapi-schema/enum-clash-member.out | 0
tests/qapi-schema/enum-dict-member.err | 1 +
tests/qapi-schema/enum-dict-member.exit | 1 +
tests/qapi-schema/enum-dict-member.json | 2 +
tests/qapi-schema/enum-dict-member.out | 0
tests/qapi-schema/enum-empty.err | 0
tests/qapi-schema/enum-empty.exit | 1 +
tests/qapi-schema/enum-empty.json | 2 +
tests/qapi-schema/enum-empty.out | 3 +
tests/qapi-schema/enum-int-member.err | 1 +
tests/qapi-schema/enum-int-member.exit | 1 +
tests/qapi-schema/enum-int-member.json | 3 +
tests/qapi-schema/enum-int-member.out | 0
tests/qapi-schema/enum-max-member.err | 1 +
tests/qapi-schema/enum-max-member.exit | 1 +
tests/qapi-schema/enum-max-member.json | 3 +
tests/qapi-schema/enum-max-member.out | 0
tests/qapi-schema/enum-missing-data.err | 1 +
tests/qapi-schema/enum-missing-data.exit | 1 +
tests/qapi-schema/enum-missing-data.json | 2 +
tests/qapi-schema/enum-missing-data.out | 0
tests/qapi-schema/enum-union-clash.err | 1 +
tests/qapi-schema/enum-union-clash.exit | 1 +
tests/qapi-schema/enum-union-clash.json | 4 +
tests/qapi-schema/enum-union-clash.out | 0
tests/qapi-schema/enum-wrong-data.err | 1 +
tests/qapi-schema/enum-wrong-data.exit | 1 +
tests/qapi-schema/enum-wrong-data.json | 2 +
tests/qapi-schema/enum-wrong-data.out | 0
tests/qapi-schema/event-case.err | 1 +
tests/qapi-schema/event-case.exit | 1 +
tests/qapi-schema/event-case.json | 2 +
tests/qapi-schema/event-case.out | 0
tests/qapi-schema/event-max.err | 1 +
tests/qapi-schema/event-max.exit | 1 +
tests/qapi-schema/event-max.json | 2 +
tests/qapi-schema/event-max.out | 0
tests/qapi-schema/event-nest-struct.err | 2 +-
tests/qapi-schema/flat-union-bad-base.err | 1 +
tests/qapi-schema/flat-union-bad-base.exit | 1 +
tests/qapi-schema/flat-union-bad-base.json | 13 +
tests/qapi-schema/flat-union-bad-base.out | 0
tests/qapi-schema/flat-union-bad-discriminator.err | 1 +
.../qapi-schema/flat-union-bad-discriminator.exit | 1 +
.../qapi-schema/flat-union-bad-discriminator.json | 15 +
tests/qapi-schema/flat-union-bad-discriminator.out | 0
tests/qapi-schema/flat-union-base-union.err | 1 +
tests/qapi-schema/flat-union-base-union.exit | 1 +
tests/qapi-schema/flat-union-base-union.json | 15 +
tests/qapi-schema/flat-union-base-union.out | 0
tests/qapi-schema/flat-union-no-base.err | 2 +-
tests/qapi-schema/flat-union-no-base.json | 7 +-
.../flat-union-optional-discriminator.err | 1 +
.../flat-union-optional-discriminator.exit | 1 +
.../flat-union-optional-discriminator.json | 9 +
.../flat-union-optional-discriminator.out | 0
tests/qapi-schema/indented-expr.json | 4 +-
tests/qapi-schema/indented-expr.out | 2 +-
tests/qapi-schema/missing-type.err | 1 +
tests/qapi-schema/missing-type.exit | 1 +
tests/qapi-schema/missing-type.json | 2 +
tests/qapi-schema/missing-type.out | 0
tests/qapi-schema/nested-struct-data.err | 1 +
tests/qapi-schema/nested-struct-data.exit | 1 +
tests/qapi-schema/nested-struct-data.json | 4 +
tests/qapi-schema/nested-struct-data.out | 0
tests/qapi-schema/nested-struct-returns.err | 1 +
tests/qapi-schema/nested-struct-returns.exit | 1 +
tests/qapi-schema/nested-struct-returns.json | 3 +
tests/qapi-schema/nested-struct-returns.out | 0
tests/qapi-schema/qapi-schema-test.json | 28 +-
tests/qapi-schema/qapi-schema-test.out | 18 +-
tests/qapi-schema/redefined-builtin.err | 1 +
tests/qapi-schema/redefined-builtin.exit | 1 +
tests/qapi-schema/redefined-builtin.json | 2 +
tests/qapi-schema/redefined-builtin.out | 0
tests/qapi-schema/redefined-command.err | 1 +
tests/qapi-schema/redefined-command.exit | 1 +
tests/qapi-schema/redefined-command.json | 3 +
tests/qapi-schema/redefined-command.out | 0
tests/qapi-schema/redefined-event.err | 1 +
tests/qapi-schema/redefined-event.exit | 1 +
tests/qapi-schema/redefined-event.json | 3 +
tests/qapi-schema/redefined-event.out | 0
tests/qapi-schema/redefined-type.err | 1 +
tests/qapi-schema/redefined-type.exit | 1 +
tests/qapi-schema/redefined-type.json | 3 +
tests/qapi-schema/redefined-type.out | 0
tests/qapi-schema/returns-alternate.err | 1 +
tests/qapi-schema/returns-alternate.exit | 1 +
tests/qapi-schema/returns-alternate.json | 3 +
tests/qapi-schema/returns-alternate.out | 0
tests/qapi-schema/returns-array-bad.err | 1 +
tests/qapi-schema/returns-array-bad.exit | 1 +
tests/qapi-schema/returns-array-bad.json | 2 +
tests/qapi-schema/returns-array-bad.out | 0
tests/qapi-schema/returns-int.err | 0
tests/qapi-schema/returns-int.exit | 1 +
tests/qapi-schema/returns-int.json | 3 +
tests/qapi-schema/returns-int.out | 3 +
tests/qapi-schema/returns-unknown.err | 1 +
tests/qapi-schema/returns-unknown.exit | 1 +
tests/qapi-schema/returns-unknown.json | 2 +
tests/qapi-schema/returns-unknown.out | 0
tests/qapi-schema/returns-whitelist.err | 1 +
tests/qapi-schema/returns-whitelist.exit | 1 +
tests/qapi-schema/returns-whitelist.json | 11 +
tests/qapi-schema/returns-whitelist.out | 0
tests/qapi-schema/type-bypass-bad-gen.err | 1 +
tests/qapi-schema/type-bypass-bad-gen.exit | 1 +
tests/qapi-schema/type-bypass-bad-gen.json | 2 +
tests/qapi-schema/type-bypass-bad-gen.out | 0
tests/qapi-schema/type-bypass-no-gen.err | 1 +
tests/qapi-schema/type-bypass-no-gen.exit | 1 +
tests/qapi-schema/type-bypass-no-gen.json | 2 +
tests/qapi-schema/type-bypass-no-gen.out | 0
tests/qapi-schema/type-bypass.err | 0
tests/qapi-schema/type-bypass.exit | 1 +
tests/qapi-schema/type-bypass.json | 2 +
tests/qapi-schema/type-bypass.out | 3 +
tests/qapi-schema/union-bad-branch.err | 1 +
tests/qapi-schema/union-bad-branch.exit | 1 +
tests/qapi-schema/union-bad-branch.json | 8 +
tests/qapi-schema/union-bad-branch.out | 0
tests/qapi-schema/union-base-no-discriminator.err | 1 +
tests/qapi-schema/union-base-no-discriminator.exit | 1 +
tests/qapi-schema/union-base-no-discriminator.json | 15 +
tests/qapi-schema/union-base-no-discriminator.out | 0
tests/qapi-schema/union-invalid-base.err | 2 +-
tests/qapi-schema/union-invalid-base.json | 4 +-
tests/qapi-schema/union-max.err | 1 +
tests/qapi-schema/union-max.exit | 1 +
tests/qapi-schema/union-max.json | 3 +
tests/qapi-schema/union-max.out | 0
tests/qapi-schema/union-optional-branch.err | 1 +
tests/qapi-schema/union-optional-branch.exit | 1 +
tests/qapi-schema/union-optional-branch.json | 2 +
tests/qapi-schema/union-optional-branch.out | 0
tests/qapi-schema/union-unknown.err | 1 +
tests/qapi-schema/union-unknown.exit | 1 +
tests/qapi-schema/union-unknown.json | 3 +
tests/qapi-schema/union-unknown.out | 0
tests/qapi-schema/unknown-expr-key.err | 1 +
tests/qapi-schema/unknown-expr-key.exit | 1 +
tests/qapi-schema/unknown-expr-key.json | 2 +
tests/qapi-schema/unknown-expr-key.out | 0
tests/test-qmp-commands.c | 35 +-
tests/test-qmp-input-strict.c | 98 ++---
tests/test-qmp-input-visitor.c | 117 +++--
tests/test-qmp-output-visitor.c | 152 +++----
tests/test-visitor-serialization.c | 84 ++--
261 files changed, 1603 insertions(+), 633 deletions(-)
create mode 100644 tests/qapi-schema/alternate-array.err
create mode 100644 tests/qapi-schema/alternate-array.exit
create mode 100644 tests/qapi-schema/alternate-array.json
create mode 100644 tests/qapi-schema/alternate-array.out
create mode 100644 tests/qapi-schema/alternate-base.err
create mode 100644 tests/qapi-schema/alternate-base.exit
create mode 100644 tests/qapi-schema/alternate-base.json
create mode 100644 tests/qapi-schema/alternate-base.out
create mode 100644 tests/qapi-schema/alternate-clash.err
create mode 100644 tests/qapi-schema/alternate-clash.exit
create mode 100644 tests/qapi-schema/alternate-clash.json
create mode 100644 tests/qapi-schema/alternate-clash.out
create mode 100644 tests/qapi-schema/alternate-conflict-dict.err
create mode 100644 tests/qapi-schema/alternate-conflict-dict.exit
create mode 100644 tests/qapi-schema/alternate-conflict-dict.json
create mode 100644 tests/qapi-schema/alternate-conflict-dict.out
create mode 100644 tests/qapi-schema/alternate-conflict-string.err
create mode 100644 tests/qapi-schema/alternate-conflict-string.exit
create mode 100644 tests/qapi-schema/alternate-conflict-string.json
create mode 100644 tests/qapi-schema/alternate-conflict-string.out
create mode 100644 tests/qapi-schema/alternate-good.err
create mode 100644 tests/qapi-schema/alternate-good.exit
create mode 100644 tests/qapi-schema/alternate-good.json
create mode 100644 tests/qapi-schema/alternate-good.out
create mode 100644 tests/qapi-schema/alternate-nested.err
create mode 100644 tests/qapi-schema/alternate-nested.exit
create mode 100644 tests/qapi-schema/alternate-nested.json
create mode 100644 tests/qapi-schema/alternate-nested.out
create mode 100644 tests/qapi-schema/alternate-unknown.err
create mode 100644 tests/qapi-schema/alternate-unknown.exit
create mode 100644 tests/qapi-schema/alternate-unknown.json
create mode 100644 tests/qapi-schema/alternate-unknown.out
create mode 100644 tests/qapi-schema/bad-base.err
create mode 100644 tests/qapi-schema/bad-base.exit
create mode 100644 tests/qapi-schema/bad-base.json
create mode 100644 tests/qapi-schema/bad-base.out
create mode 100644 tests/qapi-schema/bad-ident.err
create mode 100644 tests/qapi-schema/bad-ident.exit
create mode 100644 tests/qapi-schema/bad-ident.json
create mode 100644 tests/qapi-schema/bad-ident.out
create mode 100644 tests/qapi-schema/bad-type-bool.err
create mode 100644 tests/qapi-schema/bad-type-bool.exit
create mode 100644 tests/qapi-schema/bad-type-bool.json
create mode 100644 tests/qapi-schema/bad-type-bool.out
create mode 100644 tests/qapi-schema/bad-type-dict.err
create mode 100644 tests/qapi-schema/bad-type-dict.exit
create mode 100644 tests/qapi-schema/bad-type-dict.json
create mode 100644 tests/qapi-schema/bad-type-dict.out
create mode 100644 tests/qapi-schema/bad-type-int.err
create mode 100644 tests/qapi-schema/bad-type-int.exit
create mode 100644 tests/qapi-schema/bad-type-int.json
create mode 100644 tests/qapi-schema/bad-type-int.out
create mode 100644 tests/qapi-schema/command-int.err
create mode 100644 tests/qapi-schema/command-int.exit
create mode 100644 tests/qapi-schema/command-int.json
create mode 100644 tests/qapi-schema/command-int.out
create mode 100644 tests/qapi-schema/data-array-empty.err
create mode 100644 tests/qapi-schema/data-array-empty.exit
create mode 100644 tests/qapi-schema/data-array-empty.json
create mode 100644 tests/qapi-schema/data-array-empty.out
create mode 100644 tests/qapi-schema/data-array-unknown.err
create mode 100644 tests/qapi-schema/data-array-unknown.exit
create mode 100644 tests/qapi-schema/data-array-unknown.json
create mode 100644 tests/qapi-schema/data-array-unknown.out
create mode 100644 tests/qapi-schema/data-int.err
create mode 100644 tests/qapi-schema/data-int.exit
create mode 100644 tests/qapi-schema/data-int.json
create mode 100644 tests/qapi-schema/data-int.out
create mode 100644 tests/qapi-schema/data-member-array-bad.err
create mode 100644 tests/qapi-schema/data-member-array-bad.exit
create mode 100644 tests/qapi-schema/data-member-array-bad.json
create mode 100644 tests/qapi-schema/data-member-array-bad.out
create mode 100644 tests/qapi-schema/data-member-array.err
create mode 100644 tests/qapi-schema/data-member-array.exit
create mode 100644 tests/qapi-schema/data-member-array.json
create mode 100644 tests/qapi-schema/data-member-array.out
create mode 100644 tests/qapi-schema/data-member-unknown.err
create mode 100644 tests/qapi-schema/data-member-unknown.exit
create mode 100644 tests/qapi-schema/data-member-unknown.json
create mode 100644 tests/qapi-schema/data-member-unknown.out
create mode 100644 tests/qapi-schema/data-unknown.err
create mode 100644 tests/qapi-schema/data-unknown.exit
create mode 100644 tests/qapi-schema/data-unknown.json
create mode 100644 tests/qapi-schema/data-unknown.out
create mode 100644 tests/qapi-schema/double-data.err
create mode 100644 tests/qapi-schema/double-data.exit
create mode 100644 tests/qapi-schema/double-data.json
create mode 100644 tests/qapi-schema/double-data.out
create mode 100644 tests/qapi-schema/double-type.err
create mode 100644 tests/qapi-schema/double-type.exit
create mode 100644 tests/qapi-schema/double-type.json
create mode 100644 tests/qapi-schema/double-type.out
create mode 100644 tests/qapi-schema/enum-clash-member.err
create mode 100644 tests/qapi-schema/enum-clash-member.exit
create mode 100644 tests/qapi-schema/enum-clash-member.json
create mode 100644 tests/qapi-schema/enum-clash-member.out
create mode 100644 tests/qapi-schema/enum-dict-member.err
create mode 100644 tests/qapi-schema/enum-dict-member.exit
create mode 100644 tests/qapi-schema/enum-dict-member.json
create mode 100644 tests/qapi-schema/enum-dict-member.out
create mode 100644 tests/qapi-schema/enum-empty.err
create mode 100644 tests/qapi-schema/enum-empty.exit
create mode 100644 tests/qapi-schema/enum-empty.json
create mode 100644 tests/qapi-schema/enum-empty.out
create mode 100644 tests/qapi-schema/enum-int-member.err
create mode 100644 tests/qapi-schema/enum-int-member.exit
create mode 100644 tests/qapi-schema/enum-int-member.json
create mode 100644 tests/qapi-schema/enum-int-member.out
create mode 100644 tests/qapi-schema/enum-max-member.err
create mode 100644 tests/qapi-schema/enum-max-member.exit
create mode 100644 tests/qapi-schema/enum-max-member.json
create mode 100644 tests/qapi-schema/enum-max-member.out
create mode 100644 tests/qapi-schema/enum-missing-data.err
create mode 100644 tests/qapi-schema/enum-missing-data.exit
create mode 100644 tests/qapi-schema/enum-missing-data.json
create mode 100644 tests/qapi-schema/enum-missing-data.out
create mode 100644 tests/qapi-schema/enum-union-clash.err
create mode 100644 tests/qapi-schema/enum-union-clash.exit
create mode 100644 tests/qapi-schema/enum-union-clash.json
create mode 100644 tests/qapi-schema/enum-union-clash.out
create mode 100644 tests/qapi-schema/enum-wrong-data.err
create mode 100644 tests/qapi-schema/enum-wrong-data.exit
create mode 100644 tests/qapi-schema/enum-wrong-data.json
create mode 100644 tests/qapi-schema/enum-wrong-data.out
create mode 100644 tests/qapi-schema/event-case.err
create mode 100644 tests/qapi-schema/event-case.exit
create mode 100644 tests/qapi-schema/event-case.json
create mode 100644 tests/qapi-schema/event-case.out
create mode 100644 tests/qapi-schema/event-max.err
create mode 100644 tests/qapi-schema/event-max.exit
create mode 100644 tests/qapi-schema/event-max.json
create mode 100644 tests/qapi-schema/event-max.out
create mode 100644 tests/qapi-schema/flat-union-bad-base.err
create mode 100644 tests/qapi-schema/flat-union-bad-base.exit
create mode 100644 tests/qapi-schema/flat-union-bad-base.json
create mode 100644 tests/qapi-schema/flat-union-bad-base.out
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.err
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.exit
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.json
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.out
create mode 100644 tests/qapi-schema/flat-union-base-union.err
create mode 100644 tests/qapi-schema/flat-union-base-union.exit
create mode 100644 tests/qapi-schema/flat-union-base-union.json
create mode 100644 tests/qapi-schema/flat-union-base-union.out
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.out
create mode 100644 tests/qapi-schema/missing-type.err
create mode 100644 tests/qapi-schema/missing-type.exit
create mode 100644 tests/qapi-schema/missing-type.json
create mode 100644 tests/qapi-schema/missing-type.out
create mode 100644 tests/qapi-schema/nested-struct-data.err
create mode 100644 tests/qapi-schema/nested-struct-data.exit
create mode 100644 tests/qapi-schema/nested-struct-data.json
create mode 100644 tests/qapi-schema/nested-struct-data.out
create mode 100644 tests/qapi-schema/nested-struct-returns.err
create mode 100644 tests/qapi-schema/nested-struct-returns.exit
create mode 100644 tests/qapi-schema/nested-struct-returns.json
create mode 100644 tests/qapi-schema/nested-struct-returns.out
create mode 100644 tests/qapi-schema/redefined-builtin.err
create mode 100644 tests/qapi-schema/redefined-builtin.exit
create mode 100644 tests/qapi-schema/redefined-builtin.json
create mode 100644 tests/qapi-schema/redefined-builtin.out
create mode 100644 tests/qapi-schema/redefined-command.err
create mode 100644 tests/qapi-schema/redefined-command.exit
create mode 100644 tests/qapi-schema/redefined-command.json
create mode 100644 tests/qapi-schema/redefined-command.out
create mode 100644 tests/qapi-schema/redefined-event.err
create mode 100644 tests/qapi-schema/redefined-event.exit
create mode 100644 tests/qapi-schema/redefined-event.json
create mode 100644 tests/qapi-schema/redefined-event.out
create mode 100644 tests/qapi-schema/redefined-type.err
create mode 100644 tests/qapi-schema/redefined-type.exit
create mode 100644 tests/qapi-schema/redefined-type.json
create mode 100644 tests/qapi-schema/redefined-type.out
create mode 100644 tests/qapi-schema/returns-alternate.err
create mode 100644 tests/qapi-schema/returns-alternate.exit
create mode 100644 tests/qapi-schema/returns-alternate.json
create mode 100644 tests/qapi-schema/returns-alternate.out
create mode 100644 tests/qapi-schema/returns-array-bad.err
create mode 100644 tests/qapi-schema/returns-array-bad.exit
create mode 100644 tests/qapi-schema/returns-array-bad.json
create mode 100644 tests/qapi-schema/returns-array-bad.out
create mode 100644 tests/qapi-schema/returns-int.err
create mode 100644 tests/qapi-schema/returns-int.exit
create mode 100644 tests/qapi-schema/returns-int.json
create mode 100644 tests/qapi-schema/returns-int.out
create mode 100644 tests/qapi-schema/returns-unknown.err
create mode 100644 tests/qapi-schema/returns-unknown.exit
create mode 100644 tests/qapi-schema/returns-unknown.json
create mode 100644 tests/qapi-schema/returns-unknown.out
create mode 100644 tests/qapi-schema/returns-whitelist.err
create mode 100644 tests/qapi-schema/returns-whitelist.exit
create mode 100644 tests/qapi-schema/returns-whitelist.json
create mode 100644 tests/qapi-schema/returns-whitelist.out
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.err
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.json
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.out
create mode 100644 tests/qapi-schema/type-bypass-no-gen.err
create mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-no-gen.json
create mode 100644 tests/qapi-schema/type-bypass-no-gen.out
create mode 100644 tests/qapi-schema/type-bypass.err
create mode 100644 tests/qapi-schema/type-bypass.exit
create mode 100644 tests/qapi-schema/type-bypass.json
create mode 100644 tests/qapi-schema/type-bypass.out
create mode 100644 tests/qapi-schema/union-bad-branch.err
create mode 100644 tests/qapi-schema/union-bad-branch.exit
create mode 100644 tests/qapi-schema/union-bad-branch.json
create mode 100644 tests/qapi-schema/union-bad-branch.out
create mode 100644 tests/qapi-schema/union-base-no-discriminator.err
create mode 100644 tests/qapi-schema/union-base-no-discriminator.exit
create mode 100644 tests/qapi-schema/union-base-no-discriminator.json
create mode 100644 tests/qapi-schema/union-base-no-discriminator.out
create mode 100644 tests/qapi-schema/union-max.err
create mode 100644 tests/qapi-schema/union-max.exit
create mode 100644 tests/qapi-schema/union-max.json
create mode 100644 tests/qapi-schema/union-max.out
create mode 100644 tests/qapi-schema/union-optional-branch.err
create mode 100644 tests/qapi-schema/union-optional-branch.exit
create mode 100644 tests/qapi-schema/union-optional-branch.json
create mode 100644 tests/qapi-schema/union-optional-branch.out
create mode 100644 tests/qapi-schema/union-unknown.err
create mode 100644 tests/qapi-schema/union-unknown.exit
create mode 100644 tests/qapi-schema/union-unknown.json
create mode 100644 tests/qapi-schema/union-unknown.out
create mode 100644 tests/qapi-schema/unknown-expr-key.err
create mode 100644 tests/qapi-schema/unknown-expr-key.exit
create mode 100644 tests/qapi-schema/unknown-expr-key.json
create mode 100644 tests/qapi-schema/unknown-expr-key.out
--
2.1.0
^ permalink raw reply [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-25 18:31 ` Markus Armbruster
` (2 more replies)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 02/28] qapi: Fix generation of 'size' builtin type Eric Blake
` (28 subsequent siblings)
29 siblings, 3 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Go into more details about the various types of valid expressions
in a qapi schema, including tweaks to document fixes being done
later in the current patch series. Also fix some stale and missing
documentation in the QMP specification.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
I'm not sure if it is okay to assert GPLv2+ licensing without an
explicit Copyright, but as I am not the original author, I don't
know who to attribute any original Copyright to. Advice? Should
I split the license addition to a separate patch?
---
docs/qapi-code-gen.txt | 338 ++++++++++++++++++++++++++++++++++++++++---------
docs/qmp/qmp-spec.txt | 92 +++++++++++---
2 files changed, 350 insertions(+), 80 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 8313ba6..ce9c4b9 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -1,17 +1,18 @@
+This document is licensed under the GPLv2 (or later).
+
= How to use the QAPI code generator =
QAPI is a native C API within QEMU which provides management-level
-functionality to internal/external users. For external
+functionality to internal and external users. For external
users/processes, this interface is made available by a JSON-based
QEMU Monitor protocol that is provided by the QMP server.
-To map QMP-defined interfaces to the native C QAPI implementations,
-a JSON-based schema is used to define types and function
-signatures, and a set of scripts is used to generate types/signatures,
-and marshaling/dispatch code. The QEMU Guest Agent also uses these
-scripts, paired with a separate schema, to generate
-marshaling/dispatch code for the guest agent server running in the
-guest.
+To map QMP-defined interfaces to the native C QAPI implementations, a
+JSON-based schema is used to define types and function signatures, and
+a set of scripts is used to generate types, signatures, and
+marshaling/dispatch code. The QEMU Guest Agent also uses these
+scripts, paired with a separate schema, to generate marshaling and
+dispatch code for the guest agent server running in the guest.
This document will describe how the schemas, scripts, and resulting
code are used.
@@ -22,40 +23,150 @@ code are used.
This file defines the types, commands, and events used by QMP. It should
fully describe the interface used by QMP.
-This file is designed to be loosely based on JSON although it's technically
-executable Python. While dictionaries are used, they are parsed as
-OrderedDicts so that ordering is preserved.
+A QAPI file is designed to be loosely based on JSON, which is then
+parsed by a python code generation program. A valid QAPI schema
+consists of a list of top-level expressions, with no commas between
+them. Where dictionaries are used, they are parsed as OrderedDicts so
+that ordering is preserved (for predictable layout of generated C
+structs and parameter lists). Ordering doesn't matter for top-level
+expressions, but does matter within 'data' members of a single
+expression. QAPI input is written using 'single quotes' instead of
+JSON's "double quotes" (in contrast, QMP is strict JSON and only uses
+"double quotes"). As in JSON, trailing commas are not permitted in
+arrays or dictionaries. Input must be ASCII (although QMP supports
+full Unicode strings, the QAPI parser does not).
-There are two basic syntaxes used, type definitions and command definitions.
+Comments are allowed; anything between an unquoted # and the following
+newline is ignored. Although there is not yet a documentation
+generator, a form of stylized comments has developed for consistently
+documenting details about an expression and when it was added to the
+schema. The documentation is delimited between two lines of ##, then
+the first line names the expression, an optional overview is provided,
+then individual documentation about each member of 'data' is provided,
+and finally, a 'Since: x.y.z' tag lists the release that introduced
+the expression. Optional fields are tagged with the phrase
+'#optional', often with their default value; and extensions added
+after the expression was first released are also given a '(since
+x.y.z)' comment. For example:
-The first syntax defines a type and is represented by a dictionary. There are
-three kinds of user-defined types that are supported: complex types,
-enumeration types and union types.
+ ##
+ # @BlockStats:
+ #
+ # Statistics of a virtual block device or a block backing device.
+ #
+ # @device: #optional If the stats are for a virtual block device, the name
+ # corresponding to the virtual block device.
+ #
+ # @stats: A @BlockDeviceStats for the device.
+ #
+ # @parent: #optional This describes the file block device if it has one.
+ #
+ # @backing: #optional This describes the backing block device if it has one.
+ # (Since 2.0)
+ #
+ # Since: 0.14.0
+ ##
+ { 'type': 'BlockStats',
+ 'data': {'*device': 'str', 'stats': 'BlockDeviceStats',
+ '*parent': 'BlockStats',
+ '*backing': 'BlockStats'} }
-Generally speaking, types definitions should always use CamelCase for the type
-names. Command names should be all lower case with words separated by a hyphen.
+The schema sets up a series of types, as well as commands and events
+that will use those types. Forward references are allowed: the parser
+scans in two passes, where the first pass learns all type names, and
+the second validates the schema and generates the code. This allows
+the definition of complex structs that can have mutually recursive
+types, and allows for indefinite nesting of QMP that satisfies the
+schema. A type name should not be defined more than once.
+There are six top-level expressions recognized by the parser:
+'include', 'command', 'type', 'enum', 'union', and 'event'. There are
+several built-in types, such as 'int' and 'str'; additionally, the
+top-level expressions can define complex types, enumeration types, and
+several flavors of union types. The 'command' expression can refer to
+existing types by name, or list an anonymous type as a dictionary.
+Listing a type name inside an array refers to a single-dimension array
+of that type; multi-dimension arrays are not directly supported
+(although an array of a complex struct that contains an array member
+is possible).
+
+Types, commands, and events share a common namespace. Therefore,
+generally speaking, type definitions should always use CamelCase for
+user-defined type names, while built-in types are lowercase. Type
+definitions should not end in 'Kind', as this namespace is used for
+creating implicit C enums for visiting union types. Command names,
+and field names within a type, should be all lower case with words
+separated by a hyphen. However, some existing older commands and
+complex types use underscore; when extending such expressions,
+consistency is preferred over blindly avoiding underscore. Event
+names should be ALL_CAPS with words separated by underscore. The
+special string '**' appears for some commands that manually perform
+their own type checking rather than relying on the type-safe code
+produced by the qapi code generators.
+
+Any command, type, or field name beginning with "x-" is marked
+experimental, and may be withdrawn or changed incompatibly in a future
+release. Downstream vendors may add extensions; such extensions
+should begin with a prefix matching "__RFQDN_" (for the
+reverse-fully-qualified-domain-name of the vendor), even if the rest
+of the command or field name uses dash (example:
+__com.redhat_drive-mirror). Other than the dots used in RFQDN of a
+downstream extension, all command, type, and field names should begin
+with a letter, and contain only ASCII letters, numbers, dash, and
+underscore. It is okay to reuse names that match C keywords; the
+generator will rename a field named "default" in the QAPI to
+"q_default" in the generated C code.
+
+
+=== Built-in Types ===
+
+The following types are built-in to the parser:
+ 'str' - arbitrary UTF-8 string
+ 'int' - 64-bit signed integer (although the C code may place further
+ restrictions on acceptable range)
+ 'number' - floating point number
+ 'bool' - JSON value of true or false
+ 'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
+ bit size
+ 'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
+ 'size' - like 'uint64', but allows scaled suffix from command line
+ visitor
=== Includes ===
+Usage: { 'include': 'str' }
+
The QAPI schema definitions can be modularized using the 'include' directive:
{ 'include': 'path/to/file.json'}
The directive is evaluated recursively, and include paths are relative to the
-file using the directive. Multiple includes of the same file are safe.
+file using the directive. Multiple includes of the same file are
+safe. No other keys should appear in the expression, and the include
+value should be a string.
+
+As a matter of style, it is a good idea to have all files be
+self-contained, but at the moment, nothing prevents an included file
+from making a forward reference to a type that is only introduced by
+an outer file. The parser may be made stricter in the future to
+prevent incomplete include files.
=== Complex types ===
-A complex type is a dictionary containing a single key whose value is a
-dictionary. This corresponds to a struct in C or an Object in JSON. An
-example of a complex type is:
+Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
+
+A complex type is a dictionary containing a single 'data' key whose
+value is a dictionary. This corresponds to a struct in C or an Object
+in JSON. Each value of the 'data' dictionary must be the name of a
+complex, enum, union, or built-in type, or a one-element array
+containing a type name. An example of a complex type is:
{ 'type': 'MyType',
'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
-The use of '*' as a prefix to the name means the member is optional.
+The use of '*' as a prefix to the name means the member is optional in
+the corresponding QMP usage.
The default initialization value of an optional argument should not be changed
between versions of QEMU unless the new default maintains backward
@@ -100,22 +211,52 @@ both fields like this:
{ "file": "/some/place/my-image",
"backing": "/some/place/my-backing-file" }
+
=== Enumeration types ===
-An enumeration type is a dictionary containing a single key whose value is a
-list of strings. An example enumeration is:
+Usage: { 'enum': 'str', 'data': [ 'str' ] }
+
+An enumeration type is a dictionary containing a single 'data' key
+whose value is a list of strings. An example enumeration is:
{ 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }
+Nothing prevents an empty enumeration, although it is probably not
+useful. The list of strings should be lower case; if an enum name
+represents multiple words, use '-' between words. The string 'max' is
+not allowed as an enum value, and values should not be repeated.
+
+The enumeration values are passed as strings over the QMP protocol,
+but are encoded as C enum integral values in generated code. While
+the C code starts numbering at 0, it is better to use explicit
+comparisons to enum values than implicit comparisons to 0; the C code
+will also include a generated enum member ending in _MAX for tracking
+the size of the enum, useful when using common functions for
+converting between strings and enum values. Since the wire format
+always passes by name, it is acceptable to reorder or add new
+enumeration members in any location without breaking QMP clients;
+however, removing enum values would break compatibility. For any
+complex type that has a field that will only contain a finite set of
+string values, using an enum type for that field is better than
+open-coding the field to be type 'str'.
+
+
=== Union types ===
-Union types are used to let the user choose between several different data
-types. A union type is defined using a dictionary as explained in the
-following paragraphs.
+Usage: { 'union': 'str', 'data': 'dict' }
+or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
+ 'discriminator': 'enum-member-of-base' }
+or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
+Union types are used to let the user choose between several different
+data types. There are three flavors: simple (no discriminator), flat
+(a base type is mandatory, and discriminator is the name of an enum
+field within that base type), and anonymous (discriminator is an
+empty dictionary). A union type is defined using a data dictionary as
+explained in the following paragraphs.
-A simple union type defines a mapping from discriminator values to data types
-like in this example:
+A simple union type defines a mapping from automatic discriminator
+values to data types like in this example:
{ 'type': 'FileOptions', 'data': { 'filename': 'str' } }
{ 'type': 'Qcow2Options',
@@ -132,10 +273,17 @@ specified data type corresponding to the discriminator value:
{ "type": "qcow2", "data" : { "backing-file": "/some/place/my-image",
"lazy-refcounts": true } }
+Additionally, an implicit C enum NameKind is created, corresponding to
+the union Name, for accessing the various branches of the union. No
+branch of the union can be named 'max', as this would collide with the
+implicit enum.
-A union definition can specify a complex type as its base. In this case, the
-fields of the complex type are included as top-level fields of the union
-dictionary in the QMP wire format. An example definition is:
+
+A flat union definition specifies a complex type as its base, and
+avoids nesting on the wire. In this case, the fields of the complex
+type are included as top-level fields of the union dictionary in the
+QMP wire format, and the 'discriminator' field must be the name of an
+enum-typed member of the base type. An example definition is:
{ 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
{ 'union': 'BlockdevOptions',
@@ -150,11 +298,11 @@ And it looks like this on the wire:
"data" : { "backing-file": "/some/place/my-image",
"lazy-refcounts": true } }
-
-Flat union types avoid the nesting on the wire. They are used whenever a
-specific field of the base type is declared as the discriminator ('type' is
-then no longer generated). The discriminator must be of enumeration type.
-The above example can then be modified as follows:
+Notice that in a flat union, a 'type' field is no longer generated,
+and the keys of the 'data' dictionary must match the valid values for
+the discriminator (although not necessarily in the same order). The
+above example for simple unions can be modified to a flat union as
+follows:
{ 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
{ 'type': 'BlockdevCommonOptions',
@@ -173,13 +321,15 @@ Resulting in this JSON object:
"lazy-refcounts": true }
-A special type of unions are anonymous unions. They don't form a dictionary in
-the wire format but allow the direct use of different types in their place. As
-they aren't structured, they don't have any explicit discriminator but use
-the (QObject) data type of their value as an implicit discriminator. This means
-that they are restricted to using only one discriminator value per QObject
-type. For example, you cannot have two different complex types in an anonymous
-union, or two different integer types.
+The final flavor of unions is an anonymous union. While the other two
+union types are always passed as a dictionary in the wire format, an
+anonymous union instead allows the direct use of different types in
+its place. As they aren't structured, they don't have any explicit
+discriminator but use the (QObject) data type of their value as an
+implicit discriminator. This means that they are restricted to using
+only one discriminator value per QObject type. For example, you cannot
+have two different complex types in an anonymous union, or two
+different integer types.
Anonymous unions are declared using an empty dictionary as their discriminator.
The discriminator values never appear on the wire, they are only used in the
@@ -200,23 +350,93 @@ This example allows using both of the following example objects:
=== Commands ===
-Commands are defined by using a list containing three members. The first
-member is the command name, the second member is a dictionary containing
-arguments, and the third member is the return type.
+Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
+ '*returns': 'type',
+ '*gen': false, '*success-response': false }
-An example command is:
+Commands are defined by using a dictionary containing several members,
+where three members are most common. The 'command' member is a
+mandatory string, and determines the "execute" value passed in a QMP
+command exchange.
+
+The 'data' member is optional; if absent, the command accepts an
+optional empty dictionary. If present, it must be the string name of
+a complex type, a one-element array containing the name of a complex
+type, or a dictionary that declares an anonymous type with the same
+semantics as a 'type' expression, with one exception noted below when
+'gen' is used. The 'data' argument maps to the "arguments" dictionary
+passed in as part of a QMP command.
+
+The 'returns' member describes what will appear in the "return" field
+of a QMP reply on successful completion of a command. The member is
+optional from the command declaration; if absent, the "return" field
+will be an empty dictionary. If 'returns' is present, it must be the
+string name of a complex or built-in type, a one-element array
+containing the name of a complex or built-in type, or a dictionary
+that declares an anonymous type with the same semantics as a 'type'
+expression, with one exception noted below when 'gen' is used.
+Although it is permitted to have the 'returns' member name a built-in
+type or an array of built-in types, any command that does this cannot
+be extended to return additional information in the future; thus, new
+commands should strongly consider returning a dictionary-based type or
+an array of dictionaries, even if the dictionary only contains one
+field at the present.
+
+All commands use a dictionary to report failure, with no way to
+specify that in QAPI. Where the error return is different than the
+usual GenericError class in order to help the client react differently
+to certain error conditions, it is worth documenting this in the
+comments before the command declaration.
+
+Some example commands:
+
+ { 'command': 'my-first-command',
+ 'data': { 'arg1': 'str', '*arg2': 'str' } }
+ { 'type': 'MyType', 'data': { '*value': 'str' } }
+ { 'command': 'my-second-command',
+ 'returns': [ 'MyType' ] }
+
+which would validate this QMP transaction:
+
+ => { "execute": "my-first-command",
+ "arguments": { "arg1": "hello" } }
+ <= { "return": { } }
+ => { "execute": "my-second-command" }
+ <= { "return": [ { "value": "one" }, { } ] }
+
+In rare cases, QAPI cannot express a type-safe representation of a
+corresponding QMP command. In these cases, if the command expression
+includes the key 'gen' with boolean value false, then the 'data' or
+'returns' member that intends to bypass generated type-safety and do
+its own manual validation should use '**' rather than a valid type
+name. Please try to avoid adding new commands that rely on this, and
+instead use type-safe unions. For an example of bypass usage:
+
+ { 'command': 'netdev_add',
+ 'data': {'type': 'str', 'id': 'str', '*props': '**'},
+ 'gen': false }
+
+Normally, the QAPI schema is used to describe synchronous exchanges,
+where a response is expected. But in some cases, the action of a
+command is expected to change state in a way that a successful
+response is not possible (the command still returns a normal
+dictionary error on failure). When a successful reply is not
+possible, the command expression should include the optional key
+'success-response' with boolean value false. So far, only the
+qemu-guest-agent makes use of this field.
- { 'command': 'my-command',
- 'data': { 'arg1': 'str', '*arg2': 'str' },
- 'returns': 'str' }
=== Events ===
-Events are defined with the keyword 'event'. When 'data' is also specified,
-additional info will be included in the event. Finally there will be C API
-generated in qapi-event.h; when called by QEMU code, a message with timestamp
-will be emitted on the wire. If timestamp is -1, it means failure to retrieve
-host time.
+Usage: { 'event': 'str', '*data': 'dict-or-complex-type-name' }
+
+Events are defined with the keyword 'event'. It is not allowed to
+name an event 'MAX', since the generator also produces a C enumeration
+of all event names with a generated _MAX value at the end. When
+'data' is also specified, additional info will be included in the
+event, with similar semantics to a 'type' expression. Finally there
+will be C API generated in qapi-event.h; when called by QEMU code, a
+message with timestamp will be emitted on the wire.
An example event is:
@@ -311,7 +531,7 @@ Example:
#ifndef EXAMPLE_QAPI_TYPES_H
#define EXAMPLE_QAPI_TYPES_H
-[Builtin types omitted...]
+[Built-in types omitted...]
typedef struct UserDefOne UserDefOne;
@@ -324,7 +544,7 @@ Example:
struct UserDefOneList *next;
} UserDefOneList;
-[Functions on builtin types omitted...]
+[Functions on built-in types omitted...]
struct UserDefOne
{
@@ -423,7 +643,7 @@ Example:
#ifndef EXAMPLE_QAPI_VISIT_H
#define EXAMPLE_QAPI_VISIT_H
-[Visitors for builtin types omitted...]
+[Visitors for built-in types omitted...]
void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp);
void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp);
diff --git a/docs/qmp/qmp-spec.txt b/docs/qmp/qmp-spec.txt
index 22568c6..1fb642b 100644
--- a/docs/qmp/qmp-spec.txt
+++ b/docs/qmp/qmp-spec.txt
@@ -1,10 +1,20 @@
QEMU Machine Protocol Specification
+0. About This Document
+======================
+
+This document is licensed under the GPLv2 (or later).
+
+Last revised in March 2015.
+
1. Introduction
===============
-This document specifies the QEMU Machine Protocol (QMP), a JSON-based protocol
-which is available for applications to operate QEMU at the machine-level.
+This document specifies the QEMU Machine Protocol (QMP), a JSON-based
+protocol which is available for applications to operate QEMU at the
+machine-level. It is also in use by the QEMU Guest Agent (QGA), which
+is available for host applications to interact with the guest
+operating system.
2. Protocol Specification
=========================
@@ -23,9 +33,11 @@ the JSON standard:
http://www.ietf.org/rfc/rfc4627.txt
-For convenience, json-object members and json-array elements mentioned in
-this document will be in a certain order. However, in real protocol usage
-they can be in ANY order, thus no particular order should be assumed.
+For convenience, json-object members mentioned in this document will
+be in a certain order. However, in real protocol usage they can be in
+ANY order, thus no particular order should be assumed. On the other
+hand, use of json-array elements presumes that preserving order is
+important unless specifically documented otherwise.
2.1 General Definitions
-----------------------
@@ -52,7 +64,19 @@ The greeting message format is:
- The "version" member contains the Server's version information (the format
is the same of the query-version command)
- The "capabilities" member specify the availability of features beyond the
- baseline specification
+ baseline specification; the order of elements in this array has no
+ particular significance, so a client must search the entire array
+ when looking for a particular capability
+
+When first connecting to the server, the connection is in a capability
+exchange mode, further documented below.
+
+2.2.1 Capabilities
+------------------
+
+As of the date this document was last revised, no server or client
+capability strings have been defined.
+
2.3 Issuing Commands
--------------------
@@ -81,13 +105,15 @@ of a command execution: success or error.
The format of a success response is:
-{ "return": json-object, "id": json-value }
+{ "return": json-entity, "id": json-value }
Where,
-- The "return" member contains the command returned data, which is defined
- in a per-command basis or an empty json-object if the command does not
- return data
+- The "return" member contains the data returned by the command, which
+ is defined on a per-command basis (usually a json-object or
+ json-array of json-objects, but sometimes a json-value, json-string,
+ or json-array of json-strings); it is an empty json-object if the
+ command does not return data
- The "id" member contains the transaction identification associated
with the command execution if issued by the Client
@@ -114,7 +140,8 @@ if provided by the client.
-----------------------
As a result of state changes, the Server may send messages unilaterally
-to the Client at any time. They are called "asynchronous events".
+to the Client at any time, when not in the middle of any other
+response. They are called "asynchronous events".
The format of asynchronous events is:
@@ -126,13 +153,27 @@ The format of asynchronous events is:
- The "event" member contains the event's name
- The "data" member contains event specific data, which is defined in a
per-event basis, it is optional
-- The "timestamp" member contains the exact time of when the event occurred
- in the Server. It is a fixed json-object with time in seconds and
- microseconds
+- The "timestamp" member contains the exact time of when the event
+ occurred in the Server. It is a fixed json-object with time in
+ seconds and microseconds relative to the Unix Epoch (1 Jan 1970); if
+ there is a failure to retrieve host time, both members of the
+ timestamp will be set to -1.
For a listing of supported asynchronous events, please, refer to the
qmp-events.txt file.
+2.5 QGA Synchronization
+-----------------------
+
+When using QGA, an additional synchronization feature is built into
+the protocol. If the Client sends a raw 0xFF sentinel byte (not valid
+JSON), then the Server will reset its state and discard all pending
+data prior to the sentinel. Conversely, if the Client makes use of
+the 'guest-sync-delimited' command, the Server will send a raw 0xFF
+sentinel byte prior to its response, to aid the Client in discarding
+any data prior to the sentinel.
+
+
3. QMP Examples
===============
@@ -145,32 +186,37 @@ This section provides some examples of real QMP usage, in all of them
S: { "QMP": { "version": { "qemu": { "micro": 50, "minor": 6, "major": 1 },
"package": ""}, "capabilities": []}}
-3.2 Simple 'stop' execution
+3.2 Client QMP negotiation
+--------------------------
+C: { "execute": "qmp_capabilities" }
+S: { "return": {}}
+
+3.3 Simple 'stop' execution
---------------------------
C: { "execute": "stop" }
S: { "return": {} }
-3.3 KVM information
+3.4 KVM information
-------------------
C: { "execute": "query-kvm", "id": "example" }
S: { "return": { "enabled": true, "present": true }, "id": "example"}
-3.4 Parsing error
+3.5 Parsing error
------------------
C: { "execute": }
S: { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } }
-3.5 Powerdown event
+3.6 Powerdown event
-------------------
S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 },
"event": "POWERDOWN" }
4. Capabilities Negotiation
-----------------------------
+===========================
When a Client successfully establishes a connection, the Server is in
Capabilities Negotiation mode.
@@ -189,7 +235,7 @@ effect, all commands (except qmp_capabilities) are allowed and asynchronous
messages are delivered.
5 Compatibility Considerations
-------------------------------
+==============================
All protocol changes or new features which modify the protocol format in an
incompatible way are disabled by default and will be advertised by the
@@ -213,12 +259,16 @@ However, Clients must not assume any particular:
- Amount of errors generated by a command, that is, new errors can be added
to any existing command in newer versions of the Server
+Any command or field name beginning with "x-" is deemed experimental,
+and may be withdrawn or changed in an incompatible manner in a future
+release.
+
Of course, the Server does guarantee to send valid JSON. But apart from
this, a Client should be "conservative in what they send, and liberal in
what they accept".
6. Downstream extension of QMP
-------------------------------
+==============================
We recommend that downstream consumers of QEMU do *not* modify QMP.
Management tools should be able to support both upstream and downstream
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 02/28] qapi: Fix generation of 'size' builtin type
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 9:52 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema Eric Blake
` (27 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
We were missing the 'size' builtin type (which means that QAPI using
[ 'size' ] would fail to compile). Futhermore, there was some
redundancy between builtin_types[] and builtin_type_qtypes{}.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-types.py | 10 +++++-----
scripts/qapi-visit.py | 6 +++---
scripts/qapi.py | 9 ++-------
tests/qapi-schema/qapi-schema-test.json | 3 ++-
tests/qapi-schema/qapi-schema-test.out | 2 +-
5 files changed, 13 insertions(+), 17 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index db87218..e400b03 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -182,8 +182,8 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
for key in members:
qapi_type = members[key]
- if builtin_type_qtypes.has_key(qapi_type):
- qtype = builtin_type_qtypes[qapi_type]
+ if builtin_types.has_key(qapi_type):
+ qtype = builtin_types[qapi_type]
elif find_struct(qapi_type):
qtype = "QTYPE_QDICT"
elif find_union(qapi_type):
@@ -398,7 +398,7 @@ exprs = parse_schema(input_file)
exprs = filter(lambda expr: not expr.has_key('gen'), exprs)
fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
-for typename in builtin_types:
+for typename in builtin_types.keys():
fdecl.write(generate_fwd_struct(typename, None, builtin_type=True))
fdecl.write(guardend("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
@@ -426,7 +426,7 @@ for expr in exprs:
# to avoid header dependency hell, we always generate declarations
# for built-in types in our header files and simply guard them
fdecl.write(guardstart("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
-for typename in builtin_types:
+for typename in builtin_types.keys():
fdecl.write(generate_type_cleanup_decl(typename + "List"))
fdecl.write(guardend("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
@@ -435,7 +435,7 @@ fdecl.write(guardend("QAPI_TYPES_BUILTIN_CLEANUP_DECL"))
# over these cases
if do_builtins:
fdef.write(guardstart("QAPI_TYPES_BUILTIN_CLEANUP_DEF"))
- for typename in builtin_types:
+ for typename in builtin_types.keys():
fdef.write(generate_type_cleanup(typename + "List"))
fdef.write(guardend("QAPI_TYPES_BUILTIN_CLEANUP_DEF"))
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 8f845a2..4416677 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -261,7 +261,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
disc_type = '%sKind' % (name)
for key in members:
- assert (members[key] in builtin_types
+ assert (members[key] in builtin_types.keys()
or find_struct(members[key])
or find_union(members[key])
or find_enum(members[key])), "Invalid anonymous union member"
@@ -541,7 +541,7 @@ exprs = parse_schema(input_file)
# to avoid header dependency hell, we always generate declarations
# for built-in types in our header files and simply guard them
fdecl.write(guardstart("QAPI_VISIT_BUILTIN_VISITOR_DECL"))
-for typename in builtin_types:
+for typename in builtin_types.keys():
fdecl.write(generate_declaration(typename, None, genlist=True,
builtin_type=True))
fdecl.write(guardend("QAPI_VISIT_BUILTIN_VISITOR_DECL"))
@@ -550,7 +550,7 @@ fdecl.write(guardend("QAPI_VISIT_BUILTIN_VISITOR_DECL"))
# have the functions defined, so we use -b option to provide control
# over these cases
if do_builtins:
- for typename in builtin_types:
+ for typename in builtin_types.keys():
fdef.write(generate_visit_list(typename, None))
for expr in exprs:
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 77d46aa..d470347 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -16,13 +16,7 @@ from ordereddict import OrderedDict
import os
import sys
-builtin_types = [
- 'str', 'int', 'number', 'bool',
- 'int8', 'int16', 'int32', 'int64',
- 'uint8', 'uint16', 'uint32', 'uint64'
-]
-
-builtin_type_qtypes = {
+builtin_types = {
'str': 'QTYPE_QSTRING',
'int': 'QTYPE_QINT',
'number': 'QTYPE_QFLOAT',
@@ -35,6 +29,7 @@ builtin_type_qtypes = {
'uint16': 'QTYPE_QINT',
'uint32': 'QTYPE_QINT',
'uint64': 'QTYPE_QINT',
+ 'size': 'QTYPE_QINT',
}
def error_path(parent):
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index d43b5fd..84f0f07 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -74,7 +74,8 @@
'u64': ['uint64'],
'number': ['number'],
'boolean': ['bool'],
- 'string': ['str'] } }
+ 'string': ['str'],
+ 'sizes': ['size'] } }
# testing commands
{ 'command': 'user_def_cmd', 'data': {} }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 08d7304..915a61b 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -12,7 +12,7 @@
OrderedDict([('union', 'UserDefFlatUnion'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefA'), ('value2', 'UserDefB'), ('value3', 'UserDefB')]))]),
OrderedDict([('union', 'UserDefFlatUnion2'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefC'), ('value2', 'UserDefB'), ('value3', 'UserDefA')]))]),
OrderedDict([('union', 'UserDefAnonUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
- OrderedDict([('union', 'UserDefNativeListUnion'), ('data', OrderedDict([('integer', ['int']), ('s8', ['int8']), ('s16', ['int16']), ('s32', ['int32']), ('s64', ['int64']), ('u8', ['uint8']), ('u16', ['uint16']), ('u32', ['uint32']), ('u64', ['uint64']), ('number', ['number']), ('boolean', ['bool']), ('string', ['str'])]))]),
+ OrderedDict([('union', 'UserDefNativeListUnion'), ('data', OrderedDict([('integer', ['int']), ('s8', ['int8']), ('s16', ['int16']), ('s32', ['int32']), ('s64', ['int64']), ('u8', ['uint8']), ('u16', ['uint16']), ('u32', ['uint32']), ('u64', ['uint64']), ('number', ['number']), ('boolean', ['bool']), ('string', ['str']), ('sizes', ['size'])]))]),
OrderedDict([('command', 'user_def_cmd'), ('data', OrderedDict())]),
OrderedDict([('command', 'user_def_cmd1'), ('data', OrderedDict([('ud1a', 'UserDefOne')]))]),
OrderedDict([('command', 'user_def_cmd2'), ('data', OrderedDict([('ud1a', 'UserDefOne'), ('*ud1b', 'UserDefOne')])), ('returns', 'UserDefTwo')]),
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 02/28] qapi: Fix generation of 'size' builtin type Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-24 20:33 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 04/28] qapi: Add some enum tests Eric Blake
` (26 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Python 2 and Python 3 have a wild history of whether strings
default to ascii or unicode, where Python 3 requires checking
instanceof(foo, basestr) to cover all strings, but where that
code is not portable to Python 2. It's simpler to just state
that we don't care about Unicode strings, and to just always
use the simpler instanceof(foo, str) everywhere.
I'm no python expert, so I'm basing it on this conversation:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.html
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index d470347..20ee505 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -2,7 +2,7 @@
# QAPI helper library
#
# Copyright IBM, Corp. 2011
-# Copyright (c) 2013 Red Hat Inc.
+# Copyright (c) 2013-2015 Red Hat Inc.
#
# Authors:
# Anthony Liguori <aliguori@us.ibm.com>
@@ -354,7 +354,7 @@ def parse_schema(input_file):
return exprs
def parse_args(typeinfo):
- if isinstance(typeinfo, basestring):
+ if isinstance(typeinfo, str):
struct = find_struct(typeinfo)
assert struct != None
typeinfo = struct['data']
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 04/28] qapi: Add some enum tests
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (2 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 10:01 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 05/28] qapi: Better error messages for bad enums Eric Blake
` (25 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Demonstrate that the qapi generator doesn't deal well with enums
that aren't up to par. Later patches will update the expected
results as the generator is made stricter.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 5 ++++-
tests/qapi-schema/enum-clash-member.err | 0
tests/qapi-schema/enum-clash-member.exit | 1 +
tests/qapi-schema/enum-clash-member.json | 2 ++
tests/qapi-schema/enum-clash-member.out | 3 +++
tests/qapi-schema/enum-dict-member.err | 0
tests/qapi-schema/enum-dict-member.exit | 1 +
tests/qapi-schema/enum-dict-member.json | 2 ++
tests/qapi-schema/enum-dict-member.out | 3 +++
tests/qapi-schema/enum-empty.err | 0
tests/qapi-schema/enum-empty.exit | 1 +
tests/qapi-schema/enum-empty.json | 2 ++
tests/qapi-schema/enum-empty.out | 3 +++
tests/qapi-schema/enum-int-member.err | 1 +
tests/qapi-schema/enum-int-member.exit | 1 +
tests/qapi-schema/enum-int-member.json | 3 +++
tests/qapi-schema/enum-int-member.out | 0
tests/qapi-schema/enum-max-member.err | 0
tests/qapi-schema/enum-max-member.exit | 1 +
tests/qapi-schema/enum-max-member.json | 3 +++
tests/qapi-schema/enum-max-member.out | 3 +++
tests/qapi-schema/enum-missing-data.err | 6 ++++++
tests/qapi-schema/enum-missing-data.exit | 1 +
tests/qapi-schema/enum-missing-data.json | 2 ++
tests/qapi-schema/enum-missing-data.out | 0
tests/qapi-schema/enum-union-clash.err | 0
tests/qapi-schema/enum-union-clash.exit | 1 +
tests/qapi-schema/enum-union-clash.json | 4 ++++
tests/qapi-schema/enum-union-clash.out | 5 +++++
tests/qapi-schema/enum-wrong-data.err | 0
tests/qapi-schema/enum-wrong-data.exit | 1 +
tests/qapi-schema/enum-wrong-data.json | 2 ++
tests/qapi-schema/enum-wrong-data.out | 3 +++
33 files changed, 59 insertions(+), 1 deletion(-)
create mode 100644 tests/qapi-schema/enum-clash-member.err
create mode 100644 tests/qapi-schema/enum-clash-member.exit
create mode 100644 tests/qapi-schema/enum-clash-member.json
create mode 100644 tests/qapi-schema/enum-clash-member.out
create mode 100644 tests/qapi-schema/enum-dict-member.err
create mode 100644 tests/qapi-schema/enum-dict-member.exit
create mode 100644 tests/qapi-schema/enum-dict-member.json
create mode 100644 tests/qapi-schema/enum-dict-member.out
create mode 100644 tests/qapi-schema/enum-empty.err
create mode 100644 tests/qapi-schema/enum-empty.exit
create mode 100644 tests/qapi-schema/enum-empty.json
create mode 100644 tests/qapi-schema/enum-empty.out
create mode 100644 tests/qapi-schema/enum-int-member.err
create mode 100644 tests/qapi-schema/enum-int-member.exit
create mode 100644 tests/qapi-schema/enum-int-member.json
create mode 100644 tests/qapi-schema/enum-int-member.out
create mode 100644 tests/qapi-schema/enum-max-member.err
create mode 100644 tests/qapi-schema/enum-max-member.exit
create mode 100644 tests/qapi-schema/enum-max-member.json
create mode 100644 tests/qapi-schema/enum-max-member.out
create mode 100644 tests/qapi-schema/enum-missing-data.err
create mode 100644 tests/qapi-schema/enum-missing-data.exit
create mode 100644 tests/qapi-schema/enum-missing-data.json
create mode 100644 tests/qapi-schema/enum-missing-data.out
create mode 100644 tests/qapi-schema/enum-union-clash.err
create mode 100644 tests/qapi-schema/enum-union-clash.exit
create mode 100644 tests/qapi-schema/enum-union-clash.json
create mode 100644 tests/qapi-schema/enum-union-clash.out
create mode 100644 tests/qapi-schema/enum-wrong-data.err
create mode 100644 tests/qapi-schema/enum-wrong-data.exit
create mode 100644 tests/qapi-schema/enum-wrong-data.json
create mode 100644 tests/qapi-schema/enum-wrong-data.out
diff --git a/tests/Makefile b/tests/Makefile
index 55aa745..68bc353 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -207,7 +207,10 @@ $(foreach target,$(SYSEMU_TARGET_LIST), \
$(eval check-qtest-$(target)-y += tests/qom-test$(EXESUF))))
check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
- comments.json empty.json funny-char.json indented-expr.json \
+ comments.json empty.json enum-empty.json enum-missing-data.json \
+ enum-wrong-data.json enum-int-member.json enum-dict-member.json \
+ enum-clash-member.json enum-max-member.json enum-union-clash.json \
+ funny-char.json indented-expr.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/enum-clash-member.err b/tests/qapi-schema/enum-clash-member.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-clash-member.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
new file mode 100644
index 0000000..99d442a
--- /dev/null
+++ b/tests/qapi-schema/enum-clash-member.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject enums where members will clash when mapped to C enum
+{ 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
diff --git a/tests/qapi-schema/enum-clash-member.out b/tests/qapi-schema/enum-clash-member.out
new file mode 100644
index 0000000..0814459
--- /dev/null
+++ b/tests/qapi-schema/enum-clash-member.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', ['one', 'ONE'])])]
+[{'enum_name': 'MyEnum', 'enum_values': ['one', 'ONE']}]
+[]
diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-dict-member.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
new file mode 100644
index 0000000..de4d6bf
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject any enum member that is not a string
+{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-dict-member.out
new file mode 100644
index 0000000..8b293f8
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', [OrderedDict([('value', 'str')])])])]
+[{'enum_name': 'MyEnum', 'enum_values': [OrderedDict([('value', 'str')])]}]
+[]
diff --git a/tests/qapi-schema/enum-empty.err b/tests/qapi-schema/enum-empty.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-empty.exit b/tests/qapi-schema/enum-empty.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-empty.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-empty.json b/tests/qapi-schema/enum-empty.json
new file mode 100644
index 0000000..40d4e85
--- /dev/null
+++ b/tests/qapi-schema/enum-empty.json
@@ -0,0 +1,2 @@
+# An empty enum, although unusual, is currently acceptable
+{ 'enum': 'MyEnum', 'data': [ ] }
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
new file mode 100644
index 0000000..3b75c16
--- /dev/null
+++ b/tests/qapi-schema/enum-empty.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', [])])]
+[{'enum_name': 'MyEnum', 'enum_values': []}]
+[]
diff --git a/tests/qapi-schema/enum-int-member.err b/tests/qapi-schema/enum-int-member.err
new file mode 100644
index 0000000..071c521
--- /dev/null
+++ b/tests/qapi-schema/enum-int-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-int-member.json:3:31: Stray "1"
diff --git a/tests/qapi-schema/enum-int-member.exit b/tests/qapi-schema/enum-int-member.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/enum-int-member.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/enum-int-member.json b/tests/qapi-schema/enum-int-member.json
new file mode 100644
index 0000000..6c9c32e
--- /dev/null
+++ b/tests/qapi-schema/enum-int-member.json
@@ -0,0 +1,3 @@
+# we reject any enum member that is not a string
+# FIXME: once the parser understands integer inputs, improve the error message
+{ 'enum': 'MyEnum', 'data': [ 1 ] }
diff --git a/tests/qapi-schema/enum-int-member.out b/tests/qapi-schema/enum-int-member.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-max-member.err b/tests/qapi-schema/enum-max-member.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-max-member.exit b/tests/qapi-schema/enum-max-member.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-max-member.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-max-member.json b/tests/qapi-schema/enum-max-member.json
new file mode 100644
index 0000000..ea854c4
--- /dev/null
+++ b/tests/qapi-schema/enum-max-member.json
@@ -0,0 +1,3 @@
+# FIXME: we should either reject user-supplied 'max', or munge the implicit
+# max value we generate at the end of an array
+{ 'enum': 'MyEnum', 'data': [ 'max' ] }
diff --git a/tests/qapi-schema/enum-max-member.out b/tests/qapi-schema/enum-max-member.out
new file mode 100644
index 0000000..c933044
--- /dev/null
+++ b/tests/qapi-schema/enum-max-member.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', ['max'])])]
+[{'enum_name': 'MyEnum', 'enum_values': ['max']}]
+[]
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
new file mode 100644
index 0000000..814ab26
--- /dev/null
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -0,0 +1,6 @@
+Traceback (most recent call last):
+ File "tests/qapi-schema/test-qapi.py", line 19, in <module>
+ exprs = parse_schema(sys.argv[1])
+ File "scripts/qapi.py", line 334, in parse_schema
+ add_enum(expr['enum'], expr['data'])
+KeyError: 'data'
diff --git a/tests/qapi-schema/enum-missing-data.exit b/tests/qapi-schema/enum-missing-data.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/enum-missing-data.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/enum-missing-data.json b/tests/qapi-schema/enum-missing-data.json
new file mode 100644
index 0000000..01f3f32
--- /dev/null
+++ b/tests/qapi-schema/enum-missing-data.json
@@ -0,0 +1,2 @@
+# FIXME: we should require that all QAPI enums have a data array
+{ 'enum': 'MyEnum' }
diff --git a/tests/qapi-schema/enum-missing-data.out b/tests/qapi-schema/enum-missing-data.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-union-clash.err b/tests/qapi-schema/enum-union-clash.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-union-clash.exit b/tests/qapi-schema/enum-union-clash.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-union-clash.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-union-clash.json b/tests/qapi-schema/enum-union-clash.json
new file mode 100644
index 0000000..714ff6d
--- /dev/null
+++ b/tests/qapi-schema/enum-union-clash.json
@@ -0,0 +1,4 @@
+# FIXME: we should reject types that would conflict with implicit union enum
+{ 'enum': 'UnionKind', 'data': [ 'oops' ] }
+{ 'union': 'Union',
+ 'data': { 'a': 'int' } }
diff --git a/tests/qapi-schema/enum-union-clash.out b/tests/qapi-schema/enum-union-clash.out
new file mode 100644
index 0000000..d45f5e8
--- /dev/null
+++ b/tests/qapi-schema/enum-union-clash.out
@@ -0,0 +1,5 @@
+[OrderedDict([('enum', 'UnionKind'), ('data', ['oops'])]),
+ OrderedDict([('union', 'Union'), ('data', OrderedDict([('a', 'int')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': ['oops']},
+ {'enum_name': 'UnionKind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/enum-wrong-data.err b/tests/qapi-schema/enum-wrong-data.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-wrong-data.exit b/tests/qapi-schema/enum-wrong-data.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-wrong-data.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-wrong-data.json b/tests/qapi-schema/enum-wrong-data.json
new file mode 100644
index 0000000..61d25ec
--- /dev/null
+++ b/tests/qapi-schema/enum-wrong-data.json
@@ -0,0 +1,2 @@
+# FIXME: we should require that all qapi enums have an array for data
+{ 'enum': 'MyEnum', 'data': { 'value': 'str' } }
diff --git a/tests/qapi-schema/enum-wrong-data.out b/tests/qapi-schema/enum-wrong-data.out
new file mode 100644
index 0000000..28d2211
--- /dev/null
+++ b/tests/qapi-schema/enum-wrong-data.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', OrderedDict([('value', 'str')]))])]
+[{'enum_name': 'MyEnum', 'enum_values': OrderedDict([('value', 'str')])}]
+[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 05/28] qapi: Better error messages for bad enums
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (3 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 04/28] qapi: Add some enum tests Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 10:08 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests Eric Blake
` (24 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
The previous commit demonstrated that the generator had several
flaws with less-than-perfect enums:
- an enum that listed the same string twice (or two variant
strings that map to the same C enumerator) ended up generating
an invalid C enum
- because the generator adds a _MAX terminator to each enum,
the use of an enum member 'max' can also cause this clash
- if an enum omits 'data', the generator left a python stack
trace rather than a graceful message
- an enum that used a non-array 'data' was silently accepted by
the parser
- an enum that used non-string members in the 'data' member
was silently accepted by the parser
Add check_enum to cover these situations, and update testcases
to match. While valid .json files won't trigger any of these
cases, we might as well be nicer to developers that make a typo
while trying to add new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 34 +++++++++++++++++++++++++++-----
tests/qapi-schema/enum-clash-member.err | 1 +
tests/qapi-schema/enum-clash-member.exit | 2 +-
tests/qapi-schema/enum-clash-member.json | 2 +-
tests/qapi-schema/enum-clash-member.out | 3 ---
tests/qapi-schema/enum-dict-member.err | 1 +
tests/qapi-schema/enum-dict-member.exit | 2 +-
tests/qapi-schema/enum-dict-member.json | 2 +-
tests/qapi-schema/enum-dict-member.out | 3 ---
tests/qapi-schema/enum-max-member.err | 1 +
tests/qapi-schema/enum-max-member.exit | 2 +-
tests/qapi-schema/enum-max-member.json | 4 ++--
tests/qapi-schema/enum-max-member.out | 3 ---
tests/qapi-schema/enum-missing-data.err | 7 +------
tests/qapi-schema/enum-missing-data.json | 2 +-
tests/qapi-schema/enum-wrong-data.err | 1 +
tests/qapi-schema/enum-wrong-data.exit | 2 +-
tests/qapi-schema/enum-wrong-data.json | 2 +-
tests/qapi-schema/enum-wrong-data.out | 3 ---
19 files changed, 44 insertions(+), 33 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 20ee505..3ce8c33 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -311,13 +311,37 @@ def check_union(expr, expr_info):
# Todo: add checking for values. Key is checked as above, value can be
# also checked here, but we need more functions to handle array case.
+def check_enum(expr, expr_info):
+ name = expr['enum']
+ members = expr.get('data')
+ values = { 'MAX': '(automatic)' }
+
+ if not isinstance(members, list):
+ raise QAPIExprError(expr_info,
+ "Enum '%s' requires an array for 'data'" % name)
+ for member in members:
+ if not isinstance(member, str):
+ raise QAPIExprError(expr_info,
+ "Enum '%s' member '%s' is not a string"
+ % (name, member))
+ key = _generate_enum_string(member)
+ if key in values:
+ raise QAPIExprError(expr_info,
+ "Enum '%s' member '%s' clashes with '%s'"
+ % (name, member, values[key]))
+ values[key] = member
+
def check_exprs(schema):
for expr_elem in schema.exprs:
expr = expr_elem['expr']
- if expr.has_key('union'):
- check_union(expr, expr_elem['info'])
- if expr.has_key('event'):
- check_event(expr, expr_elem['info'])
+ info = expr_elem['info']
+
+ if expr.has_key('enum'):
+ check_enum(expr, info)
+ elif expr.has_key('union'):
+ check_union(expr, info)
+ elif expr.has_key('event'):
+ check_event(expr, info)
def parse_schema(input_file):
try:
@@ -331,7 +355,7 @@ def parse_schema(input_file):
for expr_elem in schema.exprs:
expr = expr_elem['expr']
if expr.has_key('enum'):
- add_enum(expr['enum'], expr['data'])
+ add_enum(expr['enum'], expr.get('data'))
elif expr.has_key('union'):
add_union(expr)
elif expr.has_key('type'):
diff --git a/tests/qapi-schema/enum-clash-member.err b/tests/qapi-schema/enum-clash-member.err
index e69de29..48bd136 100644
--- a/tests/qapi-schema/enum-clash-member.err
+++ b/tests/qapi-schema/enum-clash-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-clash-member.json:2: Enum 'MyEnum' member 'ONE' clashes with 'one'
diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-clash-member.exit
+++ b/tests/qapi-schema/enum-clash-member.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
index 99d442a..b7dc02a 100644
--- a/tests/qapi-schema/enum-clash-member.json
+++ b/tests/qapi-schema/enum-clash-member.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject enums where members will clash when mapped to C enum
+# we reject enums where members will clash when mapped to C enum
{ 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
diff --git a/tests/qapi-schema/enum-clash-member.out b/tests/qapi-schema/enum-clash-member.out
index 0814459..e69de29 100644
--- a/tests/qapi-schema/enum-clash-member.out
+++ b/tests/qapi-schema/enum-clash-member.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', ['one', 'ONE'])])]
-[{'enum_name': 'MyEnum', 'enum_values': ['one', 'ONE']}]
-[]
diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
index e69de29..7e966a8 100644
--- a/tests/qapi-schema/enum-dict-member.err
+++ b/tests/qapi-schema/enum-dict-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-dict-member.json:2: Enum 'MyEnum' member 'OrderedDict([('value', 'str')])' is not a string
diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-dict-member.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-dict-member.exit
+++ b/tests/qapi-schema/enum-dict-member.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
index de4d6bf..79672e0 100644
--- a/tests/qapi-schema/enum-dict-member.json
+++ b/tests/qapi-schema/enum-dict-member.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject any enum member that is not a string
+# we reject any enum member that is not a string
{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-dict-member.out
index 8b293f8..e69de29 100644
--- a/tests/qapi-schema/enum-dict-member.out
+++ b/tests/qapi-schema/enum-dict-member.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', [OrderedDict([('value', 'str')])])])]
-[{'enum_name': 'MyEnum', 'enum_values': [OrderedDict([('value', 'str')])]}]
-[]
diff --git a/tests/qapi-schema/enum-max-member.err b/tests/qapi-schema/enum-max-member.err
index e69de29..f77837f 100644
--- a/tests/qapi-schema/enum-max-member.err
+++ b/tests/qapi-schema/enum-max-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-max-member.json:3: Enum 'MyEnum' member 'max' clashes with '(automatic)'
diff --git a/tests/qapi-schema/enum-max-member.exit b/tests/qapi-schema/enum-max-member.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-max-member.exit
+++ b/tests/qapi-schema/enum-max-member.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-max-member.json b/tests/qapi-schema/enum-max-member.json
index ea854c4..6af4662 100644
--- a/tests/qapi-schema/enum-max-member.json
+++ b/tests/qapi-schema/enum-max-member.json
@@ -1,3 +1,3 @@
-# FIXME: we should either reject user-supplied 'max', or munge the implicit
-# max value we generate at the end of an array
+# we reject user-supplied 'max' for clashing with implicit enum end
+# FIXME: should we instead munge the the implicit value to avoid the clash?
{ 'enum': 'MyEnum', 'data': [ 'max' ] }
diff --git a/tests/qapi-schema/enum-max-member.out b/tests/qapi-schema/enum-max-member.out
index c933044..e69de29 100644
--- a/tests/qapi-schema/enum-max-member.out
+++ b/tests/qapi-schema/enum-max-member.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', ['max'])])]
-[{'enum_name': 'MyEnum', 'enum_values': ['max']}]
-[]
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
index 814ab26..b8ccae0 100644
--- a/tests/qapi-schema/enum-missing-data.err
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -1,6 +1 @@
-Traceback (most recent call last):
- File "tests/qapi-schema/test-qapi.py", line 19, in <module>
- exprs = parse_schema(sys.argv[1])
- File "scripts/qapi.py", line 334, in parse_schema
- add_enum(expr['enum'], expr['data'])
-KeyError: 'data'
+tests/qapi-schema/enum-missing-data.json:2: Enum 'MyEnum' requires an array for 'data'
diff --git a/tests/qapi-schema/enum-missing-data.json b/tests/qapi-schema/enum-missing-data.json
index 01f3f32..558fd35 100644
--- a/tests/qapi-schema/enum-missing-data.json
+++ b/tests/qapi-schema/enum-missing-data.json
@@ -1,2 +1,2 @@
-# FIXME: we should require that all QAPI enums have a data array
+# we require that all QAPI enums have a data array
{ 'enum': 'MyEnum' }
diff --git a/tests/qapi-schema/enum-wrong-data.err b/tests/qapi-schema/enum-wrong-data.err
index e69de29..11b4347 100644
--- a/tests/qapi-schema/enum-wrong-data.err
+++ b/tests/qapi-schema/enum-wrong-data.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-wrong-data.json:2: Enum 'MyEnum' requires an array for 'data'
diff --git a/tests/qapi-schema/enum-wrong-data.exit b/tests/qapi-schema/enum-wrong-data.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-wrong-data.exit
+++ b/tests/qapi-schema/enum-wrong-data.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-wrong-data.json b/tests/qapi-schema/enum-wrong-data.json
index 61d25ec..7b3e255 100644
--- a/tests/qapi-schema/enum-wrong-data.json
+++ b/tests/qapi-schema/enum-wrong-data.json
@@ -1,2 +1,2 @@
-# FIXME: we should require that all qapi enums have an array for data
+# we require that all qapi enums have an array for data
{ 'enum': 'MyEnum', 'data': { 'value': 'str' } }
diff --git a/tests/qapi-schema/enum-wrong-data.out b/tests/qapi-schema/enum-wrong-data.out
index 28d2211..e69de29 100644
--- a/tests/qapi-schema/enum-wrong-data.out
+++ b/tests/qapi-schema/enum-wrong-data.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', OrderedDict([('value', 'str')]))])]
-[{'enum_name': 'MyEnum', 'enum_values': OrderedDict([('value', 'str')])}]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (4 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 05/28] qapi: Better error messages for bad enums Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 13:18 ` Markus Armbruster
2015-03-26 13:23 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 07/28] qapi: Simplify tests of simple unions Eric Blake
` (23 subsequent siblings)
29 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Demonstrate that the qapi generator doesn't deal well with unions
that aren't up to par. Later patches will update the expected
reseults as the generator is made stricter.
Of particular note, we currently allow 'base' without 'discriminator'
as a way to create a simple union with a base class. However, none
of the existing QMP or QGA interfaces use it (it is exercised only
in the testsuite). Meanwhile, it would be nice to allow
'discriminator':'EnumType' without 'base' for creating a simple union
that is type-safe rather than open-coded to a generated enum (right
now, we are only type-safe when using a flat union, but that uses
a different syntax of 'discriminator':'member-name' which requires
a base class containing a 'member-name' enum field). If both 'base'
and 'discriminator' are optional, then converting a simple union
with base class to a type-safe simple union with an enum discriminator
would not be possible. So my plan is to get rid of 'base' without
'discriminator' later in the series; this will be no real loss, as any
union that needs additional common fields can always use a flat
union.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 12 +++++++++---
tests/qapi-schema/alternate-array.err | 0
tests/qapi-schema/alternate-array.exit | 1 +
tests/qapi-schema/alternate-array.json | 8 ++++++++
tests/qapi-schema/alternate-array.out | 4 ++++
tests/qapi-schema/alternate-base.err | 0
tests/qapi-schema/alternate-base.exit | 1 +
tests/qapi-schema/alternate-base.json | 7 +++++++
tests/qapi-schema/alternate-base.out | 4 ++++
tests/qapi-schema/alternate-clash.err | 0
tests/qapi-schema/alternate-clash.exit | 1 +
tests/qapi-schema/alternate-clash.json | 4 ++++
tests/qapi-schema/alternate-clash.out | 3 +++
tests/qapi-schema/alternate-conflict-dict.err | 0
tests/qapi-schema/alternate-conflict-dict.exit | 1 +
tests/qapi-schema/alternate-conflict-dict.json | 9 +++++++++
tests/qapi-schema/alternate-conflict-dict.out | 6 ++++++
tests/qapi-schema/alternate-conflict-string.err | 0
tests/qapi-schema/alternate-conflict-string.exit | 1 +
tests/qapi-schema/alternate-conflict-string.json | 8 ++++++++
tests/qapi-schema/alternate-conflict-string.out | 5 +++++
tests/qapi-schema/alternate-good.err | 0
tests/qapi-schema/alternate-good.exit | 1 +
tests/qapi-schema/alternate-good.json | 10 ++++++++++
tests/qapi-schema/alternate-good.out | 6 ++++++
tests/qapi-schema/alternate-nested.err | 0
tests/qapi-schema/alternate-nested.exit | 1 +
tests/qapi-schema/alternate-nested.json | 7 +++++++
tests/qapi-schema/alternate-nested.out | 5 +++++
tests/qapi-schema/alternate-unknown.err | 0
tests/qapi-schema/alternate-unknown.exit | 1 +
tests/qapi-schema/alternate-unknown.json | 4 ++++
tests/qapi-schema/alternate-unknown.out | 3 +++
tests/qapi-schema/flat-union-bad-base.err | 1 +
tests/qapi-schema/flat-union-bad-base.exit | 1 +
tests/qapi-schema/flat-union-bad-base.json | 13 +++++++++++++
tests/qapi-schema/flat-union-bad-base.out | 0
tests/qapi-schema/flat-union-bad-discriminator.err | 0
tests/qapi-schema/flat-union-bad-discriminator.exit | 1 +
tests/qapi-schema/flat-union-bad-discriminator.json | 14 ++++++++++++++
tests/qapi-schema/flat-union-bad-discriminator.out | 10 ++++++++++
tests/qapi-schema/flat-union-base-union.err | 1 +
tests/qapi-schema/flat-union-base-union.exit | 1 +
tests/qapi-schema/flat-union-base-union.json | 15 +++++++++++++++
tests/qapi-schema/flat-union-base-union.out | 0
tests/qapi-schema/flat-union-no-base.err | 2 +-
tests/qapi-schema/flat-union-no-base.json | 7 ++++---
tests/qapi-schema/flat-union-optional-discriminator.err | 0
tests/qapi-schema/flat-union-optional-discriminator.exit | 1 +
tests/qapi-schema/flat-union-optional-discriminator.json | 9 +++++++++
tests/qapi-schema/flat-union-optional-discriminator.out | 5 +++++
tests/qapi-schema/union-bad-branch.err | 0
tests/qapi-schema/union-bad-branch.exit | 1 +
tests/qapi-schema/union-bad-branch.json | 8 ++++++++
tests/qapi-schema/union-bad-branch.out | 6 ++++++
tests/qapi-schema/union-base-no-discriminator.err | 0
tests/qapi-schema/union-base-no-discriminator.exit | 1 +
tests/qapi-schema/union-base-no-discriminator.json | 14 ++++++++++++++
tests/qapi-schema/union-base-no-discriminator.out | 8 ++++++++
tests/qapi-schema/union-invalid-base.err | 2 +-
tests/qapi-schema/union-invalid-base.json | 4 +++-
tests/qapi-schema/union-max.err | 0
tests/qapi-schema/union-max.exit | 1 +
tests/qapi-schema/union-max.json | 3 +++
tests/qapi-schema/union-max.out | 3 +++
tests/qapi-schema/union-optional-branch.err | 0
tests/qapi-schema/union-optional-branch.exit | 1 +
tests/qapi-schema/union-optional-branch.json | 2 ++
tests/qapi-schema/union-optional-branch.out | 3 +++
tests/qapi-schema/union-unknown.err | 0
tests/qapi-schema/union-unknown.exit | 1 +
tests/qapi-schema/union-unknown.json | 3 +++
tests/qapi-schema/union-unknown.out | 3 +++
73 files changed, 249 insertions(+), 9 deletions(-)
create mode 100644 tests/qapi-schema/alternate-array.err
create mode 100644 tests/qapi-schema/alternate-array.exit
create mode 100644 tests/qapi-schema/alternate-array.json
create mode 100644 tests/qapi-schema/alternate-array.out
create mode 100644 tests/qapi-schema/alternate-base.err
create mode 100644 tests/qapi-schema/alternate-base.exit
create mode 100644 tests/qapi-schema/alternate-base.json
create mode 100644 tests/qapi-schema/alternate-base.out
create mode 100644 tests/qapi-schema/alternate-clash.err
create mode 100644 tests/qapi-schema/alternate-clash.exit
create mode 100644 tests/qapi-schema/alternate-clash.json
create mode 100644 tests/qapi-schema/alternate-clash.out
create mode 100644 tests/qapi-schema/alternate-conflict-dict.err
create mode 100644 tests/qapi-schema/alternate-conflict-dict.exit
create mode 100644 tests/qapi-schema/alternate-conflict-dict.json
create mode 100644 tests/qapi-schema/alternate-conflict-dict.out
create mode 100644 tests/qapi-schema/alternate-conflict-string.err
create mode 100644 tests/qapi-schema/alternate-conflict-string.exit
create mode 100644 tests/qapi-schema/alternate-conflict-string.json
create mode 100644 tests/qapi-schema/alternate-conflict-string.out
create mode 100644 tests/qapi-schema/alternate-good.err
create mode 100644 tests/qapi-schema/alternate-good.exit
create mode 100644 tests/qapi-schema/alternate-good.json
create mode 100644 tests/qapi-schema/alternate-good.out
create mode 100644 tests/qapi-schema/alternate-nested.err
create mode 100644 tests/qapi-schema/alternate-nested.exit
create mode 100644 tests/qapi-schema/alternate-nested.json
create mode 100644 tests/qapi-schema/alternate-nested.out
create mode 100644 tests/qapi-schema/alternate-unknown.err
create mode 100644 tests/qapi-schema/alternate-unknown.exit
create mode 100644 tests/qapi-schema/alternate-unknown.json
create mode 100644 tests/qapi-schema/alternate-unknown.out
create mode 100644 tests/qapi-schema/flat-union-bad-base.err
create mode 100644 tests/qapi-schema/flat-union-bad-base.exit
create mode 100644 tests/qapi-schema/flat-union-bad-base.json
create mode 100644 tests/qapi-schema/flat-union-bad-base.out
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.err
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.exit
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.json
create mode 100644 tests/qapi-schema/flat-union-bad-discriminator.out
create mode 100644 tests/qapi-schema/flat-union-base-union.err
create mode 100644 tests/qapi-schema/flat-union-base-union.exit
create mode 100644 tests/qapi-schema/flat-union-base-union.json
create mode 100644 tests/qapi-schema/flat-union-base-union.out
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.err
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.exit
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.json
create mode 100644 tests/qapi-schema/flat-union-optional-discriminator.out
create mode 100644 tests/qapi-schema/union-bad-branch.err
create mode 100644 tests/qapi-schema/union-bad-branch.exit
create mode 100644 tests/qapi-schema/union-bad-branch.json
create mode 100644 tests/qapi-schema/union-bad-branch.out
create mode 100644 tests/qapi-schema/union-base-no-discriminator.err
create mode 100644 tests/qapi-schema/union-base-no-discriminator.exit
create mode 100644 tests/qapi-schema/union-base-no-discriminator.json
create mode 100644 tests/qapi-schema/union-base-no-discriminator.out
create mode 100644 tests/qapi-schema/union-max.err
create mode 100644 tests/qapi-schema/union-max.exit
create mode 100644 tests/qapi-schema/union-max.json
create mode 100644 tests/qapi-schema/union-max.out
create mode 100644 tests/qapi-schema/union-optional-branch.err
create mode 100644 tests/qapi-schema/union-optional-branch.exit
create mode 100644 tests/qapi-schema/union-optional-branch.json
create mode 100644 tests/qapi-schema/union-optional-branch.out
create mode 100644 tests/qapi-schema/union-unknown.err
create mode 100644 tests/qapi-schema/union-unknown.exit
create mode 100644 tests/qapi-schema/union-unknown.json
create mode 100644 tests/qapi-schema/union-unknown.out
diff --git a/tests/Makefile b/tests/Makefile
index 68bc353..f8bc2a8 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -216,10 +216,16 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
qapi-schema-test.json quoted-structural-chars.json \
trailing-comma-list.json trailing-comma-object.json \
unclosed-list.json unclosed-object.json unclosed-string.json \
- duplicate-key.json union-invalid-base.json flat-union-no-base.json \
- flat-union-invalid-discriminator.json \
+ duplicate-key.json union-invalid-base.json union-bad-branch.json \
+ union-optional-branch.json flat-union-optional-discriminator.json \
+ flat-union-no-base.json flat-union-invalid-discriminator.json \
flat-union-invalid-branch-key.json flat-union-reverse-define.json \
- flat-union-string-discriminator.json \
+ flat-union-string-discriminator.json union-base-no-discriminator.json \
+ flat-union-bad-discriminator.json flat-union-bad-base.json \
+ flat-union-base-union.json union-unknown.json union-max.json \
+ alternate-nested.json alternate-unknown.json alternate-clash.json \
+ alternate-good.json alternate-base.json alternate-array.json \
+ alternate-conflict-string.json alternate-conflict-dict.json \
include-simple.json include-relpath.json include-format-err.json \
include-non-file.json include-no-file.json include-before-err.json \
include-nested-err.json include-self-cycle.json include-cycle.json \
diff --git a/tests/qapi-schema/alternate-array.err b/tests/qapi-schema/alternate-array.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-array.exit b/tests/qapi-schema/alternate-array.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-array.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-array.json b/tests/qapi-schema/alternate-array.json
new file mode 100644
index 0000000..b399500
--- /dev/null
+++ b/tests/qapi-schema/alternate-array.json
@@ -0,0 +1,8 @@
+# FIXME: we do not support array branches of anonymous unions yet
+# FIXME: this should fail as long as we lack support
+{ 'type': 'One',
+ 'data': { 'name': 'str' } }
+{ 'union': 'MyUnion',
+ 'discriminator': {},
+ 'data': { 'one': 'One',
+ 'two': [ 'int' ] } }
diff --git a/tests/qapi-schema/alternate-array.out b/tests/qapi-schema/alternate-array.out
new file mode 100644
index 0000000..90dc22c
--- /dev/null
+++ b/tests/qapi-schema/alternate-array.out
@@ -0,0 +1,4 @@
+[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))]),
+ OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'One'), ('two', ['int'])]))])]
+[{'enum_name': 'MyUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))])]
diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-base.exit b/tests/qapi-schema/alternate-base.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-base.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-base.json b/tests/qapi-schema/alternate-base.json
new file mode 100644
index 0000000..2d36db1
--- /dev/null
+++ b/tests/qapi-schema/alternate-base.json
@@ -0,0 +1,7 @@
+# FIXME: we should reject anonymous union with base type
+{ 'type': 'Base',
+ 'data': { 'string': 'str' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': {},
+ 'data': { 'number': 'int' } }
diff --git a/tests/qapi-schema/alternate-base.out b/tests/qapi-schema/alternate-base.out
new file mode 100644
index 0000000..7fb31f5
--- /dev/null
+++ b/tests/qapi-schema/alternate-base.out
@@ -0,0 +1,4 @@
+[OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('union', 'MyUnion'), ('base', 'Base'), ('discriminator', OrderedDict()), ('data', OrderedDict([('number', 'int')]))])]
+[{'enum_name': 'MyUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))])]
diff --git a/tests/qapi-schema/alternate-clash.err b/tests/qapi-schema/alternate-clash.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-clash.exit b/tests/qapi-schema/alternate-clash.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-clash.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-clash.json b/tests/qapi-schema/alternate-clash.json
new file mode 100644
index 0000000..7e2ef23
--- /dev/null
+++ b/tests/qapi-schema/alternate-clash.json
@@ -0,0 +1,4 @@
+# FIXME: we should detect C enum collisions in an anonymous union
+{ 'union': 'Union1',
+ 'discriminator': {},
+ 'data': { 'one': 'str', 'ONE': 'int' } }
diff --git a/tests/qapi-schema/alternate-clash.out b/tests/qapi-schema/alternate-clash.out
new file mode 100644
index 0000000..c6687fa
--- /dev/null
+++ b/tests/qapi-schema/alternate-clash.out
@@ -0,0 +1,3 @@
+[OrderedDict([('union', 'Union1'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'str'), ('ONE', 'int')]))])]
+[{'enum_name': 'Union1Kind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/alternate-conflict-dict.err b/tests/qapi-schema/alternate-conflict-dict.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-conflict-dict.exit b/tests/qapi-schema/alternate-conflict-dict.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-dict.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-conflict-dict.json b/tests/qapi-schema/alternate-conflict-dict.json
new file mode 100644
index 0000000..70fe089
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-dict.json
@@ -0,0 +1,9 @@
+# FIXME: we should reject anonymous unions with multiple object branches
+{ 'type': 'One',
+ 'data': { 'name': 'str' } }
+{ 'type': 'Two',
+ 'data': { 'value': 'int' } }
+{ 'union': 'MyUnion',
+ 'discriminator': {},
+ 'data': { 'one': 'One',
+ 'two': 'Two' } }
diff --git a/tests/qapi-schema/alternate-conflict-dict.out b/tests/qapi-schema/alternate-conflict-dict.out
new file mode 100644
index 0000000..b9ac945
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-dict.out
@@ -0,0 +1,6 @@
+[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))]),
+ OrderedDict([('type', 'Two'), ('data', OrderedDict([('value', 'int')]))]),
+ OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'One'), ('two', 'Two')]))])]
+[{'enum_name': 'MyUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))]),
+ OrderedDict([('type', 'Two'), ('data', OrderedDict([('value', 'int')]))])]
diff --git a/tests/qapi-schema/alternate-conflict-string.err b/tests/qapi-schema/alternate-conflict-string.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-conflict-string.exit b/tests/qapi-schema/alternate-conflict-string.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-string.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
new file mode 100644
index 0000000..5fd1a47
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-string.json
@@ -0,0 +1,8 @@
+# FIXME: we should reject anonymous unions with multiple string-like branches
+{ 'enum': 'Enum',
+ 'data': [ 'hello', 'world' ] }
+{ 'union': 'MyUnion',
+ 'discriminator': {},
+ 'data': { 'one': 'str',
+ 'two': 'Enum' } }
+
diff --git a/tests/qapi-schema/alternate-conflict-string.out b/tests/qapi-schema/alternate-conflict-string.out
new file mode 100644
index 0000000..e7b39a2
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-string.out
@@ -0,0 +1,5 @@
+[OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
+ OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'str'), ('two', 'Enum')]))])]
+[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
+ {'enum_name': 'MyUnionKind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/alternate-good.err b/tests/qapi-schema/alternate-good.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-good.exit b/tests/qapi-schema/alternate-good.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-good.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-good.json b/tests/qapi-schema/alternate-good.json
new file mode 100644
index 0000000..1068e2f
--- /dev/null
+++ b/tests/qapi-schema/alternate-good.json
@@ -0,0 +1,10 @@
+# Working example of anonymous union
+{ 'type': 'Data',
+ 'data': { '*number': 'int', '*name': 'str' } }
+{ 'enum': 'Enum',
+ 'data': [ 'hello', 'world' ] }
+{ 'union': 'MyUnion',
+ 'discriminator': {},
+ 'data': { 'value': 'int',
+ 'string': 'Enum',
+ 'struct': 'Data' } }
diff --git a/tests/qapi-schema/alternate-good.out b/tests/qapi-schema/alternate-good.out
new file mode 100644
index 0000000..b5117d1
--- /dev/null
+++ b/tests/qapi-schema/alternate-good.out
@@ -0,0 +1,6 @@
+[OrderedDict([('type', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))]),
+ OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
+ OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('value', 'int'), ('string', 'Enum'), ('struct', 'Data')]))])]
+[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
+ {'enum_name': 'MyUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))])]
diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-nested.exit b/tests/qapi-schema/alternate-nested.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-nested.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-nested.json b/tests/qapi-schema/alternate-nested.json
new file mode 100644
index 0000000..d5812bf
--- /dev/null
+++ b/tests/qapi-schema/alternate-nested.json
@@ -0,0 +1,7 @@
+# FIXME: we should reject a nested anonymous union branch
+{ 'union': 'Union1',
+ 'discriminator': {},
+ 'data': { 'name': 'str', 'value': 'int' } }
+{ 'union': 'Union2',
+ 'discriminator': {},
+ 'data': { 'nested': 'Union1' } }
diff --git a/tests/qapi-schema/alternate-nested.out b/tests/qapi-schema/alternate-nested.out
new file mode 100644
index 0000000..0137c1f
--- /dev/null
+++ b/tests/qapi-schema/alternate-nested.out
@@ -0,0 +1,5 @@
+[OrderedDict([('union', 'Union1'), ('discriminator', OrderedDict()), ('data', OrderedDict([('name', 'str'), ('value', 'int')]))]),
+ OrderedDict([('union', 'Union2'), ('discriminator', OrderedDict()), ('data', OrderedDict([('nested', 'Union1')]))])]
+[{'enum_name': 'Union1Kind', 'enum_values': None},
+ {'enum_name': 'Union2Kind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/alternate-unknown.err b/tests/qapi-schema/alternate-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/alternate-unknown.exit b/tests/qapi-schema/alternate-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/alternate-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/alternate-unknown.json b/tests/qapi-schema/alternate-unknown.json
new file mode 100644
index 0000000..0bab9c2
--- /dev/null
+++ b/tests/qapi-schema/alternate-unknown.json
@@ -0,0 +1,4 @@
+# FIXME: we should reject an anonymous union with unknown type in branch
+{ 'union': 'Union',
+ 'discriminator': {},
+ 'data': { 'unknown': 'MissingType' } }
diff --git a/tests/qapi-schema/alternate-unknown.out b/tests/qapi-schema/alternate-unknown.out
new file mode 100644
index 0000000..0911cdc
--- /dev/null
+++ b/tests/qapi-schema/alternate-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('union', 'Union'), ('discriminator', OrderedDict()), ('data', OrderedDict([('unknown', 'MissingType')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/flat-union-bad-base.err b/tests/qapi-schema/flat-union-bad-base.err
new file mode 100644
index 0000000..5962ff4
--- /dev/null
+++ b/tests/qapi-schema/flat-union-bad-base.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-bad-base.json:9: Base 'OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')])' is not a valid type
diff --git a/tests/qapi-schema/flat-union-bad-base.exit b/tests/qapi-schema/flat-union-bad-base.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/flat-union-bad-base.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-bad-base.json b/tests/qapi-schema/flat-union-bad-base.json
new file mode 100644
index 0000000..d69168f
--- /dev/null
+++ b/tests/qapi-schema/flat-union-bad-base.json
@@ -0,0 +1,13 @@
+# we require the base to be an existing complex type
+# FIXME: should we allow an anonymous inline base type?
+{ 'enum': 'TestEnum',
+ 'data': [ 'value1', 'value2' ] }
+{ 'type': 'TestTypeA',
+ 'data': { 'string': 'str' } }
+{ 'type': 'TestTypeB',
+ 'data': { 'integer': 'int' } }
+{ 'union': 'TestUnion',
+ 'base': { 'enum1': 'TestEnum', 'kind': 'str' },
+ 'discriminator': 'TestEnum',
+ 'data': { 'kind1': 'TestTypeA',
+ 'kind2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-bad-base.out b/tests/qapi-schema/flat-union-bad-base.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.err b/tests/qapi-schema/flat-union-bad-discriminator.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.exit b/tests/qapi-schema/flat-union-bad-discriminator.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/flat-union-bad-discriminator.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.json b/tests/qapi-schema/flat-union-bad-discriminator.json
new file mode 100644
index 0000000..1599a59
--- /dev/null
+++ b/tests/qapi-schema/flat-union-bad-discriminator.json
@@ -0,0 +1,14 @@
+# FIXME: we should require the discriminator to be a string naming a base-type member
+{ 'enum': 'TestEnum',
+ 'data': [ 'value1', 'value2' ] }
+{ 'type': 'TestBase',
+ 'data': { 'enum1': 'TestEnum', 'kind': 'str' } }
+{ 'type': 'TestTypeA',
+ 'data': { 'string': 'str' } }
+{ 'type': 'TestTypeB',
+ 'data': { 'integer': 'int' } }
+{ 'union': 'TestUnion',
+ 'base': 'TestBase',
+ 'discriminator': [],
+ 'data': { 'kind1': 'TestTypeA',
+ 'kind2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.out b/tests/qapi-schema/flat-union-bad-discriminator.out
new file mode 100644
index 0000000..b6ce217
--- /dev/null
+++ b/tests/qapi-schema/flat-union-bad-discriminator.out
@@ -0,0 +1,10 @@
+[OrderedDict([('enum', 'TestEnum'), ('data', ['value1', 'value2'])]),
+ OrderedDict([('type', 'TestBase'), ('data', OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')]))]),
+ OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
+ OrderedDict([('union', 'TestUnion'), ('base', 'TestBase'), ('discriminator', []), ('data', OrderedDict([('kind1', 'TestTypeA'), ('kind2', 'TestTypeB')]))])]
+[{'enum_name': 'TestEnum', 'enum_values': ['value1', 'value2']},
+ {'enum_name': 'TestUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'TestBase'), ('data', OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')]))]),
+ OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))])]
diff --git a/tests/qapi-schema/flat-union-base-union.err b/tests/qapi-schema/flat-union-base-union.err
new file mode 100644
index 0000000..185bf51
--- /dev/null
+++ b/tests/qapi-schema/flat-union-base-union.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-base-union.json:11: Base 'UnionBase' is not a valid type
diff --git a/tests/qapi-schema/flat-union-base-union.exit b/tests/qapi-schema/flat-union-base-union.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/flat-union-base-union.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/flat-union-base-union.json b/tests/qapi-schema/flat-union-base-union.json
new file mode 100644
index 0000000..bbaa2da
--- /dev/null
+++ b/tests/qapi-schema/flat-union-base-union.json
@@ -0,0 +1,15 @@
+# FIXME: the error message needs help: we require the base to be a struct
+{ 'enum': 'TestEnum',
+ 'data': [ 'value1', 'value2' ] }
+{ 'type': 'TestTypeA',
+ 'data': { 'string': 'str' } }
+{ 'type': 'TestTypeB',
+ 'data': { 'integer': 'int' } }
+{ 'union': 'UnionBase',
+ 'data': { 'kind1': 'TestTypeA',
+ 'kind2': 'TestTypeB' } }
+{ 'union': 'TestUnion',
+ 'base': 'UnionBase',
+ 'discriminator': 'type',
+ 'data': { 'kind1': 'TestTypeA',
+ 'kind2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-base-union.out b/tests/qapi-schema/flat-union-base-union.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/flat-union-no-base.err b/tests/qapi-schema/flat-union-no-base.err
index a59749e..eaf3592 100644
--- a/tests/qapi-schema/flat-union-no-base.err
+++ b/tests/qapi-schema/flat-union-no-base.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-no-base.json:7: Flat union 'TestUnion' must have a base field
+tests/qapi-schema/flat-union-no-base.json:8: Flat union 'TestUnion' must have a base field
diff --git a/tests/qapi-schema/flat-union-no-base.json b/tests/qapi-schema/flat-union-no-base.json
index 50f2673..6d8dc7f 100644
--- a/tests/qapi-schema/flat-union-no-base.json
+++ b/tests/qapi-schema/flat-union-no-base.json
@@ -1,10 +1,11 @@
+# FIXME: we should allow discriminator:type without base on non-flat unions
{ 'type': 'TestTypeA',
'data': { 'string': 'str' } }
-
{ 'type': 'TestTypeB',
'data': { 'integer': 'int' } }
-
+{ 'enum': 'Enum',
+ 'data': [ 'value1', 'value2' ] }
{ 'union': 'TestUnion',
- 'discriminator': 'enum1',
+ 'discriminator': 'Enum',
'data': { 'value1': 'TestTypeA',
'value2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.err b/tests/qapi-schema/flat-union-optional-discriminator.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.exit b/tests/qapi-schema/flat-union-optional-discriminator.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.json b/tests/qapi-schema/flat-union-optional-discriminator.json
new file mode 100644
index 0000000..4957c72
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator.json
@@ -0,0 +1,9 @@
+# FIXME: we should require the discriminator to be non-optional
+{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
+{ 'type': 'Base',
+ 'data': { '*switch': 'Enum' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': '*switch',
+ 'data': { 'one': 'int',
+ 'two': 'str' } }
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.out b/tests/qapi-schema/flat-union-optional-discriminator.out
new file mode 100644
index 0000000..f4b6bed
--- /dev/null
+++ b/tests/qapi-schema/flat-union-optional-discriminator.out
@@ -0,0 +1,5 @@
+[OrderedDict([('enum', 'Enum'), ('data', ['one', 'two'])]),
+ OrderedDict([('type', 'Base'), ('data', OrderedDict([('*switch', 'Enum')]))]),
+ OrderedDict([('union', 'MyUnion'), ('base', 'Base'), ('discriminator', '*switch'), ('data', OrderedDict([('one', 'int'), ('two', 'str')]))])]
+[{'enum_name': 'Enum', 'enum_values': ['one', 'two']}]
+[OrderedDict([('type', 'Base'), ('data', OrderedDict([('*switch', 'Enum')]))])]
diff --git a/tests/qapi-schema/union-bad-branch.err b/tests/qapi-schema/union-bad-branch.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/union-bad-branch.exit b/tests/qapi-schema/union-bad-branch.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/union-bad-branch.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/union-bad-branch.json b/tests/qapi-schema/union-bad-branch.json
new file mode 100644
index 0000000..f7aeae8
--- /dev/null
+++ b/tests/qapi-schema/union-bad-branch.json
@@ -0,0 +1,8 @@
+# FIXME: we should reject normal unions where branches would collide in C
+{ 'type': 'One',
+ 'data': { 'string': 'str' } }
+{ 'type': 'Two',
+ 'data': { 'number': 'int' } }
+{ 'union': 'MyUnion',
+ 'data': { 'one': 'One',
+ 'ONE': 'Two' } }
diff --git a/tests/qapi-schema/union-bad-branch.out b/tests/qapi-schema/union-bad-branch.out
new file mode 100644
index 0000000..6baf01b
--- /dev/null
+++ b/tests/qapi-schema/union-bad-branch.out
@@ -0,0 +1,6 @@
+[OrderedDict([('type', 'One'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('type', 'Two'), ('data', OrderedDict([('number', 'int')]))]),
+ OrderedDict([('union', 'MyUnion'), ('data', OrderedDict([('one', 'One'), ('ONE', 'Two')]))])]
+[{'enum_name': 'MyUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'One'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('type', 'Two'), ('data', OrderedDict([('number', 'int')]))])]
diff --git a/tests/qapi-schema/union-base-no-discriminator.err b/tests/qapi-schema/union-base-no-discriminator.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/union-base-no-discriminator.exit b/tests/qapi-schema/union-base-no-discriminator.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/union-base-no-discriminator.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/union-base-no-discriminator.json b/tests/qapi-schema/union-base-no-discriminator.json
new file mode 100644
index 0000000..b5da546
--- /dev/null
+++ b/tests/qapi-schema/union-base-no-discriminator.json
@@ -0,0 +1,14 @@
+# FIXME: either allow base in non-flat unions, or diagnose missing discriminator
+{ 'type': 'TestTypeA',
+ 'data': { 'string': 'str' } }
+
+{ 'type': 'TestTypeB',
+ 'data': { 'integer': 'int' } }
+
+{ 'type': 'Base',
+ 'data': { 'string': 'str' } }
+
+{ 'union': 'TestUnion',
+ 'base': 'Base',
+ 'data': { 'value1': 'TestTypeA',
+ 'value2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/union-base-no-discriminator.out b/tests/qapi-schema/union-base-no-discriminator.out
new file mode 100644
index 0000000..505fd57
--- /dev/null
+++ b/tests/qapi-schema/union-base-no-discriminator.out
@@ -0,0 +1,8 @@
+[OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
+ OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('union', 'TestUnion'), ('base', 'Base'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 'TestTypeB')]))])]
+[{'enum_name': 'TestUnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
+ OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
+ OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))])]
diff --git a/tests/qapi-schema/union-invalid-base.err b/tests/qapi-schema/union-invalid-base.err
index 938f969..3cc82c0 100644
--- a/tests/qapi-schema/union-invalid-base.err
+++ b/tests/qapi-schema/union-invalid-base.err
@@ -1 +1 @@
-tests/qapi-schema/union-invalid-base.json:7: Base 'TestBaseWrong' is not a valid type
+tests/qapi-schema/union-invalid-base.json:8: Base 'int' is not a valid type
diff --git a/tests/qapi-schema/union-invalid-base.json b/tests/qapi-schema/union-invalid-base.json
index 1fa4930..bc5dc8d 100644
--- a/tests/qapi-schema/union-invalid-base.json
+++ b/tests/qapi-schema/union-invalid-base.json
@@ -1,3 +1,4 @@
+# a union base type must be a struct
{ 'type': 'TestTypeA',
'data': { 'string': 'str' } }
@@ -5,6 +6,7 @@
'data': { 'integer': 'int' } }
{ 'union': 'TestUnion',
- 'base': 'TestBaseWrong',
+ 'base': 'int',
+ 'discriminator': 'int',
'data': { 'value1': 'TestTypeA',
'value2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/union-max.err b/tests/qapi-schema/union-max.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/union-max.exit b/tests/qapi-schema/union-max.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/union-max.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/union-max.json b/tests/qapi-schema/union-max.json
new file mode 100644
index 0000000..45648c4
--- /dev/null
+++ b/tests/qapi-schema/union-max.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject 'max' branch in a union, for collision with C enum
+{ 'union': 'Union',
+ 'data': { 'max': 'int' } }
diff --git a/tests/qapi-schema/union-max.out b/tests/qapi-schema/union-max.out
new file mode 100644
index 0000000..2757d36
--- /dev/null
+++ b/tests/qapi-schema/union-max.out
@@ -0,0 +1,3 @@
+[OrderedDict([('union', 'Union'), ('data', OrderedDict([('max', 'int')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/union-optional-branch.err b/tests/qapi-schema/union-optional-branch.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/union-optional-branch.exit b/tests/qapi-schema/union-optional-branch.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/union-optional-branch.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/union-optional-branch.json b/tests/qapi-schema/union-optional-branch.json
new file mode 100644
index 0000000..c513db7
--- /dev/null
+++ b/tests/qapi-schema/union-optional-branch.json
@@ -0,0 +1,2 @@
+# FIXME: union branches cannot be optional
+{ 'union': 'Union', 'data': { '*a': 'int', 'b': 'str' } }
diff --git a/tests/qapi-schema/union-optional-branch.out b/tests/qapi-schema/union-optional-branch.out
new file mode 100644
index 0000000..b03b5d2
--- /dev/null
+++ b/tests/qapi-schema/union-optional-branch.out
@@ -0,0 +1,3 @@
+[OrderedDict([('union', 'Union'), ('data', OrderedDict([('*a', 'int'), ('b', 'str')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/union-unknown.err b/tests/qapi-schema/union-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/union-unknown.exit b/tests/qapi-schema/union-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/union-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/union-unknown.json b/tests/qapi-schema/union-unknown.json
new file mode 100644
index 0000000..258f1d3
--- /dev/null
+++ b/tests/qapi-schema/union-unknown.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject a union with unknown type in branch
+{ 'union': 'Union',
+ 'data': { 'unknown': 'MissingType' } }
diff --git a/tests/qapi-schema/union-unknown.out b/tests/qapi-schema/union-unknown.out
new file mode 100644
index 0000000..8223dcf
--- /dev/null
+++ b/tests/qapi-schema/union-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('union', 'Union'), ('data', OrderedDict([('unknown', 'MissingType')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': None}]
+[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 07/28] qapi: Simplify tests of simple unions
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (5 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 13:41 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions Eric Blake
` (22 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
None of the existing QMP or QGA interfaces uses a union with a
base type but no discriminator; it is easier to avoid this in
the generator to save room for other future extensions more likely
to be useful (the previous commit added the test
union-base-no-discriminator to ensure that we eventually give an
error message). Meanwhile, the tests of UserDefNativeListUnion
serve to validate code generation of simple unions, except that it
did not have full coverage in the strict test.
Fix some indentation and long lines while at it.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-types.py | 7 ++-
scripts/qapi-visit.py | 13 +++---
tests/qapi-schema/qapi-schema-test.json | 4 --
tests/qapi-schema/qapi-schema-test.out | 2 -
tests/test-qmp-input-strict.c | 57 +++++++++++------------
tests/test-qmp-input-visitor.c | 80 +++++++++++++--------------------
tests/test-qmp-output-visitor.c | 74 +++++++++++-------------------
7 files changed, 95 insertions(+), 142 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index e400b03..f6fb930 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -242,10 +242,9 @@ struct %(name)s
''')
if base:
- base_fields = find_struct(base)['data']
- if discriminator:
- base_fields = base_fields.copy()
- del base_fields[discriminator]
+ assert discriminator
+ base_fields = find_struct(base)['data'].copy()
+ del base_fields[discriminator]
ret += generate_struct_fields(base_fields)
else:
assert not discriminator
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 4416677..3f82bd4 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -2,7 +2,7 @@
# QAPI visitor generator
#
# Copyright IBM, Corp. 2011
-# Copyright (C) 2014 Red Hat, Inc.
+# Copyright (C) 2014-2015 Red Hat, Inc.
#
# Authors:
# Anthony Liguori <aliguori@us.ibm.com>
@@ -310,16 +310,15 @@ def generate_visit_union(expr):
ret = ""
disc_type = enum_define['enum_name']
else:
- # There will always be a discriminator in the C switch code, by default it
- # is an enum type generated silently as "'%sKind' % (name)"
+ # There will always be a discriminator in the C switch code, by default
+ # it is an enum type generated silently as "'%sKind' % (name)"
ret = generate_visit_enum('%sKind' % name, members.keys())
disc_type = '%sKind' % (name)
if base:
- base_fields = find_struct(base)['data']
- if discriminator:
- base_fields = base_fields.copy()
- del base_fields[discriminator]
+ assert discriminator
+ base_fields = find_struct(base)['data'].copy()
+ del base_fields[discriminator]
ret += generate_visit_struct_fields(name, "", "", base_fields)
if discriminator:
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 84f0f07..b134f3f 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -36,10 +36,6 @@
{ 'type': 'UserDefC',
'data': { 'string1': 'str', 'string2': 'str' } }
-{ 'union': 'UserDefUnion',
- 'base': 'UserDefZero',
- 'data': { 'a' : 'UserDefA', 'b' : 'UserDefB' } }
-
{ 'type': 'UserDefUnionBase',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 915a61b..664ae7b 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -7,7 +7,6 @@
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
- OrderedDict([('union', 'UserDefUnion'), ('base', 'UserDefZero'), ('data', OrderedDict([('a', 'UserDefA'), ('b', 'UserDefB')]))]),
OrderedDict([('type', 'UserDefUnionBase'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
OrderedDict([('union', 'UserDefFlatUnion'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefA'), ('value2', 'UserDefB'), ('value3', 'UserDefB')]))]),
OrderedDict([('union', 'UserDefFlatUnion2'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefC'), ('value2', 'UserDefB'), ('value3', 'UserDefA')]))]),
@@ -24,7 +23,6 @@
OrderedDict([('event', 'EVENT_C'), ('data', OrderedDict([('*a', 'int'), ('*b', 'UserDefOne'), ('c', 'str')]))]),
OrderedDict([('event', 'EVENT_D'), ('data', OrderedDict([('a', 'EventStructOne'), ('b', 'str'), ('*c', 'str'), ('*enum3', 'EnumOne')]))])]
[{'enum_name': 'EnumOne', 'enum_values': ['value1', 'value2', 'value3']},
- {'enum_name': 'UserDefUnionKind', 'enum_values': None},
{'enum_name': 'UserDefAnonUnionKind', 'enum_values': None},
{'enum_name': 'UserDefNativeListUnionKind', 'enum_values': None}]
[OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index d5360c6..53134a1 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -1,7 +1,7 @@
/*
* QMP Input Visitor unit-tests (strict mode).
*
- * Copyright (C) 2011-2012 Red Hat Inc.
+ * Copyright (C) 2011-2012, 2015 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -141,18 +141,18 @@ static void test_validate_list(TestInputVisitorData *data,
qapi_free_UserDefOneList(head);
}
-static void test_validate_union(TestInputVisitorData *data,
- const void *unused)
+static void test_validate_union_native_list(TestInputVisitorData *data,
+ const void *unused)
{
- UserDefUnion *tmp = NULL;
+ UserDefNativeListUnion *tmp = NULL;
Visitor *v;
Error *err = NULL;
- v = validate_test_init(data, "{ 'type': 'b', 'integer': 41, 'data' : { 'integer': 42 } }");
+ v = validate_test_init(data, "{ 'type': 'integer', 'data' : [ 1, 2 ] }");
- visit_type_UserDefUnion(v, &tmp, NULL, &err);
+ visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
g_assert(!err);
- qapi_free_UserDefUnion(tmp);
+ qapi_free_UserDefNativeListUnion(tmp);
}
static void test_validate_union_flat(TestInputVisitorData *data,
@@ -232,18 +232,19 @@ static void test_validate_fail_list(TestInputVisitorData *data,
qapi_free_UserDefOneList(head);
}
-static void test_validate_fail_union(TestInputVisitorData *data,
- const void *unused)
+static void test_validate_fail_union_native_list(TestInputVisitorData *data,
+ const void *unused)
{
- UserDefUnion *tmp = NULL;
+ UserDefNativeListUnion *tmp = NULL;
Error *err = NULL;
Visitor *v;
- v = validate_test_init(data, "{ 'type': 'b', 'data' : { 'integer': 42 } }");
+ v = validate_test_init(data,
+ "{ 'type': 'integer', 'data' : [ \"string\" ] }");
- visit_type_UserDefUnion(v, &tmp, NULL, &err);
+ visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
g_assert(err);
- qapi_free_UserDefUnion(tmp);
+ qapi_free_UserDefNativeListUnion(tmp);
}
static void test_validate_fail_union_flat(TestInputVisitorData *data,
@@ -304,31 +305,31 @@ int main(int argc, char **argv)
g_test_init(&argc, &argv, NULL);
validate_test_add("/visitor/input-strict/pass/struct",
- &testdata, test_validate_struct);
+ &testdata, test_validate_struct);
validate_test_add("/visitor/input-strict/pass/struct-nested",
- &testdata, test_validate_struct_nested);
+ &testdata, test_validate_struct_nested);
validate_test_add("/visitor/input-strict/pass/list",
- &testdata, test_validate_list);
- validate_test_add("/visitor/input-strict/pass/union",
- &testdata, test_validate_union);
+ &testdata, test_validate_list);
validate_test_add("/visitor/input-strict/pass/union-flat",
- &testdata, test_validate_union_flat);
+ &testdata, test_validate_union_flat);
validate_test_add("/visitor/input-strict/pass/union-anon",
- &testdata, test_validate_union_anon);
+ &testdata, test_validate_union_anon);
+ validate_test_add("/visitor/input-strict/pass/union-native-list",
+ &testdata, test_validate_union_native_list);
validate_test_add("/visitor/input-strict/fail/struct",
- &testdata, test_validate_fail_struct);
+ &testdata, test_validate_fail_struct);
validate_test_add("/visitor/input-strict/fail/struct-nested",
- &testdata, test_validate_fail_struct_nested);
+ &testdata, test_validate_fail_struct_nested);
validate_test_add("/visitor/input-strict/fail/list",
- &testdata, test_validate_fail_list);
- validate_test_add("/visitor/input-strict/fail/union",
- &testdata, test_validate_fail_union);
+ &testdata, test_validate_fail_list);
validate_test_add("/visitor/input-strict/fail/union-flat",
- &testdata, test_validate_fail_union_flat);
+ &testdata, test_validate_fail_union_flat);
validate_test_add("/visitor/input-strict/fail/union-flat-no-discriminator",
- &testdata, test_validate_fail_union_flat_no_discrim);
+ &testdata, test_validate_fail_union_flat_no_discrim);
validate_test_add("/visitor/input-strict/fail/union-anon",
- &testdata, test_validate_fail_union_anon);
+ &testdata, test_validate_fail_union_anon);
+ validate_test_add("/visitor/input-strict/fail/union-native-list",
+ &testdata, test_validate_fail_union_native_list);
g_test_run();
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 1c8e872..cc33f64 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -1,7 +1,7 @@
/*
* QMP Input Visitor unit-tests.
*
- * Copyright (C) 2011 Red Hat Inc.
+ * Copyright (C) 2011, 2015 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -293,23 +293,6 @@ static void test_visitor_in_list(TestInputVisitorData *data,
qapi_free_UserDefOneList(head);
}
-static void test_visitor_in_union(TestInputVisitorData *data,
- const void *unused)
-{
- Visitor *v;
- Error *err = NULL;
- UserDefUnion *tmp;
-
- v = visitor_input_test_init(data, "{ 'type': 'b', 'integer': 41, 'data' : { 'integer': 42 } }");
-
- visit_type_UserDefUnion(v, &tmp, NULL, &err);
- g_assert(err == NULL);
- g_assert_cmpint(tmp->kind, ==, USER_DEF_UNION_KIND_B);
- g_assert_cmpint(tmp->integer, ==, 41);
- g_assert_cmpint(tmp->b->integer, ==, 42);
- qapi_free_UserDefUnion(tmp);
-}
-
static void test_visitor_in_union_flat(TestInputVisitorData *data,
const void *unused)
{
@@ -670,55 +653,56 @@ int main(int argc, char **argv)
input_visitor_test_add("/visitor/input/number",
&in_visitor_data, test_visitor_in_number);
input_visitor_test_add("/visitor/input/string",
- &in_visitor_data, test_visitor_in_string);
+ &in_visitor_data, test_visitor_in_string);
input_visitor_test_add("/visitor/input/enum",
- &in_visitor_data, test_visitor_in_enum);
+ &in_visitor_data, test_visitor_in_enum);
input_visitor_test_add("/visitor/input/struct",
- &in_visitor_data, test_visitor_in_struct);
+ &in_visitor_data, test_visitor_in_struct);
input_visitor_test_add("/visitor/input/struct-nested",
- &in_visitor_data, test_visitor_in_struct_nested);
+ &in_visitor_data, test_visitor_in_struct_nested);
input_visitor_test_add("/visitor/input/list",
- &in_visitor_data, test_visitor_in_list);
- input_visitor_test_add("/visitor/input/union",
- &in_visitor_data, test_visitor_in_union);
+ &in_visitor_data, test_visitor_in_list);
input_visitor_test_add("/visitor/input/union-flat",
- &in_visitor_data, test_visitor_in_union_flat);
+ &in_visitor_data, test_visitor_in_union_flat);
input_visitor_test_add("/visitor/input/union-anon",
- &in_visitor_data, test_visitor_in_union_anon);
+ &in_visitor_data, test_visitor_in_union_anon);
input_visitor_test_add("/visitor/input/errors",
- &in_visitor_data, test_visitor_in_errors);
+ &in_visitor_data, test_visitor_in_errors);
input_visitor_test_add("/visitor/input/native_list/int",
- &in_visitor_data,
- test_visitor_in_native_list_int);
+ &in_visitor_data,
+ test_visitor_in_native_list_int);
input_visitor_test_add("/visitor/input/native_list/int8",
- &in_visitor_data,
- test_visitor_in_native_list_int8);
+ &in_visitor_data,
+ test_visitor_in_native_list_int8);
input_visitor_test_add("/visitor/input/native_list/int16",
- &in_visitor_data,
- test_visitor_in_native_list_int16);
+ &in_visitor_data,
+ test_visitor_in_native_list_int16);
input_visitor_test_add("/visitor/input/native_list/int32",
- &in_visitor_data,
- test_visitor_in_native_list_int32);
+ &in_visitor_data,
+ test_visitor_in_native_list_int32);
input_visitor_test_add("/visitor/input/native_list/int64",
- &in_visitor_data,
- test_visitor_in_native_list_int64);
+ &in_visitor_data,
+ test_visitor_in_native_list_int64);
input_visitor_test_add("/visitor/input/native_list/uint8",
- &in_visitor_data,
- test_visitor_in_native_list_uint8);
+ &in_visitor_data,
+ test_visitor_in_native_list_uint8);
input_visitor_test_add("/visitor/input/native_list/uint16",
- &in_visitor_data,
- test_visitor_in_native_list_uint16);
+ &in_visitor_data,
+ test_visitor_in_native_list_uint16);
input_visitor_test_add("/visitor/input/native_list/uint32",
- &in_visitor_data,
- test_visitor_in_native_list_uint32);
+ &in_visitor_data,
+ test_visitor_in_native_list_uint32);
input_visitor_test_add("/visitor/input/native_list/uint64",
- &in_visitor_data, test_visitor_in_native_list_uint64);
+ &in_visitor_data,
+ test_visitor_in_native_list_uint64);
input_visitor_test_add("/visitor/input/native_list/bool",
- &in_visitor_data, test_visitor_in_native_list_bool);
+ &in_visitor_data, test_visitor_in_native_list_bool);
input_visitor_test_add("/visitor/input/native_list/str",
- &in_visitor_data, test_visitor_in_native_list_string);
+ &in_visitor_data,
+ test_visitor_in_native_list_string);
input_visitor_test_add("/visitor/input/native_list/number",
- &in_visitor_data, test_visitor_in_native_list_number);
+ &in_visitor_data,
+ test_visitor_in_native_list_number);
g_test_run();
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 74020de..ebe6ea3 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -1,7 +1,7 @@
/*
* QMP Output Visitor unit-tests.
*
- * Copyright (C) 2011 Red Hat Inc.
+ * Copyright (C) 2011, 2015 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -422,40 +422,6 @@ static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
qapi_free_UserDefNestedList(head);
}
-static void test_visitor_out_union(TestOutputVisitorData *data,
- const void *unused)
-{
- QObject *arg, *qvalue;
- QDict *qdict, *value;
-
- Error *err = NULL;
-
- UserDefUnion *tmp = g_malloc0(sizeof(UserDefUnion));
- tmp->kind = USER_DEF_UNION_KIND_A;
- tmp->integer = 41;
- tmp->a = g_malloc0(sizeof(UserDefA));
- tmp->a->boolean = true;
-
- visit_type_UserDefUnion(data->ov, &tmp, NULL, &err);
- g_assert(err == NULL);
- arg = qmp_output_get_qobject(data->qov);
-
- g_assert(qobject_type(arg) == QTYPE_QDICT);
- qdict = qobject_to_qdict(arg);
-
- g_assert_cmpstr(qdict_get_str(qdict, "type"), ==, "a");
- g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, 41);
-
- qvalue = qdict_get(qdict, "data");
- g_assert(data != NULL);
- g_assert(qobject_type(qvalue) == QTYPE_QDICT);
- value = qobject_to_qdict(qvalue);
- g_assert_cmpint(qdict_get_bool(value, "boolean"), ==, true);
-
- qapi_free_UserDefUnion(tmp);
- QDECREF(qdict);
-}
-
static void test_visitor_out_union_flat(TestOutputVisitorData *data,
const void *unused)
{
@@ -862,8 +828,6 @@ int main(int argc, char **argv)
&out_visitor_data, test_visitor_out_list);
output_visitor_test_add("/visitor/output/list-qapi-free",
&out_visitor_data, test_visitor_out_list_qapi_free);
- output_visitor_test_add("/visitor/output/union",
- &out_visitor_data, test_visitor_out_union);
output_visitor_test_add("/visitor/output/union-flat",
&out_visitor_data, test_visitor_out_union_flat);
output_visitor_test_add("/visitor/output/union-anon",
@@ -871,29 +835,41 @@ int main(int argc, char **argv)
output_visitor_test_add("/visitor/output/empty",
&out_visitor_data, test_visitor_out_empty);
output_visitor_test_add("/visitor/output/native_list/int",
- &out_visitor_data, test_visitor_out_native_list_int);
+ &out_visitor_data,
+ test_visitor_out_native_list_int);
output_visitor_test_add("/visitor/output/native_list/int8",
- &out_visitor_data, test_visitor_out_native_list_int8);
+ &out_visitor_data,
+ test_visitor_out_native_list_int8);
output_visitor_test_add("/visitor/output/native_list/int16",
- &out_visitor_data, test_visitor_out_native_list_int16);
+ &out_visitor_data,
+ test_visitor_out_native_list_int16);
output_visitor_test_add("/visitor/output/native_list/int32",
- &out_visitor_data, test_visitor_out_native_list_int32);
+ &out_visitor_data,
+ test_visitor_out_native_list_int32);
output_visitor_test_add("/visitor/output/native_list/int64",
- &out_visitor_data, test_visitor_out_native_list_int64);
+ &out_visitor_data,
+ test_visitor_out_native_list_int64);
output_visitor_test_add("/visitor/output/native_list/uint8",
- &out_visitor_data, test_visitor_out_native_list_uint8);
+ &out_visitor_data,
+ test_visitor_out_native_list_uint8);
output_visitor_test_add("/visitor/output/native_list/uint16",
- &out_visitor_data, test_visitor_out_native_list_uint16);
+ &out_visitor_data,
+ test_visitor_out_native_list_uint16);
output_visitor_test_add("/visitor/output/native_list/uint32",
- &out_visitor_data, test_visitor_out_native_list_uint32);
+ &out_visitor_data,
+ test_visitor_out_native_list_uint32);
output_visitor_test_add("/visitor/output/native_list/uint64",
- &out_visitor_data, test_visitor_out_native_list_uint64);
+ &out_visitor_data,
+ test_visitor_out_native_list_uint64);
output_visitor_test_add("/visitor/output/native_list/bool",
- &out_visitor_data, test_visitor_out_native_list_bool);
+ &out_visitor_data,
+ test_visitor_out_native_list_bool);
output_visitor_test_add("/visitor/output/native_list/string",
- &out_visitor_data, test_visitor_out_native_list_str);
+ &out_visitor_data,
+ test_visitor_out_native_list_str);
output_visitor_test_add("/visitor/output/native_list/number",
- &out_visitor_data, test_visitor_out_native_list_number);
+ &out_visitor_data,
+ test_visitor_out_native_list_number);
g_test_run();
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (6 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 07/28] qapi: Simplify tests of simple unions Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-24 20:38 ` Eric Blake
2015-03-26 14:20 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 09/28] qapi: Prepare for catching more semantic parse errors Eric Blake
` (21 subsequent siblings)
29 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Previous commits demonstrated that the generator had several
flaws with less-than-perfect unions:
- make the use of a base without discriminator a hard error,
since the previous patch removed all remaining uses of it
- a simple union that listed the same branch twice (or two variant
names that map to the same C enumerator, including the implicit
MAX sentinel) ended up generating invalid C code
- checking 'if discriminator' prior to 'if discriminator == {}'
leads to dead code in python, and ended up processing anonymous
unions as if they were simple unions
- an anonymous union that listed two branches with the same qtype
ended up generating invalid C code
- the generator crashed on anonymous union attempts to use an
array type
- the generator was silently ignoring a base type for anonymous
unions
- the generator allowed unknown types or nested anonymous unions
as a branch in an anonymous union
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-types.py | 13 +--
scripts/qapi.py | 96 +++++++++++++++++-----
tests/qapi-schema/alternate-array.err | 1 +
tests/qapi-schema/alternate-array.exit | 2 +-
tests/qapi-schema/alternate-array.json | 1 -
tests/qapi-schema/alternate-array.out | 4 -
tests/qapi-schema/alternate-base.err | 1 +
tests/qapi-schema/alternate-base.exit | 2 +-
tests/qapi-schema/alternate-base.json | 2 +-
tests/qapi-schema/alternate-base.out | 4 -
tests/qapi-schema/alternate-clash.err | 1 +
tests/qapi-schema/alternate-clash.exit | 2 +-
tests/qapi-schema/alternate-clash.json | 2 +-
tests/qapi-schema/alternate-clash.out | 3 -
tests/qapi-schema/alternate-conflict-dict.err | 1 +
tests/qapi-schema/alternate-conflict-dict.exit | 2 +-
tests/qapi-schema/alternate-conflict-dict.json | 2 +-
tests/qapi-schema/alternate-conflict-dict.out | 6 --
tests/qapi-schema/alternate-conflict-string.err | 1 +
tests/qapi-schema/alternate-conflict-string.exit | 2 +-
tests/qapi-schema/alternate-conflict-string.json | 2 +-
tests/qapi-schema/alternate-conflict-string.out | 5 --
tests/qapi-schema/alternate-nested.err | 1 +
tests/qapi-schema/alternate-nested.exit | 2 +-
tests/qapi-schema/alternate-nested.json | 2 +-
tests/qapi-schema/alternate-nested.out | 5 --
tests/qapi-schema/alternate-unknown.err | 1 +
tests/qapi-schema/alternate-unknown.exit | 2 +-
tests/qapi-schema/alternate-unknown.json | 2 +-
tests/qapi-schema/alternate-unknown.out | 3 -
tests/qapi-schema/flat-union-bad-discriminator.err | 1 +
.../qapi-schema/flat-union-bad-discriminator.exit | 2 +-
tests/qapi-schema/flat-union-bad-discriminator.out | 10 ---
tests/qapi-schema/union-bad-branch.err | 1 +
tests/qapi-schema/union-bad-branch.exit | 2 +-
tests/qapi-schema/union-bad-branch.json | 2 +-
tests/qapi-schema/union-bad-branch.out | 6 --
tests/qapi-schema/union-base-no-discriminator.err | 1 +
tests/qapi-schema/union-base-no-discriminator.exit | 2 +-
tests/qapi-schema/union-base-no-discriminator.json | 3 +-
tests/qapi-schema/union-base-no-discriminator.out | 8 --
tests/qapi-schema/union-max.err | 1 +
tests/qapi-schema/union-max.exit | 2 +-
tests/qapi-schema/union-max.json | 2 +-
tests/qapi-schema/union-max.out | 3 -
45 files changed, 109 insertions(+), 110 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index f6fb930..2390887 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -181,17 +181,8 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
name=name)
for key in members:
- qapi_type = members[key]
- if builtin_types.has_key(qapi_type):
- qtype = builtin_types[qapi_type]
- elif find_struct(qapi_type):
- qtype = "QTYPE_QDICT"
- elif find_union(qapi_type):
- qtype = "QTYPE_QDICT"
- elif find_enum(qapi_type):
- qtype = "QTYPE_QSTRING"
- else:
- assert False, "Invalid anonymous union member"
+ qtype = find_anonymous_member_qtype(members[key])
+ assert qtype, "Invalid anonymous union member"
ret += mcgen('''
[ %(qtype)s ] = %(abbrev)s_KIND_%(enum)s,
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 3ce8c33..fc7b7f1 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -224,6 +224,23 @@ def find_base_fields(base):
return None
return base_struct_define['data']
+# Return the qtype of an anonymous union branch, or None on error.
+def find_anonymous_member_qtype(qapi_type):
+ if builtin_types.has_key(qapi_type):
+ return builtin_types[qapi_type]
+ elif find_struct(qapi_type):
+ return "QTYPE_QDICT"
+ elif find_enum(qapi_type):
+ return "QTYPE_QSTRING"
+ else:
+ union = find_union(qapi_type)
+ if union:
+ discriminator = union.get('discriminator')
+ if discriminator == {}:
+ return None
+ return "QTYPE_QDICT"
+ return None
+
# Return the discriminator enum define if discriminator is specified as an
# enum type, otherwise return None.
def discriminator_find_enum_define(expr):
@@ -258,24 +275,28 @@ def check_union(expr, expr_info):
base = expr.get('base')
discriminator = expr.get('discriminator')
members = expr['data']
+ values = { 'MAX': '(automatic)' }
+ types_seen = {}
- # If the object has a member 'base', its value must name a complex type.
- if base:
- base_fields = find_base_fields(base)
- if not base_fields:
- raise QAPIExprError(expr_info,
- "Base '%s' is not a valid type"
- % base)
+ # Three types of unions, determined by discriminator.
- # If the union object has no member 'discriminator', it's an
- # ordinary union.
- if not discriminator:
+ # If the value of member 'discriminator' is {}, it's an
+ # anonymous union, and must not have a base.
+ if discriminator == {}:
enum_define = None
+ if base:
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' must not have a base"
+ % name)
- # Else if the value of member 'discriminator' is {}, it's an
- # anonymous union.
- elif discriminator == {}:
+ # Else if the union object has no member 'discriminator', it's an
+ # ordinary union. For now, it must not have a base.
+ elif not discriminator:
enum_define = None
+ if base:
+ raise QAPIExprError(expr_info,
+ "Simple union '%s' must not have a base"
+ % name)
# Else, it's a flat union.
else:
@@ -284,6 +305,12 @@ def check_union(expr, expr_info):
raise QAPIExprError(expr_info,
"Flat union '%s' must have a base field"
% name)
+ base_fields = find_base_fields(base)
+ if not base_fields:
+ raise QAPIExprError(expr_info,
+ "Base '%s' is not a valid type"
+ % base)
+
# The value of member 'discriminator' must name a member of the
# base type.
discriminator_type = base_fields.get(discriminator)
@@ -301,15 +328,42 @@ def check_union(expr, expr_info):
# Check every branch
for (key, value) in members.items():
- # If this named member's value names an enum type, then all members
+ # If the discriminator names an enum type, then all members
# of 'data' must also be members of the enum type.
- if enum_define and not key in enum_define['enum_values']:
- raise QAPIExprError(expr_info,
- "Discriminator value '%s' is not found in "
- "enum '%s'" %
- (key, enum_define["enum_name"]))
- # Todo: add checking for values. Key is checked as above, value can be
- # also checked here, but we need more functions to handle array case.
+ if enum_define:
+ if not key in enum_define['enum_values']:
+ raise QAPIExprError(expr_info,
+ "Discriminator value '%s' is not found in "
+ "enum '%s'" %
+ (key, enum_define["enum_name"]))
+
+ # Otherwise, check for conflicts in the generated enum
+ else:
+ c_key = _generate_enum_string(key)
+ if c_key in values:
+ raise QAPIExprError(expr_info,
+ "Union '%s' member '%s' clashes with '%s'"
+ % (name, key, values[c_key]))
+ values[c_key] = key
+
+ # Ensure anonymous unions have no type conflicts.
+ if discriminator == {}:
+ if isinstance(value, list):
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' member '%s' must "
+ "not be array type" % (name, key))
+ qtype = find_anonymous_member_qtype(value)
+ if not qtype:
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' member '%s' has "
+ "invalid type '%s'" % (name, key, value))
+ if qtype in types_seen:
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' member '%s' has "
+ "same QObject type as member '%s'"
+ % (name, key, types_seen[qtype]))
+ types_seen[qtype] = key
+
def check_enum(expr, expr_info):
name = expr['enum']
diff --git a/tests/qapi-schema/alternate-array.err b/tests/qapi-schema/alternate-array.err
index e69de29..2638b85 100644
--- a/tests/qapi-schema/alternate-array.err
+++ b/tests/qapi-schema/alternate-array.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-array.json:4: Anonymous union 'MyUnion' member 'two' must not be array type
diff --git a/tests/qapi-schema/alternate-array.exit b/tests/qapi-schema/alternate-array.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-array.exit
+++ b/tests/qapi-schema/alternate-array.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-array.json b/tests/qapi-schema/alternate-array.json
index b399500..e330d57 100644
--- a/tests/qapi-schema/alternate-array.json
+++ b/tests/qapi-schema/alternate-array.json
@@ -1,5 +1,4 @@
# FIXME: we do not support array branches of anonymous unions yet
-# FIXME: this should fail as long as we lack support
{ 'type': 'One',
'data': { 'name': 'str' } }
{ 'union': 'MyUnion',
diff --git a/tests/qapi-schema/alternate-array.out b/tests/qapi-schema/alternate-array.out
index 90dc22c..e69de29 100644
--- a/tests/qapi-schema/alternate-array.out
+++ b/tests/qapi-schema/alternate-array.out
@@ -1,4 +0,0 @@
-[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))]),
- OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'One'), ('two', ['int'])]))])]
-[{'enum_name': 'MyUnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))])]
diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
index e69de29..a2486b8 100644
--- a/tests/qapi-schema/alternate-base.err
+++ b/tests/qapi-schema/alternate-base.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-base.json:4: Anonymous union 'MyUnion' must not have a base
diff --git a/tests/qapi-schema/alternate-base.exit b/tests/qapi-schema/alternate-base.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-base.exit
+++ b/tests/qapi-schema/alternate-base.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-base.json b/tests/qapi-schema/alternate-base.json
index 2d36db1..dad7f02 100644
--- a/tests/qapi-schema/alternate-base.json
+++ b/tests/qapi-schema/alternate-base.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject anonymous union with base type
+# we reject anonymous union with base type
{ 'type': 'Base',
'data': { 'string': 'str' } }
{ 'union': 'MyUnion',
diff --git a/tests/qapi-schema/alternate-base.out b/tests/qapi-schema/alternate-base.out
index 7fb31f5..e69de29 100644
--- a/tests/qapi-schema/alternate-base.out
+++ b/tests/qapi-schema/alternate-base.out
@@ -1,4 +0,0 @@
-[OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('union', 'MyUnion'), ('base', 'Base'), ('discriminator', OrderedDict()), ('data', OrderedDict([('number', 'int')]))])]
-[{'enum_name': 'MyUnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))])]
diff --git a/tests/qapi-schema/alternate-clash.err b/tests/qapi-schema/alternate-clash.err
index e69de29..1130c12 100644
--- a/tests/qapi-schema/alternate-clash.err
+++ b/tests/qapi-schema/alternate-clash.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-clash.json:2: Union 'Union1' member 'ONE' clashes with 'one'
diff --git a/tests/qapi-schema/alternate-clash.exit b/tests/qapi-schema/alternate-clash.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-clash.exit
+++ b/tests/qapi-schema/alternate-clash.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-clash.json b/tests/qapi-schema/alternate-clash.json
index 7e2ef23..fa2d27e 100644
--- a/tests/qapi-schema/alternate-clash.json
+++ b/tests/qapi-schema/alternate-clash.json
@@ -1,4 +1,4 @@
-# FIXME: we should detect C enum collisions in an anonymous union
+# we detect C enum collisions in an anonymous union
{ 'union': 'Union1',
'discriminator': {},
'data': { 'one': 'str', 'ONE': 'int' } }
diff --git a/tests/qapi-schema/alternate-clash.out b/tests/qapi-schema/alternate-clash.out
index c6687fa..e69de29 100644
--- a/tests/qapi-schema/alternate-clash.out
+++ b/tests/qapi-schema/alternate-clash.out
@@ -1,3 +0,0 @@
-[OrderedDict([('union', 'Union1'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'str'), ('ONE', 'int')]))])]
-[{'enum_name': 'Union1Kind', 'enum_values': None}]
-[]
diff --git a/tests/qapi-schema/alternate-conflict-dict.err b/tests/qapi-schema/alternate-conflict-dict.err
index e69de29..f256d51 100644
--- a/tests/qapi-schema/alternate-conflict-dict.err
+++ b/tests/qapi-schema/alternate-conflict-dict.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-conflict-dict.json:6: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
diff --git a/tests/qapi-schema/alternate-conflict-dict.exit b/tests/qapi-schema/alternate-conflict-dict.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-conflict-dict.exit
+++ b/tests/qapi-schema/alternate-conflict-dict.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-conflict-dict.json b/tests/qapi-schema/alternate-conflict-dict.json
index 70fe089..177c163 100644
--- a/tests/qapi-schema/alternate-conflict-dict.json
+++ b/tests/qapi-schema/alternate-conflict-dict.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject anonymous unions with multiple object branches
+# we reject anonymous unions with multiple object branches
{ 'type': 'One',
'data': { 'name': 'str' } }
{ 'type': 'Two',
diff --git a/tests/qapi-schema/alternate-conflict-dict.out b/tests/qapi-schema/alternate-conflict-dict.out
index b9ac945..e69de29 100644
--- a/tests/qapi-schema/alternate-conflict-dict.out
+++ b/tests/qapi-schema/alternate-conflict-dict.out
@@ -1,6 +0,0 @@
-[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))]),
- OrderedDict([('type', 'Two'), ('data', OrderedDict([('value', 'int')]))]),
- OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'One'), ('two', 'Two')]))])]
-[{'enum_name': 'MyUnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'One'), ('data', OrderedDict([('name', 'str')]))]),
- OrderedDict([('type', 'Two'), ('data', OrderedDict([('value', 'int')]))])]
diff --git a/tests/qapi-schema/alternate-conflict-string.err b/tests/qapi-schema/alternate-conflict-string.err
index e69de29..a3b7b6d 100644
--- a/tests/qapi-schema/alternate-conflict-string.err
+++ b/tests/qapi-schema/alternate-conflict-string.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-conflict-string.json:4: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
diff --git a/tests/qapi-schema/alternate-conflict-string.exit b/tests/qapi-schema/alternate-conflict-string.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-conflict-string.exit
+++ b/tests/qapi-schema/alternate-conflict-string.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
index 5fd1a47..a1b0bea 100644
--- a/tests/qapi-schema/alternate-conflict-string.json
+++ b/tests/qapi-schema/alternate-conflict-string.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject anonymous unions with multiple string-like branches
+# we reject anonymous unions with multiple string-like branches
{ 'enum': 'Enum',
'data': [ 'hello', 'world' ] }
{ 'union': 'MyUnion',
diff --git a/tests/qapi-schema/alternate-conflict-string.out b/tests/qapi-schema/alternate-conflict-string.out
index e7b39a2..e69de29 100644
--- a/tests/qapi-schema/alternate-conflict-string.out
+++ b/tests/qapi-schema/alternate-conflict-string.out
@@ -1,5 +0,0 @@
-[OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
- OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'str'), ('two', 'Enum')]))])]
-[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
- {'enum_name': 'MyUnionKind', 'enum_values': None}]
-[]
diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
index e69de29..59df96e 100644
--- a/tests/qapi-schema/alternate-nested.err
+++ b/tests/qapi-schema/alternate-nested.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-nested.json:5: Anonymous union 'Union2' member 'nested' has invalid type 'Union1'
diff --git a/tests/qapi-schema/alternate-nested.exit b/tests/qapi-schema/alternate-nested.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-nested.exit
+++ b/tests/qapi-schema/alternate-nested.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-nested.json b/tests/qapi-schema/alternate-nested.json
index d5812bf..ed2b6b7 100644
--- a/tests/qapi-schema/alternate-nested.json
+++ b/tests/qapi-schema/alternate-nested.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject a nested anonymous union branch
+# we reject a nested anonymous union branch
{ 'union': 'Union1',
'discriminator': {},
'data': { 'name': 'str', 'value': 'int' } }
diff --git a/tests/qapi-schema/alternate-nested.out b/tests/qapi-schema/alternate-nested.out
index 0137c1f..e69de29 100644
--- a/tests/qapi-schema/alternate-nested.out
+++ b/tests/qapi-schema/alternate-nested.out
@@ -1,5 +0,0 @@
-[OrderedDict([('union', 'Union1'), ('discriminator', OrderedDict()), ('data', OrderedDict([('name', 'str'), ('value', 'int')]))]),
- OrderedDict([('union', 'Union2'), ('discriminator', OrderedDict()), ('data', OrderedDict([('nested', 'Union1')]))])]
-[{'enum_name': 'Union1Kind', 'enum_values': None},
- {'enum_name': 'Union2Kind', 'enum_values': None}]
-[]
diff --git a/tests/qapi-schema/alternate-unknown.err b/tests/qapi-schema/alternate-unknown.err
index e69de29..bf8e9ae 100644
--- a/tests/qapi-schema/alternate-unknown.err
+++ b/tests/qapi-schema/alternate-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-unknown.json:2: Anonymous union 'Union' member 'unknown' has invalid type 'MissingType'
diff --git a/tests/qapi-schema/alternate-unknown.exit b/tests/qapi-schema/alternate-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/alternate-unknown.exit
+++ b/tests/qapi-schema/alternate-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/alternate-unknown.json b/tests/qapi-schema/alternate-unknown.json
index 0bab9c2..0c305c2 100644
--- a/tests/qapi-schema/alternate-unknown.json
+++ b/tests/qapi-schema/alternate-unknown.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject an anonymous union with unknown type in branch
+# we reject an anonymous union with unknown type in branch
{ 'union': 'Union',
'discriminator': {},
'data': { 'unknown': 'MissingType' } }
diff --git a/tests/qapi-schema/alternate-unknown.out b/tests/qapi-schema/alternate-unknown.out
index 0911cdc..e69de29 100644
--- a/tests/qapi-schema/alternate-unknown.out
+++ b/tests/qapi-schema/alternate-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('union', 'Union'), ('discriminator', OrderedDict()), ('data', OrderedDict([('unknown', 'MissingType')]))])]
-[{'enum_name': 'UnionKind', 'enum_values': None}]
-[]
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.err b/tests/qapi-schema/flat-union-bad-discriminator.err
index e69de29..628432f 100644
--- a/tests/qapi-schema/flat-union-bad-discriminator.err
+++ b/tests/qapi-schema/flat-union-bad-discriminator.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-bad-discriminator.json:10: Simple union 'TestUnion' must not have a base
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.exit b/tests/qapi-schema/flat-union-bad-discriminator.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/flat-union-bad-discriminator.exit
+++ b/tests/qapi-schema/flat-union-bad-discriminator.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.out b/tests/qapi-schema/flat-union-bad-discriminator.out
index b6ce217..e69de29 100644
--- a/tests/qapi-schema/flat-union-bad-discriminator.out
+++ b/tests/qapi-schema/flat-union-bad-discriminator.out
@@ -1,10 +0,0 @@
-[OrderedDict([('enum', 'TestEnum'), ('data', ['value1', 'value2'])]),
- OrderedDict([('type', 'TestBase'), ('data', OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')]))]),
- OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
- OrderedDict([('union', 'TestUnion'), ('base', 'TestBase'), ('discriminator', []), ('data', OrderedDict([('kind1', 'TestTypeA'), ('kind2', 'TestTypeB')]))])]
-[{'enum_name': 'TestEnum', 'enum_values': ['value1', 'value2']},
- {'enum_name': 'TestUnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'TestBase'), ('data', OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')]))]),
- OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))])]
diff --git a/tests/qapi-schema/union-bad-branch.err b/tests/qapi-schema/union-bad-branch.err
index e69de29..8822735 100644
--- a/tests/qapi-schema/union-bad-branch.err
+++ b/tests/qapi-schema/union-bad-branch.err
@@ -0,0 +1 @@
+tests/qapi-schema/union-bad-branch.json:6: Union 'MyUnion' member 'ONE' clashes with 'one'
diff --git a/tests/qapi-schema/union-bad-branch.exit b/tests/qapi-schema/union-bad-branch.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/union-bad-branch.exit
+++ b/tests/qapi-schema/union-bad-branch.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/union-bad-branch.json b/tests/qapi-schema/union-bad-branch.json
index f7aeae8..5b577b8 100644
--- a/tests/qapi-schema/union-bad-branch.json
+++ b/tests/qapi-schema/union-bad-branch.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject normal unions where branches would collide in C
+# we reject normal unions where branches would collide in C
{ 'type': 'One',
'data': { 'string': 'str' } }
{ 'type': 'Two',
diff --git a/tests/qapi-schema/union-bad-branch.out b/tests/qapi-schema/union-bad-branch.out
index 6baf01b..e69de29 100644
--- a/tests/qapi-schema/union-bad-branch.out
+++ b/tests/qapi-schema/union-bad-branch.out
@@ -1,6 +0,0 @@
-[OrderedDict([('type', 'One'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('type', 'Two'), ('data', OrderedDict([('number', 'int')]))]),
- OrderedDict([('union', 'MyUnion'), ('data', OrderedDict([('one', 'One'), ('ONE', 'Two')]))])]
-[{'enum_name': 'MyUnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'One'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('type', 'Two'), ('data', OrderedDict([('number', 'int')]))])]
diff --git a/tests/qapi-schema/union-base-no-discriminator.err b/tests/qapi-schema/union-base-no-discriminator.err
index e69de29..f33392d 100644
--- a/tests/qapi-schema/union-base-no-discriminator.err
+++ b/tests/qapi-schema/union-base-no-discriminator.err
@@ -0,0 +1 @@
+tests/qapi-schema/union-base-no-discriminator.json:12: Simple union 'TestUnion' must not have a base
diff --git a/tests/qapi-schema/union-base-no-discriminator.exit b/tests/qapi-schema/union-base-no-discriminator.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/union-base-no-discriminator.exit
+++ b/tests/qapi-schema/union-base-no-discriminator.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/union-base-no-discriminator.json b/tests/qapi-schema/union-base-no-discriminator.json
index b5da546..548a633 100644
--- a/tests/qapi-schema/union-base-no-discriminator.json
+++ b/tests/qapi-schema/union-base-no-discriminator.json
@@ -1,4 +1,5 @@
-# FIXME: either allow base in non-flat unions, or diagnose missing discriminator
+# for now, we reject a base for non-flat unions
+# FIXME: might be a useful extension to allow later
{ 'type': 'TestTypeA',
'data': { 'string': 'str' } }
diff --git a/tests/qapi-schema/union-base-no-discriminator.out b/tests/qapi-schema/union-base-no-discriminator.out
index 505fd57..e69de29 100644
--- a/tests/qapi-schema/union-base-no-discriminator.out
+++ b/tests/qapi-schema/union-base-no-discriminator.out
@@ -1,8 +0,0 @@
-[OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
- OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('union', 'TestUnion'), ('base', 'Base'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 'TestTypeB')]))])]
-[{'enum_name': 'TestUnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
- OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
- OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))])]
diff --git a/tests/qapi-schema/union-max.err b/tests/qapi-schema/union-max.err
index e69de29..55ce439 100644
--- a/tests/qapi-schema/union-max.err
+++ b/tests/qapi-schema/union-max.err
@@ -0,0 +1 @@
+tests/qapi-schema/union-max.json:2: Union 'Union' member 'max' clashes with '(automatic)'
diff --git a/tests/qapi-schema/union-max.exit b/tests/qapi-schema/union-max.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/union-max.exit
+++ b/tests/qapi-schema/union-max.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/union-max.json b/tests/qapi-schema/union-max.json
index 45648c4..d6ad986 100644
--- a/tests/qapi-schema/union-max.json
+++ b/tests/qapi-schema/union-max.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject 'max' branch in a union, for collision with C enum
+# we reject 'max' branch in a union, for collision with C enum
{ 'union': 'Union',
'data': { 'max': 'int' } }
diff --git a/tests/qapi-schema/union-max.out b/tests/qapi-schema/union-max.out
index 2757d36..e69de29 100644
--- a/tests/qapi-schema/union-max.out
+++ b/tests/qapi-schema/union-max.out
@@ -1,3 +0,0 @@
-[OrderedDict([('union', 'Union'), ('data', OrderedDict([('max', 'int')]))])]
-[{'enum_name': 'UnionKind', 'enum_values': None}]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 09/28] qapi: Prepare for catching more semantic parse errors
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (7 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 14:22 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator Eric Blake
` (20 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
This patch widens the scope of a try block (with the attending
reindentation required by Python) in preparation for a future
patch adding more instances of QAPIExprError inside the block.
It's easier to separate indentation from semantic changes, so
this patch has no real behavior change.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 37 ++++++++++++++++++++-----------------
1 file changed, 20 insertions(+), 17 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index fc7b7f1..39cc88b 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -398,6 +398,7 @@ def check_exprs(schema):
check_event(expr, info)
def parse_schema(input_file):
+ # First pass: read entire file into memory
try:
schema = QAPISchema(open(input_file, "r"))
except (QAPISchemaError, QAPIExprError), e:
@@ -406,24 +407,26 @@ def parse_schema(input_file):
exprs = []
- for expr_elem in schema.exprs:
- expr = expr_elem['expr']
- if expr.has_key('enum'):
- add_enum(expr['enum'], expr.get('data'))
- elif expr.has_key('union'):
- add_union(expr)
- elif expr.has_key('type'):
- add_struct(expr)
- exprs.append(expr)
-
- # Try again for hidden UnionKind enum
- for expr_elem in schema.exprs:
- expr = expr_elem['expr']
- if expr.has_key('union'):
- if not discriminator_find_enum_define(expr):
- add_enum('%sKind' % expr['union'])
-
try:
+ # Next pass: learn the types.
+ for expr_elem in schema.exprs:
+ expr = expr_elem['expr']
+ if expr.has_key('enum'):
+ add_enum(expr['enum'], expr.get('data'))
+ elif expr.has_key('union'):
+ add_union(expr)
+ elif expr.has_key('type'):
+ add_struct(expr)
+ exprs.append(expr)
+
+ # Try again for hidden UnionKind enum
+ for expr_elem in schema.exprs:
+ expr = expr_elem['expr']
+ if expr.has_key('union'):
+ if not discriminator_find_enum_define(expr):
+ add_enum('%sKind' % expr['union'])
+
+ # Final pass - validate that exprs make sense
check_exprs(schema)
except QAPIExprError, e:
print >>sys.stderr, e
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (8 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 09/28] qapi: Prepare for catching more semantic parse errors Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 14:47 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 11/28] qapi: Rename anonymous union type in test Eric Blake
` (19 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Special-casing 'discriminator == {}' for handling anonymous unions
is getting awkward; since this particular type is not always a
dictionary on the wire, it is easier to treat it as a completely
different class of type, "alternate", so that if a type is listed
in the union_types array, we know it is not an anonymous union.
This patch just further segregates union handling, to make sure that
anonymous unions are not stored in union_types, and splitting up
check_union() into separate functions. A future patch will change
the qapi grammar, and having the segregation already in place will
make it easier to deal with the distinct meta-type.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-types.py | 6 ++--
scripts/qapi-visit.py | 4 +--
scripts/qapi.py | 94 +++++++++++++++++++++++++++++----------------------
3 files changed, 58 insertions(+), 46 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 2390887..c9e0201 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -170,7 +170,7 @@ typedef enum %(name)s
return lookup_decl + enum_decl
-def generate_anon_union_qtypes(expr):
+def generate_alternate_qtypes(expr):
name = expr['union']
members = expr['data']
@@ -181,7 +181,7 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
name=name)
for key in members:
- qtype = find_anonymous_member_qtype(members[key])
+ qtype = find_alternate_member_qtype(members[key])
assert qtype, "Invalid anonymous union member"
ret += mcgen('''
@@ -408,7 +408,7 @@ for expr in exprs:
fdef.write(generate_enum_lookup('%sKind' % expr['union'],
expr['data'].keys()))
if expr.get('discriminator') == {}:
- fdef.write(generate_anon_union_qtypes(expr))
+ fdef.write(generate_alternate_qtypes(expr))
else:
continue
fdecl.write(ret)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3f82bd4..77b0a1f 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -237,7 +237,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s *obj, const char *name, Error **er
''',
name=name)
-def generate_visit_anon_union(name, members):
+def generate_visit_alternate(name, members):
ret = mcgen('''
void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
@@ -302,7 +302,7 @@ def generate_visit_union(expr):
if discriminator == {}:
assert not base
- return generate_visit_anon_union(name, members)
+ return generate_visit_alternate(name, members)
enum_define = discriminator_find_enum_define(expr)
if enum_define:
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 39cc88b..17252e9 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -224,21 +224,16 @@ def find_base_fields(base):
return None
return base_struct_define['data']
-# Return the qtype of an anonymous union branch, or None on error.
-def find_anonymous_member_qtype(qapi_type):
+# Return the qtype of an alternate branch, or None on error.
+def find_alternate_member_qtype(qapi_type):
if builtin_types.has_key(qapi_type):
return builtin_types[qapi_type]
elif find_struct(qapi_type):
return "QTYPE_QDICT"
elif find_enum(qapi_type):
return "QTYPE_QSTRING"
- else:
- union = find_union(qapi_type)
- if union:
- discriminator = union.get('discriminator')
- if discriminator == {}:
- return None
- return "QTYPE_QDICT"
+ elif find_union(qapi_type):
+ return "QTYPE_QDICT"
return None
# Return the discriminator enum define if discriminator is specified as an
@@ -276,22 +271,13 @@ def check_union(expr, expr_info):
discriminator = expr.get('discriminator')
members = expr['data']
values = { 'MAX': '(automatic)' }
- types_seen = {}
- # Three types of unions, determined by discriminator.
+ # Two types of unions, determined by discriminator.
+ assert discriminator != {}
- # If the value of member 'discriminator' is {}, it's an
- # anonymous union, and must not have a base.
- if discriminator == {}:
- enum_define = None
- if base:
- raise QAPIExprError(expr_info,
- "Anonymous union '%s' must not have a base"
- % name)
-
- # Else if the union object has no member 'discriminator', it's an
+ # If the union object has no member 'discriminator', it's an
# ordinary union. For now, it must not have a base.
- elif not discriminator:
+ if not discriminator:
enum_define = None
if base:
raise QAPIExprError(expr_info,
@@ -346,24 +332,46 @@ def check_union(expr, expr_info):
% (name, key, values[c_key]))
values[c_key] = key
- # Ensure anonymous unions have no type conflicts.
- if discriminator == {}:
- if isinstance(value, list):
- raise QAPIExprError(expr_info,
- "Anonymous union '%s' member '%s' must "
- "not be array type" % (name, key))
- qtype = find_anonymous_member_qtype(value)
- if not qtype:
- raise QAPIExprError(expr_info,
- "Anonymous union '%s' member '%s' has "
- "invalid type '%s'" % (name, key, value))
- if qtype in types_seen:
- raise QAPIExprError(expr_info,
- "Anonymous union '%s' member '%s' has "
- "same QObject type as member '%s'"
- % (name, key, types_seen[qtype]))
- types_seen[qtype] = key
+def check_alternate(expr, expr_info):
+ name = expr['union']
+ base = expr.get('base')
+ discriminator = expr.get('discriminator')
+ members = expr['data']
+ values = { 'MAX': '(automatic)' }
+ types_seen = {}
+ assert discriminator == {}
+ if base:
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' must not have a base"
+ % name)
+
+ # Check every branch
+ for (key, value) in members.items():
+ # Check for conflicts in the generated enum
+ c_key = _generate_enum_string(key)
+ if c_key in values:
+ raise QAPIExprError(expr_info,
+ "Union '%s' member '%s' clashes with '%s'"
+ % (name, key, values[c_key]))
+ values[c_key] = key
+
+ # Ensure alternates have no type conflicts.
+ if isinstance(value, list):
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' member '%s' must "
+ "not be array type" % (name, key))
+ qtype = find_alternate_member_qtype(value)
+ if not qtype:
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' member '%s' has "
+ "invalid type '%s'" % (name, key, value))
+ if qtype in types_seen:
+ raise QAPIExprError(expr_info,
+ "Anonymous union '%s' member '%s' has "
+ "same QObject type as member '%s'"
+ % (name, key, types_seen[qtype]))
+ types_seen[qtype] = key
def check_enum(expr, expr_info):
name = expr['enum']
@@ -393,7 +401,10 @@ def check_exprs(schema):
if expr.has_key('enum'):
check_enum(expr, info)
elif expr.has_key('union'):
- check_union(expr, info)
+ if expr.get('discriminator') == {}:
+ check_alternate(expr, info)
+ else:
+ check_union(expr, info)
elif expr.has_key('event'):
check_event(expr, info)
@@ -535,7 +546,8 @@ def find_struct(name):
def add_union(definition):
global union_types
- union_types.append(definition)
+ if definition.get('discriminator') != {}:
+ union_types.append(definition)
def find_union(name):
global union_types
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 11/28] qapi: Rename anonymous union type in test
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (9 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 14:55 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union Eric Blake
` (18 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Reduce churn in the future patch that replaces anonymous unions
with a new metatype 'alternate' by changing 'AnonUnion' to
'Alternate'.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qapi-schema/qapi-schema-test.json | 2 +-
tests/qapi-schema/qapi-schema-test.out | 4 ++--
tests/test-qmp-input-strict.c | 28 ++++++++++++++--------------
tests/test-qmp-input-visitor.c | 16 ++++++++--------
tests/test-qmp-output-visitor.c | 16 ++++++++--------
5 files changed, 33 insertions(+), 33 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index b134f3f..e1d35e1 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -53,7 +53,7 @@
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefC', 'value2' : 'UserDefB', 'value3' : 'UserDefA' } }
-{ 'union': 'UserDefAnonUnion',
+{ 'union': 'UserDefAlternate',
'discriminator': {},
'data': { 'uda': 'UserDefA', 's': 'str', 'i': 'int' } }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 664ae7b..b55ab8d 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -10,7 +10,7 @@
OrderedDict([('type', 'UserDefUnionBase'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
OrderedDict([('union', 'UserDefFlatUnion'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefA'), ('value2', 'UserDefB'), ('value3', 'UserDefB')]))]),
OrderedDict([('union', 'UserDefFlatUnion2'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefC'), ('value2', 'UserDefB'), ('value3', 'UserDefA')]))]),
- OrderedDict([('union', 'UserDefAnonUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
+ OrderedDict([('union', 'UserDefAlternate'), ('discriminator', OrderedDict()), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
OrderedDict([('union', 'UserDefNativeListUnion'), ('data', OrderedDict([('integer', ['int']), ('s8', ['int8']), ('s16', ['int16']), ('s32', ['int32']), ('s64', ['int64']), ('u8', ['uint8']), ('u16', ['uint16']), ('u32', ['uint32']), ('u64', ['uint64']), ('number', ['number']), ('boolean', ['bool']), ('string', ['str']), ('sizes', ['size'])]))]),
OrderedDict([('command', 'user_def_cmd'), ('data', OrderedDict())]),
OrderedDict([('command', 'user_def_cmd1'), ('data', OrderedDict([('ud1a', 'UserDefOne')]))]),
@@ -23,7 +23,7 @@
OrderedDict([('event', 'EVENT_C'), ('data', OrderedDict([('*a', 'int'), ('*b', 'UserDefOne'), ('c', 'str')]))]),
OrderedDict([('event', 'EVENT_D'), ('data', OrderedDict([('a', 'EventStructOne'), ('b', 'str'), ('*c', 'str'), ('*enum3', 'EnumOne')]))])]
[{'enum_name': 'EnumOne', 'enum_values': ['value1', 'value2', 'value3']},
- {'enum_name': 'UserDefAnonUnionKind', 'enum_values': None},
+ {'enum_name': 'UserDefAlternateKind', 'enum_values': None},
{'enum_name': 'UserDefNativeListUnionKind', 'enum_values': None}]
[OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 53134a1..4b53560 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -173,18 +173,18 @@ static void test_validate_union_flat(TestInputVisitorData *data,
qapi_free_UserDefFlatUnion(tmp);
}
-static void test_validate_union_anon(TestInputVisitorData *data,
- const void *unused)
+static void test_validate_alternate(TestInputVisitorData *data,
+ const void *unused)
{
- UserDefAnonUnion *tmp = NULL;
+ UserDefAlternate *tmp = NULL;
Visitor *v;
Error *err = NULL;
v = validate_test_init(data, "42");
- visit_type_UserDefAnonUnion(v, &tmp, NULL, &err);
+ visit_type_UserDefAlternate(v, &tmp, NULL, &err);
g_assert(!err);
- qapi_free_UserDefAnonUnion(tmp);
+ qapi_free_UserDefAlternate(tmp);
}
static void test_validate_fail_struct(TestInputVisitorData *data,
@@ -276,18 +276,18 @@ static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
qapi_free_UserDefFlatUnion2(tmp);
}
-static void test_validate_fail_union_anon(TestInputVisitorData *data,
- const void *unused)
+static void test_validate_fail_alternate(TestInputVisitorData *data,
+ const void *unused)
{
- UserDefAnonUnion *tmp = NULL;
+ UserDefAlternate *tmp = NULL;
Visitor *v;
Error *err = NULL;
v = validate_test_init(data, "3.14");
- visit_type_UserDefAnonUnion(v, &tmp, NULL, &err);
+ visit_type_UserDefAlternate(v, &tmp, NULL, &err);
g_assert(err);
- qapi_free_UserDefAnonUnion(tmp);
+ qapi_free_UserDefAlternate(tmp);
}
static void validate_test_add(const char *testpath,
@@ -312,8 +312,8 @@ int main(int argc, char **argv)
&testdata, test_validate_list);
validate_test_add("/visitor/input-strict/pass/union-flat",
&testdata, test_validate_union_flat);
- validate_test_add("/visitor/input-strict/pass/union-anon",
- &testdata, test_validate_union_anon);
+ validate_test_add("/visitor/input-strict/pass/alternate",
+ &testdata, test_validate_alternate);
validate_test_add("/visitor/input-strict/pass/union-native-list",
&testdata, test_validate_union_native_list);
validate_test_add("/visitor/input-strict/fail/struct",
@@ -326,8 +326,8 @@ int main(int argc, char **argv)
&testdata, test_validate_fail_union_flat);
validate_test_add("/visitor/input-strict/fail/union-flat-no-discriminator",
&testdata, test_validate_fail_union_flat_no_discrim);
- validate_test_add("/visitor/input-strict/fail/union-anon",
- &testdata, test_validate_fail_union_anon);
+ validate_test_add("/visitor/input-strict/fail/alternate",
+ &testdata, test_validate_fail_alternate);
validate_test_add("/visitor/input-strict/fail/union-native-list",
&testdata, test_validate_fail_union_native_list);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index cc33f64..882359a 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -315,20 +315,20 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,
qapi_free_UserDefFlatUnion(tmp);
}
-static void test_visitor_in_union_anon(TestInputVisitorData *data,
- const void *unused)
+static void test_visitor_in_alternate(TestInputVisitorData *data,
+ const void *unused)
{
Visitor *v;
Error *err = NULL;
- UserDefAnonUnion *tmp;
+ UserDefAlternate *tmp;
v = visitor_input_test_init(data, "42");
- visit_type_UserDefAnonUnion(v, &tmp, NULL, &err);
+ visit_type_UserDefAlternate(v, &tmp, NULL, &err);
g_assert(err == NULL);
- g_assert_cmpint(tmp->kind, ==, USER_DEF_ANON_UNION_KIND_I);
+ g_assert_cmpint(tmp->kind, ==, USER_DEF_ALTERNATE_KIND_I);
g_assert_cmpint(tmp->i, ==, 42);
- qapi_free_UserDefAnonUnion(tmp);
+ qapi_free_UserDefAlternate(tmp);
}
static void test_native_list_integer_helper(TestInputVisitorData *data,
@@ -664,8 +664,8 @@ int main(int argc, char **argv)
&in_visitor_data, test_visitor_in_list);
input_visitor_test_add("/visitor/input/union-flat",
&in_visitor_data, test_visitor_in_union_flat);
- input_visitor_test_add("/visitor/input/union-anon",
- &in_visitor_data, test_visitor_in_union_anon);
+ input_visitor_test_add("/visitor/input/alternate",
+ &in_visitor_data, test_visitor_in_alternate);
input_visitor_test_add("/visitor/input/errors",
&in_visitor_data, test_visitor_in_errors);
input_visitor_test_add("/visitor/input/native_list/int",
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index ebe6ea3..602bc12 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -453,24 +453,24 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,
QDECREF(qdict);
}
-static void test_visitor_out_union_anon(TestOutputVisitorData *data,
- const void *unused)
+static void test_visitor_out_alternate(TestOutputVisitorData *data,
+ const void *unused)
{
QObject *arg;
Error *err = NULL;
- UserDefAnonUnion *tmp = g_malloc0(sizeof(UserDefAnonUnion));
- tmp->kind = USER_DEF_ANON_UNION_KIND_I;
+ UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
+ tmp->kind = USER_DEF_ALTERNATE_KIND_I;
tmp->i = 42;
- visit_type_UserDefAnonUnion(data->ov, &tmp, NULL, &err);
+ visit_type_UserDefAlternate(data->ov, &tmp, NULL, &err);
g_assert(err == NULL);
arg = qmp_output_get_qobject(data->qov);
g_assert(qobject_type(arg) == QTYPE_QINT);
g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
- qapi_free_UserDefAnonUnion(tmp);
+ qapi_free_UserDefAlternate(tmp);
}
static void test_visitor_out_empty(TestOutputVisitorData *data,
@@ -830,8 +830,8 @@ int main(int argc, char **argv)
&out_visitor_data, test_visitor_out_list_qapi_free);
output_visitor_test_add("/visitor/output/union-flat",
&out_visitor_data, test_visitor_out_union_flat);
- output_visitor_test_add("/visitor/output/union-anon",
- &out_visitor_data, test_visitor_out_union_anon);
+ output_visitor_test_add("/visitor/output/alternate",
+ &out_visitor_data, test_visitor_out_alternate);
output_visitor_test_add("/visitor/output/empty",
&out_visitor_data, test_visitor_out_empty);
output_visitor_test_add("/visitor/output/native_list/int",
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (10 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 11/28] qapi: Rename anonymous union type in test Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-24 20:41 ` Eric Blake
2015-03-26 15:42 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests Eric Blake
` (17 subsequent siblings)
29 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Rather than special-casing "'union':'foo','alternate':{}" as an
unusual union that can represent a non-dictionary, it is nicer
to designate a separate meta-type "'alternate':'foo'" for the
purpose. This involves a lot of documentation tweaks and fallout
from .json files, but I already split as much as possible into
earlier commits.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 63 +++++++++++-----------
qapi/block-core.json | 6 +--
scripts/qapi-types.py | 26 ++++++---
scripts/qapi-visit.py | 17 +++---
scripts/qapi.py | 41 +++++++-------
tests/qapi-schema/alternate-array.err | 2 +-
tests/qapi-schema/alternate-array.json | 3 +-
tests/qapi-schema/alternate-base.err | 2 +-
tests/qapi-schema/alternate-base.json | 5 +-
tests/qapi-schema/alternate-clash.err | 2 +-
tests/qapi-schema/alternate-clash.json | 5 +-
tests/qapi-schema/alternate-conflict-dict.err | 2 +-
tests/qapi-schema/alternate-conflict-dict.json | 5 +-
tests/qapi-schema/alternate-conflict-string.err | 2 +-
tests/qapi-schema/alternate-conflict-string.json | 3 +-
tests/qapi-schema/alternate-good.json | 5 +-
tests/qapi-schema/alternate-good.out | 4 +-
tests/qapi-schema/alternate-nested.err | 2 +-
tests/qapi-schema/alternate-nested.json | 8 ++-
tests/qapi-schema/alternate-unknown.err | 2 +-
tests/qapi-schema/alternate-unknown.json | 3 +-
tests/qapi-schema/flat-union-bad-base.err | 2 +-
tests/qapi-schema/flat-union-bad-discriminator.err | 2 +-
.../qapi-schema/flat-union-bad-discriminator.json | 5 +-
tests/qapi-schema/flat-union-base-union.err | 2 +-
tests/qapi-schema/flat-union-base-union.json | 2 +-
tests/qapi-schema/flat-union-no-base.err | 2 +-
tests/qapi-schema/qapi-schema-test.json | 3 +-
tests/qapi-schema/qapi-schema-test.out | 2 +-
tests/qapi-schema/union-invalid-base.err | 2 +-
30 files changed, 117 insertions(+), 113 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index ce9c4b9..8792b94 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -79,16 +79,16 @@ the definition of complex structs that can have mutually recursive
types, and allows for indefinite nesting of QMP that satisfies the
schema. A type name should not be defined more than once.
-There are six top-level expressions recognized by the parser:
-'include', 'command', 'type', 'enum', 'union', and 'event'. There are
-several built-in types, such as 'int' and 'str'; additionally, the
-top-level expressions can define complex types, enumeration types, and
-several flavors of union types. The 'command' expression can refer to
-existing types by name, or list an anonymous type as a dictionary.
-Listing a type name inside an array refers to a single-dimension array
-of that type; multi-dimension arrays are not directly supported
-(although an array of a complex struct that contains an array member
-is possible).
+There are seven top-level expressions recognized by the parser:
+'include', 'command', 'type', 'enum', 'union', 'alternate', and
+'event'. There are several built-in types, such as 'int' and 'str';
+additionally, the top-level expressions can define complex types,
+enumeration types, two flavors of union types, and an alternate type.
+The 'command' expression can refer to existing types by name, or list
+an anonymous type as a dictionary. Listing a type name inside an
+array refers to a single-dimension array of that type; multi-dimension
+arrays are not directly supported (although an array of a complex
+struct that contains an array member is possible).
Types, commands, and events share a common namespace. Therefore,
generally speaking, type definitions should always use CamelCase for
@@ -159,8 +159,8 @@ Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
A complex type is a dictionary containing a single 'data' key whose
value is a dictionary. This corresponds to a struct in C or an Object
in JSON. Each value of the 'data' dictionary must be the name of a
-complex, enum, union, or built-in type, or a one-element array
-containing a type name. An example of a complex type is:
+complex, enum, union, alternate, or built-in type, or a one-element
+array containing a type name. An example of a complex type is:
{ 'type': 'MyType',
'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
@@ -246,14 +246,12 @@ open-coding the field to be type 'str'.
Usage: { 'union': 'str', 'data': 'dict' }
or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
'discriminator': 'enum-member-of-base' }
-or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
Union types are used to let the user choose between several different
-data types. There are three flavors: simple (no discriminator), flat
-(a base type is mandatory, and discriminator is the name of an enum
-field within that base type), and anonymous (discriminator is an
-empty dictionary). A union type is defined using a data dictionary as
-explained in the following paragraphs.
+data types. There are two flavors: simple (no discriminator or base),
+and flat (a base type is mandatory, and discriminator is the name of
+an enum field within that base type). A union type is defined using a
+data dictionary as explained in the following paragraphs.
A simple union type defines a mapping from automatic discriminator
values to data types like in this example:
@@ -321,22 +319,23 @@ Resulting in this JSON object:
"lazy-refcounts": true }
-The final flavor of unions is an anonymous union. While the other two
-union types are always passed as a dictionary in the wire format, an
-anonymous union instead allows the direct use of different types in
-its place. As they aren't structured, they don't have any explicit
-discriminator but use the (QObject) data type of their value as an
-implicit discriminator. This means that they are restricted to using
-only one discriminator value per QObject type. For example, you cannot
-have two different complex types in an anonymous union, or two
-different integer types.
+=== Alternate types ===
-Anonymous unions are declared using an empty dictionary as their discriminator.
-The discriminator values never appear on the wire, they are only used in the
-generated C code. Anonymous unions cannot have a base type.
+Usage: { 'alternate: 'str', 'data': 'dict' }
- { 'union': 'BlockRef',
- 'discriminator': {},
+An alternate type is one that allows a choice between two or more
+QObject data types (string, integer, number, or dictionary, but not
+array) on the wire. The definition is similar to a simple union type,
+where each branch of the dictionary names a type, and where an
+implicit C enum NameKind is created for the alternate Name. But
+unlike a union, the discriminator string is never passed on the wire
+for QMP, instead appearing only in the generated C code. The type on
+the wire serves an implicit discriminator, which in turn means that an
+alternate can express a choice between a string and a single complex
+type (passed as a dictionary), but cannot distinguish between two
+different complex types. For example:
+
+ { 'alternate': 'BlockRef',
'data': { 'definition': 'BlockdevOptions',
'reference': 'str' } }
diff --git a/qapi/block-core.json b/qapi/block-core.json
index f525b04..581c448 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1419,8 +1419,7 @@
#
# Since: 2.2
##
-{ 'union': 'Qcow2OverlapChecks',
- 'discriminator': {},
+{ 'alternate': 'Qcow2OverlapChecks',
'data': { 'flags': 'Qcow2OverlapCheckFlags',
'mode': 'Qcow2OverlapCheckMode' } }
@@ -1711,8 +1710,7 @@
#
# Since: 1.7
##
-{ 'union': 'BlockdevRef',
- 'discriminator': {},
+{ 'alternate': 'BlockdevRef',
'data': { 'definition': 'BlockdevOptions',
'reference': 'str' } }
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index c9e0201..9c8d68c 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -172,7 +172,7 @@ typedef enum %(name)s
def generate_alternate_qtypes(expr):
- name = expr['union']
+ name = expr['alternate']
members = expr['data']
ret = mcgen('''
@@ -182,7 +182,7 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
for key in members:
qtype = find_alternate_member_qtype(members[key])
- assert qtype, "Invalid anonymous union member"
+ assert qtype, "Invalid alternate member"
ret += mcgen('''
[ %(qtype)s ] = %(abbrev)s_KIND_%(enum)s,
@@ -197,9 +197,9 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
return ret
-def generate_union(expr):
+def generate_union(expr, meta):
- name = expr['union']
+ name = expr[meta]
typeinfo = expr['data']
base = expr.get('base')
@@ -243,7 +243,7 @@ struct %(name)s
ret += mcgen('''
};
''')
- if discriminator == {}:
+ if meta == 'alternate':
ret += mcgen('''
extern const int %(name)s_qtypes[];
''',
@@ -407,8 +407,12 @@ for expr in exprs:
ret += generate_enum('%sKind' % expr['union'], expr['data'].keys())
fdef.write(generate_enum_lookup('%sKind' % expr['union'],
expr['data'].keys()))
- if expr.get('discriminator') == {}:
- fdef.write(generate_alternate_qtypes(expr))
+ elif expr.has_key('alternate'):
+ ret += generate_fwd_struct(expr['alternate'], expr['data']) + "\n"
+ ret += generate_enum('%sKind' % expr['alternate'], expr['data'].keys())
+ fdef.write(generate_enum_lookup('%sKind' % expr['alternate'],
+ expr['data'].keys()))
+ fdef.write(generate_alternate_qtypes(expr))
else:
continue
fdecl.write(ret)
@@ -438,11 +442,17 @@ for expr in exprs:
ret += generate_type_cleanup_decl(expr['type'])
fdef.write(generate_type_cleanup(expr['type']) + "\n")
elif expr.has_key('union'):
- ret += generate_union(expr)
+ ret += generate_union(expr, 'union')
ret += generate_type_cleanup_decl(expr['union'] + "List")
fdef.write(generate_type_cleanup(expr['union'] + "List") + "\n")
ret += generate_type_cleanup_decl(expr['union'])
fdef.write(generate_type_cleanup(expr['union']) + "\n")
+ elif expr.has_key('alternate'):
+ ret += generate_union(expr, 'alternate')
+ ret += generate_type_cleanup_decl(expr['alternate'] + "List")
+ fdef.write(generate_type_cleanup(expr['alternate'] + "List") + "\n")
+ ret += generate_type_cleanup_decl(expr['alternate'])
+ fdef.write(generate_type_cleanup(expr['alternate']) + "\n")
elif expr.has_key('enum'):
ret += generate_type_cleanup_decl(expr['enum'] + "List")
fdef.write(generate_type_cleanup(expr['enum'] + "List") + "\n")
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 77b0a1f..3e11089 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -256,7 +256,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
''',
name=name)
- # For anon union, always use the default enum type automatically generated
+ # For alternate, always use the default enum type automatically generated
# as "'%sKind' % (name)"
disc_type = '%sKind' % (name)
@@ -264,7 +264,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
assert (members[key] in builtin_types.keys()
or find_struct(members[key])
or find_union(members[key])
- or find_enum(members[key])), "Invalid anonymous union member"
+ or find_enum(members[key])), "Invalid alternate member"
enum_full_value = generate_enum_full_value(disc_type, key)
ret += mcgen('''
@@ -300,10 +300,6 @@ def generate_visit_union(expr):
base = expr.get('base')
discriminator = expr.get('discriminator')
- if discriminator == {}:
- assert not base
- return generate_visit_alternate(name, members)
-
enum_define = discriminator_find_enum_define(expr)
if enum_define:
# Use the enum type as discriminator
@@ -572,6 +568,15 @@ for expr in exprs:
expr['data'].keys())
ret += generate_declaration(expr['union'], expr['data'])
fdecl.write(ret)
+ elif expr.has_key('alternate'):
+ ret = generate_visit_alternate(expr['alternate'], expr['data'])
+ ret += generate_visit_list(expr['alternate'], expr['data'])
+ fdef.write(ret)
+
+ ret = generate_decl_enum('%sKind' % expr['alternate'],
+ expr['data'].keys())
+ ret += generate_declaration(expr['alternate'], expr['data'])
+ fdecl.write(ret)
elif expr.has_key('enum'):
ret = generate_visit_list(expr['enum'], expr['data'])
ret += generate_visit_enum(expr['enum'], expr['data'])
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 17252e9..018ec45 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -273,11 +273,10 @@ def check_union(expr, expr_info):
values = { 'MAX': '(automatic)' }
# Two types of unions, determined by discriminator.
- assert discriminator != {}
# If the union object has no member 'discriminator', it's an
# ordinary union. For now, it must not have a base.
- if not discriminator:
+ if discriminator is None:
enum_define = None
if base:
raise QAPIExprError(expr_info,
@@ -287,18 +286,22 @@ def check_union(expr, expr_info):
# Else, it's a flat union.
else:
# The object must have a member 'base'.
- if not base:
+ if not isinstance(base, str):
raise QAPIExprError(expr_info,
- "Flat union '%s' must have a base field"
+ "Flat union '%s' must have a string base field"
% name)
base_fields = find_base_fields(base)
if not base_fields:
raise QAPIExprError(expr_info,
- "Base '%s' is not a valid type"
+ "Base '%s' is not a valid base type"
% base)
# The value of member 'discriminator' must name a member of the
# base type.
+ if not isinstance(discriminator, str):
+ raise QAPIExprError(expr_info,
+ "Flat union '%s' must have a string "
+ "discriminator field" % name)
discriminator_type = base_fields.get(discriminator)
if not discriminator_type:
raise QAPIExprError(expr_info,
@@ -333,17 +336,15 @@ def check_union(expr, expr_info):
values[c_key] = key
def check_alternate(expr, expr_info):
- name = expr['union']
- base = expr.get('base')
- discriminator = expr.get('discriminator')
+ name = expr['alternate']
members = expr['data']
values = { 'MAX': '(automatic)' }
types_seen = {}
- assert discriminator == {}
+ base = expr.get('base')
if base:
raise QAPIExprError(expr_info,
- "Anonymous union '%s' must not have a base"
+ "Alternate '%s' must not have a base"
% name)
# Check every branch
@@ -352,23 +353,23 @@ def check_alternate(expr, expr_info):
c_key = _generate_enum_string(key)
if c_key in values:
raise QAPIExprError(expr_info,
- "Union '%s' member '%s' clashes with '%s'"
+ "Alternate '%s' member '%s' clashes with '%s'"
% (name, key, values[c_key]))
values[c_key] = key
# Ensure alternates have no type conflicts.
if isinstance(value, list):
raise QAPIExprError(expr_info,
- "Anonymous union '%s' member '%s' must "
+ "Alternate '%s' member '%s' must "
"not be array type" % (name, key))
qtype = find_alternate_member_qtype(value)
if not qtype:
raise QAPIExprError(expr_info,
- "Anonymous union '%s' member '%s' has "
+ "Alternate '%s' member '%s' has "
"invalid type '%s'" % (name, key, value))
if qtype in types_seen:
raise QAPIExprError(expr_info,
- "Anonymous union '%s' member '%s' has "
+ "Alternate '%s' member '%s' has "
"same QObject type as member '%s'"
% (name, key, types_seen[qtype]))
types_seen[qtype] = key
@@ -401,10 +402,9 @@ def check_exprs(schema):
if expr.has_key('enum'):
check_enum(expr, info)
elif expr.has_key('union'):
- if expr.get('discriminator') == {}:
- check_alternate(expr, info)
- else:
- check_union(expr, info)
+ check_union(expr, info)
+ elif expr.has_key('alternate'):
+ check_alternate(expr, info)
elif expr.has_key('event'):
check_event(expr, info)
@@ -436,6 +436,8 @@ def parse_schema(input_file):
if expr.has_key('union'):
if not discriminator_find_enum_define(expr):
add_enum('%sKind' % expr['union'])
+ elif expr.has_key('alternate'):
+ add_enum('%sKind' % expr['alternate'])
# Final pass - validate that exprs make sense
check_exprs(schema)
@@ -546,8 +548,7 @@ def find_struct(name):
def add_union(definition):
global union_types
- if definition.get('discriminator') != {}:
- union_types.append(definition)
+ union_types.append(definition)
def find_union(name):
global union_types
diff --git a/tests/qapi-schema/alternate-array.err b/tests/qapi-schema/alternate-array.err
index 2638b85..5721ed2 100644
--- a/tests/qapi-schema/alternate-array.err
+++ b/tests/qapi-schema/alternate-array.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-array.json:4: Anonymous union 'MyUnion' member 'two' must not be array type
+tests/qapi-schema/alternate-array.json:4: Alternate 'Alt' member 'two' must not be array type
diff --git a/tests/qapi-schema/alternate-array.json b/tests/qapi-schema/alternate-array.json
index e330d57..86d8fa0 100644
--- a/tests/qapi-schema/alternate-array.json
+++ b/tests/qapi-schema/alternate-array.json
@@ -1,7 +1,6 @@
# FIXME: we do not support array branches of anonymous unions yet
{ 'type': 'One',
'data': { 'name': 'str' } }
-{ 'union': 'MyUnion',
- 'discriminator': {},
+{ 'alternate': 'Alt',
'data': { 'one': 'One',
'two': [ 'int' ] } }
diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
index a2486b8..4a2566e 100644
--- a/tests/qapi-schema/alternate-base.err
+++ b/tests/qapi-schema/alternate-base.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-base.json:4: Anonymous union 'MyUnion' must not have a base
+tests/qapi-schema/alternate-base.json:4: Alternate 'Alt' must not have a base
diff --git a/tests/qapi-schema/alternate-base.json b/tests/qapi-schema/alternate-base.json
index dad7f02..66edc89 100644
--- a/tests/qapi-schema/alternate-base.json
+++ b/tests/qapi-schema/alternate-base.json
@@ -1,7 +1,6 @@
-# we reject anonymous union with base type
+# we reject alternate with base type
{ 'type': 'Base',
'data': { 'string': 'str' } }
-{ 'union': 'MyUnion',
+{ 'alternate': 'Alt',
'base': 'Base',
- 'discriminator': {},
'data': { 'number': 'int' } }
diff --git a/tests/qapi-schema/alternate-clash.err b/tests/qapi-schema/alternate-clash.err
index 1130c12..51bea3e 100644
--- a/tests/qapi-schema/alternate-clash.err
+++ b/tests/qapi-schema/alternate-clash.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-clash.json:2: Union 'Union1' member 'ONE' clashes with 'one'
+tests/qapi-schema/alternate-clash.json:2: Alternate 'Alt1' member 'ONE' clashes with 'one'
diff --git a/tests/qapi-schema/alternate-clash.json b/tests/qapi-schema/alternate-clash.json
index fa2d27e..3947935 100644
--- a/tests/qapi-schema/alternate-clash.json
+++ b/tests/qapi-schema/alternate-clash.json
@@ -1,4 +1,3 @@
-# we detect C enum collisions in an anonymous union
-{ 'union': 'Union1',
- 'discriminator': {},
+# we detect C enum collisions in an alternate
+{ 'alternate': 'Alt1',
'data': { 'one': 'str', 'ONE': 'int' } }
diff --git a/tests/qapi-schema/alternate-conflict-dict.err b/tests/qapi-schema/alternate-conflict-dict.err
index f256d51..c535332 100644
--- a/tests/qapi-schema/alternate-conflict-dict.err
+++ b/tests/qapi-schema/alternate-conflict-dict.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-conflict-dict.json:6: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
+tests/qapi-schema/alternate-conflict-dict.json:6: Alternate 'Alt' member 'two' has same QObject type as member 'one'
diff --git a/tests/qapi-schema/alternate-conflict-dict.json b/tests/qapi-schema/alternate-conflict-dict.json
index 177c163..eaac5b3 100644
--- a/tests/qapi-schema/alternate-conflict-dict.json
+++ b/tests/qapi-schema/alternate-conflict-dict.json
@@ -1,9 +1,8 @@
-# we reject anonymous unions with multiple object branches
+# we reject alternates with multiple object branches
{ 'type': 'One',
'data': { 'name': 'str' } }
{ 'type': 'Two',
'data': { 'value': 'int' } }
-{ 'union': 'MyUnion',
- 'discriminator': {},
+{ 'alternate': 'Alt',
'data': { 'one': 'One',
'two': 'Two' } }
diff --git a/tests/qapi-schema/alternate-conflict-string.err b/tests/qapi-schema/alternate-conflict-string.err
index a3b7b6d..18ef0d3 100644
--- a/tests/qapi-schema/alternate-conflict-string.err
+++ b/tests/qapi-schema/alternate-conflict-string.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-conflict-string.json:4: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
+tests/qapi-schema/alternate-conflict-string.json:4: Alternate 'Alt' member 'two' has same QObject type as member 'one'
diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
index a1b0bea..ce00307 100644
--- a/tests/qapi-schema/alternate-conflict-string.json
+++ b/tests/qapi-schema/alternate-conflict-string.json
@@ -1,8 +1,7 @@
# we reject anonymous unions with multiple string-like branches
{ 'enum': 'Enum',
'data': [ 'hello', 'world' ] }
-{ 'union': 'MyUnion',
- 'discriminator': {},
+{ 'alternate': 'Alt',
'data': { 'one': 'str',
'two': 'Enum' } }
diff --git a/tests/qapi-schema/alternate-good.json b/tests/qapi-schema/alternate-good.json
index 1068e2f..74f338c 100644
--- a/tests/qapi-schema/alternate-good.json
+++ b/tests/qapi-schema/alternate-good.json
@@ -1,10 +1,9 @@
-# Working example of anonymous union
+# Working example of alternate
{ 'type': 'Data',
'data': { '*number': 'int', '*name': 'str' } }
{ 'enum': 'Enum',
'data': [ 'hello', 'world' ] }
-{ 'union': 'MyUnion',
- 'discriminator': {},
+{ 'alternate': 'Alt',
'data': { 'value': 'int',
'string': 'Enum',
'struct': 'Data' } }
diff --git a/tests/qapi-schema/alternate-good.out b/tests/qapi-schema/alternate-good.out
index b5117d1..c3a6b77 100644
--- a/tests/qapi-schema/alternate-good.out
+++ b/tests/qapi-schema/alternate-good.out
@@ -1,6 +1,6 @@
[OrderedDict([('type', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))]),
OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
- OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('value', 'int'), ('string', 'Enum'), ('struct', 'Data')]))])]
+ OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('value', 'int'), ('string', 'Enum'), ('struct', 'Data')]))])]
[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
- {'enum_name': 'MyUnionKind', 'enum_values': None}]
+ {'enum_name': 'AltKind', 'enum_values': None}]
[OrderedDict([('type', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))])]
diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
index 59df96e..00b05c6 100644
--- a/tests/qapi-schema/alternate-nested.err
+++ b/tests/qapi-schema/alternate-nested.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-nested.json:5: Anonymous union 'Union2' member 'nested' has invalid type 'Union1'
+tests/qapi-schema/alternate-nested.json:4: Alternate 'Alt2' member 'nested' has invalid type 'Alt1'
diff --git a/tests/qapi-schema/alternate-nested.json b/tests/qapi-schema/alternate-nested.json
index ed2b6b7..2a115b9 100644
--- a/tests/qapi-schema/alternate-nested.json
+++ b/tests/qapi-schema/alternate-nested.json
@@ -1,7 +1,5 @@
# we reject a nested anonymous union branch
-{ 'union': 'Union1',
- 'discriminator': {},
+{ 'alternate': 'Alt1',
'data': { 'name': 'str', 'value': 'int' } }
-{ 'union': 'Union2',
- 'discriminator': {},
- 'data': { 'nested': 'Union1' } }
+{ 'alternate': 'Alt2',
+ 'data': { 'nested': 'Alt1' } }
diff --git a/tests/qapi-schema/alternate-unknown.err b/tests/qapi-schema/alternate-unknown.err
index bf8e9ae..7af1b4c 100644
--- a/tests/qapi-schema/alternate-unknown.err
+++ b/tests/qapi-schema/alternate-unknown.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-unknown.json:2: Anonymous union 'Union' member 'unknown' has invalid type 'MissingType'
+tests/qapi-schema/alternate-unknown.json:2: Alternate 'Alt' member 'unknown' has invalid type 'MissingType'
diff --git a/tests/qapi-schema/alternate-unknown.json b/tests/qapi-schema/alternate-unknown.json
index 0c305c2..d7b217e 100644
--- a/tests/qapi-schema/alternate-unknown.json
+++ b/tests/qapi-schema/alternate-unknown.json
@@ -1,4 +1,3 @@
# we reject an anonymous union with unknown type in branch
-{ 'union': 'Union',
- 'discriminator': {},
+{ 'alternate': 'Alt',
'data': { 'unknown': 'MissingType' } }
diff --git a/tests/qapi-schema/flat-union-bad-base.err b/tests/qapi-schema/flat-union-bad-base.err
index 5962ff4..f9c31b2 100644
--- a/tests/qapi-schema/flat-union-bad-base.err
+++ b/tests/qapi-schema/flat-union-bad-base.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-bad-base.json:9: Base 'OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')])' is not a valid type
+tests/qapi-schema/flat-union-bad-base.json:9: Flat union 'TestUnion' must have a string base field
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.err b/tests/qapi-schema/flat-union-bad-discriminator.err
index 628432f..4f0c798 100644
--- a/tests/qapi-schema/flat-union-bad-discriminator.err
+++ b/tests/qapi-schema/flat-union-bad-discriminator.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-bad-discriminator.json:10: Simple union 'TestUnion' must not have a base
+tests/qapi-schema/flat-union-bad-discriminator.json:11: Flat union 'TestUnion' must have a string discriminator field
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.json b/tests/qapi-schema/flat-union-bad-discriminator.json
index 1599a59..982f072 100644
--- a/tests/qapi-schema/flat-union-bad-discriminator.json
+++ b/tests/qapi-schema/flat-union-bad-discriminator.json
@@ -1,4 +1,5 @@
-# FIXME: we should require the discriminator to be a string naming a base-type member
+# we require the discriminator to be a string naming a base-type member
+# this tests the old syntax for anonymous unions before we added alternates
{ 'enum': 'TestEnum',
'data': [ 'value1', 'value2' ] }
{ 'type': 'TestBase',
@@ -9,6 +10,6 @@
'data': { 'integer': 'int' } }
{ 'union': 'TestUnion',
'base': 'TestBase',
- 'discriminator': [],
+ 'discriminator': {},
'data': { 'kind1': 'TestTypeA',
'kind2': 'TestTypeB' } }
diff --git a/tests/qapi-schema/flat-union-base-union.err b/tests/qapi-schema/flat-union-base-union.err
index 185bf51..318c047 100644
--- a/tests/qapi-schema/flat-union-base-union.err
+++ b/tests/qapi-schema/flat-union-base-union.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-base-union.json:11: Base 'UnionBase' is not a valid type
+tests/qapi-schema/flat-union-base-union.json:11: Base 'UnionBase' is not a valid base type
diff --git a/tests/qapi-schema/flat-union-base-union.json b/tests/qapi-schema/flat-union-base-union.json
index bbaa2da..135332d 100644
--- a/tests/qapi-schema/flat-union-base-union.json
+++ b/tests/qapi-schema/flat-union-base-union.json
@@ -1,4 +1,4 @@
-# FIXME: the error message needs help: we require the base to be a struct
+# we require the base to be a struct
{ 'enum': 'TestEnum',
'data': [ 'value1', 'value2' ] }
{ 'type': 'TestTypeA',
diff --git a/tests/qapi-schema/flat-union-no-base.err b/tests/qapi-schema/flat-union-no-base.err
index eaf3592..338bff1 100644
--- a/tests/qapi-schema/flat-union-no-base.err
+++ b/tests/qapi-schema/flat-union-no-base.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-no-base.json:8: Flat union 'TestUnion' must have a base field
+tests/qapi-schema/flat-union-no-base.json:8: Flat union 'TestUnion' must have a string base field
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index e1d35e1..dec8a7c 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -53,8 +53,7 @@
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefC', 'value2' : 'UserDefB', 'value3' : 'UserDefA' } }
-{ 'union': 'UserDefAlternate',
- 'discriminator': {},
+{ 'alternate': 'UserDefAlternate',
'data': { 'uda': 'UserDefA', 's': 'str', 'i': 'int' } }
# for testing native lists
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index b55ab8d..313ecf3 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -10,7 +10,7 @@
OrderedDict([('type', 'UserDefUnionBase'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
OrderedDict([('union', 'UserDefFlatUnion'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefA'), ('value2', 'UserDefB'), ('value3', 'UserDefB')]))]),
OrderedDict([('union', 'UserDefFlatUnion2'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefC'), ('value2', 'UserDefB'), ('value3', 'UserDefA')]))]),
- OrderedDict([('union', 'UserDefAlternate'), ('discriminator', OrderedDict()), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
+ OrderedDict([('alternate', 'UserDefAlternate'), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
OrderedDict([('union', 'UserDefNativeListUnion'), ('data', OrderedDict([('integer', ['int']), ('s8', ['int8']), ('s16', ['int16']), ('s32', ['int32']), ('s64', ['int64']), ('u8', ['uint8']), ('u16', ['uint16']), ('u32', ['uint32']), ('u64', ['uint64']), ('number', ['number']), ('boolean', ['bool']), ('string', ['str']), ('sizes', ['size'])]))]),
OrderedDict([('command', 'user_def_cmd'), ('data', OrderedDict())]),
OrderedDict([('command', 'user_def_cmd1'), ('data', OrderedDict([('ud1a', 'UserDefOne')]))]),
diff --git a/tests/qapi-schema/union-invalid-base.err b/tests/qapi-schema/union-invalid-base.err
index 3cc82c0..bedec06 100644
--- a/tests/qapi-schema/union-invalid-base.err
+++ b/tests/qapi-schema/union-invalid-base.err
@@ -1 +1 @@
-tests/qapi-schema/union-invalid-base.json:8: Base 'int' is not a valid type
+tests/qapi-schema/union-invalid-base.json:8: Base 'int' is not a valid base type
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (11 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 15:55 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 14/28] qapi: Better error messages for bad expressions Eric Blake
` (16 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Demonstrate that the qapi generator doesn't deal well with
expressions that aren't up to par. Later patches will improve
the expected results as the generator is made stricter. Only
one of the added tests actually behaves sanely at rejecting
obvious problems.
Note that in some cases, we reject bad QAPI merely because our
pseudo-JSON parser does not yet know how to parse numbers. This
series does not address that, but when a later series adds support
for numeric defaults of integer fields, the testsuite will ensure
that we don't lose the error (and hopefully that the error
message quality is improved).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 6 ++++--
tests/qapi-schema/bad-base.err | 0
tests/qapi-schema/bad-base.exit | 1 +
tests/qapi-schema/bad-base.json | 3 +++
tests/qapi-schema/bad-base.out | 4 ++++
tests/qapi-schema/bad-ident.err | 0
tests/qapi-schema/bad-ident.exit | 1 +
tests/qapi-schema/bad-ident.json | 3 +++
tests/qapi-schema/bad-ident.out | 3 +++
tests/qapi-schema/bad-type-bool.err | 1 +
tests/qapi-schema/bad-type-bool.exit | 1 +
tests/qapi-schema/bad-type-bool.json | 3 +++
tests/qapi-schema/bad-type-bool.out | 0
tests/qapi-schema/bad-type-dict.err | 0
tests/qapi-schema/bad-type-dict.exit | 1 +
tests/qapi-schema/bad-type-dict.json | 2 ++
tests/qapi-schema/bad-type-dict.out | 3 +++
tests/qapi-schema/bad-type-int.err | 1 +
tests/qapi-schema/bad-type-int.exit | 1 +
tests/qapi-schema/bad-type-int.json | 3 +++
tests/qapi-schema/bad-type-int.out | 0
tests/qapi-schema/double-data.err | 1 +
tests/qapi-schema/double-data.exit | 1 +
tests/qapi-schema/double-data.json | 2 ++
tests/qapi-schema/double-data.out | 0
tests/qapi-schema/double-type.err | 0
tests/qapi-schema/double-type.exit | 1 +
tests/qapi-schema/double-type.json | 2 ++
tests/qapi-schema/double-type.out | 3 +++
tests/qapi-schema/event-case.err | 0
tests/qapi-schema/event-case.exit | 1 +
tests/qapi-schema/event-case.json | 2 ++
tests/qapi-schema/event-case.out | 3 +++
tests/qapi-schema/missing-type.err | 0
tests/qapi-schema/missing-type.exit | 1 +
tests/qapi-schema/missing-type.json | 2 ++
tests/qapi-schema/missing-type.out | 3 +++
tests/qapi-schema/unknown-expr-key.err | 0
tests/qapi-schema/unknown-expr-key.exit | 1 +
tests/qapi-schema/unknown-expr-key.json | 2 ++
tests/qapi-schema/unknown-expr-key.out | 3 +++
41 files changed, 63 insertions(+), 2 deletions(-)
create mode 100644 tests/qapi-schema/bad-base.err
create mode 100644 tests/qapi-schema/bad-base.exit
create mode 100644 tests/qapi-schema/bad-base.json
create mode 100644 tests/qapi-schema/bad-base.out
create mode 100644 tests/qapi-schema/bad-ident.err
create mode 100644 tests/qapi-schema/bad-ident.exit
create mode 100644 tests/qapi-schema/bad-ident.json
create mode 100644 tests/qapi-schema/bad-ident.out
create mode 100644 tests/qapi-schema/bad-type-bool.err
create mode 100644 tests/qapi-schema/bad-type-bool.exit
create mode 100644 tests/qapi-schema/bad-type-bool.json
create mode 100644 tests/qapi-schema/bad-type-bool.out
create mode 100644 tests/qapi-schema/bad-type-dict.err
create mode 100644 tests/qapi-schema/bad-type-dict.exit
create mode 100644 tests/qapi-schema/bad-type-dict.json
create mode 100644 tests/qapi-schema/bad-type-dict.out
create mode 100644 tests/qapi-schema/bad-type-int.err
create mode 100644 tests/qapi-schema/bad-type-int.exit
create mode 100644 tests/qapi-schema/bad-type-int.json
create mode 100644 tests/qapi-schema/bad-type-int.out
create mode 100644 tests/qapi-schema/double-data.err
create mode 100644 tests/qapi-schema/double-data.exit
create mode 100644 tests/qapi-schema/double-data.json
create mode 100644 tests/qapi-schema/double-data.out
create mode 100644 tests/qapi-schema/double-type.err
create mode 100644 tests/qapi-schema/double-type.exit
create mode 100644 tests/qapi-schema/double-type.json
create mode 100644 tests/qapi-schema/double-type.out
create mode 100644 tests/qapi-schema/event-case.err
create mode 100644 tests/qapi-schema/event-case.exit
create mode 100644 tests/qapi-schema/event-case.json
create mode 100644 tests/qapi-schema/event-case.out
create mode 100644 tests/qapi-schema/missing-type.err
create mode 100644 tests/qapi-schema/missing-type.exit
create mode 100644 tests/qapi-schema/missing-type.json
create mode 100644 tests/qapi-schema/missing-type.out
create mode 100644 tests/qapi-schema/unknown-expr-key.err
create mode 100644 tests/qapi-schema/unknown-expr-key.exit
create mode 100644 tests/qapi-schema/unknown-expr-key.json
create mode 100644 tests/qapi-schema/unknown-expr-key.out
diff --git a/tests/Makefile b/tests/Makefile
index f8bc2a8..cf9c42b 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -210,7 +210,9 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
comments.json empty.json enum-empty.json enum-missing-data.json \
enum-wrong-data.json enum-int-member.json enum-dict-member.json \
enum-clash-member.json enum-max-member.json enum-union-clash.json \
- funny-char.json indented-expr.json \
+ funny-char.json indented-expr.json missing-type.json bad-ident.json \
+ double-type.json bad-base.json bad-type-bool.json bad-type-int.json \
+ bad-type-dict.json double-data.json unknown-expr-key.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
@@ -229,7 +231,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
include-simple.json include-relpath.json include-format-err.json \
include-non-file.json include-no-file.json include-before-err.json \
include-nested-err.json include-self-cycle.json include-cycle.json \
- include-repetition.json event-nest-struct.json)
+ include-repetition.json event-nest-struct.json event-case.json)
GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
tests/test-qmp-commands.h tests/test-qapi-event.h
diff --git a/tests/qapi-schema/bad-base.err b/tests/qapi-schema/bad-base.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-base.exit b/tests/qapi-schema/bad-base.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-base.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/bad-base.json b/tests/qapi-schema/bad-base.json
new file mode 100644
index 0000000..de964a0
--- /dev/null
+++ b/tests/qapi-schema/bad-base.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject a base that is not a struct
+{ 'union': 'Union', 'data': { 'a': 'int', 'b': 'str' } }
+{ 'type': 'MyType', 'base': 'Union', 'data': { 'c': 'int' } }
diff --git a/tests/qapi-schema/bad-base.out b/tests/qapi-schema/bad-base.out
new file mode 100644
index 0000000..91d12fc
--- /dev/null
+++ b/tests/qapi-schema/bad-base.out
@@ -0,0 +1,4 @@
+[OrderedDict([('union', 'Union'), ('data', OrderedDict([('a', 'int'), ('b', 'str')]))]),
+ OrderedDict([('type', 'MyType'), ('base', 'Union'), ('data', OrderedDict([('c', 'int')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'MyType'), ('base', 'Union'), ('data', OrderedDict([('c', 'int')]))])]
diff --git a/tests/qapi-schema/bad-ident.err b/tests/qapi-schema/bad-ident.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-ident.exit b/tests/qapi-schema/bad-ident.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-ident.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/bad-ident.json b/tests/qapi-schema/bad-ident.json
new file mode 100644
index 0000000..66333a7
--- /dev/null
+++ b/tests/qapi-schema/bad-ident.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject creating a type name with bad name
+{ 'type': '*oops', 'data': { 'i': 'int' } }
+
diff --git a/tests/qapi-schema/bad-ident.out b/tests/qapi-schema/bad-ident.out
new file mode 100644
index 0000000..165e346
--- /dev/null
+++ b/tests/qapi-schema/bad-ident.out
@@ -0,0 +1,3 @@
+[OrderedDict([('type', '*oops'), ('data', OrderedDict([('i', 'int')]))])]
+[]
+[OrderedDict([('type', '*oops'), ('data', OrderedDict([('i', 'int')]))])]
diff --git a/tests/qapi-schema/bad-type-bool.err b/tests/qapi-schema/bad-type-bool.err
new file mode 100644
index 0000000..badb7c2
--- /dev/null
+++ b/tests/qapi-schema/bad-type-bool.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-type-bool.json:3:11: Stray "t"
diff --git a/tests/qapi-schema/bad-type-bool.exit b/tests/qapi-schema/bad-type-bool.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/bad-type-bool.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/bad-type-bool.json b/tests/qapi-schema/bad-type-bool.json
new file mode 100644
index 0000000..22d6369
--- /dev/null
+++ b/tests/qapi-schema/bad-type-bool.json
@@ -0,0 +1,3 @@
+# we reject an expression with a metatype that is not a string
+# FIXME: once the parser understands bool inputs, improve the error message
+{ 'type': true, 'data': { } }
diff --git a/tests/qapi-schema/bad-type-bool.out b/tests/qapi-schema/bad-type-bool.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-type-dict.err b/tests/qapi-schema/bad-type-dict.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-type-dict.exit b/tests/qapi-schema/bad-type-dict.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/bad-type-dict.json b/tests/qapi-schema/bad-type-dict.json
new file mode 100644
index 0000000..3c392a7
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with a metatype that is not a string
+{ 'command': { } }
diff --git a/tests/qapi-schema/bad-type-dict.out b/tests/qapi-schema/bad-type-dict.out
new file mode 100644
index 0000000..c62f1ed
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/bad-type-int.err b/tests/qapi-schema/bad-type-int.err
new file mode 100644
index 0000000..9808550
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-type-int.json:3:11: Stray "1"
diff --git a/tests/qapi-schema/bad-type-int.exit b/tests/qapi-schema/bad-type-int.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/bad-type-int.json b/tests/qapi-schema/bad-type-int.json
new file mode 100644
index 0000000..398879d
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.json
@@ -0,0 +1,3 @@
+# we reject an expression with a metatype that is not a string
+# FIXME: once the parser understands integer inputs, improve the error message
+{ 'type': 1, 'data': { } }
diff --git a/tests/qapi-schema/bad-type-int.out b/tests/qapi-schema/bad-type-int.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-data.err b/tests/qapi-schema/double-data.err
new file mode 100644
index 0000000..6f1a67b
--- /dev/null
+++ b/tests/qapi-schema/double-data.err
@@ -0,0 +1 @@
+tests/qapi-schema/double-data.json:2:39: Duplicate key "data"
diff --git a/tests/qapi-schema/double-data.exit b/tests/qapi-schema/double-data.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/double-data.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/double-data.json b/tests/qapi-schema/double-data.json
new file mode 100644
index 0000000..a94b7df
--- /dev/null
+++ b/tests/qapi-schema/double-data.json
@@ -0,0 +1,2 @@
+# we reject an expression with duplicate top-level keys
+{ 'type': 'bar', 'data': { }, 'data': { 'string': 'str'} }
diff --git a/tests/qapi-schema/double-data.out b/tests/qapi-schema/double-data.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-type.exit b/tests/qapi-schema/double-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/double-type.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/double-type.json b/tests/qapi-schema/double-type.json
new file mode 100644
index 0000000..6ca96b9
--- /dev/null
+++ b/tests/qapi-schema/double-type.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with ambiguous metatype
+{ 'command': 'foo', 'type': 'bar', 'data': { } }
diff --git a/tests/qapi-schema/double-type.out b/tests/qapi-schema/double-type.out
new file mode 100644
index 0000000..3e244f5
--- /dev/null
+++ b/tests/qapi-schema/double-type.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
+[]
+[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
diff --git a/tests/qapi-schema/event-case.err b/tests/qapi-schema/event-case.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/event-case.exit b/tests/qapi-schema/event-case.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/event-case.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/event-case.json b/tests/qapi-schema/event-case.json
new file mode 100644
index 0000000..52dfc3a
--- /dev/null
+++ b/tests/qapi-schema/event-case.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an event name that is not all caps
+{ 'event': 'oops' }
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
new file mode 100644
index 0000000..3764bc7
--- /dev/null
+++ b/tests/qapi-schema/event-case.out
@@ -0,0 +1,3 @@
+[OrderedDict([('event', 'oops')])]
+[]
+[]
diff --git a/tests/qapi-schema/missing-type.err b/tests/qapi-schema/missing-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/missing-type.exit b/tests/qapi-schema/missing-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/missing-type.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/missing-type.json b/tests/qapi-schema/missing-type.json
new file mode 100644
index 0000000..1696f5c
--- /dev/null
+++ b/tests/qapi-schema/missing-type.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with missing metatype
+{ 'data': { } }
diff --git a/tests/qapi-schema/missing-type.out b/tests/qapi-schema/missing-type.out
new file mode 100644
index 0000000..67fd4fa
--- /dev/null
+++ b/tests/qapi-schema/missing-type.out
@@ -0,0 +1,3 @@
+[OrderedDict([('data', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/unknown-expr-key.exit b/tests/qapi-schema/unknown-expr-key.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
new file mode 100644
index 0000000..1e9282d
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with unknown top-level keys
+{ 'type': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
diff --git a/tests/qapi-schema/unknown-expr-key.out b/tests/qapi-schema/unknown-expr-key.out
new file mode 100644
index 0000000..c93f020
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.out
@@ -0,0 +1,3 @@
+[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
+[]
+[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 14/28] qapi: Better error messages for bad expressions
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (12 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 16:27 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 15/28] qapi: Add tests of redefined expressions Eric Blake
` (15 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
The previous commit demonstrated that the generator overlooked some
fairly basic broken expressions:
- missing metataype
- metatype key has a non-string value
- unknown key in relation to the metatype
- conflicting metatype (this patch treats the second metatype as an
unknown key of the first key visited, which is not necessarily the
first key the user typed)
Add check_keys to cover these situations, and update testcases to
match. A couple other tests (enum-missing-data, indented-expr) had
to change since the validation added here occurs so early.
While valid .json files won't trigger any of these cases, we might
as well be nicer to developers that make a typo while trying to add
new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 45 +++++++++++++++++++++++++++------
tests/qapi-schema/alternate-base.err | 2 +-
tests/qapi-schema/bad-type-dict.err | 1 +
tests/qapi-schema/bad-type-dict.exit | 2 +-
tests/qapi-schema/bad-type-dict.json | 2 +-
tests/qapi-schema/bad-type-dict.out | 3 ---
tests/qapi-schema/double-type.err | 1 +
tests/qapi-schema/double-type.exit | 2 +-
tests/qapi-schema/double-type.json | 2 +-
tests/qapi-schema/double-type.out | 3 ---
tests/qapi-schema/enum-missing-data.err | 2 +-
tests/qapi-schema/indented-expr.json | 4 +--
tests/qapi-schema/indented-expr.out | 2 +-
tests/qapi-schema/missing-type.err | 1 +
tests/qapi-schema/missing-type.exit | 2 +-
tests/qapi-schema/missing-type.json | 2 +-
tests/qapi-schema/missing-type.out | 3 ---
tests/qapi-schema/unknown-expr-key.err | 1 +
tests/qapi-schema/unknown-expr-key.exit | 2 +-
tests/qapi-schema/unknown-expr-key.json | 2 +-
tests/qapi-schema/unknown-expr-key.out | 3 ---
21 files changed, 54 insertions(+), 33 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 018ec45..90eb3bc 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -341,12 +341,6 @@ def check_alternate(expr, expr_info):
values = { 'MAX': '(automatic)' }
types_seen = {}
- base = expr.get('base')
- if base:
- raise QAPIExprError(expr_info,
- "Alternate '%s' must not have a base"
- % name)
-
# Check every branch
for (key, value) in members.items():
# Check for conflicts in the generated enum
@@ -408,6 +402,26 @@ def check_exprs(schema):
elif expr.has_key('event'):
check_event(expr, info)
+def check_keys(expr_elem, meta, required, optional=[]):
+ expr = expr_elem['expr']
+ info = expr_elem['info']
+ name = expr[meta]
+ if not isinstance(name, str):
+ raise QAPIExprError(info,
+ "%s key must have a string value" % meta)
+ required = required + [ meta ]
+ for (key, value) in expr.items():
+ if not key in required and not key in optional:
+ raise QAPIExprError(info,
+ "%s '%s' has unknown key '%s'"
+ % (meta, name, key))
+ for key in required:
+ if not expr.has_key(key):
+ raise QAPIExprError(info,
+ "%s '%s' is missing key '%s'"
+ % (meta, name, key))
+
+
def parse_schema(input_file):
# First pass: read entire file into memory
try:
@@ -419,15 +433,30 @@ def parse_schema(input_file):
exprs = []
try:
- # Next pass: learn the types.
+ # Next pass: learn the types and check for valid expression keys. At
+ # this point, top-level 'include' has already been flattened.
for expr_elem in schema.exprs:
expr = expr_elem['expr']
if expr.has_key('enum'):
- add_enum(expr['enum'], expr.get('data'))
+ check_keys(expr_elem, 'enum', ['data'])
+ add_enum(expr['enum'], expr['data'])
elif expr.has_key('union'):
+ check_keys(expr_elem, 'union', ['data'],
+ ['base', 'discriminator'])
add_union(expr)
+ elif expr.has_key('alternate'):
+ check_keys(expr_elem, 'alternate', ['data'])
elif expr.has_key('type'):
+ check_keys(expr_elem, 'type', ['data'], ['base'])
add_struct(expr)
+ elif expr.has_key('command'):
+ check_keys(expr_elem, 'command', [],
+ ['data', 'returns', 'gen', 'success-response'])
+ elif expr.has_key('event'):
+ check_keys(expr_elem, 'event', [], ['data'])
+ else:
+ raise QAPIExprError(expr_elem['info'],
+ "Expression is missing metatype")
exprs.append(expr)
# Try again for hidden UnionKind enum
diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
index 4a2566e..f9e30e7 100644
--- a/tests/qapi-schema/alternate-base.err
+++ b/tests/qapi-schema/alternate-base.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-base.json:4: Alternate 'Alt' must not have a base
+tests/qapi-schema/alternate-base.json:4: alternate 'Alt' has unknown key 'base'
diff --git a/tests/qapi-schema/bad-type-dict.err b/tests/qapi-schema/bad-type-dict.err
index e69de29..b7c73cc 100644
--- a/tests/qapi-schema/bad-type-dict.err
+++ b/tests/qapi-schema/bad-type-dict.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-type-dict.json:2: command key must have a string value
diff --git a/tests/qapi-schema/bad-type-dict.exit b/tests/qapi-schema/bad-type-dict.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/bad-type-dict.exit
+++ b/tests/qapi-schema/bad-type-dict.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/bad-type-dict.json b/tests/qapi-schema/bad-type-dict.json
index 3c392a7..2a91b24 100644
--- a/tests/qapi-schema/bad-type-dict.json
+++ b/tests/qapi-schema/bad-type-dict.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with a metatype that is not a string
+# we reject an expression with a metatype that is not a string
{ 'command': { } }
diff --git a/tests/qapi-schema/bad-type-dict.out b/tests/qapi-schema/bad-type-dict.out
index c62f1ed..e69de29 100644
--- a/tests/qapi-schema/bad-type-dict.out
+++ b/tests/qapi-schema/bad-type-dict.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', OrderedDict())])]
-[]
-[]
diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
index e69de29..394127b 100644
--- a/tests/qapi-schema/double-type.err
+++ b/tests/qapi-schema/double-type.err
@@ -0,0 +1 @@
+tests/qapi-schema/double-type.json:2: type 'bar' has unknown key 'command'
diff --git a/tests/qapi-schema/double-type.exit b/tests/qapi-schema/double-type.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/double-type.exit
+++ b/tests/qapi-schema/double-type.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/double-type.json b/tests/qapi-schema/double-type.json
index 6ca96b9..471623a 100644
--- a/tests/qapi-schema/double-type.json
+++ b/tests/qapi-schema/double-type.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with ambiguous metatype
+# we reject an expression with ambiguous metatype
{ 'command': 'foo', 'type': 'bar', 'data': { } }
diff --git a/tests/qapi-schema/double-type.out b/tests/qapi-schema/double-type.out
index 3e244f5..e69de29 100644
--- a/tests/qapi-schema/double-type.out
+++ b/tests/qapi-schema/double-type.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
-[]
-[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
index b8ccae0..62f4a7f 100644
--- a/tests/qapi-schema/enum-missing-data.err
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -1 +1 @@
-tests/qapi-schema/enum-missing-data.json:2: Enum 'MyEnum' requires an array for 'data'
+tests/qapi-schema/enum-missing-data.json:2: enum 'MyEnum' is missing key 'data'
diff --git a/tests/qapi-schema/indented-expr.json b/tests/qapi-schema/indented-expr.json
index d80af60..7115d31 100644
--- a/tests/qapi-schema/indented-expr.json
+++ b/tests/qapi-schema/indented-expr.json
@@ -1,2 +1,2 @@
-{ 'id' : 'eins' }
- { 'id' : 'zwei' }
+{ 'command' : 'eins' }
+ { 'command' : 'zwei' }
diff --git a/tests/qapi-schema/indented-expr.out b/tests/qapi-schema/indented-expr.out
index 98af89a..b5ce915 100644
--- a/tests/qapi-schema/indented-expr.out
+++ b/tests/qapi-schema/indented-expr.out
@@ -1,3 +1,3 @@
-[OrderedDict([('id', 'eins')]), OrderedDict([('id', 'zwei')])]
+[OrderedDict([('command', 'eins')]), OrderedDict([('command', 'zwei')])]
[]
[]
diff --git a/tests/qapi-schema/missing-type.err b/tests/qapi-schema/missing-type.err
index e69de29..b3e7b14 100644
--- a/tests/qapi-schema/missing-type.err
+++ b/tests/qapi-schema/missing-type.err
@@ -0,0 +1 @@
+tests/qapi-schema/missing-type.json:2: Expression is missing metatype
diff --git a/tests/qapi-schema/missing-type.exit b/tests/qapi-schema/missing-type.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/missing-type.exit
+++ b/tests/qapi-schema/missing-type.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/missing-type.json b/tests/qapi-schema/missing-type.json
index 1696f5c..ff5349d 100644
--- a/tests/qapi-schema/missing-type.json
+++ b/tests/qapi-schema/missing-type.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with missing metatype
+# we reject an expression with missing metatype
{ 'data': { } }
diff --git a/tests/qapi-schema/missing-type.out b/tests/qapi-schema/missing-type.out
index 67fd4fa..e69de29 100644
--- a/tests/qapi-schema/missing-type.out
+++ b/tests/qapi-schema/missing-type.out
@@ -1,3 +0,0 @@
-[OrderedDict([('data', OrderedDict())])]
-[]
-[]
diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
index e69de29..3b35508 100644
--- a/tests/qapi-schema/unknown-expr-key.err
+++ b/tests/qapi-schema/unknown-expr-key.err
@@ -0,0 +1 @@
+tests/qapi-schema/unknown-expr-key.json:2: type 'bar' has unknown key 'bogus'
diff --git a/tests/qapi-schema/unknown-expr-key.exit b/tests/qapi-schema/unknown-expr-key.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/unknown-expr-key.exit
+++ b/tests/qapi-schema/unknown-expr-key.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
index 1e9282d..ba7bdf3 100644
--- a/tests/qapi-schema/unknown-expr-key.json
+++ b/tests/qapi-schema/unknown-expr-key.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with unknown top-level keys
+# we reject an expression with unknown top-level keys
{ 'type': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
diff --git a/tests/qapi-schema/unknown-expr-key.out b/tests/qapi-schema/unknown-expr-key.out
index c93f020..e69de29 100644
--- a/tests/qapi-schema/unknown-expr-key.out
+++ b/tests/qapi-schema/unknown-expr-key.out
@@ -1,3 +0,0 @@
-[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
-[]
-[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 15/28] qapi: Add tests of redefined expressions
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (13 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 14/28] qapi: Better error messages for bad expressions Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 17:05 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions Eric Blake
` (14 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Demonstrate that the qapi generator doesn't deal very well with
redefined expressions. At the parse level, they are silently
accepted; and while the testsuite just stops at parsing, I've
further tested that many of them cause generator crashes or
invalid C code if they were appended to qapi-schema-test.json.
A later patch will tighten things up and adjust the testsuite
to match.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 2 ++
tests/qapi-schema/command-int.err | 0
tests/qapi-schema/command-int.exit | 1 +
tests/qapi-schema/command-int.json | 3 +++
tests/qapi-schema/command-int.out | 3 +++
tests/qapi-schema/event-max.err | 0
tests/qapi-schema/event-max.exit | 1 +
tests/qapi-schema/event-max.json | 2 ++
tests/qapi-schema/event-max.out | 3 +++
tests/qapi-schema/redefined-builtin.err | 0
tests/qapi-schema/redefined-builtin.exit | 1 +
tests/qapi-schema/redefined-builtin.json | 2 ++
tests/qapi-schema/redefined-builtin.out | 3 +++
tests/qapi-schema/redefined-command.err | 0
tests/qapi-schema/redefined-command.exit | 1 +
tests/qapi-schema/redefined-command.json | 3 +++
tests/qapi-schema/redefined-command.out | 4 ++++
tests/qapi-schema/redefined-event.err | 0
tests/qapi-schema/redefined-event.exit | 1 +
tests/qapi-schema/redefined-event.json | 3 +++
tests/qapi-schema/redefined-event.out | 4 ++++
tests/qapi-schema/redefined-type.err | 0
tests/qapi-schema/redefined-type.exit | 1 +
tests/qapi-schema/redefined-type.json | 3 +++
tests/qapi-schema/redefined-type.out | 4 ++++
25 files changed, 45 insertions(+)
create mode 100644 tests/qapi-schema/command-int.err
create mode 100644 tests/qapi-schema/command-int.exit
create mode 100644 tests/qapi-schema/command-int.json
create mode 100644 tests/qapi-schema/command-int.out
create mode 100644 tests/qapi-schema/event-max.err
create mode 100644 tests/qapi-schema/event-max.exit
create mode 100644 tests/qapi-schema/event-max.json
create mode 100644 tests/qapi-schema/event-max.out
create mode 100644 tests/qapi-schema/redefined-builtin.err
create mode 100644 tests/qapi-schema/redefined-builtin.exit
create mode 100644 tests/qapi-schema/redefined-builtin.json
create mode 100644 tests/qapi-schema/redefined-builtin.out
create mode 100644 tests/qapi-schema/redefined-command.err
create mode 100644 tests/qapi-schema/redefined-command.exit
create mode 100644 tests/qapi-schema/redefined-command.json
create mode 100644 tests/qapi-schema/redefined-command.out
create mode 100644 tests/qapi-schema/redefined-event.err
create mode 100644 tests/qapi-schema/redefined-event.exit
create mode 100644 tests/qapi-schema/redefined-event.json
create mode 100644 tests/qapi-schema/redefined-event.out
create mode 100644 tests/qapi-schema/redefined-type.err
create mode 100644 tests/qapi-schema/redefined-type.exit
create mode 100644 tests/qapi-schema/redefined-type.json
create mode 100644 tests/qapi-schema/redefined-type.out
diff --git a/tests/Makefile b/tests/Makefile
index cf9c42b..7079e2a 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -213,6 +213,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
funny-char.json indented-expr.json missing-type.json bad-ident.json \
double-type.json bad-base.json bad-type-bool.json bad-type-int.json \
bad-type-dict.json double-data.json unknown-expr-key.json \
+ redefined-type.json redefined-command.json redefined-builtin.json \
+ redefined-event.json command-int.json event-max.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/command-int.err b/tests/qapi-schema/command-int.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/command-int.exit b/tests/qapi-schema/command-int.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/command-int.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/command-int.json b/tests/qapi-schema/command-int.json
new file mode 100644
index 0000000..fcbb643
--- /dev/null
+++ b/tests/qapi-schema/command-int.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject collisions between commands and types
+{ 'command': 'int', 'data': { 'character': 'str' },
+ 'returns': { 'value': 'int' } }
diff --git a/tests/qapi-schema/command-int.out b/tests/qapi-schema/command-int.out
new file mode 100644
index 0000000..d8e1854
--- /dev/null
+++ b/tests/qapi-schema/command-int.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'int'), ('data', OrderedDict([('character', 'str')])), ('returns', OrderedDict([('value', 'int')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/event-max.err b/tests/qapi-schema/event-max.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/event-max.exit b/tests/qapi-schema/event-max.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/event-max.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/event-max.json b/tests/qapi-schema/event-max.json
new file mode 100644
index 0000000..997c61c
--- /dev/null
+++ b/tests/qapi-schema/event-max.json
@@ -0,0 +1,2 @@
+# FIXME: an event named 'MAX' would conflict with implicit C enum
+{ 'event': 'MAX' }
diff --git a/tests/qapi-schema/event-max.out b/tests/qapi-schema/event-max.out
new file mode 100644
index 0000000..010c42b
--- /dev/null
+++ b/tests/qapi-schema/event-max.out
@@ -0,0 +1,3 @@
+[OrderedDict([('event', 'MAX')])]
+[]
+[]
diff --git a/tests/qapi-schema/redefined-builtin.err b/tests/qapi-schema/redefined-builtin.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-builtin.exit b/tests/qapi-schema/redefined-builtin.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-builtin.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-builtin.json b/tests/qapi-schema/redefined-builtin.json
new file mode 100644
index 0000000..a10050d
--- /dev/null
+++ b/tests/qapi-schema/redefined-builtin.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject types that duplicate builtin names
+{ 'type': 'size', 'data': { 'myint': 'size' } }
diff --git a/tests/qapi-schema/redefined-builtin.out b/tests/qapi-schema/redefined-builtin.out
new file mode 100644
index 0000000..b0a9aea
--- /dev/null
+++ b/tests/qapi-schema/redefined-builtin.out
@@ -0,0 +1,3 @@
+[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
+[]
+[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
diff --git a/tests/qapi-schema/redefined-command.err b/tests/qapi-schema/redefined-command.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-command.exit b/tests/qapi-schema/redefined-command.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-command.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-command.json b/tests/qapi-schema/redefined-command.json
new file mode 100644
index 0000000..d8c9975
--- /dev/null
+++ b/tests/qapi-schema/redefined-command.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject commands defined more than once
+{ 'command': 'foo', 'data': { 'one': 'str' } }
+{ 'command': 'foo', 'data': { '*two': 'str' } }
diff --git a/tests/qapi-schema/redefined-command.out b/tests/qapi-schema/redefined-command.out
new file mode 100644
index 0000000..44ddba6
--- /dev/null
+++ b/tests/qapi-schema/redefined-command.out
@@ -0,0 +1,4 @@
+[OrderedDict([('command', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
+ OrderedDict([('command', 'foo'), ('data', OrderedDict([('*two', 'str')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/redefined-event.err b/tests/qapi-schema/redefined-event.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-event.exit b/tests/qapi-schema/redefined-event.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-event.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-event.json b/tests/qapi-schema/redefined-event.json
new file mode 100644
index 0000000..152cce7
--- /dev/null
+++ b/tests/qapi-schema/redefined-event.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject duplicate events
+{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
+{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
diff --git a/tests/qapi-schema/redefined-event.out b/tests/qapi-schema/redefined-event.out
new file mode 100644
index 0000000..261b183
--- /dev/null
+++ b/tests/qapi-schema/redefined-event.out
@@ -0,0 +1,4 @@
+[OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))]),
+ OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/redefined-type.err b/tests/qapi-schema/redefined-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-type.exit b/tests/qapi-schema/redefined-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-type.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-type.json b/tests/qapi-schema/redefined-type.json
new file mode 100644
index 0000000..7972194
--- /dev/null
+++ b/tests/qapi-schema/redefined-type.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject types defined more than once
+{ 'type': 'foo', 'data': { 'one': 'str' } }
+{ 'enum': 'foo', 'data': [ 'two' ] }
diff --git a/tests/qapi-schema/redefined-type.out b/tests/qapi-schema/redefined-type.out
new file mode 100644
index 0000000..b509e57
--- /dev/null
+++ b/tests/qapi-schema/redefined-type.out
@@ -0,0 +1,4 @@
+[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
+ OrderedDict([('enum', 'foo'), ('data', ['two'])])]
+[{'enum_name': 'foo', 'enum_values': ['two']}]
+[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))])]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (14 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 15/28] qapi: Add tests of redefined expressions Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 17:21 ` Markus Armbruster
2015-03-27 7:52 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json Eric Blake
` (13 subsequent siblings)
29 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
The previous commit demonstrated that the generator overlooked
duplicate expressions:
- a complex type or command reusing a built-in type name
- redeclaration of a type name, whether by the same or different
metatype
- redeclaration of a command or event
- collision of a type with implicit 'Kind' enum for a union
- collision with an implicit MAX enum constant, or with various
case spellings of events
Since the c_type() function in the generator treats all names
as being in the same namespace, this patch adds a global array
to track all known names and their source, to prevent collisions
before it can cause further problems. While valid .json files
won't trigger any of these cases, we might as well be nicer to
developers that make a typo while trying to add new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 68 +++++++++++++++++++++++++-------
tests/qapi-schema/bad-type-dict.err | 2 +-
tests/qapi-schema/command-int.err | 1 +
tests/qapi-schema/command-int.exit | 2 +-
tests/qapi-schema/command-int.json | 2 +-
tests/qapi-schema/command-int.out | 3 --
tests/qapi-schema/enum-union-clash.err | 1 +
tests/qapi-schema/enum-union-clash.exit | 2 +-
tests/qapi-schema/enum-union-clash.json | 2 +-
tests/qapi-schema/enum-union-clash.out | 5 ---
tests/qapi-schema/event-case.err | 1 +
tests/qapi-schema/event-case.exit | 2 +-
tests/qapi-schema/event-case.json | 2 +-
tests/qapi-schema/event-case.out | 3 --
tests/qapi-schema/event-max.err | 1 +
tests/qapi-schema/event-max.exit | 2 +-
tests/qapi-schema/event-max.json | 2 +-
tests/qapi-schema/event-max.out | 3 --
tests/qapi-schema/redefined-builtin.err | 1 +
tests/qapi-schema/redefined-builtin.exit | 2 +-
tests/qapi-schema/redefined-builtin.json | 2 +-
tests/qapi-schema/redefined-builtin.out | 3 --
tests/qapi-schema/redefined-command.err | 1 +
tests/qapi-schema/redefined-command.exit | 2 +-
tests/qapi-schema/redefined-command.json | 2 +-
tests/qapi-schema/redefined-command.out | 4 --
tests/qapi-schema/redefined-event.err | 1 +
tests/qapi-schema/redefined-event.exit | 2 +-
tests/qapi-schema/redefined-event.json | 2 +-
tests/qapi-schema/redefined-event.out | 4 --
tests/qapi-schema/redefined-type.err | 1 +
tests/qapi-schema/redefined-type.exit | 2 +-
tests/qapi-schema/redefined-type.json | 2 +-
tests/qapi-schema/redefined-type.out | 4 --
34 files changed, 78 insertions(+), 61 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 90eb3bc..5d0dc91 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -32,6 +32,12 @@ builtin_types = {
'size': 'QTYPE_QINT',
}
+enum_types = []
+struct_types = []
+union_types = []
+events = []
+all_names = {}
+
def error_path(parent):
res = ""
while parent:
@@ -256,7 +262,17 @@ def discriminator_find_enum_define(expr):
return find_enum(discriminator_type)
def check_event(expr, expr_info):
+ global events
+ name = expr['event']
params = expr.get('data')
+
+ if name == 'MAX':
+ raise QAPIExprError(expr_info, "Event name 'MAX' cannot be created")
+ if name != name.upper():
+ raise QAPIExprError(expr_info, "Event name '%s' should be upper case"
+ % name)
+ events.append(name)
+
if params:
for argname, argentry, optional, structured in parse_args(params):
if structured:
@@ -408,7 +424,7 @@ def check_keys(expr_elem, meta, required, optional=[]):
name = expr[meta]
if not isinstance(name, str):
raise QAPIExprError(info,
- "%s key must have a string value" % meta)
+ "'%s' key must have a string value" % meta)
required = required + [ meta ]
for (key, value) in expr.items():
if not key in required and not key in optional:
@@ -423,6 +439,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
def parse_schema(input_file):
+ global all_names
+ exprs = []
+
# First pass: read entire file into memory
try:
schema = QAPISchema(open(input_file, "r"))
@@ -430,30 +449,34 @@ def parse_schema(input_file):
print >>sys.stderr, e
exit(1)
- exprs = []
-
try:
# Next pass: learn the types and check for valid expression keys. At
# this point, top-level 'include' has already been flattened.
+ for builtin in builtin_types.keys():
+ all_names[builtin] = 'built-in'
for expr_elem in schema.exprs:
expr = expr_elem['expr']
+ info = expr_elem['info']
if expr.has_key('enum'):
check_keys(expr_elem, 'enum', ['data'])
- add_enum(expr['enum'], expr['data'])
+ add_enum(expr['enum'], info, expr['data'])
elif expr.has_key('union'):
check_keys(expr_elem, 'union', ['data'],
['base', 'discriminator'])
- add_union(expr)
+ add_union(expr, info)
elif expr.has_key('alternate'):
check_keys(expr_elem, 'alternate', ['data'])
+ add_name(expr['alternate'], info, 'alternate')
elif expr.has_key('type'):
check_keys(expr_elem, 'type', ['data'], ['base'])
- add_struct(expr)
+ add_struct(expr, info)
elif expr.has_key('command'):
check_keys(expr_elem, 'command', [],
['data', 'returns', 'gen', 'success-response'])
+ add_name(expr['command'], info, 'command')
elif expr.has_key('event'):
check_keys(expr_elem, 'event', [], ['data'])
+ add_name(expr['event'], info, 'event')
else:
raise QAPIExprError(expr_elem['info'],
"Expression is missing metatype")
@@ -464,9 +487,11 @@ def parse_schema(input_file):
expr = expr_elem['expr']
if expr.has_key('union'):
if not discriminator_find_enum_define(expr):
- add_enum('%sKind' % expr['union'])
+ add_enum('%sKind' % expr['union'], expr_elem['info'],
+ implicit=True)
elif expr.has_key('alternate'):
- add_enum('%sKind' % expr['alternate'])
+ add_enum('%sKind' % expr['alternate'], expr_elem['info'],
+ implicit=True)
# Final pass - validate that exprs make sense
check_exprs(schema)
@@ -560,12 +585,22 @@ def type_name(name):
return c_list_type(name[0])
return name
-enum_types = []
-struct_types = []
-union_types = []
+def add_name(name, info, meta, implicit = False):
+ global all_names
+ if name in all_names:
+ raise QAPIExprError(info,
+ "%s '%s' is already defined"
+ %(all_names[name], name))
+ if not implicit and name[-4:] == 'Kind':
+ raise QAPIExprError(info,
+ "%s '%s' should not end in 'Kind'"
+ %(meta, name))
+ all_names[name] = meta
-def add_struct(definition):
+def add_struct(definition, info):
global struct_types
+ name = definition['type']
+ add_name(name, info, 'struct')
struct_types.append(definition)
def find_struct(name):
@@ -575,8 +610,10 @@ def find_struct(name):
return struct
return None
-def add_union(definition):
+def add_union(definition, info):
global union_types
+ name = definition['union']
+ add_name(name, info, 'union')
union_types.append(definition)
def find_union(name):
@@ -586,8 +623,9 @@ def find_union(name):
return union
return None
-def add_enum(name, enum_values = None):
+def add_enum(name, info, enum_values = None, implicit = False):
global enum_types
+ add_name(name, info, 'enum', implicit)
enum_types.append({"enum_name": name, "enum_values": enum_values})
def find_enum(name):
@@ -629,7 +667,7 @@ def c_type(name, is_param=False):
return name
elif name == None or len(name) == 0:
return 'void'
- elif name == name.upper():
+ elif name in events:
return '%sEvent *%s' % (camel_case(name), eatspace)
else:
return '%s *%s' % (name, eatspace)
diff --git a/tests/qapi-schema/bad-type-dict.err b/tests/qapi-schema/bad-type-dict.err
index b7c73cc..0b2a2ae 100644
--- a/tests/qapi-schema/bad-type-dict.err
+++ b/tests/qapi-schema/bad-type-dict.err
@@ -1 +1 @@
-tests/qapi-schema/bad-type-dict.json:2: command key must have a string value
+tests/qapi-schema/bad-type-dict.json:2: 'command' key must have a string value
diff --git a/tests/qapi-schema/command-int.err b/tests/qapi-schema/command-int.err
index e69de29..0f93006 100644
--- a/tests/qapi-schema/command-int.err
+++ b/tests/qapi-schema/command-int.err
@@ -0,0 +1 @@
+tests/qapi-schema/command-int.json:2: built-in 'int' is already defined
diff --git a/tests/qapi-schema/command-int.exit b/tests/qapi-schema/command-int.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/command-int.exit
+++ b/tests/qapi-schema/command-int.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/command-int.json b/tests/qapi-schema/command-int.json
index fcbb643..c90d408 100644
--- a/tests/qapi-schema/command-int.json
+++ b/tests/qapi-schema/command-int.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject collisions between commands and types
+# we reject collisions between commands and types
{ 'command': 'int', 'data': { 'character': 'str' },
'returns': { 'value': 'int' } }
diff --git a/tests/qapi-schema/command-int.out b/tests/qapi-schema/command-int.out
index d8e1854..e69de29 100644
--- a/tests/qapi-schema/command-int.out
+++ b/tests/qapi-schema/command-int.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'int'), ('data', OrderedDict([('character', 'str')])), ('returns', OrderedDict([('value', 'int')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/enum-union-clash.err b/tests/qapi-schema/enum-union-clash.err
index e69de29..c04e1a8 100644
--- a/tests/qapi-schema/enum-union-clash.err
+++ b/tests/qapi-schema/enum-union-clash.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-union-clash.json:2: enum 'UnionKind' should not end in 'Kind'
diff --git a/tests/qapi-schema/enum-union-clash.exit b/tests/qapi-schema/enum-union-clash.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-union-clash.exit
+++ b/tests/qapi-schema/enum-union-clash.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-union-clash.json b/tests/qapi-schema/enum-union-clash.json
index 714ff6d..593282b 100644
--- a/tests/qapi-schema/enum-union-clash.json
+++ b/tests/qapi-schema/enum-union-clash.json
@@ -1,4 +1,4 @@
-# FIXME: we should reject types that would conflict with implicit union enum
+# we reject types that would conflict with implicit union enum
{ 'enum': 'UnionKind', 'data': [ 'oops' ] }
{ 'union': 'Union',
'data': { 'a': 'int' } }
diff --git a/tests/qapi-schema/enum-union-clash.out b/tests/qapi-schema/enum-union-clash.out
index d45f5e8..e69de29 100644
--- a/tests/qapi-schema/enum-union-clash.out
+++ b/tests/qapi-schema/enum-union-clash.out
@@ -1,5 +0,0 @@
-[OrderedDict([('enum', 'UnionKind'), ('data', ['oops'])]),
- OrderedDict([('union', 'Union'), ('data', OrderedDict([('a', 'int')]))])]
-[{'enum_name': 'UnionKind', 'enum_values': ['oops']},
- {'enum_name': 'UnionKind', 'enum_values': None}]
-[]
diff --git a/tests/qapi-schema/event-case.err b/tests/qapi-schema/event-case.err
index e69de29..26857be 100644
--- a/tests/qapi-schema/event-case.err
+++ b/tests/qapi-schema/event-case.err
@@ -0,0 +1 @@
+tests/qapi-schema/event-case.json:2: Event name 'oops' should be upper case
diff --git a/tests/qapi-schema/event-case.exit b/tests/qapi-schema/event-case.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/event-case.exit
+++ b/tests/qapi-schema/event-case.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/event-case.json b/tests/qapi-schema/event-case.json
index 52dfc3a..0dfa5ae 100644
--- a/tests/qapi-schema/event-case.json
+++ b/tests/qapi-schema/event-case.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an event name that is not all caps
+# we reject an event name that is not all caps
{ 'event': 'oops' }
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index 3764bc7..e69de29 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1,3 +0,0 @@
-[OrderedDict([('event', 'oops')])]
-[]
-[]
diff --git a/tests/qapi-schema/event-max.err b/tests/qapi-schema/event-max.err
index e69de29..c856534 100644
--- a/tests/qapi-schema/event-max.err
+++ b/tests/qapi-schema/event-max.err
@@ -0,0 +1 @@
+tests/qapi-schema/event-max.json:2: Event name 'MAX' cannot be created
diff --git a/tests/qapi-schema/event-max.exit b/tests/qapi-schema/event-max.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/event-max.exit
+++ b/tests/qapi-schema/event-max.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/event-max.json b/tests/qapi-schema/event-max.json
index 997c61c..f3d7de2 100644
--- a/tests/qapi-schema/event-max.json
+++ b/tests/qapi-schema/event-max.json
@@ -1,2 +1,2 @@
-# FIXME: an event named 'MAX' would conflict with implicit C enum
+# an event named 'MAX' would conflict with implicit C enum
{ 'event': 'MAX' }
diff --git a/tests/qapi-schema/event-max.out b/tests/qapi-schema/event-max.out
index 010c42b..e69de29 100644
--- a/tests/qapi-schema/event-max.out
+++ b/tests/qapi-schema/event-max.out
@@ -1,3 +0,0 @@
-[OrderedDict([('event', 'MAX')])]
-[]
-[]
diff --git a/tests/qapi-schema/redefined-builtin.err b/tests/qapi-schema/redefined-builtin.err
index e69de29..b275722 100644
--- a/tests/qapi-schema/redefined-builtin.err
+++ b/tests/qapi-schema/redefined-builtin.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-builtin.json:2: built-in 'size' is already defined
diff --git a/tests/qapi-schema/redefined-builtin.exit b/tests/qapi-schema/redefined-builtin.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-builtin.exit
+++ b/tests/qapi-schema/redefined-builtin.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-builtin.json b/tests/qapi-schema/redefined-builtin.json
index a10050d..df328cc 100644
--- a/tests/qapi-schema/redefined-builtin.json
+++ b/tests/qapi-schema/redefined-builtin.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject types that duplicate builtin names
+# we reject types that duplicate builtin names
{ 'type': 'size', 'data': { 'myint': 'size' } }
diff --git a/tests/qapi-schema/redefined-builtin.out b/tests/qapi-schema/redefined-builtin.out
index b0a9aea..e69de29 100644
--- a/tests/qapi-schema/redefined-builtin.out
+++ b/tests/qapi-schema/redefined-builtin.out
@@ -1,3 +0,0 @@
-[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
-[]
-[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
diff --git a/tests/qapi-schema/redefined-command.err b/tests/qapi-schema/redefined-command.err
index e69de29..82ae256 100644
--- a/tests/qapi-schema/redefined-command.err
+++ b/tests/qapi-schema/redefined-command.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-command.json:3: command 'foo' is already defined
diff --git a/tests/qapi-schema/redefined-command.exit b/tests/qapi-schema/redefined-command.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-command.exit
+++ b/tests/qapi-schema/redefined-command.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-command.json b/tests/qapi-schema/redefined-command.json
index d8c9975..247e401 100644
--- a/tests/qapi-schema/redefined-command.json
+++ b/tests/qapi-schema/redefined-command.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject commands defined more than once
+# we reject commands defined more than once
{ 'command': 'foo', 'data': { 'one': 'str' } }
{ 'command': 'foo', 'data': { '*two': 'str' } }
diff --git a/tests/qapi-schema/redefined-command.out b/tests/qapi-schema/redefined-command.out
index 44ddba6..e69de29 100644
--- a/tests/qapi-schema/redefined-command.out
+++ b/tests/qapi-schema/redefined-command.out
@@ -1,4 +0,0 @@
-[OrderedDict([('command', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
- OrderedDict([('command', 'foo'), ('data', OrderedDict([('*two', 'str')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/redefined-event.err b/tests/qapi-schema/redefined-event.err
index e69de29..35429cb 100644
--- a/tests/qapi-schema/redefined-event.err
+++ b/tests/qapi-schema/redefined-event.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-event.json:3: event 'EVENT_A' is already defined
diff --git a/tests/qapi-schema/redefined-event.exit b/tests/qapi-schema/redefined-event.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-event.exit
+++ b/tests/qapi-schema/redefined-event.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-event.json b/tests/qapi-schema/redefined-event.json
index 152cce7..7717e91 100644
--- a/tests/qapi-schema/redefined-event.json
+++ b/tests/qapi-schema/redefined-event.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject duplicate events
+# we reject duplicate events
{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
diff --git a/tests/qapi-schema/redefined-event.out b/tests/qapi-schema/redefined-event.out
index 261b183..e69de29 100644
--- a/tests/qapi-schema/redefined-event.out
+++ b/tests/qapi-schema/redefined-event.out
@@ -1,4 +0,0 @@
-[OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))]),
- OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/redefined-type.err b/tests/qapi-schema/redefined-type.err
index e69de29..06ea78c 100644
--- a/tests/qapi-schema/redefined-type.err
+++ b/tests/qapi-schema/redefined-type.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-type.json:3: struct 'foo' is already defined
diff --git a/tests/qapi-schema/redefined-type.exit b/tests/qapi-schema/redefined-type.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-type.exit
+++ b/tests/qapi-schema/redefined-type.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-type.json b/tests/qapi-schema/redefined-type.json
index 7972194..e6a5f24 100644
--- a/tests/qapi-schema/redefined-type.json
+++ b/tests/qapi-schema/redefined-type.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject types defined more than once
+# we reject types defined more than once
{ 'type': 'foo', 'data': { 'one': 'str' } }
{ 'enum': 'foo', 'data': [ 'two' ] }
diff --git a/tests/qapi-schema/redefined-type.out b/tests/qapi-schema/redefined-type.out
index b509e57..e69de29 100644
--- a/tests/qapi-schema/redefined-type.out
+++ b/tests/qapi-schema/redefined-type.out
@@ -1,4 +0,0 @@
-[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
- OrderedDict([('enum', 'foo'), ('data', ['two'])])]
-[{'enum_name': 'foo', 'enum_values': ['two']}]
-[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))])]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (15 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 17:32 ` Markus Armbruster
2015-03-31 15:23 ` Kevin Wolf
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests Eric Blake
` (12 subsequent siblings)
29 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
From: Fam Zheng <famz@redhat.com>
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.
In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
'string':{'type':'str', 'optional':true, 'default':null}}
We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.
Update the testsuite to match an improved error message.
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 21 ++++++++++++++++++---
tests/qapi-schema/bad-type-bool.err | 2 +-
tests/qapi-schema/bad-type-bool.json | 1 -
3 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 5d0dc91..6ed6a34 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -158,6 +158,20 @@ class QAPISchema:
return
else:
string += ch
+ elif self.tok in "tfn":
+ val = self.src[self.cursor - 1:]
+ if val.startswith("true"):
+ self.val = True
+ self.cursor += 3
+ return
+ elif val.startswith("false"):
+ self.val = False
+ self.cursor += 4
+ return
+ elif val.startswith("null"):
+ self.val = None
+ self.cursor += 3
+ return
elif self.tok == '\n':
if self.cursor == len(self.src):
self.tok = None
@@ -197,8 +211,9 @@ class QAPISchema:
if self.tok == ']':
self.accept()
return expr
- if not self.tok in [ '{', '[', "'" ]:
- raise QAPISchemaError(self, 'Expected "{", "[", "]" or string')
+ if not self.tok in "{['tfn":
+ raise QAPISchemaError(self, 'Expected "{", "[", "]", string, '
+ 'boolean or "null"')
while True:
expr.append(self.get_expr(True))
if self.tok == ']':
@@ -217,7 +232,7 @@ class QAPISchema:
elif self.tok == '[':
self.accept()
expr = self.get_values()
- elif self.tok == "'":
+ elif self.tok in "'tfn":
expr = self.val
self.accept()
else:
diff --git a/tests/qapi-schema/bad-type-bool.err b/tests/qapi-schema/bad-type-bool.err
index badb7c2..de6168c 100644
--- a/tests/qapi-schema/bad-type-bool.err
+++ b/tests/qapi-schema/bad-type-bool.err
@@ -1 +1 @@
-tests/qapi-schema/bad-type-bool.json:3:11: Stray "t"
+tests/qapi-schema/bad-type-bool.json:2: 'type' key must have a string value
diff --git a/tests/qapi-schema/bad-type-bool.json b/tests/qapi-schema/bad-type-bool.json
index 22d6369..e1e9fb0 100644
--- a/tests/qapi-schema/bad-type-bool.json
+++ b/tests/qapi-schema/bad-type-bool.json
@@ -1,3 +1,2 @@
# we reject an expression with a metatype that is not a string
-# FIXME: once the parser understands bool inputs, improve the error message
{ 'type': true, 'data': { } }
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (16 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 17:38 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests Eric Blake
` (11 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
For a few QMP commands, we are forced to pass an arbitrary type
without tracking it properly in QAPI. Among the existing clients,
this unnamed type was spelled 'dict', 'visitor', and '**'; this
patch standardizes on '**'.
Meanwhile, for both 'gen' and 'success-response' keys, we have been
ignoring the value, although the schema consistently used "'no'".
But now that we can support a literal "false" in the schema, we
might as well use that rather than ignoring the value or
special-casing a random string.
There is no difference to the generated code. As these features
were previously undocumented before this series, add some tests
and documentation on what we'd like to guarantee, although it will
take later patches to clean up test results and actually enforce
the use of a bool parameter.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
qapi-schema.json | 14 +++++++-------
qga/qapi-schema.json | 8 ++++----
tests/Makefile | 1 +
tests/qapi-schema/type-bypass-bad-gen.err | 0
tests/qapi-schema/type-bypass-bad-gen.exit | 1 +
tests/qapi-schema/type-bypass-bad-gen.json | 2 ++
tests/qapi-schema/type-bypass-bad-gen.out | 3 +++
tests/qapi-schema/type-bypass-no-gen.err | 0
tests/qapi-schema/type-bypass-no-gen.exit | 1 +
tests/qapi-schema/type-bypass-no-gen.json | 2 ++
tests/qapi-schema/type-bypass-no-gen.out | 3 +++
tests/qapi-schema/type-bypass.err | 0
tests/qapi-schema/type-bypass.exit | 1 +
tests/qapi-schema/type-bypass.json | 2 ++
tests/qapi-schema/type-bypass.out | 3 +++
15 files changed, 30 insertions(+), 11 deletions(-)
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.err
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.json
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.out
create mode 100644 tests/qapi-schema/type-bypass-no-gen.err
create mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-no-gen.json
create mode 100644 tests/qapi-schema/type-bypass-no-gen.out
create mode 100644 tests/qapi-schema/type-bypass.err
create mode 100644 tests/qapi-schema/type-bypass.exit
create mode 100644 tests/qapi-schema/type-bypass.json
create mode 100644 tests/qapi-schema/type-bypass.out
diff --git a/qapi-schema.json b/qapi-schema.json
index ac9594d..7f4cf86 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1561,8 +1561,8 @@
##
{ 'command': 'qom-get',
'data': { 'path': 'str', 'property': 'str' },
- 'returns': 'visitor',
- 'gen': 'no' }
+ 'returns': '**',
+ 'gen': false }
##
# @qom-set:
@@ -1579,8 +1579,8 @@
# Since: 1.2
##
{ 'command': 'qom-set',
- 'data': { 'path': 'str', 'property': 'str', 'value': 'visitor' },
- 'gen': 'no' }
+ 'data': { 'path': 'str', 'property': 'str', 'value': '**' },
+ 'gen': false }
##
# @set_password:
@@ -1943,7 +1943,7 @@
##
{ 'command': 'netdev_add',
'data': {'type': 'str', 'id': 'str', '*props': '**'},
- 'gen': 'no' }
+ 'gen': false }
##
# @netdev_del:
@@ -1976,8 +1976,8 @@
# Since: 2.0
##
{ 'command': 'object-add',
- 'data': {'qom-type': 'str', 'id': 'str', '*props': 'dict'},
- 'gen': 'no' }
+ 'data': {'qom-type': 'str', 'id': 'str', '*props': '**'},
+ 'gen': false }
##
# @object-del:
diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
index 95f49e3..fe5be7e 100644
--- a/qga/qapi-schema.json
+++ b/qga/qapi-schema.json
@@ -195,7 +195,7 @@
# Since: 0.15.0
##
{ 'command': 'guest-shutdown', 'data': { '*mode': 'str' },
- 'success-response': 'no' }
+ 'success-response': false }
##
# @guest-file-open:
@@ -470,7 +470,7 @@
#
# Since: 1.1
##
-{ 'command': 'guest-suspend-disk', 'success-response': 'no' }
+{ 'command': 'guest-suspend-disk', 'success-response': false }
##
# @guest-suspend-ram
@@ -502,7 +502,7 @@
#
# Since: 1.1
##
-{ 'command': 'guest-suspend-ram', 'success-response': 'no' }
+{ 'command': 'guest-suspend-ram', 'success-response': false }
##
# @guest-suspend-hybrid
@@ -529,7 +529,7 @@
#
# Since: 1.1
##
-{ 'command': 'guest-suspend-hybrid', 'success-response': 'no' }
+{ 'command': 'guest-suspend-hybrid', 'success-response': false }
##
# @GuestIpAddressType:
diff --git a/tests/Makefile b/tests/Makefile
index 7079e2a..941e839 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -215,6 +215,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
bad-type-dict.json double-data.json unknown-expr-key.json \
redefined-type.json redefined-command.json redefined-builtin.json \
redefined-event.json command-int.json event-max.json \
+ type-bypass.json type-bypass-no-gen.json type-bypass-bad-gen.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/type-bypass-bad-gen.err b/tests/qapi-schema/type-bypass-bad-gen.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/type-bypass-bad-gen.exit b/tests/qapi-schema/type-bypass-bad-gen.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-bad-gen.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/type-bypass-bad-gen.json b/tests/qapi-schema/type-bypass-bad-gen.json
new file mode 100644
index 0000000..bb70bee
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-bad-gen.json
@@ -0,0 +1,2 @@
+# FIXME: 'gen' should only appear with value false
+{ 'command': 'foo', 'gen': 'whatever' }
diff --git a/tests/qapi-schema/type-bypass-bad-gen.out b/tests/qapi-schema/type-bypass-bad-gen.out
new file mode 100644
index 0000000..e678f2c
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-bad-gen.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('gen', 'whatever')])]
+[]
+[]
diff --git a/tests/qapi-schema/type-bypass-no-gen.err b/tests/qapi-schema/type-bypass-no-gen.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/type-bypass-no-gen.exit b/tests/qapi-schema/type-bypass-no-gen.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-no-gen.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/type-bypass-no-gen.json b/tests/qapi-schema/type-bypass-no-gen.json
new file mode 100644
index 0000000..af87c19
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-no-gen.json
@@ -0,0 +1,2 @@
+# FIXME: type bypass should only work with 'gen':false
+{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**' }
diff --git a/tests/qapi-schema/type-bypass-no-gen.out b/tests/qapi-schema/type-bypass-no-gen.out
new file mode 100644
index 0000000..8b2a9ac
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-no-gen.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**')])]
+[]
+[]
diff --git a/tests/qapi-schema/type-bypass.err b/tests/qapi-schema/type-bypass.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/type-bypass.exit b/tests/qapi-schema/type-bypass.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/type-bypass.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/type-bypass.json b/tests/qapi-schema/type-bypass.json
new file mode 100644
index 0000000..48b2137
--- /dev/null
+++ b/tests/qapi-schema/type-bypass.json
@@ -0,0 +1,2 @@
+# Use of 'gen':false allows bypassing type system
+{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**', 'gen': false }
diff --git a/tests/qapi-schema/type-bypass.out b/tests/qapi-schema/type-bypass.out
new file mode 100644
index 0000000..eaf20f8
--- /dev/null
+++ b/tests/qapi-schema/type-bypass.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**'), ('gen', False)])]
+[]
+[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (17 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-26 17:58 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types Eric Blake
` (10 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Demonstrate that the qapi generator silently parses confusing
types, which may cause other errors later on. Later patches
will update the expected results as the generator is made stricter.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 8 ++++++--
tests/qapi-schema/data-array-empty.err | 0
tests/qapi-schema/data-array-empty.exit | 1 +
tests/qapi-schema/data-array-empty.json | 2 ++
tests/qapi-schema/data-array-empty.out | 3 +++
tests/qapi-schema/data-array-unknown.err | 0
tests/qapi-schema/data-array-unknown.exit | 1 +
tests/qapi-schema/data-array-unknown.json | 2 ++
tests/qapi-schema/data-array-unknown.out | 3 +++
tests/qapi-schema/data-int.err | 0
tests/qapi-schema/data-int.exit | 1 +
tests/qapi-schema/data-int.json | 2 ++
tests/qapi-schema/data-int.out | 3 +++
tests/qapi-schema/data-member-array-bad.err | 0
tests/qapi-schema/data-member-array-bad.exit | 1 +
tests/qapi-schema/data-member-array-bad.json | 2 ++
tests/qapi-schema/data-member-array-bad.out | 3 +++
tests/qapi-schema/data-member-array.err | 0
tests/qapi-schema/data-member-array.exit | 1 +
tests/qapi-schema/data-member-array.json | 4 ++++
tests/qapi-schema/data-member-array.out | 5 +++++
tests/qapi-schema/data-member-unknown.err | 0
tests/qapi-schema/data-member-unknown.exit | 1 +
tests/qapi-schema/data-member-unknown.json | 2 ++
tests/qapi-schema/data-member-unknown.out | 3 +++
tests/qapi-schema/data-unknown.err | 0
tests/qapi-schema/data-unknown.exit | 1 +
tests/qapi-schema/data-unknown.json | 2 ++
tests/qapi-schema/data-unknown.out | 3 +++
tests/qapi-schema/nested-struct-data.err | 0
tests/qapi-schema/nested-struct-data.exit | 1 +
tests/qapi-schema/nested-struct-data.json | 4 ++++
tests/qapi-schema/nested-struct-data.out | 3 +++
tests/qapi-schema/nested-struct-returns.err | 0
tests/qapi-schema/nested-struct-returns.exit | 1 +
tests/qapi-schema/nested-struct-returns.json | 3 +++
tests/qapi-schema/nested-struct-returns.out | 3 +++
tests/qapi-schema/returns-alternate.err | 0
tests/qapi-schema/returns-alternate.exit | 1 +
tests/qapi-schema/returns-alternate.json | 3 +++
tests/qapi-schema/returns-alternate.out | 4 ++++
tests/qapi-schema/returns-array-bad.err | 0
tests/qapi-schema/returns-array-bad.exit | 1 +
tests/qapi-schema/returns-array-bad.json | 2 ++
tests/qapi-schema/returns-array-bad.out | 3 +++
tests/qapi-schema/returns-int.err | 0
tests/qapi-schema/returns-int.exit | 1 +
tests/qapi-schema/returns-int.json | 2 ++
tests/qapi-schema/returns-int.out | 3 +++
tests/qapi-schema/returns-unknown.err | 0
tests/qapi-schema/returns-unknown.exit | 1 +
tests/qapi-schema/returns-unknown.json | 2 ++
tests/qapi-schema/returns-unknown.out | 3 +++
tests/qapi-schema/returns-whitelist.err | 0
tests/qapi-schema/returns-whitelist.exit | 1 +
tests/qapi-schema/returns-whitelist.json | 11 +++++++++++
tests/qapi-schema/returns-whitelist.out | 7 +++++++
57 files changed, 112 insertions(+), 2 deletions(-)
create mode 100644 tests/qapi-schema/data-array-empty.err
create mode 100644 tests/qapi-schema/data-array-empty.exit
create mode 100644 tests/qapi-schema/data-array-empty.json
create mode 100644 tests/qapi-schema/data-array-empty.out
create mode 100644 tests/qapi-schema/data-array-unknown.err
create mode 100644 tests/qapi-schema/data-array-unknown.exit
create mode 100644 tests/qapi-schema/data-array-unknown.json
create mode 100644 tests/qapi-schema/data-array-unknown.out
create mode 100644 tests/qapi-schema/data-int.err
create mode 100644 tests/qapi-schema/data-int.exit
create mode 100644 tests/qapi-schema/data-int.json
create mode 100644 tests/qapi-schema/data-int.out
create mode 100644 tests/qapi-schema/data-member-array-bad.err
create mode 100644 tests/qapi-schema/data-member-array-bad.exit
create mode 100644 tests/qapi-schema/data-member-array-bad.json
create mode 100644 tests/qapi-schema/data-member-array-bad.out
create mode 100644 tests/qapi-schema/data-member-array.err
create mode 100644 tests/qapi-schema/data-member-array.exit
create mode 100644 tests/qapi-schema/data-member-array.json
create mode 100644 tests/qapi-schema/data-member-array.out
create mode 100644 tests/qapi-schema/data-member-unknown.err
create mode 100644 tests/qapi-schema/data-member-unknown.exit
create mode 100644 tests/qapi-schema/data-member-unknown.json
create mode 100644 tests/qapi-schema/data-member-unknown.out
create mode 100644 tests/qapi-schema/data-unknown.err
create mode 100644 tests/qapi-schema/data-unknown.exit
create mode 100644 tests/qapi-schema/data-unknown.json
create mode 100644 tests/qapi-schema/data-unknown.out
create mode 100644 tests/qapi-schema/nested-struct-data.err
create mode 100644 tests/qapi-schema/nested-struct-data.exit
create mode 100644 tests/qapi-schema/nested-struct-data.json
create mode 100644 tests/qapi-schema/nested-struct-data.out
create mode 100644 tests/qapi-schema/nested-struct-returns.err
create mode 100644 tests/qapi-schema/nested-struct-returns.exit
create mode 100644 tests/qapi-schema/nested-struct-returns.json
create mode 100644 tests/qapi-schema/nested-struct-returns.out
create mode 100644 tests/qapi-schema/returns-alternate.err
create mode 100644 tests/qapi-schema/returns-alternate.exit
create mode 100644 tests/qapi-schema/returns-alternate.json
create mode 100644 tests/qapi-schema/returns-alternate.out
create mode 100644 tests/qapi-schema/returns-array-bad.err
create mode 100644 tests/qapi-schema/returns-array-bad.exit
create mode 100644 tests/qapi-schema/returns-array-bad.json
create mode 100644 tests/qapi-schema/returns-array-bad.out
create mode 100644 tests/qapi-schema/returns-int.err
create mode 100644 tests/qapi-schema/returns-int.exit
create mode 100644 tests/qapi-schema/returns-int.json
create mode 100644 tests/qapi-schema/returns-int.out
create mode 100644 tests/qapi-schema/returns-unknown.err
create mode 100644 tests/qapi-schema/returns-unknown.exit
create mode 100644 tests/qapi-schema/returns-unknown.json
create mode 100644 tests/qapi-schema/returns-unknown.out
create mode 100644 tests/qapi-schema/returns-whitelist.err
create mode 100644 tests/qapi-schema/returns-whitelist.exit
create mode 100644 tests/qapi-schema/returns-whitelist.json
create mode 100644 tests/qapi-schema/returns-whitelist.out
diff --git a/tests/Makefile b/tests/Makefile
index 941e839..9877a5b 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -216,8 +216,12 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
redefined-type.json redefined-command.json redefined-builtin.json \
redefined-event.json command-int.json event-max.json \
type-bypass.json type-bypass-no-gen.json type-bypass-bad-gen.json \
- missing-colon.json missing-comma-list.json \
- missing-comma-object.json non-objects.json \
+ data-array-empty.json data-array-unknown.json data-int.json \
+ data-unknown.json data-member-unknown.json data-member-array.json \
+ data-member-array-bad.json returns-array-bad.json returns-int.json \
+ returns-unknown.json returns-alternate.json returns-whitelist.json \
+ missing-colon.json missing-comma-list.json missing-comma-object.json \
+ nested-struct-data.json nested-struct-returns.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
trailing-comma-list.json trailing-comma-object.json \
unclosed-list.json unclosed-object.json unclosed-string.json \
diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-array-empty.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
new file mode 100644
index 0000000..edb543a
--- /dev/null
+++ b/tests/qapi-schema/data-array-empty.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an array for data if it does not contain a known type
+{ 'command': 'oops', 'data': { 'empty': [ ] } }
diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
new file mode 100644
index 0000000..6f25c9e
--- /dev/null
+++ b/tests/qapi-schema/data-array-empty.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', OrderedDict([('empty', [])]))])]
+[]
+[]
diff --git a/tests/qapi-schema/data-array-unknown.err b/tests/qapi-schema/data-array-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-array-unknown.exit b/tests/qapi-schema/data-array-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-array-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-array-unknown.json b/tests/qapi-schema/data-array-unknown.json
new file mode 100644
index 0000000..20cd3c0
--- /dev/null
+++ b/tests/qapi-schema/data-array-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an array for data if it does not contain a known type
+{ 'command': 'oops', 'data': { 'array': [ 'NoSuchType' ] } }
diff --git a/tests/qapi-schema/data-array-unknown.out b/tests/qapi-schema/data-array-unknown.out
new file mode 100644
index 0000000..4314ab5
--- /dev/null
+++ b/tests/qapi-schema/data-array-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', OrderedDict([('array', ['NoSuchType'])]))])]
+[]
+[]
diff --git a/tests/qapi-schema/data-int.err b/tests/qapi-schema/data-int.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-int.exit b/tests/qapi-schema/data-int.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-int.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-int.json b/tests/qapi-schema/data-int.json
new file mode 100644
index 0000000..37916e0
--- /dev/null
+++ b/tests/qapi-schema/data-int.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject commands where data is not an array or complex type
+{ 'command': 'oops', 'data': 'int' }
diff --git a/tests/qapi-schema/data-int.out b/tests/qapi-schema/data-int.out
new file mode 100644
index 0000000..e589a4f
--- /dev/null
+++ b/tests/qapi-schema/data-int.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', 'int')])]
+[]
+[]
diff --git a/tests/qapi-schema/data-member-array-bad.err b/tests/qapi-schema/data-member-array-bad.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-member-array-bad.exit b/tests/qapi-schema/data-member-array-bad.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-member-array-bad.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-member-array-bad.json b/tests/qapi-schema/data-member-array-bad.json
new file mode 100644
index 0000000..c954af1
--- /dev/null
+++ b/tests/qapi-schema/data-member-array-bad.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject data if it does not contain a valid array type
+{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
diff --git a/tests/qapi-schema/data-member-array-bad.out b/tests/qapi-schema/data-member-array-bad.out
new file mode 100644
index 0000000..0e00c41
--- /dev/null
+++ b/tests/qapi-schema/data-member-array-bad.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', [OrderedDict([('nested', 'str')])])]))])]
+[]
+[]
diff --git a/tests/qapi-schema/data-member-array.err b/tests/qapi-schema/data-member-array.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-member-array.exit b/tests/qapi-schema/data-member-array.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-member-array.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-member-array.json b/tests/qapi-schema/data-member-array.json
new file mode 100644
index 0000000..7cce276
--- /dev/null
+++ b/tests/qapi-schema/data-member-array.json
@@ -0,0 +1,4 @@
+# valid array members
+{ 'enum': 'abc', 'data': [ 'a', 'b', 'c' ] }
+{ 'type': 'def', 'data': { 'array': [ 'abc' ] } }
+{ 'command': 'okay', 'data': { 'member1': [ 'int' ], 'member2': [ 'def' ] } }
diff --git a/tests/qapi-schema/data-member-array.out b/tests/qapi-schema/data-member-array.out
new file mode 100644
index 0000000..8287120
--- /dev/null
+++ b/tests/qapi-schema/data-member-array.out
@@ -0,0 +1,5 @@
+[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
+ OrderedDict([('type', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
+ OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), ('member2', ['def'])]))])]
+[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
+[OrderedDict([('type', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
diff --git a/tests/qapi-schema/data-member-unknown.err b/tests/qapi-schema/data-member-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-member-unknown.exit b/tests/qapi-schema/data-member-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-member-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-member-unknown.json b/tests/qapi-schema/data-member-unknown.json
new file mode 100644
index 0000000..40e6252
--- /dev/null
+++ b/tests/qapi-schema/data-member-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject data if it does not contain a known type
+{ 'command': 'oops', 'data': { 'member': 'NoSuchType' } }
diff --git a/tests/qapi-schema/data-member-unknown.out b/tests/qapi-schema/data-member-unknown.out
new file mode 100644
index 0000000..36252a5
--- /dev/null
+++ b/tests/qapi-schema/data-member-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', 'NoSuchType')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/data-unknown.err b/tests/qapi-schema/data-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-unknown.exit b/tests/qapi-schema/data-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-unknown.json b/tests/qapi-schema/data-unknown.json
new file mode 100644
index 0000000..776bd34
--- /dev/null
+++ b/tests/qapi-schema/data-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject data if it does not contain a known type
+{ 'command': 'oops', 'data': 'NoSuchType' }
diff --git a/tests/qapi-schema/data-unknown.out b/tests/qapi-schema/data-unknown.out
new file mode 100644
index 0000000..2c60726
--- /dev/null
+++ b/tests/qapi-schema/data-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', 'NoSuchType')])]
+[]
+[]
diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/nested-struct-data.exit b/tests/qapi-schema/nested-struct-data.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-data.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/nested-struct-data.json b/tests/qapi-schema/nested-struct-data.json
new file mode 100644
index 0000000..0247c8c
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-data.json
@@ -0,0 +1,4 @@
+# FIXME: inline subtypes collide with our desired future use of defaults
+{ 'command': 'foo',
+ 'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' },
+ 'returns': {} }
diff --git a/tests/qapi-schema/nested-struct-data.out b/tests/qapi-schema/nested-struct-data.out
new file mode 100644
index 0000000..999cbb8
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-data.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('data', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')])), ('returns', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/nested-struct-returns.err b/tests/qapi-schema/nested-struct-returns.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/nested-struct-returns.exit b/tests/qapi-schema/nested-struct-returns.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-returns.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/nested-struct-returns.json b/tests/qapi-schema/nested-struct-returns.json
new file mode 100644
index 0000000..5a46840
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-returns.json
@@ -0,0 +1,3 @@
+# FIXME: inline subtypes collide with our desired future use of defaults
+{ 'command': 'foo',
+ 'returns': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }
diff --git a/tests/qapi-schema/nested-struct-returns.out b/tests/qapi-schema/nested-struct-returns.out
new file mode 100644
index 0000000..c53d23b
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-returns.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('returns', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-alternate.err b/tests/qapi-schema/returns-alternate.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-alternate.exit b/tests/qapi-schema/returns-alternate.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-alternate.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-alternate.json b/tests/qapi-schema/returns-alternate.json
new file mode 100644
index 0000000..b3b91fd
--- /dev/null
+++ b/tests/qapi-schema/returns-alternate.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject returns if it is an alternate type
+{ 'alternate': 'Alt', 'data': { 'a': 'int', 'b': 'str' } }
+{ 'command': 'oops', 'returns': 'Alt' }
diff --git a/tests/qapi-schema/returns-alternate.out b/tests/qapi-schema/returns-alternate.out
new file mode 100644
index 0000000..8a03ed3
--- /dev/null
+++ b/tests/qapi-schema/returns-alternate.out
@@ -0,0 +1,4 @@
+[OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('a', 'int'), ('b', 'str')]))]),
+ OrderedDict([('command', 'oops'), ('returns', 'Alt')])]
+[{'enum_name': 'AltKind', 'enum_values': None}]
+[]
diff --git a/tests/qapi-schema/returns-array-bad.err b/tests/qapi-schema/returns-array-bad.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-array-bad.exit b/tests/qapi-schema/returns-array-bad.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-array-bad.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-array-bad.json b/tests/qapi-schema/returns-array-bad.json
new file mode 100644
index 0000000..14882c1
--- /dev/null
+++ b/tests/qapi-schema/returns-array-bad.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an array return that is not a single type
+{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
diff --git a/tests/qapi-schema/returns-array-bad.out b/tests/qapi-schema/returns-array-bad.out
new file mode 100644
index 0000000..eccad55
--- /dev/null
+++ b/tests/qapi-schema/returns-array-bad.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('returns', ['str', 'str'])])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-int.err b/tests/qapi-schema/returns-int.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-int.exit b/tests/qapi-schema/returns-int.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-int.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-int.json b/tests/qapi-schema/returns-int.json
new file mode 100644
index 0000000..7888fb1
--- /dev/null
+++ b/tests/qapi-schema/returns-int.json
@@ -0,0 +1,2 @@
+# It is okay (although not extensible) to return a non-dictionary
+{ 'command': 'okay', 'returns': 'int' }
diff --git a/tests/qapi-schema/returns-int.out b/tests/qapi-schema/returns-int.out
new file mode 100644
index 0000000..36b00a9
--- /dev/null
+++ b/tests/qapi-schema/returns-int.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'okay'), ('returns', 'int')])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-unknown.err b/tests/qapi-schema/returns-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-unknown.exit b/tests/qapi-schema/returns-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-unknown.json b/tests/qapi-schema/returns-unknown.json
new file mode 100644
index 0000000..61f20eb
--- /dev/null
+++ b/tests/qapi-schema/returns-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject returns if it does not contain a known type
+{ 'command': 'oops', 'returns': 'NoSuchType' }
diff --git a/tests/qapi-schema/returns-unknown.out b/tests/qapi-schema/returns-unknown.out
new file mode 100644
index 0000000..a482c83
--- /dev/null
+++ b/tests/qapi-schema/returns-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-whitelist.err b/tests/qapi-schema/returns-whitelist.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-whitelist.exit b/tests/qapi-schema/returns-whitelist.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-whitelist.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-whitelist.json b/tests/qapi-schema/returns-whitelist.json
new file mode 100644
index 0000000..8328563
--- /dev/null
+++ b/tests/qapi-schema/returns-whitelist.json
@@ -0,0 +1,11 @@
+# FIXME: we should enforce that 'returns' be a dict or array of dict unless whitelisted
+{ 'command': 'human-monitor-command',
+ 'data': {'command-line': 'str', '*cpu-index': 'int'},
+ 'returns': 'str' }
+{ 'enum': 'TpmModel', 'data': [ 'tpm-tis' ] }
+{ 'command': 'query-tpm-models', 'returns': ['TpmModel'] }
+{ 'command': 'guest-get-time',
+ 'returns': 'int' }
+
+{ 'command': 'no-way-this-will-get-whitelisted',
+ 'returns': [ 'int' ] }
diff --git a/tests/qapi-schema/returns-whitelist.out b/tests/qapi-schema/returns-whitelist.out
new file mode 100644
index 0000000..2adcd8b
--- /dev/null
+++ b/tests/qapi-schema/returns-whitelist.out
@@ -0,0 +1,7 @@
+[OrderedDict([('command', 'human-monitor-command'), ('data', OrderedDict([('command-line', 'str'), ('*cpu-index', 'int')])), ('returns', 'str')]),
+ OrderedDict([('enum', 'TpmModel'), ('data', ['tpm-tis'])]),
+ OrderedDict([('command', 'query-tpm-models'), ('returns', ['TpmModel'])]),
+ OrderedDict([('command', 'guest-get-time'), ('returns', 'int')]),
+ OrderedDict([('command', 'no-way-this-will-get-whitelisted'), ('returns', ['int'])])]
+[{'enum_name': 'TpmModel', 'enum_values': ['tpm-tis']}]
+[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (18 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 8:23 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names Eric Blake
` (9 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Now that we know every expression is valid with regards to
its keys, we can add further tests that those keys refer to
valid types. With this patch, all uses of a type (the 'data':
of command, type, union, alternate, and event; the 'returns':
of command; the 'base': of type and union) must resolve to an
appropriate subset of metatypes declared by the current qapi
parse; this includes recursing into each member of a data
dictionary. Dealing with '**' and nested anonymous structs
will be done in later patches.
Update the testsuite to match improved output.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 94 +++++++++++++++++++++++++---
tests/qapi-schema/alternate-array.err | 2 +-
tests/qapi-schema/alternate-nested.err | 2 +-
tests/qapi-schema/alternate-unknown.err | 2 +-
tests/qapi-schema/bad-base.err | 1 +
tests/qapi-schema/bad-base.exit | 2 +-
tests/qapi-schema/bad-base.json | 2 +-
tests/qapi-schema/bad-base.out | 4 --
tests/qapi-schema/data-array-empty.err | 1 +
tests/qapi-schema/data-array-empty.exit | 2 +-
tests/qapi-schema/data-array-empty.json | 2 +-
tests/qapi-schema/data-array-empty.out | 3 -
tests/qapi-schema/data-array-unknown.err | 1 +
tests/qapi-schema/data-array-unknown.exit | 2 +-
tests/qapi-schema/data-array-unknown.json | 2 +-
tests/qapi-schema/data-array-unknown.out | 3 -
tests/qapi-schema/data-int.err | 1 +
tests/qapi-schema/data-int.exit | 2 +-
tests/qapi-schema/data-int.json | 2 +-
tests/qapi-schema/data-int.out | 3 -
tests/qapi-schema/data-member-array-bad.err | 1 +
tests/qapi-schema/data-member-array-bad.exit | 2 +-
tests/qapi-schema/data-member-array-bad.json | 2 +-
tests/qapi-schema/data-member-array-bad.out | 3 -
tests/qapi-schema/data-member-unknown.err | 1 +
tests/qapi-schema/data-member-unknown.exit | 2 +-
tests/qapi-schema/data-member-unknown.json | 2 +-
tests/qapi-schema/data-member-unknown.out | 3 -
tests/qapi-schema/data-unknown.err | 1 +
tests/qapi-schema/data-unknown.exit | 2 +-
tests/qapi-schema/data-unknown.json | 2 +-
tests/qapi-schema/data-unknown.out | 3 -
tests/qapi-schema/returns-array-bad.err | 1 +
tests/qapi-schema/returns-array-bad.exit | 2 +-
tests/qapi-schema/returns-array-bad.json | 2 +-
tests/qapi-schema/returns-array-bad.out | 3 -
tests/qapi-schema/returns-unknown.err | 1 +
tests/qapi-schema/returns-unknown.exit | 2 +-
tests/qapi-schema/returns-unknown.json | 2 +-
tests/qapi-schema/returns-unknown.out | 3 -
tests/qapi-schema/union-unknown.err | 1 +
tests/qapi-schema/union-unknown.exit | 2 +-
tests/qapi-schema/union-unknown.json | 2 +-
tests/qapi-schema/union-unknown.out | 3 -
44 files changed, 118 insertions(+), 63 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6ed6a34..c42683b 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -276,6 +276,63 @@ def discriminator_find_enum_define(expr):
return find_enum(discriminator_type)
+def check_type(expr_info, source, data, allow_array = False,
+ allowed_metas = [], allow_dict = True):
+ global all_names
+
+ if data is None:
+ return
+
+ if data == '**':
+ return
+
+ # Check if array type for data is okay
+ if isinstance(data, list):
+ if not allow_array:
+ raise QAPIExprError(expr_info,
+ "%s cannot be an array" % source)
+ if len(data) != 1 or not isinstance(data[0], str):
+ raise QAPIExprError(expr_info,
+ "%s: array type must contain single type name"
+ % source)
+ data = data[0]
+
+ # Check if type name for data is okay
+ if isinstance(data, str):
+ if not data in all_names:
+ raise QAPIExprError(expr_info,
+ "%s uses unknown type '%s'"
+ % (source, data))
+ if not all_names[data] in allowed_metas:
+ raise QAPIExprError(expr_info,
+ "%s cannot use %s type '%s'"
+ % (source, all_names[data], data))
+ return
+
+ # data is a dictionary, check that each member is okay
+ if not isinstance(data, OrderedDict):
+ raise QAPIExprError(expr_info,
+ "%s should be a dictionary" % source)
+ if not allow_dict:
+ raise QAPIExprError(expr_info,
+ "%s should be a type name" % source)
+ for (key, value) in data.items():
+ check_type(expr_info, "Member '%s' of %s" % (key, source), value,
+ allow_array=True,
+ allowed_metas=['built-in', 'union', 'alternate', 'struct',
+ 'enum'],
+ allow_dict=True)
+
+def check_command(expr, expr_info):
+ name = expr['command']
+ check_type(expr_info, "'data' for command '%s'" % name,
+ expr.get('data'),
+ allowed_metas=['union', 'struct'])
+ check_type(expr_info, "'returns' for command '%s'" % name,
+ expr.get('returns'), allow_array=True,
+ allowed_metas=['built-in', 'union', 'alternate', 'struct',
+ 'enum'])
+
def check_event(expr, expr_info):
global events
name = expr['event']
@@ -287,7 +344,8 @@ def check_event(expr, expr_info):
raise QAPIExprError(expr_info, "Event name '%s' should be upper case"
% name)
events.append(name)
-
+ check_type(expr_info, "'data' for event '%s'" % name,
+ expr.get('data'), allowed_metas=['union', 'struct'])
if params:
for argname, argentry, optional, structured in parse_args(params):
if structured:
@@ -348,6 +406,13 @@ def check_union(expr, expr_info):
# Check every branch
for (key, value) in members.items():
+ # Each value must name a known type
+ check_type(expr_info, "Member '%s' of union '%s'" % (key, name),
+ value, allow_array=True,
+ allowed_metas=['built-in', 'union', 'alternate', 'struct',
+ 'enum'],
+ allow_dict=False)
+
# If the discriminator names an enum type, then all members
# of 'data' must also be members of the enum type.
if enum_define:
@@ -383,15 +448,12 @@ def check_alternate(expr, expr_info):
values[c_key] = key
# Ensure alternates have no type conflicts.
- if isinstance(value, list):
- raise QAPIExprError(expr_info,
- "Alternate '%s' member '%s' must "
- "not be array type" % (name, key))
+ check_type(expr_info, "Member '%s' of alternate '%s'" % (key, name),
+ value,
+ allowed_metas=['built-in', 'union', 'struct', 'enum'],
+ allow_dict=False)
qtype = find_alternate_member_qtype(value)
- if not qtype:
- raise QAPIExprError(expr_info,
- "Alternate '%s' member '%s' has "
- "invalid type '%s'" % (name, key, value))
+ assert qtype
if qtype in types_seen:
raise QAPIExprError(expr_info,
"Alternate '%s' member '%s' has "
@@ -419,6 +481,14 @@ def check_enum(expr, expr_info):
% (name, member, values[key]))
values[key] = member
+def check_struct(expr, expr_info):
+ name = expr['type']
+ members = expr['data']
+
+ check_type(expr_info, "'data' for type '%s'" % name, members)
+ check_type(expr_info, "'base' for type '%s'" % name, expr.get('base'),
+ allowed_metas=['struct'], allow_dict=False)
+
def check_exprs(schema):
for expr_elem in schema.exprs:
expr = expr_elem['expr']
@@ -430,8 +500,14 @@ def check_exprs(schema):
check_union(expr, info)
elif expr.has_key('alternate'):
check_alternate(expr, info)
+ elif expr.has_key('type'):
+ check_struct(expr, info)
+ elif expr.has_key('command'):
+ check_command(expr, info)
elif expr.has_key('event'):
check_event(expr, info)
+ else:
+ assert False, 'unexpected meta type'
def check_keys(expr_elem, meta, required, optional=[]):
expr = expr_elem['expr']
diff --git a/tests/qapi-schema/alternate-array.err b/tests/qapi-schema/alternate-array.err
index 5721ed2..f4f0c1d 100644
--- a/tests/qapi-schema/alternate-array.err
+++ b/tests/qapi-schema/alternate-array.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-array.json:4: Alternate 'Alt' member 'two' must not be array type
+tests/qapi-schema/alternate-array.json:4: Member 'two' of alternate 'Alt' cannot be an array
diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
index 00b05c6..4d1187e 100644
--- a/tests/qapi-schema/alternate-nested.err
+++ b/tests/qapi-schema/alternate-nested.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-nested.json:4: Alternate 'Alt2' member 'nested' has invalid type 'Alt1'
+tests/qapi-schema/alternate-nested.json:4: Member 'nested' of alternate 'Alt2' cannot use alternate type 'Alt1'
diff --git a/tests/qapi-schema/alternate-unknown.err b/tests/qapi-schema/alternate-unknown.err
index 7af1b4c..dea45dc 100644
--- a/tests/qapi-schema/alternate-unknown.err
+++ b/tests/qapi-schema/alternate-unknown.err
@@ -1 +1 @@
-tests/qapi-schema/alternate-unknown.json:2: Alternate 'Alt' member 'unknown' has invalid type 'MissingType'
+tests/qapi-schema/alternate-unknown.json:2: Member 'unknown' of alternate 'Alt' uses unknown type 'MissingType'
diff --git a/tests/qapi-schema/bad-base.err b/tests/qapi-schema/bad-base.err
index e69de29..f398bbb 100644
--- a/tests/qapi-schema/bad-base.err
+++ b/tests/qapi-schema/bad-base.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-base.json:3: 'base' for type 'MyType' cannot use union type 'Union'
diff --git a/tests/qapi-schema/bad-base.exit b/tests/qapi-schema/bad-base.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/bad-base.exit
+++ b/tests/qapi-schema/bad-base.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/bad-base.json b/tests/qapi-schema/bad-base.json
index de964a0..a690470 100644
--- a/tests/qapi-schema/bad-base.json
+++ b/tests/qapi-schema/bad-base.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject a base that is not a struct
+# we reject a base that is not a struct
{ 'union': 'Union', 'data': { 'a': 'int', 'b': 'str' } }
{ 'type': 'MyType', 'base': 'Union', 'data': { 'c': 'int' } }
diff --git a/tests/qapi-schema/bad-base.out b/tests/qapi-schema/bad-base.out
index 91d12fc..e69de29 100644
--- a/tests/qapi-schema/bad-base.out
+++ b/tests/qapi-schema/bad-base.out
@@ -1,4 +0,0 @@
-[OrderedDict([('union', 'Union'), ('data', OrderedDict([('a', 'int'), ('b', 'str')]))]),
- OrderedDict([('type', 'MyType'), ('base', 'Union'), ('data', OrderedDict([('c', 'int')]))])]
-[{'enum_name': 'UnionKind', 'enum_values': None}]
-[OrderedDict([('type', 'MyType'), ('base', 'Union'), ('data', OrderedDict([('c', 'int')]))])]
diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
index e69de29..f713f14 100644
--- a/tests/qapi-schema/data-array-empty.err
+++ b/tests/qapi-schema/data-array-empty.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-array-empty.json:2: Member 'empty' of 'data' for command 'oops': array type must contain single type name
diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-array-empty.exit
+++ b/tests/qapi-schema/data-array-empty.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
index edb543a..652dcfb 100644
--- a/tests/qapi-schema/data-array-empty.json
+++ b/tests/qapi-schema/data-array-empty.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an array for data if it does not contain a known type
+# we reject an array for data if it does not contain a known type
{ 'command': 'oops', 'data': { 'empty': [ ] } }
diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
index 6f25c9e..e69de29 100644
--- a/tests/qapi-schema/data-array-empty.out
+++ b/tests/qapi-schema/data-array-empty.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', OrderedDict([('empty', [])]))])]
-[]
-[]
diff --git a/tests/qapi-schema/data-array-unknown.err b/tests/qapi-schema/data-array-unknown.err
index e69de29..21c39ca 100644
--- a/tests/qapi-schema/data-array-unknown.err
+++ b/tests/qapi-schema/data-array-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-array-unknown.json:2: Member 'array' of 'data' for command 'oops' uses unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/data-array-unknown.exit b/tests/qapi-schema/data-array-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-array-unknown.exit
+++ b/tests/qapi-schema/data-array-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-array-unknown.json b/tests/qapi-schema/data-array-unknown.json
index 20cd3c0..6f3e883 100644
--- a/tests/qapi-schema/data-array-unknown.json
+++ b/tests/qapi-schema/data-array-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an array for data if it does not contain a known type
+# we reject an array for data if it does not contain a known type
{ 'command': 'oops', 'data': { 'array': [ 'NoSuchType' ] } }
diff --git a/tests/qapi-schema/data-array-unknown.out b/tests/qapi-schema/data-array-unknown.out
index 4314ab5..e69de29 100644
--- a/tests/qapi-schema/data-array-unknown.out
+++ b/tests/qapi-schema/data-array-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', OrderedDict([('array', ['NoSuchType'])]))])]
-[]
-[]
diff --git a/tests/qapi-schema/data-int.err b/tests/qapi-schema/data-int.err
index e69de29..1a9b077 100644
--- a/tests/qapi-schema/data-int.err
+++ b/tests/qapi-schema/data-int.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-int.json:2: 'data' for command 'oops' cannot use built-in type 'int'
diff --git a/tests/qapi-schema/data-int.exit b/tests/qapi-schema/data-int.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-int.exit
+++ b/tests/qapi-schema/data-int.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-int.json b/tests/qapi-schema/data-int.json
index 37916e0..a334d92 100644
--- a/tests/qapi-schema/data-int.json
+++ b/tests/qapi-schema/data-int.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject commands where data is not an array or complex type
+# we reject commands where data is not an array or complex type
{ 'command': 'oops', 'data': 'int' }
diff --git a/tests/qapi-schema/data-int.out b/tests/qapi-schema/data-int.out
index e589a4f..e69de29 100644
--- a/tests/qapi-schema/data-int.out
+++ b/tests/qapi-schema/data-int.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', 'int')])]
-[]
-[]
diff --git a/tests/qapi-schema/data-member-array-bad.err b/tests/qapi-schema/data-member-array-bad.err
index e69de29..2c072d5 100644
--- a/tests/qapi-schema/data-member-array-bad.err
+++ b/tests/qapi-schema/data-member-array-bad.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-member-array-bad.json:2: Member 'member' of 'data' for command 'oops': array type must contain single type name
diff --git a/tests/qapi-schema/data-member-array-bad.exit b/tests/qapi-schema/data-member-array-bad.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-member-array-bad.exit
+++ b/tests/qapi-schema/data-member-array-bad.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-member-array-bad.json b/tests/qapi-schema/data-member-array-bad.json
index c954af1..b2ff144 100644
--- a/tests/qapi-schema/data-member-array-bad.json
+++ b/tests/qapi-schema/data-member-array-bad.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject data if it does not contain a valid array type
+# we reject data if it does not contain a valid array type
{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
diff --git a/tests/qapi-schema/data-member-array-bad.out b/tests/qapi-schema/data-member-array-bad.out
index 0e00c41..e69de29 100644
--- a/tests/qapi-schema/data-member-array-bad.out
+++ b/tests/qapi-schema/data-member-array-bad.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', [OrderedDict([('nested', 'str')])])]))])]
-[]
-[]
diff --git a/tests/qapi-schema/data-member-unknown.err b/tests/qapi-schema/data-member-unknown.err
index e69de29..ab905db 100644
--- a/tests/qapi-schema/data-member-unknown.err
+++ b/tests/qapi-schema/data-member-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-member-unknown.json:2: Member 'member' of 'data' for command 'oops' uses unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/data-member-unknown.exit b/tests/qapi-schema/data-member-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-member-unknown.exit
+++ b/tests/qapi-schema/data-member-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-member-unknown.json b/tests/qapi-schema/data-member-unknown.json
index 40e6252..342a41e 100644
--- a/tests/qapi-schema/data-member-unknown.json
+++ b/tests/qapi-schema/data-member-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject data if it does not contain a known type
+# we reject data if it does not contain a known type
{ 'command': 'oops', 'data': { 'member': 'NoSuchType' } }
diff --git a/tests/qapi-schema/data-member-unknown.out b/tests/qapi-schema/data-member-unknown.out
index 36252a5..e69de29 100644
--- a/tests/qapi-schema/data-member-unknown.out
+++ b/tests/qapi-schema/data-member-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', 'NoSuchType')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/data-unknown.err b/tests/qapi-schema/data-unknown.err
index e69de29..5b07277 100644
--- a/tests/qapi-schema/data-unknown.err
+++ b/tests/qapi-schema/data-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-unknown.json:2: 'data' for command 'oops' uses unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/data-unknown.exit b/tests/qapi-schema/data-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-unknown.exit
+++ b/tests/qapi-schema/data-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-unknown.json b/tests/qapi-schema/data-unknown.json
index 776bd34..32aba43 100644
--- a/tests/qapi-schema/data-unknown.json
+++ b/tests/qapi-schema/data-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject data if it does not contain a known type
+# we reject data if it does not contain a known type
{ 'command': 'oops', 'data': 'NoSuchType' }
diff --git a/tests/qapi-schema/data-unknown.out b/tests/qapi-schema/data-unknown.out
index 2c60726..e69de29 100644
--- a/tests/qapi-schema/data-unknown.out
+++ b/tests/qapi-schema/data-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', 'NoSuchType')])]
-[]
-[]
diff --git a/tests/qapi-schema/returns-array-bad.err b/tests/qapi-schema/returns-array-bad.err
index e69de29..138095c 100644
--- a/tests/qapi-schema/returns-array-bad.err
+++ b/tests/qapi-schema/returns-array-bad.err
@@ -0,0 +1 @@
+tests/qapi-schema/returns-array-bad.json:2: 'returns' for command 'oops': array type must contain single type name
diff --git a/tests/qapi-schema/returns-array-bad.exit b/tests/qapi-schema/returns-array-bad.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/returns-array-bad.exit
+++ b/tests/qapi-schema/returns-array-bad.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/returns-array-bad.json b/tests/qapi-schema/returns-array-bad.json
index 14882c1..09b0b1f 100644
--- a/tests/qapi-schema/returns-array-bad.json
+++ b/tests/qapi-schema/returns-array-bad.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an array return that is not a single type
+# we reject an array return that is not a single type
{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
diff --git a/tests/qapi-schema/returns-array-bad.out b/tests/qapi-schema/returns-array-bad.out
index eccad55..e69de29 100644
--- a/tests/qapi-schema/returns-array-bad.out
+++ b/tests/qapi-schema/returns-array-bad.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('returns', ['str', 'str'])])]
-[]
-[]
diff --git a/tests/qapi-schema/returns-unknown.err b/tests/qapi-schema/returns-unknown.err
index e69de29..1f43e3a 100644
--- a/tests/qapi-schema/returns-unknown.err
+++ b/tests/qapi-schema/returns-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/returns-unknown.json:2: 'returns' for command 'oops' uses unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/returns-unknown.exit b/tests/qapi-schema/returns-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/returns-unknown.exit
+++ b/tests/qapi-schema/returns-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/returns-unknown.json b/tests/qapi-schema/returns-unknown.json
index 61f20eb..25bd498 100644
--- a/tests/qapi-schema/returns-unknown.json
+++ b/tests/qapi-schema/returns-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject returns if it does not contain a known type
+# we reject returns if it does not contain a known type
{ 'command': 'oops', 'returns': 'NoSuchType' }
diff --git a/tests/qapi-schema/returns-unknown.out b/tests/qapi-schema/returns-unknown.out
index a482c83..e69de29 100644
--- a/tests/qapi-schema/returns-unknown.out
+++ b/tests/qapi-schema/returns-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
-[]
-[]
diff --git a/tests/qapi-schema/union-unknown.err b/tests/qapi-schema/union-unknown.err
index e69de29..54fe456 100644
--- a/tests/qapi-schema/union-unknown.err
+++ b/tests/qapi-schema/union-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/union-unknown.json:2: Member 'unknown' of union 'Union' uses unknown type 'MissingType'
diff --git a/tests/qapi-schema/union-unknown.exit b/tests/qapi-schema/union-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/union-unknown.exit
+++ b/tests/qapi-schema/union-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/union-unknown.json b/tests/qapi-schema/union-unknown.json
index 258f1d3..aa7e814 100644
--- a/tests/qapi-schema/union-unknown.json
+++ b/tests/qapi-schema/union-unknown.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject a union with unknown type in branch
+# we reject a union with unknown type in branch
{ 'union': 'Union',
'data': { 'unknown': 'MissingType' } }
diff --git a/tests/qapi-schema/union-unknown.out b/tests/qapi-schema/union-unknown.out
index 8223dcf..e69de29 100644
--- a/tests/qapi-schema/union-unknown.out
+++ b/tests/qapi-schema/union-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('union', 'Union'), ('data', OrderedDict([('unknown', 'MissingType')]))])]
-[{'enum_name': 'UnionKind', 'enum_values': None}]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (19 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 8:48 ` Markus Armbruster
` (2 more replies)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary Eric Blake
` (8 subsequent siblings)
29 siblings, 3 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Previous commits demonstrated that the generator overlooked various
bad naming situations:
- types, commands, and events need a valid name
- union and alternate branches cannot be marked optional
The set of valid names includes [a-zA-Z0-9._-] (where '.' is
useful only in downstream extensions).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 57 ++++++++++++++++------
tests/qapi-schema/bad-ident.err | 1 +
tests/qapi-schema/bad-ident.exit | 2 +-
tests/qapi-schema/bad-ident.json | 2 +-
tests/qapi-schema/bad-ident.out | 3 --
tests/qapi-schema/flat-union-bad-discriminator.err | 2 +-
.../flat-union-optional-discriminator.err | 1 +
.../flat-union-optional-discriminator.exit | 2 +-
.../flat-union-optional-discriminator.json | 2 +-
.../flat-union-optional-discriminator.out | 5 --
tests/qapi-schema/union-optional-branch.err | 1 +
tests/qapi-schema/union-optional-branch.exit | 2 +-
tests/qapi-schema/union-optional-branch.json | 2 +-
tests/qapi-schema/union-optional-branch.out | 3 --
14 files changed, 53 insertions(+), 32 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index c42683b..ed5385a 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -15,6 +15,7 @@ import re
from ordereddict import OrderedDict
import os
import sys
+import string
builtin_types = {
'str': 'QTYPE_QSTRING',
@@ -276,8 +277,27 @@ def discriminator_find_enum_define(expr):
return find_enum(discriminator_type)
+valid_characters = set(string.ascii_letters + string.digits + '.' + '-' + '_')
+def check_name(expr_info, source, name, allow_optional = False):
+ membername = name
+
+ if not isinstance(name, str):
+ raise QAPIExprError(expr_info,
+ "%s requires a string name" % source)
+ if name == '**':
+ return
+ if name.startswith('*'):
+ membername = name[1:]
+ if not allow_optional:
+ raise QAPIExprError(expr_info,
+ "%s does not allow optional name '%s'"
+ % (source, name))
+ if not set(membername) <= valid_characters:
+ raise QAPIExprError(expr_info,
+ "%s uses invalid name '%s'" % (source, name))
+
def check_type(expr_info, source, data, allow_array = False,
- allowed_metas = [], allow_dict = True):
+ allowed_metas = [], allow_dict = True, allow_optional = False):
global all_names
if data is None:
@@ -317,21 +337,23 @@ def check_type(expr_info, source, data, allow_array = False,
raise QAPIExprError(expr_info,
"%s should be a type name" % source)
for (key, value) in data.items():
+ check_name(expr_info, "Member of %s" % source, key,
+ allow_optional=allow_optional)
check_type(expr_info, "Member '%s' of %s" % (key, source), value,
allow_array=True,
allowed_metas=['built-in', 'union', 'alternate', 'struct',
'enum'],
- allow_dict=True)
+ allow_dict=True, allow_optional=True)
def check_command(expr, expr_info):
name = expr['command']
check_type(expr_info, "'data' for command '%s'" % name,
expr.get('data'),
- allowed_metas=['union', 'struct'])
+ allowed_metas=['union', 'struct'], allow_optional=True)
check_type(expr_info, "'returns' for command '%s'" % name,
expr.get('returns'), allow_array=True,
allowed_metas=['built-in', 'union', 'alternate', 'struct',
- 'enum'])
+ 'enum'], allow_optional=True)
def check_event(expr, expr_info):
global events
@@ -345,7 +367,8 @@ def check_event(expr, expr_info):
% name)
events.append(name)
check_type(expr_info, "'data' for event '%s'" % name,
- expr.get('data'), allowed_metas=['union', 'struct'])
+ expr.get('data'), allowed_metas=['union', 'struct'],
+ allow_optional=True)
if params:
for argname, argentry, optional, structured in parse_args(params):
if structured:
@@ -385,12 +408,10 @@ def check_union(expr, expr_info):
"Base '%s' is not a valid base type"
% base)
- # The value of member 'discriminator' must name a member of the
- # base type.
- if not isinstance(discriminator, str):
- raise QAPIExprError(expr_info,
- "Flat union '%s' must have a string "
- "discriminator field" % name)
+ # The value of member 'discriminator' must name a non-optional
+ # member of the base type.
+ check_name(expr_info, "Discriminator of flat union '%s'" % name,
+ discriminator)
discriminator_type = base_fields.get(discriminator)
if not discriminator_type:
raise QAPIExprError(expr_info,
@@ -406,6 +427,8 @@ def check_union(expr, expr_info):
# Check every branch
for (key, value) in members.items():
+ check_name(expr_info, "Member of union '%s'" % name, key)
+
# Each value must name a known type
check_type(expr_info, "Member '%s' of union '%s'" % (key, name),
value, allow_array=True,
@@ -439,6 +462,8 @@ def check_alternate(expr, expr_info):
# Check every branch
for (key, value) in members.items():
+ check_name(expr_info, "Member of alternate '%s'" % name, key)
+
# Check for conflicts in the generated enum
c_key = _generate_enum_string(key)
if c_key in values:
@@ -485,7 +510,8 @@ def check_struct(expr, expr_info):
name = expr['type']
members = expr['data']
- check_type(expr_info, "'data' for type '%s'" % name, members)
+ check_type(expr_info, "'data' for type '%s'" % name, members,
+ allow_optional=True)
check_type(expr_info, "'base' for type '%s'" % name, expr.get('base'),
allowed_metas=['struct'], allow_dict=False)
@@ -676,8 +702,11 @@ def type_name(name):
return c_list_type(name[0])
return name
-def add_name(name, info, meta, implicit = False):
+def add_name(name, info, meta, implicit = False, source = None):
global all_names
+ if not source:
+ source = "'%s'" % meta
+ check_name(info, source, name)
if name in all_names:
raise QAPIExprError(info,
"%s '%s' is already defined"
@@ -691,7 +720,7 @@ def add_name(name, info, meta, implicit = False):
def add_struct(definition, info):
global struct_types
name = definition['type']
- add_name(name, info, 'struct')
+ add_name(name, info, 'struct', source="'type'")
struct_types.append(definition)
def find_struct(name):
diff --git a/tests/qapi-schema/bad-ident.err b/tests/qapi-schema/bad-ident.err
index e69de29..42b490c 100644
--- a/tests/qapi-schema/bad-ident.err
+++ b/tests/qapi-schema/bad-ident.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-ident.json:2: 'type' does not allow optional name '*oops'
diff --git a/tests/qapi-schema/bad-ident.exit b/tests/qapi-schema/bad-ident.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/bad-ident.exit
+++ b/tests/qapi-schema/bad-ident.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/bad-ident.json b/tests/qapi-schema/bad-ident.json
index 66333a7..7418364 100644
--- a/tests/qapi-schema/bad-ident.json
+++ b/tests/qapi-schema/bad-ident.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject creating a type name with bad name
+# we reject creating a type name with bad name
{ 'type': '*oops', 'data': { 'i': 'int' } }
diff --git a/tests/qapi-schema/bad-ident.out b/tests/qapi-schema/bad-ident.out
index 165e346..e69de29 100644
--- a/tests/qapi-schema/bad-ident.out
+++ b/tests/qapi-schema/bad-ident.out
@@ -1,3 +0,0 @@
-[OrderedDict([('type', '*oops'), ('data', OrderedDict([('i', 'int')]))])]
-[]
-[OrderedDict([('type', '*oops'), ('data', OrderedDict([('i', 'int')]))])]
diff --git a/tests/qapi-schema/flat-union-bad-discriminator.err b/tests/qapi-schema/flat-union-bad-discriminator.err
index 4f0c798..c38cc8e 100644
--- a/tests/qapi-schema/flat-union-bad-discriminator.err
+++ b/tests/qapi-schema/flat-union-bad-discriminator.err
@@ -1 +1 @@
-tests/qapi-schema/flat-union-bad-discriminator.json:11: Flat union 'TestUnion' must have a string discriminator field
+tests/qapi-schema/flat-union-bad-discriminator.json:11: Discriminator of flat union 'TestUnion' requires a string name
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.err b/tests/qapi-schema/flat-union-optional-discriminator.err
index e69de29..df290ce 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.err
+++ b/tests/qapi-schema/flat-union-optional-discriminator.err
@@ -0,0 +1 @@
+tests/qapi-schema/flat-union-optional-discriminator.json:5: Discriminator of flat union 'MyUnion' does not allow optional name '*switch'
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.exit b/tests/qapi-schema/flat-union-optional-discriminator.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.exit
+++ b/tests/qapi-schema/flat-union-optional-discriminator.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.json b/tests/qapi-schema/flat-union-optional-discriminator.json
index 4957c72..cbf0afa 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.json
+++ b/tests/qapi-schema/flat-union-optional-discriminator.json
@@ -1,4 +1,4 @@
-# FIXME: we should require the discriminator to be non-optional
+# we require the discriminator to be non-optional
{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
{ 'type': 'Base',
'data': { '*switch': 'Enum' } }
diff --git a/tests/qapi-schema/flat-union-optional-discriminator.out b/tests/qapi-schema/flat-union-optional-discriminator.out
index f4b6bed..e69de29 100644
--- a/tests/qapi-schema/flat-union-optional-discriminator.out
+++ b/tests/qapi-schema/flat-union-optional-discriminator.out
@@ -1,5 +0,0 @@
-[OrderedDict([('enum', 'Enum'), ('data', ['one', 'two'])]),
- OrderedDict([('type', 'Base'), ('data', OrderedDict([('*switch', 'Enum')]))]),
- OrderedDict([('union', 'MyUnion'), ('base', 'Base'), ('discriminator', '*switch'), ('data', OrderedDict([('one', 'int'), ('two', 'str')]))])]
-[{'enum_name': 'Enum', 'enum_values': ['one', 'two']}]
-[OrderedDict([('type', 'Base'), ('data', OrderedDict([('*switch', 'Enum')]))])]
diff --git a/tests/qapi-schema/union-optional-branch.err b/tests/qapi-schema/union-optional-branch.err
index e69de29..3ada133 100644
--- a/tests/qapi-schema/union-optional-branch.err
+++ b/tests/qapi-schema/union-optional-branch.err
@@ -0,0 +1 @@
+tests/qapi-schema/union-optional-branch.json:2: Member of union 'Union' does not allow optional name '*a'
diff --git a/tests/qapi-schema/union-optional-branch.exit b/tests/qapi-schema/union-optional-branch.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/union-optional-branch.exit
+++ b/tests/qapi-schema/union-optional-branch.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/union-optional-branch.json b/tests/qapi-schema/union-optional-branch.json
index c513db7..591615f 100644
--- a/tests/qapi-schema/union-optional-branch.json
+++ b/tests/qapi-schema/union-optional-branch.json
@@ -1,2 +1,2 @@
-# FIXME: union branches cannot be optional
+# union branches cannot be optional
{ 'union': 'Union', 'data': { '*a': 'int', 'b': 'str' } }
diff --git a/tests/qapi-schema/union-optional-branch.out b/tests/qapi-schema/union-optional-branch.out
index b03b5d2..e69de29 100644
--- a/tests/qapi-schema/union-optional-branch.out
+++ b/tests/qapi-schema/union-optional-branch.out
@@ -1,3 +0,0 @@
-[OrderedDict([('union', 'Union'), ('data', OrderedDict([('*a', 'int'), ('b', 'str')]))])]
-[{'enum_name': 'UnionKind', 'enum_values': None}]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (20 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 9:11 ` Markus Armbruster
2015-03-27 16:19 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 23/28] qapi: More rigorous checking for type safety bypass Eric Blake
` (7 subsequent siblings)
29 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
...or an array of dictionaries. Although we have to cater to
existing commands, returning a non-dictionary means the command
is not extensible (no new name/value pairs can be added if more
information must be returned in parallel). By making the
whitelist explicit, any new command that falls foul of this
practice will have to be self-documenting, which will encourage
developers to either justify the action or rework the design to
use a dictionary after all.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 30 ++++++++++++++++++++++++++++--
tests/qapi-schema/returns-alternate.err | 1 +
tests/qapi-schema/returns-alternate.exit | 2 +-
tests/qapi-schema/returns-alternate.json | 2 +-
tests/qapi-schema/returns-alternate.out | 4 ----
tests/qapi-schema/returns-int.json | 3 ++-
tests/qapi-schema/returns-int.out | 2 +-
tests/qapi-schema/returns-whitelist.err | 1 +
tests/qapi-schema/returns-whitelist.exit | 2 +-
tests/qapi-schema/returns-whitelist.json | 2 +-
tests/qapi-schema/returns-whitelist.out | 7 -------
11 files changed, 37 insertions(+), 19 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index ed5385a..9421431 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -33,6 +33,30 @@ builtin_types = {
'size': 'QTYPE_QINT',
}
+# Whitelist of commands allowed to return a non-dictionary
+returns_whitelist = [
+ # From QMP:
+ 'human-monitor-command',
+ 'query-migrate-cache-size',
+ 'query-tpm-models',
+ 'query-tpm-types',
+ 'ringbuf-read',
+
+ # From QGA:
+ 'guest-file-open',
+ 'guest-fsfreeze-freeze',
+ 'guest-fsfreeze-freeze-list',
+ 'guest-fsfreeze-status',
+ 'guest-fsfreeze-thaw',
+ 'guest-get-time',
+ 'guest-set-vcpus',
+ 'guest-sync',
+ 'guest-sync-delimited',
+
+ # From qapi-schema-test:
+ 'user_def_cmd3',
+]
+
enum_types = []
struct_types = []
union_types = []
@@ -350,10 +374,12 @@ def check_command(expr, expr_info):
check_type(expr_info, "'data' for command '%s'" % name,
expr.get('data'),
allowed_metas=['union', 'struct'], allow_optional=True)
+ returns_meta = ['union', 'struct']
+ if name in returns_whitelist:
+ returns_meta += ['built-in', 'alternate', 'enum']
check_type(expr_info, "'returns' for command '%s'" % name,
expr.get('returns'), allow_array=True,
- allowed_metas=['built-in', 'union', 'alternate', 'struct',
- 'enum'], allow_optional=True)
+ allowed_metas=returns_meta, allow_optional=True)
def check_event(expr, expr_info):
global events
diff --git a/tests/qapi-schema/returns-alternate.err b/tests/qapi-schema/returns-alternate.err
index e69de29..dfbb419 100644
--- a/tests/qapi-schema/returns-alternate.err
+++ b/tests/qapi-schema/returns-alternate.err
@@ -0,0 +1 @@
+tests/qapi-schema/returns-alternate.json:3: 'returns' for command 'oops' cannot use alternate type 'Alt'
diff --git a/tests/qapi-schema/returns-alternate.exit b/tests/qapi-schema/returns-alternate.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/returns-alternate.exit
+++ b/tests/qapi-schema/returns-alternate.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/returns-alternate.json b/tests/qapi-schema/returns-alternate.json
index b3b91fd..972390c 100644
--- a/tests/qapi-schema/returns-alternate.json
+++ b/tests/qapi-schema/returns-alternate.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject returns if it is an alternate type
+# we reject returns if it is an alternate type
{ 'alternate': 'Alt', 'data': { 'a': 'int', 'b': 'str' } }
{ 'command': 'oops', 'returns': 'Alt' }
diff --git a/tests/qapi-schema/returns-alternate.out b/tests/qapi-schema/returns-alternate.out
index 8a03ed3..e69de29 100644
--- a/tests/qapi-schema/returns-alternate.out
+++ b/tests/qapi-schema/returns-alternate.out
@@ -1,4 +0,0 @@
-[OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('a', 'int'), ('b', 'str')]))]),
- OrderedDict([('command', 'oops'), ('returns', 'Alt')])]
-[{'enum_name': 'AltKind', 'enum_values': None}]
-[]
diff --git a/tests/qapi-schema/returns-int.json b/tests/qapi-schema/returns-int.json
index 7888fb1..870ec63 100644
--- a/tests/qapi-schema/returns-int.json
+++ b/tests/qapi-schema/returns-int.json
@@ -1,2 +1,3 @@
# It is okay (although not extensible) to return a non-dictionary
-{ 'command': 'okay', 'returns': 'int' }
+# But to make it work, the name must be in a whitelist
+{ 'command': 'guest-get-time', 'returns': 'int' }
diff --git a/tests/qapi-schema/returns-int.out b/tests/qapi-schema/returns-int.out
index 36b00a9..70b3ac5 100644
--- a/tests/qapi-schema/returns-int.out
+++ b/tests/qapi-schema/returns-int.out
@@ -1,3 +1,3 @@
-[OrderedDict([('command', 'okay'), ('returns', 'int')])]
+[OrderedDict([('command', 'guest-get-time'), ('returns', 'int')])]
[]
[]
diff --git a/tests/qapi-schema/returns-whitelist.err b/tests/qapi-schema/returns-whitelist.err
index e69de29..f47c1ee 100644
--- a/tests/qapi-schema/returns-whitelist.err
+++ b/tests/qapi-schema/returns-whitelist.err
@@ -0,0 +1 @@
+tests/qapi-schema/returns-whitelist.json:10: 'returns' for command 'no-way-this-will-get-whitelisted' cannot use built-in type 'int'
diff --git a/tests/qapi-schema/returns-whitelist.exit b/tests/qapi-schema/returns-whitelist.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/returns-whitelist.exit
+++ b/tests/qapi-schema/returns-whitelist.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/returns-whitelist.json b/tests/qapi-schema/returns-whitelist.json
index 8328563..e8b3cea 100644
--- a/tests/qapi-schema/returns-whitelist.json
+++ b/tests/qapi-schema/returns-whitelist.json
@@ -1,4 +1,4 @@
-# FIXME: we should enforce that 'returns' be a dict or array of dict unless whitelisted
+# we enforce that 'returns' be a dict or array of dict unless whitelisted
{ 'command': 'human-monitor-command',
'data': {'command-line': 'str', '*cpu-index': 'int'},
'returns': 'str' }
diff --git a/tests/qapi-schema/returns-whitelist.out b/tests/qapi-schema/returns-whitelist.out
index 2adcd8b..e69de29 100644
--- a/tests/qapi-schema/returns-whitelist.out
+++ b/tests/qapi-schema/returns-whitelist.out
@@ -1,7 +0,0 @@
-[OrderedDict([('command', 'human-monitor-command'), ('data', OrderedDict([('command-line', 'str'), ('*cpu-index', 'int')])), ('returns', 'str')]),
- OrderedDict([('enum', 'TpmModel'), ('data', ['tpm-tis'])]),
- OrderedDict([('command', 'query-tpm-models'), ('returns', ['TpmModel'])]),
- OrderedDict([('command', 'guest-get-time'), ('returns', 'int')]),
- OrderedDict([('command', 'no-way-this-will-get-whitelisted'), ('returns', ['int'])])]
-[{'enum_name': 'TpmModel', 'enum_values': ['tpm-tis']}]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 23/28] qapi: More rigorous checking for type safety bypass
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (21 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 9:45 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
` (6 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
Now that we have a way to validate every type, we can also be
stricter about enforcing that callers that want to bypass
type safety in generated code. Prior to this patch, it didn't
matter what value was associated with the key 'gen', but it
looked odd that 'gen':'yes' could result in bypassing the
generated code. These changes also enforce the changes made
earlier in the series for documentation and consolidation of
using '**' as the wildcard type, as well as 'gen':false as the
canonical spelling for requesting type bypass.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 23 ++++++++++++++++++-----
tests/qapi-schema/type-bypass-bad-gen.err | 1 +
tests/qapi-schema/type-bypass-bad-gen.exit | 2 +-
tests/qapi-schema/type-bypass-bad-gen.json | 2 +-
tests/qapi-schema/type-bypass-bad-gen.out | 3 ---
tests/qapi-schema/type-bypass-no-gen.err | 1 +
tests/qapi-schema/type-bypass-no-gen.exit | 2 +-
tests/qapi-schema/type-bypass-no-gen.json | 2 +-
tests/qapi-schema/type-bypass-no-gen.out | 3 ---
9 files changed, 24 insertions(+), 15 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 9421431..800e8e4 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -321,13 +321,14 @@ def check_name(expr_info, source, name, allow_optional = False):
"%s uses invalid name '%s'" % (source, name))
def check_type(expr_info, source, data, allow_array = False,
- allowed_metas = [], allow_dict = True, allow_optional = False):
+ allowed_metas = [], allow_dict = True, allow_optional = False,
+ allow_star = False):
global all_names
if data is None:
return
- if data == '**':
+ if allow_star and data == '**':
return
# Check if array type for data is okay
@@ -343,6 +344,10 @@ def check_type(expr_info, source, data, allow_array = False,
# Check if type name for data is okay
if isinstance(data, str):
+ if data == '**':
+ raise QAPIExprError(expr_info,
+ "%s uses '**' but did not request 'gen':false"
+ % source)
if not data in all_names:
raise QAPIExprError(expr_info,
"%s uses unknown type '%s'"
@@ -367,19 +372,23 @@ def check_type(expr_info, source, data, allow_array = False,
allow_array=True,
allowed_metas=['built-in', 'union', 'alternate', 'struct',
'enum'],
- allow_dict=True, allow_optional=True)
+ allow_dict=True, allow_optional=True, allow_star=allow_star)
def check_command(expr, expr_info):
name = expr['command']
+ allow_star = expr.has_key('gen')
+
check_type(expr_info, "'data' for command '%s'" % name,
expr.get('data'),
- allowed_metas=['union', 'struct'], allow_optional=True)
+ allowed_metas=['union', 'struct'], allow_optional=True,
+ allow_star=allow_star)
returns_meta = ['union', 'struct']
if name in returns_whitelist:
returns_meta += ['built-in', 'alternate', 'enum']
check_type(expr_info, "'returns' for command '%s'" % name,
expr.get('returns'), allow_array=True,
- allowed_metas=returns_meta, allow_optional=True)
+ allowed_metas=returns_meta, allow_optional=True,
+ allow_star=allow_star)
def check_event(expr, expr_info):
global events
@@ -574,6 +583,10 @@ def check_keys(expr_elem, meta, required, optional=[]):
raise QAPIExprError(info,
"%s '%s' has unknown key '%s'"
% (meta, name, key))
+ if (key == 'gen' or key == 'success-response') and value != False:
+ raise QAPIExprError(info,
+ "'%s' of %s '%s' should only use false value"
+ % (key, meta, name))
for key in required:
if not expr.has_key(key):
raise QAPIExprError(info,
diff --git a/tests/qapi-schema/type-bypass-bad-gen.err b/tests/qapi-schema/type-bypass-bad-gen.err
index e69de29..a83c3c6 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.err
+++ b/tests/qapi-schema/type-bypass-bad-gen.err
@@ -0,0 +1 @@
+tests/qapi-schema/type-bypass-bad-gen.json:2: 'gen' of command 'foo' should only use false value
diff --git a/tests/qapi-schema/type-bypass-bad-gen.exit b/tests/qapi-schema/type-bypass-bad-gen.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.exit
+++ b/tests/qapi-schema/type-bypass-bad-gen.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/type-bypass-bad-gen.json b/tests/qapi-schema/type-bypass-bad-gen.json
index bb70bee..e8dec34 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.json
+++ b/tests/qapi-schema/type-bypass-bad-gen.json
@@ -1,2 +1,2 @@
-# FIXME: 'gen' should only appear with value false
+# 'gen' should only appear with value false
{ 'command': 'foo', 'gen': 'whatever' }
diff --git a/tests/qapi-schema/type-bypass-bad-gen.out b/tests/qapi-schema/type-bypass-bad-gen.out
index e678f2c..e69de29 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.out
+++ b/tests/qapi-schema/type-bypass-bad-gen.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('gen', 'whatever')])]
-[]
-[]
diff --git a/tests/qapi-schema/type-bypass-no-gen.err b/tests/qapi-schema/type-bypass-no-gen.err
index e69de29..20cef0a 100644
--- a/tests/qapi-schema/type-bypass-no-gen.err
+++ b/tests/qapi-schema/type-bypass-no-gen.err
@@ -0,0 +1 @@
+tests/qapi-schema/type-bypass-no-gen.json:2: Member 'arg' of 'data' for command 'unsafe' uses '**' but did not request 'gen':false
diff --git a/tests/qapi-schema/type-bypass-no-gen.exit b/tests/qapi-schema/type-bypass-no-gen.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/type-bypass-no-gen.exit
+++ b/tests/qapi-schema/type-bypass-no-gen.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/type-bypass-no-gen.json b/tests/qapi-schema/type-bypass-no-gen.json
index af87c19..49b5742 100644
--- a/tests/qapi-schema/type-bypass-no-gen.json
+++ b/tests/qapi-schema/type-bypass-no-gen.json
@@ -1,2 +1,2 @@
-# FIXME: type bypass should only work with 'gen':false
+# type bypass only works with 'gen':'no'
{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**' }
diff --git a/tests/qapi-schema/type-bypass-no-gen.out b/tests/qapi-schema/type-bypass-no-gen.out
index 8b2a9ac..e69de29 100644
--- a/tests/qapi-schema/type-bypass-no-gen.out
+++ b/tests/qapi-schema/type-bypass-no-gen.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**')])]
-[]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (22 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 23/28] qapi: More rigorous checking for type safety bypass Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 9:52 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 25/28] qapi: Drop tests for inline nested structs Eric Blake
` (5 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
In the testsuite, UserDefTwo and UserDefNested were identical
types other than the member names. Reduce code duplication by
having just one type, and choose names that also favor reuse.
This will also make it easier for a later patch to get rid of
inline nested types in QAPI; it means that the type is now boxed
instead of unboxed in C code, but has no difference to the QMP
wire protocol. When touching code to add new allocations, also
convert existing allocations to consistently prefer typesafe
g_new0 over g_malloc0.
Ensure that 'make check-qapi-schema check-unit' still passes.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qapi-schema/qapi-schema-test.json | 10 +----
tests/qapi-schema/qapi-schema-test.out | 6 +--
tests/test-qmp-commands.c | 32 +++++++--------
tests/test-qmp-input-strict.c | 17 ++++----
tests/test-qmp-input-visitor.c | 17 ++++----
tests/test-qmp-output-visitor.c | 48 +++++++++++-----------
tests/test-visitor-serialization.c | 71 +++++++++++++++++----------------
7 files changed, 100 insertions(+), 101 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index dec8a7c..fb6a350 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -15,16 +15,10 @@
'data': { 'string': 'str', '*enum1': 'EnumOne' } }
{ 'type': 'UserDefTwo',
- 'data': { 'string': 'str',
- 'dict': { 'string': 'str',
- 'dict': { 'userdef': 'UserDefOne', 'string': 'str' },
- '*dict2': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
-
-{ 'type': 'UserDefNested',
'data': { 'string0': 'str',
'dict1': { 'string1': 'str',
- 'dict2': { 'userdef1': 'UserDefOne', 'string2': 'str' },
- '*dict3': { 'userdef2': 'UserDefOne', 'string3': 'str' } } } }
+ 'dict2': { 'userdef': 'UserDefOne', 'string': 'str' },
+ '*dict3': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
# for testing unions
{ 'type': 'UserDefA',
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 313ecf3..4d73bc0 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -2,8 +2,7 @@
OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string', 'str'), ('dict', OrderedDict([('string', 'str'), ('dict', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
- OrderedDict([('type', 'UserDefNested'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef1', 'UserDefOne'), ('string2', 'str')])), ('*dict3', OrderedDict([('userdef2', 'UserDefOne'), ('string3', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
@@ -28,8 +27,7 @@
[OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string', 'str'), ('dict', OrderedDict([('string', 'str'), ('dict', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
- OrderedDict([('type', 'UserDefNested'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef1', 'UserDefOne'), ('string2', 'str')])), ('*dict3', OrderedDict([('userdef2', 'UserDefOne'), ('string3', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 554e222..9189cd2 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -32,13 +32,13 @@ UserDefTwo *qmp_user_def_cmd2(UserDefOne *ud1a,
ud1d->base->integer = has_udb1 ? ud1b->base->integer : 0;
ret = g_malloc0(sizeof(UserDefTwo));
- ret->string = strdup("blah1");
- ret->dict.string = strdup("blah2");
- ret->dict.dict.userdef = ud1c;
- ret->dict.dict.string = strdup("blah3");
- ret->dict.has_dict2 = true;
- ret->dict.dict2.userdef = ud1d;
- ret->dict.dict2.string = strdup("blah4");
+ ret->string0 = strdup("blah1");
+ ret->dict1.string1 = strdup("blah2");
+ ret->dict1.dict2.userdef = ud1c;
+ ret->dict1.dict2.string = strdup("blah3");
+ ret->dict1.has_dict3 = true;
+ ret->dict1.dict3.userdef = ud1d;
+ ret->dict1.dict3.string = strdup("blah4");
return ret;
}
@@ -120,15 +120,15 @@ static void test_dispatch_cmd_io(void)
ret = qobject_to_qdict(test_qmp_dispatch(req));
- assert(!strcmp(qdict_get_str(ret, "string"), "blah1"));
- ret_dict = qdict_get_qdict(ret, "dict");
- assert(!strcmp(qdict_get_str(ret_dict, "string"), "blah2"));
- ret_dict_dict = qdict_get_qdict(ret_dict, "dict");
+ assert(!strcmp(qdict_get_str(ret, "string0"), "blah1"));
+ ret_dict = qdict_get_qdict(ret, "dict1");
+ assert(!strcmp(qdict_get_str(ret_dict, "string1"), "blah2"));
+ ret_dict_dict = qdict_get_qdict(ret_dict, "dict2");
ret_dict_dict_userdef = qdict_get_qdict(ret_dict_dict, "userdef");
assert(qdict_get_int(ret_dict_dict_userdef, "integer") == 42);
assert(!strcmp(qdict_get_str(ret_dict_dict_userdef, "string"), "hello"));
assert(!strcmp(qdict_get_str(ret_dict_dict, "string"), "blah3"));
- ret_dict_dict2 = qdict_get_qdict(ret_dict, "dict2");
+ ret_dict_dict2 = qdict_get_qdict(ret_dict, "dict3");
ret_dict_dict2_userdef = qdict_get_qdict(ret_dict_dict2, "userdef");
assert(qdict_get_int(ret_dict_dict2_userdef, "integer") == 422);
assert(!strcmp(qdict_get_str(ret_dict_dict2_userdef, "string"), "hello2"));
@@ -192,7 +192,7 @@ static void test_dealloc_partial(void)
QmpInputVisitor *qiv;
ud2_dict = qdict_new();
- qdict_put_obj(ud2_dict, "string", QOBJECT(qstring_from_str(text)));
+ qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
qiv = qmp_input_visitor_new(QOBJECT(ud2_dict));
visit_type_UserDefTwo(qmp_input_get_visitor(qiv), &ud2, NULL, &err);
@@ -202,9 +202,9 @@ static void test_dealloc_partial(void)
/* verify partial success */
assert(ud2 != NULL);
- assert(ud2->string != NULL);
- assert(strcmp(ud2->string, text) == 0);
- assert(ud2->dict.dict.userdef == NULL);
+ assert(ud2->string0 != NULL);
+ assert(strcmp(ud2->string0, text) == 0);
+ assert(ud2->dict1.dict2.userdef == NULL);
/* confirm & release construction error */
assert(err != NULL);
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 4b53560..7965df0 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -116,15 +116,18 @@ static void test_validate_struct(TestInputVisitorData *data,
static void test_validate_struct_nested(TestInputVisitorData *data,
const void *unused)
{
- UserDefNested *udp = NULL;
+ UserDefTwo *udp = NULL;
Error *err = NULL;
Visitor *v;
- v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string' }, 'string2': 'string2'}}}");
+ v = validate_test_init(data, "{ 'string0': 'string0', "
+ "'dict1': { 'string1': 'string1', "
+ "'dict2': { 'userdef': { 'integer': 42, "
+ "'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefNested(v, &udp, NULL, &err);
+ visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(!err);
- qapi_free_UserDefNested(udp);
+ qapi_free_UserDefTwo(udp);
}
static void test_validate_list(TestInputVisitorData *data,
@@ -207,15 +210,15 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
static void test_validate_fail_struct_nested(TestInputVisitorData *data,
const void *unused)
{
- UserDefNested *udp = NULL;
+ UserDefTwo *udp = NULL;
Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");
- visit_type_UserDefNested(v, &udp, NULL, &err);
+ visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(err);
- qapi_free_UserDefNested(udp);
+ qapi_free_UserDefTwo(udp);
}
static void test_validate_fail_list(TestInputVisitorData *data,
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 882359a..115fb22 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -248,23 +248,26 @@ static void check_and_free_str(char *str, const char *cmp)
static void test_visitor_in_struct_nested(TestInputVisitorData *data,
const void *unused)
{
- UserDefNested *udp = NULL;
+ UserDefTwo *udp = NULL;
Error *err = NULL;
Visitor *v;
- v = visitor_input_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string' }, 'string2': 'string2'}}}");
+ v = visitor_input_test_init(data, "{ 'string0': 'string0', "
+ "'dict1': { 'string1': 'string1', "
+ "'dict2': { 'userdef': { 'integer': 42, "
+ "'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefNested(v, &udp, NULL, &err);
+ visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(!err);
check_and_free_str(udp->string0, "string0");
check_and_free_str(udp->dict1.string1, "string1");
- g_assert_cmpint(udp->dict1.dict2.userdef1->base->integer, ==, 42);
- check_and_free_str(udp->dict1.dict2.userdef1->string, "string");
- check_and_free_str(udp->dict1.dict2.string2, "string2");
+ g_assert_cmpint(udp->dict1.dict2.userdef->base->integer, ==, 42);
+ check_and_free_str(udp->dict1.dict2.userdef->string, "string");
+ check_and_free_str(udp->dict1.dict2.string, "string2");
g_assert(udp->dict1.has_dict3 == false);
- g_free(udp->dict1.dict2.userdef1);
+ g_free(udp->dict1.dict2.userdef);
g_free(udp);
}
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 602bc12..525c36a 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -234,7 +234,7 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
{
int64_t value = 42;
Error *err = NULL;
- UserDefNested *ud2;
+ UserDefTwo *ud2;
QObject *obj;
QDict *qdict, *dict1, *dict2, *dict3, *userdef;
const char *string = "user def string";
@@ -245,20 +245,20 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
ud2->string0 = g_strdup(strings[0]);
ud2->dict1.string1 = g_strdup(strings[1]);
- ud2->dict1.dict2.userdef1 = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict2.userdef1->string = g_strdup(string);
- ud2->dict1.dict2.userdef1->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict2.userdef1->base->integer = value;
- ud2->dict1.dict2.string2 = g_strdup(strings[2]);
+ ud2->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1.dict2.userdef->string = g_strdup(string);
+ ud2->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1.dict2.userdef->base->integer = value;
+ ud2->dict1.dict2.string = g_strdup(strings[2]);
ud2->dict1.has_dict3 = true;
- ud2->dict1.dict3.userdef2 = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict3.userdef2->string = g_strdup(string);
- ud2->dict1.dict3.userdef2->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict3.userdef2->base->integer = value;
- ud2->dict1.dict3.string3 = g_strdup(strings[3]);
+ ud2->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1.dict3.userdef->string = g_strdup(string);
+ ud2->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1.dict3.userdef->base->integer = value;
+ ud2->dict1.dict3.string = g_strdup(strings[3]);
- visit_type_UserDefNested(data->ov, &ud2, "unused", &err);
+ visit_type_UserDefTwo(data->ov, &ud2, "unused", &err);
g_assert(!err);
obj = qmp_output_get_qobject(data->qov);
@@ -275,22 +275,22 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
dict2 = qdict_get_qdict(dict1, "dict2");
g_assert_cmpint(qdict_size(dict2), ==, 2);
- g_assert_cmpstr(qdict_get_str(dict2, "string2"), ==, strings[2]);
- userdef = qdict_get_qdict(dict2, "userdef1");
+ g_assert_cmpstr(qdict_get_str(dict2, "string"), ==, strings[2]);
+ userdef = qdict_get_qdict(dict2, "userdef");
g_assert_cmpint(qdict_size(userdef), ==, 2);
g_assert_cmpint(qdict_get_int(userdef, "integer"), ==, value);
g_assert_cmpstr(qdict_get_str(userdef, "string"), ==, string);
dict3 = qdict_get_qdict(dict1, "dict3");
g_assert_cmpint(qdict_size(dict3), ==, 2);
- g_assert_cmpstr(qdict_get_str(dict3, "string3"), ==, strings[3]);
- userdef = qdict_get_qdict(dict3, "userdef2");
+ g_assert_cmpstr(qdict_get_str(dict3, "string"), ==, strings[3]);
+ userdef = qdict_get_qdict(dict3, "userdef");
g_assert_cmpint(qdict_size(userdef), ==, 2);
g_assert_cmpint(qdict_get_int(userdef, "integer"), ==, value);
g_assert_cmpstr(qdict_get_str(userdef, "string"), ==, string);
QDECREF(qdict);
- qapi_free_UserDefNested(ud2);
+ qapi_free_UserDefTwo(ud2);
}
static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
@@ -398,7 +398,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
const void *unused)
{
- UserDefNestedList *p, *head = NULL;
+ UserDefTwoList *p, *head = NULL;
const char string[] = "foo bar";
int i, max_count = 1024;
@@ -408,18 +408,18 @@ static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
p->value->string0 = g_strdup(string);
p->value->dict1.string1 = g_strdup(string);
- p->value->dict1.dict2.userdef1 = g_malloc0(sizeof(UserDefOne));
- p->value->dict1.dict2.userdef1->string = g_strdup(string);
- p->value->dict1.dict2.userdef1->base = g_new0(UserDefZero, 1);
- p->value->dict1.dict2.userdef1->base->integer = 42;
- p->value->dict1.dict2.string2 = g_strdup(string);
+ p->value->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
+ p->value->dict1.dict2.userdef->string = g_strdup(string);
+ p->value->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
+ p->value->dict1.dict2.userdef->base->integer = 42;
+ p->value->dict1.dict2.string = g_strdup(string);
p->value->dict1.has_dict3 = false;
p->next = head;
head = p;
}
- qapi_free_UserDefNestedList(head);
+ qapi_free_UserDefTwoList(head);
}
static void test_visitor_out_union_flat(TestOutputVisitorData *data,
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 7ad1886..133f882 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -249,57 +249,57 @@ static void visit_struct(Visitor *v, void **native, Error **errp)
visit_type_TestStruct(v, (TestStruct **)native, NULL, errp);
}
-static UserDefNested *nested_struct_create(void)
+static UserDefTwo *nested_struct_create(void)
{
- UserDefNested *udnp = g_malloc0(sizeof(*udnp));
+ UserDefTwo *udnp = g_malloc0(sizeof(*udnp));
udnp->string0 = strdup("test_string0");
udnp->dict1.string1 = strdup("test_string1");
- udnp->dict1.dict2.userdef1 = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict2.userdef1->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict2.userdef1->base->integer = 42;
- udnp->dict1.dict2.userdef1->string = strdup("test_string");
- udnp->dict1.dict2.string2 = strdup("test_string2");
+ udnp->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1.dict2.userdef->base->integer = 42;
+ udnp->dict1.dict2.userdef->string = strdup("test_string");
+ udnp->dict1.dict2.string = strdup("test_string2");
udnp->dict1.has_dict3 = true;
- udnp->dict1.dict3.userdef2 = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict3.userdef2->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict3.userdef2->base->integer = 43;
- udnp->dict1.dict3.userdef2->string = strdup("test_string");
- udnp->dict1.dict3.string3 = strdup("test_string3");
+ udnp->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1.dict3.userdef->base->integer = 43;
+ udnp->dict1.dict3.userdef->string = strdup("test_string");
+ udnp->dict1.dict3.string = strdup("test_string3");
return udnp;
}
-static void nested_struct_compare(UserDefNested *udnp1, UserDefNested *udnp2)
+static void nested_struct_compare(UserDefTwo *udnp1, UserDefTwo *udnp2)
{
g_assert(udnp1);
g_assert(udnp2);
g_assert_cmpstr(udnp1->string0, ==, udnp2->string0);
g_assert_cmpstr(udnp1->dict1.string1, ==, udnp2->dict1.string1);
- g_assert_cmpint(udnp1->dict1.dict2.userdef1->base->integer, ==,
- udnp2->dict1.dict2.userdef1->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict2.userdef1->string, ==,
- udnp2->dict1.dict2.userdef1->string);
- g_assert_cmpstr(udnp1->dict1.dict2.string2, ==, udnp2->dict1.dict2.string2);
+ g_assert_cmpint(udnp1->dict1.dict2.userdef->base->integer, ==,
+ udnp2->dict1.dict2.userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1.dict2.userdef->string, ==,
+ udnp2->dict1.dict2.userdef->string);
+ g_assert_cmpstr(udnp1->dict1.dict2.string, ==, udnp2->dict1.dict2.string);
g_assert(udnp1->dict1.has_dict3 == udnp2->dict1.has_dict3);
- g_assert_cmpint(udnp1->dict1.dict3.userdef2->base->integer, ==,
- udnp2->dict1.dict3.userdef2->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict3.userdef2->string, ==,
- udnp2->dict1.dict3.userdef2->string);
- g_assert_cmpstr(udnp1->dict1.dict3.string3, ==, udnp2->dict1.dict3.string3);
+ g_assert_cmpint(udnp1->dict1.dict3.userdef->base->integer, ==,
+ udnp2->dict1.dict3.userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1.dict3.userdef->string, ==,
+ udnp2->dict1.dict3.userdef->string);
+ g_assert_cmpstr(udnp1->dict1.dict3.string, ==, udnp2->dict1.dict3.string);
}
-static void nested_struct_cleanup(UserDefNested *udnp)
+static void nested_struct_cleanup(UserDefTwo *udnp)
{
- qapi_free_UserDefNested(udnp);
+ qapi_free_UserDefTwo(udnp);
}
static void visit_nested_struct(Visitor *v, void **native, Error **errp)
{
- visit_type_UserDefNested(v, (UserDefNested **)native, NULL, errp);
+ visit_type_UserDefTwo(v, (UserDefTwo **)native, NULL, errp);
}
static void visit_nested_struct_list(Visitor *v, void **native, Error **errp)
{
- visit_type_UserDefNestedList(v, (UserDefNestedList **)native, NULL, errp);
+ visit_type_UserDefTwoList(v, (UserDefTwoList **)native, NULL, errp);
}
/* test cases */
@@ -715,13 +715,14 @@ static void test_nested_struct(gconstpointer opaque)
{
TestArgs *args = (TestArgs *) opaque;
const SerializeOps *ops = args->ops;
- UserDefNested *udnp = nested_struct_create();
- UserDefNested *udnp_copy = NULL;
+ UserDefTwo *udnp = nested_struct_create();
+ UserDefTwo *udnp_copy = NULL;
Error *err = NULL;
void *serialize_data;
-
+
ops->serialize(udnp, &serialize_data, visit_nested_struct, &err);
- ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct, &err);
+ ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct,
+ &err);
g_assert(err == NULL);
nested_struct_compare(udnp, udnp_copy);
@@ -737,13 +738,13 @@ static void test_nested_struct_list(gconstpointer opaque)
{
TestArgs *args = (TestArgs *) opaque;
const SerializeOps *ops = args->ops;
- UserDefNestedList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
+ UserDefTwoList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
Error *err = NULL;
void *serialize_data;
int i = 0;
for (i = 0; i < 8; i++) {
- tmp = g_malloc0(sizeof(UserDefNestedList));
+ tmp = g_malloc0(sizeof(UserDefTwoList));
tmp->value = nested_struct_create();
tmp->next = listp;
listp = tmp;
@@ -764,8 +765,8 @@ static void test_nested_struct_list(gconstpointer opaque)
listp_copy = listp_copy->next;
}
- qapi_free_UserDefNestedList(tmp);
- qapi_free_UserDefNestedList(tmp_copy);
+ qapi_free_UserDefTwoList(tmp);
+ qapi_free_UserDefTwoList(tmp_copy);
ops->cleanup(serialize_data);
g_free(args);
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 25/28] qapi: Drop tests for inline nested structs
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (23 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 10:30 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 26/28] qapi: Drop inline nested type in query-version Eric Blake
` (4 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline nested structs conflicts with that goal.
This patch fixes the testsuite to avoid inline nested types, by
breaking the nesting into explicit types; it means that the type
is now boxed instead of unboxed in C code, but makes no difference
on the wire. When touching code to add new allocations, also
convert existing allocations to consistently prefer typesafe
g_new0 over g_malloc0.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qapi-schema/qapi-schema-test.json | 12 ++++++--
tests/qapi-schema/qapi-schema-test.out | 8 +++--
tests/test-qmp-commands.c | 17 ++++++-----
tests/test-qmp-input-visitor.c | 14 +++++----
tests/test-qmp-output-visitor.c | 44 +++++++++++++++------------
tests/test-visitor-serialization.c | 53 ++++++++++++++++++---------------
6 files changed, 87 insertions(+), 61 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index fb6a350..7aeb490 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -14,11 +14,17 @@
'base': 'UserDefZero',
'data': { 'string': 'str', '*enum1': 'EnumOne' } }
+{ 'type': 'UserDefTwoDictDict',
+ 'data': { 'userdef': 'UserDefOne', 'string': 'str' } }
+
+{ 'type': 'UserDefTwoDict',
+ 'data': { 'string1': 'str',
+ 'dict2': 'UserDefTwoDictDict',
+ '*dict3': 'UserDefTwoDictDict' } }
+
{ 'type': 'UserDefTwo',
'data': { 'string0': 'str',
- 'dict1': { 'string1': 'str',
- 'dict2': { 'userdef': 'UserDefOne', 'string': 'str' },
- '*dict3': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
+ 'dict1': 'UserDefTwoDict' } }
# for testing unions
{ 'type': 'UserDefA',
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 4d73bc0..251f6f1 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -2,7 +2,9 @@
OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwoDictDict'), ('data', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]),
+ OrderedDict([('type', 'UserDefTwoDict'), ('data', OrderedDict([('string1', 'str'), ('dict2', 'UserDefTwoDictDict'), ('*dict3', 'UserDefTwoDictDict')]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', 'UserDefTwoDict')]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
@@ -27,7 +29,9 @@
[OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwoDictDict'), ('data', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]),
+ OrderedDict([('type', 'UserDefTwoDict'), ('data', OrderedDict([('string1', 'str'), ('dict2', 'UserDefTwoDictDict'), ('*dict3', 'UserDefTwoDictDict')]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', 'UserDefTwoDict')]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefC'), ('data', OrderedDict([('string1', 'str'), ('string2', 'str')]))]),
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 9189cd2..dc199d3 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -33,12 +33,15 @@ UserDefTwo *qmp_user_def_cmd2(UserDefOne *ud1a,
ret = g_malloc0(sizeof(UserDefTwo));
ret->string0 = strdup("blah1");
- ret->dict1.string1 = strdup("blah2");
- ret->dict1.dict2.userdef = ud1c;
- ret->dict1.dict2.string = strdup("blah3");
- ret->dict1.has_dict3 = true;
- ret->dict1.dict3.userdef = ud1d;
- ret->dict1.dict3.string = strdup("blah4");
+ ret->dict1 = g_malloc0(sizeof(UserDefTwoDict));
+ ret->dict1->string1 = strdup("blah2");
+ ret->dict1->dict2 = g_malloc0(sizeof(UserDefTwoDictDict));
+ ret->dict1->dict2->userdef = ud1c;
+ ret->dict1->dict2->string = strdup("blah3");
+ ret->dict1->dict3 = g_malloc0(sizeof(UserDefTwoDictDict));
+ ret->dict1->has_dict3 = true;
+ ret->dict1->dict3->userdef = ud1d;
+ ret->dict1->dict3->string = strdup("blah4");
return ret;
}
@@ -204,7 +207,7 @@ static void test_dealloc_partial(void)
assert(ud2 != NULL);
assert(ud2->string0 != NULL);
assert(strcmp(ud2->string0, text) == 0);
- assert(ud2->dict1.dict2.userdef == NULL);
+ assert(ud2->dict1 == NULL);
/* confirm & release construction error */
assert(err != NULL);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 115fb22..b961953 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -261,13 +261,15 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
g_assert(!err);
check_and_free_str(udp->string0, "string0");
- check_and_free_str(udp->dict1.string1, "string1");
- g_assert_cmpint(udp->dict1.dict2.userdef->base->integer, ==, 42);
- check_and_free_str(udp->dict1.dict2.userdef->string, "string");
- check_and_free_str(udp->dict1.dict2.string, "string2");
- g_assert(udp->dict1.has_dict3 == false);
+ check_and_free_str(udp->dict1->string1, "string1");
+ g_assert_cmpint(udp->dict1->dict2->userdef->base->integer, ==, 42);
+ check_and_free_str(udp->dict1->dict2->userdef->string, "string");
+ check_and_free_str(udp->dict1->dict2->string, "string2");
+ g_assert(udp->dict1->has_dict3 == false);
- g_free(udp->dict1.dict2.userdef);
+ g_free(udp->dict1->dict2->userdef);
+ g_free(udp->dict1->dict2);
+ g_free(udp->dict1);
g_free(udp);
}
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 525c36a..955d4f0 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -244,19 +244,23 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
ud2 = g_malloc0(sizeof(*ud2));
ud2->string0 = g_strdup(strings[0]);
- ud2->dict1.string1 = g_strdup(strings[1]);
- ud2->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict2.userdef->string = g_strdup(string);
- ud2->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict2.userdef->base->integer = value;
- ud2->dict1.dict2.string = g_strdup(strings[2]);
+ ud2->dict1 = g_malloc0(sizeof(*ud2->dict1));
+ ud2->dict1->string1 = g_strdup(strings[1]);
- ud2->dict1.has_dict3 = true;
- ud2->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict3.userdef->string = g_strdup(string);
- ud2->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict3.userdef->base->integer = value;
- ud2->dict1.dict3.string = g_strdup(strings[3]);
+ ud2->dict1->dict2 = g_malloc0(sizeof(*ud2->dict1->dict2));
+ ud2->dict1->dict2->userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1->dict2->userdef->string = g_strdup(string);
+ ud2->dict1->dict2->userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1->dict2->userdef->base->integer = value;
+ ud2->dict1->dict2->string = g_strdup(strings[2]);
+
+ ud2->dict1->dict3 = g_malloc0(sizeof(*ud2->dict1->dict3));
+ ud2->dict1->has_dict3 = true;
+ ud2->dict1->dict3->userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1->dict3->userdef->string = g_strdup(string);
+ ud2->dict1->dict3->userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1->dict3->userdef->base->integer = value;
+ ud2->dict1->dict3->string = g_strdup(strings[3]);
visit_type_UserDefTwo(data->ov, &ud2, "unused", &err);
g_assert(!err);
@@ -407,13 +411,15 @@ static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
p->value = g_malloc0(sizeof(*p->value));
p->value->string0 = g_strdup(string);
- p->value->dict1.string1 = g_strdup(string);
- p->value->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
- p->value->dict1.dict2.userdef->string = g_strdup(string);
- p->value->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
- p->value->dict1.dict2.userdef->base->integer = 42;
- p->value->dict1.dict2.string = g_strdup(string);
- p->value->dict1.has_dict3 = false;
+ p->value->dict1 = g_malloc0(sizeof(UserDefTwoDict));
+ p->value->dict1->string1 = g_strdup(string);
+ p->value->dict1->dict2 = g_malloc0(sizeof(UserDefTwoDictDict));
+ p->value->dict1->dict2->userdef = g_malloc0(sizeof(UserDefOne));
+ p->value->dict1->dict2->userdef->string = g_strdup(string);
+ p->value->dict1->dict2->userdef->base = g_new0(UserDefZero, 1);
+ p->value->dict1->dict2->userdef->base->integer = 42;
+ p->value->dict1->dict2->string = g_strdup(string);
+ p->value->dict1->has_dict3 = false;
p->next = head;
head = p;
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 133f882..40169ff 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -253,18 +253,21 @@ static UserDefTwo *nested_struct_create(void)
{
UserDefTwo *udnp = g_malloc0(sizeof(*udnp));
udnp->string0 = strdup("test_string0");
- udnp->dict1.string1 = strdup("test_string1");
- udnp->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict2.userdef->base->integer = 42;
- udnp->dict1.dict2.userdef->string = strdup("test_string");
- udnp->dict1.dict2.string = strdup("test_string2");
- udnp->dict1.has_dict3 = true;
- udnp->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict3.userdef->base->integer = 43;
- udnp->dict1.dict3.userdef->string = strdup("test_string");
- udnp->dict1.dict3.string = strdup("test_string3");
+ udnp->dict1 = g_malloc0(sizeof(*udnp->dict1));
+ udnp->dict1->string1 = strdup("test_string1");
+ udnp->dict1->dict2 = g_malloc0(sizeof(*udnp->dict1->dict2));
+ udnp->dict1->dict2->userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1->dict2->userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1->dict2->userdef->base->integer = 42;
+ udnp->dict1->dict2->userdef->string = strdup("test_string");
+ udnp->dict1->dict2->string = strdup("test_string2");
+ udnp->dict1->dict3 = g_malloc0(sizeof(*udnp->dict1->dict3));
+ udnp->dict1->has_dict3 = true;
+ udnp->dict1->dict3->userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1->dict3->userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1->dict3->userdef->base->integer = 43;
+ udnp->dict1->dict3->userdef->string = strdup("test_string");
+ udnp->dict1->dict3->string = strdup("test_string3");
return udnp;
}
@@ -273,18 +276,20 @@ static void nested_struct_compare(UserDefTwo *udnp1, UserDefTwo *udnp2)
g_assert(udnp1);
g_assert(udnp2);
g_assert_cmpstr(udnp1->string0, ==, udnp2->string0);
- g_assert_cmpstr(udnp1->dict1.string1, ==, udnp2->dict1.string1);
- g_assert_cmpint(udnp1->dict1.dict2.userdef->base->integer, ==,
- udnp2->dict1.dict2.userdef->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict2.userdef->string, ==,
- udnp2->dict1.dict2.userdef->string);
- g_assert_cmpstr(udnp1->dict1.dict2.string, ==, udnp2->dict1.dict2.string);
- g_assert(udnp1->dict1.has_dict3 == udnp2->dict1.has_dict3);
- g_assert_cmpint(udnp1->dict1.dict3.userdef->base->integer, ==,
- udnp2->dict1.dict3.userdef->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict3.userdef->string, ==,
- udnp2->dict1.dict3.userdef->string);
- g_assert_cmpstr(udnp1->dict1.dict3.string, ==, udnp2->dict1.dict3.string);
+ g_assert_cmpstr(udnp1->dict1->string1, ==, udnp2->dict1->string1);
+ g_assert_cmpint(udnp1->dict1->dict2->userdef->base->integer, ==,
+ udnp2->dict1->dict2->userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1->dict2->userdef->string, ==,
+ udnp2->dict1->dict2->userdef->string);
+ g_assert_cmpstr(udnp1->dict1->dict2->string, ==,
+ udnp2->dict1->dict2->string);
+ g_assert(udnp1->dict1->has_dict3 == udnp2->dict1->has_dict3);
+ g_assert_cmpint(udnp1->dict1->dict3->userdef->base->integer, ==,
+ udnp2->dict1->dict3->userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1->dict3->userdef->string, ==,
+ udnp2->dict1->dict3->userdef->string);
+ g_assert_cmpstr(udnp1->dict1->dict3->string, ==,
+ udnp2->dict1->dict3->string);
}
static void nested_struct_cleanup(UserDefTwo *udnp)
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 26/28] qapi: Drop inline nested type in query-version
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (24 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 25/28] qapi: Drop tests for inline nested structs Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 10:34 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 27/28] qapi: Drop inline nested types in query-pci Eric Blake
` (3 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline nested structs conflicts with that goal.
This patch fixes one of only two commands relying on nested
types, by breaking the nesting into an explicit type; it means
that the type is now boxed instead of unboxed in C code, but the
QMP wire format is unaffected by this change.
Prefer the safer g_new0() while making the conversion.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
hmp.c | 2 +-
qapi/common.json | 26 +++++++++++++++++++-------
qmp.c | 9 +++++----
3 files changed, 25 insertions(+), 12 deletions(-)
diff --git a/hmp.c b/hmp.c
index f31ae27..c59b823 100644
--- a/hmp.c
+++ b/hmp.c
@@ -60,7 +60,7 @@ void hmp_info_version(Monitor *mon, const QDict *qdict)
info = qmp_query_version(NULL);
monitor_printf(mon, "%" PRId64 ".%" PRId64 ".%" PRId64 "%s\n",
- info->qemu.major, info->qemu.minor, info->qemu.micro,
+ info->qemu->major, info->qemu->minor, info->qemu->micro,
info->package);
qapi_free_VersionInfo(info);
diff --git a/qapi/common.json b/qapi/common.json
index 63ef3b4..ed3aa57 100644
--- a/qapi/common.json
+++ b/qapi/common.json
@@ -29,15 +29,28 @@
'DeviceNotActive', 'DeviceNotFound', 'KVMMissingCap' ] }
##
+# @VersionTriple
+#
+# A three-part version number.
+#
+# @qemu.major: The major version number.
+#
+# @qemu.minor: The minor version number.
+#
+# @qemu.micro: The micro version number.
+#
+# Since: 2.4
+##
+{ 'type': 'VersionTriple',
+ 'data': {'major': 'int', 'minor': 'int', 'micro': 'int'} }
+
+
+##
# @VersionInfo:
#
# A description of QEMU's version.
#
-# @qemu.major: The major version of QEMU
-#
-# @qemu.minor: The minor version of QEMU
-#
-# @qemu.micro: The micro version of QEMU. By current convention, a micro
+# @qemu: The version of QEMU. By current convention, a micro
# version of 50 signifies a development branch. A micro version
# greater than or equal to 90 signifies a release candidate for
# the next minor version. A micro version of less than 50
@@ -51,8 +64,7 @@
# Since: 0.14.0
##
{ 'type': 'VersionInfo',
- 'data': {'qemu': {'major': 'int', 'minor': 'int', 'micro': 'int'},
- 'package': 'str'} }
+ 'data': {'qemu': 'VersionTriple', 'package': 'str'} }
##
# @query-version:
diff --git a/qmp.c b/qmp.c
index c479e77..78a5c23 100644
--- a/qmp.c
+++ b/qmp.c
@@ -45,15 +45,16 @@ NameInfo *qmp_query_name(Error **errp)
VersionInfo *qmp_query_version(Error **errp)
{
- VersionInfo *info = g_malloc0(sizeof(*info));
+ VersionInfo *info = g_new0(VersionInfo, 1);
const char *version = QEMU_VERSION;
char *tmp;
- info->qemu.major = strtol(version, &tmp, 10);
+ info->qemu = g_new0(VersionTriple, 1);
+ info->qemu->major = strtol(version, &tmp, 10);
tmp++;
- info->qemu.minor = strtol(tmp, &tmp, 10);
+ info->qemu->minor = strtol(tmp, &tmp, 10);
tmp++;
- info->qemu.micro = strtol(tmp, &tmp, 10);
+ info->qemu->micro = strtol(tmp, &tmp, 10);
info->package = g_strdup(QEMU_PKGVERSION);
return info;
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 27/28] qapi: Drop inline nested types in query-pci
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (25 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 26/28] qapi: Drop inline nested type in query-version Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 10:37 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 28/28] qapi: Drop support for inline nested types Eric Blake
` (2 subsequent siblings)
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline nested structs conflicts with that goal.
This patch fixes one of only two commands relying on nested
types, by breaking the nesting into an explicit type; it means
that the type is now boxed instead of unboxed in C code, but the
QMP wire format is unaffected by this change.
Prefer the safer g_new0() while making the conversion, and reduce
some long lines.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
hmp.c | 26 ++++++++--------
hw/pci/pci.c | 42 ++++++++++++++------------
qapi-schema.json | 90 ++++++++++++++++++++++++++++++++++++++------------------
3 files changed, 98 insertions(+), 60 deletions(-)
diff --git a/hmp.c b/hmp.c
index c59b823..dbc7fec 100644
--- a/hmp.c
+++ b/hmp.c
@@ -648,14 +648,14 @@ static void hmp_info_pci_device(Monitor *mon, const PciDeviceInfo *dev)
dev->slot, dev->function);
monitor_printf(mon, " ");
- if (dev->class_info.has_desc) {
- monitor_printf(mon, "%s", dev->class_info.desc);
+ if (dev->class_info->has_desc) {
+ monitor_printf(mon, "%s", dev->class_info->desc);
} else {
- monitor_printf(mon, "Class %04" PRId64, dev->class_info.q_class);
+ monitor_printf(mon, "Class %04" PRId64, dev->class_info->q_class);
}
monitor_printf(mon, ": PCI device %04" PRIx64 ":%04" PRIx64 "\n",
- dev->id.vendor, dev->id.device);
+ dev->id->vendor, dev->id->device);
if (dev->has_irq) {
monitor_printf(mon, " IRQ %" PRId64 ".\n", dev->irq);
@@ -663,25 +663,25 @@ static void hmp_info_pci_device(Monitor *mon, const PciDeviceInfo *dev)
if (dev->has_pci_bridge) {
monitor_printf(mon, " BUS %" PRId64 ".\n",
- dev->pci_bridge->bus.number);
+ dev->pci_bridge->bus->number);
monitor_printf(mon, " secondary bus %" PRId64 ".\n",
- dev->pci_bridge->bus.secondary);
+ dev->pci_bridge->bus->secondary);
monitor_printf(mon, " subordinate bus %" PRId64 ".\n",
- dev->pci_bridge->bus.subordinate);
+ dev->pci_bridge->bus->subordinate);
monitor_printf(mon, " IO range [0x%04"PRIx64", 0x%04"PRIx64"]\n",
- dev->pci_bridge->bus.io_range->base,
- dev->pci_bridge->bus.io_range->limit);
+ dev->pci_bridge->bus->io_range->base,
+ dev->pci_bridge->bus->io_range->limit);
monitor_printf(mon,
" memory range [0x%08"PRIx64", 0x%08"PRIx64"]\n",
- dev->pci_bridge->bus.memory_range->base,
- dev->pci_bridge->bus.memory_range->limit);
+ dev->pci_bridge->bus->memory_range->base,
+ dev->pci_bridge->bus->memory_range->limit);
monitor_printf(mon, " prefetchable memory range "
"[0x%08"PRIx64", 0x%08"PRIx64"]\n",
- dev->pci_bridge->bus.prefetchable_range->base,
- dev->pci_bridge->bus.prefetchable_range->limit);
+ dev->pci_bridge->bus->prefetchable_range->base,
+ dev->pci_bridge->bus->prefetchable_range->limit);
}
for (region = dev->regions; region; region = region->next) {
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 6941a82..c1d0068 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -1456,24 +1456,26 @@ static PciBridgeInfo *qmp_query_pci_bridge(PCIDevice *dev, PCIBus *bus,
int bus_num)
{
PciBridgeInfo *info;
+ PciMemoryRange *range;
- info = g_malloc0(sizeof(*info));
+ info = g_new0(PciBridgeInfo, 1);
- info->bus.number = dev->config[PCI_PRIMARY_BUS];
- info->bus.secondary = dev->config[PCI_SECONDARY_BUS];
- info->bus.subordinate = dev->config[PCI_SUBORDINATE_BUS];
+ info->bus = g_new0(PciBusInfo, 1);
+ info->bus->number = dev->config[PCI_PRIMARY_BUS];
+ info->bus->secondary = dev->config[PCI_SECONDARY_BUS];
+ info->bus->subordinate = dev->config[PCI_SUBORDINATE_BUS];
- info->bus.io_range = g_malloc0(sizeof(*info->bus.io_range));
- info->bus.io_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_IO);
- info->bus.io_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_IO);
+ range = info->bus->io_range = g_new0(PciMemoryRange, 1);
+ range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_IO);
+ range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_IO);
- info->bus.memory_range = g_malloc0(sizeof(*info->bus.memory_range));
- info->bus.memory_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
- info->bus.memory_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
+ range = info->bus->memory_range = g_new0(PciMemoryRange, 1);
+ range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
+ range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
- info->bus.prefetchable_range = g_malloc0(sizeof(*info->bus.prefetchable_range));
- info->bus.prefetchable_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
- info->bus.prefetchable_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
+ range = info->bus->prefetchable_range = g_new0(PciMemoryRange, 1);
+ range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
+ range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
if (dev->config[PCI_SECONDARY_BUS] != 0) {
PCIBus *child_bus = pci_find_bus_nr(bus, dev->config[PCI_SECONDARY_BUS]);
@@ -1494,21 +1496,23 @@ static PciDeviceInfo *qmp_query_pci_device(PCIDevice *dev, PCIBus *bus,
uint8_t type;
int class;
- info = g_malloc0(sizeof(*info));
+ info = g_new0(PciDeviceInfo, 1);
info->bus = bus_num;
info->slot = PCI_SLOT(dev->devfn);
info->function = PCI_FUNC(dev->devfn);
+ info->class_info = g_new0(PciDeviceClass, 1);
class = pci_get_word(dev->config + PCI_CLASS_DEVICE);
- info->class_info.q_class = class;
+ info->class_info->q_class = class;
desc = get_class_desc(class);
if (desc->desc) {
- info->class_info.has_desc = true;
- info->class_info.desc = g_strdup(desc->desc);
+ info->class_info->has_desc = true;
+ info->class_info->desc = g_strdup(desc->desc);
}
- info->id.vendor = pci_get_word(dev->config + PCI_VENDOR_ID);
- info->id.device = pci_get_word(dev->config + PCI_DEVICE_ID);
+ info->id = g_new0(PciDeviceId, 1);
+ info->id->vendor = pci_get_word(dev->config + PCI_VENDOR_ID);
+ info->id->device = pci_get_word(dev->config + PCI_DEVICE_ID);
info->regions = qmp_query_pci_regions(dev);
info->qdev_id = g_strdup(dev->qdev.id ? dev->qdev.id : "");
diff --git a/qapi-schema.json b/qapi-schema.json
index 7f4cf86..6b280b7 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1041,36 +1041,75 @@
'*prefetch': 'bool', '*mem_type_64': 'bool' } }
##
+# @PciBusInfo:
+#
+# Information about a bus of a PCI Bridge device
+#
+# @number: primary bus interface number. This should be the number of the
+# bus the device resides on.
+#
+# @secondary: secondary bus interface number. This is the number of the
+# main bus for the bridge
+#
+# @subordinate: This is the highest number bus that resides below the
+# bridge.
+#
+# @io_range: The PIO range for all devices on this bridge
+#
+# @memory_range: The MMIO range for all devices on this bridge
+#
+# @prefetchable_range: The range of prefetchable MMIO for all devices on
+# this bridge
+#
+# Since: 2.4
+##
+{ 'type': 'PciBusInfo',
+ 'data': {'number': 'int', 'secondary': 'int', 'subordinate': 'int',
+ 'io_range': 'PciMemoryRange',
+ 'memory_range': 'PciMemoryRange',
+ 'prefetchable_range': 'PciMemoryRange' } }
+
+##
# @PciBridgeInfo:
#
# Information about a PCI Bridge device
#
-# @bus.number: primary bus interface number. This should be the number of the
-# bus the device resides on.
-#
-# @bus.secondary: secondary bus interface number. This is the number of the
-# main bus for the bridge
-#
-# @bus.subordinate: This is the highest number bus that resides below the
-# bridge.
-#
-# @bus.io_range: The PIO range for all devices on this bridge
-#
-# @bus.memory_range: The MMIO range for all devices on this bridge
-#
-# @bus.prefetchable_range: The range of prefetchable MMIO for all devices on
-# this bridge
+# @bus: information about the bus the device resides on
#
# @devices: a list of @PciDeviceInfo for each device on this bridge
#
# Since: 0.14.0
##
{ 'type': 'PciBridgeInfo',
- 'data': {'bus': { 'number': 'int', 'secondary': 'int', 'subordinate': 'int',
- 'io_range': 'PciMemoryRange',
- 'memory_range': 'PciMemoryRange',
- 'prefetchable_range': 'PciMemoryRange' },
- '*devices': ['PciDeviceInfo']} }
+ 'data': {'bus': 'PciBusInfo', '*devices': ['PciDeviceInfo']} }
+
+##
+# @PciDeviceClass:
+#
+# Information about the Class of a PCI device
+#
+# @desc: #optional a string description of the device's class
+#
+# @class: the class code of the device
+#
+# Since: 2.4
+##
+{ 'type': 'PciDeviceClass',
+ 'data': {'*desc': 'str', 'class': 'int'} }
+
+##
+# @PciDeviceId:
+#
+# Information about the Id of a PCI device
+#
+# @device: the PCI device id
+#
+# @vendor: the PCI vendor id
+#
+# Since: 2.4
+##
+{ 'type': 'PciDeviceId',
+ 'data': {'device': 'int', 'vendor': 'int'} }
##
# @PciDeviceInfo:
@@ -1083,13 +1122,9 @@
#
# @function: the function of the slot used by the device
#
-# @class_info.desc: #optional a string description of the device's class
+# @class_info: the class of the device
#
-# @class_info.class: the class code of the device
-#
-# @id.device: the PCI device id
-#
-# @id.vendor: the PCI vendor id
+# @id: the PCI device id
#
# @irq: #optional if an IRQ is assigned to the device, the IRQ number
#
@@ -1106,8 +1141,7 @@
##
{ 'type': 'PciDeviceInfo',
'data': {'bus': 'int', 'slot': 'int', 'function': 'int',
- 'class_info': {'*desc': 'str', 'class': 'int'},
- 'id': {'device': 'int', 'vendor': 'int'},
+ 'class_info': 'PciDeviceClass', 'id': 'PciDeviceId',
'*irq': 'int', 'qdev_id': 'str', '*pci_bridge': 'PciBridgeInfo',
'regions': ['PciMemoryRegion']} }
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* [Qemu-devel] [PATCH v5 28/28] qapi: Drop support for inline nested types
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (26 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 27/28] qapi: Drop inline nested types in query-pci Eric Blake
@ 2015-03-24 20:03 ` Eric Blake
2015-03-27 10:45 ` Markus Armbruster
2015-03-27 12:50 ` [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Markus Armbruster
2015-03-29 16:03 ` Markus Armbruster
29 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:03 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, famz, armbru, wenchaoqemu, lcapitulino
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline nested structs conflicts with that goal.
Now that all commands have been changed to avoid inline nested
structs, nuke support for them, and turn it into a hard error.
Update the testsuite to reflect tighter parsing rules.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-commands.py | 8 +++---
scripts/qapi-event.py | 4 +--
scripts/qapi-types.py | 9 ++-----
scripts/qapi-visit.py | 37 ++++------------------------
scripts/qapi.py | 18 +++++---------
tests/qapi-schema/event-nest-struct.err | 2 +-
tests/qapi-schema/nested-struct-data.err | 1 +
tests/qapi-schema/nested-struct-data.exit | 2 +-
tests/qapi-schema/nested-struct-data.json | 2 +-
tests/qapi-schema/nested-struct-data.out | 3 ---
tests/qapi-schema/nested-struct-returns.err | 1 +
tests/qapi-schema/nested-struct-returns.exit | 2 +-
tests/qapi-schema/nested-struct-returns.json | 2 +-
tests/qapi-schema/nested-struct-returns.out | 3 ---
14 files changed, 26 insertions(+), 68 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 053ba85..db81044 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -28,7 +28,7 @@ def type_visitor(name):
def generate_command_decl(name, args, ret_type):
arglist=""
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
argtype = c_type(argtype, is_param=True)
if optional:
arglist += "bool has_%s, " % c_var(argname)
@@ -53,7 +53,7 @@ def gen_sync_call(name, args, ret_type, indent=0):
retval=""
if ret_type:
retval = "retval = "
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
if optional:
arglist += "has_%s, " % c_var(argname)
arglist += "%s, " % (c_var(argname))
@@ -96,7 +96,7 @@ Visitor *v;
def gen_visitor_input_vars_decl(args):
ret = ""
push_indent()
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
if optional:
ret += mcgen('''
bool has_%(argname)s = false;
@@ -139,7 +139,7 @@ v = qapi_dealloc_get_visitor(md);
v = qmp_input_get_visitor(mi);
''')
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
if optional:
ret += mcgen('''
visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 601e307..47dc041 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -21,7 +21,7 @@ def _generate_event_api_name(event_name, params):
l = len(api_name)
if params:
- for argname, argentry, optional, structured in parse_args(params):
+ for argname, argentry, optional in parse_args(params):
if optional:
api_name += "bool has_%s,\n" % c_var(argname)
api_name += "".ljust(l)
@@ -93,7 +93,7 @@ def generate_event_implement(api_name, event_name, params):
""",
event_name = event_name)
- for argname, argentry, optional, structured in parse_args(params):
+ for argname, argentry, optional in parse_args(params):
if optional:
ret += mcgen("""
if (has_%(var)s) {
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 9c8d68c..4789528 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -63,18 +63,13 @@ typedef struct %(name)sList
def generate_struct_fields(members):
ret = ''
- for argname, argentry, optional, structured in parse_args(members):
+ for argname, argentry, optional in parse_args(members):
if optional:
ret += mcgen('''
bool has_%(c_name)s;
''',
c_name=c_var(argname))
- if structured:
- push_indent()
- ret += generate_struct({ "field": argname, "data": argentry})
- pop_indent()
- else:
- ret += mcgen('''
+ ret += mcgen('''
%(c_type)s %(c_name)s;
''',
c_type=c_type(argentry), c_name=c_var(argname))
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3e11089..10b08c6 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -51,27 +51,6 @@ def generate_visit_struct_fields(name, field_prefix, fn_prefix, members, base =
else:
full_name = "%s_%s" % (name, fn_prefix)
- for argname, argentry, optional, structured in parse_args(members):
- if structured:
- if not fn_prefix:
- nested_fn_prefix = argname
- else:
- nested_fn_prefix = "%s_%s" % (fn_prefix, argname)
-
- nested_field_prefix = "%s%s." % (field_prefix, argname)
- ret += generate_visit_struct_fields(name, nested_field_prefix,
- nested_fn_prefix, argentry)
- ret += mcgen('''
-
-static void visit_type_%(full_name)s_field_%(c_name)s(Visitor *m, %(name)s **obj, Error **errp)
-{
-''',
- name=name, full_name=full_name, c_name=c_var(argname))
- ret += generate_visit_struct_body(full_name, argname, argentry)
- ret += mcgen('''
-}
-''')
-
if base:
ret += generate_visit_implicit_struct(base)
@@ -94,7 +73,7 @@ if (err) {
c_prefix=c_var(field_prefix),
type=type_name(base), c_name=c_var('base'))
- for argname, argentry, optional, structured in parse_args(members):
+ for argname, argentry, optional in parse_args(members):
if optional:
ret += mcgen('''
visit_optional(m, &(*obj)->%(c_prefix)shas_%(c_name)s, "%(name)s", &err);
@@ -104,18 +83,12 @@ if (!err && (*obj)->%(prefix)shas_%(c_name)s) {
c_name=c_var(argname), name=argname)
push_indent()
- if structured:
- ret += mcgen('''
-visit_type_%(full_name)s_field_%(c_name)s(m, obj, &err);
-''',
- full_name=full_name, c_name=c_var(argname))
- else:
- ret += mcgen('''
+ ret += mcgen('''
visit_type_%(type)s(m, &(*obj)->%(c_prefix)s%(c_name)s, "%(name)s", &err);
''',
- c_prefix=c_var(field_prefix), prefix=field_prefix,
- type=type_name(argentry), c_name=c_var(argname),
- name=argname)
+ c_prefix=c_var(field_prefix), prefix=field_prefix,
+ type=type_name(argentry), c_name=c_var(argname),
+ name=argname)
if optional:
pop_indent()
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 800e8e4..8e5b4ad 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -368,11 +368,13 @@ def check_type(expr_info, source, data, allow_array = False,
for (key, value) in data.items():
check_name(expr_info, "Member of %s" % source, key,
allow_optional=allow_optional)
+ # Todo: allow dictionaries to represent default values of
+ # an optional argument.
check_type(expr_info, "Member '%s' of %s" % (key, source), value,
allow_array=True,
allowed_metas=['built-in', 'union', 'alternate', 'struct',
'enum'],
- allow_dict=True, allow_optional=True, allow_star=allow_star)
+ allow_dict=False, allow_star=allow_star)
def check_command(expr, expr_info):
name = expr['command']
@@ -404,13 +406,6 @@ def check_event(expr, expr_info):
check_type(expr_info, "'data' for event '%s'" % name,
expr.get('data'), allowed_metas=['union', 'struct'],
allow_optional=True)
- if params:
- for argname, argentry, optional, structured in parse_args(params):
- if structured:
- raise QAPIExprError(expr_info,
- "Nested structure define in event is not "
- "supported, event '%s', argname '%s'"
- % (expr['event'], argname))
def check_union(expr, expr_info):
name = expr['union']
@@ -667,13 +662,12 @@ def parse_args(typeinfo):
argname = member
argentry = typeinfo[member]
optional = False
- structured = False
if member.startswith('*'):
argname = member[1:]
optional = True
- if isinstance(argentry, OrderedDict):
- structured = True
- yield (argname, argentry, optional, structured)
+ # Todo: allow argentry to be OrderedDict, for providing the
+ # value of an optional argument.
+ yield (argname, argentry, optional)
def de_camel_case(name):
new_name = ''
diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
index 91bde1c..5a42701 100644
--- a/tests/qapi-schema/event-nest-struct.err
+++ b/tests/qapi-schema/event-nest-struct.err
@@ -1 +1 @@
-tests/qapi-schema/event-nest-struct.json:1: Nested structure define in event is not supported, event 'EVENT_A', argname 'a'
+tests/qapi-schema/event-nest-struct.json:1: Member 'a' of 'data' for event 'EVENT_A' should be a type name
diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
index e69de29..da767ba 100644
--- a/tests/qapi-schema/nested-struct-data.err
+++ b/tests/qapi-schema/nested-struct-data.err
@@ -0,0 +1 @@
+tests/qapi-schema/nested-struct-data.json:2: Member 'a' of 'data' for command 'foo' should be a type name
diff --git a/tests/qapi-schema/nested-struct-data.exit b/tests/qapi-schema/nested-struct-data.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/nested-struct-data.exit
+++ b/tests/qapi-schema/nested-struct-data.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/nested-struct-data.json b/tests/qapi-schema/nested-struct-data.json
index 0247c8c..3d52d2b 100644
--- a/tests/qapi-schema/nested-struct-data.json
+++ b/tests/qapi-schema/nested-struct-data.json
@@ -1,4 +1,4 @@
-# FIXME: inline subtypes collide with our desired future use of defaults
+# inline subtypes collide with our desired future use of defaults
{ 'command': 'foo',
'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' },
'returns': {} }
diff --git a/tests/qapi-schema/nested-struct-data.out b/tests/qapi-schema/nested-struct-data.out
index 999cbb8..e69de29 100644
--- a/tests/qapi-schema/nested-struct-data.out
+++ b/tests/qapi-schema/nested-struct-data.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('data', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')])), ('returns', OrderedDict())])]
-[]
-[]
diff --git a/tests/qapi-schema/nested-struct-returns.err b/tests/qapi-schema/nested-struct-returns.err
index e69de29..5238d07 100644
--- a/tests/qapi-schema/nested-struct-returns.err
+++ b/tests/qapi-schema/nested-struct-returns.err
@@ -0,0 +1 @@
+tests/qapi-schema/nested-struct-returns.json:2: Member 'a' of 'returns' for command 'foo' should be a type name
diff --git a/tests/qapi-schema/nested-struct-returns.exit b/tests/qapi-schema/nested-struct-returns.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/nested-struct-returns.exit
+++ b/tests/qapi-schema/nested-struct-returns.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/nested-struct-returns.json b/tests/qapi-schema/nested-struct-returns.json
index 5a46840..d2cd047 100644
--- a/tests/qapi-schema/nested-struct-returns.json
+++ b/tests/qapi-schema/nested-struct-returns.json
@@ -1,3 +1,3 @@
-# FIXME: inline subtypes collide with our desired future use of defaults
+# inline subtypes collide with our desired future use of defaults
{ 'command': 'foo',
'returns': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }
diff --git a/tests/qapi-schema/nested-struct-returns.out b/tests/qapi-schema/nested-struct-returns.out
index c53d23b..e69de29 100644
--- a/tests/qapi-schema/nested-struct-returns.out
+++ b/tests/qapi-schema/nested-struct-returns.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('returns', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')]))])]
-[]
-[]
--
2.1.0
^ permalink raw reply related [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema Eric Blake
@ 2015-03-24 20:33 ` Eric Blake
2015-03-26 9:54 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:33 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, lcapitulino, famz, armbru, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]
On 03/24/2015 02:03 PM, Eric Blake wrote:
> Python 2 and Python 3 have a wild history of whether strings
> default to ascii or unicode, where Python 3 requires checking
> instanceof(foo, basestr) to cover all strings, but where that
> code is not portable to Python 2. It's simpler to just state
> that we don't care about Unicode strings, and to just always
> use the simpler instanceof(foo, str) everywhere.
And for all my proof-reading, I already have a commit message change:
s/instanceof/isinstance/
(you can tell I'm not that proficient in python...)
>
> I'm no python expert, so I'm basing it on this conversation:
> https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.html
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> @@ -354,7 +354,7 @@ def parse_schema(input_file):
> return exprs
>
> def parse_args(typeinfo):
> - if isinstance(typeinfo, basestring):
> + if isinstance(typeinfo, str):
at least the code is right.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions Eric Blake
@ 2015-03-24 20:38 ` Eric Blake
2015-03-26 14:20 ` Markus Armbruster
1 sibling, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:38 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, lcapitulino, famz, armbru, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]
On 03/24/2015 02:03 PM, Eric Blake wrote:
> Previous commits demonstrated that the generator had several
> flaws with less-than-perfect unions:
> - make the use of a base without discriminator a hard error,
> since the previous patch removed all remaining uses of it
> - a simple union that listed the same branch twice (or two variant
> names that map to the same C enumerator, including the implicit
> MAX sentinel) ended up generating invalid C code
> - checking 'if discriminator' prior to 'if discriminator == {}'
Another commit typo; should read:
checking 'if not discriminator' prior to 'if discriminator == {}'
> leads to dead code in python, and ended up processing anonymous
> unions as if they were simple unions
> - an anonymous union that listed two branches with the same qtype
> ended up generating invalid C code
> - the generator crashed on anonymous union attempts to use an
> array type
> - the generator was silently ignoring a base type for anonymous
> unions
> - the generator allowed unknown types or nested anonymous unions
> as a branch in an anonymous union
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union Eric Blake
@ 2015-03-24 20:41 ` Eric Blake
2015-03-26 15:42 ` Markus Armbruster
1 sibling, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-24 20:41 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, lcapitulino, famz, armbru, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 642 bytes --]
On 03/24/2015 02:03 PM, Eric Blake wrote:
> Rather than special-casing "'union':'foo','alternate':{}" as an
And continuing my belated proofreading:
s/alternate/discriminator/
> unusual union that can represent a non-dictionary, it is nicer
> to designate a separate meta-type "'alternate':'foo'" for the
> purpose. This involves a lot of documentation tweaks and fallout
> from .json files, but I already split as much as possible into
> earlier commits.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
@ 2015-03-25 18:31 ` Markus Armbruster
2015-03-25 20:11 ` Eric Blake
2015-03-31 15:09 ` Kevin Wolf
2015-04-01 15:29 ` Markus Armbruster
2 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-25 18:31 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Go into more details about the various types of valid expressions
> in a qapi schema, including tweaks to document fixes being done
> later in the current patch series. Also fix some stale and missing
> documentation in the QMP specification.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
>
> I'm not sure if it is okay to assert GPLv2+ licensing without an
> explicit Copyright, but as I am not the original author, I don't
> know who to attribute any original Copyright to. Advice? Should
> I split the license addition to a separate patch?
> ---
> docs/qapi-code-gen.txt | 338 ++++++++++++++++++++++++++++++++++++++++---------
> docs/qmp/qmp-spec.txt | 92 +++++++++++---
> 2 files changed, 350 insertions(+), 80 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 8313ba6..ce9c4b9 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -1,17 +1,18 @@
> +This document is licensed under the GPLv2 (or later).
> +
> = How to use the QAPI code generator =
>
> QAPI is a native C API within QEMU which provides management-level
> -functionality to internal/external users. For external
> +functionality to internal and external users. For external
> users/processes, this interface is made available by a JSON-based
> QEMU Monitor protocol that is provided by the QMP server.
>
> -To map QMP-defined interfaces to the native C QAPI implementations,
> -a JSON-based schema is used to define types and function
> -signatures, and a set of scripts is used to generate types/signatures,
> -and marshaling/dispatch code. The QEMU Guest Agent also uses these
> -scripts, paired with a separate schema, to generate
> -marshaling/dispatch code for the guest agent server running in the
> -guest.
> +To map QMP-defined interfaces to the native C QAPI implementations, a
> +JSON-based schema is used to define types and function signatures, and
> +a set of scripts is used to generate types, signatures, and
> +marshaling/dispatch code. The QEMU Guest Agent also uses these
> +scripts, paired with a separate schema, to generate marshaling and
> +dispatch code for the guest agent server running in the guest.
>
> This document will describe how the schemas, scripts, and resulting
> code are used.
> @@ -22,40 +23,150 @@ code are used.
> This file defines the types, commands, and events used by QMP. It should
> fully describe the interface used by QMP.
>
> -This file is designed to be loosely based on JSON although it's technically
> -executable Python. While dictionaries are used, they are parsed as
> -OrderedDicts so that ordering is preserved.
> +A QAPI file is designed to be loosely based on JSON, which is then
> +parsed by a python code generation program. A valid QAPI schema
> +consists of a list of top-level expressions, with no commas between
> +them. Where dictionaries are used, they are parsed as OrderedDicts so
> +that ordering is preserved (for predictable layout of generated C
> +structs and parameter lists). Ordering doesn't matter for top-level
> +expressions, but does matter within 'data' members of a single
> +expression. QAPI input is written using 'single quotes' instead of
> +JSON's "double quotes" (in contrast, QMP is strict JSON and only uses
> +"double quotes"). As in JSON, trailing commas are not permitted in
> +arrays or dictionaries. Input must be ASCII (although QMP supports
> +full Unicode strings, the QAPI parser does not).
>
> -There are two basic syntaxes used, type definitions and command definitions.
> +Comments are allowed; anything between an unquoted # and the following
> +newline is ignored. Although there is not yet a documentation
> +generator, a form of stylized comments has developed for consistently
> +documenting details about an expression and when it was added to the
> +schema. The documentation is delimited between two lines of ##, then
> +the first line names the expression, an optional overview is provided,
> +then individual documentation about each member of 'data' is provided,
> +and finally, a 'Since: x.y.z' tag lists the release that introduced
> +the expression. Optional fields are tagged with the phrase
> +'#optional', often with their default value; and extensions added
> +after the expression was first released are also given a '(since
> +x.y.z)' comment. For example:
>
> -The first syntax defines a type and is represented by a dictionary. There are
> -three kinds of user-defined types that are supported: complex types,
> -enumeration types and union types.
> + ##
> + # @BlockStats:
> + #
> + # Statistics of a virtual block device or a block backing device.
> + #
> + # @device: #optional If the stats are for a virtual block device, the name
> + # corresponding to the virtual block device.
> + #
> + # @stats: A @BlockDeviceStats for the device.
> + #
> + # @parent: #optional This describes the file block device if it has one.
> + #
> + # @backing: #optional This describes the backing block device if it has one.
> + # (Since 2.0)
> + #
> + # Since: 0.14.0
> + ##
> + { 'type': 'BlockStats',
> + 'data': {'*device': 'str', 'stats': 'BlockDeviceStats',
> + '*parent': 'BlockStats',
> + '*backing': 'BlockStats'} }
>
> -Generally speaking, types definitions should always use CamelCase for the type
> -names. Command names should be all lower case with words separated by a hyphen.
> +The schema sets up a series of types, as well as commands and events
> +that will use those types. Forward references are allowed: the parser
> +scans in two passes, where the first pass learns all type names, and
> +the second validates the schema and generates the code. This allows
> +the definition of complex structs that can have mutually recursive
> +types, and allows for indefinite nesting of QMP that satisfies the
> +schema. A type name should not be defined more than once.
>
> +There are six top-level expressions recognized by the parser:
> +'include', 'command', 'type', 'enum', 'union', and 'event'. There are
> +several built-in types, such as 'int' and 'str'; additionally, the
> +top-level expressions can define complex types, enumeration types, and
> +several flavors of union types. The 'command' expression can refer to
> +existing types by name, or list an anonymous type as a dictionary.
'event' can do that, too.
> +
> +Listing a type name inside an array refers to a single-dimension array
> +of that type; multi-dimension arrays are not directly supported
> +(although an array of a complex struct that contains an array member
> +is possible).
> +
> +Types, commands, and events share a common namespace. Therefore,
> +generally speaking, type definitions should always use CamelCase for
> +user-defined type names, while built-in types are lowercase. Type
> +definitions should not end in 'Kind', as this namespace is used for
> +creating implicit C enums for visiting union types. Command names,
> +and field names within a type, should be all lower case with words
> +separated by a hyphen. However, some existing older commands and
> +complex types use underscore; when extending such expressions,
> +consistency is preferred over blindly avoiding underscore. Event
> +names should be ALL_CAPS with words separated by underscore. The
> +special string '**' appears for some commands that manually perform
> +their own type checking rather than relying on the type-safe code
> +produced by the qapi code generators.
The generator generates C identifiers pretty thoughtlessly, and is
therefore prone to generate clashes. Fixable, but we've got bigger fish
to fry, and this series is hefty enough as it is. Documenting the mess
instead makes sense.
> +
> +Any command, type, or field name beginning with "x-" is marked
> +experimental, and may be withdrawn or changed incompatibly in a future
> +release. Downstream vendors may add extensions; such extensions
> +should begin with a prefix matching "__RFQDN_" (for the
> +reverse-fully-qualified-domain-name of the vendor), even if the rest
> +of the command or field name uses dash (example:
> +__com.redhat_drive-mirror). Other than the dots used in RFQDN of a
> +downstream extension, all command, type, and field names should begin
> +with a letter, and contain only ASCII letters, numbers, dash, and
> +underscore. It is okay to reuse names that match C keywords; the
> +generator will rename a field named "default" in the QAPI to
> +"q_default" in the generated C code.
> +
> +
> +=== Built-in Types ===
> +
> +The following types are built-in to the parser:
> + 'str' - arbitrary UTF-8 string
> + 'int' - 64-bit signed integer (although the C code may place further
> + restrictions on acceptable range)
> + 'number' - floating point number
> + 'bool' - JSON value of true or false
> + 'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
> + bit size
> + 'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
Apropos uint64_t: QInt wraps int64_t, and I wouldn't be willing to bet
on uint64_t values it can't represent to actually work. But if they
don't, it's a bug that is outside the scope of this series.
> + 'size' - like 'uint64', but allows scaled suffix from command line
> + visitor
>
> === Includes ===
>
> +Usage: { 'include': 'str' }
> +
> The QAPI schema definitions can be modularized using the 'include' directive:
>
> { 'include': 'path/to/file.json'}
If you need to respin for some reason, insert a space before the closing
brace.
>
> The directive is evaluated recursively, and include paths are relative to the
> -file using the directive. Multiple includes of the same file are safe.
> +file using the directive. Multiple includes of the same file are
> +safe. No other keys should appear in the expression, and the include
> +value should be a string.
> +
> +As a matter of style, it is a good idea to have all files be
> +self-contained, but at the moment, nothing prevents an included file
> +from making a forward reference to a type that is only introduced by
> +an outer file. The parser may be made stricter in the future to
> +prevent incomplete include files.
>
>
> === Complex types ===
>
> -A complex type is a dictionary containing a single key whose value is a
> -dictionary. This corresponds to a struct in C or an Object in JSON. An
> -example of a complex type is:
> +Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
Here, the reader needs to get that 'type' and 'data' and 'base' are
literals, but 'str', 'dict' and 'complex-type-name' are placeholders. I
like setting apart placeholders with typograhic conventions. If we want
to do that, let's do it on top.
'*base' is interesting. It's a tacit use of the schema language's
"'*name' means member 'name' is optional" rule on meta-schema-level. We
can take care of that when/if we clarify placeholders.
> +
> +A complex type is a dictionary containing a single 'data' key whose
> +value is a dictionary. This corresponds to a struct in C or an Object
> +in JSON. Each value of the 'data' dictionary must be the name of a
> +complex, enum, union, or built-in type,
Are there any others? If not, we can simplify to
Each value of the 'data' dictionary must be a type name
> or a one-element array
> +containing a type name. An example of a complex type is:
>
> { 'type': 'MyType',
> 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
>
> -The use of '*' as a prefix to the name means the member is optional.
> +The use of '*' as a prefix to the name means the member is optional in
> +the corresponding QMP usage.
Not just in QMP, also QGA and (internal) C interfaces. Since explaining
that is tiresome, I'd be tempted to stick to the original sentence here.
Preexisting: we sometimes say "QMP wire format", and sometimes simply
"JSON". Again, we got bigger fish to fry.
>
> The default initialization value of an optional argument should not be changed
> between versions of QEMU unless the new default maintains backward
> @@ -100,22 +211,52 @@ both fields like this:
> { "file": "/some/place/my-image",
> "backing": "/some/place/my-backing-file" }
>
> +
> === Enumeration types ===
>
> -An enumeration type is a dictionary containing a single key whose value is a
> -list of strings. An example enumeration is:
> +Usage: { 'enum': 'str', 'data': [ 'str' ] }
> +
> +An enumeration type is a dictionary containing a single 'data' key
> +whose value is a list of strings. An example enumeration is:
>
> { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }
>
> +Nothing prevents an empty enumeration, although it is probably not
> +useful. The list of strings should be lower case; if an enum name
> +represents multiple words, use '-' between words. The string 'max' is
> +not allowed as an enum value, and values should not be repeated.
> +
> +The enumeration values are passed as strings over the QMP protocol,
> +but are encoded as C enum integral values in generated code. While
> +the C code starts numbering at 0, it is better to use explicit
> +comparisons to enum values than implicit comparisons to 0; the C code
> +will also include a generated enum member ending in _MAX for tracking
> +the size of the enum, useful when using common functions for
> +converting between strings and enum values. Since the wire format
> +always passes by name, it is acceptable to reorder or add new
> +enumeration members in any location without breaking QMP clients;
> +however, removing enum values would break compatibility. For any
> +complex type that has a field that will only contain a finite set of
> +string values, using an enum type for that field is better than
> +open-coding the field to be type 'str'.
> +
> +
> === Union types ===
>
> -Union types are used to let the user choose between several different data
> -types. A union type is defined using a dictionary as explained in the
> -following paragraphs.
> +Usage: { 'union': 'str', 'data': 'dict' }
> +or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
> + 'discriminator': 'enum-member-of-base' }
> +or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
>
> +Union types are used to let the user choose between several different
> +data types. There are three flavors: simple (no discriminator), flat
> +(a base type is mandatory, and discriminator is the name of an enum
> +field within that base type), and anonymous (discriminator is an
> +empty dictionary). A union type is defined using a data dictionary as
> +explained in the following paragraphs.
>
> -A simple union type defines a mapping from discriminator values to data types
> -like in this example:
> +A simple union type defines a mapping from automatic discriminator
> +values to data types like in this example:
>
> { 'type': 'FileOptions', 'data': { 'filename': 'str' } }
> { 'type': 'Qcow2Options',
> @@ -132,10 +273,17 @@ specified data type corresponding to the discriminator value:
> { "type": "qcow2", "data" : { "backing-file": "/some/place/my-image",
> "lazy-refcounts": true } }
>
> +Additionally, an implicit C enum NameKind is created, corresponding to
> +the union Name, for accessing the various branches of the union. No
> +branch of the union can be named 'max', as this would collide with the
> +implicit enum.
Aside: I guess this implicit enum is just like a normal QAPI enum,
except it doesn't have a name in the schema.
>
> -A union definition can specify a complex type as its base. In this case, the
> -fields of the complex type are included as top-level fields of the union
> -dictionary in the QMP wire format. An example definition is:
> +
> +A flat union definition specifies a complex type as its base, and
> +avoids nesting on the wire. In this case, the fields of the complex
> +type are included as top-level fields of the union dictionary in the
> +QMP wire format, and the 'discriminator' field must be the name of an
> +enum-typed member of the base type. An example definition is:
>
> { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
> { 'union': 'BlockdevOptions',
Worth mentioning explicitly that the base type's members are common to
all the union's cases?
> @@ -150,11 +298,11 @@ And it looks like this on the wire:
> "data" : { "backing-file": "/some/place/my-image",
> "lazy-refcounts": true } }
>
> -
> -Flat union types avoid the nesting on the wire. They are used whenever a
> -specific field of the base type is declared as the discriminator ('type' is
> -then no longer generated). The discriminator must be of enumeration type.
> -The above example can then be modified as follows:
> +Notice that in a flat union, a 'type' field is no longer generated,
> +and the keys of the 'data' dictionary must match the valid values for
> +the discriminator (although not necessarily in the same order). The
> +above example for simple unions can be modified to a flat union as
> +follows:
>
> { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
> { 'type': 'BlockdevCommonOptions',
> @@ -173,13 +321,15 @@ Resulting in this JSON object:
> "lazy-refcounts": true }
>
>
> -A special type of unions are anonymous unions. They don't form a dictionary in
> -the wire format but allow the direct use of different types in their place. As
> -they aren't structured, they don't have any explicit discriminator but use
> -the (QObject) data type of their value as an implicit discriminator. This means
> -that they are restricted to using only one discriminator value per QObject
> -type. For example, you cannot have two different complex types in an anonymous
> -union, or two different integer types.
> +The final flavor of unions is an anonymous union. While the other two
> +union types are always passed as a dictionary in the wire format, an
> +anonymous union instead allows the direct use of different types in
> +its place. As they aren't structured, they don't have any explicit
> +discriminator but use the (QObject) data type of their value as an
> +implicit discriminator. This means that they are restricted to using
> +only one discriminator value per QObject type. For example, you cannot
> +have two different complex types in an anonymous union, or two
> +different integer types.
This is the first mention of "QObject type". How it's related to JSON
is left to the reader's deductive skills :)
The QObject types are QTYPE_NONE, QTYPE_QINT, QTYPE_QSTRING,
QTYPE_QDICT, QTYPE_QLIST, QTYPE_QFLOAT, QTYPE_QBOOL, QTYPE_QERROR.
The connections JSON string - QTYPE_QSTRING, JSON object - QTYPE_QDICT,
JSON array - QTYPE_QLIST and JSON boolean - QTYPE_QBOOL are obvious
enough.
If I remember correctly, our JSON parser chokes on the JSON keyword
null.
That leaves just JSON numbers - QTYPE_QINT or QTYPE_QFLOAT. Can an
anonymous union have a separate case for each of the two?
For completeness: on the QTYPE_ side, it leaves QTYPE_QERROR and
QTYPE_NONE. QTYPE_QERROR is internal only, and will hopefully be gone
soon. I can't see QTYPE_NONE objects being created anywhere.
Should we explain this in terms of JSON types instead of QObject types?
>
> Anonymous unions are declared using an empty dictionary as their discriminator.
> The discriminator values never appear on the wire, they are only used in the
> @@ -200,23 +350,93 @@ This example allows using both of the following example objects:
>
> === Commands ===
>
> -Commands are defined by using a list containing three members. The first
> -member is the command name, the second member is a dictionary containing
> -arguments, and the third member is the return type.
> +Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
> + '*returns': 'type',
Shoudn't this be '*returns': 'dict-or-type-name'?
> + '*gen': false, '*success-response': false }
>
> -An example command is:
> +Commands are defined by using a dictionary containing several members,
> +where three members are most common. The 'command' member is a
> +mandatory string, and determines the "execute" value passed in a QMP
> +command exchange.
> +
> +The 'data' member is optional; if absent, the command accepts an
> +optional empty dictionary.
Suggest: The 'data' member is optional, and defaults to {}.
> + If present, it must be the string name of
> +a complex type, a one-element array containing the name of a complex
> +type, or a dictionary that declares an anonymous type with the same
> +semantics as a 'type' expression, with one exception noted below when
> +'gen' is used. The 'data' argument maps to the "arguments" dictionary
> +passed in as part of a QMP command.
Suggest to move the last sentence to the beginning of the paragaph.
> +
> +The 'returns' member describes what will appear in the "return" field
> +of a QMP reply on successful completion of a command. The member is
> +optional from the command declaration; if absent, the "return" field
> +will be an empty dictionary. If 'returns' is present, it must be the
> +string name of a complex or built-in type, a one-element array
> +containing the name of a complex or built-in type, or a dictionary
> +that declares an anonymous type with the same semantics as a 'type'
> +expression, with one exception noted below when 'gen' is used.
Oh, 'gen' affect 'returns', too? Live and learn...
> +Although it is permitted to have the 'returns' member name a built-in
> +type or an array of built-in types, any command that does this cannot
> +be extended to return additional information in the future; thus, new
> +commands should strongly consider returning a dictionary-based type or
> +an array of dictionaries, even if the dictionary only contains one
> +field at the present.
> +
> +All commands use a dictionary to report failure, with no way to
> +specify that in QAPI. Where the error return is different than the
> +usual GenericError class in order to help the client react differently
> +to certain error conditions, it is worth documenting this in the
> +comments before the command declaration.
> +
> +Some example commands:
> +
> + { 'command': 'my-first-command',
> + 'data': { 'arg1': 'str', '*arg2': 'str' } }
> + { 'type': 'MyType', 'data': { '*value': 'str' } }
> + { 'command': 'my-second-command',
> + 'returns': [ 'MyType' ] }
> +
> +which would validate this QMP transaction:
> +
> + => { "execute": "my-first-command",
> + "arguments": { "arg1": "hello" } }
> + <= { "return": { } }
> + => { "execute": "my-second-command" }
> + <= { "return": [ { "value": "one" }, { } ] }
> +
> +In rare cases, QAPI cannot express a type-safe representation of a
> +corresponding QMP command. In these cases, if the command expression
> +includes the key 'gen' with boolean value false, then the 'data' or
> +'returns' member that intends to bypass generated type-safety and do
> +its own manual validation should use '**' rather than a valid type
> +name. Please try to avoid adding new commands that rely on this, and
> +instead use type-safe unions. For an example of bypass usage:
> +
> + { 'command': 'netdev_add',
> + 'data': {'type': 'str', 'id': 'str', '*props': '**'},
> + 'gen': false }
We use 'gen': 'no' until PATCH 18.
> +
> +Normally, the QAPI schema is used to describe synchronous exchanges,
> +where a response is expected. But in some cases, the action of a
> +command is expected to change state in a way that a successful
> +response is not possible (the command still returns a normal
> +dictionary error on failure). When a successful reply is not
> +possible, the command expression should include the optional key
> +'success-response' with boolean value false. So far, only the
> +qemu-guest-agent makes use of this field.
I had no idea :)
>
> - { 'command': 'my-command',
> - 'data': { 'arg1': 'str', '*arg2': 'str' },
> - 'returns': 'str' }
>
> === Events ===
>
> -Events are defined with the keyword 'event'. When 'data' is also specified,
> -additional info will be included in the event. Finally there will be C API
> -generated in qapi-event.h; when called by QEMU code, a message with timestamp
> -will be emitted on the wire. If timestamp is -1, it means failure to retrieve
> -host time.
> +Usage: { 'event': 'str', '*data': 'dict-or-complex-type-name' }
> +
> +Events are defined with the keyword 'event'. It is not allowed to
> +name an event 'MAX', since the generator also produces a C enumeration
> +of all event names with a generated _MAX value at the end. When
> +'data' is also specified, additional info will be included in the
> +event, with similar semantics to a 'type' expression. Finally there
> +will be C API generated in qapi-event.h; when called by QEMU code, a
> +message with timestamp will be emitted on the wire.
>
> An example event is:
>
> @@ -311,7 +531,7 @@ Example:
> #ifndef EXAMPLE_QAPI_TYPES_H
> #define EXAMPLE_QAPI_TYPES_H
>
> -[Builtin types omitted...]
> +[Built-in types omitted...]
>
> typedef struct UserDefOne UserDefOne;
>
> @@ -324,7 +544,7 @@ Example:
> struct UserDefOneList *next;
> } UserDefOneList;
>
> -[Functions on builtin types omitted...]
> +[Functions on built-in types omitted...]
>
> struct UserDefOne
> {
> @@ -423,7 +643,7 @@ Example:
> #ifndef EXAMPLE_QAPI_VISIT_H
> #define EXAMPLE_QAPI_VISIT_H
>
> -[Visitors for builtin types omitted...]
> +[Visitors for built-in types omitted...]
>
> void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp);
> void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp);
> diff --git a/docs/qmp/qmp-spec.txt b/docs/qmp/qmp-spec.txt
> index 22568c6..1fb642b 100644
> --- a/docs/qmp/qmp-spec.txt
> +++ b/docs/qmp/qmp-spec.txt
> @@ -1,10 +1,20 @@
> QEMU Machine Protocol Specification
>
> +0. About This Document
> +======================
> +
> +This document is licensed under the GPLv2 (or later).
> +
> +Last revised in March 2015.
> +
> 1. Introduction
> ===============
>
> -This document specifies the QEMU Machine Protocol (QMP), a JSON-based protocol
> -which is available for applications to operate QEMU at the machine-level.
> +This document specifies the QEMU Machine Protocol (QMP), a JSON-based
> +protocol which is available for applications to operate QEMU at the
> +machine-level. It is also in use by the QEMU Guest Agent (QGA), which
> +is available for host applications to interact with the guest
> +operating system.
>
> 2. Protocol Specification
> =========================
> @@ -23,9 +33,11 @@ the JSON standard:
>
> http://www.ietf.org/rfc/rfc4627.txt
Upgrade to RFC 7159?
>
> -For convenience, json-object members and json-array elements mentioned in
> -this document will be in a certain order. However, in real protocol usage
> -they can be in ANY order, thus no particular order should be assumed.
> +For convenience, json-object members mentioned in this document will
> +be in a certain order. However, in real protocol usage they can be in
> +ANY order, thus no particular order should be assumed. On the other
> +hand, use of json-array elements presumes that preserving order is
> +important unless specifically documented otherwise.
>
> 2.1 General Definitions
> -----------------------
> @@ -52,7 +64,19 @@ The greeting message format is:
> - The "version" member contains the Server's version information (the format
> is the same of the query-version command)
> - The "capabilities" member specify the availability of features beyond the
> - baseline specification
> + baseline specification; the order of elements in this array has no
> + particular significance, so a client must search the entire array
> + when looking for a particular capability
> +
> +When first connecting to the server, the connection is in a capability
> +exchange mode, further documented below.
> +
> +2.2.1 Capabilities
> +------------------
> +
> +As of the date this document was last revised, no server or client
> +capability strings have been defined.
> +
>
> 2.3 Issuing Commands
> --------------------
> @@ -81,13 +105,15 @@ of a command execution: success or error.
>
> The format of a success response is:
>
> -{ "return": json-object, "id": json-value }
> +{ "return": json-entity, "id": json-value }
Unlike the other json-FOOs we use, "entity" isn't defined in RFC4627.
"value" is, and we already use json-value. What's the difference
between the two?
>
> Where,
>
> -- The "return" member contains the command returned data, which is defined
> - in a per-command basis or an empty json-object if the command does not
> - return data
> +- The "return" member contains the data returned by the command, which
> + is defined on a per-command basis (usually a json-object or
> + json-array of json-objects, but sometimes a json-value, json-string,
> + or json-array of json-strings); it is an empty json-object if the
> + command does not return data
Err, aren't json-object, -string, -array all json-value? At least
that's how the JSON RFC uses "value".
> - The "id" member contains the transaction identification associated
> with the command execution if issued by the Client
>
> @@ -114,7 +140,8 @@ if provided by the client.
> -----------------------
>
> As a result of state changes, the Server may send messages unilaterally
> -to the Client at any time. They are called "asynchronous events".
> +to the Client at any time, when not in the middle of any other
> +response. They are called "asynchronous events".
>
> The format of asynchronous events is:
>
> @@ -126,13 +153,27 @@ The format of asynchronous events is:
> - The "event" member contains the event's name
> - The "data" member contains event specific data, which is defined in a
> per-event basis, it is optional
> -- The "timestamp" member contains the exact time of when the event occurred
> - in the Server. It is a fixed json-object with time in seconds and
> - microseconds
> +- The "timestamp" member contains the exact time of when the event
> + occurred in the Server. It is a fixed json-object with time in
> + seconds and microseconds relative to the Unix Epoch (1 Jan 1970); if
> + there is a failure to retrieve host time, both members of the
> + timestamp will be set to -1.
>
> For a listing of supported asynchronous events, please, refer to the
> qmp-events.txt file.
>
> +2.5 QGA Synchronization
> +-----------------------
> +
> +When using QGA, an additional synchronization feature is built into
> +the protocol. If the Client sends a raw 0xFF sentinel byte (not valid
> +JSON), then the Server will reset its state and discard all pending
> +data prior to the sentinel. Conversely, if the Client makes use of
> +the 'guest-sync-delimited' command, the Server will send a raw 0xFF
> +sentinel byte prior to its response, to aid the Client in discarding
> +any data prior to the sentinel.
> +
> +
> 3. QMP Examples
> ===============
>
> @@ -145,32 +186,37 @@ This section provides some examples of real QMP usage, in all of them
> S: { "QMP": { "version": { "qemu": { "micro": 50, "minor": 6, "major": 1 },
> "package": ""}, "capabilities": []}}
>
> -3.2 Simple 'stop' execution
> +3.2 Client QMP negotiation
> +--------------------------
> +C: { "execute": "qmp_capabilities" }
> +S: { "return": {}}
> +
> +3.3 Simple 'stop' execution
> ---------------------------
>
> C: { "execute": "stop" }
> S: { "return": {} }
>
> -3.3 KVM information
> +3.4 KVM information
> -------------------
>
> C: { "execute": "query-kvm", "id": "example" }
> S: { "return": { "enabled": true, "present": true }, "id": "example"}
>
> -3.4 Parsing error
> +3.5 Parsing error
> ------------------
>
> C: { "execute": }
> S: { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } }
>
> -3.5 Powerdown event
> +3.6 Powerdown event
> -------------------
>
> S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 },
> "event": "POWERDOWN" }
>
> 4. Capabilities Negotiation
> -----------------------------
> +===========================
>
> When a Client successfully establishes a connection, the Server is in
> Capabilities Negotiation mode.
> @@ -189,7 +235,7 @@ effect, all commands (except qmp_capabilities) are allowed and asynchronous
> messages are delivered.
>
> 5 Compatibility Considerations
> -------------------------------
> +==============================
>
> All protocol changes or new features which modify the protocol format in an
> incompatible way are disabled by default and will be advertised by the
> @@ -213,12 +259,16 @@ However, Clients must not assume any particular:
> - Amount of errors generated by a command, that is, new errors can be added
> to any existing command in newer versions of the Server
>
> +Any command or field name beginning with "x-" is deemed experimental,
> +and may be withdrawn or changed in an incompatible manner in a future
> +release.
> +
> Of course, the Server does guarantee to send valid JSON. But apart from
> this, a Client should be "conservative in what they send, and liberal in
> what they accept".
>
> 6. Downstream extension of QMP
> -------------------------------
> +==============================
>
> We recommend that downstream consumers of QEMU do *not* modify QMP.
> Management tools should be able to support both upstream and downstream
Massive improvement. We can always improve some more on top, thus:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-25 18:31 ` Markus Armbruster
@ 2015-03-25 20:11 ` Eric Blake
2015-03-25 21:15 ` Eric Blake
` (2 more replies)
0 siblings, 3 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-25 20:11 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 14541 bytes --]
On 03/25/2015 12:31 PM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Go into more details about the various types of valid expressions
>> in a qapi schema, including tweaks to document fixes being done
>> later in the current patch series. Also fix some stale and missing
>> documentation in the QMP specification.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>>
>> I'm not sure if it is okay to assert GPLv2+ licensing without an
>> explicit Copyright, but as I am not the original author, I don't
>> know who to attribute any original Copyright to. Advice? Should
>> I split the license addition to a separate patch?
No thoughts to this question?
>>
>> +There are six top-level expressions recognized by the parser:
>> +'include', 'command', 'type', 'enum', 'union', and 'event'. There are
>> +several built-in types, such as 'int' and 'str'; additionally, the
>> +top-level expressions can define complex types, enumeration types, and
>> +several flavors of union types. The 'command' expression can refer to
>> +existing types by name, or list an anonymous type as a dictionary.
>
> 'event' can do that, too.
Yep. I can fix that on top if no respin is needed.
>> +Types, commands, and events share a common namespace. Therefore,
>> +generally speaking, type definitions should always use CamelCase for
>> +user-defined type names, while built-in types are lowercase. Type
>> +definitions should not end in 'Kind', as this namespace is used for
>> +creating implicit C enums for visiting union types. Command names,
>> +and field names within a type, should be all lower case with words
>> +separated by a hyphen. However, some existing older commands and
>> +complex types use underscore; when extending such expressions,
>> +consistency is preferred over blindly avoiding underscore. Event
>> +names should be ALL_CAPS with words separated by underscore. The
>> +special string '**' appears for some commands that manually perform
>> +their own type checking rather than relying on the type-safe code
>> +produced by the qapi code generators.
>
> The generator generates C identifiers pretty thoughtlessly, and is
> therefore prone to generate clashes. Fixable, but we've got bigger fish
> to fry, and this series is hefty enough as it is. Documenting the mess
> instead makes sense.
And I actually DO tighten up the parser to flag some of these problems
later in the series.
>> The QAPI schema definitions can be modularized using the 'include' directive:
>>
>> { 'include': 'path/to/file.json'}
>
> If you need to respin for some reason, insert a space before the closing
> brace.
Sure; added to my on-top-or-respin pile.
>>
>> -A complex type is a dictionary containing a single key whose value is a
>> -dictionary. This corresponds to a struct in C or an Object in JSON. An
>> -example of a complex type is:
>> +Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
>
> Here, the reader needs to get that 'type' and 'data' and 'base' are
> literals, but 'str', 'dict' and 'complex-type-name' are placeholders. I
> like setting apart placeholders with typograhic conventions. If we want
> to do that, let's do it on top.
I was trying to borrow from qapi syntax, in that the 'data' dictionary
uses '*name':'type' for a literal "name" coupled with a
value-of-that-type pair in the dictionary sent over the wire for
"arguments". As for a typographical convention, would either of these
be any easier to read?
Usage: { 'type': 'STR', 'data': 'DICT', '*base': 'COMPLEX-TYPE-NAME' }
or
Usage: { 'type': string, 'data': dict [, 'base': complex-type-name] }
>
> '*base' is interesting. It's a tacit use of the schema language's
> "'*name' means member 'name' is optional" rule on meta-schema-level. We
> can take care of that when/if we clarify placeholders.
>
>> +
>> +A complex type is a dictionary containing a single 'data' key whose
>> +value is a dictionary. This corresponds to a struct in C or an Object
>> +in JSON. Each value of the 'data' dictionary must be the name of a
>> +complex, enum, union, or built-in type,
>
> Are there any others? If not, we can simplify to
>
> Each value of the 'data' dictionary must be a type name
Works for me, especially since the addition of 'alternate' later in this
series is also a type that works in this context.
>
>> or a one-element array
>> +containing a type name. An example of a complex type is:
>>
>> { 'type': 'MyType',
>> 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
>>
>> -The use of '*' as a prefix to the name means the member is optional.
>> +The use of '*' as a prefix to the name means the member is optional in
>> +the corresponding QMP usage.
>
> Not just in QMP, also QGA and (internal) C interfaces. Since explaining
> that is tiresome, I'd be tempted to stick to the original sentence here.
>
> Preexisting: we sometimes say "QMP wire format", and sometimes simply
> "JSON". Again, we got bigger fish to fry.
If I do anything for this, it will be a separate patch on top scrubbing
for all uses.
>> +Additionally, an implicit C enum NameKind is created, corresponding to
>> +the union Name, for accessing the various branches of the union. No
>> +branch of the union can be named 'max', as this would collide with the
>> +implicit enum.
>
> Aside: I guess this implicit enum is just like a normal QAPI enum,
> except it doesn't have a name in the schema.
Implemented identically; and why I want to eventually add:
{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
{ 'union': 'Name', 'discriminator': 'Enum',
'data': { 'one': 'str', 'two': 'int' } }
{ 'command': 'foo', 'data': 'Name' }
to allow this on the wire:
{ "execute": "foo", "arguments": { "type": "one", "data": "string" } }
{ "execute": "foo", "arguments": { "type": "two", "data": 1 } }
>> { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
>> { 'union': 'BlockdevOptions',
>
> Worth mentioning explicitly that the base type's members are common to
> all the union's cases?
Sure.
>> +The final flavor of unions is an anonymous union. While the other two
>> +union types are always passed as a dictionary in the wire format, an
>> +anonymous union instead allows the direct use of different types in
>> +its place. As they aren't structured, they don't have any explicit
>> +discriminator but use the (QObject) data type of their value as an
>> +implicit discriminator. This means that they are restricted to using
>> +only one discriminator value per QObject type. For example, you cannot
>> +have two different complex types in an anonymous union, or two
>> +different integer types.
>
> This is the first mention of "QObject type". How it's related to JSON
> is left to the reader's deductive skills :)
>
> The QObject types are QTYPE_NONE, QTYPE_QINT, QTYPE_QSTRING,
> QTYPE_QDICT, QTYPE_QLIST, QTYPE_QFLOAT, QTYPE_QBOOL, QTYPE_QERROR.
>
> The connections JSON string - QTYPE_QSTRING, JSON object - QTYPE_QDICT,
> JSON array - QTYPE_QLIST and JSON boolean - QTYPE_QBOOL are obvious
> enough.
>
> If I remember correctly, our JSON parser chokes on the JSON keyword
> null.
>
> That leaves just JSON numbers - QTYPE_QINT or QTYPE_QFLOAT. Can an
> anonymous union have a separate case for each of the two?
>
> For completeness: on the QTYPE_ side, it leaves QTYPE_QERROR and
> QTYPE_NONE. QTYPE_QERROR is internal only, and will hopefully be gone
> soon. I can't see QTYPE_NONE objects being created anywhere.
>
> Should we explain this in terms of JSON types instead of QObject types?
Yeah, makes sense. Sounds like my on-top patch is getting bigger.
>
>>
>> Anonymous unions are declared using an empty dictionary as their discriminator.
>> The discriminator values never appear on the wire, they are only used in the
>> @@ -200,23 +350,93 @@ This example allows using both of the following example objects:
>>
>> === Commands ===
>>
>> -Commands are defined by using a list containing three members. The first
>> -member is the command name, the second member is a dictionary containing
>> -arguments, and the third member is the return type.
>> +Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
>> + '*returns': 'type',
>
> Shoudn't this be '*returns': 'dict-or-type-name'?
Yep. Good catch.
>
>> + '*gen': false, '*success-response': false }
>>
>> -An example command is:
>> +Commands are defined by using a dictionary containing several members,
>> +where three members are most common. The 'command' member is a
>> +mandatory string, and determines the "execute" value passed in a QMP
>> +command exchange.
>> +
>> +The 'data' member is optional; if absent, the command accepts an
>> +optional empty dictionary.
>
> Suggest: The 'data' member is optional, and defaults to {}.
>
>> + If present, it must be the string name of
>> +a complex type, a one-element array containing the name of a complex
>> +type, or a dictionary that declares an anonymous type with the same
>> +semantics as a 'type' expression, with one exception noted below when
>> +'gen' is used. The 'data' argument maps to the "arguments" dictionary
>> +passed in as part of a QMP command.
>
> Suggest to move the last sentence to the beginning of the paragaph.
Good ideas.
>
>> +
>> +The 'returns' member describes what will appear in the "return" field
>> +of a QMP reply on successful completion of a command. The member is
>> +optional from the command declaration; if absent, the "return" field
>> +will be an empty dictionary. If 'returns' is present, it must be the
>> +string name of a complex or built-in type, a one-element array
>> +containing the name of a complex or built-in type, or a dictionary
>> +that declares an anonymous type with the same semantics as a 'type'
>> +expression, with one exception noted below when 'gen' is used.
>
> Oh, 'gen' affect 'returns', too? Live and learn...
Yes; thanks to 'qom-get'.
>> +
>> + { 'command': 'netdev_add',
>> + 'data': {'type': 'str', 'id': 'str', '*props': '**'},
>> + 'gen': false }
>
> We use 'gen': 'no' until PATCH 18.
Yeah. I debated about that one, whether to do all my docs to their
final state up front, or whether to doc existing practice and then tweak
it to be more specific later in the series as fixes were made. I guess
you can see which way I chose. And of course, if I do more followups on
top of the series, I no longer have all changes in one place. Oh well;
doc changes don't affect bisection :)
(well, I did more docs later when creating 'alternate' in place of
anonymous union, but still limited to two patches)
>
>> +
>> +Normally, the QAPI schema is used to describe synchronous exchanges,
>> +where a response is expected. But in some cases, the action of a
>> +command is expected to change state in a way that a successful
>> +response is not possible (the command still returns a normal
>> +dictionary error on failure). When a successful reply is not
>> +possible, the command expression should include the optional key
>> +'success-response' with boolean value false. So far, only the
>> +qemu-guest-agent makes use of this field.
>
> I had no idea :)
You said that last time :)
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg04254.html
>>
>> http://www.ietf.org/rfc/rfc4627.txt
>
> Upgrade to RFC 7159?
Sure; wasn't even aware it was out there.
/me reads it
Looks like it added hints about objects being true dictionaries
(repeating a name leads to unpredictable results, you can't rely on
order being preserved in the dictionary by default), clarified that
arrays need not be homogeneous types; gave more details on the limits of
numbers, spoke more about consequences of encoding in other than UTF-8,
and required that escape sequences be normalized before comparing
strings for equality. None of those should really hurt our usage (well,
we DO document that our use of json-object for 'data' member is
special-cased in that order is significant to the generated C code but
not to the QMP wire format).
>>
>> The format of a success response is:
>>
>> -{ "return": json-object, "id": json-value }
>> +{ "return": json-entity, "id": json-value }
>
> Unlike the other json-FOOs we use, "entity" isn't defined in RFC4627.
> "value" is, and we already use json-value. What's the difference
> between the two?
Umm, when I wrote that, I was thinking "id":json-value meant integer, so
I wanted something that was not an integer. But sure enough, json-value
is precisely the term I wanted to use:
$ qemu-kvm -qmp stdio -nodefaults
{"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 2},
"package": " (qemu-2.3.0-0.1.rc0.fc21)"}, "capabilities": []}}
{"execute":"qmp_capabilities","id":[]}
{"return": {}, "id": []}
{"execute":"huh", "id":{"a":1,"b":true}}
{"id": {"a": 1, "b": true}, "error": {"class": "CommandNotFound",
"desc": "The command huh has not been found"}}
So _I_ learned something (the id can be anything at all!)
Looks like I have some wording improvements to add.
>
>>
>> Where,
>>
>> -- The "return" member contains the command returned data, which is defined
>> - in a per-command basis or an empty json-object if the command does not
>> - return data
>> +- The "return" member contains the data returned by the command, which
>> + is defined on a per-command basis (usually a json-object or
>> + json-array of json-objects, but sometimes a json-value, json-string,
>> + or json-array of json-strings); it is an empty json-object if the
>> + command does not return data
>
> Err, aren't json-object, -string, -array all json-value? At least
> that's how the JSON RFC uses "value".
Yep.
>
> Massive improvement. We can always improve some more on top, thus:
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Thank you. If there's cause for respin, I can fold in some improvements
then; otherwise, I'm already working on putting my changes into a
followup patch.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-25 20:11 ` Eric Blake
@ 2015-03-25 21:15 ` Eric Blake
2015-03-26 9:09 ` Markus Armbruster
2015-03-26 7:52 ` Markus Armbruster
2015-03-26 8:09 ` Markus Armbruster
2 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-25 21:15 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 2906 bytes --]
On 03/25/2015 02:11 PM, Eric Blake wrote:
>> The QObject types are QTYPE_NONE, QTYPE_QINT, QTYPE_QSTRING,
>> QTYPE_QDICT, QTYPE_QLIST, QTYPE_QFLOAT, QTYPE_QBOOL, QTYPE_QERROR.
>>
>> The connections JSON string - QTYPE_QSTRING, JSON object - QTYPE_QDICT,
>> JSON array - QTYPE_QLIST and JSON boolean - QTYPE_QBOOL are obvious
>> enough.
>>
>> If I remember correctly, our JSON parser chokes on the JSON keyword
>> null.
Yep, that is sadly still the case [1]:
$ qemu-kvm -qmp stdio -nodefaults
{"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 2},
"package": " (qemu-2.3.0-0.1.rc0.fc21)"}, "capabilities": []}}
{"execute":"qmp_capabilities","id":null}
{"error": {"class": "GenericError", "desc": "Invalid JSON syntax"}}
Patch 17 adds support for it to qapi schemas, but not to our JSON
parser. I guess that's yet another task to be solved before we turn on
argument defaults and introspection (as returning 'null' would be a
logical solution for introspecting the default value of an optional
string argument. On the other hand, we could argue that using 'null' in
output of introspection still doesn't require the parser to accept
'null' on input; the output direction of introspection is different
than the input direction of parsing a 'null' in user commands).
>>
>> That leaves just JSON numbers - QTYPE_QINT or QTYPE_QFLOAT. Can an
>> anonymous union have a separate case for each of the two?
Yes. Don't know why we'd want it, but the code currently handles it.
That is, this compiles just fine:
{ 'alternate': 'Foo', 'data': { 'a': 'int', 'b': 'number' } }
{ 'command': 'bar', 'data': { 'value': 'Foo' } }
allowing:
{"execute":"bar", "arguments":{ "value":1 } }
{"execute":"bar", "arguments":{ "value":1.0 } }
as operating on uint64_t vs. double (in practice, since '1' is also
valid input as a number, a double is sufficient without needing the
alternative of a uint64_t, unless you are simultaneously worrying about
precise integral values larger than 2**53 that lose data when converted
to double, while still allowing for inputs larger than 2**63 via double)
>>> The format of a success response is:
>>>
>>> -{ "return": json-object, "id": json-value }
>>> +{ "return": json-entity, "id": json-value }
>>
>> Unlike the other json-FOOs we use, "entity" isn't defined in RFC4627.
>> "value" is, and we already use json-value. What's the difference
>> between the two?
>
> Umm, when I wrote that, I was thinking "id":json-value meant integer, so
> I wanted something that was not an integer. But sure enough, json-value
> is precisely the term I wanted to use:
Well, given above at [1] that 'null' is a valid json-value but NOT
accepted by our parser, I guess we are not quite accurate here.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-25 20:11 ` Eric Blake
2015-03-25 21:15 ` Eric Blake
@ 2015-03-26 7:52 ` Markus Armbruster
2015-03-30 15:23 ` Eric Blake
2015-03-26 8:09 ` Markus Armbruster
2 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 7:52 UTC (permalink / raw)
To: Eric Blake
Cc: kwolf, famz, wenchaoqemu, Michael Roth, qemu-devel, lcapitulino
Copyright note question, copying Michael.
Eric Blake <eblake@redhat.com> writes:
> On 03/25/2015 12:31 PM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Go into more details about the various types of valid expressions
>>> in a qapi schema, including tweaks to document fixes being done
>>> later in the current patch series. Also fix some stale and missing
>>> documentation in the QMP specification.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>>>
>>> I'm not sure if it is okay to assert GPLv2+ licensing without an
>>> explicit Copyright, but as I am not the original author, I don't
>>> know who to attribute any original Copyright to. Advice? Should
>>> I split the license addition to a separate patch?
>
> No thoughts to this question?
Missed it.
I wish we didn't need to clutter copyright and licensing boiler plate
everywhere, but I accept it's the prudent thing to do in a tree with so
many differently licensed parts.
Making GPLv2+ explicit is obviously fine, because anything without an
explicit licensing note is GPLv2+ (see ./LICENSE). That leaves the
copyright part, as you say.
According to git-log, the file was created by Michael Roth. git-blame
blames 219 out of 590 lines in current master on his initial commit.
$ git-blame -w master docs/qapi-code-gen.txt | cut -c 11-28 | sort | uniq -c | sort -nr
219 Michael Roth
124 Markus Armbruster
116 Kevin Wolf
95 Eric Blake
21 Wenchao Xia
10 Lluís Vilanova
2 Stefan Weil
2 Stefan Hajnoczi
1 Benoît Canet
If you want to add a copyright note, I suggest to steal one from
Michael's work elsewhere, and update it for later major contributors.
Here's my stab at it:
= How to use the QAPI code generator =
Copyright IBM Corp. 2011
Copyright (C) 2012-2015 Red Hat, Inc.
Authors:
Michael Roth <mdroth@linux.vnet.ibm.com>
Kevin Wolf <kwolf@redhat.com>
Markus Armbruster <armbru@redhat.com>
Eric Blake <eblake@redhat.com>
This work is licensed under the terms of the GNU GPL, version 2 or later.
See the COPYING file in the top-level directory.
== Introduction ==
QAPI is a native C API within QEMU which provides management-level
functionality to internal and external users. For external
The Authors paragraph is informational and could be omitted without
compromising the copyright note.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-25 20:11 ` Eric Blake
2015-03-25 21:15 ` Eric Blake
2015-03-26 7:52 ` Markus Armbruster
@ 2015-03-26 8:09 ` Markus Armbruster
2 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 8:09 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/25/2015 12:31 PM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Go into more details about the various types of valid expressions
>>> in a qapi schema, including tweaks to document fixes being done
>>> later in the current patch series. Also fix some stale and missing
>>> documentation in the QMP specification.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>>>
>>> I'm not sure if it is okay to assert GPLv2+ licensing without an
>>> explicit Copyright, but as I am not the original author, I don't
>>> know who to attribute any original Copyright to. Advice? Should
>>> I split the license addition to a separate patch?
>
> No thoughts to this question?
Separate message.
>>>
>>> +There are six top-level expressions recognized by the parser:
>>> +'include', 'command', 'type', 'enum', 'union', and 'event'. There are
>>> +several built-in types, such as 'int' and 'str'; additionally, the
>>> +top-level expressions can define complex types, enumeration types, and
>>> +several flavors of union types. The 'command' expression can refer to
>>> +existing types by name, or list an anonymous type as a dictionary.
>>
>> 'event' can do that, too.
>
> Yep. I can fix that on top if no respin is needed.
>
>
>>> +Types, commands, and events share a common namespace. Therefore,
>>> +generally speaking, type definitions should always use CamelCase for
>>> +user-defined type names, while built-in types are lowercase. Type
>>> +definitions should not end in 'Kind', as this namespace is used for
>>> +creating implicit C enums for visiting union types. Command names,
>>> +and field names within a type, should be all lower case with words
>>> +separated by a hyphen. However, some existing older commands and
>>> +complex types use underscore; when extending such expressions,
>>> +consistency is preferred over blindly avoiding underscore. Event
>>> +names should be ALL_CAPS with words separated by underscore. The
>>> +special string '**' appears for some commands that manually perform
>>> +their own type checking rather than relying on the type-safe code
>>> +produced by the qapi code generators.
>>
>> The generator generates C identifiers pretty thoughtlessly, and is
>> therefore prone to generate clashes. Fixable, but we've got bigger fish
>> to fry, and this series is hefty enough as it is. Documenting the mess
>> instead makes sense.
>
> And I actually DO tighten up the parser to flag some of these problems
> later in the series.
>
>
>>> The QAPI schema definitions can be modularized using the 'include'
>>> directive:
>>>
>>> { 'include': 'path/to/file.json'}
>>
>> If you need to respin for some reason, insert a space before the closing
>> brace.
>
> Sure; added to my on-top-or-respin pile.
>
>
>>>
>>> -A complex type is a dictionary containing a single key whose value is a
>>> -dictionary. This corresponds to a struct in C or an Object in JSON. An
>>> -example of a complex type is:
>>> +Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
>>
>> Here, the reader needs to get that 'type' and 'data' and 'base' are
>> literals, but 'str', 'dict' and 'complex-type-name' are placeholders. I
>> like setting apart placeholders with typograhic conventions. If we want
>> to do that, let's do it on top.
>
> I was trying to borrow from qapi syntax, in that the 'data' dictionary
> uses '*name':'type' for a literal "name" coupled with a
> value-of-that-type pair in the dictionary sent over the wire for
> "arguments". As for a typographical convention, would either of these
> be any easier to read?
>
> Usage: { 'type': 'STR', 'data': 'DICT', '*base': 'COMPLEX-TYPE-NAME' }
>
> or
>
> Usage: { 'type': string, 'data': dict [, 'base': complex-type-name] }
Or
Usage: { 'type': STRING, 'data': DICT [, 'base': COMPLEX-TYPE-NAME] }
Makes the placeholders stand out. Unambiguous as long as we don't use
ALL-CAPS for anything else outside quotes.
Using QAPI's *-prefix instead of EBNF's brackets
Usage: { 'type': STRING, 'data': DICT , '*base': COMPLEX-TYPE-NAME }
would also work, but I feel the *-prefix needs to be explained earlier
then.
>>
>> '*base' is interesting. It's a tacit use of the schema language's
>> "'*name' means member 'name' is optional" rule on meta-schema-level. We
>> can take care of that when/if we clarify placeholders.
>>
>>> +
>>> +A complex type is a dictionary containing a single 'data' key whose
>>> +value is a dictionary. This corresponds to a struct in C or an Object
>>> +in JSON. Each value of the 'data' dictionary must be the name of a
>>> +complex, enum, union, or built-in type,
>>
>> Are there any others? If not, we can simplify to
>>
>> Each value of the 'data' dictionary must be a type name
>
> Works for me, especially since the addition of 'alternate' later in this
> series is also a type that works in this context.
>
>>
>>> or a one-element array
>>> +containing a type name. An example of a complex type is:
>>>
>>> { 'type': 'MyType',
>>> 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
>>>
>>> -The use of '*' as a prefix to the name means the member is optional.
>>> +The use of '*' as a prefix to the name means the member is optional in
>>> +the corresponding QMP usage.
>>
>> Not just in QMP, also QGA and (internal) C interfaces. Since explaining
>> that is tiresome, I'd be tempted to stick to the original sentence here.
>>
>> Preexisting: we sometimes say "QMP wire format", and sometimes simply
>> "JSON". Again, we got bigger fish to fry.
>
> If I do anything for this, it will be a separate patch on top scrubbing
> for all uses.
Absolutely.
>>> +Additionally, an implicit C enum NameKind is created, corresponding to
>>> +the union Name, for accessing the various branches of the union. No
>>> +branch of the union can be named 'max', as this would collide with the
>>> +implicit enum.
>>
>> Aside: I guess this implicit enum is just like a normal QAPI enum,
>> except it doesn't have a name in the schema.
>
> Implemented identically; and why I want to eventually add:
>
> { 'enum': 'Enum', 'data': [ 'one', 'two' ] }
> { 'union': 'Name', 'discriminator': 'Enum',
> 'data': { 'one': 'str', 'two': 'int' } }
> { 'command': 'foo', 'data': 'Name' }
>
> to allow this on the wire:
>
> { "execute": "foo", "arguments": { "type": "one", "data": "string" } }
> { "execute": "foo", "arguments": { "type": "two", "data": 1 } }
>
>>> { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
>>> { 'union': 'BlockdevOptions',
>>
>> Worth mentioning explicitly that the base type's members are common to
>> all the union's cases?
>
> Sure.
>
>
>>> +The final flavor of unions is an anonymous union. While the other two
>>> +union types are always passed as a dictionary in the wire format, an
>>> +anonymous union instead allows the direct use of different types in
>>> +its place. As they aren't structured, they don't have any explicit
>>> +discriminator but use the (QObject) data type of their value as an
>>> +implicit discriminator. This means that they are restricted to using
>>> +only one discriminator value per QObject type. For example, you cannot
>>> +have two different complex types in an anonymous union, or two
>>> +different integer types.
>>
>> This is the first mention of "QObject type". How it's related to JSON
>> is left to the reader's deductive skills :)
>>
>> The QObject types are QTYPE_NONE, QTYPE_QINT, QTYPE_QSTRING,
>> QTYPE_QDICT, QTYPE_QLIST, QTYPE_QFLOAT, QTYPE_QBOOL, QTYPE_QERROR.
>>
>> The connections JSON string - QTYPE_QSTRING, JSON object - QTYPE_QDICT,
>> JSON array - QTYPE_QLIST and JSON boolean - QTYPE_QBOOL are obvious
>> enough.
>>
>> If I remember correctly, our JSON parser chokes on the JSON keyword
>> null.
>>
>> That leaves just JSON numbers - QTYPE_QINT or QTYPE_QFLOAT. Can an
>> anonymous union have a separate case for each of the two?
>>
>> For completeness: on the QTYPE_ side, it leaves QTYPE_QERROR and
>> QTYPE_NONE. QTYPE_QERROR is internal only, and will hopefully be gone
>> soon. I can't see QTYPE_NONE objects being created anywhere.
>>
>> Should we explain this in terms of JSON types instead of QObject types?
>
> Yeah, makes sense. Sounds like my on-top patch is getting bigger.
>
>>
>>>
>>> Anonymous unions are declared using an empty dictionary as their
>>> discriminator.
>>> The discriminator values never appear on the wire, they are only used in the
>>> @@ -200,23 +350,93 @@ This example allows using both of the
>>> following example objects:
>>>
>>> === Commands ===
>>>
>>> -Commands are defined by using a list containing three members. The first
>>> -member is the command name, the second member is a dictionary containing
>>> -arguments, and the third member is the return type.
>>> +Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
>>> + '*returns': 'type',
>>
>> Shoudn't this be '*returns': 'dict-or-type-name'?
>
> Yep. Good catch.
>
>>
>>> + '*gen': false, '*success-response': false }
>>>
>>> -An example command is:
>>> +Commands are defined by using a dictionary containing several members,
>>> +where three members are most common. The 'command' member is a
>>> +mandatory string, and determines the "execute" value passed in a QMP
>>> +command exchange.
>>> +
>>> +The 'data' member is optional; if absent, the command accepts an
>>> +optional empty dictionary.
>>
>> Suggest: The 'data' member is optional, and defaults to {}.
>>
>>> + If present, it must be the string name of
>>> +a complex type, a one-element array containing the name of a complex
>>> +type, or a dictionary that declares an anonymous type with the same
>>> +semantics as a 'type' expression, with one exception noted below when
>>> +'gen' is used. The 'data' argument maps to the "arguments" dictionary
>>> +passed in as part of a QMP command.
>>
>> Suggest to move the last sentence to the beginning of the paragaph.
>
> Good ideas.
>
>>
>>> +
>>> +The 'returns' member describes what will appear in the "return" field
>>> +of a QMP reply on successful completion of a command. The member is
>>> +optional from the command declaration; if absent, the "return" field
>>> +will be an empty dictionary. If 'returns' is present, it must be the
>>> +string name of a complex or built-in type, a one-element array
>>> +containing the name of a complex or built-in type, or a dictionary
>>> +that declares an anonymous type with the same semantics as a 'type'
>>> +expression, with one exception noted below when 'gen' is used.
>>
>> Oh, 'gen' affect 'returns', too? Live and learn...
>
> Yes; thanks to 'qom-get'.
>
>
>>> +
>>> + { 'command': 'netdev_add',
>>> + 'data': {'type': 'str', 'id': 'str', '*props': '**'},
>>> + 'gen': false }
>>
>> We use 'gen': 'no' until PATCH 18.
>
> Yeah. I debated about that one, whether to do all my docs to their
> final state up front, or whether to doc existing practice and then tweak
> it to be more specific later in the series as fixes were made. I guess
> you can see which way I chose. And of course, if I do more followups on
> top of the series, I no longer have all changes in one place. Oh well;
> doc changes don't affect bisection :)
>
> (well, I did more docs later when creating 'alternate' in place of
> anonymous union, but still limited to two patches)
Not enough OCD to make you evolve the documentation in lockstep with the
code 100% ;-)
>>> +
>>> +Normally, the QAPI schema is used to describe synchronous exchanges,
>>> +where a response is expected. But in some cases, the action of a
>>> +command is expected to change state in a way that a successful
>>> +response is not possible (the command still returns a normal
>>> +dictionary error on failure). When a successful reply is not
>>> +possible, the command expression should include the optional key
>>> +'success-response' with boolean value false. So far, only the
>>> +qemu-guest-agent makes use of this field.
>>
>> I had no idea :)
>
> You said that last time :)
> https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg04254.html
I guess we're approaching the state where we'll have forgotten more
about the QAPI schema language than the average QEMU developer needs to
know...
>>>
>>> http://www.ietf.org/rfc/rfc4627.txt
>>
>> Upgrade to RFC 7159?
>
> Sure; wasn't even aware it was out there.
Me neither, until I stumbled over it the other day.
> /me reads it
>
> Looks like it added hints about objects being true dictionaries
> (repeating a name leads to unpredictable results, you can't rely on
> order being preserved in the dictionary by default), clarified that
> arrays need not be homogeneous types; gave more details on the limits of
> numbers, spoke more about consequences of encoding in other than UTF-8,
> and required that escape sequences be normalized before comparing
> strings for equality. None of those should really hurt our usage (well,
> we DO document that our use of json-object for 'data' member is
> special-cased in that order is significant to the generated C code but
> not to the QMP wire format).
>
>
>>>
>>> The format of a success response is:
>>>
>>> -{ "return": json-object, "id": json-value }
>>> +{ "return": json-entity, "id": json-value }
>>
>> Unlike the other json-FOOs we use, "entity" isn't defined in RFC4627.
>> "value" is, and we already use json-value. What's the difference
>> between the two?
>
> Umm, when I wrote that, I was thinking "id":json-value meant integer, so
> I wanted something that was not an integer. But sure enough, json-value
> is precisely the term I wanted to use:
>
> $ qemu-kvm -qmp stdio -nodefaults
> {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 2},
> "package": " (qemu-2.3.0-0.1.rc0.fc21)"}, "capabilities": []}}
> {"execute":"qmp_capabilities","id":[]}
> {"return": {}, "id": []}
> {"execute":"huh", "id":{"a":1,"b":true}}
> {"id": {"a": 1, "b": true}, "error": {"class": "CommandNotFound",
> "desc": "The command huh has not been found"}}
>
> So _I_ learned something (the id can be anything at all!)
>
> Looks like I have some wording improvements to add.
>
>>
>>>
>>> Where,
>>>
>>> -- The "return" member contains the command returned data, which is defined
>>> - in a per-command basis or an empty json-object if the command does not
>>> - return data
>>> +- The "return" member contains the data returned by the command, which
>>> + is defined on a per-command basis (usually a json-object or
>>> + json-array of json-objects, but sometimes a json-value, json-string,
>>> + or json-array of json-strings); it is an empty json-object if the
>>> + command does not return data
>>
>> Err, aren't json-object, -string, -array all json-value? At least
>> that's how the JSON RFC uses "value".
>
> Yep.
>
>>
>> Massive improvement. We can always improve some more on top, thus:
>>
>> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
> Thank you. If there's cause for respin, I can fold in some improvements
> then; otherwise, I'm already working on putting my changes into a
> followup patch.
Followup patch works for me.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-25 21:15 ` Eric Blake
@ 2015-03-26 9:09 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 9:09 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 03/25/2015 02:11 PM, Eric Blake wrote:
>
>>> The QObject types are QTYPE_NONE, QTYPE_QINT, QTYPE_QSTRING,
>>> QTYPE_QDICT, QTYPE_QLIST, QTYPE_QFLOAT, QTYPE_QBOOL, QTYPE_QERROR.
>>>
>>> The connections JSON string - QTYPE_QSTRING, JSON object - QTYPE_QDICT,
>>> JSON array - QTYPE_QLIST and JSON boolean - QTYPE_QBOOL are obvious
>>> enough.
>>>
>>> If I remember correctly, our JSON parser chokes on the JSON keyword
>>> null.
>
> Yep, that is sadly still the case [1]:
>
> $ qemu-kvm -qmp stdio -nodefaults
> {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 2},
> "package": " (qemu-2.3.0-0.1.rc0.fc21)"}, "capabilities": []}}
> {"execute":"qmp_capabilities","id":null}
> {"error": {"class": "GenericError", "desc": "Invalid JSON syntax"}}
>
> Patch 17 adds support for it to qapi schemas, but not to our JSON
> parser. I guess that's yet another task to be solved before we turn on
> argument defaults and introspection (as returning 'null' would be a
> logical solution for introspecting the default value of an optional
> string argument. On the other hand, we could argue that using 'null' in
> output of introspection still doesn't require the parser to accept
> 'null' on input; the output direction of introspection is different
> than the input direction of parsing a 'null' in user commands).
Yes.
Our parser choking on null is simply a bug that happens not to matter
with our current usage.
>>>
>>> That leaves just JSON numbers - QTYPE_QINT or QTYPE_QFLOAT. Can an
>>> anonymous union have a separate case for each of the two?
>
> Yes. Don't know why we'd want it, but the code currently handles it.
> That is, this compiles just fine:
>
> { 'alternate': 'Foo', 'data': { 'a': 'int', 'b': 'number' } }
> { 'command': 'bar', 'data': { 'value': 'Foo' } }
>
> allowing:
> {"execute":"bar", "arguments":{ "value":1 } }
> {"execute":"bar", "arguments":{ "value":1.0 } }
> as operating on uint64_t vs. double (in practice, since '1' is also
> valid input as a number, a double is sufficient without needing the
> alternative of a uint64_t, unless you are simultaneously worrying about
> precise integral values larger than 2**53 that lose data when converted
> to double, while still allowing for inputs larger than 2**63 via double)
JSON leaves defining limits on range and precision of numbers to
implementations.
Many implementations limit them to IEEE double precision. Integers that
double can't represent exactly get rounded. Whether you write an
integer as string of digits or the same sting followed by a redundant
exponent or fraction such as .0 doesn't matter.
We limit differently. If the JSON number has neither a fraction nor an
exponent part, and it's representable as int64_t, then we parse it as
that. Else we parse it as double. See the big comment in
parse_literal(). Yes, the code there fails error handling 101.
Buys us exact representation of more integers at the price of subtly
different interpretation of JSON numbers with neither fraction nor
exponent part.
A few examples:
JSON QTYPE_ value exact?
1 QINT 1 yes
1.0 QFLOAT 1.0 yes
10000000000000001.0 QFLOAT 10000000000000000.0 no
10000000000000001 QINT 10000000000000001 yes
9223372036854775807 QINT 9223372036854775807 yes
9223372036854775808 QFLOAT 9223372036854775808.0 yes
9223372036854775809 QFLOAT 9223372036854775808.0 no
-9223372036854775808 QINT -9223372036854775808 yes
Yes, that means the integers in [2^63,2^64-1] are parsed as double.
uint64 in the schema is a pious lie.
>>>> The format of a success response is:
>>>>
>>>> -{ "return": json-object, "id": json-value }
>>>> +{ "return": json-entity, "id": json-value }
>>>
>>> Unlike the other json-FOOs we use, "entity" isn't defined in RFC4627.
>>> "value" is, and we already use json-value. What's the difference
>>> between the two?
>>
>> Umm, when I wrote that, I was thinking "id":json-value meant integer, so
>> I wanted something that was not an integer. But sure enough, json-value
>> is precisely the term I wanted to use:
>
> Well, given above at [1] that 'null' is a valid json-value but NOT
> accepted by our parser, I guess we are not quite accurate here.
I regard the parser choking on null as a bug. Bugs are implementation
detail ;)
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 02/28] qapi: Fix generation of 'size' builtin type
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 02/28] qapi: Fix generation of 'size' builtin type Eric Blake
@ 2015-03-26 9:52 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 9:52 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> We were missing the 'size' builtin type (which means that QAPI using
> [ 'size' ] would fail to compile).
I suspect it chokes on 'size' in anonymous unions, too.
> Futhermore, there was some
> redundancy between builtin_types[] and builtin_type_qtypes{}.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Split off "[PATCH v4 10/19] qapi: Better error messages for duplicated
expressions". Good.
> ---
> scripts/qapi-types.py | 10 +++++-----
> scripts/qapi-visit.py | 6 +++---
> scripts/qapi.py | 9 ++-------
> tests/qapi-schema/qapi-schema-test.json | 3 ++-
> tests/qapi-schema/qapi-schema-test.out | 2 +-
> 5 files changed, 13 insertions(+), 17 deletions(-)
>
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index db87218..e400b03 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -182,8 +182,8 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
>
> for key in members:
> qapi_type = members[key]
> - if builtin_type_qtypes.has_key(qapi_type):
> - qtype = builtin_type_qtypes[qapi_type]
> + if builtin_types.has_key(qapi_type):
> + qtype = builtin_types[qapi_type]
> elif find_struct(qapi_type):
> qtype = "QTYPE_QDICT"
> elif find_union(qapi_type):
Took me a bit of digging to make the connection to "there was some
redundancy between builtin_types[] and builtin_type_qtypes{}", and to
the patch to qapi.py.
You could split this patch up: one part for folding _qtypes into _types,
and another part for fixing 'size'. Not worth a respin by itself, so:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema
2015-03-24 20:33 ` Eric Blake
@ 2015-03-26 9:54 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 9:54 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/24/2015 02:03 PM, Eric Blake wrote:
>> Python 2 and Python 3 have a wild history of whether strings
>> default to ascii or unicode, where Python 3 requires checking
>> instanceof(foo, basestr) to cover all strings, but where that
>> code is not portable to Python 2. It's simpler to just state
>> that we don't care about Unicode strings, and to just always
>> use the simpler instanceof(foo, str) everywhere.
>
> And for all my proof-reading, I already have a commit message change:
>
> s/instanceof/isinstance/
>
> (you can tell I'm not that proficient in python...)
>
>>
>> I'm no python expert, so I'm basing it on this conversation:
>> https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.html
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> scripts/qapi.py | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>
>> @@ -354,7 +354,7 @@ def parse_schema(input_file):
>> return exprs
>>
>> def parse_args(typeinfo):
>> - if isinstance(typeinfo, basestring):
>> + if isinstance(typeinfo, str):
>
> at least the code is right.
Yup. With the spelling fix:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 04/28] qapi: Add some enum tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 04/28] qapi: Add some enum tests Eric Blake
@ 2015-03-26 10:01 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 10:01 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal well with enums
> that aren't up to par. Later patches will update the expected
> results as the generator is made stricter.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
This is v4 plus the new enum-union-clash test.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 05/28] qapi: Better error messages for bad enums
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 05/28] qapi: Better error messages for bad enums Eric Blake
@ 2015-03-26 10:08 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 10:08 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> The previous commit demonstrated that the generator had several
> flaws with less-than-perfect enums:
> - an enum that listed the same string twice (or two variant
> strings that map to the same C enumerator) ended up generating
> an invalid C enum
> - because the generator adds a _MAX terminator to each enum,
> the use of an enum member 'max' can also cause this clash
> - if an enum omits 'data', the generator left a python stack
> trace rather than a graceful message
> - an enum that used a non-array 'data' was silently accepted by
> the parser
> - an enum that used non-string members in the 'data' member
> was silently accepted by the parser
>
> Add check_enum to cover these situations, and update testcases
> to match. While valid .json files won't trigger any of these
> cases, we might as well be nicer to developers that make a typo
> while trying to add new QAPI code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests Eric Blake
@ 2015-03-26 13:18 ` Markus Armbruster
2015-03-26 15:04 ` Eric Blake
2015-03-26 13:23 ` Markus Armbruster
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 13:18 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal well with unions
> that aren't up to par. Later patches will update the expected
> reseults as the generator is made stricter.
>
> Of particular note, we currently allow 'base' without 'discriminator'
> as a way to create a simple union with a base class. However, none
> of the existing QMP or QGA interfaces use it (it is exercised only
> in the testsuite).
qapi-code-gen.txt documents 'base' only for flat unions. We should
either document its use in simple unions, or outlaw it.
> Meanwhile, it would be nice to allow
> 'discriminator':'EnumType' without 'base' for creating a simple union
> that is type-safe rather than open-coded to a generated enum (right
> now, we are only type-safe when using a flat union, but that uses
> a different syntax of 'discriminator':'member-name' which requires
> a base class containing a 'member-name' enum field).
I'm afraid I don't get you. Can you give examples?
> If both 'base'
> and 'discriminator' are optional, then converting a simple union
> with base class to a type-safe simple union with an enum discriminator
> would not be possible. So my plan is to get rid of 'base' without
> 'discriminator' later in the series;
Aha: you're going to outlaw 'base' in simple unions. Yes, please.
> this will be no real loss, as any
> union that needs additional common fields can always use a flat
> union.
The mathematical concept behind unions is the sum type.
We have three implementations, and we call them simple, flat, anonymous.
Anonymous unions are implicitly tagged. They come with the obvious
restrictions required to make implicit tagging work.
The other two are explicitly tagged. The difference between them is
just notation. I like my unions flat, because for me the extra wrapping
is just notational overhead.
In particular, I can define simple unions in terms of flat ones by
restricting all union cases to a single member named 'data'. They're
not implemented that way, but that's a historical accident. Simple
unions are a redundant concept.
This proves your "can always use a flat union" proposition. The only
way they can earn their keep is making the schema easier to read. I
haven't thought about that.
Another historical accident is how we express members common to all
union cases: base type. Probably just because complex types already had
them. Are you planning to change anything there?
[...]
> diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/alternate-nested.exit b/tests/qapi-schema/alternate-nested.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-nested.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/alternate-nested.json b/tests/qapi-schema/alternate-nested.json
> new file mode 100644
> index 0000000..d5812bf
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-nested.json
> @@ -0,0 +1,7 @@
> +# FIXME: we should reject a nested anonymous union branch
Same reason we want to reject nested / anonymous complex types
elsewhere? Or is there a deeper reason?
> +{ 'union': 'Union1',
> + 'discriminator': {},
> + 'data': { 'name': 'str', 'value': 'int' } }
> +{ 'union': 'Union2',
> + 'discriminator': {},
> + 'data': { 'nested': 'Union1' } }
> diff --git a/tests/qapi-schema/alternate-nested.out b/tests/qapi-schema/alternate-nested.out
> new file mode 100644
> index 0000000..0137c1f
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-nested.out
> @@ -0,0 +1,5 @@
> +[OrderedDict([('union', 'Union1'), ('discriminator', OrderedDict()), ('data', OrderedDict([('name', 'str'), ('value', 'int')]))]),
> + OrderedDict([('union', 'Union2'), ('discriminator', OrderedDict()), ('data', OrderedDict([('nested', 'Union1')]))])]
> +[{'enum_name': 'Union1Kind', 'enum_values': None},
> + {'enum_name': 'Union2Kind', 'enum_values': None}]
> +[]
[...]
> diff --git a/tests/qapi-schema/flat-union-bad-base.err b/tests/qapi-schema/flat-union-bad-base.err
> new file mode 100644
> index 0000000..5962ff4
> --- /dev/null
> +++ b/tests/qapi-schema/flat-union-bad-base.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/flat-union-bad-base.json:9: Base 'OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')])' is not a valid type
> diff --git a/tests/qapi-schema/flat-union-bad-base.exit b/tests/qapi-schema/flat-union-bad-base.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/flat-union-bad-base.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/flat-union-bad-base.json b/tests/qapi-schema/flat-union-bad-base.json
> new file mode 100644
> index 0000000..d69168f
> --- /dev/null
> +++ b/tests/qapi-schema/flat-union-bad-base.json
> @@ -0,0 +1,13 @@
> +# we require the base to be an existing complex type
> +# FIXME: should we allow an anonymous inline base type?
What do you mean by "anonymous inline base type"?
> +{ 'enum': 'TestEnum',
> + 'data': [ 'value1', 'value2' ] }
> +{ 'type': 'TestTypeA',
> + 'data': { 'string': 'str' } }
> +{ 'type': 'TestTypeB',
> + 'data': { 'integer': 'int' } }
> +{ 'union': 'TestUnion',
> + 'base': { 'enum1': 'TestEnum', 'kind': 'str' },
> + 'discriminator': 'TestEnum',
> + 'data': { 'kind1': 'TestTypeA',
> + 'kind2': 'TestTypeB' } }
> diff --git a/tests/qapi-schema/flat-union-bad-base.out b/tests/qapi-schema/flat-union-bad-base.out
> new file mode 100644
> index 0000000..e69de29
[...]
"Doesn't deal well" is an understatement :)
Since all my questions are about your intentions and rationale:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests Eric Blake
2015-03-26 13:18 ` Markus Armbruster
@ 2015-03-26 13:23 ` Markus Armbruster
2015-03-26 13:51 ` Eric Blake
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 13:23 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
One more:
[...]
> diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
> new file mode 100644
> index 0000000..5fd1a47
> --- /dev/null
> +++ b/tests/qapi-schema/alternate-conflict-string.json
> @@ -0,0 +1,8 @@
> +# FIXME: we should reject anonymous unions with multiple string-like branches
> +{ 'enum': 'Enum',
> + 'data': [ 'hello', 'world' ] }
> +{ 'union': 'MyUnion',
> + 'discriminator': {},
> + 'data': { 'one': 'str',
> + 'two': 'Enum' } }
> +
/home/armbru/work/qemu/.git/rebase-apply/patch:325: new blank line at EOF.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 07/28] qapi: Simplify tests of simple unions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 07/28] qapi: Simplify tests of simple unions Eric Blake
@ 2015-03-26 13:41 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 13:41 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> None of the existing QMP or QGA interfaces uses a union with a
> base type but no discriminator; it is easier to avoid this in
> the generator to save room for other future extensions more likely
> to be useful (the previous commit added the test
> union-base-no-discriminator to ensure that we eventually give an
> error message). Meanwhile, the tests of UserDefNativeListUnion
> serve to validate code generation of simple unions, except that it
> did not have full coverage in the strict test.
>
> Fix some indentation and long lines while at it.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
I figure I'd split this commit into two or three parts
* kill the undocumented "base in simple union" feature
* add the missing tests for UserDefNativeListUnion
* style cleanup, possibly squashed into previous
Your choice.
[...]
> diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
> index d5360c6..53134a1 100644
> --- a/tests/test-qmp-input-strict.c
> +++ b/tests/test-qmp-input-strict.c
[...]
> @@ -232,18 +232,19 @@ static void test_validate_fail_list(TestInputVisitorData *data,
> qapi_free_UserDefOneList(head);
> }
>
> -static void test_validate_fail_union(TestInputVisitorData *data,
> - const void *unused)
> +static void test_validate_fail_union_native_list(TestInputVisitorData *data,
> + const void *unused)
> {
> - UserDefUnion *tmp = NULL;
> + UserDefNativeListUnion *tmp = NULL;
> Error *err = NULL;
> Visitor *v;
>
> - v = validate_test_init(data, "{ 'type': 'b', 'data' : { 'integer': 42 } }");
> + v = validate_test_init(data,
> + "{ 'type': 'integer', 'data' : [ \"string\" ] }");
Everywhere else we use ' instead of \". Not worth a respin on its own,
clean up on top then.
>
> - visit_type_UserDefUnion(v, &tmp, NULL, &err);
> + visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
> g_assert(err);
> - qapi_free_UserDefUnion(tmp);
> + qapi_free_UserDefNativeListUnion(tmp);
> }
>
> static void test_validate_fail_union_flat(TestInputVisitorData *data,
[...]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-26 13:23 ` Markus Armbruster
@ 2015-03-26 13:51 ` Eric Blake
2015-03-26 15:58 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-26 13:51 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1045 bytes --]
On 03/26/2015 07:23 AM, Markus Armbruster wrote:
> One more:
>
> [...]
>> diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
>> new file mode 100644
>> index 0000000..5fd1a47
>> --- /dev/null
>> +++ b/tests/qapi-schema/alternate-conflict-string.json
>> @@ -0,0 +1,8 @@
>> +# FIXME: we should reject anonymous unions with multiple string-like branches
>> +{ 'enum': 'Enum',
>> + 'data': [ 'hello', 'world' ] }
>> +{ 'union': 'MyUnion',
>> + 'discriminator': {},
>> + 'data': { 'one': 'str',
>> + 'two': 'Enum' } }
>> +
>
> /home/armbru/work/qemu/.git/rebase-apply/patch:325: new blank line at EOF.
Huh. I thought I had git set up to reject me from making commits like
that locally, but obviously not.
http://wiki.qemu.org/Contribute/SubmitAPatch should probably mention the
magic one-time setup to use to turn this type of checking on...
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions Eric Blake
2015-03-24 20:38 ` Eric Blake
@ 2015-03-26 14:20 ` Markus Armbruster
1 sibling, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 14:20 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Subject suggests you're just polishing error messages here. In fact,
you're fixing the generator to detect errors. Suggest something like
qapi: Tighten checking of unions
Eric Blake <eblake@redhat.com> writes:
> Previous commits demonstrated that the generator had several
> flaws with less-than-perfect unions:
> - make the use of a base without discriminator a hard error,
> since the previous patch removed all remaining uses of it
> - a simple union that listed the same branch twice (or two variant
> names that map to the same C enumerator, including the implicit
> MAX sentinel) ended up generating invalid C code
> - checking 'if discriminator' prior to 'if discriminator == {}'
> leads to dead code in python, and ended up processing anonymous
> unions as if they were simple unions
> - an anonymous union that listed two branches with the same qtype
> ended up generating invalid C code
> - the generator crashed on anonymous union attempts to use an
> array type
> - the generator was silently ignoring a base type for anonymous
> unions
> - the generator allowed unknown types or nested anonymous unions
> as a branch in an anonymous union
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
[...]
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index f6fb930..2390887 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -181,17 +181,8 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
> name=name)
>
> for key in members:
> - qapi_type = members[key]
> - if builtin_types.has_key(qapi_type):
> - qtype = builtin_types[qapi_type]
> - elif find_struct(qapi_type):
> - qtype = "QTYPE_QDICT"
> - elif find_union(qapi_type):
> - qtype = "QTYPE_QDICT"
> - elif find_enum(qapi_type):
> - qtype = "QTYPE_QSTRING"
> - else:
> - assert False, "Invalid anonymous union member"
> + qtype = find_anonymous_member_qtype(members[key])
> + assert qtype, "Invalid anonymous union member"
>
> ret += mcgen('''
> [ %(qtype)s ] = %(abbrev)s_KIND_%(enum)s,
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 3ce8c33..fc7b7f1 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -224,6 +224,23 @@ def find_base_fields(base):
> return None
> return base_struct_define['data']
>
> +# Return the qtype of an anonymous union branch, or None on error.
> +def find_anonymous_member_qtype(qapi_type):
> + if builtin_types.has_key(qapi_type):
> + return builtin_types[qapi_type]
> + elif find_struct(qapi_type):
> + return "QTYPE_QDICT"
> + elif find_enum(qapi_type):
> + return "QTYPE_QSTRING"
> + else:
> + union = find_union(qapi_type)
> + if union:
> + discriminator = union.get('discriminator')
> + if discriminator == {}:
> + return None
> + return "QTYPE_QDICT"
> + return None
> +
> # Return the discriminator enum define if discriminator is specified as an
> # enum type, otherwise return None.
> def discriminator_find_enum_define(expr):
> @@ -258,24 +275,28 @@ def check_union(expr, expr_info):
> base = expr.get('base')
> discriminator = expr.get('discriminator')
> members = expr['data']
> + values = { 'MAX': '(automatic)' }
> + types_seen = {}
>
> - # If the object has a member 'base', its value must name a complex type.
> - if base:
> - base_fields = find_base_fields(base)
> - if not base_fields:
> - raise QAPIExprError(expr_info,
> - "Base '%s' is not a valid type"
> - % base)
> + # Three types of unions, determined by discriminator.
>
> - # If the union object has no member 'discriminator', it's an
> - # ordinary union.
> - if not discriminator:
> + # If the value of member 'discriminator' is {}, it's an
> + # anonymous union, and must not have a base.
> + if discriminator == {}:
> enum_define = None
> + if base:
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' must not have a base"
> + % name)
>
> - # Else if the value of member 'discriminator' is {}, it's an
> - # anonymous union.
> - elif discriminator == {}:
> + # Else if the union object has no member 'discriminator', it's an
> + # ordinary union. For now, it must not have a base.
Since you're touching this anyway, you could s/ordinary/simple/.
> + elif not discriminator:
> enum_define = None
> + if base:
> + raise QAPIExprError(expr_info,
> + "Simple union '%s' must not have a base"
> + % name)
>
> # Else, it's a flat union.
> else:
> @@ -284,6 +305,12 @@ def check_union(expr, expr_info):
> raise QAPIExprError(expr_info,
> "Flat union '%s' must have a base field"
> % name)
> + base_fields = find_base_fields(base)
> + if not base_fields:
> + raise QAPIExprError(expr_info,
> + "Base '%s' is not a valid type"
> + % base)
> +
> # The value of member 'discriminator' must name a member of the
> # base type.
> discriminator_type = base_fields.get(discriminator)
> @@ -301,15 +328,42 @@ def check_union(expr, expr_info):
>
> # Check every branch
> for (key, value) in members.items():
> - # If this named member's value names an enum type, then all members
> + # If the discriminator names an enum type, then all members
> # of 'data' must also be members of the enum type.
> - if enum_define and not key in enum_define['enum_values']:
> - raise QAPIExprError(expr_info,
> - "Discriminator value '%s' is not found in "
> - "enum '%s'" %
> - (key, enum_define["enum_name"]))
> - # Todo: add checking for values. Key is checked as above, value can be
> - # also checked here, but we need more functions to handle array case.
> + if enum_define:
> + if not key in enum_define['enum_values']:
> + raise QAPIExprError(expr_info,
> + "Discriminator value '%s' is not found in "
> + "enum '%s'" %
> + (key, enum_define["enum_name"]))
> +
> + # Otherwise, check for conflicts in the generated enum
> + else:
> + c_key = _generate_enum_string(key)
> + if c_key in values:
> + raise QAPIExprError(expr_info,
> + "Union '%s' member '%s' clashes with '%s'"
> + % (name, key, values[c_key]))
> + values[c_key] = key
> +
> + # Ensure anonymous unions have no type conflicts.
> + if discriminator == {}:
> + if isinstance(value, list):
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' member '%s' must "
> + "not be array type" % (name, key))
> + qtype = find_anonymous_member_qtype(value)
> + if not qtype:
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' member '%s' has "
> + "invalid type '%s'" % (name, key, value))
> + if qtype in types_seen:
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' member '%s' has "
> + "same QObject type as member '%s'"
> + % (name, key, types_seen[qtype]))
> + types_seen[qtype] = key
> +
>
> def check_enum(expr, expr_info):
> name = expr['enum']
[...]
> diff --git a/tests/qapi-schema/alternate-conflict-string.err b/tests/qapi-schema/alternate-conflict-string.err
> index e69de29..a3b7b6d 100644
> --- a/tests/qapi-schema/alternate-conflict-string.err
> +++ b/tests/qapi-schema/alternate-conflict-string.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/alternate-conflict-string.json:4: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
Explains the problem in terms of the implementation. That's okay for a
development tool, but what about "Anonymous union 'MyUnion' member 'two'
can't be distinguished from member 'one'"?
> diff --git a/tests/qapi-schema/alternate-conflict-string.exit b/tests/qapi-schema/alternate-conflict-string.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/alternate-conflict-string.exit
> +++ b/tests/qapi-schema/alternate-conflict-string.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
> index 5fd1a47..a1b0bea 100644
> --- a/tests/qapi-schema/alternate-conflict-string.json
> +++ b/tests/qapi-schema/alternate-conflict-string.json
> @@ -1,4 +1,4 @@
> -# FIXME: we should reject anonymous unions with multiple string-like branches
> +# we reject anonymous unions with multiple string-like branches
> { 'enum': 'Enum',
> 'data': [ 'hello', 'world' ] }
> { 'union': 'MyUnion',
> diff --git a/tests/qapi-schema/alternate-conflict-string.out b/tests/qapi-schema/alternate-conflict-string.out
> index e7b39a2..e69de29 100644
> --- a/tests/qapi-schema/alternate-conflict-string.out
> +++ b/tests/qapi-schema/alternate-conflict-string.out
> @@ -1,5 +0,0 @@
> -[OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
> - OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('one', 'str'), ('two', 'Enum')]))])]
> -[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
> - {'enum_name': 'MyUnionKind', 'enum_values': None}]
> -[]
[...]
> diff --git a/tests/qapi-schema/union-base-no-discriminator.err b/tests/qapi-schema/union-base-no-discriminator.err
> index e69de29..f33392d 100644
> --- a/tests/qapi-schema/union-base-no-discriminator.err
> +++ b/tests/qapi-schema/union-base-no-discriminator.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/union-base-no-discriminator.json:12: Simple union 'TestUnion' must not have a base
> diff --git a/tests/qapi-schema/union-base-no-discriminator.exit b/tests/qapi-schema/union-base-no-discriminator.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/union-base-no-discriminator.exit
> +++ b/tests/qapi-schema/union-base-no-discriminator.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/union-base-no-discriminator.json b/tests/qapi-schema/union-base-no-discriminator.json
> index b5da546..548a633 100644
> --- a/tests/qapi-schema/union-base-no-discriminator.json
> +++ b/tests/qapi-schema/union-base-no-discriminator.json
> @@ -1,4 +1,5 @@
> -# FIXME: either allow base in non-flat unions, or diagnose missing discriminator
> +# for now, we reject a base for non-flat unions
> +# FIXME: might be a useful extension to allow later
This isn't a FIXME, it's an idea :)
> { 'type': 'TestTypeA',
> 'data': { 'string': 'str' } }
>
> diff --git a/tests/qapi-schema/union-base-no-discriminator.out b/tests/qapi-schema/union-base-no-discriminator.out
> index 505fd57..e69de29 100644
> --- a/tests/qapi-schema/union-base-no-discriminator.out
> +++ b/tests/qapi-schema/union-base-no-discriminator.out
> @@ -1,8 +0,0 @@
> -[OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
> - OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
> - OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))]),
> - OrderedDict([('union', 'TestUnion'), ('base', 'Base'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 'TestTypeB')]))])]
> -[{'enum_name': 'TestUnionKind', 'enum_values': None}]
> -[OrderedDict([('type', 'TestTypeA'), ('data', OrderedDict([('string', 'str')]))]),
> - OrderedDict([('type', 'TestTypeB'), ('data', OrderedDict([('integer', 'int')]))]),
> - OrderedDict([('type', 'Base'), ('data', OrderedDict([('string', 'str')]))])]
> diff --git a/tests/qapi-schema/union-max.err b/tests/qapi-schema/union-max.err
> index e69de29..55ce439 100644
> --- a/tests/qapi-schema/union-max.err
> +++ b/tests/qapi-schema/union-max.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/union-max.json:2: Union 'Union' member 'max' clashes with '(automatic)'
Cryptic error message, but it'll do.
> diff --git a/tests/qapi-schema/union-max.exit b/tests/qapi-schema/union-max.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/union-max.exit
> +++ b/tests/qapi-schema/union-max.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/union-max.json b/tests/qapi-schema/union-max.json
> index 45648c4..d6ad986 100644
> --- a/tests/qapi-schema/union-max.json
> +++ b/tests/qapi-schema/union-max.json
> @@ -1,3 +1,3 @@
> -# FIXME: we should reject 'max' branch in a union, for collision with C enum
> +# we reject 'max' branch in a union, for collision with C enum
> { 'union': 'Union',
> 'data': { 'max': 'int' } }
> diff --git a/tests/qapi-schema/union-max.out b/tests/qapi-schema/union-max.out
> index 2757d36..e69de29 100644
> --- a/tests/qapi-schema/union-max.out
> +++ b/tests/qapi-schema/union-max.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('union', 'Union'), ('data', OrderedDict([('max', 'int')]))])]
> -[{'enum_name': 'UnionKind', 'enum_values': None}]
> -[]
Feel free to address my comments on top.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 09/28] qapi: Prepare for catching more semantic parse errors
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 09/28] qapi: Prepare for catching more semantic parse errors Eric Blake
@ 2015-03-26 14:22 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 14:22 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> This patch widens the scope of a try block (with the attending
> reindentation required by Python) in preparation for a future
> patch adding more instances of QAPIExprError inside the block.
> It's easier to separate indentation from semantic changes, so
> this patch has no real behavior change.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator Eric Blake
@ 2015-03-26 14:47 ` Markus Armbruster
2015-03-26 15:26 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 14:47 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Special-casing 'discriminator == {}' for handling anonymous unions
> is getting awkward; since this particular type is not always a
> dictionary on the wire, it is easier to treat it as a completely
> different class of type, "alternate", so that if a type is listed
> in the union_types array, we know it is not an anonymous union.
>
> This patch just further segregates union handling, to make sure that
> anonymous unions are not stored in union_types, and splitting up
> check_union() into separate functions. A future patch will change
> the qapi grammar, and having the segregation already in place will
> make it easier to deal with the distinct meta-type.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi-types.py | 6 ++--
> scripts/qapi-visit.py | 4 +--
> scripts/qapi.py | 94 +++++++++++++++++++++++++++++----------------------
> 3 files changed, 58 insertions(+), 46 deletions(-)
>
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index 2390887..c9e0201 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -170,7 +170,7 @@ typedef enum %(name)s
>
> return lookup_decl + enum_decl
>
> -def generate_anon_union_qtypes(expr):
> +def generate_alternate_qtypes(expr):
>
> name = expr['union']
> members = expr['data']
> @@ -181,7 +181,7 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
> name=name)
>
> for key in members:
> - qtype = find_anonymous_member_qtype(members[key])
> + qtype = find_alternate_member_qtype(members[key])
> assert qtype, "Invalid anonymous union member"
>
> ret += mcgen('''
> @@ -408,7 +408,7 @@ for expr in exprs:
> fdef.write(generate_enum_lookup('%sKind' % expr['union'],
> expr['data'].keys()))
> if expr.get('discriminator') == {}:
> - fdef.write(generate_anon_union_qtypes(expr))
> + fdef.write(generate_alternate_qtypes(expr))
> else:
> continue
> fdecl.write(ret)
> diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
> index 3f82bd4..77b0a1f 100644
> --- a/scripts/qapi-visit.py
> +++ b/scripts/qapi-visit.py
> @@ -237,7 +237,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s *obj, const char *name, Error **er
> ''',
> name=name)
>
> -def generate_visit_anon_union(name, members):
> +def generate_visit_alternate(name, members):
> ret = mcgen('''
>
> void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **errp)
> @@ -302,7 +302,7 @@ def generate_visit_union(expr):
>
> if discriminator == {}:
> assert not base
> - return generate_visit_anon_union(name, members)
> + return generate_visit_alternate(name, members)
>
> enum_define = discriminator_find_enum_define(expr)
> if enum_define:
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 39cc88b..17252e9 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -224,21 +224,16 @@ def find_base_fields(base):
> return None
> return base_struct_define['data']
>
> -# Return the qtype of an anonymous union branch, or None on error.
> -def find_anonymous_member_qtype(qapi_type):
> +# Return the qtype of an alternate branch, or None on error.
> +def find_alternate_member_qtype(qapi_type):
> if builtin_types.has_key(qapi_type):
> return builtin_types[qapi_type]
> elif find_struct(qapi_type):
> return "QTYPE_QDICT"
> elif find_enum(qapi_type):
> return "QTYPE_QSTRING"
> - else:
> - union = find_union(qapi_type)
> - if union:
> - discriminator = union.get('discriminator')
> - if discriminator == {}:
> - return None
> - return "QTYPE_QDICT"
> + elif find_union(qapi_type):
> + return "QTYPE_QDICT"
> return None
>
> # Return the discriminator enum define if discriminator is specified as an
> @@ -276,22 +271,13 @@ def check_union(expr, expr_info):
> discriminator = expr.get('discriminator')
> members = expr['data']
> values = { 'MAX': '(automatic)' }
> - types_seen = {}
>
> - # Three types of unions, determined by discriminator.
> + # Two types of unions, determined by discriminator.
> + assert discriminator != {}
>
> - # If the value of member 'discriminator' is {}, it's an
> - # anonymous union, and must not have a base.
> - if discriminator == {}:
> - enum_define = None
> - if base:
> - raise QAPIExprError(expr_info,
> - "Anonymous union '%s' must not have a base"
> - % name)
> -
> - # Else if the union object has no member 'discriminator', it's an
> + # If the union object has no member 'discriminator', it's an
> # ordinary union. For now, it must not have a base.
> - elif not discriminator:
> + if not discriminator:
> enum_define = None
> if base:
> raise QAPIExprError(expr_info,
> @@ -346,24 +332,46 @@ def check_union(expr, expr_info):
> % (name, key, values[c_key]))
> values[c_key] = key
>
> - # Ensure anonymous unions have no type conflicts.
> - if discriminator == {}:
> - if isinstance(value, list):
> - raise QAPIExprError(expr_info,
> - "Anonymous union '%s' member '%s' must "
> - "not be array type" % (name, key))
> - qtype = find_anonymous_member_qtype(value)
> - if not qtype:
> - raise QAPIExprError(expr_info,
> - "Anonymous union '%s' member '%s' has "
> - "invalid type '%s'" % (name, key, value))
> - if qtype in types_seen:
> - raise QAPIExprError(expr_info,
> - "Anonymous union '%s' member '%s' has "
> - "same QObject type as member '%s'"
> - % (name, key, types_seen[qtype]))
> - types_seen[qtype] = key
> +def check_alternate(expr, expr_info):
> + name = expr['union']
> + base = expr.get('base')
> + discriminator = expr.get('discriminator')
> + members = expr['data']
> + values = { 'MAX': '(automatic)' }
> + types_seen = {}
>
> + assert discriminator == {}
> + if base:
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' must not have a base"
> + % name)
> +
> + # Check every branch
> + for (key, value) in members.items():
> + # Check for conflicts in the generated enum
> + c_key = _generate_enum_string(key)
> + if c_key in values:
> + raise QAPIExprError(expr_info,
> + "Union '%s' member '%s' clashes with '%s'"
> + % (name, key, values[c_key]))
> + values[c_key] = key
> +
> + # Ensure alternates have no type conflicts.
> + if isinstance(value, list):
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' member '%s' must "
> + "not be array type" % (name, key))
> + qtype = find_alternate_member_qtype(value)
> + if not qtype:
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' member '%s' has "
> + "invalid type '%s'" % (name, key, value))
> + if qtype in types_seen:
> + raise QAPIExprError(expr_info,
> + "Anonymous union '%s' member '%s' has "
> + "same QObject type as member '%s'"
> + % (name, key, types_seen[qtype]))
> + types_seen[qtype] = key
>
> def check_enum(expr, expr_info):
> name = expr['enum']
> @@ -393,7 +401,10 @@ def check_exprs(schema):
> if expr.has_key('enum'):
> check_enum(expr, info)
> elif expr.has_key('union'):
> - check_union(expr, info)
> + if expr.get('discriminator') == {}:
> + check_alternate(expr, info)
> + else:
> + check_union(expr, info)
> elif expr.has_key('event'):
> check_event(expr, info)
>
> @@ -535,7 +546,8 @@ def find_struct(name):
>
> def add_union(definition):
> global union_types
> - union_types.append(definition)
> + if definition.get('discriminator') != {}:
> + union_types.append(definition)
>
> def find_union(name):
> global union_types
This is the only unobvious hunk.
union_types is used only through find_union. The hunk makes
find_union(N) return None when N names an anonymous union.
find_union() is used in two places:
* find_alternate_member_qtype()
Patched further up. It really wants only non-anonymous unions, and
this change to find_union() renders its check for anonymous unions
superfluous. Good.
* generate_visit_alternate()
Asserts that each member's type is either a built-in type, a complex
type, a union type, or an enum type.
The change relaxes the assertion not to trigger on anonymous union
types. Why is that okay?
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 11/28] qapi: Rename anonymous union type in test
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 11/28] qapi: Rename anonymous union type in test Eric Blake
@ 2015-03-26 14:55 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 14:55 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Reduce churn in the future patch that replaces anonymous unions
> with a new metatype 'alternate' by changing 'AnonUnion' to
> 'Alternate'.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-26 13:18 ` Markus Armbruster
@ 2015-03-26 15:04 ` Eric Blake
2015-03-27 12:30 ` Markus Armbruster
2015-03-31 17:13 ` Kevin Wolf
0 siblings, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-26 15:04 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 9358 bytes --]
On 03/26/2015 07:18 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Demonstrate that the qapi generator doesn't deal well with unions
>> that aren't up to par. Later patches will update the expected
>> reseults as the generator is made stricter.
>>
>> Of particular note, we currently allow 'base' without 'discriminator'
>> as a way to create a simple union with a base class. However, none
>> of the existing QMP or QGA interfaces use it (it is exercised only
>> in the testsuite).
>
> qapi-code-gen.txt documents 'base' only for flat unions. We should
> either document its use in simple unions, or outlaw it.
I went with outlaw later in the series, and the rest of my commit
message was an attempt to explain my reasoning in that choice. But I
can certainly try to improve the wording, if we need a respin.
>
>> Meanwhile, it would be nice to allow
>> 'discriminator':'EnumType' without 'base' for creating a simple union
>> that is type-safe rather than open-coded to a generated enum (right
>> now, we are only type-safe when using a flat union, but that uses
>> a different syntax of 'discriminator':'member-name' which requires
>> a base class containing a 'member-name' enum field).
>
> I'm afraid I don't get you. Can you give examples?
Using this common code with the appropriate union for each example:
{ 'command':'foo', 'data':UNION }
Right now, we have flat unions which are required to be type-safe (all
branches MUST map back to the enum type of the discriminator, enforced
by the generator, so that if the enum later adds a member, the union
must also be updated to match):
[1]
{ 'union':'Safe', 'base':'Base', 'discriminator':'switch',
'data':{ 'one':'str', 'two':'int' }}
{"execute":"foo", "arguments":{'switch':'one', 'data':'hello'}}
and simple unions which cannot be typesafe (the branches of the union
are open-coded - even if they correlate to an existing enum, there is
nothing enforcing that extensions to the enum be reflected into the union):
[2]
{ 'union':'SimpleButOpenCoded',
'data':{ 'one': 'str', 'two':'int' }}
{"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
I'm hoping to add as a followup series a variant of simple unions that
is type-safe:
[3]
{ 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
'data':{ 'one':'str', 'two':'int' }}
{"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
But the existing, unused-except-in-testsuite, notion of a simple union
with a base class looks like:
[4]
{ 'type':'Shared', 'data':{'common':'int'}}
{ 'union':'SimpleWithBase', 'base':'Shared',
'data':{ 'one':'str', 'two':'int' }}
{"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
If we were to allow the addition of 'discriminator':'EnumType' to a
simple union [3], but then add that discriminator to an existing case of
a simple union with base [4], it would look like:
{ 'type':'Shared', 'data':{'common':'int'}}
{ 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
'discriminator':'MyEnum',
'data':{ 'one':'str', 'two':'int' }}
Yuck. That is indistinguishable from flat unions [1], except by whether
discriminator names an enum type or a member of the base class.
>
>> If both 'base'
>> and 'discriminator' are optional, then converting a simple union
>> with base class to a type-safe simple union with an enum discriminator
>> would not be possible. So my plan is to get rid of 'base' without
>> 'discriminator' later in the series;
>
> Aha: you're going to outlaw 'base' in simple unions. Yes, please.
Okay, you came to my desired conclusion; it's just that my wording in
the middle didn't help.
>
>> this will be no real loss, as any
>> union that needs additional common fields can always use a flat
>> union.
>
> The mathematical concept behind unions is the sum type.
>
> We have three implementations, and we call them simple, flat, anonymous.
>
> Anonymous unions are implicitly tagged. They come with the obvious
> restrictions required to make implicit tagging work.
and get renamed to 'alternate' later in the series, so they are not
worth worrying about here.
>
> The other two are explicitly tagged. The difference between them is
> just notation. I like my unions flat, because for me the extra wrapping
> is just notational overhead.
>
> In particular, I can define simple unions in terms of flat ones by
> restricting all union cases to a single member named 'data'. They're
> not implemented that way, but that's a historical accident. Simple
> unions are a redundant concept.
Cool. Or more concretely,
{ 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
is identical on the wire to:
{ 'enum': 'MyEnum', 'data': ['one', 'two'] }
{ 'type': 'Base', 'data': { 'type': 'MyEnum' } }
{ 'type': 'Branch1', 'data': { 'data': 'str' } }
{ 'type': 'Branch2', 'data': { 'data': 'int' } }
{ 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
'data': { 'one': 'Branch1', 'two': 'Branch2' } }
Probably worth mentioning in my commit message.
Hmm - that makes me wonder - do we support non-dict branches in a flat
union? The usual use case of a flat union is that all dictionary keys
in the branch's dictionary are treated as keys at the top level of the
resulting overall union; but a non-dictionary branch has no keys to
flatten into the top level. I may need to tweak a subsequent patch to
ensure that flat union branches can only use dictionaries (while simple
unions can use any type).
>
> This proves your "can always use a flat union" proposition. The only
> way they can earn their keep is making the schema easier to read. I
> haven't thought about that.
>
> Another historical accident is how we express members common to all
> union cases: base type. Probably just because complex types already had
> them. Are you planning to change anything there?
Maybe, per your own review. More at point [5] below...
>
> [...]
>> diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
>> new file mode 100644
>> index 0000000..e69de29
>> diff --git a/tests/qapi-schema/alternate-nested.exit b/tests/qapi-schema/alternate-nested.exit
>> new file mode 100644
>> index 0000000..573541a
>> --- /dev/null
>> +++ b/tests/qapi-schema/alternate-nested.exit
>> @@ -0,0 +1 @@
>> +0
>> diff --git a/tests/qapi-schema/alternate-nested.json b/tests/qapi-schema/alternate-nested.json
>> new file mode 100644
>> index 0000000..d5812bf
>> --- /dev/null
>> +++ b/tests/qapi-schema/alternate-nested.json
>> @@ -0,0 +1,7 @@
>> +# FIXME: we should reject a nested anonymous union branch
>
> Same reason we want to reject nested / anonymous complex types
> elsewhere? Or is there a deeper reason?
Similar to the reason as we want to reject {'command':'foo',
'data':'alternate-type'} for allowing non-dictionaries - we aren't
currently using it, and it's easier to reject than to worry about making
corner cases of the generator work on something we won't use.
Also, if I have an alternate A that chooses between string and dict,
then want to create an alternate B that chooses between int and
alternate A, will that even generate correct code? An alternate
represents multiple QObject types at once, so determining the QObject
type of the next token while parsing the code for union B would have to
worry about BOTH cases of nested union A.
So the FIXME is correct, and later in the series, I nuke this as
unsupported.
>> +++ b/tests/qapi-schema/flat-union-bad-base.json
>> @@ -0,0 +1,13 @@
>> +# we require the base to be an existing complex type
>> +# FIXME: should we allow an anonymous inline base type?
>
> What do you mean by "anonymous inline base type"?
[5] basically, that the following example could be legal shorthand...
>
>> +{ 'enum': 'TestEnum',
>> + 'data': [ 'value1', 'value2' ] }
>> +{ 'type': 'TestTypeA',
>> + 'data': { 'string': 'str' } }
>> +{ 'type': 'TestTypeB',
>> + 'data': { 'integer': 'int' } }
>> +{ 'union': 'TestUnion',
>> + 'base': { 'enum1': 'TestEnum', 'kind': 'str' },
>> + 'discriminator': 'TestEnum',
>> + 'data': { 'kind1': 'TestTypeA',
>> + 'kind2': 'TestTypeB' } }
where the { 'enum1':'TestEnum', 'kind':'str' } anonymous type defined at
'base' should be usable instead of having to name the type with a more
verbose:
{ 'type': 'Base', 'data': {'enum1':'TestEnum', 'kind':'str' }}
{ 'union': 'TestUnion', 'base':'Base', ... }
or in other words, that unions behave more like 'command' allowing
'data':{dict} as an anonymous type in place of 'data':'name'.
>> diff --git a/tests/qapi-schema/flat-union-bad-base.out b/tests/qapi-schema/flat-union-bad-base.out
>> new file mode 100644
>> index 0000000..e69de29
> [...]
>
> "Doesn't deal well" is an understatement :)
>
> Since all my questions are about your intentions and rationale:
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator
2015-03-26 14:47 ` Markus Armbruster
@ 2015-03-26 15:26 ` Eric Blake
2015-03-27 12:32 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-26 15:26 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 2421 bytes --]
On 03/26/2015 08:47 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Special-casing 'discriminator == {}' for handling anonymous unions
>> is getting awkward; since this particular type is not always a
>> dictionary on the wire, it is easier to treat it as a completely
>> different class of type, "alternate", so that if a type is listed
>> in the union_types array, we know it is not an anonymous union.
>>
>> This patch just further segregates union handling, to make sure that
>> anonymous unions are not stored in union_types, and splitting up
>> check_union() into separate functions. A future patch will change
>> the qapi grammar, and having the segregation already in place will
>> make it easier to deal with the distinct meta-type.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> @@ -535,7 +546,8 @@ def find_struct(name):
>>
>> def add_union(definition):
>> global union_types
>> - union_types.append(definition)
>> + if definition.get('discriminator') != {}:
>> + union_types.append(definition)
>>
>> def find_union(name):
>> global union_types
>
> This is the only unobvious hunk.
>
> union_types is used only through find_union. The hunk makes
> find_union(N) return None when N names an anonymous union.
>
> find_union() is used in two places:
>
> * find_alternate_member_qtype()
>
> Patched further up. It really wants only non-anonymous unions, and
> this change to find_union() renders its check for anonymous unions
> superfluous. Good.
>
> * generate_visit_alternate()
>
> Asserts that each member's type is either a built-in type, a complex
> type, a union type, or an enum type.
>
> The change relaxes the assertion not to trigger on anonymous union
> types. Why is that okay?
No, the change tightens the assertion so that it will now fail on an
anonymous union nested as a branch of another anonymous union (where
before it could silently pass the assertion), because the anonymous
union is no longer found by find_union(). And this is okay because the
earlier change to find_alternate_member_qtype means that we don't allow
nested anonymous unions, so making the assertion fail if an anonymous
union gets through anyway is correct.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union Eric Blake
2015-03-24 20:41 ` Eric Blake
@ 2015-03-26 15:42 ` Markus Armbruster
1 sibling, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 15:42 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Rather than special-casing "'union':'foo','alternate':{}" as an
> unusual union that can represent a non-dictionary, it is nicer
> to designate a separate meta-type "'alternate':'foo'" for the
> purpose. This involves a lot of documentation tweaks and fallout
> from .json files, but I already split as much as possible into
> earlier commits.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> docs/qapi-code-gen.txt | 63 +++++++++++-----------
> qapi/block-core.json | 6 +--
> scripts/qapi-types.py | 26 ++++++---
> scripts/qapi-visit.py | 17 +++---
> scripts/qapi.py | 41 +++++++-------
> tests/qapi-schema/alternate-array.err | 2 +-
> tests/qapi-schema/alternate-array.json | 3 +-
> tests/qapi-schema/alternate-base.err | 2 +-
> tests/qapi-schema/alternate-base.json | 5 +-
> tests/qapi-schema/alternate-clash.err | 2 +-
> tests/qapi-schema/alternate-clash.json | 5 +-
> tests/qapi-schema/alternate-conflict-dict.err | 2 +-
> tests/qapi-schema/alternate-conflict-dict.json | 5 +-
> tests/qapi-schema/alternate-conflict-string.err | 2 +-
> tests/qapi-schema/alternate-conflict-string.json | 3 +-
> tests/qapi-schema/alternate-good.json | 5 +-
> tests/qapi-schema/alternate-good.out | 4 +-
> tests/qapi-schema/alternate-nested.err | 2 +-
> tests/qapi-schema/alternate-nested.json | 8 ++-
> tests/qapi-schema/alternate-unknown.err | 2 +-
> tests/qapi-schema/alternate-unknown.json | 3 +-
> tests/qapi-schema/flat-union-bad-base.err | 2 +-
> tests/qapi-schema/flat-union-bad-discriminator.err | 2 +-
> .../qapi-schema/flat-union-bad-discriminator.json | 5 +-
> tests/qapi-schema/flat-union-base-union.err | 2 +-
> tests/qapi-schema/flat-union-base-union.json | 2 +-
> tests/qapi-schema/flat-union-no-base.err | 2 +-
> tests/qapi-schema/qapi-schema-test.json | 3 +-
> tests/qapi-schema/qapi-schema-test.out | 2 +-
> tests/qapi-schema/union-invalid-base.err | 2 +-
> 30 files changed, 117 insertions(+), 113 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index ce9c4b9..8792b94 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -79,16 +79,16 @@ the definition of complex structs that can have mutually recursive
> types, and allows for indefinite nesting of QMP that satisfies the
> schema. A type name should not be defined more than once.
>
> -There are six top-level expressions recognized by the parser:
> -'include', 'command', 'type', 'enum', 'union', and 'event'. There are
> -several built-in types, such as 'int' and 'str'; additionally, the
> -top-level expressions can define complex types, enumeration types, and
> -several flavors of union types. The 'command' expression can refer to
> -existing types by name, or list an anonymous type as a dictionary.
> -Listing a type name inside an array refers to a single-dimension array
> -of that type; multi-dimension arrays are not directly supported
> -(although an array of a complex struct that contains an array member
> -is possible).
> +There are seven top-level expressions recognized by the parser:
> +'include', 'command', 'type', 'enum', 'union', 'alternate', and
> +'event'. There are several built-in types, such as 'int' and 'str';
> +additionally, the top-level expressions can define complex types,
> +enumeration types, two flavors of union types, and an alternate type.
> +The 'command' expression can refer to existing types by name, or list
> +an anonymous type as a dictionary. Listing a type name inside an
Preexisting: we can't quite decide whether to use JSON terminology or
our own. 'dictionary' and 'list' are QObject, JSON calls them 'object'
and 'array'. Let's ignore this for now.
> +array refers to a single-dimension array of that type; multi-dimension
> +arrays are not directly supported (although an array of a complex
> +struct that contains an array member is possible).
>
> Types, commands, and events share a common namespace. Therefore,
> generally speaking, type definitions should always use CamelCase for
> @@ -159,8 +159,8 @@ Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
> A complex type is a dictionary containing a single 'data' key whose
> value is a dictionary. This corresponds to a struct in C or an Object
> in JSON. Each value of the 'data' dictionary must be the name of a
> -complex, enum, union, or built-in type, or a one-element array
> -containing a type name. An example of a complex type is:
> +complex, enum, union, alternate, or built-in type, or a one-element
> +array containing a type name. An example of a complex type is:
>
> { 'type': 'MyType',
> 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
> @@ -246,14 +246,12 @@ open-coding the field to be type 'str'.
> Usage: { 'union': 'str', 'data': 'dict' }
> or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
> 'discriminator': 'enum-member-of-base' }
> -or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
>
> Union types are used to let the user choose between several different
> -data types. There are three flavors: simple (no discriminator), flat
> -(a base type is mandatory, and discriminator is the name of an enum
> -field within that base type), and anonymous (discriminator is an
> -empty dictionary). A union type is defined using a data dictionary as
> -explained in the following paragraphs.
> +data types. There are two flavors: simple (no discriminator or base),
> +and flat (a base type is mandatory, and discriminator is the name of
> +an enum field within that base type). A union type is defined using a
> +data dictionary as explained in the following paragraphs.
Suggest to simplify to
and flat (both discriminator and base). A union type is defined using a
because we'll explain them in more detail further down already.
>
> A simple union type defines a mapping from automatic discriminator
> values to data types like in this example:
> @@ -321,22 +319,23 @@ Resulting in this JSON object:
> "lazy-refcounts": true }
>
>
> -The final flavor of unions is an anonymous union. While the other two
> -union types are always passed as a dictionary in the wire format, an
> -anonymous union instead allows the direct use of different types in
> -its place. As they aren't structured, they don't have any explicit
> -discriminator but use the (QObject) data type of their value as an
> -implicit discriminator. This means that they are restricted to using
> -only one discriminator value per QObject type. For example, you cannot
> -have two different complex types in an anonymous union, or two
> -different integer types.
> +=== Alternate types ===
>
> -Anonymous unions are declared using an empty dictionary as their discriminator.
> -The discriminator values never appear on the wire, they are only used in the
> -generated C code. Anonymous unions cannot have a base type.
> +Usage: { 'alternate: 'str', 'data': 'dict' }
>
> - { 'union': 'BlockRef',
> - 'discriminator': {},
> +An alternate type is one that allows a choice between two or more
> +QObject data types (string, integer, number, or dictionary, but not
> +array) on the wire. The definition is similar to a simple union type,
> +where each branch of the dictionary names a type, and where an
> +implicit C enum NameKind is created for the alternate Name. But
> +unlike a union, the discriminator string is never passed on the wire
> +for QMP, instead appearing only in the generated C code. The type on
> +the wire serves an implicit discriminator, which in turn means that an
> +alternate can express a choice between a string and a single complex
> +type (passed as a dictionary), but cannot distinguish between two
> +different complex types. For example:
> +
> + { 'alternate': 'BlockRef',
> 'data': { 'definition': 'BlockdevOptions',
> 'reference': 'str' } }
>
Same as for PATCH 01: should we explain this in terms of JSON types
instead of QObject types?
Perhaps:
An alternate type is one that allows a choice between values of several
distinct types (string, integer, number, or object, but currently not
array) on the wire. The definition is similar to a simple union type,
where each branch of the union names a type. For example:
{ 'alternate': 'BlockRef',
'data': { 'definition': 'BlockdevOptions',
'reference': 'str' } }
Just like for a simple union, an implicit C enum is generated to
enumerate the branches.
Unlike a union, the discriminator string is never passed on the wire for
QMP. The value's JSON type serves an implicit discriminator, which in
turn means that an alternate can only express a choice between types
represented differently in JSON, such as 'string' (JSON string) and
complex type (JSON object). Two different complex types, for instance,
aren't permitted, because both are represented as JSON objects.
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index f525b04..581c448 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1419,8 +1419,7 @@
> #
> # Since: 2.2
> ##
> -{ 'union': 'Qcow2OverlapChecks',
> - 'discriminator': {},
> +{ 'alternate': 'Qcow2OverlapChecks',
> 'data': { 'flags': 'Qcow2OverlapCheckFlags',
> 'mode': 'Qcow2OverlapCheckMode' } }
>
> @@ -1711,8 +1710,7 @@
> #
> # Since: 1.7
> ##
> -{ 'union': 'BlockdevRef',
> - 'discriminator': {},
> +{ 'alternate': 'BlockdevRef',
> 'data': { 'definition': 'BlockdevOptions',
> 'reference': 'str' } }
>
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index c9e0201..9c8d68c 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -172,7 +172,7 @@ typedef enum %(name)s
>
> def generate_alternate_qtypes(expr):
>
> - name = expr['union']
> + name = expr['alternate']
> members = expr['data']
>
> ret = mcgen('''
> @@ -182,7 +182,7 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
>
> for key in members:
> qtype = find_alternate_member_qtype(members[key])
> - assert qtype, "Invalid anonymous union member"
> + assert qtype, "Invalid alternate member"
>
> ret += mcgen('''
> [ %(qtype)s ] = %(abbrev)s_KIND_%(enum)s,
> @@ -197,9 +197,9 @@ const int %(name)s_qtypes[QTYPE_MAX] = {
> return ret
>
>
> -def generate_union(expr):
> +def generate_union(expr, meta):
>
> - name = expr['union']
> + name = expr[meta]
> typeinfo = expr['data']
>
> base = expr.get('base')
> @@ -243,7 +243,7 @@ struct %(name)s
> ret += mcgen('''
> };
> ''')
> - if discriminator == {}:
> + if meta == 'alternate':
> ret += mcgen('''
> extern const int %(name)s_qtypes[];
> ''',
> @@ -407,8 +407,12 @@ for expr in exprs:
> ret += generate_enum('%sKind' % expr['union'], expr['data'].keys())
> fdef.write(generate_enum_lookup('%sKind' % expr['union'],
> expr['data'].keys()))
> - if expr.get('discriminator') == {}:
> - fdef.write(generate_alternate_qtypes(expr))
> + elif expr.has_key('alternate'):
> + ret += generate_fwd_struct(expr['alternate'], expr['data']) + "\n"
> + ret += generate_enum('%sKind' % expr['alternate'], expr['data'].keys())
> + fdef.write(generate_enum_lookup('%sKind' % expr['alternate'],
> + expr['data'].keys()))
> + fdef.write(generate_alternate_qtypes(expr))
> else:
> continue
> fdecl.write(ret)
> @@ -438,11 +442,17 @@ for expr in exprs:
> ret += generate_type_cleanup_decl(expr['type'])
> fdef.write(generate_type_cleanup(expr['type']) + "\n")
> elif expr.has_key('union'):
> - ret += generate_union(expr)
> + ret += generate_union(expr, 'union')
> ret += generate_type_cleanup_decl(expr['union'] + "List")
> fdef.write(generate_type_cleanup(expr['union'] + "List") + "\n")
> ret += generate_type_cleanup_decl(expr['union'])
> fdef.write(generate_type_cleanup(expr['union']) + "\n")
> + elif expr.has_key('alternate'):
> + ret += generate_union(expr, 'alternate')
> + ret += generate_type_cleanup_decl(expr['alternate'] + "List")
> + fdef.write(generate_type_cleanup(expr['alternate'] + "List") + "\n")
> + ret += generate_type_cleanup_decl(expr['alternate'])
> + fdef.write(generate_type_cleanup(expr['alternate']) + "\n")
> elif expr.has_key('enum'):
> ret += generate_type_cleanup_decl(expr['enum'] + "List")
> fdef.write(generate_type_cleanup(expr['enum'] + "List") + "\n")
> diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
> index 77b0a1f..3e11089 100644
> --- a/scripts/qapi-visit.py
> +++ b/scripts/qapi-visit.py
> @@ -256,7 +256,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
> ''',
> name=name)
>
> - # For anon union, always use the default enum type automatically generated
> + # For alternate, always use the default enum type automatically generated
> # as "'%sKind' % (name)"
> disc_type = '%sKind' % (name)
>
> @@ -264,7 +264,7 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error **e
> assert (members[key] in builtin_types.keys()
> or find_struct(members[key])
> or find_union(members[key])
> - or find_enum(members[key])), "Invalid anonymous union member"
> + or find_enum(members[key])), "Invalid alternate member"
>
> enum_full_value = generate_enum_full_value(disc_type, key)
> ret += mcgen('''
> @@ -300,10 +300,6 @@ def generate_visit_union(expr):
> base = expr.get('base')
> discriminator = expr.get('discriminator')
>
> - if discriminator == {}:
> - assert not base
> - return generate_visit_alternate(name, members)
> -
> enum_define = discriminator_find_enum_define(expr)
> if enum_define:
> # Use the enum type as discriminator
> @@ -572,6 +568,15 @@ for expr in exprs:
> expr['data'].keys())
> ret += generate_declaration(expr['union'], expr['data'])
> fdecl.write(ret)
> + elif expr.has_key('alternate'):
> + ret = generate_visit_alternate(expr['alternate'], expr['data'])
> + ret += generate_visit_list(expr['alternate'], expr['data'])
> + fdef.write(ret)
> +
> + ret = generate_decl_enum('%sKind' % expr['alternate'],
> + expr['data'].keys())
> + ret += generate_declaration(expr['alternate'], expr['data'])
> + fdecl.write(ret)
> elif expr.has_key('enum'):
> ret = generate_visit_list(expr['enum'], expr['data'])
> ret += generate_visit_enum(expr['enum'], expr['data'])
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 17252e9..018ec45 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -273,11 +273,10 @@ def check_union(expr, expr_info):
> values = { 'MAX': '(automatic)' }
>
> # Two types of unions, determined by discriminator.
> - assert discriminator != {}
>
> # If the union object has no member 'discriminator', it's an
> # ordinary union. For now, it must not have a base.
> - if not discriminator:
> + if discriminator is None:
> enum_define = None
> if base:
> raise QAPIExprError(expr_info,
> @@ -287,18 +286,22 @@ def check_union(expr, expr_info):
> # Else, it's a flat union.
> else:
> # The object must have a member 'base'.
> - if not base:
> + if not isinstance(base, str):
> raise QAPIExprError(expr_info,
> - "Flat union '%s' must have a base field"
> + "Flat union '%s' must have a string base field"
> % name)
Sure this bug fix belongs to this patch?
> base_fields = find_base_fields(base)
> if not base_fields:
> raise QAPIExprError(expr_info,
> - "Base '%s' is not a valid type"
> + "Base '%s' is not a valid base type"
> % base)
>
Likewise.
> # The value of member 'discriminator' must name a member of the
> # base type.
> + if not isinstance(discriminator, str):
> + raise QAPIExprError(expr_info,
> + "Flat union '%s' must have a string "
> + "discriminator field" % name)
> discriminator_type = base_fields.get(discriminator)
> if not discriminator_type:
> raise QAPIExprError(expr_info,
Likewise.
The bug fixes are clearly visible in the test results.
> @@ -333,17 +336,15 @@ def check_union(expr, expr_info):
> values[c_key] = key
>
> def check_alternate(expr, expr_info):
> - name = expr['union']
> - base = expr.get('base')
> - discriminator = expr.get('discriminator')
> + name = expr['alternate']
> members = expr['data']
> values = { 'MAX': '(automatic)' }
> types_seen = {}
>
> - assert discriminator == {}
> + base = expr.get('base')
> if base:
> raise QAPIExprError(expr_info,
> - "Anonymous union '%s' must not have a base"
> + "Alternate '%s' must not have a base"
> % name)
>
> # Check every branch
> @@ -352,23 +353,23 @@ def check_alternate(expr, expr_info):
> c_key = _generate_enum_string(key)
> if c_key in values:
> raise QAPIExprError(expr_info,
> - "Union '%s' member '%s' clashes with '%s'"
> + "Alternate '%s' member '%s' clashes with '%s'"
> % (name, key, values[c_key]))
> values[c_key] = key
>
> # Ensure alternates have no type conflicts.
> if isinstance(value, list):
> raise QAPIExprError(expr_info,
> - "Anonymous union '%s' member '%s' must "
> + "Alternate '%s' member '%s' must "
> "not be array type" % (name, key))
> qtype = find_alternate_member_qtype(value)
> if not qtype:
> raise QAPIExprError(expr_info,
> - "Anonymous union '%s' member '%s' has "
> + "Alternate '%s' member '%s' has "
> "invalid type '%s'" % (name, key, value))
> if qtype in types_seen:
> raise QAPIExprError(expr_info,
> - "Anonymous union '%s' member '%s' has "
> + "Alternate '%s' member '%s' has "
> "same QObject type as member '%s'"
> % (name, key, types_seen[qtype]))
> types_seen[qtype] = key
> @@ -401,10 +402,9 @@ def check_exprs(schema):
> if expr.has_key('enum'):
> check_enum(expr, info)
> elif expr.has_key('union'):
> - if expr.get('discriminator') == {}:
> - check_alternate(expr, info)
> - else:
> - check_union(expr, info)
> + check_union(expr, info)
> + elif expr.has_key('alternate'):
> + check_alternate(expr, info)
> elif expr.has_key('event'):
> check_event(expr, info)
>
> @@ -436,6 +436,8 @@ def parse_schema(input_file):
> if expr.has_key('union'):
> if not discriminator_find_enum_define(expr):
> add_enum('%sKind' % expr['union'])
> + elif expr.has_key('alternate'):
> + add_enum('%sKind' % expr['alternate'])
>
> # Final pass - validate that exprs make sense
> check_exprs(schema)
> @@ -546,8 +548,7 @@ def find_struct(name):
>
> def add_union(definition):
> global union_types
> - if definition.get('discriminator') != {}:
> - union_types.append(definition)
> + union_types.append(definition)
>
> def find_union(name):
> global union_types
> diff --git a/tests/qapi-schema/alternate-array.err b/tests/qapi-schema/alternate-array.err
> index 2638b85..5721ed2 100644
> --- a/tests/qapi-schema/alternate-array.err
> +++ b/tests/qapi-schema/alternate-array.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-array.json:4: Anonymous union 'MyUnion' member 'two' must not be array type
> +tests/qapi-schema/alternate-array.json:4: Alternate 'Alt' member 'two' must not be array type
> diff --git a/tests/qapi-schema/alternate-array.json b/tests/qapi-schema/alternate-array.json
> index e330d57..86d8fa0 100644
> --- a/tests/qapi-schema/alternate-array.json
> +++ b/tests/qapi-schema/alternate-array.json
> @@ -1,7 +1,6 @@
> # FIXME: we do not support array branches of anonymous unions yet
> { 'type': 'One',
> 'data': { 'name': 'str' } }
> -{ 'union': 'MyUnion',
> - 'discriminator': {},
> +{ 'alternate': 'Alt',
> 'data': { 'one': 'One',
> 'two': [ 'int' ] } }
> diff --git a/tests/qapi-schema/alternate-base.err b/tests/qapi-schema/alternate-base.err
> index a2486b8..4a2566e 100644
> --- a/tests/qapi-schema/alternate-base.err
> +++ b/tests/qapi-schema/alternate-base.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-base.json:4: Anonymous union 'MyUnion' must not have a base
> +tests/qapi-schema/alternate-base.json:4: Alternate 'Alt' must not have a base
> diff --git a/tests/qapi-schema/alternate-base.json b/tests/qapi-schema/alternate-base.json
> index dad7f02..66edc89 100644
> --- a/tests/qapi-schema/alternate-base.json
> +++ b/tests/qapi-schema/alternate-base.json
> @@ -1,7 +1,6 @@
> -# we reject anonymous union with base type
> +# we reject alternate with base type
> { 'type': 'Base',
> 'data': { 'string': 'str' } }
> -{ 'union': 'MyUnion',
> +{ 'alternate': 'Alt',
> 'base': 'Base',
> - 'discriminator': {},
> 'data': { 'number': 'int' } }
> diff --git a/tests/qapi-schema/alternate-clash.err b/tests/qapi-schema/alternate-clash.err
> index 1130c12..51bea3e 100644
> --- a/tests/qapi-schema/alternate-clash.err
> +++ b/tests/qapi-schema/alternate-clash.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-clash.json:2: Union 'Union1' member 'ONE' clashes with 'one'
> +tests/qapi-schema/alternate-clash.json:2: Alternate 'Alt1' member 'ONE' clashes with 'one'
> diff --git a/tests/qapi-schema/alternate-clash.json b/tests/qapi-schema/alternate-clash.json
> index fa2d27e..3947935 100644
> --- a/tests/qapi-schema/alternate-clash.json
> +++ b/tests/qapi-schema/alternate-clash.json
> @@ -1,4 +1,3 @@
> -# we detect C enum collisions in an anonymous union
> -{ 'union': 'Union1',
> - 'discriminator': {},
> +# we detect C enum collisions in an alternate
> +{ 'alternate': 'Alt1',
> 'data': { 'one': 'str', 'ONE': 'int' } }
> diff --git a/tests/qapi-schema/alternate-conflict-dict.err b/tests/qapi-schema/alternate-conflict-dict.err
> index f256d51..c535332 100644
> --- a/tests/qapi-schema/alternate-conflict-dict.err
> +++ b/tests/qapi-schema/alternate-conflict-dict.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-conflict-dict.json:6: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
> +tests/qapi-schema/alternate-conflict-dict.json:6: Alternate 'Alt' member 'two' has same QObject type as member 'one'
> diff --git a/tests/qapi-schema/alternate-conflict-dict.json b/tests/qapi-schema/alternate-conflict-dict.json
> index 177c163..eaac5b3 100644
> --- a/tests/qapi-schema/alternate-conflict-dict.json
> +++ b/tests/qapi-schema/alternate-conflict-dict.json
> @@ -1,9 +1,8 @@
> -# we reject anonymous unions with multiple object branches
> +# we reject alternates with multiple object branches
> { 'type': 'One',
> 'data': { 'name': 'str' } }
> { 'type': 'Two',
> 'data': { 'value': 'int' } }
> -{ 'union': 'MyUnion',
> - 'discriminator': {},
> +{ 'alternate': 'Alt',
> 'data': { 'one': 'One',
> 'two': 'Two' } }
> diff --git a/tests/qapi-schema/alternate-conflict-string.err b/tests/qapi-schema/alternate-conflict-string.err
> index a3b7b6d..18ef0d3 100644
> --- a/tests/qapi-schema/alternate-conflict-string.err
> +++ b/tests/qapi-schema/alternate-conflict-string.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-conflict-string.json:4: Anonymous union 'MyUnion' member 'two' has same QObject type as member 'one'
> +tests/qapi-schema/alternate-conflict-string.json:4: Alternate 'Alt' member 'two' has same QObject type as member 'one'
> diff --git a/tests/qapi-schema/alternate-conflict-string.json b/tests/qapi-schema/alternate-conflict-string.json
> index a1b0bea..ce00307 100644
> --- a/tests/qapi-schema/alternate-conflict-string.json
> +++ b/tests/qapi-schema/alternate-conflict-string.json
> @@ -1,8 +1,7 @@
> # we reject anonymous unions with multiple string-like branches
> { 'enum': 'Enum',
> 'data': [ 'hello', 'world' ] }
> -{ 'union': 'MyUnion',
> - 'discriminator': {},
> +{ 'alternate': 'Alt',
> 'data': { 'one': 'str',
> 'two': 'Enum' } }
>
> diff --git a/tests/qapi-schema/alternate-good.json b/tests/qapi-schema/alternate-good.json
> index 1068e2f..74f338c 100644
> --- a/tests/qapi-schema/alternate-good.json
> +++ b/tests/qapi-schema/alternate-good.json
> @@ -1,10 +1,9 @@
> -# Working example of anonymous union
> +# Working example of alternate
> { 'type': 'Data',
> 'data': { '*number': 'int', '*name': 'str' } }
> { 'enum': 'Enum',
> 'data': [ 'hello', 'world' ] }
> -{ 'union': 'MyUnion',
> - 'discriminator': {},
> +{ 'alternate': 'Alt',
> 'data': { 'value': 'int',
> 'string': 'Enum',
> 'struct': 'Data' } }
> diff --git a/tests/qapi-schema/alternate-good.out b/tests/qapi-schema/alternate-good.out
> index b5117d1..c3a6b77 100644
> --- a/tests/qapi-schema/alternate-good.out
> +++ b/tests/qapi-schema/alternate-good.out
> @@ -1,6 +1,6 @@
> [OrderedDict([('type', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))]),
> OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
> - OrderedDict([('union', 'MyUnion'), ('discriminator', OrderedDict()), ('data', OrderedDict([('value', 'int'), ('string', 'Enum'), ('struct', 'Data')]))])]
> + OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('value', 'int'), ('string', 'Enum'), ('struct', 'Data')]))])]
> [{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
> - {'enum_name': 'MyUnionKind', 'enum_values': None}]
> + {'enum_name': 'AltKind', 'enum_values': None}]
> [OrderedDict([('type', 'Data'), ('data', OrderedDict([('*number', 'int'), ('*name', 'str')]))])]
> diff --git a/tests/qapi-schema/alternate-nested.err b/tests/qapi-schema/alternate-nested.err
> index 59df96e..00b05c6 100644
> --- a/tests/qapi-schema/alternate-nested.err
> +++ b/tests/qapi-schema/alternate-nested.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-nested.json:5: Anonymous union 'Union2' member 'nested' has invalid type 'Union1'
> +tests/qapi-schema/alternate-nested.json:4: Alternate 'Alt2' member 'nested' has invalid type 'Alt1'
> diff --git a/tests/qapi-schema/alternate-nested.json b/tests/qapi-schema/alternate-nested.json
> index ed2b6b7..2a115b9 100644
> --- a/tests/qapi-schema/alternate-nested.json
> +++ b/tests/qapi-schema/alternate-nested.json
> @@ -1,7 +1,5 @@
> # we reject a nested anonymous union branch
> -{ 'union': 'Union1',
> - 'discriminator': {},
> +{ 'alternate': 'Alt1',
> 'data': { 'name': 'str', 'value': 'int' } }
> -{ 'union': 'Union2',
> - 'discriminator': {},
> - 'data': { 'nested': 'Union1' } }
> +{ 'alternate': 'Alt2',
> + 'data': { 'nested': 'Alt1' } }
> diff --git a/tests/qapi-schema/alternate-unknown.err b/tests/qapi-schema/alternate-unknown.err
> index bf8e9ae..7af1b4c 100644
> --- a/tests/qapi-schema/alternate-unknown.err
> +++ b/tests/qapi-schema/alternate-unknown.err
> @@ -1 +1 @@
> -tests/qapi-schema/alternate-unknown.json:2: Anonymous union 'Union' member 'unknown' has invalid type 'MissingType'
> +tests/qapi-schema/alternate-unknown.json:2: Alternate 'Alt' member 'unknown' has invalid type 'MissingType'
> diff --git a/tests/qapi-schema/alternate-unknown.json b/tests/qapi-schema/alternate-unknown.json
> index 0c305c2..d7b217e 100644
> --- a/tests/qapi-schema/alternate-unknown.json
> +++ b/tests/qapi-schema/alternate-unknown.json
> @@ -1,4 +1,3 @@
> # we reject an anonymous union with unknown type in branch
> -{ 'union': 'Union',
> - 'discriminator': {},
> +{ 'alternate': 'Alt',
> 'data': { 'unknown': 'MissingType' } }
> diff --git a/tests/qapi-schema/flat-union-bad-base.err b/tests/qapi-schema/flat-union-bad-base.err
> index 5962ff4..f9c31b2 100644
> --- a/tests/qapi-schema/flat-union-bad-base.err
> +++ b/tests/qapi-schema/flat-union-bad-base.err
> @@ -1 +1 @@
> -tests/qapi-schema/flat-union-bad-base.json:9: Base 'OrderedDict([('enum1', 'TestEnum'), ('kind', 'str')])' is not a valid type
> +tests/qapi-schema/flat-union-bad-base.json:9: Flat union 'TestUnion' must have a string base field
Bug fix visible here...
> diff --git a/tests/qapi-schema/flat-union-bad-discriminator.err b/tests/qapi-schema/flat-union-bad-discriminator.err
> index 628432f..4f0c798 100644
> --- a/tests/qapi-schema/flat-union-bad-discriminator.err
> +++ b/tests/qapi-schema/flat-union-bad-discriminator.err
> @@ -1 +1 @@
> -tests/qapi-schema/flat-union-bad-discriminator.json:10: Simple union 'TestUnion' must not have a base
> +tests/qapi-schema/flat-union-bad-discriminator.json:11: Flat union 'TestUnion' must have a string discriminator field
... and here ...
> diff --git a/tests/qapi-schema/flat-union-bad-discriminator.json b/tests/qapi-schema/flat-union-bad-discriminator.json
> index 1599a59..982f072 100644
> --- a/tests/qapi-schema/flat-union-bad-discriminator.json
> +++ b/tests/qapi-schema/flat-union-bad-discriminator.json
> @@ -1,4 +1,5 @@
> -# FIXME: we should require the discriminator to be a string naming a base-type member
> +# we require the discriminator to be a string naming a base-type member
> +# this tests the old syntax for anonymous unions before we added alternates
> { 'enum': 'TestEnum',
> 'data': [ 'value1', 'value2' ] }
> { 'type': 'TestBase',
> @@ -9,6 +10,6 @@
> 'data': { 'integer': 'int' } }
> { 'union': 'TestUnion',
> 'base': 'TestBase',
> - 'discriminator': [],
> + 'discriminator': {},
> 'data': { 'kind1': 'TestTypeA',
> 'kind2': 'TestTypeB' } }
> diff --git a/tests/qapi-schema/flat-union-base-union.err b/tests/qapi-schema/flat-union-base-union.err
> index 185bf51..318c047 100644
> --- a/tests/qapi-schema/flat-union-base-union.err
> +++ b/tests/qapi-schema/flat-union-base-union.err
> @@ -1 +1 @@
> -tests/qapi-schema/flat-union-base-union.json:11: Base 'UnionBase' is not a valid type
> +tests/qapi-schema/flat-union-base-union.json:11: Base 'UnionBase' is not a valid base type
> diff --git a/tests/qapi-schema/flat-union-base-union.json b/tests/qapi-schema/flat-union-base-union.json
> index bbaa2da..135332d 100644
> --- a/tests/qapi-schema/flat-union-base-union.json
> +++ b/tests/qapi-schema/flat-union-base-union.json
> @@ -1,4 +1,4 @@
> -# FIXME: the error message needs help: we require the base to be a struct
> +# we require the base to be a struct
> { 'enum': 'TestEnum',
> 'data': [ 'value1', 'value2' ] }
> { 'type': 'TestTypeA',
> diff --git a/tests/qapi-schema/flat-union-no-base.err b/tests/qapi-schema/flat-union-no-base.err
> index eaf3592..338bff1 100644
> --- a/tests/qapi-schema/flat-union-no-base.err
> +++ b/tests/qapi-schema/flat-union-no-base.err
> @@ -1 +1 @@
> -tests/qapi-schema/flat-union-no-base.json:8: Flat union 'TestUnion' must have a base field
> +tests/qapi-schema/flat-union-no-base.json:8: Flat union 'TestUnion' must have a string base field
... and here.
> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index e1d35e1..dec8a7c 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -53,8 +53,7 @@
> 'discriminator': 'enum1',
> 'data': { 'value1' : 'UserDefC', 'value2' : 'UserDefB', 'value3' : 'UserDefA' } }
>
> -{ 'union': 'UserDefAlternate',
> - 'discriminator': {},
> +{ 'alternate': 'UserDefAlternate',
> 'data': { 'uda': 'UserDefA', 's': 'str', 'i': 'int' } }
>
> # for testing native lists
> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> index b55ab8d..313ecf3 100644
> --- a/tests/qapi-schema/qapi-schema-test.out
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -10,7 +10,7 @@
> OrderedDict([('type', 'UserDefUnionBase'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
> OrderedDict([('union', 'UserDefFlatUnion'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefA'), ('value2', 'UserDefB'), ('value3', 'UserDefB')]))]),
> OrderedDict([('union', 'UserDefFlatUnion2'), ('base', 'UserDefUnionBase'), ('discriminator', 'enum1'), ('data', OrderedDict([('value1', 'UserDefC'), ('value2', 'UserDefB'), ('value3', 'UserDefA')]))]),
> - OrderedDict([('union', 'UserDefAlternate'), ('discriminator', OrderedDict()), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
> + OrderedDict([('alternate', 'UserDefAlternate'), ('data', OrderedDict([('uda', 'UserDefA'), ('s', 'str'), ('i', 'int')]))]),
> OrderedDict([('union', 'UserDefNativeListUnion'), ('data', OrderedDict([('integer', ['int']), ('s8', ['int8']), ('s16', ['int16']), ('s32', ['int32']), ('s64', ['int64']), ('u8', ['uint8']), ('u16', ['uint16']), ('u32', ['uint32']), ('u64', ['uint64']), ('number', ['number']), ('boolean', ['bool']), ('string', ['str']), ('sizes', ['size'])]))]),
> OrderedDict([('command', 'user_def_cmd'), ('data', OrderedDict())]),
> OrderedDict([('command', 'user_def_cmd1'), ('data', OrderedDict([('ud1a', 'UserDefOne')]))]),
> diff --git a/tests/qapi-schema/union-invalid-base.err b/tests/qapi-schema/union-invalid-base.err
> index 3cc82c0..bedec06 100644
> --- a/tests/qapi-schema/union-invalid-base.err
> +++ b/tests/qapi-schema/union-invalid-base.err
> @@ -1 +1 @@
> -tests/qapi-schema/union-invalid-base.json:8: Base 'int' is not a valid type
> +tests/qapi-schema/union-invalid-base.json:8: Base 'int' is not a valid base type
A few 'anonymous' linger in tests/qapi-schema/:
tests/qapi-schema/alternate-array.json:# FIXME: we do not support array branches of anonymous unions yet
tests/qapi-schema/alternate-conflict-string.json:# we reject anonymous unions with multiple string-like branches
tests/qapi-schema/alternate-nested.json:# we reject a nested anonymous union branch
tests/qapi-schema/alternate-unknown.json:# we reject an anonymous union with unknown type in branch
tests/qapi-schema/flat-union-bad-base.json:# FIXME: should we allow an anonymous inline base type?
tests/qapi-schema/flat-union-bad-discriminator.json:# this tests the old syntax for anonymous unions before we added alternates
Since I found nothing wrong with this patch, just a few opportunities
for improvement, and bug fixes that should perhaps be separate:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests Eric Blake
@ 2015-03-26 15:55 ` Markus Armbruster
2015-03-26 19:02 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 15:55 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal well with
> expressions that aren't up to par. Later patches will improve
> the expected results as the generator is made stricter. Only
> one of the added tests actually behaves sanely at rejecting
> obvious problems.
>
> Note that in some cases, we reject bad QAPI merely because our
> pseudo-JSON parser does not yet know how to parse numbers. This
> series does not address that, but when a later series adds support
> for numeric defaults of integer fields, the testsuite will ensure
> that we don't lose the error (and hopefully that the error
> message quality is improved).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
[...]
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/event-case.exit b/tests/qapi-schema/event-case.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/event-case.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/event-case.json b/tests/qapi-schema/event-case.json
> new file mode 100644
> index 0000000..52dfc3a
> --- /dev/null
> +++ b/tests/qapi-schema/event-case.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject an event name that is not all caps
> +{ 'event': 'oops' }
qapi-code-gen.txt documents the naming conventions:
Types, commands, and events share a common namespace. Therefore,
generally speaking, type definitions should always use CamelCase for
user-defined type names, while built-in types are lowercase. Type
definitions should not end in 'Kind', as this namespace is used for
creating implicit C enums for visiting union types. Command names,
and field names within a type, should be all lower case with words
separated by a hyphen. However, some existing older commands and
complex types use underscore; when extending such expressions,
consistency is preferred over blindly avoiding underscore. Event
names should be ALL_CAPS with words separated by underscore. The
special string '**' appears for some commands that manually perform
their own type checking rather than relying on the type-safe code
produced by the qapi code generators.
We should either enforce the conventions consistently, or not at all.
Enforcing them makes certain kinds of name clashes in generated C
impossible. If we don't enforce them, we should catch the clashes.
Since I haven't read to the end of your series, I have to ask: do you
intend to enforce them?
> diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
> new file mode 100644
> index 0000000..3764bc7
> --- /dev/null
> +++ b/tests/qapi-schema/event-case.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('event', 'oops')])]
> +[]
> +[]
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-26 13:51 ` Eric Blake
@ 2015-03-26 15:58 ` Markus Armbruster
2015-03-30 22:45 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 15:58 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/26/2015 07:23 AM, Markus Armbruster wrote:
>> One more:
>>
>> [...]
>>> diff --git a/tests/qapi-schema/alternate-conflict-string.json
>>> b/tests/qapi-schema/alternate-conflict-string.json
>>> new file mode 100644
>>> index 0000000..5fd1a47
>>> --- /dev/null
>>> +++ b/tests/qapi-schema/alternate-conflict-string.json
>>> @@ -0,0 +1,8 @@
>>> +# FIXME: we should reject anonymous unions with multiple
>>> string-like branches
>>> +{ 'enum': 'Enum',
>>> + 'data': [ 'hello', 'world' ] }
>>> +{ 'union': 'MyUnion',
>>> + 'discriminator': {},
>>> + 'data': { 'one': 'str',
>>> + 'two': 'Enum' } }
>>> +
>>
>> /home/armbru/work/qemu/.git/rebase-apply/patch:325: new blank line at EOF.
>
> Huh. I thought I had git set up to reject me from making commits like
> that locally, but obviously not.
There's another one in PATCH 13:
/home/armbru/work/qemu/.git/rebase-apply/patch:156: new blank line at EOF.
> http://wiki.qemu.org/Contribute/SubmitAPatch should probably mention the
> magic one-time setup to use to turn this type of checking on...
Feel free to add it :)
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 14/28] qapi: Better error messages for bad expressions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 14/28] qapi: Better error messages for bad expressions Eric Blake
@ 2015-03-26 16:27 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 16:27 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> The previous commit demonstrated that the generator overlooked some
> fairly basic broken expressions:
> - missing metataype
> - metatype key has a non-string value
> - unknown key in relation to the metatype
> - conflicting metatype (this patch treats the second metatype as an
> unknown key of the first key visited, which is not necessarily the
> first key the user typed)
Not ideal, but good enough for a development tool.
>
> Add check_keys to cover these situations, and update testcases to
> match. A couple other tests (enum-missing-data, indented-expr) had
> to change since the validation added here occurs so early.
>
> While valid .json files won't trigger any of these cases, we might
> as well be nicer to developers that make a typo while trying to add
> new QAPI code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 15/28] qapi: Add tests of redefined expressions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 15/28] qapi: Add tests of redefined expressions Eric Blake
@ 2015-03-26 17:05 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 17:05 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal very well with
> redefined expressions. At the parse level, they are silently
> accepted; and while the testsuite just stops at parsing, I've
> further tested that many of them cause generator crashes or
> invalid C code if they were appended to qapi-schema-test.json.
> A later patch will tighten things up and adjust the testsuite
> to match.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions Eric Blake
@ 2015-03-26 17:21 ` Markus Armbruster
2015-03-27 7:52 ` Markus Armbruster
1 sibling, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 17:21 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> The previous commit demonstrated that the generator overlooked
> duplicate expressions:
> - a complex type or command reusing a built-in type name
> - redeclaration of a type name, whether by the same or different
> metatype
> - redeclaration of a command or event
> - collision of a type with implicit 'Kind' enum for a union
> - collision with an implicit MAX enum constant, or with various
> case spellings of events
>
> Since the c_type() function in the generator treats all names
> as being in the same namespace, this patch adds a global array
> to track all known names and their source, to prevent collisions
> before it can cause further problems. While valid .json files
> won't trigger any of these cases, we might as well be nicer to
> developers that make a typo while trying to add new QAPI code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 68 +++++++++++++++++++++++++-------
> tests/qapi-schema/bad-type-dict.err | 2 +-
> tests/qapi-schema/command-int.err | 1 +
> tests/qapi-schema/command-int.exit | 2 +-
> tests/qapi-schema/command-int.json | 2 +-
> tests/qapi-schema/command-int.out | 3 --
> tests/qapi-schema/enum-union-clash.err | 1 +
> tests/qapi-schema/enum-union-clash.exit | 2 +-
> tests/qapi-schema/enum-union-clash.json | 2 +-
> tests/qapi-schema/enum-union-clash.out | 5 ---
> tests/qapi-schema/event-case.err | 1 +
> tests/qapi-schema/event-case.exit | 2 +-
> tests/qapi-schema/event-case.json | 2 +-
> tests/qapi-schema/event-case.out | 3 --
> tests/qapi-schema/event-max.err | 1 +
> tests/qapi-schema/event-max.exit | 2 +-
> tests/qapi-schema/event-max.json | 2 +-
> tests/qapi-schema/event-max.out | 3 --
> tests/qapi-schema/redefined-builtin.err | 1 +
> tests/qapi-schema/redefined-builtin.exit | 2 +-
> tests/qapi-schema/redefined-builtin.json | 2 +-
> tests/qapi-schema/redefined-builtin.out | 3 --
> tests/qapi-schema/redefined-command.err | 1 +
> tests/qapi-schema/redefined-command.exit | 2 +-
> tests/qapi-schema/redefined-command.json | 2 +-
> tests/qapi-schema/redefined-command.out | 4 --
> tests/qapi-schema/redefined-event.err | 1 +
> tests/qapi-schema/redefined-event.exit | 2 +-
> tests/qapi-schema/redefined-event.json | 2 +-
> tests/qapi-schema/redefined-event.out | 4 --
> tests/qapi-schema/redefined-type.err | 1 +
> tests/qapi-schema/redefined-type.exit | 2 +-
> tests/qapi-schema/redefined-type.json | 2 +-
> tests/qapi-schema/redefined-type.out | 4 --
> 34 files changed, 78 insertions(+), 61 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 90eb3bc..5d0dc91 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -32,6 +32,12 @@ builtin_types = {
> 'size': 'QTYPE_QINT',
> }
>
> +enum_types = []
> +struct_types = []
> +union_types = []
> +events = []
> +all_names = {}
> +
> def error_path(parent):
> res = ""
> while parent:
> @@ -256,7 +262,17 @@ def discriminator_find_enum_define(expr):
> return find_enum(discriminator_type)
>
> def check_event(expr, expr_info):
> + global events
> + name = expr['event']
> params = expr.get('data')
> +
> + if name == 'MAX':
> + raise QAPIExprError(expr_info, "Event name 'MAX' cannot be created")
> + if name != name.upper():
> + raise QAPIExprError(expr_info, "Event name '%s' should be upper case"
> + % name)
> + events.append(name)
> +
> if params:
> for argname, argentry, optional, structured in parse_args(params):
> if structured:
> @@ -408,7 +424,7 @@ def check_keys(expr_elem, meta, required, optional=[]):
> name = expr[meta]
> if not isinstance(name, str):
> raise QAPIExprError(info,
> - "%s key must have a string value" % meta)
> + "'%s' key must have a string value" % meta)
> required = required + [ meta ]
> for (key, value) in expr.items():
> if not key in required and not key in optional:
Let's squash this hunk into PATCH 14.
> @@ -423,6 +439,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
>
>
> def parse_schema(input_file):
> + global all_names
> + exprs = []
> +
> # First pass: read entire file into memory
> try:
> schema = QAPISchema(open(input_file, "r"))
> @@ -430,30 +449,34 @@ def parse_schema(input_file):
> print >>sys.stderr, e
> exit(1)
>
> - exprs = []
> -
> try:
> # Next pass: learn the types and check for valid expression keys. At
> # this point, top-level 'include' has already been flattened.
> + for builtin in builtin_types.keys():
> + all_names[builtin] = 'built-in'
> for expr_elem in schema.exprs:
> expr = expr_elem['expr']
> + info = expr_elem['info']
> if expr.has_key('enum'):
> check_keys(expr_elem, 'enum', ['data'])
> - add_enum(expr['enum'], expr['data'])
> + add_enum(expr['enum'], info, expr['data'])
> elif expr.has_key('union'):
> check_keys(expr_elem, 'union', ['data'],
> ['base', 'discriminator'])
> - add_union(expr)
> + add_union(expr, info)
> elif expr.has_key('alternate'):
> check_keys(expr_elem, 'alternate', ['data'])
> + add_name(expr['alternate'], info, 'alternate')
> elif expr.has_key('type'):
> check_keys(expr_elem, 'type', ['data'], ['base'])
> - add_struct(expr)
> + add_struct(expr, info)
> elif expr.has_key('command'):
> check_keys(expr_elem, 'command', [],
> ['data', 'returns', 'gen', 'success-response'])
> + add_name(expr['command'], info, 'command')
> elif expr.has_key('event'):
> check_keys(expr_elem, 'event', [], ['data'])
> + add_name(expr['event'], info, 'event')
> else:
> raise QAPIExprError(expr_elem['info'],
> "Expression is missing metatype")
> @@ -464,9 +487,11 @@ def parse_schema(input_file):
> expr = expr_elem['expr']
> if expr.has_key('union'):
> if not discriminator_find_enum_define(expr):
> - add_enum('%sKind' % expr['union'])
> + add_enum('%sKind' % expr['union'], expr_elem['info'],
> + implicit=True)
> elif expr.has_key('alternate'):
> - add_enum('%sKind' % expr['alternate'])
> + add_enum('%sKind' % expr['alternate'], expr_elem['info'],
> + implicit=True)
>
> # Final pass - validate that exprs make sense
> check_exprs(schema)
> @@ -560,12 +585,22 @@ def type_name(name):
> return c_list_type(name[0])
> return name
>
> -enum_types = []
> -struct_types = []
> -union_types = []
> +def add_name(name, info, meta, implicit = False):
> + global all_names
> + if name in all_names:
> + raise QAPIExprError(info,
> + "%s '%s' is already defined"
> + %(all_names[name], name))
> + if not implicit and name[-4:] == 'Kind':
> + raise QAPIExprError(info,
> + "%s '%s' should not end in 'Kind'"
> + %(meta, name))
> + all_names[name] = meta
>
> -def add_struct(definition):
> +def add_struct(definition, info):
> global struct_types
> + name = definition['type']
> + add_name(name, info, 'struct')
> struct_types.append(definition)
>
> def find_struct(name):
> @@ -575,8 +610,10 @@ def find_struct(name):
> return struct
> return None
>
> -def add_union(definition):
> +def add_union(definition, info):
> global union_types
> + name = definition['union']
> + add_name(name, info, 'union')
> union_types.append(definition)
>
> def find_union(name):
> @@ -586,8 +623,9 @@ def find_union(name):
> return union
> return None
>
> -def add_enum(name, enum_values = None):
> +def add_enum(name, info, enum_values = None, implicit = False):
> global enum_types
> + add_name(name, info, 'enum', implicit)
> enum_types.append({"enum_name": name, "enum_values": enum_values})
>
> def find_enum(name):
> @@ -629,7 +667,7 @@ def c_type(name, is_param=False):
> return name
> elif name == None or len(name) == 0:
> return 'void'
> - elif name == name.upper():
> + elif name in events:
> return '%sEvent *%s' % (camel_case(name), eatspace)
> else:
> return '%s *%s' % (name, eatspace)
One instance kitten-killing grossness down :)
[Tests look good...]
Since the misplaced hunk is just polish:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json Eric Blake
@ 2015-03-26 17:32 ` Markus Armbruster
2015-03-31 15:23 ` Kevin Wolf
1 sibling, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 17:32 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> From: Fam Zheng <famz@redhat.com>
>
> In the near term, we will use it for a sensible-looking
> 'gen':false inside command declarations, instead of the
> current ugly 'gen':'no'.
>
> In the long term, it will allow conversion from shorthand
> with defaults mentioned only in side-band documentation:
> 'data':{'*flag':'bool', '*string':'str'}
> into an explicit default value documentation, as in:
> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
> 'string':{'type':'str', 'optional':true, 'default':null}}
>
> We still don't parse integer values (also necessary before
> we can allow explicit defaults), but that can come in a later
> series.
>
> Update the testsuite to match an improved error message.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 21 ++++++++++++++++++---
> tests/qapi-schema/bad-type-bool.err | 2 +-
> tests/qapi-schema/bad-type-bool.json | 1 -
> 3 files changed, 19 insertions(+), 5 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 5d0dc91..6ed6a34 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -158,6 +158,20 @@ class QAPISchema:
> return
> else:
> string += ch
> + elif self.tok in "tfn":
> + val = self.src[self.cursor - 1:]
> + if val.startswith("true"):
> + self.val = True
> + self.cursor += 3
> + return
> + elif val.startswith("false"):
> + self.val = False
> + self.cursor += 4
> + return
> + elif val.startswith("null"):
> + self.val = None
> + self.cursor += 3
> + return
> elif self.tok == '\n':
> if self.cursor == len(self.src):
> self.tok = None
> @@ -197,8 +211,9 @@ class QAPISchema:
> if self.tok == ']':
> self.accept()
> return expr
> - if not self.tok in [ '{', '[', "'" ]:
> - raise QAPISchemaError(self, 'Expected "{", "[", "]" or string')
> + if not self.tok in "{['tfn":
> + raise QAPISchemaError(self, 'Expected "{", "[", "]", string, '
> + 'boolean or "null"')
> while True:
> expr.append(self.get_expr(True))
> if self.tok == ']':
> @@ -217,7 +232,7 @@ class QAPISchema:
> elif self.tok == '[':
> self.accept()
> expr = self.get_values()
> - elif self.tok == "'":
> + elif self.tok in "'tfn":
> expr = self.val
> self.accept()
> else:
Exploiting that the three literal names start with different letters is
a a hack, but it'll do.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests Eric Blake
@ 2015-03-26 17:38 ` Markus Armbruster
2015-03-26 19:05 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 17:38 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> For a few QMP commands, we are forced to pass an arbitrary type
> without tracking it properly in QAPI. Among the existing clients,
> this unnamed type was spelled 'dict', 'visitor', and '**'; this
> patch standardizes on '**'.
>
> Meanwhile, for both 'gen' and 'success-response' keys, we have been
> ignoring the value, although the schema consistently used "'no'".
> But now that we can support a literal "false" in the schema, we
> might as well use that rather than ignoring the value or
> special-casing a random string.
>
> There is no difference to the generated code. As these features
> were previously undocumented before this series, add some tests
> and documentation on what we'd like to guarantee, although it will
> take later patches to clean up test results and actually enforce
> the use of a bool parameter.
You don't actually add documentation in this patch.
Aside: 'gen': false is required when '**' is used anywhere in the
command. If it was permitted only then, it would be redundant. I think
we happily accept 'gen': false without '**' so far, although we don't
use it. That's okay.
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests Eric Blake
@ 2015-03-26 17:58 ` Markus Armbruster
2015-03-26 19:07 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-26 17:58 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator silently parses confusing
> types, which may cause other errors later on. Later patches
> will update the expected results as the generator is made stricter.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
[...]
> diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-empty.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
> new file mode 100644
> index 0000000..edb543a
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-empty.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject an array for data if it does not contain a known type
> +{ 'command': 'oops', 'data': { 'empty': [ ] } }
v4 tested
{ 'command': 'oops', 'data': [ ] }
Is that still covered somewhere else?
> diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
> new file mode 100644
> index 0000000..6f25c9e
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-empty.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', OrderedDict([('empty', [])]))])]
> +[]
> +[]
[...]
> diff --git a/tests/qapi-schema/returns-whitelist.err b/tests/qapi-schema/returns-whitelist.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/returns-whitelist.exit b/tests/qapi-schema/returns-whitelist.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/returns-whitelist.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/returns-whitelist.json b/tests/qapi-schema/returns-whitelist.json
> new file mode 100644
> index 0000000..8328563
> --- /dev/null
> +++ b/tests/qapi-schema/returns-whitelist.json
> @@ -0,0 +1,11 @@
> +# FIXME: we should enforce that 'returns' be a dict or array of dict unless whitelisted
> +{ 'command': 'human-monitor-command',
> + 'data': {'command-line': 'str', '*cpu-index': 'int'},
> + 'returns': 'str' }
> +{ 'enum': 'TpmModel', 'data': [ 'tpm-tis' ] }
> +{ 'command': 'query-tpm-models', 'returns': ['TpmModel'] }
> +{ 'command': 'guest-get-time',
> + 'returns': 'int' }
> +
> +{ 'command': 'no-way-this-will-get-whitelisted',
> + 'returns': [ 'int' ] }
I like the pattern "add tests to demonstrate issues, then code to
address the issues". But this test appears too much out of the blue for
my taste.
You could use the commit message to prepare the reader.
Or you could deviate from the pattern and add this test together with
the actual whitelist. That's what I'd try.
> diff --git a/tests/qapi-schema/returns-whitelist.out b/tests/qapi-schema/returns-whitelist.out
> new file mode 100644
> index 0000000..2adcd8b
> --- /dev/null
> +++ b/tests/qapi-schema/returns-whitelist.out
> @@ -0,0 +1,7 @@
> +[OrderedDict([('command', 'human-monitor-command'), ('data', OrderedDict([('command-line', 'str'), ('*cpu-index', 'int')])), ('returns', 'str')]),
> + OrderedDict([('enum', 'TpmModel'), ('data', ['tpm-tis'])]),
> + OrderedDict([('command', 'query-tpm-models'), ('returns', ['TpmModel'])]),
> + OrderedDict([('command', 'guest-get-time'), ('returns', 'int')]),
> + OrderedDict([('command', 'no-way-this-will-get-whitelisted'), ('returns', ['int'])])]
> +[{'enum_name': 'TpmModel', 'enum_values': ['tpm-tis']}]
> +[]
Since I found nothing that's actually wrong:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests
2015-03-26 15:55 ` Markus Armbruster
@ 2015-03-26 19:02 ` Eric Blake
2015-03-27 12:38 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-26 19:02 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 2006 bytes --]
On 03/26/2015 09:55 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Demonstrate that the qapi generator doesn't deal well with
>> expressions that aren't up to par. Later patches will improve
>> the expected results as the generator is made stricter. Only
>> one of the added tests actually behaves sanely at rejecting
>> obvious problems.
>>
>
> qapi-code-gen.txt documents the naming conventions:
>
> Types, commands, and events share a common namespace. Therefore,
> generally speaking, type definitions should always use CamelCase for
> user-defined type names, while built-in types are lowercase. Type
> definitions should not end in 'Kind', as this namespace is used for
> creating implicit C enums for visiting union types. Command names,
> and field names within a type, should be all lower case with words
> separated by a hyphen. However, some existing older commands and
> complex types use underscore; when extending such expressions,
> consistency is preferred over blindly avoiding underscore. Event
> names should be ALL_CAPS with words separated by underscore. The
> special string '**' appears for some commands that manually perform
> their own type checking rather than relying on the type-safe code
> produced by the qapi code generators.
>
> We should either enforce the conventions consistently, or not at all.
>
> Enforcing them makes certain kinds of name clashes in generated C
> impossible. If we don't enforce them, we should catch the clashes.
>
> Since I haven't read to the end of your series, I have to ask: do you
> intend to enforce them?
I added tests to enforce it for event names, but did not enforce things
for command names or complex type members. I guess that can be added on
top, if desired.
So, did this patch get R-b?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests
2015-03-26 17:38 ` Markus Armbruster
@ 2015-03-26 19:05 ` Eric Blake
2015-03-27 12:40 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-26 19:05 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1859 bytes --]
On 03/26/2015 11:38 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> For a few QMP commands, we are forced to pass an arbitrary type
>> without tracking it properly in QAPI. Among the existing clients,
>> this unnamed type was spelled 'dict', 'visitor', and '**'; this
>> patch standardizes on '**'.
>>
>> Meanwhile, for both 'gen' and 'success-response' keys, we have been
>> ignoring the value, although the schema consistently used "'no'".
>> But now that we can support a literal "false" in the schema, we
>> might as well use that rather than ignoring the value or
>> special-casing a random string.
>>
>> There is no difference to the generated code. As these features
>> were previously undocumented before this series, add some tests
>> and documentation on what we'd like to guarantee, although it will
>> take later patches to clean up test results and actually enforce
>> the use of a bool parameter.
>
> You don't actually add documentation in this patch.
Hmm, more evidence that I waffled about per-commit doc fixes, vs.
lumping it all in patch 1, and I obviously failed to scrub the commit
messages after changing my mind.
>
> Aside: 'gen': false is required when '**' is used anywhere in the
> command. If it was permitted only then, it would be redundant. I think
> we happily accept 'gen': false without '**' so far, although we don't
> use it. That's okay.
Also, even though the code accepts 'gen':false, it rejects 'gen':true
('gen' is only a one-way switch away from the default). Also something
I didn't think worth worrying about.
>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests
2015-03-26 17:58 ` Markus Armbruster
@ 2015-03-26 19:07 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-26 19:07 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 2913 bytes --]
On 03/26/2015 11:58 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Demonstrate that the qapi generator silently parses confusing
>> types, which may cause other errors later on. Later patches
>> will update the expected results as the generator is made stricter.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
> [...]
>> diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
>> new file mode 100644
>> index 0000000..e69de29
>> diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
>> new file mode 100644
>> index 0000000..573541a
>> --- /dev/null
>> +++ b/tests/qapi-schema/data-array-empty.exit
>> @@ -0,0 +1 @@
>> +0
>> diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
>> new file mode 100644
>> index 0000000..edb543a
>> --- /dev/null
>> +++ b/tests/qapi-schema/data-array-empty.json
>> @@ -0,0 +1,2 @@
>> +# FIXME: we should reject an array for data if it does not contain a known type
>> +{ 'command': 'oops', 'data': { 'empty': [ ] } }
>
> v4 tested
>
> { 'command': 'oops', 'data': [ ] }
>
> Is that still covered somewhere else?
Probably not. Sounds like a good test case to add, presumably on top if
I don't have to respin.
>> +
>> +{ 'command': 'no-way-this-will-get-whitelisted',
>> + 'returns': [ 'int' ] }
>
> I like the pattern "add tests to demonstrate issues, then code to
> address the issues". But this test appears too much out of the blue for
> my taste.
>
> You could use the commit message to prepare the reader.
>
> Or you could deviate from the pattern and add this test together with
> the actual whitelist. That's what I'd try.
Of course, if I do need to respin, I can shuffle and split patches as
needed to make the resubmission prettier.
>
>> diff --git a/tests/qapi-schema/returns-whitelist.out b/tests/qapi-schema/returns-whitelist.out
>> new file mode 100644
>> index 0000000..2adcd8b
>> --- /dev/null
>> +++ b/tests/qapi-schema/returns-whitelist.out
>> @@ -0,0 +1,7 @@
>> +[OrderedDict([('command', 'human-monitor-command'), ('data', OrderedDict([('command-line', 'str'), ('*cpu-index', 'int')])), ('returns', 'str')]),
>> + OrderedDict([('enum', 'TpmModel'), ('data', ['tpm-tis'])]),
>> + OrderedDict([('command', 'query-tpm-models'), ('returns', ['TpmModel'])]),
>> + OrderedDict([('command', 'guest-get-time'), ('returns', 'int')]),
>> + OrderedDict([('command', 'no-way-this-will-get-whitelisted'), ('returns', ['int'])])]
>> +[{'enum_name': 'TpmModel', 'enum_values': ['tpm-tis']}]
>> +[]
>
> Since I found nothing that's actually wrong:
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions Eric Blake
2015-03-26 17:21 ` Markus Armbruster
@ 2015-03-27 7:52 ` Markus Armbruster
2015-03-27 19:53 ` Eric Blake
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 7:52 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
One more...
Eric Blake <eblake@redhat.com> writes:
[...]
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 90eb3bc..5d0dc91 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
[...]
> @@ -560,12 +585,22 @@ def type_name(name):
> return c_list_type(name[0])
> return name
>
> -enum_types = []
> -struct_types = []
> -union_types = []
> +def add_name(name, info, meta, implicit = False):
> + global all_names
> + if name in all_names:
> + raise QAPIExprError(info,
> + "%s '%s' is already defined"
> + %(all_names[name], name))
We say "struct 'Foo'", and expect the user to know that 'struct' means
'complex type'. It'll do, it's just a development tool.
I'm not really happy with 'complex type', though. Isn't a union type
complex, too? Anyway, we can clean up our confused terminology later;
this series is long enough.
> + if not implicit and name[-4:] == 'Kind':
> + raise QAPIExprError(info,
> + "%s '%s' should not end in 'Kind'"
> + %(meta, name))
> + all_names[name] = meta
>
> -def add_struct(definition):
> +def add_struct(definition, info):
> global struct_types
> + name = definition['type']
> + add_name(name, info, 'struct')
> struct_types.append(definition)
>
> def find_struct(name):
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types Eric Blake
@ 2015-03-27 8:23 ` Markus Armbruster
2015-03-27 20:03 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 8:23 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Now that we know every expression is valid with regards to
> its keys, we can add further tests that those keys refer to
> valid types. With this patch, all uses of a type (the 'data':
> of command, type, union, alternate, and event; the 'returns':
> of command; the 'base': of type and union) must resolve to an
> appropriate subset of metatypes declared by the current qapi
> parse; this includes recursing into each member of a data
> dictionary. Dealing with '**' and nested anonymous structs
> will be done in later patches.
>
> Update the testsuite to match improved output.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
[...]
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 6ed6a34..c42683b 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -276,6 +276,63 @@ def discriminator_find_enum_define(expr):
>
> return find_enum(discriminator_type)
>
> +def check_type(expr_info, source, data, allow_array = False,
> + allowed_metas = [], allow_dict = True):
I'd allow_dict = False here, because I like defaulting to false. Matter
of taste.
I'd name the third parameter def rather than data. Matter of taste
again.
> + global all_names
> +
> + if data is None:
> + return
> +
> + if data == '**':
> + return
> +
> + # Check if array type for data is okay
> + if isinstance(data, list):
> + if not allow_array:
> + raise QAPIExprError(expr_info,
> + "%s cannot be an array" % source)
> + if len(data) != 1 or not isinstance(data[0], str):
> + raise QAPIExprError(expr_info,
> + "%s: array type must contain single type name"
> + % source)
> + data = data[0]
> +
> + # Check if type name for data is okay
> + if isinstance(data, str):
> + if not data in all_names:
> + raise QAPIExprError(expr_info,
> + "%s uses unknown type '%s'"
> + % (source, data))
> + if not all_names[data] in allowed_metas:
> + raise QAPIExprError(expr_info,
> + "%s cannot use %s type '%s'"
> + % (source, all_names[data], data))
> + return
> +
> + # data is a dictionary, check that each member is okay
> + if not isinstance(data, OrderedDict):
> + raise QAPIExprError(expr_info,
> + "%s should be a dictionary" % source)
> + if not allow_dict:
> + raise QAPIExprError(expr_info,
> + "%s should be a type name" % source)
> + for (key, value) in data.items():
> + check_type(expr_info, "Member '%s' of %s" % (key, source), value,
This can produce messages like
Member 'inner' of Member 'outer' of ...
I figure the problem will go away when you ditch nested structs. Not
worth worrying about it then.
> + allow_array=True,
> + allowed_metas=['built-in', 'union', 'alternate', 'struct',
> + 'enum'],
> + allow_dict=True)
> +
> +def check_command(expr, expr_info):
> + name = expr['command']
> + check_type(expr_info, "'data' for command '%s'" % name,
> + expr.get('data'),
> + allowed_metas=['union', 'struct'])
> + check_type(expr_info, "'returns' for command '%s'" % name,
> + expr.get('returns'), allow_array=True,
> + allowed_metas=['built-in', 'union', 'alternate', 'struct',
> + 'enum'])
> +
> def check_event(expr, expr_info):
> global events
> name = expr['event']
> @@ -287,7 +344,8 @@ def check_event(expr, expr_info):
> raise QAPIExprError(expr_info, "Event name '%s' should be upper case"
> % name)
> events.append(name)
> -
> + check_type(expr_info, "'data' for event '%s'" % name,
> + expr.get('data'), allowed_metas=['union', 'struct'])
> if params:
> for argname, argentry, optional, structured in parse_args(params):
> if structured:
> @@ -348,6 +406,13 @@ def check_union(expr, expr_info):
>
> # Check every branch
> for (key, value) in members.items():
> + # Each value must name a known type
> + check_type(expr_info, "Member '%s' of union '%s'" % (key, name),
> + value, allow_array=True,
> + allowed_metas=['built-in', 'union', 'alternate', 'struct',
> + 'enum'],
> + allow_dict=False)
> +
> # If the discriminator names an enum type, then all members
> # of 'data' must also be members of the enum type.
> if enum_define:
> @@ -383,15 +448,12 @@ def check_alternate(expr, expr_info):
> values[c_key] = key
>
> # Ensure alternates have no type conflicts.
> - if isinstance(value, list):
> - raise QAPIExprError(expr_info,
> - "Alternate '%s' member '%s' must "
> - "not be array type" % (name, key))
> + check_type(expr_info, "Member '%s' of alternate '%s'" % (key, name),
> + value,
> + allowed_metas=['built-in', 'union', 'struct', 'enum'],
> + allow_dict=False)
> qtype = find_alternate_member_qtype(value)
> - if not qtype:
> - raise QAPIExprError(expr_info,
> - "Alternate '%s' member '%s' has "
> - "invalid type '%s'" % (name, key, value))
> + assert qtype
> if qtype in types_seen:
> raise QAPIExprError(expr_info,
> "Alternate '%s' member '%s' has "
> @@ -419,6 +481,14 @@ def check_enum(expr, expr_info):
> % (name, member, values[key]))
> values[key] = member
>
> +def check_struct(expr, expr_info):
> + name = expr['type']
> + members = expr['data']
> +
> + check_type(expr_info, "'data' for type '%s'" % name, members)
This one gave me pause, until I realized that allowed_metas=[],
allow_dict=True accepts exactly dictionary members, which is what we
want.
> + check_type(expr_info, "'base' for type '%s'" % name, expr.get('base'),
> + allowed_metas=['struct'], allow_dict=False)
> +
> def check_exprs(schema):
> for expr_elem in schema.exprs:
> expr = expr_elem['expr']
> @@ -430,8 +500,14 @@ def check_exprs(schema):
> check_union(expr, info)
> elif expr.has_key('alternate'):
> check_alternate(expr, info)
> + elif expr.has_key('type'):
> + check_struct(expr, info)
> + elif expr.has_key('command'):
> + check_command(expr, info)
> elif expr.has_key('event'):
> check_event(expr, info)
> + else:
> + assert False, 'unexpected meta type'
>
> def check_keys(expr_elem, meta, required, optional=[]):
> expr = expr_elem['expr']
[...]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names Eric Blake
@ 2015-03-27 8:48 ` Markus Armbruster
2015-03-27 20:15 ` Eric Blake
2015-03-27 17:14 ` Markus Armbruster
2015-03-29 9:06 ` Markus Armbruster
2 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 8:48 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Previous commits demonstrated that the generator overlooked various
> bad naming situations:
> - types, commands, and events need a valid name
> - union and alternate branches cannot be marked optional
>
> The set of valid names includes [a-zA-Z0-9._-] (where '.' is
> useful only in downstream extensions).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 57 ++++++++++++++++------
> tests/qapi-schema/bad-ident.err | 1 +
> tests/qapi-schema/bad-ident.exit | 2 +-
> tests/qapi-schema/bad-ident.json | 2 +-
> tests/qapi-schema/bad-ident.out | 3 --
> tests/qapi-schema/flat-union-bad-discriminator.err | 2 +-
> .../flat-union-optional-discriminator.err | 1 +
> .../flat-union-optional-discriminator.exit | 2 +-
> .../flat-union-optional-discriminator.json | 2 +-
> .../flat-union-optional-discriminator.out | 5 --
> tests/qapi-schema/union-optional-branch.err | 1 +
> tests/qapi-schema/union-optional-branch.exit | 2 +-
> tests/qapi-schema/union-optional-branch.json | 2 +-
> tests/qapi-schema/union-optional-branch.out | 3 --
> 14 files changed, 53 insertions(+), 32 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index c42683b..ed5385a 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -15,6 +15,7 @@ import re
> from ordereddict import OrderedDict
> import os
> import sys
> +import string
>
> builtin_types = {
> 'str': 'QTYPE_QSTRING',
> @@ -276,8 +277,27 @@ def discriminator_find_enum_define(expr):
>
> return find_enum(discriminator_type)
>
> +valid_characters = set(string.ascii_letters + string.digits + '.' + '-' + '_')
strings.ascii_letters depends on the locale...
> +def check_name(expr_info, source, name, allow_optional = False):
> + membername = name
> +
> + if not isinstance(name, str):
> + raise QAPIExprError(expr_info,
> + "%s requires a string name" % source)
> + if name == '**':
> + return
Doesn't this permit '**' anywhere, not just as pseudo-type in command
arguments and results?
> + if name.startswith('*'):
> + membername = name[1:]
> + if not allow_optional:
> + raise QAPIExprError(expr_info,
> + "%s does not allow optional name '%s'"
> + % (source, name))
> + if not set(membername) <= valid_characters:
... so this check would break if we called locale.setlocale() in this
program. While I don't think we need to worry about it, I think you
could just as well use something like
valid_name = re.compile(r"^[A-Za-z0-9-._]+$")
if not valid_name.match(membername):
> + raise QAPIExprError(expr_info,
> + "%s uses invalid name '%s'" % (source, name))
> +
> def check_type(expr_info, source, data, allow_array = False,
> - allowed_metas = [], allow_dict = True):
> + allowed_metas = [], allow_dict = True, allow_optional = False):
> global all_names
>
> if data is None:
> @@ -317,21 +337,23 @@ def check_type(expr_info, source, data, allow_array = False,
> raise QAPIExprError(expr_info,
> "%s should be a type name" % source)
> for (key, value) in data.items():
> + check_name(expr_info, "Member of %s" % source, key,
> + allow_optional=allow_optional)
> check_type(expr_info, "Member '%s' of %s" % (key, source), value,
> allow_array=True,
> allowed_metas=['built-in', 'union', 'alternate', 'struct',
> 'enum'],
> - allow_dict=True)
> + allow_dict=True, allow_optional=True)
>
> def check_command(expr, expr_info):
> name = expr['command']
> check_type(expr_info, "'data' for command '%s'" % name,
> expr.get('data'),
> - allowed_metas=['union', 'struct'])
> + allowed_metas=['union', 'struct'], allow_optional=True)
> check_type(expr_info, "'returns' for command '%s'" % name,
> expr.get('returns'), allow_array=True,
> allowed_metas=['built-in', 'union', 'alternate', 'struct',
> - 'enum'])
> + 'enum'], allow_optional=True)
>
> def check_event(expr, expr_info):
> global events
> @@ -345,7 +367,8 @@ def check_event(expr, expr_info):
> % name)
> events.append(name)
> check_type(expr_info, "'data' for event '%s'" % name,
> - expr.get('data'), allowed_metas=['union', 'struct'])
> + expr.get('data'), allowed_metas=['union', 'struct'],
> + allow_optional=True)
> if params:
> for argname, argentry, optional, structured in parse_args(params):
> if structured:
> @@ -385,12 +408,10 @@ def check_union(expr, expr_info):
> "Base '%s' is not a valid base type"
> % base)
>
> - # The value of member 'discriminator' must name a member of the
> - # base type.
> - if not isinstance(discriminator, str):
> - raise QAPIExprError(expr_info,
> - "Flat union '%s' must have a string "
> - "discriminator field" % name)
> + # The value of member 'discriminator' must name a non-optional
> + # member of the base type.
> + check_name(expr_info, "Discriminator of flat union '%s'" % name,
> + discriminator)
> discriminator_type = base_fields.get(discriminator)
> if not discriminator_type:
> raise QAPIExprError(expr_info,
What happens when I try 'discriminator': '**'?
> @@ -406,6 +427,8 @@ def check_union(expr, expr_info):
>
> # Check every branch
> for (key, value) in members.items():
> + check_name(expr_info, "Member of union '%s'" % name, key)
> +
> # Each value must name a known type
> check_type(expr_info, "Member '%s' of union '%s'" % (key, name),
> value, allow_array=True,
> @@ -439,6 +462,8 @@ def check_alternate(expr, expr_info):
>
> # Check every branch
> for (key, value) in members.items():
> + check_name(expr_info, "Member of alternate '%s'" % name, key)
> +
> # Check for conflicts in the generated enum
> c_key = _generate_enum_string(key)
> if c_key in values:
> @@ -485,7 +510,8 @@ def check_struct(expr, expr_info):
> name = expr['type']
> members = expr['data']
>
> - check_type(expr_info, "'data' for type '%s'" % name, members)
> + check_type(expr_info, "'data' for type '%s'" % name, members,
> + allow_optional=True)
> check_type(expr_info, "'base' for type '%s'" % name, expr.get('base'),
> allowed_metas=['struct'], allow_dict=False)
>
> @@ -676,8 +702,11 @@ def type_name(name):
> return c_list_type(name[0])
> return name
>
> -def add_name(name, info, meta, implicit = False):
> +def add_name(name, info, meta, implicit = False, source = None):
> global all_names
> + if not source:
> + source = "'%s'" % meta
> + check_name(info, source, name)
> if name in all_names:
> raise QAPIExprError(info,
> "%s '%s' is already defined"
> @@ -691,7 +720,7 @@ def add_name(name, info, meta, implicit = False):
> def add_struct(definition, info):
> global struct_types
> name = definition['type']
> - add_name(name, info, 'struct')
> + add_name(name, info, 'struct', source="'type'")
> struct_types.append(definition)
>
> def find_struct(name):
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary Eric Blake
@ 2015-03-27 9:11 ` Markus Armbruster
2015-03-27 20:20 ` Eric Blake
2015-03-27 16:19 ` Markus Armbruster
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 9:11 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> ...or an array of dictionaries. Although we have to cater to
> existing commands, returning a non-dictionary means the command
> is not extensible (no new name/value pairs can be added if more
> information must be returned in parallel). By making the
> whitelist explicit, any new command that falls foul of this
> practice will have to be self-documenting, which will encourage
> developers to either justify the action or rework the design to
> use a dictionary after all.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 30 ++++++++++++++++++++++++++++--
> tests/qapi-schema/returns-alternate.err | 1 +
> tests/qapi-schema/returns-alternate.exit | 2 +-
> tests/qapi-schema/returns-alternate.json | 2 +-
> tests/qapi-schema/returns-alternate.out | 4 ----
> tests/qapi-schema/returns-int.json | 3 ++-
> tests/qapi-schema/returns-int.out | 2 +-
> tests/qapi-schema/returns-whitelist.err | 1 +
> tests/qapi-schema/returns-whitelist.exit | 2 +-
> tests/qapi-schema/returns-whitelist.json | 2 +-
> tests/qapi-schema/returns-whitelist.out | 7 -------
> 11 files changed, 37 insertions(+), 19 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index ed5385a..9421431 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -33,6 +33,30 @@ builtin_types = {
> 'size': 'QTYPE_QINT',
> }
>
> +# Whitelist of commands allowed to return a non-dictionary
> +returns_whitelist = [
> + # From QMP:
> + 'human-monitor-command',
> + 'query-migrate-cache-size',
> + 'query-tpm-models',
> + 'query-tpm-types',
> + 'ringbuf-read',
> +
> + # From QGA:
> + 'guest-file-open',
> + 'guest-fsfreeze-freeze',
> + 'guest-fsfreeze-freeze-list',
> + 'guest-fsfreeze-status',
> + 'guest-fsfreeze-thaw',
> + 'guest-get-time',
> + 'guest-set-vcpus',
> + 'guest-sync',
> + 'guest-sync-delimited',
> +
> + # From qapi-schema-test:
> + 'user_def_cmd3',
> +]
> +
Since there's just one whitelist, all schemata share it, and that means
it's too permissive for each of them. Sloppy, but good enough.
If the sloppiness bothers us, here are two alternatives:
* Program takes the whitelist as argument, say
scripts/qapi-commands.py --legacy-returns qmp-legacy-returns ...
* Leave enforcing to C
If a command 'frobnicate' returns a non-dictionary, generate something
like
#ifndef FROBNICATE_LEGACY_RETURN_OK
#error Command 'frobnicate' should return a dictionary
#endif
Then manually define the macros necessary to keep the current use
working in a suitable header.
> enum_types = []
> struct_types = []
> union_types = []
> @@ -350,10 +374,12 @@ def check_command(expr, expr_info):
> check_type(expr_info, "'data' for command '%s'" % name,
> expr.get('data'),
> allowed_metas=['union', 'struct'], allow_optional=True)
> + returns_meta = ['union', 'struct']
> + if name in returns_whitelist:
> + returns_meta += ['built-in', 'alternate', 'enum']
> check_type(expr_info, "'returns' for command '%s'" % name,
> expr.get('returns'), allow_array=True,
> - allowed_metas=['built-in', 'union', 'alternate', 'struct',
> - 'enum'], allow_optional=True)
> + allowed_metas=returns_meta, allow_optional=True)
>
> def check_event(expr, expr_info):
> global events
[...]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 23/28] qapi: More rigorous checking for type safety bypass
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 23/28] qapi: More rigorous checking for type safety bypass Eric Blake
@ 2015-03-27 9:45 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 9:45 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Now that we have a way to validate every type, we can also be
> stricter about enforcing that callers that want to bypass
> type safety in generated code. Prior to this patch, it didn't
> matter what value was associated with the key 'gen', but it
> looked odd that 'gen':'yes' could result in bypassing the
> generated code. These changes also enforce the changes made
> earlier in the series for documentation and consolidation of
> using '**' as the wildcard type, as well as 'gen':false as the
> canonical spelling for requesting type bypass.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 23 ++++++++++++++++++-----
> tests/qapi-schema/type-bypass-bad-gen.err | 1 +
> tests/qapi-schema/type-bypass-bad-gen.exit | 2 +-
> tests/qapi-schema/type-bypass-bad-gen.json | 2 +-
> tests/qapi-schema/type-bypass-bad-gen.out | 3 ---
> tests/qapi-schema/type-bypass-no-gen.err | 1 +
> tests/qapi-schema/type-bypass-no-gen.exit | 2 +-
> tests/qapi-schema/type-bypass-no-gen.json | 2 +-
> tests/qapi-schema/type-bypass-no-gen.out | 3 ---
> 9 files changed, 24 insertions(+), 15 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 9421431..800e8e4 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -321,13 +321,14 @@ def check_name(expr_info, source, name, allow_optional = False):
> "%s uses invalid name '%s'" % (source, name))
>
> def check_type(expr_info, source, data, allow_array = False,
> - allowed_metas = [], allow_dict = True, allow_optional = False):
> + allowed_metas = [], allow_dict = True, allow_optional = False,
> + allow_star = False):
> global all_names
>
> if data is None:
> return
>
> - if data == '**':
> + if allow_star and data == '**':
> return
>
> # Check if array type for data is okay
Aha! Takes us back to my question on PATCH 21.
State so far: check_name() accepts '**', but some of its callers reject
it. Whether it's rejected everywhere we don't want it isn't obvious,
even if we add rejects that are currently missing.
Perhaps we should try to do this the other way round: require extra code
in places where we want to accept it.
I can accept this as is, because it's an improvement, and we can plug
remaining holes on top.
> @@ -343,6 +344,10 @@ def check_type(expr_info, source, data, allow_array = False,
>
> # Check if type name for data is okay
> if isinstance(data, str):
> + if data == '**':
> + raise QAPIExprError(expr_info,
> + "%s uses '**' but did not request 'gen':false"
> + % source)
> if not data in all_names:
> raise QAPIExprError(expr_info,
> "%s uses unknown type '%s'"
> @@ -367,19 +372,23 @@ def check_type(expr_info, source, data, allow_array = False,
> allow_array=True,
> allowed_metas=['built-in', 'union', 'alternate', 'struct',
> 'enum'],
> - allow_dict=True, allow_optional=True)
> + allow_dict=True, allow_optional=True, allow_star=allow_star)
>
> def check_command(expr, expr_info):
> name = expr['command']
> + allow_star = expr.has_key('gen')
> +
> check_type(expr_info, "'data' for command '%s'" % name,
> expr.get('data'),
> - allowed_metas=['union', 'struct'], allow_optional=True)
> + allowed_metas=['union', 'struct'], allow_optional=True,
> + allow_star=allow_star)
> returns_meta = ['union', 'struct']
> if name in returns_whitelist:
> returns_meta += ['built-in', 'alternate', 'enum']
> check_type(expr_info, "'returns' for command '%s'" % name,
> expr.get('returns'), allow_array=True,
> - allowed_metas=returns_meta, allow_optional=True)
> + allowed_metas=returns_meta, allow_optional=True,
> + allow_star=allow_star)
>
> def check_event(expr, expr_info):
> global events
> @@ -574,6 +583,10 @@ def check_keys(expr_elem, meta, required, optional=[]):
> raise QAPIExprError(info,
> "%s '%s' has unknown key '%s'"
> % (meta, name, key))
> + if (key == 'gen' or key == 'success-response') and value != False:
> + raise QAPIExprError(info,
> + "'%s' of %s '%s' should only use false value"
> + % (key, meta, name))
> for key in required:
> if not expr.has_key(key):
> raise QAPIExprError(info,
> diff --git a/tests/qapi-schema/type-bypass-bad-gen.err b/tests/qapi-schema/type-bypass-bad-gen.err
> index e69de29..a83c3c6 100644
> --- a/tests/qapi-schema/type-bypass-bad-gen.err
> +++ b/tests/qapi-schema/type-bypass-bad-gen.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/type-bypass-bad-gen.json:2: 'gen' of command 'foo' should only use false value
> diff --git a/tests/qapi-schema/type-bypass-bad-gen.exit b/tests/qapi-schema/type-bypass-bad-gen.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/type-bypass-bad-gen.exit
> +++ b/tests/qapi-schema/type-bypass-bad-gen.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/type-bypass-bad-gen.json b/tests/qapi-schema/type-bypass-bad-gen.json
> index bb70bee..e8dec34 100644
> --- a/tests/qapi-schema/type-bypass-bad-gen.json
> +++ b/tests/qapi-schema/type-bypass-bad-gen.json
> @@ -1,2 +1,2 @@
> -# FIXME: 'gen' should only appear with value false
> +# 'gen' should only appear with value false
> { 'command': 'foo', 'gen': 'whatever' }
> diff --git a/tests/qapi-schema/type-bypass-bad-gen.out b/tests/qapi-schema/type-bypass-bad-gen.out
> index e678f2c..e69de29 100644
> --- a/tests/qapi-schema/type-bypass-bad-gen.out
> +++ b/tests/qapi-schema/type-bypass-bad-gen.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('command', 'foo'), ('gen', 'whatever')])]
> -[]
> -[]
> diff --git a/tests/qapi-schema/type-bypass-no-gen.err b/tests/qapi-schema/type-bypass-no-gen.err
> index e69de29..20cef0a 100644
> --- a/tests/qapi-schema/type-bypass-no-gen.err
> +++ b/tests/qapi-schema/type-bypass-no-gen.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/type-bypass-no-gen.json:2: Member 'arg' of 'data' for command 'unsafe' uses '**' but did not request 'gen':false
> diff --git a/tests/qapi-schema/type-bypass-no-gen.exit b/tests/qapi-schema/type-bypass-no-gen.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/type-bypass-no-gen.exit
> +++ b/tests/qapi-schema/type-bypass-no-gen.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/type-bypass-no-gen.json b/tests/qapi-schema/type-bypass-no-gen.json
> index af87c19..49b5742 100644
> --- a/tests/qapi-schema/type-bypass-no-gen.json
> +++ b/tests/qapi-schema/type-bypass-no-gen.json
> @@ -1,2 +1,2 @@
> -# FIXME: type bypass should only work with 'gen':false
> +# type bypass only works with 'gen':'no'
'gen': false
> { 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**' }
> diff --git a/tests/qapi-schema/type-bypass-no-gen.out b/tests/qapi-schema/type-bypass-no-gen.out
> index 8b2a9ac..e69de29 100644
> --- a/tests/qapi-schema/type-bypass-no-gen.out
> +++ b/tests/qapi-schema/type-bypass-no-gen.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**')])]
> -[]
> -[]
With the trivial fix in type-bypass-no-gen.json:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
@ 2015-03-27 9:52 ` Markus Armbruster
2015-03-27 20:30 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 9:52 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> In the testsuite, UserDefTwo and UserDefNested were identical
> types other than the member names. Reduce code duplication by
> having just one type, and choose names that also favor reuse.
> This will also make it easier for a later patch to get rid of
> inline nested types in QAPI; it means that the type is now boxed
> instead of unboxed in C code, but has no difference to the QMP
> wire protocol.
I can't see a change from boxed to unboxed in this patch. Does the
remark apply to the elimination of nested types? If yes, it's premature
here.
> When touching code to add new allocations, also
> convert existing allocations to consistently prefer typesafe
> g_new0 over g_malloc0.
I can't see any change from g_malloc0() to g_new0(). Is this stale?
>
> Ensure that 'make check-qapi-schema check-unit' still passes.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 25/28] qapi: Drop tests for inline nested structs
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 25/28] qapi: Drop tests for inline nested structs Eric Blake
@ 2015-03-27 10:30 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 10:30 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> A future patch will be using a 'name':{dictionary} entry in the
> QAPI schema to specify a default value for an optional argument;
> but existing use of inline nested structs conflicts with that goal.
I'm okay with this rationale as is, just want to mention the more
general motivation again: we want to make the schema language
extensible.
A definition in the schema associates a name with a set of properties in
a certain lexical environment.
Example 1: { 'type': 'Foo', 'data': { MEMBERS... } }
at the top level associates global name 'Foo' with properties (meta-type
struct) and MEMBERS...
Example 2: 'mumble': TYPE
within the MEMBERS... above associates 'mumble' with properties (type
TYPE) and (optional false) within type Foo.
The syntax in example 1 is extensible: when we need another property, we
can add another member, just like we added 'base'.
The syntax in example 2 isn't extensible that way, because the right
hand side can only be a type.
Property optional is encoded in the name: "'*mumble': 'int'" associates
'mumble' with (type int) and (optional true). Convenient, but won't
scale.
What we want to do is change the core syntax to
NAME: { 'type': TYPE }
with syntactic sugar
NAME: TYPE --> NAME: { 'type': TYPE, 'optional': false }
*ONAME: TYPE --> ONAME: { 'type': TYPE, 'optional': true }
where *ONAME is a string starting with '*', and ONAME its tail.
Now we can extend by adding another member.
Naturally, the schema will mostly be written in the sugared form.
Adding optional defaults is an instance of this general "extend the
schema language" pattern.
Feel free to crib from the above if you want more of the motivation in
the commit log,
> This patch fixes the testsuite to avoid inline nested types, by
> breaking the nesting into explicit types; it means that the type
> is now boxed instead of unboxed in C code, but makes no difference
> on the wire.
Yes, this patch actually adds boxing.
The QAPI code generator boxes too much for my taste, but this series is
not the place to mess with that.
> When touching code to add new allocations, also
> convert existing allocations to consistently prefer typesafe
> g_new0 over g_malloc0.
Again, I can't see new uses of g_new0().
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> tests/qapi-schema/qapi-schema-test.json | 12 ++++++--
> tests/qapi-schema/qapi-schema-test.out | 8 +++--
> tests/test-qmp-commands.c | 17 ++++++-----
> tests/test-qmp-input-visitor.c | 14 +++++----
> tests/test-qmp-output-visitor.c | 44 +++++++++++++++------------
> tests/test-visitor-serialization.c | 53 ++++++++++++++++++---------------
> 6 files changed, 87 insertions(+), 61 deletions(-)
>
> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index fb6a350..7aeb490 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -14,11 +14,17 @@
> 'base': 'UserDefZero',
> 'data': { 'string': 'str', '*enum1': 'EnumOne' } }
>
> +{ 'type': 'UserDefTwoDictDict',
> + 'data': { 'userdef': 'UserDefOne', 'string': 'str' } }
> +
> +{ 'type': 'UserDefTwoDict',
> + 'data': { 'string1': 'str',
> + 'dict2': 'UserDefTwoDictDict',
> + '*dict3': 'UserDefTwoDictDict' } }
> +
> { 'type': 'UserDefTwo',
> 'data': { 'string0': 'str',
> - 'dict1': { 'string1': 'str',
> - 'dict2': { 'userdef': 'UserDefOne', 'string': 'str' },
> - '*dict3': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
> + 'dict1': 'UserDefTwoDict' } }
>
> # for testing unions
> { 'type': 'UserDefA',
[...]
> diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
> index 9189cd2..dc199d3 100644
> --- a/tests/test-qmp-commands.c
> +++ b/tests/test-qmp-commands.c
> @@ -33,12 +33,15 @@ UserDefTwo *qmp_user_def_cmd2(UserDefOne *ud1a,
>
> ret = g_malloc0(sizeof(UserDefTwo));
> ret->string0 = strdup("blah1");
> - ret->dict1.string1 = strdup("blah2");
> - ret->dict1.dict2.userdef = ud1c;
> - ret->dict1.dict2.string = strdup("blah3");
> - ret->dict1.has_dict3 = true;
> - ret->dict1.dict3.userdef = ud1d;
> - ret->dict1.dict3.string = strdup("blah4");
> + ret->dict1 = g_malloc0(sizeof(UserDefTwoDict));
Hmm, you could g_new0(UserDefTwoDict) here. More of the same below, and
in the previous patch.
> + ret->dict1->string1 = strdup("blah2");
> + ret->dict1->dict2 = g_malloc0(sizeof(UserDefTwoDictDict));
> + ret->dict1->dict2->userdef = ud1c;
> + ret->dict1->dict2->string = strdup("blah3");
> + ret->dict1->dict3 = g_malloc0(sizeof(UserDefTwoDictDict));
> + ret->dict1->has_dict3 = true;
> + ret->dict1->dict3->userdef = ud1d;
> + ret->dict1->dict3->string = strdup("blah4");
>
> return ret;
> }
[...]
With either the commit message corrected not to claim use of g_new0(),
or the patch updated to actually do it:
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 26/28] qapi: Drop inline nested type in query-version
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 26/28] qapi: Drop inline nested type in query-version Eric Blake
@ 2015-03-27 10:34 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 10:34 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> A future patch will be using a 'name':{dictionary} entry in the
> QAPI schema to specify a default value for an optional argument;
> but existing use of inline nested structs conflicts with that goal.
> This patch fixes one of only two commands relying on nested
> types, by breaking the nesting into an explicit type; it means
> that the type is now boxed instead of unboxed in C code, but the
> QMP wire format is unaffected by this change.
>
> Prefer the safer g_new0() while making the conversion.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 27/28] qapi: Drop inline nested types in query-pci
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 27/28] qapi: Drop inline nested types in query-pci Eric Blake
@ 2015-03-27 10:37 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 10:37 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> A future patch will be using a 'name':{dictionary} entry in the
> QAPI schema to specify a default value for an optional argument;
> but existing use of inline nested structs conflicts with that goal.
> This patch fixes one of only two commands relying on nested
> types, by breaking the nesting into an explicit type; it means
> that the type is now boxed instead of unboxed in C code, but the
> QMP wire format is unaffected by this change.
>
> Prefer the safer g_new0() while making the conversion, and reduce
> some long lines.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 28/28] qapi: Drop support for inline nested types
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 28/28] qapi: Drop support for inline nested types Eric Blake
@ 2015-03-27 10:45 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 10:45 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> A future patch will be using a 'name':{dictionary} entry in the
> QAPI schema to specify a default value for an optional argument;
> but existing use of inline nested structs conflicts with that goal.
> Now that all commands have been changed to avoid inline nested
> structs, nuke support for them, and turn it into a hard error.
> Update the testsuite to reflect tighter parsing rules.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi-commands.py | 8 +++---
> scripts/qapi-event.py | 4 +--
> scripts/qapi-types.py | 9 ++-----
> scripts/qapi-visit.py | 37 ++++------------------------
> scripts/qapi.py | 18 +++++---------
> tests/qapi-schema/event-nest-struct.err | 2 +-
> tests/qapi-schema/nested-struct-data.err | 1 +
> tests/qapi-schema/nested-struct-data.exit | 2 +-
> tests/qapi-schema/nested-struct-data.json | 2 +-
> tests/qapi-schema/nested-struct-data.out | 3 ---
> tests/qapi-schema/nested-struct-returns.err | 1 +
> tests/qapi-schema/nested-struct-returns.exit | 2 +-
> tests/qapi-schema/nested-struct-returns.json | 2 +-
> tests/qapi-schema/nested-struct-returns.out | 3 ---
> 14 files changed, 26 insertions(+), 68 deletions(-)
>
> diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
> index 053ba85..db81044 100644
> --- a/scripts/qapi-commands.py
> +++ b/scripts/qapi-commands.py
> @@ -28,7 +28,7 @@ def type_visitor(name):
>
> def generate_command_decl(name, args, ret_type):
> arglist=""
> - for argname, argtype, optional, structured in parse_args(args):
> + for argname, argtype, optional in parse_args(args):
> argtype = c_type(argtype, is_param=True)
> if optional:
> arglist += "bool has_%s, " % c_var(argname)
> @@ -53,7 +53,7 @@ def gen_sync_call(name, args, ret_type, indent=0):
> retval=""
> if ret_type:
> retval = "retval = "
> - for argname, argtype, optional, structured in parse_args(args):
> + for argname, argtype, optional in parse_args(args):
> if optional:
> arglist += "has_%s, " % c_var(argname)
> arglist += "%s, " % (c_var(argname))
> @@ -96,7 +96,7 @@ Visitor *v;
> def gen_visitor_input_vars_decl(args):
> ret = ""
> push_indent()
> - for argname, argtype, optional, structured in parse_args(args):
> + for argname, argtype, optional in parse_args(args):
> if optional:
> ret += mcgen('''
> bool has_%(argname)s = false;
> @@ -139,7 +139,7 @@ v = qapi_dealloc_get_visitor(md);
> v = qmp_input_get_visitor(mi);
> ''')
>
> - for argname, argtype, optional, structured in parse_args(args):
> + for argname, argtype, optional in parse_args(args):
> if optional:
> ret += mcgen('''
> visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
> diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
> index 601e307..47dc041 100644
> --- a/scripts/qapi-event.py
> +++ b/scripts/qapi-event.py
> @@ -21,7 +21,7 @@ def _generate_event_api_name(event_name, params):
> l = len(api_name)
>
> if params:
> - for argname, argentry, optional, structured in parse_args(params):
> + for argname, argentry, optional in parse_args(params):
> if optional:
> api_name += "bool has_%s,\n" % c_var(argname)
> api_name += "".ljust(l)
> @@ -93,7 +93,7 @@ def generate_event_implement(api_name, event_name, params):
> """,
> event_name = event_name)
>
> - for argname, argentry, optional, structured in parse_args(params):
> + for argname, argentry, optional in parse_args(params):
> if optional:
> ret += mcgen("""
> if (has_%(var)s) {
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index 9c8d68c..4789528 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -63,18 +63,13 @@ typedef struct %(name)sList
> def generate_struct_fields(members):
> ret = ''
>
> - for argname, argentry, optional, structured in parse_args(members):
> + for argname, argentry, optional in parse_args(members):
> if optional:
> ret += mcgen('''
> bool has_%(c_name)s;
> ''',
> c_name=c_var(argname))
> - if structured:
> - push_indent()
> - ret += generate_struct({ "field": argname, "data": argentry})
> - pop_indent()
> - else:
> - ret += mcgen('''
> + ret += mcgen('''
> %(c_type)s %(c_name)s;
> ''',
> c_type=c_type(argentry), c_name=c_var(argname))
> diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
> index 3e11089..10b08c6 100644
> --- a/scripts/qapi-visit.py
> +++ b/scripts/qapi-visit.py
> @@ -51,27 +51,6 @@ def generate_visit_struct_fields(name, field_prefix, fn_prefix, members, base =
> else:
> full_name = "%s_%s" % (name, fn_prefix)
>
> - for argname, argentry, optional, structured in parse_args(members):
> - if structured:
> - if not fn_prefix:
> - nested_fn_prefix = argname
> - else:
> - nested_fn_prefix = "%s_%s" % (fn_prefix, argname)
> -
> - nested_field_prefix = "%s%s." % (field_prefix, argname)
> - ret += generate_visit_struct_fields(name, nested_field_prefix,
> - nested_fn_prefix, argentry)
> - ret += mcgen('''
> -
> -static void visit_type_%(full_name)s_field_%(c_name)s(Visitor *m, %(name)s **obj, Error **errp)
> -{
> -''',
> - name=name, full_name=full_name, c_name=c_var(argname))
> - ret += generate_visit_struct_body(full_name, argname, argentry)
> - ret += mcgen('''
> -}
> -''')
> -
> if base:
> ret += generate_visit_implicit_struct(base)
>
> @@ -94,7 +73,7 @@ if (err) {
> c_prefix=c_var(field_prefix),
> type=type_name(base), c_name=c_var('base'))
>
> - for argname, argentry, optional, structured in parse_args(members):
> + for argname, argentry, optional in parse_args(members):
> if optional:
> ret += mcgen('''
> visit_optional(m, &(*obj)->%(c_prefix)shas_%(c_name)s, "%(name)s", &err);
> @@ -104,18 +83,12 @@ if (!err && (*obj)->%(prefix)shas_%(c_name)s) {
> c_name=c_var(argname), name=argname)
> push_indent()
>
> - if structured:
> - ret += mcgen('''
> -visit_type_%(full_name)s_field_%(c_name)s(m, obj, &err);
> -''',
> - full_name=full_name, c_name=c_var(argname))
> - else:
> - ret += mcgen('''
> + ret += mcgen('''
> visit_type_%(type)s(m, &(*obj)->%(c_prefix)s%(c_name)s, "%(name)s", &err);
> ''',
> - c_prefix=c_var(field_prefix), prefix=field_prefix,
> - type=type_name(argentry), c_name=c_var(argname),
> - name=argname)
> + c_prefix=c_var(field_prefix), prefix=field_prefix,
> + type=type_name(argentry), c_name=c_var(argname),
> + name=argname)
>
> if optional:
> pop_indent()
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 800e8e4..8e5b4ad 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -368,11 +368,13 @@ def check_type(expr_info, source, data, allow_array = False,
> for (key, value) in data.items():
> check_name(expr_info, "Member of %s" % source, key,
> allow_optional=allow_optional)
> + # Todo: allow dictionaries to represent default values of
> + # an optional argument.
> check_type(expr_info, "Member '%s' of %s" % (key, source), value,
> allow_array=True,
> allowed_metas=['built-in', 'union', 'alternate', 'struct',
> 'enum'],
> - allow_dict=True, allow_optional=True, allow_star=allow_star)
> + allow_dict=False, allow_star=allow_star)
check_type() uses allow_optional only when allow_dict. Okay.
>
> def check_command(expr, expr_info):
> name = expr['command']
> @@ -404,13 +406,6 @@ def check_event(expr, expr_info):
> check_type(expr_info, "'data' for event '%s'" % name,
> expr.get('data'), allowed_metas=['union', 'struct'],
> allow_optional=True)
> - if params:
> - for argname, argentry, optional, structured in parse_args(params):
> - if structured:
> - raise QAPIExprError(expr_info,
> - "Nested structure define in event is not "
> - "supported, event '%s', argname '%s'"
> - % (expr['event'], argname))
>
> def check_union(expr, expr_info):
> name = expr['union']
> @@ -667,13 +662,12 @@ def parse_args(typeinfo):
> argname = member
> argentry = typeinfo[member]
> optional = False
> - structured = False
> if member.startswith('*'):
> argname = member[1:]
> optional = True
> - if isinstance(argentry, OrderedDict):
> - structured = True
> - yield (argname, argentry, optional, structured)
> + # Todo: allow argentry to be OrderedDict, for providing the
> + # value of an optional argument.
> + yield (argname, argentry, optional)
>
> def de_camel_case(name):
> new_name = ''
> diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
> index 91bde1c..5a42701 100644
> --- a/tests/qapi-schema/event-nest-struct.err
> +++ b/tests/qapi-schema/event-nest-struct.err
> @@ -1 +1 @@
> -tests/qapi-schema/event-nest-struct.json:1: Nested structure define in event is not supported, event 'EVENT_A', argname 'a'
> +tests/qapi-schema/event-nest-struct.json:1: Member 'a' of 'data' for event 'EVENT_A' should be a type name
> diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
> index e69de29..da767ba 100644
> --- a/tests/qapi-schema/nested-struct-data.err
> +++ b/tests/qapi-schema/nested-struct-data.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/nested-struct-data.json:2: Member 'a' of 'data' for command 'foo' should be a type name
> diff --git a/tests/qapi-schema/nested-struct-data.exit b/tests/qapi-schema/nested-struct-data.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/nested-struct-data.exit
> +++ b/tests/qapi-schema/nested-struct-data.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/nested-struct-data.json b/tests/qapi-schema/nested-struct-data.json
> index 0247c8c..3d52d2b 100644
> --- a/tests/qapi-schema/nested-struct-data.json
> +++ b/tests/qapi-schema/nested-struct-data.json
> @@ -1,4 +1,4 @@
> -# FIXME: inline subtypes collide with our desired future use of defaults
Seen only now: I'd mark this TODO rather than FIXME in PATCH 19, because
it's not actually broken. Only if you respin anyway, of course.
> +# inline subtypes collide with our desired future use of defaults
> { 'command': 'foo',
> 'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' },
> 'returns': {} }
> diff --git a/tests/qapi-schema/nested-struct-data.out b/tests/qapi-schema/nested-struct-data.out
> index 999cbb8..e69de29 100644
> --- a/tests/qapi-schema/nested-struct-data.out
> +++ b/tests/qapi-schema/nested-struct-data.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('command', 'foo'), ('data', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')])), ('returns', OrderedDict())])]
> -[]
> -[]
> diff --git a/tests/qapi-schema/nested-struct-returns.err b/tests/qapi-schema/nested-struct-returns.err
> index e69de29..5238d07 100644
> --- a/tests/qapi-schema/nested-struct-returns.err
> +++ b/tests/qapi-schema/nested-struct-returns.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/nested-struct-returns.json:2: Member 'a' of 'returns' for command 'foo' should be a type name
> diff --git a/tests/qapi-schema/nested-struct-returns.exit b/tests/qapi-schema/nested-struct-returns.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/nested-struct-returns.exit
> +++ b/tests/qapi-schema/nested-struct-returns.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/nested-struct-returns.json b/tests/qapi-schema/nested-struct-returns.json
> index 5a46840..d2cd047 100644
> --- a/tests/qapi-schema/nested-struct-returns.json
> +++ b/tests/qapi-schema/nested-struct-returns.json
> @@ -1,3 +1,3 @@
> -# FIXME: inline subtypes collide with our desired future use of defaults
> +# inline subtypes collide with our desired future use of defaults
> { 'command': 'foo',
> 'returns': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }
> diff --git a/tests/qapi-schema/nested-struct-returns.out b/tests/qapi-schema/nested-struct-returns.out
> index c53d23b..e69de29 100644
> --- a/tests/qapi-schema/nested-struct-returns.out
> +++ b/tests/qapi-schema/nested-struct-returns.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('command', 'foo'), ('returns', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')]))])]
> -[]
> -[]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-26 15:04 ` Eric Blake
@ 2015-03-27 12:30 ` Markus Armbruster
2015-03-27 19:47 ` Eric Blake
2015-03-31 17:13 ` Kevin Wolf
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 12:30 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/26/2015 07:18 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Demonstrate that the qapi generator doesn't deal well with unions
>>> that aren't up to par. Later patches will update the expected
>>> reseults as the generator is made stricter.
>>>
>>> Of particular note, we currently allow 'base' without 'discriminator'
>>> as a way to create a simple union with a base class. However, none
>>> of the existing QMP or QGA interfaces use it (it is exercised only
>>> in the testsuite).
>>
>> qapi-code-gen.txt documents 'base' only for flat unions. We should
>> either document its use in simple unions, or outlaw it.
>
> I went with outlaw later in the series, and the rest of my commit
> message was an attempt to explain my reasoning in that choice. But I
> can certainly try to improve the wording, if we need a respin.
If you respin, I suggest to add that it's undocumented.
>>
>>> Meanwhile, it would be nice to allow
>>> 'discriminator':'EnumType' without 'base' for creating a simple union
>>> that is type-safe rather than open-coded to a generated enum (right
>>> now, we are only type-safe when using a flat union, but that uses
>>> a different syntax of 'discriminator':'member-name' which requires
>>> a base class containing a 'member-name' enum field).
>>
>> I'm afraid I don't get you. Can you give examples?
>
> Using this common code with the appropriate union for each example:
> { 'command':'foo', 'data':UNION }
>
> Right now, we have flat unions which are required to be type-safe (all
> branches MUST map back to the enum type of the discriminator, enforced
> by the generator, so that if the enum later adds a member, the union
> must also be updated to match):
>
> [1]
> { 'union':'Safe', 'base':'Base', 'discriminator':'switch',
> 'data':{ 'one':'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'switch':'one', 'data':'hello'}}
As far as I can tell, the generator rejects flat union 'data' members
that aren't discriminator values. It doesn't require the union to cover
all discriminator members. Makes sense to me, because the union may not
have additional data for some discriminator values.
Your claim "so that if the enum later adds a member, the union must also
be updated to match" appears to be wrong.
> and simple unions which cannot be typesafe (the branches of the union
> are open-coded - even if they correlate to an existing enum, there is
> nothing enforcing that extensions to the enum be reflected into the union):
>
> [2]
> { 'union':'SimpleButOpenCoded',
> 'data':{ 'one': 'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
Ah, now I get what you mean! We have a user-defined enum and the simple
union's implicit enum, and no way to connect the two. In C, we get two
distinct enum types.
Perhaps you can clarify the commit message a bit. Or maybe just drop
the part that confused me. Okay since the relevant FIXME doesn't pass
judgement:
# FIXME: either allow base in non-flat unions, or diagnose missing discriminator
> I'm hoping to add as a followup series a variant of simple unions that
> is type-safe:
>
> [3]
> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
> 'data':{ 'one':'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>
> But the existing, unused-except-in-testsuite, notion of a simple union
> with a base class looks like:
>
> [4]
> { 'type':'Shared', 'data':{'common':'int'}}
> { 'union':'SimpleWithBase', 'base':'Shared',
> 'data':{ 'one':'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
>
> If we were to allow the addition of 'discriminator':'EnumType' to a
> simple union [3], but then add that discriminator to an existing case of
> a simple union with base [4], it would look like:
>
> { 'type':'Shared', 'data':{'common':'int'}}
> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
> 'discriminator':'MyEnum',
> 'data':{ 'one':'str', 'two':'int' }}
>
> Yuck. That is indistinguishable from flat unions [1], except by whether
> discriminator names an enum type or a member of the base class.
Too subtle. The difference should be syntactic, so it's immediately
visible.
>>> If both 'base'
>>> and 'discriminator' are optional, then converting a simple union
>>> with base class to a type-safe simple union with an enum discriminator
>>> would not be possible. So my plan is to get rid of 'base' without
>>> 'discriminator' later in the series;
>>
>> Aha: you're going to outlaw 'base' in simple unions. Yes, please.
>
> Okay, you came to my desired conclusion; it's just that my wording in
> the middle didn't help.
>
>>
>>> this will be no real loss, as any
>>> union that needs additional common fields can always use a flat
>>> union.
>>
>> The mathematical concept behind unions is the sum type.
>>
>> We have three implementations, and we call them simple, flat, anonymous.
>>
>> Anonymous unions are implicitly tagged. They come with the obvious
>> restrictions required to make implicit tagging work.
>
> and get renamed to 'alternate' later in the series, so they are not
> worth worrying about here.
Yes.
>> The other two are explicitly tagged. The difference between them is
>> just notation. I like my unions flat, because for me the extra wrapping
>> is just notational overhead.
>>
>> In particular, I can define simple unions in terms of flat ones by
>> restricting all union cases to a single member named 'data'. They're
>> not implemented that way, but that's a historical accident. Simple
>> unions are a redundant concept.
>
> Cool. Or more concretely,
>
> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>
> is identical on the wire to:
>
> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
> { 'type': 'Branch1', 'data': { 'data': 'str' } }
> { 'type': 'Branch2', 'data': { 'data': 'int' } }
> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
>
> Probably worth mentioning in my commit message.
Yes, please.
> Hmm - that makes me wonder - do we support non-dict branches in a flat
> union? The usual use case of a flat union is that all dictionary keys
> in the branch's dictionary are treated as keys at the top level of the
> resulting overall union; but a non-dictionary branch has no keys to
> flatten into the top level.
I think you just showed why non-dictionary branches make no sense.
> I may need to tweak a subsequent patch to
> ensure that flat union branches can only use dictionaries (while simple
> unions can use any type).
Yes, please. Follow-up patch okay, if that's easier for you.
>> This proves your "can always use a flat union" proposition. The only
>> way they can earn their keep is making the schema easier to read. I
>> haven't thought about that.
>>
>> Another historical accident is how we express members common to all
>> union cases: base type. Probably just because complex types already had
>> them. Are you planning to change anything there?
>
> Maybe, per your own review. More at point [5] below...
>
>>
>> [...]
>>> diff --git a/tests/qapi-schema/alternate-nested.err
>>> b/tests/qapi-schema/alternate-nested.err
>>> new file mode 100644
>>> index 0000000..e69de29
>>> diff --git a/tests/qapi-schema/alternate-nested.exit
>>> b/tests/qapi-schema/alternate-nested.exit
>>> new file mode 100644
>>> index 0000000..573541a
>>> --- /dev/null
>>> +++ b/tests/qapi-schema/alternate-nested.exit
>>> @@ -0,0 +1 @@
>>> +0
>>> diff --git a/tests/qapi-schema/alternate-nested.json
>>> b/tests/qapi-schema/alternate-nested.json
>>> new file mode 100644
>>> index 0000000..d5812bf
>>> --- /dev/null
>>> +++ b/tests/qapi-schema/alternate-nested.json
>>> @@ -0,0 +1,7 @@
>>> +# FIXME: we should reject a nested anonymous union branch
>>
>> Same reason we want to reject nested / anonymous complex types
>> elsewhere? Or is there a deeper reason?
>
> Similar to the reason as we want to reject {'command':'foo',
> 'data':'alternate-type'} for allowing non-dictionaries - we aren't
> currently using it, and it's easier to reject than to worry about making
> corner cases of the generator work on something we won't use.
Okay, that makes sense.
> Also, if I have an alternate A that chooses between string and dict,
> then want to create an alternate B that chooses between int and
> alternate A, will that even generate correct code? An alternate
> represents multiple QObject types at once, so determining the QObject
> type of the next token while parsing the code for union B would have to
> worry about BOTH cases of nested union A.
>
> So the FIXME is correct, and later in the series, I nuke this as
> unsupported.
Good.
>>> +++ b/tests/qapi-schema/flat-union-bad-base.json
>>> @@ -0,0 +1,13 @@
>>> +# we require the base to be an existing complex type
>>> +# FIXME: should we allow an anonymous inline base type?
>>
>> What do you mean by "anonymous inline base type"?
>
> [5] basically, that the following example could be legal shorthand...
>
>>
>>> +{ 'enum': 'TestEnum',
>>> + 'data': [ 'value1', 'value2' ] }
>>> +{ 'type': 'TestTypeA',
>>> + 'data': { 'string': 'str' } }
>>> +{ 'type': 'TestTypeB',
>>> + 'data': { 'integer': 'int' } }
>>> +{ 'union': 'TestUnion',
>>> + 'base': { 'enum1': 'TestEnum', 'kind': 'str' },
>>> + 'discriminator': 'TestEnum',
Shouldn't this be 'discriminator': 'enum1'?
>>> + 'data': { 'kind1': 'TestTypeA',
>>> + 'kind2': 'TestTypeB' } }
>
> where the { 'enum1':'TestEnum', 'kind':'str' } anonymous type defined at
> 'base' should be usable instead of having to name the type with a more
> verbose:
>
> { 'type': 'Base', 'data': {'enum1':'TestEnum', 'kind':'str' }}
> { 'union': 'TestUnion', 'base':'Base', ... }
>
> or in other words, that unions behave more like 'command' allowing
> 'data':{dict} as an anonymous type in place of 'data':'name'.
Got it.
Since there's nothing currently broken, I wouldn't label the question
"should we allow an anonymous inline base type?" FIXME.
>>> diff --git a/tests/qapi-schema/flat-union-bad-base.out
>>> b/tests/qapi-schema/flat-union-bad-base.out
>>> new file mode 100644
>>> index 0000000..e69de29
>> [...]
>>
>> "Doesn't deal well" is an understatement :)
>>
>> Since all my questions are about your intentions and rationale:
>>
>> Reviewed-by: Markus Armbruster <armbru@redhat.com>
R-by stands, but I encourage you to polish the commit message and drop
the FIXME: from the comment in flat-union-bad-base.json.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator
2015-03-26 15:26 ` Eric Blake
@ 2015-03-27 12:32 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 12:32 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/26/2015 08:47 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Special-casing 'discriminator == {}' for handling anonymous unions
>>> is getting awkward; since this particular type is not always a
>>> dictionary on the wire, it is easier to treat it as a completely
>>> different class of type, "alternate", so that if a type is listed
>>> in the union_types array, we know it is not an anonymous union.
>>>
>>> This patch just further segregates union handling, to make sure that
>>> anonymous unions are not stored in union_types, and splitting up
>>> check_union() into separate functions. A future patch will change
>>> the qapi grammar, and having the segregation already in place will
>>> make it easier to deal with the distinct meta-type.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> ---
>
>>> @@ -535,7 +546,8 @@ def find_struct(name):
>>>
>>> def add_union(definition):
>>> global union_types
>>> - union_types.append(definition)
>>> + if definition.get('discriminator') != {}:
>>> + union_types.append(definition)
>>>
>>> def find_union(name):
>>> global union_types
>>
>> This is the only unobvious hunk.
>>
>> union_types is used only through find_union. The hunk makes
>> find_union(N) return None when N names an anonymous union.
>>
>> find_union() is used in two places:
>>
>> * find_alternate_member_qtype()
>>
>> Patched further up. It really wants only non-anonymous unions, and
>> this change to find_union() renders its check for anonymous unions
>> superfluous. Good.
>>
>> * generate_visit_alternate()
>>
>> Asserts that each member's type is either a built-in type, a complex
>> type, a union type, or an enum type.
>>
>> The change relaxes the assertion not to trigger on anonymous union
>> types. Why is that okay?
>
> No, the change tightens the assertion so that it will now fail on an
> anonymous union nested as a branch of another anonymous union (where
> before it could silently pass the assertion), because the anonymous
> union is no longer found by find_union(). And this is okay because the
> earlier change to find_alternate_member_qtype means that we don't allow
> nested anonymous unions, so making the assertion fail if an anonymous
> union gets through anyway is correct.
Thanks for unconfusing me.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests
2015-03-26 19:02 ` Eric Blake
@ 2015-03-27 12:38 ` Markus Armbruster
2015-03-27 19:39 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 12:38 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/26/2015 09:55 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Demonstrate that the qapi generator doesn't deal well with
>>> expressions that aren't up to par. Later patches will improve
>>> the expected results as the generator is made stricter. Only
>>> one of the added tests actually behaves sanely at rejecting
>>> obvious problems.
>>>
>>
>> qapi-code-gen.txt documents the naming conventions:
>>
>> Types, commands, and events share a common namespace. Therefore,
>> generally speaking, type definitions should always use CamelCase for
>> user-defined type names, while built-in types are lowercase. Type
>> definitions should not end in 'Kind', as this namespace is used for
>> creating implicit C enums for visiting union types. Command names,
>> and field names within a type, should be all lower case with words
>> separated by a hyphen. However, some existing older commands and
>> complex types use underscore; when extending such expressions,
>> consistency is preferred over blindly avoiding underscore. Event
>> names should be ALL_CAPS with words separated by underscore. The
>> special string '**' appears for some commands that manually perform
>> their own type checking rather than relying on the type-safe code
>> produced by the qapi code generators.
>>
>> We should either enforce the conventions consistently, or not at all.
>>
>> Enforcing them makes certain kinds of name clashes in generated C
>> impossible. If we don't enforce them, we should catch the clashes.
>>
>> Since I haven't read to the end of your series, I have to ask: do you
>> intend to enforce them?
>
> I added tests to enforce it for event names, but did not enforce things
> for command names or complex type members. I guess that can be added on
> top, if desired.
>
> So, did this patch get R-b?
I'd rather not enforce naming conventions just for events.
If we want to enforce them, let's do it consistently, and in a separate
series that includes this patch. Okay?
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests
2015-03-26 19:05 ` Eric Blake
@ 2015-03-27 12:40 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 12:40 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/26/2015 11:38 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> For a few QMP commands, we are forced to pass an arbitrary type
>>> without tracking it properly in QAPI. Among the existing clients,
>>> this unnamed type was spelled 'dict', 'visitor', and '**'; this
>>> patch standardizes on '**'.
>>>
>>> Meanwhile, for both 'gen' and 'success-response' keys, we have been
>>> ignoring the value, although the schema consistently used "'no'".
>>> But now that we can support a literal "false" in the schema, we
>>> might as well use that rather than ignoring the value or
>>> special-casing a random string.
>>>
>>> There is no difference to the generated code. As these features
>>> were previously undocumented before this series, add some tests
>>> and documentation on what we'd like to guarantee, although it will
>>> take later patches to clean up test results and actually enforce
>>> the use of a bool parameter.
>>
>> You don't actually add documentation in this patch.
>
> Hmm, more evidence that I waffled about per-commit doc fixes, vs.
> lumping it all in patch 1, and I obviously failed to scrub the commit
> messages after changing my mind.
Fortunately, commit messages don't need testing, so this is easy enough
to fix in a respin :)
>> Aside: 'gen': false is required when '**' is used anywhere in the
>> command. If it was permitted only then, it would be redundant. I think
>> we happily accept 'gen': false without '**' so far, although we don't
>> use it. That's okay.
>
> Also, even though the code accepts 'gen':false, it rejects 'gen':true
> ('gen' is only a one-way switch away from the default). Also something
> I didn't think worth worrying about.
Agree.
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>>
>>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (27 preceding siblings ...)
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 28/28] qapi: Drop support for inline nested types Eric Blake
@ 2015-03-27 12:50 ` Markus Armbruster
2015-03-29 16:03 ` Markus Armbruster
29 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 12:50 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> After several months of sitting on this, I finally made progress
> on it. Let's get it in 2.4, and I promise to kick out a v6
> (if needed) with much less delay.
>
> We want to eventually allow qapi defaults, by making:
> 'data':{'*flag':'bool'}
> shorthand for:
> 'data':{'flag':{'type':'bool', 'optional':true}}
> so that the default can be specified:
> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true}}
>
> This series does not quite get us there, but it DOES do a number
> of other things. It gets rid of the three uses of nested inline
> structs, changes anonymous unions to use a specific 'alternate'
> metatype rather than abusing 'union', and fixes lots of other
> parser bugs found while designing the testsuite. The testsuite
> changes make the bulk of this series, with a repeating pattern
> of writing tests that expose weaknesses in the old parser, then
> beefing up the generator to catch the problem during the initial
> parse rather than choking with an obscure python message or even
> causing a C compilation failure.
>
> v4 was here:
> https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg04022.html
>
> Changes since then: lots of refactoring; 8 new commits; lots of
> testsuite additions; address Markus' review comments where I
> could. However, so much has changed, and so much time elapsed,
> that it is probably easier to just review this series fresh than
> to try and compare it to earlier versions.
Okay, I'm through. I like it well, and I think it's really close.
There are a few questions / ideas you haven't replied to already in
PATCH 21ff. With those conversations wrapped up, I may well agree to
taking the series as is, but I think I'd prefer a *conservative* respin
addressing stuff that doesn't need new testing, like commit messages and
comments, and *selected* (by you) improvments that do need new testing.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary Eric Blake
2015-03-27 9:11 ` Markus Armbruster
@ 2015-03-27 16:19 ` Markus Armbruster
2015-03-27 20:29 ` Eric Blake
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 16:19 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> ...or an array of dictionaries. Although we have to cater to
> existing commands, returning a non-dictionary means the command
> is not extensible (no new name/value pairs can be added if more
> information must be returned in parallel). By making the
> whitelist explicit, any new command that falls foul of this
> practice will have to be self-documenting, which will encourage
> developers to either justify the action or rework the design to
> use a dictionary after all.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 30 ++++++++++++++++++++++++++++--
> tests/qapi-schema/returns-alternate.err | 1 +
> tests/qapi-schema/returns-alternate.exit | 2 +-
> tests/qapi-schema/returns-alternate.json | 2 +-
> tests/qapi-schema/returns-alternate.out | 4 ----
> tests/qapi-schema/returns-int.json | 3 ++-
> tests/qapi-schema/returns-int.out | 2 +-
> tests/qapi-schema/returns-whitelist.err | 1 +
> tests/qapi-schema/returns-whitelist.exit | 2 +-
> tests/qapi-schema/returns-whitelist.json | 2 +-
> tests/qapi-schema/returns-whitelist.out | 7 -------
> 11 files changed, 37 insertions(+), 19 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index ed5385a..9421431 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -33,6 +33,30 @@ builtin_types = {
> 'size': 'QTYPE_QINT',
> }
>
> +# Whitelist of commands allowed to return a non-dictionary
> +returns_whitelist = [
> + # From QMP:
> + 'human-monitor-command',
> + 'query-migrate-cache-size',
> + 'query-tpm-models',
> + 'query-tpm-types',
> + 'ringbuf-read',
> +
> + # From QGA:
> + 'guest-file-open',
> + 'guest-fsfreeze-freeze',
> + 'guest-fsfreeze-freeze-list',
> + 'guest-fsfreeze-status',
> + 'guest-fsfreeze-thaw',
> + 'guest-get-time',
> + 'guest-set-vcpus',
> + 'guest-sync',
> + 'guest-sync-delimited',
> +
> + # From qapi-schema-test:
> + 'user_def_cmd3',
> +]
> +
> enum_types = []
> struct_types = []
> union_types = []
> @@ -350,10 +374,12 @@ def check_command(expr, expr_info):
> check_type(expr_info, "'data' for command '%s'" % name,
> expr.get('data'),
> allowed_metas=['union', 'struct'], allow_optional=True)
> + returns_meta = ['union', 'struct']
> + if name in returns_whitelist:
> + returns_meta += ['built-in', 'alternate', 'enum']
> check_type(expr_info, "'returns' for command '%s'" % name,
> expr.get('returns'), allow_array=True,
> - allowed_metas=['built-in', 'union', 'alternate', 'struct',
> - 'enum'], allow_optional=True)
> + allowed_metas=returns_meta, allow_optional=True)
>
> def check_event(expr, expr_info):
> global events
[...]
Thinking on introspection, I started to wonder whether there's actually
a command returning a union, yet. So I applied the appended stupid
patch on top, and found the following commands returning (list of)
non-struct type:
* qapi-schema.json:
'ringbuf-read' built-in type 'str'
'human-monitor-command' built-in type 'str'
'query-migrate-cache-size' built-in type 'int'
'query-tpm-models' enum type 'TpmModel'
'query-tpm-types' enum type 'TpmType'
'query-memory-devices' union type 'MemoryDeviceInfo'
* qga/qapi-schema.json:
'guest-sync-delimited' built-in type 'int'
'guest-sync' built-in type 'int'
'guest-get-time' built-in type 'int'
'guest-file-open' built-in type 'int'
'guest-fsfreeze-status' enum type 'GuestFsfreezeStatus'
'guest-fsfreeze-freeze' built-in type 'int'
'guest-fsfreeze-freeze-list' built-in type 'int'
'guest-fsfreeze-thaw' built-in type 'int'
'guest-set-vcpus' built-in type 'int'
The sole example for union is 'MemoryDeviceInfo'. It has one case %-}
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8e5b4ad..c4c6a6e 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -353,9 +353,12 @@ def check_type(expr_info, source, data, allow_array = False,
"%s uses unknown type '%s'"
% (source, data))
if not all_names[data] in allowed_metas:
- raise QAPIExprError(expr_info,
- "%s cannot use %s type '%s'"
- % (source, all_names[data], data))
+# raise QAPIExprError(expr_info,
+# "%s cannot use %s type '%s'"
+# % (source, all_names[data], data))
+ print >>sys.stderr, QAPIExprError(expr_info,
+ "%s cannot use %s type '%s'"
+ % (source, all_names[data], data))
return
# data is a dictionary, check that each member is okay
@@ -387,6 +390,7 @@ def check_command(expr, expr_info):
returns_meta = ['union', 'struct']
if name in returns_whitelist:
returns_meta += ['built-in', 'alternate', 'enum']
+ returns_meta = ['struct']
check_type(expr_info, "'returns' for command '%s'" % name,
expr.get('returns'), allow_array=True,
allowed_metas=returns_meta, allow_optional=True,
^ permalink raw reply related [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names Eric Blake
2015-03-27 8:48 ` Markus Armbruster
@ 2015-03-27 17:14 ` Markus Armbruster
2015-03-27 20:17 ` Eric Blake
2015-03-29 9:06 ` Markus Armbruster
2 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-27 17:14 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> Previous commits demonstrated that the generator overlooked various
> bad naming situations:
> - types, commands, and events need a valid name
> - union and alternate branches cannot be marked optional
>
> The set of valid names includes [a-zA-Z0-9._-] (where '.' is
> useful only in downstream extensions).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 57 ++++++++++++++++------
> tests/qapi-schema/bad-ident.err | 1 +
> tests/qapi-schema/bad-ident.exit | 2 +-
> tests/qapi-schema/bad-ident.json | 2 +-
> tests/qapi-schema/bad-ident.out | 3 --
> tests/qapi-schema/flat-union-bad-discriminator.err | 2 +-
> .../flat-union-optional-discriminator.err | 1 +
> .../flat-union-optional-discriminator.exit | 2 +-
> .../flat-union-optional-discriminator.json | 2 +-
> .../flat-union-optional-discriminator.out | 5 --
> tests/qapi-schema/union-optional-branch.err | 1 +
> tests/qapi-schema/union-optional-branch.exit | 2 +-
> tests/qapi-schema/union-optional-branch.json | 2 +-
> tests/qapi-schema/union-optional-branch.out | 3 --
> 14 files changed, 53 insertions(+), 32 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index c42683b..ed5385a 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -15,6 +15,7 @@ import re
> from ordereddict import OrderedDict
> import os
> import sys
> +import string
>
> builtin_types = {
> 'str': 'QTYPE_QSTRING',
> @@ -276,8 +277,27 @@ def discriminator_find_enum_define(expr):
>
> return find_enum(discriminator_type)
>
> +valid_characters = set(string.ascii_letters + string.digits + '.' + '-' + '_')
> +def check_name(expr_info, source, name, allow_optional = False):
> + membername = name
> +
> + if not isinstance(name, str):
> + raise QAPIExprError(expr_info,
> + "%s requires a string name" % source)
> + if name == '**':
> + return
I'm afraid this conditional is superfluous or wrong. Our schemata don't
trigger it.
As far as I know, '**' may occur as pseudo-type in a command's 'data' or
'returns', and nowhere else.
Example 1 (qom-get):
'returns': '**'
Example 2 (qom-set):
'data': { 'path': 'str', 'property': 'str', 'value': '**' },
Example 3 (hypothetical)
'returns': { 'frob': '**' }
Both 'data' and 'returns' are checked by check_type().
Example 1 takes the early exit on data = '**' there.
Example 2 goes through this loop
for (key, value) in data.items():
check_name(expr_info, "Member of %s" % source, key,
allow_optional=allow_optional)
check_type(expr_info, "Member '%s' of %s" % (key, source), value,
allow_array=True,
allowed_metas=['built-in', 'union', 'alternate', 'struct',
'enum'],
allow_dict=True, allow_optional=True)
The '**' is fed to check_type(), *not* to check_name(). check_type()
takes the same early exit.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests
2015-03-27 12:38 ` Markus Armbruster
@ 2015-03-27 19:39 ` Eric Blake
2015-03-29 8:27 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-27 19:39 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 2650 bytes --]
On 03/27/2015 06:38 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 03/26/2015 09:55 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> Demonstrate that the qapi generator doesn't deal well with
>>>> expressions that aren't up to par. Later patches will improve
>>>> the expected results as the generator is made stricter. Only
>>>> one of the added tests actually behaves sanely at rejecting
>>>> obvious problems.
>>>>
>>>
>>> qapi-code-gen.txt documents the naming conventions:
>>>
>>> Types, commands, and events share a common namespace. Therefore,
>>> generally speaking, type definitions should always use CamelCase for
>>> user-defined type names, while built-in types are lowercase. Type
>>> definitions should not end in 'Kind', as this namespace is used for
>>> creating implicit C enums for visiting union types. Command names,
>>> and field names within a type, should be all lower case with words
>>> separated by a hyphen. However, some existing older commands and
>>> complex types use underscore; when extending such expressions,
>>> consistency is preferred over blindly avoiding underscore. Event
>>> names should be ALL_CAPS with words separated by underscore. The
>>> special string '**' appears for some commands that manually perform
>>> their own type checking rather than relying on the type-safe code
>>> produced by the qapi code generators.
>>>
>>> We should either enforce the conventions consistently, or not at all.
>>>
>>> Enforcing them makes certain kinds of name clashes in generated C
>>> impossible. If we don't enforce them, we should catch the clashes.
>>>
>>> Since I haven't read to the end of your series, I have to ask: do you
>>> intend to enforce them?
>>
>> I added tests to enforce it for event names, but did not enforce things
>> for command names or complex type members. I guess that can be added on
>> top, if desired.
>>
>> So, did this patch get R-b?
>
> I'd rather not enforce naming conventions just for events.
>
> If we want to enforce them, let's do it consistently, and in a separate
> series that includes this patch. Okay?
Sounds like I need a v6 respin then, where I drop my patch that attempts
to enforce all-caps event naming but did not enforce type or command
naming; but I will keep everything else (enforcing that names are valid
C identifiers + '-' and '.' (which both get flattened to '_').
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-27 12:30 ` Markus Armbruster
@ 2015-03-27 19:47 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-27 19:47 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 4033 bytes --]
On 03/27/2015 06:30 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 03/26/2015 07:18 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> Demonstrate that the qapi generator doesn't deal well with unions
>>>> that aren't up to par. Later patches will update the expected
>>>> reseults as the generator is made stricter.
Oh my. s/reseults/results/
Looks like I'll be doing a v6 to add R-b and clean up these nits, as it
is turning into too large a collection to ask a maintainer to do on my
behalf. I'll try and split up some of the patches where I crammed
multiple things into one commit, as part of the respin.
>>>>
>>>> Of particular note, we currently allow 'base' without 'discriminator'
>>>> as a way to create a simple union with a base class. However, none
>>>> of the existing QMP or QGA interfaces use it (it is exercised only
>>>> in the testsuite).
>>>
>>> qapi-code-gen.txt documents 'base' only for flat unions. We should
>>> either document its use in simple unions, or outlaw it.
>>
>> I went with outlaw later in the series, and the rest of my commit
>> message was an attempt to explain my reasoning in that choice. But I
>> can certainly try to improve the wording, if we need a respin.
>
> If you respin, I suggest to add that it's undocumented.
Sure.
>> I'm hoping to add as a followup series a variant of simple unions that
>> is type-safe:
>>
>> [3]
>> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> Hmm - that makes me wonder - do we support non-dict branches in a flat
>> union? The usual use case of a flat union is that all dictionary keys
>> in the branch's dictionary are treated as keys at the top level of the
>> resulting overall union; but a non-dictionary branch has no keys to
>> flatten into the top level.
>
> I think you just showed why non-dictionary branches make no sense.
>
>> I may need to tweak a subsequent patch to
>> ensure that flat union branches can only use dictionaries (while simple
>> unions can use any type).
>
> Yes, please. Follow-up patch okay, if that's easier for you.
At this point, it may indeed be easier (keep the front half of the patch
down to the bare minimum, and just make the entire series longer, rather
than trying to rebase things into perfection).
>>> What do you mean by "anonymous inline base type"?
>>
>> [5] basically, that the following example could be legal shorthand...
>>
>>>
>>>> +{ 'enum': 'TestEnum',
>>>> + 'data': [ 'value1', 'value2' ] }
>>>> +{ 'type': 'TestTypeA',
>>>> + 'data': { 'string': 'str' } }
>>>> +{ 'type': 'TestTypeB',
>>>> + 'data': { 'integer': 'int' } }
>>>> +{ 'union': 'TestUnion',
>>>> + 'base': { 'enum1': 'TestEnum', 'kind': 'str' },
>>>> + 'discriminator': 'TestEnum',
>
> Shouldn't this be 'discriminator': 'enum1'?
Yes. Good catch.
>
> Since there's nothing currently broken, I wouldn't label the question
> "should we allow an anonymous inline base type?" FIXME.
As part of my respin, I'll change any FIXME into something more benign
like TODO or RFC if it is merely intended to document a possible future
direction and not and existing bug to be corrected by the series (so
that the remaining additions of FIXME in the earlier part of the series
will all disappear by the end of the series).
>>> Since all my questions are about your intentions and rationale:
>>>
>>> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
> R-by stands, but I encourage you to polish the commit message and drop
> the FIXME: from the comment in flat-union-bad-base.json.
I think we've racked up enough to warrant a respin; I'll keep your R-b
where the only things I touch are obviously trivial, but it may mean a
few patches come through without R-b.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions
2015-03-27 7:52 ` Markus Armbruster
@ 2015-03-27 19:53 ` Eric Blake
2015-03-29 8:38 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-27 19:53 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]
On 03/27/2015 01:52 AM, Markus Armbruster wrote:
> One more...
>
> Eric Blake <eblake@redhat.com> writes:
>
> [...]
>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>> index 90eb3bc..5d0dc91 100644
>> --- a/scripts/qapi.py
>> +++ b/scripts/qapi.py
> [...]
>> @@ -560,12 +585,22 @@ def type_name(name):
>> return c_list_type(name[0])
>> return name
>>
>> -enum_types = []
>> -struct_types = []
>> -union_types = []
>> +def add_name(name, info, meta, implicit = False):
>> + global all_names
>> + if name in all_names:
>> + raise QAPIExprError(info,
>> + "%s '%s' is already defined"
>> + %(all_names[name], name))
>
> We say "struct 'Foo'", and expect the user to know that 'struct' means
> 'complex type'. It'll do, it's just a development tool.
In fact, I considered making it "type 'Foo'", to match that the item is
declared with { 'type':'Foo' ...} and not { 'struct':'Foo' ...}. But
type is an ambiguous word. I'm half tempted to do a global
search-and-replace of s/'type'/'struct'/ in the json files, since
'union' is also a type. Obviously as its own patch.
>
> I'm not really happy with 'complex type', though. Isn't a union type
> complex, too? Anyway, we can clean up our confused terminology later;
> this series is long enough.
Hmm. If I _do_ the global rename, then we have a nice hierarchy:
type - simple type: built-in, enum
- alternate
- complex type: struct, union
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types
2015-03-27 8:23 ` Markus Armbruster
@ 2015-03-27 20:03 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-27 20:03 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 2957 bytes --]
On 03/27/2015 02:23 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Now that we know every expression is valid with regards to
>> its keys, we can add further tests that those keys refer to
>> valid types. With this patch, all uses of a type (the 'data':
>> of command, type, union, alternate, and event; the 'returns':
>> of command; the 'base': of type and union) must resolve to an
>> appropriate subset of metatypes declared by the current qapi
>> parse; this includes recursing into each member of a data
>> dictionary. Dealing with '**' and nested anonymous structs
>> will be done in later patches.
>>
>> Update the testsuite to match improved output.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
> [...]
>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>> index 6ed6a34..c42683b 100644
>> --- a/scripts/qapi.py
>> +++ b/scripts/qapi.py
>> @@ -276,6 +276,63 @@ def discriminator_find_enum_define(expr):
>>
>> return find_enum(discriminator_type)
>>
>> +def check_type(expr_info, source, data, allow_array = False,
>> + allowed_metas = [], allow_dict = True):
>
> I'd allow_dict = False here, because I like defaulting to false. Matter
> of taste.
Not too hard.
>
> I'd name the third parameter def rather than data. Matter of taste
> again.
>
Oh, definitely a better name. When I first prototyped the function, I
was only crawling the 'data' member, but since I now also use it to
crawl 'returns' and 'base' it does make a difference.
>> + for (key, value) in data.items():
>> + check_type(expr_info, "Member '%s' of %s" % (key, source), value,
>
> This can produce messages like
>
> Member 'inner' of Member 'outer' of ...
>
> I figure the problem will go away when you ditch nested structs. Not
> worth worrying about it then.
Yep, it disappears quite nicely. (I was torn on whether to make the
message start with lower-case, to avoid the mid-sentence Cap, but the
prevailing practice in this file before my patches was to start all
QAPIExprError with a capital)
>>
>> +def check_struct(expr, expr_info):
>> + name = expr['type']
>> + members = expr['data']
>> +
>> + check_type(expr_info, "'data' for type '%s'" % name, members)
>
> This one gave me pause, until I realized that allowed_metas=[],
> allow_dict=True accepts exactly dictionary members, which is what we
> want.
And your idea of having allow_dict default to False in the prototype to
make it explicit here might make it easier to understand (where the
default allows nothing, and then callers add the pieces that they want).
Indeed, just typing this email, that sounds nicer than the current case
of defaults cater to the majority, but force some callers to have to
explicitly opt out.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-27 8:48 ` Markus Armbruster
@ 2015-03-27 20:15 ` Eric Blake
2015-03-29 10:17 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-27 20:15 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 4298 bytes --]
On 03/27/2015 02:48 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Previous commits demonstrated that the generator overlooked various
>> bad naming situations:
>> - types, commands, and events need a valid name
>> - union and alternate branches cannot be marked optional
>>
>> The set of valid names includes [a-zA-Z0-9._-] (where '.' is
>> useful only in downstream extensions).
>>
>> +valid_characters = set(string.ascii_letters + string.digits + '.' + '-' + '_')
>
> strings.ascii_letters depends on the locale...
https://docs.python.org/2/library/string.html
string.ascii_letters
The concatenation of the ascii_lowercase and ascii_uppercase
constants described below. This value is not locale-dependent.
You are thinking of string.letters, which IS locale-dependent. I
intentionally used ascii_letters.
>
>> +def check_name(expr_info, source, name, allow_optional = False):
>> + membername = name
>> +
>> + if not isinstance(name, str):
>> + raise QAPIExprError(expr_info,
>> + "%s requires a string name" % source)
>> + if name == '**':
>> + return
>
> Doesn't this permit '**' anywhere, not just as pseudo-type in command
> arguments and results?
Yes, on the grounds that check_type then filters it appropriately. But
worthy of a comment (probably both in the commit message AND in the code
base). Grounds for a v6 respin.
>
>> + if name.startswith('*'):
>> + membername = name[1:]
>> + if not allow_optional:
>> + raise QAPIExprError(expr_info,
>> + "%s does not allow optional name '%s'"
>> + % (source, name))
>> + if not set(membername) <= valid_characters:
>
> ... so this check would break if we called locale.setlocale() in this
> program. While I don't think we need to worry about it, I think you
> could just as well use something like
>
> valid_name = re.compile(r"^[A-Za-z0-9-._]+$")
>
> if not valid_name.match(membername):
regex is slightly slower than string matching _if the regex is
precompiled_, and MUCH slower than string matching if the regex is
compiled every time. In turn, string matching is slower than
open-coding things, but has the benefit of being more compact and
maintainable (open-coded loops are the worst on that front). Here's
where I got my inspiration:
https://stackoverflow.com/questions/1323364/in-python-how-to-check-if-a-string-only-contains-certain-characters
But I may just go with the regex after all (I don't know how efficient
python is about reusing a regex when a function is called multiple
times, rather than recompiling the regex every time. Personal side
note: back in 2009 or so, I was able to make 'm4' significantly faster
in the context of 'autoconf' when I taught it to cache the compilation
of the 8 most-recently-encountered regex, rather than recompiling every
time; and then made 'autoconf' even faster when I taught it to do
actions that didn't require regex use from 'm4' in the first place.)
The nice thing, though, is that I factored things so that the
implementation of this one function can change without having to hunt
down all call-sites, if I keep the contract the same.
>> discriminator_type = base_fields.get(discriminator)
>> if not discriminator_type:
>> raise QAPIExprError(expr_info,
>
> What happens when I try 'discriminator': '**'?
No clue. Good thing for me to add somewhere in my series. However, I
did manage to have this series at least think about a base type with
'*switch':'Enum', then use 'discriminator':'*switch', which got past the
generator (who knows what the C code would have done if have_switch was
false?), so I plugged that hole; but in the process of testing it, I
noticed that '*switch':'Enum' paired with 'discriminator':'switch'
complained that 'switch' was not a member of the base class (which is a
lie; it is present in the base class, but as an optional member). Proof
that the generator is a bunch of hacks strung together :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-27 17:14 ` Markus Armbruster
@ 2015-03-27 20:17 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-27 20:17 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]
On 03/27/2015 11:14 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Previous commits demonstrated that the generator overlooked various
>> bad naming situations:
>> - types, commands, and events need a valid name
>> - union and alternate branches cannot be marked optional
>>
>> The set of valid names includes [a-zA-Z0-9._-] (where '.' is
>> useful only in downstream extensions).
>>
>>
>> +valid_characters = set(string.ascii_letters + string.digits + '.' + '-' + '_')
>> +def check_name(expr_info, source, name, allow_optional = False):
>> + membername = name
>> +
>> + if not isinstance(name, str):
>> + raise QAPIExprError(expr_info,
>> + "%s requires a string name" % source)
>> + if name == '**':
>> + return
>
> I'm afraid this conditional is superfluous or wrong. Our schemata don't
> trigger it.
Hmm, you're right. Given a 'name':'type' pair in a dictionary,
check_name is used on the 'name' portion; but we only use 'name':'**'
(and NOT '**':'type') in the schemata. I'll drop it in the respin.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary
2015-03-27 9:11 ` Markus Armbruster
@ 2015-03-27 20:20 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-27 20:20 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]
On 03/27/2015 03:11 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> ...or an array of dictionaries. Although we have to cater to
>> existing commands, returning a non-dictionary means the command
>> is not extensible (no new name/value pairs can be added if more
>> information must be returned in parallel). By making the
>> whitelist explicit, any new command that falls foul of this
>> practice will have to be self-documenting, which will encourage
>> developers to either justify the action or rework the design to
>> use a dictionary after all.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
> Since there's just one whitelist, all schemata share it, and that means
> it's too permissive for each of them. Sloppy, but good enough.
>
> If the sloppiness bothers us, here are two alternatives:
Doesn't bother me enough to use either alternative, but I _will_ update
the commit message to at least remind someone of the potential for
tightening things, should the need arise in the future.
>
> * Program takes the whitelist as argument, say
>
> scripts/qapi-commands.py --legacy-returns qmp-legacy-returns ...
Early exit (generator fails if you don't whitelist), but noisy (have to
touch a Makefile in addition to the .json schema).
>
> * Leave enforcing to C
>
> If a command 'frobnicate' returns a non-dictionary, generate something
> like
>
> #ifndef FROBNICATE_LEGACY_RETURN_OK
> #error Command 'frobnicate' should return a dictionary
> #endif
>
> Then manually define the macros necessary to keep the current use
> working in a suitable header.
Late exit (failure does not occur at python generation time, but only at
C compilation time) - and the point of this series was to promote
several late exits into early exits. But slightly cleaner than having
to touch Makefiles to add in the whitelist.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary
2015-03-27 16:19 ` Markus Armbruster
@ 2015-03-27 20:29 ` Eric Blake
2015-03-29 10:22 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-27 20:29 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 3060 bytes --]
On 03/27/2015 10:19 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> ...or an array of dictionaries. Although we have to cater to
>> existing commands, returning a non-dictionary means the command
>> is not extensible (no new name/value pairs can be added if more
>> information must be returned in parallel). By making the
>> whitelist explicit, any new command that falls foul of this
>> practice will have to be self-documenting, which will encourage
>> developers to either justify the action or rework the design to
>> use a dictionary after all.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>
> Thinking on introspection, I started to wonder whether there's actually
> a command returning a union, yet. So I applied the appended stupid
> patch on top, and found the following commands returning (list of)
> non-struct type:
>
> * qapi-schema.json:
>
> 'ringbuf-read' built-in type 'str'
> 'human-monitor-command' built-in type 'str'
> 'query-migrate-cache-size' built-in type 'int'
> 'query-tpm-models' enum type 'TpmModel'
More precisely, "array of enum type 'TpmModel'" (or "list", depending on
whether we go with JSON array/object terminology, or QObject dict/list).
I wonder if it is worth trying to tweak the error message to more
precisely track when I strip away the [] earlier in check_type to still
report sane messages about 'array of ...' if a later check fails.
> 'query-tpm-types' enum type 'TpmType'
> 'query-memory-devices' union type 'MemoryDeviceInfo'
>
> * qga/qapi-schema.json:
>
> 'guest-sync-delimited' built-in type 'int'
> 'guest-sync' built-in type 'int'
> 'guest-get-time' built-in type 'int'
> 'guest-file-open' built-in type 'int'
> 'guest-fsfreeze-status' enum type 'GuestFsfreezeStatus'
> 'guest-fsfreeze-freeze' built-in type 'int'
> 'guest-fsfreeze-freeze-list' built-in type 'int'
> 'guest-fsfreeze-thaw' built-in type 'int'
> 'guest-set-vcpus' built-in type 'int'
Good - your patch found all of my whitelists, plus...
>
> The sole example for union is 'MemoryDeviceInfo'. It has one case %-}
Yeah, MemoryDeviceInfo as a union currently has only one type, but it
was done that way in case we add other memory devices. So it was
actually quite forward-thinking.
...one additional thing. But returning (an array of) a union should be
okay (it is a dictionary, and therefore extensible); this patch was only
about flagging non-dictionaries.
[side note: again, my idea of renaming 'type' into 'struct' in the .json
files would make it easier to talk about "complex types" as the set of
"struct" and "union" types, rather than the current confusion of
deciding if "type" means all meta-types or just struct meta-types.]
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests
2015-03-27 9:52 ` Markus Armbruster
@ 2015-03-27 20:30 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-27 20:30 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]
On 03/27/2015 03:52 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> In the testsuite, UserDefTwo and UserDefNested were identical
>> types other than the member names. Reduce code duplication by
>> having just one type, and choose names that also favor reuse.
>> This will also make it easier for a later patch to get rid of
>> inline nested types in QAPI; it means that the type is now boxed
>> instead of unboxed in C code, but has no difference to the QMP
>> wire protocol.
>
> I can't see a change from boxed to unboxed in this patch. Does the
> remark apply to the elimination of nested types? If yes, it's premature
> here.
May be a stale comment; either as the result of too much copy-and-paste
between similar patches 25 and 26, or as the result of begin first
written last September and several of your tree cleanups hitting the git
repo in the meantime. I'll clean up the commit wording in v6.
>
>> When touching code to add new allocations, also
>> convert existing allocations to consistently prefer typesafe
>> g_new0 over g_malloc0.
>
> I can't see any change from g_malloc0() to g_new0(). Is this stale?
>
>>
>> Ensure that 'make check-qapi-schema check-unit' still passes.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests
2015-03-27 19:39 ` Eric Blake
@ 2015-03-29 8:27 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 8:27 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 03/27/2015 06:38 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> On 03/26/2015 09:55 AM, Markus Armbruster wrote:
>>>> Eric Blake <eblake@redhat.com> writes:
>>>>
>>>>> Demonstrate that the qapi generator doesn't deal well with
>>>>> expressions that aren't up to par. Later patches will improve
>>>>> the expected results as the generator is made stricter. Only
>>>>> one of the added tests actually behaves sanely at rejecting
>>>>> obvious problems.
>>>>>
>>>>
>>>> qapi-code-gen.txt documents the naming conventions:
>>>>
>>>> Types, commands, and events share a common namespace. Therefore,
>>>> generally speaking, type definitions should always use CamelCase for
>>>> user-defined type names, while built-in types are lowercase. Type
>>>> definitions should not end in 'Kind', as this namespace is used for
>>>> creating implicit C enums for visiting union types. Command names,
>>>> and field names within a type, should be all lower case with words
>>>> separated by a hyphen. However, some existing older commands and
>>>> complex types use underscore; when extending such expressions,
>>>> consistency is preferred over blindly avoiding underscore. Event
>>>> names should be ALL_CAPS with words separated by underscore. The
>>>> special string '**' appears for some commands that manually perform
>>>> their own type checking rather than relying on the type-safe code
>>>> produced by the qapi code generators.
>>>>
>>>> We should either enforce the conventions consistently, or not at all.
>>>>
>>>> Enforcing them makes certain kinds of name clashes in generated C
>>>> impossible. If we don't enforce them, we should catch the clashes.
>>>>
>>>> Since I haven't read to the end of your series, I have to ask: do you
>>>> intend to enforce them?
>>>
>>> I added tests to enforce it for event names, but did not enforce things
>>> for command names or complex type members. I guess that can be added on
>>> top, if desired.
>>>
>>> So, did this patch get R-b?
>>
>> I'd rather not enforce naming conventions just for events.
>>
>> If we want to enforce them, let's do it consistently, and in a separate
>> series that includes this patch. Okay?
>
> Sounds like I need a v6 respin then, where I drop my patch that attempts
> to enforce all-caps event naming but did not enforce type or command
> naming; but I will keep everything else (enforcing that names are valid
> C identifiers + '-' and '.' (which both get flattened to '_').
Sounds good!
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions
2015-03-27 19:53 ` Eric Blake
@ 2015-03-29 8:38 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 8:38 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/27/2015 01:52 AM, Markus Armbruster wrote:
>> One more...
>>
>> Eric Blake <eblake@redhat.com> writes:
>>
>> [...]
>>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>>> index 90eb3bc..5d0dc91 100644
>>> --- a/scripts/qapi.py
>>> +++ b/scripts/qapi.py
>> [...]
>>> @@ -560,12 +585,22 @@ def type_name(name):
>>> return c_list_type(name[0])
>>> return name
>>>
>>> -enum_types = []
>>> -struct_types = []
>>> -union_types = []
>>> +def add_name(name, info, meta, implicit = False):
>>> + global all_names
>>> + if name in all_names:
>>> + raise QAPIExprError(info,
>>> + "%s '%s' is already defined"
>>> + %(all_names[name], name))
>>
>> We say "struct 'Foo'", and expect the user to know that 'struct' means
>> 'complex type'. It'll do, it's just a development tool.
>
> In fact, I considered making it "type 'Foo'", to match that the item is
> declared with { 'type':'Foo' ...} and not { 'struct':'Foo' ...}. But
> type is an ambiguous word. I'm half tempted to do a global
> search-and-replace of s/'type'/'struct'/ in the json files, since
> 'union' is also a type. Obviously as its own patch.
No objections. The churn will be a bit annoying with git-blame, but I'd
prefer that to continuing with confusing terminology.
>>
>> I'm not really happy with 'complex type', though. Isn't a union type
>> complex, too? Anyway, we can clean up our confused terminology later;
>> this series is long enough.
>
> Hmm. If I _do_ the global rename, then we have a nice hierarchy:
>
> type - simple type: built-in, enum
> - alternate
> - complex type: struct, union
Indeed.
The odd in-between-ness of alternate here is additional justification
for you separating it from union.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names Eric Blake
2015-03-27 8:48 ` Markus Armbruster
2015-03-27 17:14 ` Markus Armbruster
@ 2015-03-29 9:06 ` Markus Armbruster
2 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 9:06 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
[...]
> +valid_characters = set(string.ascii_letters + string.digits + '.' + '-' + '_')
> +def check_name(expr_info, source, name, allow_optional = False):
> + membername = name
> +
> + if not isinstance(name, str):
> + raise QAPIExprError(expr_info,
> + "%s requires a string name" % source)
> + if name == '**':
> + return
> + if name.startswith('*'):
> + membername = name[1:]
> + if not allow_optional:
> + raise QAPIExprError(expr_info,
> + "%s does not allow optional name '%s'"
> + % (source, name))
> + if not set(membername) <= valid_characters:
> + raise QAPIExprError(expr_info,
> + "%s uses invalid name '%s'" % (source, name))
> +
This accepts names starting with a digit. Sure we generate valid C
identifiers for such beauties?
qapi-code-gen.txt:
Downstream vendors may add extensions; such extensions should begin
with a prefix matching "__RFQDN_" (for the reverse-fully-qualified-
domain-name of the vendor), even if the rest of the command or field
name uses dash (example: __com.redhat_drive-mirror). Other than the
dots used in RFQDN of a downstream extension, all command, type, and
field names should begin with a letter, and contain only ASCII
letters, numbers, dash, and underscore.
One, I think "all command, type, and field names" is too narrow, what
about event names, or enumeration value names? Suggest say just "all
names".
Two, "letters, digits, dash, and underscore", please.
Three, I think check_name() should enforce "starts with letter or '_'".
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-27 20:15 ` Eric Blake
@ 2015-03-29 10:17 ` Markus Armbruster
2015-03-29 14:23 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 10:17 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/27/2015 02:48 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Previous commits demonstrated that the generator overlooked various
>>> bad naming situations:
>>> - types, commands, and events need a valid name
>>> - union and alternate branches cannot be marked optional
>>>
>>> The set of valid names includes [a-zA-Z0-9._-] (where '.' is
>>> useful only in downstream extensions).
>>>
>
>>> +valid_characters = set(string.ascii_letters + string.digits + '.'
>>> + '-' + '_')
>>
>> strings.ascii_letters depends on the locale...
>
> https://docs.python.org/2/library/string.html
>
> string.ascii_letters
>
> The concatenation of the ascii_lowercase and ascii_uppercase
> constants described below. This value is not locale-dependent.
>
> You are thinking of string.letters, which IS locale-dependent. I
> intentionally used ascii_letters.
You're right, I misread the docs.
>>> +def check_name(expr_info, source, name, allow_optional = False):
>>> + membername = name
>>> +
>>> + if not isinstance(name, str):
>>> + raise QAPIExprError(expr_info,
>>> + "%s requires a string name" % source)
>>> + if name == '**':
>>> + return
>>
>> Doesn't this permit '**' anywhere, not just as pseudo-type in command
>> arguments and results?
>
> Yes, on the grounds that check_type then filters it appropriately. But
> worthy of a comment (probably both in the commit message AND in the code
> base). Grounds for a v6 respin.
>
>>
>>> + if name.startswith('*'):
>>> + membername = name[1:]
>>> + if not allow_optional:
>>> + raise QAPIExprError(expr_info,
>>> + "%s does not allow optional name '%s'"
>>> + % (source, name))
>>> + if not set(membername) <= valid_characters:
>>
>> ... so this check would break if we called locale.setlocale() in this
>> program. While I don't think we need to worry about it, I think you
>> could just as well use something like
>>
>> valid_name = re.compile(r"^[A-Za-z0-9-._]+$")
>>
>> if not valid_name.match(membername):
>
> regex is slightly slower than string matching _if the regex is
> precompiled_, and MUCH slower than string matching if the regex is
> compiled every time. In turn, string matching is slower than
> open-coding things, but has the benefit of being more compact and
> maintainable (open-coded loops are the worst on that front). Here's
> where I got my inspiration:
>
> https://stackoverflow.com/questions/1323364/in-python-how-to-check-if-a-string-only-contains-certain-characters
>
> But I may just go with the regex after all (I don't know how efficient
> python is about reusing a regex when a function is called multiple
> times, rather than recompiling the regex every time. Personal side
> note: back in 2009 or so, I was able to make 'm4' significantly faster
> in the context of 'autoconf' when I taught it to cache the compilation
> of the 8 most-recently-encountered regex, rather than recompiling every
> time; and then made 'autoconf' even faster when I taught it to do
> actions that didn't require regex use from 'm4' in the first place.)
Neat.
> The nice thing, though, is that I factored things so that the
> implementation of this one function can change without having to hunt
> down all call-sites, if I keep the contract the same.
I don't care for matching performance here, I do care for readability.
Please pick the solution you find easiest to understand.
>>> discriminator_type = base_fields.get(discriminator)
>>> if not discriminator_type:
>>> raise QAPIExprError(expr_info,
>>
>> What happens when I try 'discriminator': '**'?
>
> No clue. Good thing for me to add somewhere in my series. However, I
I had a second look. I think the generator accepting '**' in exactly
the right places relies on:
(1) check_name() accepts only proper names, not '**'.
(2) All names get checked with check_name().
(3) Except check_type() accepts special type name '**' when allow_star.
(4) allow_star is set for exactly the right places.
With check_name()'s superfluous / incorrect '**' check gone, (1)
obviously holds. (2) isn't obvious, thus food for code review. (3) is
trivial. (4) is trivial except for "exactly the right places".
qapi-code-gen.txt:
In rare cases, QAPI cannot express a type-safe representation of a
corresponding Client JSON Protocol command. In these cases, if the
command expression includes the key 'gen' with boolean value false,
then the 'data' or 'returns' member that intends to bypass generated
type-safety and do its own manual validation should use '**' rather
than a valid type name.
Unfortunately, "the 'data' or 'returns' member that intends to bypass
[...] should use '**' rather than a valid type name" can be read in (at
least) two ways:
1. It applies to the 'data', 'returns' members of the command object.
2. It applies to members of 'data', 'returns' members of the command
object.
The schema uses both readings: qom-get has 'returns': '**', and qom-set,
netdev_add, object_add have 'data' members of the form KEY: '**'.
Note that neither code nor tests have 'data': '**' or KEY: '**' in
'returns'.
qapi.py appears to implement a third way: '**' may appear as type name
anywhere within 'data' or 'returns'. May well make sense, and may well
work, but we have no test coverage for it.
We can extend tests to cover what the generator accepts (separate series
recommended), or restrict the generator to what we actually use and
test. I'm leaning towards the latter.
Further note that '**' can only be used right in a command declaration.
Factoring out the right hand side of 'data' or 'returns' into a separate
struct type declaration isn't possible when it contains '**'.
> did manage to have this series at least think about a base type with
> '*switch':'Enum', then use 'discriminator':'*switch', which got past the
> generator (who knows what the C code would have done if have_switch was
> false?), so I plugged that hole; but in the process of testing it, I
> noticed that '*switch':'Enum' paired with 'discriminator':'switch'
> complained that 'switch' was not a member of the base class (which is a
> lie; it is present in the base class, but as an optional member). Proof
> that the generator is a bunch of hacks strung together :)
Indeed!
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary
2015-03-27 20:29 ` Eric Blake
@ 2015-03-29 10:22 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 10:22 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 03/27/2015 10:19 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> ...or an array of dictionaries. Although we have to cater to
>>> existing commands, returning a non-dictionary means the command
>>> is not extensible (no new name/value pairs can be added if more
>>> information must be returned in parallel). By making the
>>> whitelist explicit, any new command that falls foul of this
>>> practice will have to be self-documenting, which will encourage
>>> developers to either justify the action or rework the design to
>>> use a dictionary after all.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> ---
>
>
>>
>> Thinking on introspection, I started to wonder whether there's actually
>> a command returning a union, yet. So I applied the appended stupid
>> patch on top, and found the following commands returning (list of)
>> non-struct type:
>>
>> * qapi-schema.json:
>>
>> 'ringbuf-read' built-in type 'str'
>> 'human-monitor-command' built-in type 'str'
>> 'query-migrate-cache-size' built-in type 'int'
>> 'query-tpm-models' enum type 'TpmModel'
>
> More precisely, "array of enum type 'TpmModel'" (or "list", depending on
> whether we go with JSON array/object terminology, or QObject dict/list).
> I wonder if it is worth trying to tweak the error message to more
> precisely track when I strip away the [] earlier in check_type to still
> report sane messages about 'array of ...' if a later check fails.
I figure the error message is understandable enough as is. But if it
improving it would be easy, then why not. Could be done on top.
>> 'query-tpm-types' enum type 'TpmType'
>> 'query-memory-devices' union type 'MemoryDeviceInfo'
>>
>> * qga/qapi-schema.json:
>>
>> 'guest-sync-delimited' built-in type 'int'
>> 'guest-sync' built-in type 'int'
>> 'guest-get-time' built-in type 'int'
>> 'guest-file-open' built-in type 'int'
>> 'guest-fsfreeze-status' enum type 'GuestFsfreezeStatus'
>> 'guest-fsfreeze-freeze' built-in type 'int'
>> 'guest-fsfreeze-freeze-list' built-in type 'int'
>> 'guest-fsfreeze-thaw' built-in type 'int'
>> 'guest-set-vcpus' built-in type 'int'
>
> Good - your patch found all of my whitelists, plus...
>
>>
>> The sole example for union is 'MemoryDeviceInfo'. It has one case %-}
>
> Yeah, MemoryDeviceInfo as a union currently has only one type, but it
> was done that way in case we add other memory devices. So it was
> actually quite forward-thinking.
>
> ...one additional thing. But returning (an array of) a union should be
> okay (it is a dictionary, and therefore extensible); this patch was only
> about flagging non-dictionaries.
>
> [side note: again, my idea of renaming 'type' into 'struct' in the .json
> files would make it easier to talk about "complex types" as the set of
> "struct" and "union" types, rather than the current confusion of
> deciding if "type" means all meta-types or just struct meta-types.]
Yes.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names
2015-03-29 10:17 ` Markus Armbruster
@ 2015-03-29 14:23 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 14:23 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Markus Armbruster <armbru@redhat.com> writes:
[...]
> I had a second look. I think the generator accepting '**' in exactly
> the right places relies on:
>
> (1) check_name() accepts only proper names, not '**'.
>
> (2) All names get checked with check_name().
>
> (3) Except check_type() accepts special type name '**' when allow_star.
>
> (4) allow_star is set for exactly the right places.
>
> With check_name()'s superfluous / incorrect '**' check gone, (1)
> obviously holds. (2) isn't obvious, thus food for code review. (3) is
> trivial. (4) is trivial except for "exactly the right places".
> qapi-code-gen.txt:
>
> In rare cases, QAPI cannot express a type-safe representation of a
> corresponding Client JSON Protocol command. In these cases, if the
> command expression includes the key 'gen' with boolean value false,
> then the 'data' or 'returns' member that intends to bypass generated
> type-safety and do its own manual validation should use '**' rather
> than a valid type name.
>
> Unfortunately, "the 'data' or 'returns' member that intends to bypass
> [...] should use '**' rather than a valid type name" can be read in (at
> least) two ways:
>
> 1. It applies to the 'data', 'returns' members of the command object.
>
> 2. It applies to members of 'data', 'returns' members of the command
> object.
>
> The schema uses both readings: qom-get has 'returns': '**', and qom-set,
> netdev_add, object_add have 'data' members of the form KEY: '**'.
>
> Note that neither code nor tests have 'data': '**' or KEY: '**' in
> 'returns'.
Type '**' generally means "any JSON value". However, a command's 'data'
must be a JSON object (see docs/qmp/qmp-spec.txt). Ways to deal with
this:
A. Ignore. Conforming to the schema is necessary but not sufficient for
a command being accepted anyway.
B. The meaning of type '**' depends on context: it's "any JSON object"
for a command's 'data', else "any JSON value".
C. Outlaw 'data': '**' for now.
I prefer C, I can accept A, I dislike B.
> qapi.py appears to implement a third way: '**' may appear as type name
> anywhere within 'data' or 'returns'. May well make sense, and may well
> work, but we have no test coverage for it.
>
> We can extend tests to cover what the generator accepts (separate series
> recommended), or restrict the generator to what we actually use and
> test. I'm leaning towards the latter.
>
> Further note that '**' can only be used right in a command declaration.
> Factoring out the right hand side of 'data' or 'returns' into a separate
> struct type declaration isn't possible when it contains '**'.
We could treat '**' as top type rather than as hack for a few commands.
Conceptually clean, but hard to justify without users.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
` (28 preceding siblings ...)
2015-03-27 12:50 ` [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Markus Armbruster
@ 2015-03-29 16:03 ` Markus Armbruster
2015-03-31 4:30 ` Eric Blake
29 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-29 16:03 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
What happens when I define a member key multiple times in a struct or
union type?
If I do it directly, the parser rejects the duplicate key in
get_members(). Covered by tests/qapi-schema/duplicate-key.json.
What if I hide the duplicate in a base type?
If I stick this into qapi-schema-test.json:
{ 'type': 'Base', 'data': { 'foo': 'str', 'bar': 'str' } }
{ 'type': 'Clash', 'base': 'Base', 'data': { 'foo': 'int' } }
{ 'command': 'clash', 'data': 'Clash' }
the resulting test-qmp-commands.h declares qmp_clash(), but
test-qmp-marshal.c doesn't define it. WTF?!?
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-26 7:52 ` Markus Armbruster
@ 2015-03-30 15:23 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-30 15:23 UTC (permalink / raw)
To: Markus Armbruster
Cc: kwolf, famz, wenchaoqemu, Michael Roth, qemu-devel, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 2887 bytes --]
On 03/26/2015 01:52 AM, Markus Armbruster wrote:
>>>> I'm not sure if it is okay to assert GPLv2+ licensing without an
>>>> explicit Copyright, but as I am not the original author, I don't
>>>> know who to attribute any original Copyright to. Advice? Should
>>>> I split the license addition to a separate patch?
>>
>> No thoughts to this question?
>
> Missed it.
>
> I wish we didn't need to clutter copyright and licensing boiler plate
> everywhere, but I accept it's the prudent thing to do in a tree with so
> many differently licensed parts.
>
> Making GPLv2+ explicit is obviously fine, because anything without an
> explicit licensing note is GPLv2+ (see ./LICENSE). That leaves the
> copyright part, as you say.
>
> According to git-log, the file was created by Michael Roth. git-blame
> blames 219 out of 590 lines in current master on his initial commit.
>
> $ git-blame -w master docs/qapi-code-gen.txt | cut -c 11-28 | sort | uniq -c | sort -nr
Nice, but the 'cut' renders it not quite reusable when column widths
vary on other files or authorship information. 'git blame --help' suggests:
kept by the reader. The --line-porcelain option can be used to output
full commit information for each line, allowing simpler (but less
efficient) usage like:
# count the number of lines attributed to each author
git blame --line-porcelain file |
sed -n 's/^author //p' |
sort | uniq -c | sort -rn
So with the updated formula, I also checked doc/qmp/qmp-spec.txt:
201 Luiz Capitulino
55 Markus Armbruster
17 Paolo Bonzini
which is entirely Red Hat (a bit easier to handle).
>
> If you want to add a copyright note, I suggest to steal one from
> Michael's work elsewhere, and update it for later major contributors.
>
> Here's my stab at it:
>
> = How to use the QAPI code generator =
>
> Copyright IBM Corp. 2011
> Copyright (C) 2012-2015 Red Hat, Inc.
>
> Authors:
> Michael Roth <mdroth@linux.vnet.ibm.com>
> Kevin Wolf <kwolf@redhat.com>
> Markus Armbruster <armbru@redhat.com>
> Eric Blake <eblake@redhat.com>
>
> This work is licensed under the terms of the GNU GPL, version 2 or later.
> See the COPYING file in the top-level directory.
>
> == Introduction ==
>
> QAPI is a native C API within QEMU which provides management-level
> functionality to internal and external users. For external
>
> The Authors paragraph is informational and could be omitted without
> compromising the copyright note.
I prefer omitting Authors information (it is too easy to become stale).
I'll split this into a separate patch, as part of my v6.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-26 15:58 ` Markus Armbruster
@ 2015-03-30 22:45 ` Eric Blake
2015-03-31 23:40 ` Eric Blake
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-30 22:45 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 888 bytes --]
On 03/26/2015 09:58 AM, Markus Armbruster wrote:
>>> /home/armbru/work/qemu/.git/rebase-apply/patch:325: new blank line at EOF.
>>
>> Huh. I thought I had git set up to reject me from making commits like
>> that locally, but obviously not.
>
> There's another one in PATCH 13:
>
> /home/armbru/work/qemu/.git/rebase-apply/patch:156: new blank line at EOF.
>
>> http://wiki.qemu.org/Contribute/SubmitAPatch should probably mention the
>> magic one-time setup to use to turn this type of checking on...
>
> Feel free to add it :)
The wiki already mentions how:
*
[http://blog.vmsplice.net/2011/03/how-to-automatically-run-checkpatchpl.html
Automate a checkpatch run on commit]
but I had failed to 'chmod +x .git/hooks/pre-commit' to actually use it.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs
2015-03-29 16:03 ` Markus Armbruster
@ 2015-03-31 4:30 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-31 4:30 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 933 bytes --]
On 03/29/2015 10:03 AM, Markus Armbruster wrote:
> What happens when I define a member key multiple times in a struct or
> union type?
>
> If I do it directly, the parser rejects the duplicate key in
> get_members(). Covered by tests/qapi-schema/duplicate-key.json.
>
> What if I hide the duplicate in a base type?
>
> If I stick this into qapi-schema-test.json:
>
> { 'type': 'Base', 'data': { 'foo': 'str', 'bar': 'str' } }
> { 'type': 'Clash', 'base': 'Base', 'data': { 'foo': 'int' } }
> { 'command': 'clash', 'data': 'Clash' }
>
> the resulting test-qmp-commands.h declares qmp_clash(), but
> test-qmp-marshal.c doesn't define it. WTF?!?
Nice test; I'll see if I can add a patch on to the end of the series to
flag it. I suspect flat unions have potential for the same issue.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
2015-03-25 18:31 ` Markus Armbruster
@ 2015-03-31 15:09 ` Kevin Wolf
2015-03-31 17:07 ` Eric Blake
2015-04-01 15:29 ` Markus Armbruster
2 siblings, 1 reply; 124+ messages in thread
From: Kevin Wolf @ 2015-03-31 15:09 UTC (permalink / raw)
To: Eric Blake; +Cc: armbru, famz, qemu-devel, wenchaoqemu, lcapitulino
Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
> Go into more details about the various types of valid expressions
> in a qapi schema, including tweaks to document fixes being done
> later in the current patch series. Also fix some stale and missing
> documentation in the QMP specification.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> === Union types ===
>
> -Union types are used to let the user choose between several different data
> -types. A union type is defined using a dictionary as explained in the
> -following paragraphs.
> +Usage: { 'union': 'str', 'data': 'dict' }
> +or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
> + 'discriminator': 'enum-member-of-base' }
> +or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
>
> +Union types are used to let the user choose between several different
> +data types. There are three flavors: simple (no discriminator), flat
> +(a base type is mandatory, and discriminator is the name of an enum
> +field within that base type), and anonymous (discriminator is an
> +empty dictionary). A union type is defined using a data dictionary as
> +explained in the following paragraphs.
>
> -A simple union type defines a mapping from discriminator values to data types
> -like in this example:
> +A simple union type defines a mapping from automatic discriminator
> +values to data types like in this example:
>
> { 'type': 'FileOptions', 'data': { 'filename': 'str' } }
> { 'type': 'Qcow2Options',
> @@ -132,10 +273,17 @@ specified data type corresponding to the discriminator value:
> { "type": "qcow2", "data" : { "backing-file": "/some/place/my-image",
> "lazy-refcounts": true } }
>
> +Additionally, an implicit C enum NameKind is created, corresponding to
> +the union Name, for accessing the various branches of the union. No
> +branch of the union can be named 'max', as this would collide with the
> +implicit enum.
>
> -A union definition can specify a complex type as its base. In this case, the
> -fields of the complex type are included as top-level fields of the union
> -dictionary in the QMP wire format. An example definition is:
> +
> +A flat union definition specifies a complex type as its base, and
> +avoids nesting on the wire. In this case, the fields of the complex
> +type are included as top-level fields of the union dictionary in the
> +QMP wire format, and the 'discriminator' field must be the name of an
> +enum-typed member of the base type. An example definition is:
Adding full context:
{ 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
{ 'union': 'BlockdevOptions',
'base': 'BlockdevCommonOptions',
'data': { 'raw': 'RawOptions',
'qcow2': 'Qcow2Options' } }
And it looks like this on the wire:
{ "type": "qcow2",
"readonly": false,
"data" : { "backing-file": "/some/place/my-image",
"lazy-refcounts": true } }
This is not a flat union for me, but really a simple one with an added
base.
With this example treated as a flat union, I'm afraid the explanation
for the real flat unions below becomes hard to understand, because you
don't mention any more that the defining property for that case is the
existence of a 'discriminator' key, which must refer to a key in the
base type.
Somehow it seems to be expected that the reader already knows this and
you only need to explain the details. I would appreciate if you could
reintroduce the information that the old text of this section had.
> -
> -Flat union types avoid the nesting on the wire. They are used whenever a
> -specific field of the base type is declared as the discriminator ('type' is
> -then no longer generated). The discriminator must be of enumeration type.
> -The above example can then be modified as follows:
> +Notice that in a flat union, a 'type' field is no longer generated,
> +and the keys of the 'data' dictionary must match the valid values for
> +the discriminator (although not necessarily in the same order). The
> +above example for simple unions can be modified to a flat union as
> +follows:
>
> { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
> { 'type': 'BlockdevCommonOptions',
Everything else (that hasn't been mentioned in this thread yet) looks
good to me.
Kevin
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json Eric Blake
2015-03-26 17:32 ` Markus Armbruster
@ 2015-03-31 15:23 ` Kevin Wolf
2015-03-31 20:09 ` Markus Armbruster
1 sibling, 1 reply; 124+ messages in thread
From: Kevin Wolf @ 2015-03-31 15:23 UTC (permalink / raw)
To: Eric Blake; +Cc: armbru, famz, qemu-devel, wenchaoqemu, lcapitulino
Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
> From: Fam Zheng <famz@redhat.com>
>
> In the near term, we will use it for a sensible-looking
> 'gen':false inside command declarations, instead of the
> current ugly 'gen':'no'.
>
> In the long term, it will allow conversion from shorthand
> with defaults mentioned only in side-band documentation:
> 'data':{'*flag':'bool', '*string':'str'}
> into an explicit default value documentation, as in:
> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
> 'string':{'type':'str', 'optional':true, 'default':null}}
FWIW, I don't think that's a very friendly syntax for humans, it's a bit
verbose. But that's no reason not to allow true/false/null, of course.
> We still don't parse integer values (also necessary before
> we can allow explicit defaults), but that can come in a later
> series.
>
> Update the testsuite to match an improved error message.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Kevin
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-31 15:09 ` Kevin Wolf
@ 2015-03-31 17:07 ` Eric Blake
2015-03-31 17:15 ` Kevin Wolf
0 siblings, 1 reply; 124+ messages in thread
From: Eric Blake @ 2015-03-31 17:07 UTC (permalink / raw)
To: Kevin Wolf; +Cc: armbru, famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]
On 03/31/2015 09:09 AM, Kevin Wolf wrote:
> Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
>> Go into more details about the various types of valid expressions
>> in a qapi schema, including tweaks to document fixes being done
>> later in the current patch series. Also fix some stale and missing
>> documentation in the QMP specification.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> +A flat union definition specifies a complex type as its base, and
>> +avoids nesting on the wire. In this case, the fields of the complex
>> +type are included as top-level fields of the union dictionary in the
>> +QMP wire format, and the 'discriminator' field must be the name of an
>> +enum-typed member of the base type. An example definition is:
>
> Adding full context:
>
> { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
> { 'union': 'BlockdevOptions',
> 'base': 'BlockdevCommonOptions',
> 'data': { 'raw': 'RawOptions',
> 'qcow2': 'Qcow2Options' } }
Oh, yikes. I even outlawed this type of "simple union with base class"
later in the series, since no one was using it, and since it gets in the
way of my desire to move towards "simple union with type-safe enum
branching". I'll have to revisit that text and document an actual flat
union in v6.
>
> And it looks like this on the wire:
>
> { "type": "qcow2",
> "readonly": false,
> "data" : { "backing-file": "/some/place/my-image",
> "lazy-refcounts": true } }
>
> This is not a flat union for me, but really a simple one with an added
> base.
Correct - it is a bad example.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-26 15:04 ` Eric Blake
2015-03-27 12:30 ` Markus Armbruster
@ 2015-03-31 17:13 ` Kevin Wolf
2015-03-31 18:15 ` Eric Blake
2015-03-31 20:46 ` Markus Armbruster
1 sibling, 2 replies; 124+ messages in thread
From: Kevin Wolf @ 2015-03-31 17:13 UTC (permalink / raw)
To: Eric Blake; +Cc: lcapitulino, famz, Markus Armbruster, wenchaoqemu, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 5532 bytes --]
Am 26.03.2015 um 16:04 hat Eric Blake geschrieben:
> On 03/26/2015 07:18 AM, Markus Armbruster wrote:
> > Eric Blake <eblake@redhat.com> writes:
> >> Meanwhile, it would be nice to allow
> >> 'discriminator':'EnumType' without 'base' for creating a simple union
> >> that is type-safe rather than open-coded to a generated enum (right
> >> now, we are only type-safe when using a flat union, but that uses
> >> a different syntax of 'discriminator':'member-name' which requires
> >> a base class containing a 'member-name' enum field).
> >
> > I'm afraid I don't get you. Can you give examples?
>
> Using this common code with the appropriate union for each example:
> { 'command':'foo', 'data':UNION }
>
> Right now, we have flat unions which are required to be type-safe (all
> branches MUST map back to the enum type of the discriminator, enforced
> by the generator, so that if the enum later adds a member, the union
> must also be updated to match):
>
> [1]
> { 'union':'Safe', 'base':'Base', 'discriminator':'switch',
> 'data':{ 'one':'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'switch':'one', 'data':'hello'}}
>
> and simple unions which cannot be typesafe (the branches of the union
> are open-coded - even if they correlate to an existing enum, there is
> nothing enforcing that extensions to the enum be reflected into the union):
>
> [2]
> { 'union':'SimpleButOpenCoded',
> 'data':{ 'one': 'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>
> I'm hoping to add as a followup series a variant of simple unions that
> is type-safe:
>
> [3]
> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
> 'data':{ 'one':'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
This would overload 'discriminator' with two different meanings:
* In case [1] it refers to one key in the JSON object which contains the
name of the union branch to select. That is, it is the _name_ of the
key used as a discriminator.
* In case [3], the 'type' key is used in the JSON object to select a
union branch. 'discriminator' describes the _valid values_ of it, i.e.
the branch names.
We shouldn't mix these meanings. If you need [3], we could call it
'discriminator-type' or something like that. If both options are
consistently used for simple and flat unions, you end up with these
rules:
* 'discriminator' is the key that is used to select the union branch.
- For flat unions it is required and must refer to an explicitly
declared key in the base type.
- For simple unions it is optional and defaults to 'type'. The key is
implicitly created in the union type.
* 'discrimiator-type' describes the valid values of 'discriminator',
either by referring to an enum type or to 'str'.
- For flat unions, this must match the type of the explicit
declaration of the discriminator field. It is optional and defauls
to the only valid value.
- For simple unions it is optional, too, and defaults to 'str'.
- If it is the name of an enum type, that enum type is reused and the
declared union branches must match the valid values of the enum.
- If it is 'str', a new enum is generated, and all the declared union
branches are used as valid values.
There's quite some duplication in it where we need to make sure that the
schema matches in all places, but without an explicit declaration of the
disriminator field in simple unions, this seems to be the best I can
come up with.
> But the existing, unused-except-in-testsuite, notion of a simple union
> with a base class looks like:
>
> [4]
> { 'type':'Shared', 'data':{'common':'int'}}
> { 'union':'SimpleWithBase', 'base':'Shared',
> 'data':{ 'one':'str', 'two':'int' }}
>
> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
>
> If we were to allow the addition of 'discriminator':'EnumType' to a
> simple union [3], but then add that discriminator to an existing case of
> a simple union with base [4], it would look like:
>
> { 'type':'Shared', 'data':{'common':'int'}}
> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
> 'discriminator':'MyEnum',
> 'data':{ 'one':'str', 'two':'int' }}
>
> Yuck. That is indistinguishable from flat unions [1], except by whether
> discriminator names an enum type or a member of the base class.
Which could even be ambiguous, couldn't it?
> > In particular, I can define simple unions in terms of flat ones by
> > restricting all union cases to a single member named 'data'. They're
> > not implemented that way, but that's a historical accident. Simple
> > unions are a redundant concept.
>
> Cool. Or more concretely,
>
> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>
> is identical on the wire to:
>
> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
> { 'type': 'Branch1', 'data': { 'data': 'str' } }
> { 'type': 'Branch2', 'data': { 'data': 'int' } }
> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
Perhaps we should expose all unions for schema introspection in a way
similar to this (possibly not splitting out Branch1 and Branch2 as
independent types, though). We would just have to give a name to
implicitly generated enums and base types.
Kevin
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-31 17:07 ` Eric Blake
@ 2015-03-31 17:15 ` Kevin Wolf
0 siblings, 0 replies; 124+ messages in thread
From: Kevin Wolf @ 2015-03-31 17:15 UTC (permalink / raw)
To: Eric Blake; +Cc: armbru, famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 1599 bytes --]
Am 31.03.2015 um 19:07 hat Eric Blake geschrieben:
> On 03/31/2015 09:09 AM, Kevin Wolf wrote:
> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
> >> Go into more details about the various types of valid expressions
> >> in a qapi schema, including tweaks to document fixes being done
> >> later in the current patch series. Also fix some stale and missing
> >> documentation in the QMP specification.
> >>
> >> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> >> +A flat union definition specifies a complex type as its base, and
> >> +avoids nesting on the wire. In this case, the fields of the complex
> >> +type are included as top-level fields of the union dictionary in the
> >> +QMP wire format, and the 'discriminator' field must be the name of an
> >> +enum-typed member of the base type. An example definition is:
> >
> > Adding full context:
> >
> > { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
> > { 'union': 'BlockdevOptions',
> > 'base': 'BlockdevCommonOptions',
> > 'data': { 'raw': 'RawOptions',
> > 'qcow2': 'Qcow2Options' } }
>
> Oh, yikes. I even outlawed this type of "simple union with base class"
> later in the series, since no one was using it, and since it gets in the
> way of my desire to move towards "simple union with type-safe enum
> branching". I'll have to revisit that text and document an actual flat
> union in v6.
There is already an example for a real flat union. You just need to drop
this one and probably change the text around it a bit.
Kevin
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-31 17:13 ` Kevin Wolf
@ 2015-03-31 18:15 ` Eric Blake
2015-03-31 18:31 ` Eric Blake
2015-03-31 18:34 ` Kevin Wolf
2015-03-31 20:46 ` Markus Armbruster
1 sibling, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-31 18:15 UTC (permalink / raw)
To: Kevin Wolf; +Cc: lcapitulino, famz, Markus Armbruster, wenchaoqemu, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 5188 bytes --]
On 03/31/2015 11:13 AM, Kevin Wolf wrote:
>> [3]
>> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>
> This would overload 'discriminator' with two different meanings:
>
> * In case [1] it refers to one key in the JSON object which contains the
> name of the union branch to select. That is, it is the _name_ of the
> key used as a discriminator.
>
> * In case [3], the 'type' key is used in the JSON object to select a
> union branch. 'discriminator' describes the _valid values_ of it, i.e.
> the branch names.
>
> We shouldn't mix these meanings. If you need [3], we could call it
> 'discriminator-type' or something like that. If both options are
> consistently used for simple and flat unions, you end up with these
> rules:
>
Nice idea. Good thing I haven't actually implemented the extension yet.
> * 'discriminator' is the key that is used to select the union branch.
>
> - For flat unions it is required and must refer to an explicitly
> declared key in the base type.
>
> - For simple unions it is optional and defaults to 'type'. The key is
> implicitly created in the union type.
>
> * 'discrimiator-type' describes the valid values of 'discriminator',
> either by referring to an enum type or to 'str'.
Referring to 'str' should only be allowed for implicit enums. We
already require it to resolve to an enum for flat unions.
>
> - For flat unions, this must match the type of the explicit
> declaration of the discriminator field. It is optional and defauls
> to the only valid value.
>
> - For simple unions it is optional, too, and defaults to 'str'.
>
> - If it is the name of an enum type, that enum type is reused and the
> declared union branches must match the valid values of the enum.
>
> - If it is 'str', a new enum is generated, and all the declared union
> branches are used as valid values.
Sounds reasonable to me. But even with this approach, I still don't
know that we have room for a 'base' type on a simple union. I guess the
joy of future extensions is that we can make legal whatever we want, and
that this series is adding enough tests to make it obvious whether those
new additions break any existing test cases.
>
> There's quite some duplication in it where we need to make sure that the
> schema matches in all places, but without an explicit declaration of the
> disriminator field in simple unions, this seems to be the best I can
> come up with.
Yeah, it's a bit more verbose than the overloaded version, but may be
worth it.
>
>> But the existing, unused-except-in-testsuite, notion of a simple union
>> with a base class looks like:
>>
>> [4]
>> { 'type':'Shared', 'data':{'common':'int'}}
>> { 'union':'SimpleWithBase', 'base':'Shared',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
>>
>> If we were to allow the addition of 'discriminator':'EnumType' to a
>> simple union [3], but then add that discriminator to an existing case of
>> a simple union with base [4], it would look like:
>>
>> { 'type':'Shared', 'data':{'common':'int'}}
>> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
>> 'discriminator':'MyEnum',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> Yuck. That is indistinguishable from flat unions [1], except by whether
>> discriminator names an enum type or a member of the base class.
>
> Which could even be ambiguous, couldn't it?
My whole argument for why I _don't_ want to allow base types for simple
unions.
>
>>> In particular, I can define simple unions in terms of flat ones by
>>> restricting all union cases to a single member named 'data'. They're
>>> not implemented that way, but that's a historical accident. Simple
>>> unions are a redundant concept.
>>
>> Cool. Or more concretely,
>>
>> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>>
>> is identical on the wire to:
>>
>> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
>> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
>> { 'type': 'Branch1', 'data': { 'data': 'str' } }
>> { 'type': 'Branch2', 'data': { 'data': 'int' } }
>> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
>> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
>
> Perhaps we should expose all unions for schema introspection in a way
> similar to this (possibly not splitting out Branch1 and Branch2 as
> independent types, though). We would just have to give a name to
> implicitly generated enums and base types.
So maybe we update the schema to allow anonymous types. That is:
{ 'union': 'Simple',
'data': { 'one': { 'name': 'str', 'value': 'int' } } }
would be nicer than the current requirement of:
{ 'type': 'Branch1', 'data': { 'name': 'str', 'value': 'int' } }
{ 'union': 'Simple',
'data': { 'one': 'Branch1' } }
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-31 18:15 ` Eric Blake
@ 2015-03-31 18:31 ` Eric Blake
2015-03-31 18:34 ` Kevin Wolf
1 sibling, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-31 18:31 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, famz, Markus Armbruster, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 2057 bytes --]
On 03/31/2015 12:15 PM, Eric Blake wrote:
>>> Cool. Or more concretely,
>>>
>>> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>>>
>>> is identical on the wire to:
>>>
>>> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
>>> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
>>> { 'type': 'Branch1', 'data': { 'data': 'str' } }
>>> { 'type': 'Branch2', 'data': { 'data': 'int' } }
>>> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
>>> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
>>
>> Perhaps we should expose all unions for schema introspection in a way
>> similar to this (possibly not splitting out Branch1 and Branch2 as
>> independent types, though). We would just have to give a name to
>> implicitly generated enums and base types.
>
> So maybe we update the schema to allow anonymous types. That is:
>
> { 'union': 'Simple',
> 'data': { 'one': { 'name': 'str', 'value': 'int' } } }
>
> would be nicer than the current requirement of:
>
> { 'type': 'Branch1', 'data': { 'name': 'str', 'value': 'int' } }
> { 'union': 'Simple',
> 'data': { 'one': 'Branch1' } }
Hmm; maybe we could support a notion of a 'common' dictionary for simple
unions, containing all non-discriminator fields present in all branches,
where (using anonymous types for compactness), and assuming the enum
definition:
{ 'union': 'Simple',
'common': { 'readonly': 'bool' },
'data': { 'one': 'str', 'two': 'int' } }
is shorthand for:
{ 'union': 'Flat',
'base': { 'readonly': 'bool', 'type': 'Enum' },
'discriminator': 'type',
'data': { 'one': { 'data': 'str' },
'two': { 'data': 'int' } } }
that is, where 'base' for flat unions is the union of the 'common' type
and 'discriminator':'discriminator-type' member of simple unions.
At any rate, any changes along these lines will be a later series. Time
for me to get back to work on publishing v6 :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-31 18:15 ` Eric Blake
2015-03-31 18:31 ` Eric Blake
@ 2015-03-31 18:34 ` Kevin Wolf
1 sibling, 0 replies; 124+ messages in thread
From: Kevin Wolf @ 2015-03-31 18:34 UTC (permalink / raw)
To: Eric Blake; +Cc: lcapitulino, famz, Markus Armbruster, wenchaoqemu, qemu-devel
[-- Attachment #1: Type: text/plain, Size: 6296 bytes --]
Am 31.03.2015 um 20:15 hat Eric Blake geschrieben:
> On 03/31/2015 11:13 AM, Kevin Wolf wrote:
>
> >> [3]
> >> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
> >
> > This would overload 'discriminator' with two different meanings:
> >
> > * In case [1] it refers to one key in the JSON object which contains the
> > name of the union branch to select. That is, it is the _name_ of the
> > key used as a discriminator.
> >
> > * In case [3], the 'type' key is used in the JSON object to select a
> > union branch. 'discriminator' describes the _valid values_ of it, i.e.
> > the branch names.
> >
> > We shouldn't mix these meanings. If you need [3], we could call it
> > 'discriminator-type' or something like that. If both options are
> > consistently used for simple and flat unions, you end up with these
> > rules:
> >
>
> Nice idea. Good thing I haven't actually implemented the extension yet.
>
> > * 'discriminator' is the key that is used to select the union branch.
> >
> > - For flat unions it is required and must refer to an explicitly
> > declared key in the base type.
> >
> > - For simple unions it is optional and defaults to 'type'. The key is
> > implicitly created in the union type.
> >
> > * 'discrimiator-type' describes the valid values of 'discriminator',
> > either by referring to an enum type or to 'str'.
>
> Referring to 'str' should only be allowed for implicit enums. We
> already require it to resolve to an enum for flat unions.
>
> >
> > - For flat unions, this must match the type of the explicit
> > declaration of the discriminator field. It is optional and defauls
> > to the only valid value.
> >
> > - For simple unions it is optional, too, and defaults to 'str'.
> >
> > - If it is the name of an enum type, that enum type is reused and the
> > declared union branches must match the valid values of the enum.
> >
> > - If it is 'str', a new enum is generated, and all the declared union
> > branches are used as valid values.
>
> Sounds reasonable to me. But even with this approach, I still don't
> know that we have room for a 'base' type on a simple union. I guess the
> joy of future extensions is that we can make legal whatever we want, and
> that this series is adding enough tests to make it obvious whether those
> new additions break any existing test cases.
I think we have room for it, but probably still no use case. And they
are kind of ugly with mixing top-level fields for the base type and
nested ones for the branch-specific fields. (To be honest, I find all
non-flat unions ugly, but mixing is even uglier.)
So I certainly don't object to removing simple unions with base.
> > There's quite some duplication in it where we need to make sure that the
> > schema matches in all places, but without an explicit declaration of the
> > disriminator field in simple unions, this seems to be the best I can
> > come up with.
>
> Yeah, it's a bit more verbose than the overloaded version, but may be
> worth it.
>
> >
> >> But the existing, unused-except-in-testsuite, notion of a simple union
> >> with a base class looks like:
> >>
> >> [4]
> >> { 'type':'Shared', 'data':{'common':'int'}}
> >> { 'union':'SimpleWithBase', 'base':'Shared',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
> >>
> >> If we were to allow the addition of 'discriminator':'EnumType' to a
> >> simple union [3], but then add that discriminator to an existing case of
> >> a simple union with base [4], it would look like:
> >>
> >> { 'type':'Shared', 'data':{'common':'int'}}
> >> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
> >> 'discriminator':'MyEnum',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> Yuck. That is indistinguishable from flat unions [1], except by whether
> >> discriminator names an enum type or a member of the base class.
> >
> > Which could even be ambiguous, couldn't it?
>
> My whole argument for why I _don't_ want to allow base types for simple
> unions.
Essentially the problem is that we distinguish between simple and flat
unions only by looking at currently arbitrarily restricted options. So
another option to solve it would be a boolean 'flat', which we would set
as false for all existing simple unions, and let it default to true.
That would give us nicer QMP for new unions, too.
(I still don't think we need to save simple unions with base types, but
it's a thought I wanted to mention anyway because I like flat unions.)
> >>> In particular, I can define simple unions in terms of flat ones by
> >>> restricting all union cases to a single member named 'data'. They're
> >>> not implemented that way, but that's a historical accident. Simple
> >>> unions are a redundant concept.
> >>
> >> Cool. Or more concretely,
> >>
> >> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
> >>
> >> is identical on the wire to:
> >>
> >> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
> >> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
> >> { 'type': 'Branch1', 'data': { 'data': 'str' } }
> >> { 'type': 'Branch2', 'data': { 'data': 'int' } }
> >> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
> >> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
> >
> > Perhaps we should expose all unions for schema introspection in a way
> > similar to this (possibly not splitting out Branch1 and Branch2 as
> > independent types, though). We would just have to give a name to
> > implicitly generated enums and base types.
>
> So maybe we update the schema to allow anonymous types. That is:
>
> { 'union': 'Simple',
> 'data': { 'one': { 'name': 'str', 'value': 'int' } } }
>
> would be nicer than the current requirement of:
>
> { 'type': 'Branch1', 'data': { 'name': 'str', 'value': 'int' } }
> { 'union': 'Simple',
> 'data': { 'one': 'Branch1' } }
Hm... You just removed them, right? Maybe my suggestion isn't that
constructive then. :-)
Kevin
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-03-31 15:23 ` Kevin Wolf
@ 2015-03-31 20:09 ` Markus Armbruster
2015-04-01 8:31 ` Kevin Wolf
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-31 20:09 UTC (permalink / raw)
To: Kevin Wolf; +Cc: famz, lcapitulino, qemu-devel, wenchaoqemu
Kevin Wolf <kwolf@redhat.com> writes:
> Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
>> From: Fam Zheng <famz@redhat.com>
>>
>> In the near term, we will use it for a sensible-looking
>> 'gen':false inside command declarations, instead of the
>> current ugly 'gen':'no'.
>>
>> In the long term, it will allow conversion from shorthand
>> with defaults mentioned only in side-band documentation:
>> 'data':{'*flag':'bool', '*string':'str'}
>> into an explicit default value documentation, as in:
>> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
>> 'string':{'type':'str', 'optional':true, 'default':null}}
>
> FWIW, I don't think that's a very friendly syntax for humans, it's a bit
> verbose. But that's no reason not to allow true/false/null, of course.
Here's my current thinking.
Longhand:
# mandatory
'name': { 'type': 'str' }
# optional, with a default
'flag': { 'type': 'bool', 'default': true }
# optional, no default
'string': { 'type': 'str', 'default': null }
Presence of 'default' implies optional.
Equivalent shorthand, if any:
'name': 'str'
'*string': 'str'
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-31 17:13 ` Kevin Wolf
2015-03-31 18:15 ` Eric Blake
@ 2015-03-31 20:46 ` Markus Armbruster
2015-04-01 8:23 ` Kevin Wolf
1 sibling, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-03-31 20:46 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, famz, wenchaoqemu, lcapitulino
Kevin Wolf <kwolf@redhat.com> writes:
> Am 26.03.2015 um 16:04 hat Eric Blake geschrieben:
>> On 03/26/2015 07:18 AM, Markus Armbruster wrote:
>> > Eric Blake <eblake@redhat.com> writes:
>> >> Meanwhile, it would be nice to allow
>> >> 'discriminator':'EnumType' without 'base' for creating a simple union
>> >> that is type-safe rather than open-coded to a generated enum (right
>> >> now, we are only type-safe when using a flat union, but that uses
>> >> a different syntax of 'discriminator':'member-name' which requires
>> >> a base class containing a 'member-name' enum field).
>> >
>> > I'm afraid I don't get you. Can you give examples?
>>
>> Using this common code with the appropriate union for each example:
>> { 'command':'foo', 'data':UNION }
>>
>> Right now, we have flat unions which are required to be type-safe (all
>> branches MUST map back to the enum type of the discriminator, enforced
>> by the generator, so that if the enum later adds a member, the union
>> must also be updated to match):
>>
>> [1]
>> { 'union':'Safe', 'base':'Base', 'discriminator':'switch',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> {"execute":"foo", "arguments":{'switch':'one', 'data':'hello'}}
>>
>> and simple unions which cannot be typesafe (the branches of the union
>> are open-coded - even if they correlate to an existing enum, there is
>> nothing enforcing that extensions to the enum be reflected into the union):
>>
>> [2]
>> { 'union':'SimpleButOpenCoded',
>> 'data':{ 'one': 'str', 'two':'int' }}
>>
>> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>>
>> I'm hoping to add as a followup series a variant of simple unions that
>> is type-safe:
>>
>> [3]
>> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>
> This would overload 'discriminator' with two different meanings:
>
> * In case [1] it refers to one key in the JSON object which contains the
> name of the union branch to select. That is, it is the _name_ of the
> key used as a discriminator.
>
> * In case [3], the 'type' key is used in the JSON object to select a
> union branch. 'discriminator' describes the _valid values_ of it, i.e.
> the branch names.
>
> We shouldn't mix these meanings. If you need [3], we could call it
> 'discriminator-type' or something like that. If both options are
> consistently used for simple and flat unions, you end up with these
> rules:
>
> * 'discriminator' is the key that is used to select the union branch.
>
> - For flat unions it is required and must refer to an explicitly
> declared key in the base type.
>
> - For simple unions it is optional and defaults to 'type'. The key is
> implicitly created in the union type.
>
> * 'discrimiator-type' describes the valid values of 'discriminator',
> either by referring to an enum type or to 'str'.
>
> - For flat unions, this must match the type of the explicit
> declaration of the discriminator field. It is optional and defauls
> to the only valid value.
>
> - For simple unions it is optional, too, and defaults to 'str'.
>
> - If it is the name of an enum type, that enum type is reused and the
> declared union branches must match the valid values of the enum.
>
> - If it is 'str', a new enum is generated, and all the declared union
> branches are used as valid values.
>
> There's quite some duplication in it where we need to make sure that the
> schema matches in all places, but without an explicit declaration of the
> disriminator field in simple unions, this seems to be the best I can
> come up with.
>
>> But the existing, unused-except-in-testsuite, notion of a simple union
>> with a base class looks like:
>>
>> [4]
>> { 'type':'Shared', 'data':{'common':'int'}}
>> { 'union':'SimpleWithBase', 'base':'Shared',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
>>
>> If we were to allow the addition of 'discriminator':'EnumType' to a
>> simple union [3], but then add that discriminator to an existing case of
>> a simple union with base [4], it would look like:
>>
>> { 'type':'Shared', 'data':{'common':'int'}}
>> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
>> 'discriminator':'MyEnum',
>> 'data':{ 'one':'str', 'two':'int' }}
>>
>> Yuck. That is indistinguishable from flat unions [1], except by whether
>> discriminator names an enum type or a member of the base class.
>
> Which could even be ambiguous, couldn't it?
>
>> > In particular, I can define simple unions in terms of flat ones by
>> > restricting all union cases to a single member named 'data'. They're
>> > not implemented that way, but that's a historical accident. Simple
>> > unions are a redundant concept.
>>
>> Cool. Or more concretely,
>>
>> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>>
>> is identical on the wire to:
>>
>> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
>> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
>> { 'type': 'Branch1', 'data': { 'data': 'str' } }
>> { 'type': 'Branch2', 'data': { 'data': 'int' } }
>> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
>> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
>
> Perhaps we should expose all unions for schema introspection in a way
> similar to this (possibly not splitting out Branch1 and Branch2 as
> independent types, though).
My current thinking on this is a bit more radical. I suspect there's a
straightforward object type buried in this mess, struggling to get out:
the good old variant record. It consists of
* a name
* an optional base type name (must be a object type without variants)
* list of own members (empty okay)
Effective members = own members + effective base type members
* optionally variants
- one of the effective members is the discriminator (must be enum)
- for each discriminator value a list of variant members (empty okay)
All existing type/union types are specializations:
* The complex type (struct type) is an object type without variants.
* The simple union type is an object type
- without a base type
- with an implicitly defined own member of an implicitly defined
enumeration type serving as discriminator
- with no other own members
- with a single variant member for each discriminator value
* The flat union type is an object type
- with a base type
- without own members
- with a discriminator
I further suspect lowering these types to the general form will make the
generator simpler, not just the introspection.
> We would just have to give a name to
> implicitly generated enums and base types.
Introspection doesn't care whether we defined something implicitly or
explicitly. Let's make up names to hide that.
I'm trying to get a proof-of-concept introspection patch working this
week. It'll probably be ugly enough to curdle the milk in your tea.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-30 22:45 ` Eric Blake
@ 2015-03-31 23:40 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-03-31 23:40 UTC (permalink / raw)
To: Markus Armbruster
Cc: kwolf, famz, wenchaoqemu, qemu-devel, lcapitulino, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]
On 03/30/2015 04:45 PM, Eric Blake wrote:
> On 03/26/2015 09:58 AM, Markus Armbruster wrote:
>
>>>> /home/armbru/work/qemu/.git/rebase-apply/patch:325: new blank line at EOF.
>>>
>>> Huh. I thought I had git set up to reject me from making commits like
>>> that locally, but obviously not.
>>
>> There's another one in PATCH 13:
>>
>> /home/armbru/work/qemu/.git/rebase-apply/patch:156: new blank line at EOF.
>>
>>> http://wiki.qemu.org/Contribute/SubmitAPatch should probably mention the
>>> magic one-time setup to use to turn this type of checking on...
>>
>> Feel free to add it :)
>
> The wiki already mentions how:
> *
> [http://blog.vmsplice.net/2011/03/how-to-automatically-run-checkpatchpl.html
> Automate a checkpatch run on commit]
>
> but I had failed to 'chmod +x .git/hooks/pre-commit' to actually use it.
Actually, scripts/checkpatch.pl does NOT flag blank lines; so I want to
also wire up git's default .git/hooks/pre-commit checker which can flag
that type of damage (Stefan's blog and/or checkpatch.pl may need an update).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-03-31 20:46 ` Markus Armbruster
@ 2015-04-01 8:23 ` Kevin Wolf
2015-04-01 9:14 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Kevin Wolf @ 2015-04-01 8:23 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, famz, wenchaoqemu, lcapitulino
Am 31.03.2015 um 22:46 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 26.03.2015 um 16:04 hat Eric Blake geschrieben:
> >> On 03/26/2015 07:18 AM, Markus Armbruster wrote:
> >> > Eric Blake <eblake@redhat.com> writes:
> >> >> Meanwhile, it would be nice to allow
> >> >> 'discriminator':'EnumType' without 'base' for creating a simple union
> >> >> that is type-safe rather than open-coded to a generated enum (right
> >> >> now, we are only type-safe when using a flat union, but that uses
> >> >> a different syntax of 'discriminator':'member-name' which requires
> >> >> a base class containing a 'member-name' enum field).
> >> >
> >> > I'm afraid I don't get you. Can you give examples?
> >>
> >> Using this common code with the appropriate union for each example:
> >> { 'command':'foo', 'data':UNION }
> >>
> >> Right now, we have flat unions which are required to be type-safe (all
> >> branches MUST map back to the enum type of the discriminator, enforced
> >> by the generator, so that if the enum later adds a member, the union
> >> must also be updated to match):
> >>
> >> [1]
> >> { 'union':'Safe', 'base':'Base', 'discriminator':'switch',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> {"execute":"foo", "arguments":{'switch':'one', 'data':'hello'}}
> >>
> >> and simple unions which cannot be typesafe (the branches of the union
> >> are open-coded - even if they correlate to an existing enum, there is
> >> nothing enforcing that extensions to the enum be reflected into the union):
> >>
> >> [2]
> >> { 'union':'SimpleButOpenCoded',
> >> 'data':{ 'one': 'str', 'two':'int' }}
> >>
> >> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
> >>
> >> I'm hoping to add as a followup series a variant of simple unions that
> >> is type-safe:
> >>
> >> [3]
> >> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
> >
> > This would overload 'discriminator' with two different meanings:
> >
> > * In case [1] it refers to one key in the JSON object which contains the
> > name of the union branch to select. That is, it is the _name_ of the
> > key used as a discriminator.
> >
> > * In case [3], the 'type' key is used in the JSON object to select a
> > union branch. 'discriminator' describes the _valid values_ of it, i.e.
> > the branch names.
> >
> > We shouldn't mix these meanings. If you need [3], we could call it
> > 'discriminator-type' or something like that. If both options are
> > consistently used for simple and flat unions, you end up with these
> > rules:
> >
> > * 'discriminator' is the key that is used to select the union branch.
> >
> > - For flat unions it is required and must refer to an explicitly
> > declared key in the base type.
> >
> > - For simple unions it is optional and defaults to 'type'. The key is
> > implicitly created in the union type.
> >
> > * 'discrimiator-type' describes the valid values of 'discriminator',
> > either by referring to an enum type or to 'str'.
> >
> > - For flat unions, this must match the type of the explicit
> > declaration of the discriminator field. It is optional and defauls
> > to the only valid value.
> >
> > - For simple unions it is optional, too, and defaults to 'str'.
> >
> > - If it is the name of an enum type, that enum type is reused and the
> > declared union branches must match the valid values of the enum.
> >
> > - If it is 'str', a new enum is generated, and all the declared union
> > branches are used as valid values.
> >
> > There's quite some duplication in it where we need to make sure that the
> > schema matches in all places, but without an explicit declaration of the
> > disriminator field in simple unions, this seems to be the best I can
> > come up with.
> >
> >> But the existing, unused-except-in-testsuite, notion of a simple union
> >> with a base class looks like:
> >>
> >> [4]
> >> { 'type':'Shared', 'data':{'common':'int'}}
> >> { 'union':'SimpleWithBase', 'base':'Shared',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
> >>
> >> If we were to allow the addition of 'discriminator':'EnumType' to a
> >> simple union [3], but then add that discriminator to an existing case of
> >> a simple union with base [4], it would look like:
> >>
> >> { 'type':'Shared', 'data':{'common':'int'}}
> >> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
> >> 'discriminator':'MyEnum',
> >> 'data':{ 'one':'str', 'two':'int' }}
> >>
> >> Yuck. That is indistinguishable from flat unions [1], except by whether
> >> discriminator names an enum type or a member of the base class.
> >
> > Which could even be ambiguous, couldn't it?
> >
> >> > In particular, I can define simple unions in terms of flat ones by
> >> > restricting all union cases to a single member named 'data'. They're
> >> > not implemented that way, but that's a historical accident. Simple
> >> > unions are a redundant concept.
> >>
> >> Cool. Or more concretely,
> >>
> >> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
> >>
> >> is identical on the wire to:
> >>
> >> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
> >> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
> >> { 'type': 'Branch1', 'data': { 'data': 'str' } }
> >> { 'type': 'Branch2', 'data': { 'data': 'int' } }
> >> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
> >> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
> >
> > Perhaps we should expose all unions for schema introspection in a way
> > similar to this (possibly not splitting out Branch1 and Branch2 as
> > independent types, though).
>
> My current thinking on this is a bit more radical. I suspect there's a
> straightforward object type buried in this mess, struggling to get out:
> the good old variant record. It consists of
>
> * a name
>
> * an optional base type name (must be a object type without variants)
>
> * list of own members (empty okay)
>
> Effective members = own members + effective base type members
>
> * optionally variants
>
> - one of the effective members is the discriminator (must be enum)
>
> - for each discriminator value a list of variant members (empty okay)
>
> All existing type/union types are specializations:
>
> * The complex type (struct type) is an object type without variants.
>
> * The simple union type is an object type
>
> - without a base type
>
> - with an implicitly defined own member of an implicitly defined
> enumeration type serving as discriminator
>
> - with no other own members
>
> - with a single variant member for each discriminator value
>
> * The flat union type is an object type
>
> - with a base type
>
> - without own members
>
> - with a discriminator
>
> I further suspect lowering these types to the general form will make the
> generator simpler, not just the introspection.
That seems to be essentially the --verbose version of what I said above,
except that you also include simple structs. So yes, I think I agree.
Or maybe I'm missing what else you think is different?
> > We would just have to give a name to
> > implicitly generated enums and base types.
>
> Introspection doesn't care whether we defined something implicitly or
> explicitly. Let's make up names to hide that.
We just need to find a good way to make up names that stay stable and
don't cause clashes. I'm afraid that unlike block device IDs, we might
not have additional characters that could select a different namespace
here?
Not that this would be super hard, but if we need to use the same
namespace for automatically generated names, we need to properly
document which other identifiers are automatically used when you
declare a type (and detect clashes in the generator, of course).
> I'm trying to get a proof-of-concept introspection patch working this
> week. It'll probably be ugly enough to curdle the milk in your tea.
Who put that milk into my tea?! I never do that! ;-)
Kevin
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-03-31 20:09 ` Markus Armbruster
@ 2015-04-01 8:31 ` Kevin Wolf
2015-04-01 9:33 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Kevin Wolf @ 2015-04-01 8:31 UTC (permalink / raw)
To: Markus Armbruster; +Cc: famz, lcapitulino, qemu-devel, wenchaoqemu
Am 31.03.2015 um 22:09 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
> >> From: Fam Zheng <famz@redhat.com>
> >>
> >> In the near term, we will use it for a sensible-looking
> >> 'gen':false inside command declarations, instead of the
> >> current ugly 'gen':'no'.
> >>
> >> In the long term, it will allow conversion from shorthand
> >> with defaults mentioned only in side-band documentation:
> >> 'data':{'*flag':'bool', '*string':'str'}
> >> into an explicit default value documentation, as in:
> >> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
> >> 'string':{'type':'str', 'optional':true, 'default':null}}
> >
> > FWIW, I don't think that's a very friendly syntax for humans, it's a bit
> > verbose. But that's no reason not to allow true/false/null, of course.
>
> Here's my current thinking.
>
> Longhand:
>
> # mandatory
> 'name': { 'type': 'str' }
> # optional, with a default
> 'flag': { 'type': 'bool', 'default': true }
> # optional, no default
> 'string': { 'type': 'str', 'default': null }
>
> Presence of 'default' implies optional.
>
> Equivalent shorthand, if any:
>
> 'name': 'str'
> '*string': 'str'
A nice shorthand for defaults would be:
'*name': 'str' = 'default'
Though that would be neither valid JSON nor Python any more. Do we
actually rely on this property anywhere or is it only parsed by the QAPI
generator anyway and we can extend the language in such ways?
Kevin
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests
2015-04-01 8:23 ` Kevin Wolf
@ 2015-04-01 9:14 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-04-01 9:14 UTC (permalink / raw)
To: Kevin Wolf; +Cc: lcapitulino, famz, qemu-devel, wenchaoqemu
Kevin Wolf <kwolf@redhat.com> writes:
> Am 31.03.2015 um 22:46 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 26.03.2015 um 16:04 hat Eric Blake geschrieben:
>> >> On 03/26/2015 07:18 AM, Markus Armbruster wrote:
>> >> > Eric Blake <eblake@redhat.com> writes:
>> >> >> Meanwhile, it would be nice to allow
>> >> >> 'discriminator':'EnumType' without 'base' for creating a simple union
>> >> >> that is type-safe rather than open-coded to a generated enum (right
>> >> >> now, we are only type-safe when using a flat union, but that uses
>> >> >> a different syntax of 'discriminator':'member-name' which requires
>> >> >> a base class containing a 'member-name' enum field).
>> >> >
>> >> > I'm afraid I don't get you. Can you give examples?
>> >>
>> >> Using this common code with the appropriate union for each example:
>> >> { 'command':'foo', 'data':UNION }
>> >>
>> >> Right now, we have flat unions which are required to be type-safe (all
>> >> branches MUST map back to the enum type of the discriminator, enforced
>> >> by the generator, so that if the enum later adds a member, the union
>> >> must also be updated to match):
>> >>
>> >> [1]
>> >> { 'union':'Safe', 'base':'Base', 'discriminator':'switch',
>> >> 'data':{ 'one':'str', 'two':'int' }}
>> >>
>> >> {"execute":"foo", "arguments":{'switch':'one', 'data':'hello'}}
>> >>
>> >> and simple unions which cannot be typesafe (the branches of the union
>> >> are open-coded - even if they correlate to an existing enum, there is
>> >> nothing enforcing that extensions to the enum be reflected into the union):
>> >>
>> >> [2]
>> >> { 'union':'SimpleButOpenCoded',
>> >> 'data':{ 'one': 'str', 'two':'int' }}
>> >>
>> >> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>> >>
>> >> I'm hoping to add as a followup series a variant of simple unions that
>> >> is type-safe:
>> >>
>> >> [3]
>> >> { 'union':'SimpleAndSafe', 'discriminator':'MyEnum',
>> >> 'data':{ 'one':'str', 'two':'int' }}
>> >>
>> >> {"execute":"foo", "arguments":{'type':'one', 'data':'hello'}}
>> >
>> > This would overload 'discriminator' with two different meanings:
>> >
>> > * In case [1] it refers to one key in the JSON object which contains the
>> > name of the union branch to select. That is, it is the _name_ of the
>> > key used as a discriminator.
>> >
>> > * In case [3], the 'type' key is used in the JSON object to select a
>> > union branch. 'discriminator' describes the _valid values_ of it, i.e.
>> > the branch names.
>> >
>> > We shouldn't mix these meanings. If you need [3], we could call it
>> > 'discriminator-type' or something like that. If both options are
>> > consistently used for simple and flat unions, you end up with these
>> > rules:
>> >
>> > * 'discriminator' is the key that is used to select the union branch.
>> >
>> > - For flat unions it is required and must refer to an explicitly
>> > declared key in the base type.
>> >
>> > - For simple unions it is optional and defaults to 'type'. The key is
>> > implicitly created in the union type.
>> >
>> > * 'discrimiator-type' describes the valid values of 'discriminator',
>> > either by referring to an enum type or to 'str'.
>> >
>> > - For flat unions, this must match the type of the explicit
>> > declaration of the discriminator field. It is optional and defauls
>> > to the only valid value.
>> >
>> > - For simple unions it is optional, too, and defaults to 'str'.
>> >
>> > - If it is the name of an enum type, that enum type is reused and the
>> > declared union branches must match the valid values of the enum.
>> >
>> > - If it is 'str', a new enum is generated, and all the declared union
>> > branches are used as valid values.
>> >
>> > There's quite some duplication in it where we need to make sure that the
>> > schema matches in all places, but without an explicit declaration of the
>> > disriminator field in simple unions, this seems to be the best I can
>> > come up with.
>> >
>> >> But the existing, unused-except-in-testsuite, notion of a simple union
>> >> with a base class looks like:
>> >>
>> >> [4]
>> >> { 'type':'Shared', 'data':{'common':'int'}}
>> >> { 'union':'SimpleWithBase', 'base':'Shared',
>> >> 'data':{ 'one':'str', 'two':'int' }}
>> >>
>> >> {"execute":"foo", "arguments":{'common':1, 'type':'one', 'data':'hello'}}
>> >>
>> >> If we were to allow the addition of 'discriminator':'EnumType' to a
>> >> simple union [3], but then add that discriminator to an existing case of
>> >> a simple union with base [4], it would look like:
>> >>
>> >> { 'type':'Shared', 'data':{'common':'int'}}
>> >> { 'union':'SimpleWithBaseAndDiscriminator', 'base':'Shared',
>> >> 'discriminator':'MyEnum',
>> >> 'data':{ 'one':'str', 'two':'int' }}
>> >>
>> >> Yuck. That is indistinguishable from flat unions [1], except by whether
>> >> discriminator names an enum type or a member of the base class.
>> >
>> > Which could even be ambiguous, couldn't it?
>> >
>> >> > In particular, I can define simple unions in terms of flat ones by
>> >> > restricting all union cases to a single member named 'data'. They're
>> >> > not implemented that way, but that's a historical accident. Simple
>> >> > unions are a redundant concept.
>> >>
>> >> Cool. Or more concretely,
>> >>
>> >> { 'union': 'Simple', 'data': { 'one': 'str', 'two': 'int' } }
>> >>
>> >> is identical on the wire to:
>> >>
>> >> { 'enum': 'MyEnum', 'data': ['one', 'two'] }
>> >> { 'type': 'Base', 'data': { 'type': 'MyEnum' } }
>> >> { 'type': 'Branch1', 'data': { 'data': 'str' } }
>> >> { 'type': 'Branch2', 'data': { 'data': 'int' } }
>> >> { 'union': 'Flat', 'base': 'Base', 'discriminator': 'type',
>> >> 'data': { 'one': 'Branch1', 'two': 'Branch2' } }
>> >
>> > Perhaps we should expose all unions for schema introspection in a way
>> > similar to this (possibly not splitting out Branch1 and Branch2 as
>> > independent types, though).
>>
>> My current thinking on this is a bit more radical. I suspect there's a
>> straightforward object type buried in this mess, struggling to get out:
>> the good old variant record. It consists of
>>
>> * a name
>>
>> * an optional base type name (must be a object type without variants)
>>
>> * list of own members (empty okay)
>>
>> Effective members = own members + effective base type members
>>
>> * optionally variants
>>
>> - one of the effective members is the discriminator (must be enum)
>>
>> - for each discriminator value a list of variant members (empty okay)
>>
>> All existing type/union types are specializations:
>>
>> * The complex type (struct type) is an object type without variants.
>>
>> * The simple union type is an object type
>>
>> - without a base type
>>
>> - with an implicitly defined own member of an implicitly defined
>> enumeration type serving as discriminator
>>
>> - with no other own members
>>
>> - with a single variant member for each discriminator value
>>
>> * The flat union type is an object type
>>
>> - with a base type
>>
>> - without own members
>>
>> - with a discriminator
>>
>> I further suspect lowering these types to the general form will make the
>> generator simpler, not just the introspection.
>
> That seems to be essentially the --verbose version of what I said above,
> except that you also include simple structs. So yes, I think I agree.
>
> Or maybe I'm missing what else you think is different?
Lamport's quip "If you're thinking without writing, you only think
you're thinking" certainly applies to me. And then I can just as well
send it out, to give you an opportunity to point out holes or
misunderstandings.
For actually coding something, I need to think --verbose. Computers
have so far stubbornly refused all my attempts to make them DWIM %-}
>> > We would just have to give a name to
>> > implicitly generated enums and base types.
>>
>> Introspection doesn't care whether we defined something implicitly or
>> explicitly. Let's make up names to hide that.
>
> We just need to find a good way to make up names that stay stable and
> don't cause clashes. I'm afraid that unlike block device IDs, we might
> not have additional characters that could select a different namespace
> here?
>
> Not that this would be super hard, but if we need to use the same
> namespace for automatically generated names, we need to properly
> document which other identifiers are automatically used when you
> declare a type (and detect clashes in the generator, of course).
"[PATCH 21] qapi: Require valid names" makes it easy. I agree that
introspection documentation should explain automatically generated
identifiers.
>> I'm trying to get a proof-of-concept introspection patch working this
>> week. It'll probably be ugly enough to curdle the milk in your tea.
>
> Who put that milk into my tea?! I never do that! ;-)
Enjoy!
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 8:31 ` Kevin Wolf
@ 2015-04-01 9:33 ` Markus Armbruster
2015-04-01 9:58 ` Kevin Wolf
2015-04-01 12:17 ` Eric Blake
0 siblings, 2 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-04-01 9:33 UTC (permalink / raw)
To: Kevin Wolf; +Cc: famz, qemu-devel, wenchaoqemu, lcapitulino
Kevin Wolf <kwolf@redhat.com> writes:
> Am 31.03.2015 um 22:09 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
>> >> From: Fam Zheng <famz@redhat.com>
>> >>
>> >> In the near term, we will use it for a sensible-looking
>> >> 'gen':false inside command declarations, instead of the
>> >> current ugly 'gen':'no'.
>> >>
>> >> In the long term, it will allow conversion from shorthand
>> >> with defaults mentioned only in side-band documentation:
>> >> 'data':{'*flag':'bool', '*string':'str'}
>> >> into an explicit default value documentation, as in:
>> >> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
>> >> 'string':{'type':'str', 'optional':true, 'default':null}}
>> >
>> > FWIW, I don't think that's a very friendly syntax for humans, it's a bit
>> > verbose. But that's no reason not to allow true/false/null, of course.
>>
>> Here's my current thinking.
>>
>> Longhand:
>>
>> # mandatory
>> 'name': { 'type': 'str' }
>> # optional, with a default
>> 'flag': { 'type': 'bool', 'default': true }
>> # optional, no default
>> 'string': { 'type': 'str', 'default': null }
>>
>> Presence of 'default' implies optional.
>>
>> Equivalent shorthand, if any:
>>
>> 'name': 'str'
>> '*string': 'str'
>
> A nice shorthand for defaults would be:
>
> '*name': 'str' = 'default'
>
> Though that would be neither valid JSON nor Python any more. Do we
> actually rely on this property anywhere or is it only parsed by the QAPI
> generator anyway and we can extend the language in such ways?
I guess JSON / Python was chosen as QAPI schema language to save us the
bother of defining a syntax and building the tools to work with it, like
an Emacs mode. JSON's not exactly my favourite choice, but at least
it's not XML.
What we have now isn't JSON, but it's still a subset of Python, and the
Python tools work. If we go beyond Python, they'll break.
If we decide to sacrifice these tools for readability, then we can just
as well replace the syntax entirely. Preferably by something where I
don't have to put every identifier in quotes.
In short, you're welcome to hack up qapi.py some more for schema
readability, but either keep Emacs Python mode working, or provide a
replacement :)
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 9:33 ` Markus Armbruster
@ 2015-04-01 9:58 ` Kevin Wolf
2015-04-01 11:03 ` Markus Armbruster
2015-04-01 12:17 ` Eric Blake
1 sibling, 1 reply; 124+ messages in thread
From: Kevin Wolf @ 2015-04-01 9:58 UTC (permalink / raw)
To: Markus Armbruster; +Cc: famz, qemu-devel, wenchaoqemu, lcapitulino
Am 01.04.2015 um 11:33 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 31.03.2015 um 22:09 hat Markus Armbruster geschrieben:
> >> Kevin Wolf <kwolf@redhat.com> writes:
> >>
> >> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
> >> >> From: Fam Zheng <famz@redhat.com>
> >> >>
> >> >> In the near term, we will use it for a sensible-looking
> >> >> 'gen':false inside command declarations, instead of the
> >> >> current ugly 'gen':'no'.
> >> >>
> >> >> In the long term, it will allow conversion from shorthand
> >> >> with defaults mentioned only in side-band documentation:
> >> >> 'data':{'*flag':'bool', '*string':'str'}
> >> >> into an explicit default value documentation, as in:
> >> >> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
> >> >> 'string':{'type':'str', 'optional':true, 'default':null}}
> >> >
> >> > FWIW, I don't think that's a very friendly syntax for humans, it's a bit
> >> > verbose. But that's no reason not to allow true/false/null, of course.
> >>
> >> Here's my current thinking.
> >>
> >> Longhand:
> >>
> >> # mandatory
> >> 'name': { 'type': 'str' }
> >> # optional, with a default
> >> 'flag': { 'type': 'bool', 'default': true }
> >> # optional, no default
> >> 'string': { 'type': 'str', 'default': null }
> >>
> >> Presence of 'default' implies optional.
> >>
> >> Equivalent shorthand, if any:
> >>
> >> 'name': 'str'
> >> '*string': 'str'
> >
> > A nice shorthand for defaults would be:
> >
> > '*name': 'str' = 'default'
> >
> > Though that would be neither valid JSON nor Python any more. Do we
> > actually rely on this property anywhere or is it only parsed by the QAPI
> > generator anyway and we can extend the language in such ways?
>
> I guess JSON / Python was chosen as QAPI schema language to save us the
> bother of defining a syntax and building the tools to work with it, like
> an Emacs mode. JSON's not exactly my favourite choice, but at least
> it's not XML.
>
> What we have now isn't JSON, but it's still a subset of Python, and the
> Python tools work. If we go beyond Python, they'll break.
>
> If we decide to sacrifice these tools for readability, then we can just
> as well replace the syntax entirely. Preferably by something where I
> don't have to put every identifier in quotes.
>
> In short, you're welcome to hack up qapi.py some more for schema
> readability, but either keep Emacs Python mode working, or provide a
> replacement :)
What features does this Python mode provide that you use?
For the JSON schema, the only thing I really use from the editor is
syntax highlighting, and that shouldn't be bothered by an addition like
this (in vim it doesn't hurt anyway). I also don't use any other Python
tools, though I'm not sure that none exist that would make sense with
the schema.
So if there are practically relevant advantages in being real Python
code, then we need to consider that, of course. That's why I was asking.
But if there aren't, it's just an arbitrary restriction that could be
removed.
Kevin
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 9:58 ` Kevin Wolf
@ 2015-04-01 11:03 ` Markus Armbruster
2015-04-01 11:17 ` Kevin Wolf
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-04-01 11:03 UTC (permalink / raw)
To: Kevin Wolf; +Cc: lcapitulino, famz, qemu-devel, wenchaoqemu
Kevin Wolf <kwolf@redhat.com> writes:
> Am 01.04.2015 um 11:33 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 31.03.2015 um 22:09 hat Markus Armbruster geschrieben:
>> >> Kevin Wolf <kwolf@redhat.com> writes:
>> >>
>> >> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
>> >> >> From: Fam Zheng <famz@redhat.com>
>> >> >>
>> >> >> In the near term, we will use it for a sensible-looking
>> >> >> 'gen':false inside command declarations, instead of the
>> >> >> current ugly 'gen':'no'.
>> >> >>
>> >> >> In the long term, it will allow conversion from shorthand
>> >> >> with defaults mentioned only in side-band documentation:
>> >> >> 'data':{'*flag':'bool', '*string':'str'}
>> >> >> into an explicit default value documentation, as in:
>> >> >> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
>> >> >> 'string':{'type':'str', 'optional':true, 'default':null}}
>> >> >
>> >> > FWIW, I don't think that's a very friendly syntax for humans, it's a bit
>> >> > verbose. But that's no reason not to allow true/false/null, of course.
>> >>
>> >> Here's my current thinking.
>> >>
>> >> Longhand:
>> >>
>> >> # mandatory
>> >> 'name': { 'type': 'str' }
>> >> # optional, with a default
>> >> 'flag': { 'type': 'bool', 'default': true }
>> >> # optional, no default
>> >> 'string': { 'type': 'str', 'default': null }
>> >>
>> >> Presence of 'default' implies optional.
>> >>
>> >> Equivalent shorthand, if any:
>> >>
>> >> 'name': 'str'
>> >> '*string': 'str'
>> >
>> > A nice shorthand for defaults would be:
>> >
>> > '*name': 'str' = 'default'
>> >
>> > Though that would be neither valid JSON nor Python any more. Do we
>> > actually rely on this property anywhere or is it only parsed by the QAPI
>> > generator anyway and we can extend the language in such ways?
>>
>> I guess JSON / Python was chosen as QAPI schema language to save us the
>> bother of defining a syntax and building the tools to work with it, like
>> an Emacs mode. JSON's not exactly my favourite choice, but at least
>> it's not XML.
>>
>> What we have now isn't JSON, but it's still a subset of Python, and the
>> Python tools work. If we go beyond Python, they'll break.
>>
>> If we decide to sacrifice these tools for readability, then we can just
>> as well replace the syntax entirely. Preferably by something where I
>> don't have to put every identifier in quotes.
>>
>> In short, you're welcome to hack up qapi.py some more for schema
>> readability, but either keep Emacs Python mode working, or provide a
>> replacement :)
>
> What features does this Python mode provide that you use?
Syntax highlighting, automatic indentation, possibly more I rely on
without noticing. My fingers know, but they don't talk.
> For the JSON schema, the only thing I really use from the editor is
> syntax highlighting, and that shouldn't be bothered by an addition like
> this (in vim it doesn't hurt anyway). I also don't use any other Python
> tools, though I'm not sure that none exist that would make sense with
> the schema.
>
> So if there are practically relevant advantages in being real Python
> code, then we need to consider that, of course. That's why I was asking.
> But if there aren't, it's just an arbitrary restriction that could be
> removed.
If we decide to revise the decision to borrow existing syntax and roll
our own instead, let's 'make' 'our' 'own' 'syntax' 'not' 'suck'.
Anyway, we got bigger fish to fry right now.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 11:03 ` Markus Armbruster
@ 2015-04-01 11:17 ` Kevin Wolf
2015-04-01 14:51 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Kevin Wolf @ 2015-04-01 11:17 UTC (permalink / raw)
To: Markus Armbruster; +Cc: lcapitulino, famz, qemu-devel, wenchaoqemu
Am 01.04.2015 um 13:03 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 01.04.2015 um 11:33 hat Markus Armbruster geschrieben:
> >> Kevin Wolf <kwolf@redhat.com> writes:
> >>
> >> > Am 31.03.2015 um 22:09 hat Markus Armbruster geschrieben:
> >> >> Kevin Wolf <kwolf@redhat.com> writes:
> >> >>
> >> >> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
> >> >> >> From: Fam Zheng <famz@redhat.com>
> >> >> >>
> >> >> >> In the near term, we will use it for a sensible-looking
> >> >> >> 'gen':false inside command declarations, instead of the
> >> >> >> current ugly 'gen':'no'.
> >> >> >>
> >> >> >> In the long term, it will allow conversion from shorthand
> >> >> >> with defaults mentioned only in side-band documentation:
> >> >> >> 'data':{'*flag':'bool', '*string':'str'}
> >> >> >> into an explicit default value documentation, as in:
> >> >> >> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
> >> >> >> 'string':{'type':'str', 'optional':true, 'default':null}}
> >> >> >
> >> >> > FWIW, I don't think that's a very friendly syntax for humans, it's a bit
> >> >> > verbose. But that's no reason not to allow true/false/null, of course.
> >> >>
> >> >> Here's my current thinking.
> >> >>
> >> >> Longhand:
> >> >>
> >> >> # mandatory
> >> >> 'name': { 'type': 'str' }
> >> >> # optional, with a default
> >> >> 'flag': { 'type': 'bool', 'default': true }
> >> >> # optional, no default
> >> >> 'string': { 'type': 'str', 'default': null }
> >> >>
> >> >> Presence of 'default' implies optional.
> >> >>
> >> >> Equivalent shorthand, if any:
> >> >>
> >> >> 'name': 'str'
> >> >> '*string': 'str'
> >> >
> >> > A nice shorthand for defaults would be:
> >> >
> >> > '*name': 'str' = 'default'
> >> >
> >> > Though that would be neither valid JSON nor Python any more. Do we
> >> > actually rely on this property anywhere or is it only parsed by the QAPI
> >> > generator anyway and we can extend the language in such ways?
> >>
> >> I guess JSON / Python was chosen as QAPI schema language to save us the
> >> bother of defining a syntax and building the tools to work with it, like
> >> an Emacs mode. JSON's not exactly my favourite choice, but at least
> >> it's not XML.
> >>
> >> What we have now isn't JSON, but it's still a subset of Python, and the
> >> Python tools work. If we go beyond Python, they'll break.
> >>
> >> If we decide to sacrifice these tools for readability, then we can just
> >> as well replace the syntax entirely. Preferably by something where I
> >> don't have to put every identifier in quotes.
> >>
> >> In short, you're welcome to hack up qapi.py some more for schema
> >> readability, but either keep Emacs Python mode working, or provide a
> >> replacement :)
> >
> > What features does this Python mode provide that you use?
>
> Syntax highlighting, automatic indentation, possibly more I rely on
> without noticing. My fingers know, but they don't talk.
>
> > For the JSON schema, the only thing I really use from the editor is
> > syntax highlighting, and that shouldn't be bothered by an addition like
> > this (in vim it doesn't hurt anyway). I also don't use any other Python
> > tools, though I'm not sure that none exist that would make sense with
> > the schema.
> >
> > So if there are practically relevant advantages in being real Python
> > code, then we need to consider that, of course. That's why I was asking.
> > But if there aren't, it's just an arbitrary restriction that could be
> > removed.
>
> If we decide to revise the decision to borrow existing syntax and roll
> our own instead, let's 'make' 'our' 'own' 'syntax' 'not' 'suck'.
>
> Anyway, we got bigger fish to fry right now.
Okay, I got it. Asking me to create a completely new syntax and the
generator for it is the longhand version for "no".
Kevin
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 9:33 ` Markus Armbruster
2015-04-01 9:58 ` Kevin Wolf
@ 2015-04-01 12:17 ` Eric Blake
2015-04-01 14:55 ` Markus Armbruster
2015-04-01 15:43 ` Eric Blake
1 sibling, 2 replies; 124+ messages in thread
From: Eric Blake @ 2015-04-01 12:17 UTC (permalink / raw)
To: Markus Armbruster, Kevin Wolf; +Cc: lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 2141 bytes --]
On 04/01/2015 03:33 AM, Markus Armbruster wrote:
>>> Longhand:
>>>
>>> # mandatory
>>> 'name': { 'type': 'str' }
>>> # optional, with a default
>>> 'flag': { 'type': 'bool', 'default': true }
>>> # optional, no default
>>> 'string': { 'type': 'str', 'default': null }
>>>
>>> Presence of 'default' implies optional.
>>>
>>> Equivalent shorthand, if any:
>>>
>>> 'name': 'str'
>>> '*string': 'str'
>>
>> A nice shorthand for defaults would be:
>>
>> '*name': 'str' = 'default'
>>
>> Though that would be neither valid JSON nor Python any more. Do we
>> actually rely on this property anywhere or is it only parsed by the QAPI
>> generator anyway and we can extend the language in such ways?
>
> I guess JSON / Python was chosen as QAPI schema language to save us the
> bother of defining a syntax and building the tools to work with it, like
> an Emacs mode. JSON's not exactly my favourite choice, but at least
> it's not XML.
>
> What we have now isn't JSON, but it's still a subset of Python, and the
> Python tools work. If we go beyond Python, they'll break.
Well, we were a subset of Python, until this patch added true, false,
and null (the Python way would have been True, False, and None). We are
also similar to JSON5, http://json5.org/
Among other things, JSON5 allows trailing commas, allows unquoted keys
in a dictionary, allows single-quoted strings, and allows C-style comments.
>
> If we decide to sacrifice these tools for readability, then we can just
> as well replace the syntax entirely. Preferably by something where I
> don't have to put every identifier in quotes.
>
> In short, you're welcome to hack up qapi.py some more for schema
> readability, but either keep Emacs Python mode working, or provide a
> replacement :)
Since we're not quite python or JSON, we've already rolled our own
parser; so rewriting QAPI to use a syntax of our own choosing is not
that much of a leap. But not for this series.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 11:17 ` Kevin Wolf
@ 2015-04-01 14:51 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-04-01 14:51 UTC (permalink / raw)
To: Kevin Wolf; +Cc: famz, qemu-devel, wenchaoqemu, lcapitulino
Kevin Wolf <kwolf@redhat.com> writes:
> Am 01.04.2015 um 13:03 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 01.04.2015 um 11:33 hat Markus Armbruster geschrieben:
>> >> Kevin Wolf <kwolf@redhat.com> writes:
>> >>
>> >> > Am 31.03.2015 um 22:09 hat Markus Armbruster geschrieben:
>> >> >> Kevin Wolf <kwolf@redhat.com> writes:
>> >> >>
>> >> >> > Am 24.03.2015 um 21:03 hat Eric Blake geschrieben:
>> >> >> >> From: Fam Zheng <famz@redhat.com>
>> >> >> >>
>> >> >> >> In the near term, we will use it for a sensible-looking
>> >> >> >> 'gen':false inside command declarations, instead of the
>> >> >> >> current ugly 'gen':'no'.
>> >> >> >>
>> >> >> >> In the long term, it will allow conversion from shorthand
>> >> >> >> with defaults mentioned only in side-band documentation:
>> >> >> >> 'data':{'*flag':'bool', '*string':'str'}
>> >> >> >> into an explicit default value documentation, as in:
>> >> >> >> 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
>> >> >> >> 'string':{'type':'str', 'optional':true, 'default':null}}
>> >> >> >
>> >> >> > FWIW, I don't think that's a very friendly syntax for
>> >> >> > humans, it's a bit
>> >> >> > verbose. But that's no reason not to allow true/false/null, of course.
>> >> >>
>> >> >> Here's my current thinking.
>> >> >>
>> >> >> Longhand:
>> >> >>
>> >> >> # mandatory
>> >> >> 'name': { 'type': 'str' }
>> >> >> # optional, with a default
>> >> >> 'flag': { 'type': 'bool', 'default': true }
>> >> >> # optional, no default
>> >> >> 'string': { 'type': 'str', 'default': null }
>> >> >>
>> >> >> Presence of 'default' implies optional.
>> >> >>
>> >> >> Equivalent shorthand, if any:
>> >> >>
>> >> >> 'name': 'str'
>> >> >> '*string': 'str'
>> >> >
>> >> > A nice shorthand for defaults would be:
>> >> >
>> >> > '*name': 'str' = 'default'
>> >> >
>> >> > Though that would be neither valid JSON nor Python any more. Do we
>> >> > actually rely on this property anywhere or is it only parsed by the QAPI
>> >> > generator anyway and we can extend the language in such ways?
>> >>
>> >> I guess JSON / Python was chosen as QAPI schema language to save us the
>> >> bother of defining a syntax and building the tools to work with it, like
>> >> an Emacs mode. JSON's not exactly my favourite choice, but at least
>> >> it's not XML.
>> >>
>> >> What we have now isn't JSON, but it's still a subset of Python, and the
>> >> Python tools work. If we go beyond Python, they'll break.
>> >>
>> >> If we decide to sacrifice these tools for readability, then we can just
>> >> as well replace the syntax entirely. Preferably by something where I
>> >> don't have to put every identifier in quotes.
>> >>
>> >> In short, you're welcome to hack up qapi.py some more for schema
>> >> readability, but either keep Emacs Python mode working, or provide a
>> >> replacement :)
>> >
>> > What features does this Python mode provide that you use?
>>
>> Syntax highlighting, automatic indentation, possibly more I rely on
>> without noticing. My fingers know, but they don't talk.
>>
>> > For the JSON schema, the only thing I really use from the editor is
>> > syntax highlighting, and that shouldn't be bothered by an addition like
>> > this (in vim it doesn't hurt anyway). I also don't use any other Python
>> > tools, though I'm not sure that none exist that would make sense with
>> > the schema.
>> >
>> > So if there are practically relevant advantages in being real Python
>> > code, then we need to consider that, of course. That's why I was asking.
>> > But if there aren't, it's just an arbitrary restriction that could be
>> > removed.
>>
>> If we decide to revise the decision to borrow existing syntax and roll
>> our own instead, let's 'make' 'our' 'own' 'syntax' 'not' 'suck'.
>>
>> Anyway, we got bigger fish to fry right now.
>
> Okay, I got it. Asking me to create a completely new syntax and the
> generator for it is the longhand version for "no".
It's advice, not a proactive NAK.
Anyway, let's get Eric's series in, and crack the introspection problem
before we start even more QAPI generator projects.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 12:17 ` Eric Blake
@ 2015-04-01 14:55 ` Markus Armbruster
2015-04-01 15:43 ` Eric Blake
1 sibling, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2015-04-01 14:55 UTC (permalink / raw)
To: Eric Blake; +Cc: Kevin Wolf, famz, qemu-devel, wenchaoqemu, lcapitulino
Eric Blake <eblake@redhat.com> writes:
> On 04/01/2015 03:33 AM, Markus Armbruster wrote:
>
>>>> Longhand:
>>>>
>>>> # mandatory
>>>> 'name': { 'type': 'str' }
>>>> # optional, with a default
>>>> 'flag': { 'type': 'bool', 'default': true }
>>>> # optional, no default
>>>> 'string': { 'type': 'str', 'default': null }
>>>>
>>>> Presence of 'default' implies optional.
>>>>
>>>> Equivalent shorthand, if any:
>>>>
>>>> 'name': 'str'
>>>> '*string': 'str'
>>>
>>> A nice shorthand for defaults would be:
>>>
>>> '*name': 'str' = 'default'
>>>
>>> Though that would be neither valid JSON nor Python any more. Do we
>>> actually rely on this property anywhere or is it only parsed by the QAPI
>>> generator anyway and we can extend the language in such ways?
>>
>> I guess JSON / Python was chosen as QAPI schema language to save us the
>> bother of defining a syntax and building the tools to work with it, like
>> an Emacs mode. JSON's not exactly my favourite choice, but at least
>> it's not XML.
>>
>> What we have now isn't JSON, but it's still a subset of Python, and the
>> Python tools work. If we go beyond Python, they'll break.
>
> Well, we were a subset of Python, until this patch added true, false,
> and null (the Python way would have been True, False, and None). We are
> also similar to JSON5, http://json5.org/
>
> Among other things, JSON5 allows trailing commas, allows unquoted keys
> in a dictionary, allows single-quoted strings, and allows C-style comments.
I'm open to consider adopting it once the dust settles.
>> If we decide to sacrifice these tools for readability, then we can just
>> as well replace the syntax entirely. Preferably by something where I
>> don't have to put every identifier in quotes.
>>
>> In short, you're welcome to hack up qapi.py some more for schema
>> readability, but either keep Emacs Python mode working, or provide a
>> replacement :)
>
> Since we're not quite python or JSON, we've already rolled our own
> parser; so rewriting QAPI to use a syntax of our own choosing is not
> that much of a leap.
Point taken.
> But not for this series.
Certainly.
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
2015-03-25 18:31 ` Markus Armbruster
2015-03-31 15:09 ` Kevin Wolf
@ 2015-04-01 15:29 ` Markus Armbruster
2015-04-01 15:36 ` Eric Blake
2 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2015-04-01 15:29 UTC (permalink / raw)
To: Eric Blake; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
[...]
> === Union types ===
>
> -Union types are used to let the user choose between several different data
> -types. A union type is defined using a dictionary as explained in the
> -following paragraphs.
> +Usage: { 'union': 'str', 'data': 'dict' }
> +or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
> + 'discriminator': 'enum-member-of-base' }
> +or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
>
> +Union types are used to let the user choose between several different
> +data types. There are three flavors: simple (no discriminator), flat
> +(a base type is mandatory, and discriminator is the name of an enum
> +field within that base type), and anonymous (discriminator is an
> +empty dictionary). A union type is defined using a data dictionary as
> +explained in the following paragraphs.
>
> -A simple union type defines a mapping from discriminator values to data types
> -like in this example:
> +A simple union type defines a mapping from automatic discriminator
> +values to data types like in this example:
>
> { 'type': 'FileOptions', 'data': { 'filename': 'str' } }
> { 'type': 'Qcow2Options',
> @@ -132,10 +273,17 @@ specified data type corresponding to the discriminator value:
> { "type": "qcow2", "data" : { "backing-file": "/some/place/my-image",
> "lazy-refcounts": true } }
>
> +Additionally, an implicit C enum NameKind is created, corresponding to
> +the union Name, for accessing the various branches of the union. No
> +branch of the union can be named 'max', as this would collide with the
> +implicit enum.
>
> -A union definition can specify a complex type as its base. In this case, the
> -fields of the complex type are included as top-level fields of the union
> -dictionary in the QMP wire format. An example definition is:
> +
> +A flat union definition specifies a complex type as its base, and
> +avoids nesting on the wire. In this case, the fields of the complex
> +type are included as top-level fields of the union dictionary in the
> +QMP wire format, and the 'discriminator' field must be the name of an
> +enum-typed member of the base type. An example definition is:
>
> { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
> { 'union': 'BlockdevOptions',
'base': 'BlockdevCommonOptions',
'data': { 'raw': 'RawOptions',
'qcow2': 'Qcow2Options' } }
Where's 'discriminator'?
And it looks like this on the wire:
{ "type": "qcow2",
"readonly": false,
> "data" : { "backing-file": "/some/place/my-image",
> "lazy-refcounts": true } }
Is the variant part really wrapped in "data"?
Here's the real schema's flat union BlockdevOptions:
{ 'union': 'BlockdevOptions',
'base': 'BlockdevOptionsBase',
'discriminator': 'driver',
'data': {
'qcow2': 'BlockdevOptionsQcow2',
'raw': 'BlockdevOptionsGenericFormat',
[many more omitted...]
} }
Same thing, just different names.
One the wire, it looks like
{ "driver": "qcow2",
"id": "disk1",
"file": { "driver": "file", "filename": "tmp.qcow2" } }
>
> -
> -Flat union types avoid the nesting on the wire. They are used whenever a
> -specific field of the base type is declared as the discriminator ('type' is
> -then no longer generated). The discriminator must be of enumeration type.
> -The above example can then be modified as follows:
> +Notice that in a flat union, a 'type' field is no longer generated,
> +and the keys of the 'data' dictionary must match the valid values for
> +the discriminator (although not necessarily in the same order). The
> +above example for simple unions can be modified to a flat union as
> +follows:
>
> { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
> { 'type': 'BlockdevCommonOptions',
'data': { 'driver': 'BlockdevDriver', 'readonly': 'bool' } }
{ 'union': 'BlockdevOptions',
'base': 'BlockdevCommonOptions',
'discriminator': 'driver',
'data': { 'raw': 'RawOptions',
'qcow2': 'Qcow2Options' } }
Resulting in this JSON object:
{ "driver": "qcow2",
"readonly": false,
"backing-file": "/some/place/my-image",
"lazy-refcounts": true }
This actually matches the above example with the "data" wrapper peeled
off the variant part.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations
2015-04-01 15:29 ` Markus Armbruster
@ 2015-04-01 15:36 ` Eric Blake
0 siblings, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-04-01 15:36 UTC (permalink / raw)
To: Markus Armbruster; +Cc: kwolf, lcapitulino, famz, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]
On 04/01/2015 09:29 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
> [...]
>> === Union types ===
>>
>> +
>> +A flat union definition specifies a complex type as its base, and
>> +avoids nesting on the wire. In this case, the fields of the complex
>> +type are included as top-level fields of the union dictionary in the
>> +QMP wire format, and the 'discriminator' field must be the name of an
>> +enum-typed member of the base type. An example definition is:
>>
>> { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
>> { 'union': 'BlockdevOptions',
> 'base': 'BlockdevCommonOptions',
> 'data': { 'raw': 'RawOptions',
> 'qcow2': 'Qcow2Options' } }
>
> Where's 'discriminator'?
Kevin already flagged this. I forgot to remove simple-with-base
documentation in v5; it's already gone from my working copy of v6.
> Resulting in this JSON object:
>
> { "driver": "qcow2",
> "readonly": false,
> "backing-file": "/some/place/my-image",
> "lazy-refcounts": true }
>
> This actually matches the above example with the "data" wrapper peeled
> off the variant part.
Yep - with the bad intermediate stuff gone, the remaining stuff makes sense.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json
2015-04-01 12:17 ` Eric Blake
2015-04-01 14:55 ` Markus Armbruster
@ 2015-04-01 15:43 ` Eric Blake
1 sibling, 0 replies; 124+ messages in thread
From: Eric Blake @ 2015-04-01 15:43 UTC (permalink / raw)
To: Markus Armbruster, Kevin Wolf; +Cc: famz, qemu-devel, wenchaoqemu, lcapitulino
[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]
On 04/01/2015 06:17 AM, Eric Blake wrote:
>> I guess JSON / Python was chosen as QAPI schema language to save us the
>> bother of defining a syntax and building the tools to work with it, like
>> an Emacs mode. JSON's not exactly my favourite choice, but at least
>> it's not XML.
>>
>> What we have now isn't JSON, but it's still a subset of Python, and the
>> Python tools work. If we go beyond Python, they'll break.
>
> Well, we were a subset of Python, until this patch added true, false,
> and null (the Python way would have been True, False, and None). We are
> also similar to JSON5, http://json5.org/
>
> Among other things, JSON5 allows trailing commas, allows unquoted keys
> in a dictionary, allows single-quoted strings, and allows C-style comments.
Another thing I just noticed: JSON allows '\u0061' as a synonym for the
one-byte string 'a', but our parser does not (instead, our parser treats
it like the five bytes 'u0061'). I guess we haven't noticed, since qapi
has never needed non-ascii names...
> Since we're not quite python or JSON, we've already rolled our own
> parser; so rewriting QAPI to use a syntax of our own choosing is not
> that much of a leap. But not for this series.
So of course this still holds, and I won't stall my v6 posting because
of it.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 124+ messages in thread
end of thread, other threads:[~2015-04-01 15:43 UTC | newest]
Thread overview: 124+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-24 20:03 [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 01/28] qapi: Document type-safety considerations Eric Blake
2015-03-25 18:31 ` Markus Armbruster
2015-03-25 20:11 ` Eric Blake
2015-03-25 21:15 ` Eric Blake
2015-03-26 9:09 ` Markus Armbruster
2015-03-26 7:52 ` Markus Armbruster
2015-03-30 15:23 ` Eric Blake
2015-03-26 8:09 ` Markus Armbruster
2015-03-31 15:09 ` Kevin Wolf
2015-03-31 17:07 ` Eric Blake
2015-03-31 17:15 ` Kevin Wolf
2015-04-01 15:29 ` Markus Armbruster
2015-04-01 15:36 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 02/28] qapi: Fix generation of 'size' builtin type Eric Blake
2015-03-26 9:52 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 03/28] qapi: Require ASCII in schema Eric Blake
2015-03-24 20:33 ` Eric Blake
2015-03-26 9:54 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 04/28] qapi: Add some enum tests Eric Blake
2015-03-26 10:01 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 05/28] qapi: Better error messages for bad enums Eric Blake
2015-03-26 10:08 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 06/28] qapi: Add some union tests Eric Blake
2015-03-26 13:18 ` Markus Armbruster
2015-03-26 15:04 ` Eric Blake
2015-03-27 12:30 ` Markus Armbruster
2015-03-27 19:47 ` Eric Blake
2015-03-31 17:13 ` Kevin Wolf
2015-03-31 18:15 ` Eric Blake
2015-03-31 18:31 ` Eric Blake
2015-03-31 18:34 ` Kevin Wolf
2015-03-31 20:46 ` Markus Armbruster
2015-04-01 8:23 ` Kevin Wolf
2015-04-01 9:14 ` Markus Armbruster
2015-03-26 13:23 ` Markus Armbruster
2015-03-26 13:51 ` Eric Blake
2015-03-26 15:58 ` Markus Armbruster
2015-03-30 22:45 ` Eric Blake
2015-03-31 23:40 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 07/28] qapi: Simplify tests of simple unions Eric Blake
2015-03-26 13:41 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 08/28] qapi: Better error messages for bad unions Eric Blake
2015-03-24 20:38 ` Eric Blake
2015-03-26 14:20 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 09/28] qapi: Prepare for catching more semantic parse errors Eric Blake
2015-03-26 14:22 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 10/28] qapi: Segregate anonymous unions into alternates in generator Eric Blake
2015-03-26 14:47 ` Markus Armbruster
2015-03-26 15:26 ` Eric Blake
2015-03-27 12:32 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 11/28] qapi: Rename anonymous union type in test Eric Blake
2015-03-26 14:55 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 12/28] qapi: Introduce 'alternate' to replace anonymous union Eric Blake
2015-03-24 20:41 ` Eric Blake
2015-03-26 15:42 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 13/28] qapi: Add some expr tests Eric Blake
2015-03-26 15:55 ` Markus Armbruster
2015-03-26 19:02 ` Eric Blake
2015-03-27 12:38 ` Markus Armbruster
2015-03-27 19:39 ` Eric Blake
2015-03-29 8:27 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 14/28] qapi: Better error messages for bad expressions Eric Blake
2015-03-26 16:27 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 15/28] qapi: Add tests of redefined expressions Eric Blake
2015-03-26 17:05 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 16/28] qapi: Better error messages for duplicated expressions Eric Blake
2015-03-26 17:21 ` Markus Armbruster
2015-03-27 7:52 ` Markus Armbruster
2015-03-27 19:53 ` Eric Blake
2015-03-29 8:38 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 17/28] qapi: Allow true, false and null in schema json Eric Blake
2015-03-26 17:32 ` Markus Armbruster
2015-03-31 15:23 ` Kevin Wolf
2015-03-31 20:09 ` Markus Armbruster
2015-04-01 8:31 ` Kevin Wolf
2015-04-01 9:33 ` Markus Armbruster
2015-04-01 9:58 ` Kevin Wolf
2015-04-01 11:03 ` Markus Armbruster
2015-04-01 11:17 ` Kevin Wolf
2015-04-01 14:51 ` Markus Armbruster
2015-04-01 12:17 ` Eric Blake
2015-04-01 14:55 ` Markus Armbruster
2015-04-01 15:43 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 18/28] qapi: Unify type bypass and add tests Eric Blake
2015-03-26 17:38 ` Markus Armbruster
2015-03-26 19:05 ` Eric Blake
2015-03-27 12:40 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 19/28] qapi: Add some type check tests Eric Blake
2015-03-26 17:58 ` Markus Armbruster
2015-03-26 19:07 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 20/28] qapi: More rigourous checking of types Eric Blake
2015-03-27 8:23 ` Markus Armbruster
2015-03-27 20:03 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 21/28] qapi: Require valid names Eric Blake
2015-03-27 8:48 ` Markus Armbruster
2015-03-27 20:15 ` Eric Blake
2015-03-29 10:17 ` Markus Armbruster
2015-03-29 14:23 ` Markus Armbruster
2015-03-27 17:14 ` Markus Armbruster
2015-03-27 20:17 ` Eric Blake
2015-03-29 9:06 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 22/28] qapi: Whitelist commands that don't return dictionary Eric Blake
2015-03-27 9:11 ` Markus Armbruster
2015-03-27 20:20 ` Eric Blake
2015-03-27 16:19 ` Markus Armbruster
2015-03-27 20:29 ` Eric Blake
2015-03-29 10:22 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 23/28] qapi: More rigorous checking for type safety bypass Eric Blake
2015-03-27 9:45 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 24/28] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
2015-03-27 9:52 ` Markus Armbruster
2015-03-27 20:30 ` Eric Blake
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 25/28] qapi: Drop tests for inline nested structs Eric Blake
2015-03-27 10:30 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 26/28] qapi: Drop inline nested type in query-version Eric Blake
2015-03-27 10:34 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 27/28] qapi: Drop inline nested types in query-pci Eric Blake
2015-03-27 10:37 ` Markus Armbruster
2015-03-24 20:03 ` [Qemu-devel] [PATCH v5 28/28] qapi: Drop support for inline nested types Eric Blake
2015-03-27 10:45 ` Markus Armbruster
2015-03-27 12:50 ` [Qemu-devel] [PATCH v5 00/28] drop qapi nested structs Markus Armbruster
2015-03-29 16:03 ` Markus Armbruster
2015-03-31 4:30 ` Eric Blake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.