* [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs
@ 2014-09-19 22:24 Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile Eric Blake
` (19 more replies)
0 siblings, 20 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
After dragging my feet due to some libvirt work, I've finally
revived this patch series as promised.
v3 was here:
https://lists.gnu.org/archive/html/qemu-devel/2014-08/msg00725.html
v4:
update commit message style [Markus]
add docs (patches 3-4 are new) [Markus]
lots of new tests (spread among 5, 7, 9, 11, 12)
add comments to all new tests [Markus]
rework check for valid expressions (patches 8, 10 are pretty much
new, later patches rebased and simplified as a result)
rebase to master
fix tests to match file name (broken test in v3 patch 7 now fixed
in patch 12)
drop expr_name (v3 patch 8); series now validates expressions
earlier (patch 8)
consolidate '**' magic, and reject 'gen':'yes' in commands (patch
14 is new)
probably other stuff - it's been more than a month, so reviewers
will probably want to start from scratch anyways :)
v3:
No code changes, but fix mis-send of 11-14 [Peter]
v2:
New patches: 1-2, 5-9
consistent TAB usage in Makefile [Fam]
catch more bad coding constructs, and test them
avoid code duplication in type validity checks (patch 14 [former 7] is
simpler because of patch 9)
Eric Blake (19):
qapi: Consistent whitespace in tests/Makefile
qapi: Ignore files created during make check
qapi: Update docs given recent event, spacing fixes
qapi: Document type-safety considerations
qapi: Add some enum tests
qapi: Better error messages for bad enums
qapi: Add some expr tests
qapi: Better error messages for bad expressions
qapi: Add tests of redefined expressions
qapi: Better error messages for duplicated expressions
qapi: Add tests of type bypass
qapi: Add some type check tests
qapi: More rigourous checking of types
qapi: More rigorous checking for type safety bypass
qapi: Merge UserDefTwo and UserDefNested in tests
qapi: Drop tests for inline subtypes
qapi: Drop inline subtype in query-version
qapi: Drop inline subtype in query-pci
qapi: Drop support for inline subtypes
docs/qapi-code-gen.txt | 349 +++++++++++++++++++++++----
hmp.c | 28 +--
hw/pci/pci.c | 42 ++--
qapi-schema.json | 96 +++++---
qapi/common.json | 26 +-
qmp.c | 9 +-
scripts/qapi-commands.py | 8 +-
scripts/qapi-event.py | 4 +-
scripts/qapi-types.py | 9 +-
scripts/qapi-visit.py | 37 +--
scripts/qapi.py | 261 ++++++++++++++++----
tests/.gitignore | 3 +
tests/Makefile | 43 ++--
tests/qapi-schema/bad-type-dict.err | 1 +
tests/qapi-schema/bad-type-dict.exit | 1 +
tests/qapi-schema/bad-type-dict.json | 2 +
tests/qapi-schema/bad-type-dict.out | 0
tests/qapi-schema/bad-type-int.err | 1 +
tests/qapi-schema/bad-type-int.exit | 1 +
tests/qapi-schema/bad-type-int.json | 3 +
tests/qapi-schema/bad-type-int.out | 0
tests/qapi-schema/data-array-empty.err | 1 +
tests/qapi-schema/data-array-empty.exit | 1 +
tests/qapi-schema/data-array-empty.json | 2 +
tests/qapi-schema/data-array-empty.out | 0
tests/qapi-schema/data-array-unknown.err | 1 +
tests/qapi-schema/data-array-unknown.exit | 1 +
tests/qapi-schema/data-array-unknown.json | 2 +
tests/qapi-schema/data-array-unknown.out | 0
tests/qapi-schema/data-int.err | 1 +
tests/qapi-schema/data-int.exit | 1 +
tests/qapi-schema/data-int.json | 2 +
tests/qapi-schema/data-int.out | 0
tests/qapi-schema/data-member-array-bad.err | 1 +
tests/qapi-schema/data-member-array-bad.exit | 1 +
tests/qapi-schema/data-member-array-bad.json | 2 +
tests/qapi-schema/data-member-array-bad.out | 0
tests/qapi-schema/data-member-array.err | 0
tests/qapi-schema/data-member-array.exit | 1 +
tests/qapi-schema/data-member-array.json | 4 +
tests/qapi-schema/data-member-array.out | 5 +
tests/qapi-schema/data-member-unknown.err | 1 +
tests/qapi-schema/data-member-unknown.exit | 1 +
tests/qapi-schema/data-member-unknown.json | 2 +
tests/qapi-schema/data-member-unknown.out | 0
tests/qapi-schema/data-unknown.err | 1 +
tests/qapi-schema/data-unknown.exit | 1 +
tests/qapi-schema/data-unknown.json | 2 +
tests/qapi-schema/data-unknown.out | 0
tests/qapi-schema/double-data.err | 1 +
tests/qapi-schema/double-data.exit | 1 +
tests/qapi-schema/double-data.json | 2 +
tests/qapi-schema/double-data.out | 0
tests/qapi-schema/double-type.err | 1 +
tests/qapi-schema/double-type.exit | 1 +
tests/qapi-schema/double-type.json | 2 +
tests/qapi-schema/double-type.out | 0
tests/qapi-schema/enum-clash-member.err | 1 +
tests/qapi-schema/enum-clash-member.exit | 1 +
tests/qapi-schema/enum-clash-member.json | 2 +
tests/qapi-schema/enum-clash-member.out | 0
tests/qapi-schema/enum-dict-member.err | 1 +
tests/qapi-schema/enum-dict-member.exit | 1 +
tests/qapi-schema/enum-dict-member.json | 2 +
tests/qapi-schema/enum-dict-member.out | 0
tests/qapi-schema/enum-empty.err | 0
tests/qapi-schema/enum-empty.exit | 1 +
tests/qapi-schema/enum-empty.json | 2 +
tests/qapi-schema/enum-empty.out | 3 +
tests/qapi-schema/enum-int-member.err | 1 +
tests/qapi-schema/enum-int-member.exit | 1 +
tests/qapi-schema/enum-int-member.json | 3 +
tests/qapi-schema/enum-int-member.out | 0
tests/qapi-schema/enum-max-member.err | 1 +
tests/qapi-schema/enum-max-member.exit | 1 +
tests/qapi-schema/enum-max-member.json | 3 +
tests/qapi-schema/enum-max-member.out | 0
tests/qapi-schema/enum-missing-data.err | 1 +
tests/qapi-schema/enum-missing-data.exit | 1 +
tests/qapi-schema/enum-missing-data.json | 2 +
tests/qapi-schema/enum-missing-data.out | 0
tests/qapi-schema/enum-wrong-data.err | 1 +
tests/qapi-schema/enum-wrong-data.exit | 1 +
tests/qapi-schema/enum-wrong-data.json | 2 +
tests/qapi-schema/enum-wrong-data.out | 0
tests/qapi-schema/event-nest-struct.err | 2 +-
tests/qapi-schema/indented-expr.json | 4 +-
tests/qapi-schema/indented-expr.out | 2 +-
tests/qapi-schema/missing-type.err | 1 +
tests/qapi-schema/missing-type.exit | 1 +
tests/qapi-schema/missing-type.json | 2 +
tests/qapi-schema/missing-type.out | 0
tests/qapi-schema/nested-struct-data.err | 1 +
tests/qapi-schema/nested-struct-data.exit | 1 +
tests/qapi-schema/nested-struct-data.json | 4 +
tests/qapi-schema/nested-struct-data.out | 0
tests/qapi-schema/nested-struct-returns.err | 1 +
tests/qapi-schema/nested-struct-returns.exit | 1 +
tests/qapi-schema/nested-struct-returns.json | 3 +
tests/qapi-schema/nested-struct-returns.out | 0
tests/qapi-schema/qapi-schema-test.json | 18 +-
tests/qapi-schema/qapi-schema-test.out | 10 +-
tests/qapi-schema/redefined-builtin.err | 1 +
tests/qapi-schema/redefined-builtin.exit | 1 +
tests/qapi-schema/redefined-builtin.json | 2 +
tests/qapi-schema/redefined-builtin.out | 0
tests/qapi-schema/redefined-command.err | 1 +
tests/qapi-schema/redefined-command.exit | 1 +
tests/qapi-schema/redefined-command.json | 3 +
tests/qapi-schema/redefined-command.out | 0
tests/qapi-schema/redefined-event.err | 1 +
tests/qapi-schema/redefined-event.exit | 1 +
tests/qapi-schema/redefined-event.json | 3 +
tests/qapi-schema/redefined-event.out | 0
tests/qapi-schema/redefined-type.err | 1 +
tests/qapi-schema/redefined-type.exit | 1 +
tests/qapi-schema/redefined-type.json | 3 +
tests/qapi-schema/redefined-type.out | 0
tests/qapi-schema/returns-array-bad.err | 1 +
tests/qapi-schema/returns-array-bad.exit | 1 +
tests/qapi-schema/returns-array-bad.json | 2 +
tests/qapi-schema/returns-array-bad.out | 0
tests/qapi-schema/returns-int.err | 0
tests/qapi-schema/returns-int.exit | 1 +
tests/qapi-schema/returns-int.json | 2 +
tests/qapi-schema/returns-int.out | 3 +
tests/qapi-schema/returns-unknown.err | 1 +
tests/qapi-schema/returns-unknown.exit | 1 +
tests/qapi-schema/returns-unknown.json | 2 +
tests/qapi-schema/returns-unknown.out | 0
tests/qapi-schema/type-bypass-bad-gen.err | 1 +
tests/qapi-schema/type-bypass-bad-gen.exit | 1 +
tests/qapi-schema/type-bypass-bad-gen.json | 2 +
tests/qapi-schema/type-bypass-bad-gen.out | 0
tests/qapi-schema/type-bypass-no-gen.err | 1 +
tests/qapi-schema/type-bypass-no-gen.exit | 1 +
tests/qapi-schema/type-bypass-no-gen.json | 2 +
tests/qapi-schema/type-bypass-no-gen.out | 0
tests/qapi-schema/type-bypass.err | 0
tests/qapi-schema/type-bypass.exit | 1 +
tests/qapi-schema/type-bypass.json | 2 +
tests/qapi-schema/type-bypass.out | 3 +
tests/qapi-schema/unknown-expr-key.err | 1 +
tests/qapi-schema/unknown-expr-key.exit | 1 +
tests/qapi-schema/unknown-expr-key.json | 2 +
tests/qapi-schema/unknown-expr-key.out | 0
tests/test-qmp-commands.c | 35 +--
tests/test-qmp-input-strict.c | 19 +-
tests/test-qmp-input-visitor.c | 25 +-
tests/test-qmp-output-visitor.c | 64 ++---
tests/test-visitor-serialization.c | 84 ++++---
151 files changed, 979 insertions(+), 348 deletions(-)
create mode 100644 tests/qapi-schema/bad-type-dict.err
create mode 100644 tests/qapi-schema/bad-type-dict.exit
create mode 100644 tests/qapi-schema/bad-type-dict.json
create mode 100644 tests/qapi-schema/bad-type-dict.out
create mode 100644 tests/qapi-schema/bad-type-int.err
create mode 100644 tests/qapi-schema/bad-type-int.exit
create mode 100644 tests/qapi-schema/bad-type-int.json
create mode 100644 tests/qapi-schema/bad-type-int.out
create mode 100644 tests/qapi-schema/data-array-empty.err
create mode 100644 tests/qapi-schema/data-array-empty.exit
create mode 100644 tests/qapi-schema/data-array-empty.json
create mode 100644 tests/qapi-schema/data-array-empty.out
create mode 100644 tests/qapi-schema/data-array-unknown.err
create mode 100644 tests/qapi-schema/data-array-unknown.exit
create mode 100644 tests/qapi-schema/data-array-unknown.json
create mode 100644 tests/qapi-schema/data-array-unknown.out
create mode 100644 tests/qapi-schema/data-int.err
create mode 100644 tests/qapi-schema/data-int.exit
create mode 100644 tests/qapi-schema/data-int.json
create mode 100644 tests/qapi-schema/data-int.out
create mode 100644 tests/qapi-schema/data-member-array-bad.err
create mode 100644 tests/qapi-schema/data-member-array-bad.exit
create mode 100644 tests/qapi-schema/data-member-array-bad.json
create mode 100644 tests/qapi-schema/data-member-array-bad.out
create mode 100644 tests/qapi-schema/data-member-array.err
create mode 100644 tests/qapi-schema/data-member-array.exit
create mode 100644 tests/qapi-schema/data-member-array.json
create mode 100644 tests/qapi-schema/data-member-array.out
create mode 100644 tests/qapi-schema/data-member-unknown.err
create mode 100644 tests/qapi-schema/data-member-unknown.exit
create mode 100644 tests/qapi-schema/data-member-unknown.json
create mode 100644 tests/qapi-schema/data-member-unknown.out
create mode 100644 tests/qapi-schema/data-unknown.err
create mode 100644 tests/qapi-schema/data-unknown.exit
create mode 100644 tests/qapi-schema/data-unknown.json
create mode 100644 tests/qapi-schema/data-unknown.out
create mode 100644 tests/qapi-schema/double-data.err
create mode 100644 tests/qapi-schema/double-data.exit
create mode 100644 tests/qapi-schema/double-data.json
create mode 100644 tests/qapi-schema/double-data.out
create mode 100644 tests/qapi-schema/double-type.err
create mode 100644 tests/qapi-schema/double-type.exit
create mode 100644 tests/qapi-schema/double-type.json
create mode 100644 tests/qapi-schema/double-type.out
create mode 100644 tests/qapi-schema/enum-clash-member.err
create mode 100644 tests/qapi-schema/enum-clash-member.exit
create mode 100644 tests/qapi-schema/enum-clash-member.json
create mode 100644 tests/qapi-schema/enum-clash-member.out
create mode 100644 tests/qapi-schema/enum-dict-member.err
create mode 100644 tests/qapi-schema/enum-dict-member.exit
create mode 100644 tests/qapi-schema/enum-dict-member.json
create mode 100644 tests/qapi-schema/enum-dict-member.out
create mode 100644 tests/qapi-schema/enum-empty.err
create mode 100644 tests/qapi-schema/enum-empty.exit
create mode 100644 tests/qapi-schema/enum-empty.json
create mode 100644 tests/qapi-schema/enum-empty.out
create mode 100644 tests/qapi-schema/enum-int-member.err
create mode 100644 tests/qapi-schema/enum-int-member.exit
create mode 100644 tests/qapi-schema/enum-int-member.json
create mode 100644 tests/qapi-schema/enum-int-member.out
create mode 100644 tests/qapi-schema/enum-max-member.err
create mode 100644 tests/qapi-schema/enum-max-member.exit
create mode 100644 tests/qapi-schema/enum-max-member.json
create mode 100644 tests/qapi-schema/enum-max-member.out
create mode 100644 tests/qapi-schema/enum-missing-data.err
create mode 100644 tests/qapi-schema/enum-missing-data.exit
create mode 100644 tests/qapi-schema/enum-missing-data.json
create mode 100644 tests/qapi-schema/enum-missing-data.out
create mode 100644 tests/qapi-schema/enum-wrong-data.err
create mode 100644 tests/qapi-schema/enum-wrong-data.exit
create mode 100644 tests/qapi-schema/enum-wrong-data.json
create mode 100644 tests/qapi-schema/enum-wrong-data.out
create mode 100644 tests/qapi-schema/missing-type.err
create mode 100644 tests/qapi-schema/missing-type.exit
create mode 100644 tests/qapi-schema/missing-type.json
create mode 100644 tests/qapi-schema/missing-type.out
create mode 100644 tests/qapi-schema/nested-struct-data.err
create mode 100644 tests/qapi-schema/nested-struct-data.exit
create mode 100644 tests/qapi-schema/nested-struct-data.json
create mode 100644 tests/qapi-schema/nested-struct-data.out
create mode 100644 tests/qapi-schema/nested-struct-returns.err
create mode 100644 tests/qapi-schema/nested-struct-returns.exit
create mode 100644 tests/qapi-schema/nested-struct-returns.json
create mode 100644 tests/qapi-schema/nested-struct-returns.out
create mode 100644 tests/qapi-schema/redefined-builtin.err
create mode 100644 tests/qapi-schema/redefined-builtin.exit
create mode 100644 tests/qapi-schema/redefined-builtin.json
create mode 100644 tests/qapi-schema/redefined-builtin.out
create mode 100644 tests/qapi-schema/redefined-command.err
create mode 100644 tests/qapi-schema/redefined-command.exit
create mode 100644 tests/qapi-schema/redefined-command.json
create mode 100644 tests/qapi-schema/redefined-command.out
create mode 100644 tests/qapi-schema/redefined-event.err
create mode 100644 tests/qapi-schema/redefined-event.exit
create mode 100644 tests/qapi-schema/redefined-event.json
create mode 100644 tests/qapi-schema/redefined-event.out
create mode 100644 tests/qapi-schema/redefined-type.err
create mode 100644 tests/qapi-schema/redefined-type.exit
create mode 100644 tests/qapi-schema/redefined-type.json
create mode 100644 tests/qapi-schema/redefined-type.out
create mode 100644 tests/qapi-schema/returns-array-bad.err
create mode 100644 tests/qapi-schema/returns-array-bad.exit
create mode 100644 tests/qapi-schema/returns-array-bad.json
create mode 100644 tests/qapi-schema/returns-array-bad.out
create mode 100644 tests/qapi-schema/returns-int.err
create mode 100644 tests/qapi-schema/returns-int.exit
create mode 100644 tests/qapi-schema/returns-int.json
create mode 100644 tests/qapi-schema/returns-int.out
create mode 100644 tests/qapi-schema/returns-unknown.err
create mode 100644 tests/qapi-schema/returns-unknown.exit
create mode 100644 tests/qapi-schema/returns-unknown.json
create mode 100644 tests/qapi-schema/returns-unknown.out
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.err
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.json
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.out
create mode 100644 tests/qapi-schema/type-bypass-no-gen.err
create mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-no-gen.json
create mode 100644 tests/qapi-schema/type-bypass-no-gen.out
create mode 100644 tests/qapi-schema/type-bypass.err
create mode 100644 tests/qapi-schema/type-bypass.exit
create mode 100644 tests/qapi-schema/type-bypass.json
create mode 100644 tests/qapi-schema/type-bypass.out
create mode 100644 tests/qapi-schema/unknown-expr-key.err
create mode 100644 tests/qapi-schema/unknown-expr-key.exit
create mode 100644 tests/qapi-schema/unknown-expr-key.json
create mode 100644 tests/qapi-schema/unknown-expr-key.out
--
1.9.3
^ permalink raw reply [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-22 12:40 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 02/19] qapi: Ignore files created during make check Eric Blake
` (18 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
tests/Makefile had a mix of TAB vs. 8-space indentation; given
that it is a Makefile, TAB is more idiomatic even though in these
particular cases the choice of whitespace didn't matter.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/tests/Makefile b/tests/Makefile
index a5e3d0c..976388a 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -195,23 +195,23 @@ $(foreach target,$(SYSEMU_TARGET_LIST), \
$(eval check-qtest-$(target)-y += tests/qom-test$(EXESUF)))
check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
- comments.json empty.json funny-char.json indented-expr.json \
- missing-colon.json missing-comma-list.json \
- missing-comma-object.json non-objects.json \
- qapi-schema-test.json quoted-structural-chars.json \
- trailing-comma-list.json trailing-comma-object.json \
- unclosed-list.json unclosed-object.json unclosed-string.json \
- duplicate-key.json union-invalid-base.json flat-union-no-base.json \
- flat-union-invalid-discriminator.json \
- flat-union-invalid-branch-key.json flat-union-reverse-define.json \
- flat-union-string-discriminator.json \
- include-simple.json include-relpath.json include-format-err.json \
- include-non-file.json include-no-file.json include-before-err.json \
- include-nested-err.json include-self-cycle.json include-cycle.json \
- include-repetition.json event-nest-struct.json)
+ comments.json empty.json funny-char.json indented-expr.json \
+ missing-colon.json missing-comma-list.json \
+ missing-comma-object.json non-objects.json \
+ qapi-schema-test.json quoted-structural-chars.json \
+ trailing-comma-list.json trailing-comma-object.json \
+ unclosed-list.json unclosed-object.json unclosed-string.json \
+ duplicate-key.json union-invalid-base.json flat-union-no-base.json \
+ flat-union-invalid-discriminator.json \
+ flat-union-invalid-branch-key.json flat-union-reverse-define.json \
+ flat-union-string-discriminator.json \
+ include-simple.json include-relpath.json include-format-err.json \
+ include-non-file.json include-no-file.json include-before-err.json \
+ include-nested-err.json include-self-cycle.json include-cycle.json \
+ include-repetition.json event-nest-struct.json)
GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
- tests/test-qmp-commands.h tests/test-qapi-event.h
+ tests/test-qmp-commands.h tests/test-qapi-event.h
test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/check-qlist.o tests/check-qfloat.o tests/check-qjson.o \
@@ -223,7 +223,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
tests/test-opts-visitor.o tests/test-qmp-event.o
test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
- tests/test-qapi-event.o
+ tests/test-qapi-event.o
$(test-obj-y): QEMU_INCLUDES += -Itests
QEMU_CFLAGS += -I$(SRC_PATH)/tests
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 02/19] qapi: Ignore files created during make check
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-23 8:07 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 03/19] qapi: Update docs given recent event, spacing fixes Eric Blake
` (17 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
After an in-tree build and run of 'make check-{qapi-schema,unit}',
I noticed some leftover files.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Wenchao Xia <wenchaoqemu@gmail.com>
---
tests/.gitignore | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tests/.gitignore b/tests/.gitignore
index c71c110..e2e4957 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -14,11 +14,14 @@ test-int128
test-iov
test-mul64
test-opts-visitor
+test-qapi-event.[ch]
test-qapi-types.[ch]
test-qapi-visit.[ch]
test-qdev-global-props
+test-qemu-opts
test-qmp-commands
test-qmp-commands.h
+test-qmp-event
test-qmp-input-strict
test-qmp-input-visitor
test-qmp-marshal.c
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 03/19] qapi: Update docs given recent event, spacing fixes
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 02/19] qapi: Ignore files created during make check Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-22 12:40 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations Eric Blake
` (16 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Commit 21cd70d added event support but didn't document what the
generated code looks like. Commit 05dfb26 removed some unwanted
spaces in the generated code, but didn't reflect those changes
into the documentation. Finally, the docs start with a big
disclaimer about QMP not using QAPI yet, which feels rather stale.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 100 +++++++++++++++++++++++++++++++++++++++----------
1 file changed, 80 insertions(+), 20 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index a6197a9..8313ba6 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -1,10 +1,5 @@
= How to use the QAPI code generator =
-* Note: as of this writing, QMP does not use QAPI. Eventually QMP
-commands will be converted to use QAPI internally. The following
-information describes QMP/QAPI as it will exist after the
-conversion.
-
QAPI is a native C API within QEMU which provides management-level
functionality to internal/external users. For external
users/processes, this interface is made available by a JSON-based
@@ -19,7 +14,7 @@ marshaling/dispatch code for the guest agent server running in the
guest.
This document will describe how the schemas, scripts, and resulting
-code is used.
+code are used.
== QMP/Guest agent schema ==
@@ -234,6 +229,7 @@ Resulting in this JSON object:
"data": { "b": "test string" },
"timestamp": { "seconds": 1267020223, "microseconds": 435656 } }
+
== Code generation ==
Schemas are fed into 3 scripts to generate all the code/files that, paired
@@ -256,6 +252,8 @@ command which takes that type as a parameter and returns the same type:
'data': {'arg1': 'UserDefOne'},
'returns': 'UserDefOne' }
+ { 'event': 'MY_EVENT' }
+
=== scripts/qapi-types.py ===
Used to generate the C types defined by a schema. The following files are
@@ -277,7 +275,7 @@ Example:
$ cat qapi-generated/example-qapi-types.c
[Uninteresting stuff omitted...]
- void qapi_free_UserDefOneList(UserDefOneList * obj)
+ void qapi_free_UserDefOneList(UserDefOneList *obj)
{
QapiDeallocVisitor *md;
Visitor *v;
@@ -292,7 +290,7 @@ Example:
qapi_dealloc_visitor_cleanup(md);
}
- void qapi_free_UserDefOne(UserDefOne * obj)
+ void qapi_free_UserDefOne(UserDefOne *obj)
{
QapiDeallocVisitor *md;
Visitor *v;
@@ -331,11 +329,11 @@ Example:
struct UserDefOne
{
int64_t integer;
- char * string;
+ char *string;
};
- void qapi_free_UserDefOneList(UserDefOneList * obj);
- void qapi_free_UserDefOne(UserDefOne * obj);
+ void qapi_free_UserDefOneList(UserDefOneList *obj);
+ void qapi_free_UserDefOne(UserDefOne *obj);
#endif
@@ -364,7 +362,7 @@ Example:
$ cat qapi-generated/example-qapi-visit.c
[Uninteresting stuff omitted...]
- static void visit_type_UserDefOne_fields(Visitor *m, UserDefOne ** obj, Error **errp)
+ static void visit_type_UserDefOne_fields(Visitor *m, UserDefOne **obj, Error **errp)
{
Error *err = NULL;
visit_type_int(m, &(*obj)->integer, "integer", &err);
@@ -380,7 +378,7 @@ Example:
error_propagate(errp, err);
}
- void visit_type_UserDefOne(Visitor *m, UserDefOne ** obj, const char *name, Error **errp)
+ void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp)
{
Error *err = NULL;
@@ -394,7 +392,7 @@ Example:
error_propagate(errp, err);
}
- void visit_type_UserDefOneList(Visitor *m, UserDefOneList ** obj, const char *name, Error **errp)
+ void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp)
{
Error *err = NULL;
GenericList *i, **prev;
@@ -427,8 +425,8 @@ Example:
[Visitors for builtin types omitted...]
- void visit_type_UserDefOne(Visitor *m, UserDefOne ** obj, const char *name, Error **errp);
- void visit_type_UserDefOneList(Visitor *m, UserDefOneList ** obj, const char *name, Error **errp);
+ void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp);
+ void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp);
#endif
@@ -451,10 +449,12 @@ $(prefix)qmp-commands.h: Function prototypes for the QMP commands
Example:
+ $ python scripts/qapi-commands.py --output-dir="qapi-generated"
+ --prefix="example-" --input-file=example-schema.json
$ cat qapi-generated/example-qmp-marshal.c
[Uninteresting stuff omitted...]
- static void qmp_marshal_output_my_command(UserDefOne * ret_in, QObject **ret_out, Error **errp)
+ static void qmp_marshal_output_my_command(UserDefOne *ret_in, QObject **ret_out, Error **errp)
{
Error *local_err = NULL;
QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -480,11 +480,11 @@ Example:
static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error **errp)
{
Error *local_err = NULL;
- UserDefOne * retval = NULL;
+ UserDefOne *retval = NULL;
QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
QapiDeallocVisitor *md;
Visitor *v;
- UserDefOne * arg1 = NULL;
+ UserDefOne *arg1 = NULL;
v = qmp_input_get_visitor(mi);
visit_type_UserDefOne(v, &arg1, "arg1", &local_err);
@@ -525,6 +525,66 @@ Example:
#include "qapi/qmp/qdict.h"
#include "qapi/error.h"
- UserDefOne * qmp_my_command(UserDefOne * arg1, Error **errp);
+ UserDefOne *qmp_my_command(UserDefOne *arg1, Error **errp);
+
+ #endif
+
+=== scripts/qapi-event.py ===
+
+Used to generate the event-related C code defined by a schema. The
+following files are created:
+
+$(prefix)qapi-event.h - Function prototypes for each event type, plus an
+ enumeration of all event names
+$(prefix)qapi-event.c - Implementation of functions to send an event
+
+Example:
+
+ $ python scripts/qapi-event.py --output-dir="qapi-generated"
+ --prefix="example-" --input-file=example-schema.json
+ $ cat qapi-generated/example-qapi-event.c
+[Uninteresting stuff omitted...]
+
+ void qapi_event_send_my_event(Error **errp)
+ {
+ QDict *qmp;
+ Error *local_err = NULL;
+ QMPEventFuncEmit emit;
+ emit = qmp_event_get_func_emit();
+ if (!emit) {
+ return;
+ }
+
+ qmp = qmp_event_build_dict("MY_EVENT");
+
+ emit(EXAMPLE_QAPI_EVENT_MY_EVENT, qmp, &local_err);
+
+ error_propagate(errp, local_err);
+ QDECREF(qmp);
+ }
+
+ const char *EXAMPLE_QAPIEvent_lookup[] = {
+ "MY_EVENT",
+ NULL,
+ };
+ $ cat qapi-generated/example-qapi-event.h
+[Uninteresting stuff omitted...]
+
+ #ifndef EXAMPLE_QAPI_EVENT_H
+ #define EXAMPLE_QAPI_EVENT_H
+
+ #include "qapi/error.h"
+ #include "qapi/qmp/qdict.h"
+ #include "example-qapi-types.h"
+
+
+ void qapi_event_send_my_event(Error **errp);
+
+ extern const char *EXAMPLE_QAPIEvent_lookup[];
+ typedef enum EXAMPLE_QAPIEvent
+ {
+ EXAMPLE_QAPI_EVENT_MY_EVENT = 0,
+ EXAMPLE_QAPI_EVENT_MAX = 1,
+ } EXAMPLE_QAPIEvent;
#endif
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (2 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 03/19] qapi: Update docs given recent event, spacing fixes Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-22 12:37 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 05/19] qapi: Add some enum tests Eric Blake
` (15 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Go into more details about the various types of valid expressions
in a qapi schema, including tweaks to document fixes being done
later in the current patch series.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
docs/qapi-code-gen.txt | 249 ++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 217 insertions(+), 32 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 8313ba6..3a79629 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -22,35 +22,136 @@ code are used.
This file defines the types, commands, and events used by QMP. It should
fully describe the interface used by QMP.
-This file is designed to be loosely based on JSON although it's technically
-executable Python. While dictionaries are used, they are parsed as
-OrderedDicts so that ordering is preserved.
+A QAPI file is designed to be loosely based on JSON although it's
+technically executable Python. A valid QAPI schema consists of a list
+of top-level expressions, with no commas between them. While
+dictionaries are used, they are parsed as OrderedDicts so that
+ordering is preserved; ordering doesn't matter for top-level
+expressions, but does matter within 'data' members. QAPI input is
+written using 'single quotes' instead of JSON's "double quotes" (in
+contrast, QMP is strict JSON and only uses "double quotes"). As in
+JSON, trailing commas are not permitted in arrays or dictionaries.
-There are two basic syntaxes used, type definitions and command definitions.
+Comments are allowed; anything between an unquoted # and the following
+newline is ignored. Although there is not yet a documentation
+generator, a form of stylized comments has developed for consistently
+documenting details about an expression and when it was added to the
+schema. The documentation is delimited between two lines of ##, then
+the first line names the expression, an optional overview is provided,
+then individual documentation about each member of 'data' is provided,
+and finally, a 'Since: x.y.z' tag lists the release that introduced
+the expression. Optional fields are tagged with the phrase
+'#optional', often with their default value; and extensions added
+after the expression was first released are also given a '(since
+x.y.z)' comment. For example:
-The first syntax defines a type and is represented by a dictionary. There are
-three kinds of user-defined types that are supported: complex types,
-enumeration types and union types.
+ ##
+ # @BlockStats:
+ #
+ # Statistics of a virtual block device or a block backing device.
+ #
+ # @device: #optional If the stats are for a virtual block device, the name
+ # corresponding to the virtual block device.
+ #
+ # @stats: A @BlockDeviceStats for the device.
+ #
+ # @parent: #optional This describes the file block device if it has one.
+ #
+ # @backing: #optional This describes the backing block device if it has one.
+ # (Since 2.0)
+ #
+ # Since: 0.14.0
+ ##
+ { 'type': 'BlockStats',
+ 'data': {'*device': 'str', 'stats': 'BlockDeviceStats',
+ '*parent': 'BlockStats',
+ '*backing': 'BlockStats'} }
-Generally speaking, types definitions should always use CamelCase for the type
-names. Command names should be all lower case with words separated by a hyphen.
+The schema sets up a series of types, as well as commands and events
+that will use those types. Forward references are allowed: the parser
+scans in two passes, where the first pass learns all type names, and
+the second validates the schema and generates the code. This allows
+the definition of complex structs that can have mutually recursive
+types, and allows for indefinite nesting of QMP that satisfies the
+schema. A type name should not be defined more than once.
+There are six top-level expressions recognized by the parser:
+'include', 'command', 'type', 'enum', 'union', and 'event'. There are
+several built-in types, such as 'int' and 'str'; additionally, the
+top-level expressions can define complex types, enumeration types, and
+union types. The 'command' expression can refer to existing types by
+name, or list an anonymous type as a dictionary. Listing a type name
+inside an array refers to a single-dimension array of that type;
+multi-dimension arrays are not directly supported (although an array
+of a complex struct that contains an array member is possible).
+
+Generally speaking, types definitions should always use CamelCase for
+user-defined type names, while built-in types are lowercase. Command
+names, and field names within a type, should be all lower case with
+words separated by a hyphen. However, some existing older commands
+and complex types use underscore; when extending such expressions,
+consistency is preferred over blindly avoiding underscore. Event
+names should be ALL_CAPS with words separated by underscore. The
+special string '**' appears for some commands that manually perform
+their own type checking rather than relying on the type-safe code
+produced by the qapi code generators.
+
+Any command, type, or field name beginning with "x-" is marked
+experimental, and may be withdrawn in a future release. Downstream
+vendors may add extensions; such extensions should begin with a prefix
+matching "__RFQDN_" (for the reverse-fully-qualified-domain-name of
+the vendor), even if the rest of the command or field name uses dash
+(example: __com.redhat_drive-mirror). Other than the dots used in
+RFQDN of a downstream extension, all command, type, and field names
+should begin with a letter, and contain only ASCII letters, numbers,
+dash, and underscore. It is okay to reuse names that might collide
+with programming languages; the generator will rename a field named
+"default" in the QAPI to "q_default" in the generated C code.
+
+
+=== Built-in Types ===
+
+The following types are built-in to the parser:
+ 'str' - arbitrary UTF-8 string
+ 'int' - 64-bit signed integer (although the C code may place further
+ restrictions on acceptable range)
+ 'number' - floating point number
+ 'bool' - JSON value of true or false
+ 'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
+ bit size
+ 'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
+ 'size' - like 'uint64', but allows scaled suffix from command line
+ visitor
=== Includes ===
+Usage: { 'include': 'str' }
+
The QAPI schema definitions can be modularized using the 'include' directive:
{ 'include': 'path/to/file.json'}
The directive is evaluated recursively, and include paths are relative to the
-file using the directive. Multiple includes of the same file are safe.
+file using the directive. Multiple includes of the same file are
+safe. No other keys should appear in the expression, and the include
+value should be a string.
+
+As a matter of style, it is a good idea to have all files be
+self-contained, but at the moment, nothing prevents an included file
+from making a forward reference to a type that is only introduced by
+an outer file. The parser may be made stricter in the future to
+prevent incomplete include files.
=== Complex types ===
-A complex type is a dictionary containing a single key whose value is a
-dictionary. This corresponds to a struct in C or an Object in JSON. An
-example of a complex type is:
+Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
+
+A complex type is a dictionary containing a single 'data' key whose
+value is a dictionary. This corresponds to a struct in C or an Object
+in JSON. Each value of the 'data' dictionary must be the name of a
+complex, enum, union, or built-in type, or a one-element array
+containing a type name. An example of a complex type is:
{ 'type': 'MyType',
'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
@@ -100,15 +201,42 @@ both fields like this:
{ "file": "/some/place/my-image",
"backing": "/some/place/my-backing-file" }
+
=== Enumeration types ===
-An enumeration type is a dictionary containing a single key whose value is a
-list of strings. An example enumeration is:
+Usage: { 'enum': 'str', 'data': [ 'str' ] }
+
+An enumeration type is a dictionary containing a single 'data' key
+whose value is a list of strings. An example enumeration is:
{ 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }
+Nothing prevents an empty enumeration, although it is probably not
+useful. The list of strings should be lower case; if an enum name
+represents multiple words, use '-' between words. The string 'max' is
+not allowed as an enum value, and values should not be repeated.
+
+The enumeration values are passed as strings over the QMP protocol,
+but are encoded as C enum integral values in generated code. While
+the C code starts numbering at 0, it is better to use explicit
+comparisons to enum values than implicit comparisons to 0; the C code
+will also include a generated enum member ending in _MAX for tracking
+the size of the enum, useful when using common functions for
+converting between strings and enum values. Since the wire format
+always passes by name, it is acceptable to reorder or add new
+enumeration members in any location without breaking QMP clients;
+however, removing enum values would break compatibility. For any
+complex type that has a field that will only contain a finite set of
+string values, using an enum type for that field is better than
+open-coding the field to be type 'str'.
+
+
=== Union types ===
+Usage: { 'union': 'str', 'data': 'dict', '*base': 'complex-type-name',
+ '*discriminator': 'enum-type-name' }
+or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
+
Union types are used to let the user choose between several different data
types. A union type is defined using a dictionary as explained in the
following paragraphs.
@@ -153,8 +281,10 @@ And it looks like this on the wire:
Flat union types avoid the nesting on the wire. They are used whenever a
specific field of the base type is declared as the discriminator ('type' is
-then no longer generated). The discriminator must be of enumeration type.
-The above example can then be modified as follows:
+then no longer generated). The discriminator must be of enumeration
+type, and the keys of the 'data' dictionary must match the enumeration
+keys (although not necessarily in the same order). The above example
+can then be modified as follows:
{ 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
{ 'type': 'BlockdevCommonOptions',
@@ -200,23 +330,78 @@ This example allows using both of the following example objects:
=== Commands ===
-Commands are defined by using a list containing three members. The first
-member is the command name, the second member is a dictionary containing
-arguments, and the third member is the return type.
+Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
+ '*returns': 'type',
+ '*gen': 'no', '*success-response': 'no' }
-An example command is:
+Commands are defined by using a dictionary containing several members,
+where three members are most common. The 'data' member is optional; if
+absent, the command accepts an optional empty dictionary. If present,
+it must be the string name of a complex type, a one-element array
+containing the name of a complex type, or a dictionary that declares
+an anonymous type with the same semantics as a 'type' expression, with
+one exception noted below. The 'returns' member is optional; if
+absent, the command returns an empty dictionary. If present, it must
+be the string name of a complex or built-in type, a one-element array
+containing the name of a complex or built-in type, or a dictionary
+that declares an anonymous type with the same semantics as a 'type'
+expression, with one exception noted below.
+
+Although it is permitted to have the 'returns' member name a built-in
+type, any command that does this cannot be extended to return
+additional information in the future; thus, new commands should
+strongly consider returning a dictionary-based type, even if it only
+contains one field at the present. Besides, all commands return a
+dictionary to report failure.
+
+Some example commands:
+
+ { 'command': 'my-first-command',
+ 'data': { 'arg1': 'str', '*arg2': 'str' } }
+ { 'type': 'MyType', 'data': { '*value': 'str' } }
+ { 'command': 'my-second-command',
+ 'returns': [ 'MyType' ] }
+
+which would validate this QMP transaction:
+
+ => { "execute": "my-first-command",
+ "arguments": { "arg1": "hello" } }
+ <= { "return": { } }
+ => { "execute": "my-second-command" }
+ <= { "return": [ { "value": "one" }, { } ] }
+
+In rare cases, QAPI cannot express a type-safe representation of a
+corresponding QMP command. In these cases, if the command expression
+includes the key 'gen' with value 'no', then the 'data' or 'returns'
+member that intends to bypass generated type-safety and do its own
+manual validation should use '**' rather than a valid type name.
+Please try to avoid adding new commands that rely on this, and instead
+use type-safe unions. For an example of bypass usage:
+
+ { 'command': 'netdev_add',
+ 'data': {'type': 'str', 'id': 'str', '*props': '**'},
+ 'gen': 'no' }
+
+Normally, the QAPI schema is used to describe synchronous exchanges,
+where a response is expected. But in some cases, the action of a
+command is expected to change state in a way that a successful
+response is not possible (a failure message still returns a
+dictionary). In this case, the command expression should include the
+optional key 'success-response' with value 'no'.
- { 'command': 'my-command',
- 'data': { 'arg1': 'str', '*arg2': 'str' },
- 'returns': 'str' }
=== Events ===
-Events are defined with the keyword 'event'. When 'data' is also specified,
-additional info will be included in the event. Finally there will be C API
-generated in qapi-event.h; when called by QEMU code, a message with timestamp
-will be emitted on the wire. If timestamp is -1, it means failure to retrieve
-host time.
+Usage: { 'event': 'str', '*data': 'dict-or-complex-type-name' }
+
+Events are defined with the keyword 'event'. It is not allowed to
+name an event 'MAX', since the generator also produces a C enumeration
+of all event names with a generated _MAX value at the end. When
+'data' is also specified, additional info will be included in the
+event, with similar semantics to a 'type' expression. Finally there
+will be C API generated in qapi-event.h; when called by QEMU code, a
+message with timestamp will be emitted on the wire. If timestamp is
+-1, it means failure to retrieve host time.
An example event is:
@@ -311,7 +496,7 @@ Example:
#ifndef EXAMPLE_QAPI_TYPES_H
#define EXAMPLE_QAPI_TYPES_H
-[Builtin types omitted...]
+[Built-in types omitted...]
typedef struct UserDefOne UserDefOne;
@@ -324,7 +509,7 @@ Example:
struct UserDefOneList *next;
} UserDefOneList;
-[Functions on builtin types omitted...]
+[Functions on built-in types omitted...]
struct UserDefOne
{
@@ -423,7 +608,7 @@ Example:
#ifndef EXAMPLE_QAPI_VISIT_H
#define EXAMPLE_QAPI_VISIT_H
-[Visitors for builtin types omitted...]
+[Visitors for built-in types omitted...]
void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp);
void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp);
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 05/19] qapi: Add some enum tests
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (3 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-22 12:43 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums Eric Blake
` (14 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Demonstrate that the qapi generator doesn't deal well with enums
that aren't up to par. Later patches will update the expected
results as the generator is made stricter.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 5 ++++-
tests/qapi-schema/enum-clash-member.err | 0
tests/qapi-schema/enum-clash-member.exit | 1 +
tests/qapi-schema/enum-clash-member.json | 2 ++
tests/qapi-schema/enum-clash-member.out | 3 +++
tests/qapi-schema/enum-dict-member.err | 0
tests/qapi-schema/enum-dict-member.exit | 1 +
tests/qapi-schema/enum-dict-member.json | 2 ++
tests/qapi-schema/enum-dict-member.out | 3 +++
tests/qapi-schema/enum-empty.err | 0
tests/qapi-schema/enum-empty.exit | 1 +
tests/qapi-schema/enum-empty.json | 2 ++
tests/qapi-schema/enum-empty.out | 3 +++
tests/qapi-schema/enum-int-member.err | 1 +
tests/qapi-schema/enum-int-member.exit | 1 +
tests/qapi-schema/enum-int-member.json | 3 +++
tests/qapi-schema/enum-int-member.out | 0
tests/qapi-schema/enum-max-member.err | 0
tests/qapi-schema/enum-max-member.exit | 1 +
tests/qapi-schema/enum-max-member.json | 3 +++
tests/qapi-schema/enum-max-member.out | 3 +++
tests/qapi-schema/enum-missing-data.err | 6 ++++++
tests/qapi-schema/enum-missing-data.exit | 1 +
tests/qapi-schema/enum-missing-data.json | 2 ++
tests/qapi-schema/enum-missing-data.out | 0
tests/qapi-schema/enum-wrong-data.err | 0
tests/qapi-schema/enum-wrong-data.exit | 1 +
tests/qapi-schema/enum-wrong-data.json | 2 ++
tests/qapi-schema/enum-wrong-data.out | 3 +++
29 files changed, 49 insertions(+), 1 deletion(-)
create mode 100644 tests/qapi-schema/enum-clash-member.err
create mode 100644 tests/qapi-schema/enum-clash-member.exit
create mode 100644 tests/qapi-schema/enum-clash-member.json
create mode 100644 tests/qapi-schema/enum-clash-member.out
create mode 100644 tests/qapi-schema/enum-dict-member.err
create mode 100644 tests/qapi-schema/enum-dict-member.exit
create mode 100644 tests/qapi-schema/enum-dict-member.json
create mode 100644 tests/qapi-schema/enum-dict-member.out
create mode 100644 tests/qapi-schema/enum-empty.err
create mode 100644 tests/qapi-schema/enum-empty.exit
create mode 100644 tests/qapi-schema/enum-empty.json
create mode 100644 tests/qapi-schema/enum-empty.out
create mode 100644 tests/qapi-schema/enum-int-member.err
create mode 100644 tests/qapi-schema/enum-int-member.exit
create mode 100644 tests/qapi-schema/enum-int-member.json
create mode 100644 tests/qapi-schema/enum-int-member.out
create mode 100644 tests/qapi-schema/enum-max-member.err
create mode 100644 tests/qapi-schema/enum-max-member.exit
create mode 100644 tests/qapi-schema/enum-max-member.json
create mode 100644 tests/qapi-schema/enum-max-member.out
create mode 100644 tests/qapi-schema/enum-missing-data.err
create mode 100644 tests/qapi-schema/enum-missing-data.exit
create mode 100644 tests/qapi-schema/enum-missing-data.json
create mode 100644 tests/qapi-schema/enum-missing-data.out
create mode 100644 tests/qapi-schema/enum-wrong-data.err
create mode 100644 tests/qapi-schema/enum-wrong-data.exit
create mode 100644 tests/qapi-schema/enum-wrong-data.json
create mode 100644 tests/qapi-schema/enum-wrong-data.out
diff --git a/tests/Makefile b/tests/Makefile
index 976388a..6a4e5a6 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -195,7 +195,10 @@ $(foreach target,$(SYSEMU_TARGET_LIST), \
$(eval check-qtest-$(target)-y += tests/qom-test$(EXESUF)))
check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
- comments.json empty.json funny-char.json indented-expr.json \
+ comments.json empty.json enum-empty.json enum-missing-data.json \
+ enum-wrong-data.json enum-int-member.json enum-dict-member.json \
+ enum-clash-member.json enum-max-member.json \
+ funny-char.json indented-expr.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/enum-clash-member.err b/tests/qapi-schema/enum-clash-member.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-clash-member.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
new file mode 100644
index 0000000..cb4b428
--- /dev/null
+++ b/tests/qapi-schema/enum-clash-member.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject enums where members will clash in C switch
+{ 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
diff --git a/tests/qapi-schema/enum-clash-member.out b/tests/qapi-schema/enum-clash-member.out
new file mode 100644
index 0000000..0814459
--- /dev/null
+++ b/tests/qapi-schema/enum-clash-member.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', ['one', 'ONE'])])]
+[{'enum_name': 'MyEnum', 'enum_values': ['one', 'ONE']}]
+[]
diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-dict-member.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
new file mode 100644
index 0000000..de4d6bf
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject any enum member that is not a string
+{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-dict-member.out
new file mode 100644
index 0000000..8b293f8
--- /dev/null
+++ b/tests/qapi-schema/enum-dict-member.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', [OrderedDict([('value', 'str')])])])]
+[{'enum_name': 'MyEnum', 'enum_values': [OrderedDict([('value', 'str')])]}]
+[]
diff --git a/tests/qapi-schema/enum-empty.err b/tests/qapi-schema/enum-empty.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-empty.exit b/tests/qapi-schema/enum-empty.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-empty.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-empty.json b/tests/qapi-schema/enum-empty.json
new file mode 100644
index 0000000..40d4e85
--- /dev/null
+++ b/tests/qapi-schema/enum-empty.json
@@ -0,0 +1,2 @@
+# An empty enum, although unusual, is currently acceptable
+{ 'enum': 'MyEnum', 'data': [ ] }
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
new file mode 100644
index 0000000..3b75c16
--- /dev/null
+++ b/tests/qapi-schema/enum-empty.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', [])])]
+[{'enum_name': 'MyEnum', 'enum_values': []}]
+[]
diff --git a/tests/qapi-schema/enum-int-member.err b/tests/qapi-schema/enum-int-member.err
new file mode 100644
index 0000000..071c521
--- /dev/null
+++ b/tests/qapi-schema/enum-int-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-int-member.json:3:31: Stray "1"
diff --git a/tests/qapi-schema/enum-int-member.exit b/tests/qapi-schema/enum-int-member.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/enum-int-member.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/enum-int-member.json b/tests/qapi-schema/enum-int-member.json
new file mode 100644
index 0000000..6c9c32e
--- /dev/null
+++ b/tests/qapi-schema/enum-int-member.json
@@ -0,0 +1,3 @@
+# we reject any enum member that is not a string
+# FIXME: once the parser understands integer inputs, improve the error message
+{ 'enum': 'MyEnum', 'data': [ 1 ] }
diff --git a/tests/qapi-schema/enum-int-member.out b/tests/qapi-schema/enum-int-member.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-max-member.err b/tests/qapi-schema/enum-max-member.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-max-member.exit b/tests/qapi-schema/enum-max-member.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-max-member.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-max-member.json b/tests/qapi-schema/enum-max-member.json
new file mode 100644
index 0000000..ea854c4
--- /dev/null
+++ b/tests/qapi-schema/enum-max-member.json
@@ -0,0 +1,3 @@
+# FIXME: we should either reject user-supplied 'max', or munge the implicit
+# max value we generate at the end of an array
+{ 'enum': 'MyEnum', 'data': [ 'max' ] }
diff --git a/tests/qapi-schema/enum-max-member.out b/tests/qapi-schema/enum-max-member.out
new file mode 100644
index 0000000..c933044
--- /dev/null
+++ b/tests/qapi-schema/enum-max-member.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', ['max'])])]
+[{'enum_name': 'MyEnum', 'enum_values': ['max']}]
+[]
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
new file mode 100644
index 0000000..1fec213
--- /dev/null
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -0,0 +1,6 @@
+Traceback (most recent call last):
+ File "tests/qapi-schema/test-qapi.py", line 19, in <module>
+ exprs = parse_schema(sys.argv[1])
+ File "scripts/qapi.py", line 339, in parse_schema
+ add_enum(expr['enum'], expr['data'])
+KeyError: 'data'
diff --git a/tests/qapi-schema/enum-missing-data.exit b/tests/qapi-schema/enum-missing-data.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/enum-missing-data.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/enum-missing-data.json b/tests/qapi-schema/enum-missing-data.json
new file mode 100644
index 0000000..01f3f32
--- /dev/null
+++ b/tests/qapi-schema/enum-missing-data.json
@@ -0,0 +1,2 @@
+# FIXME: we should require that all QAPI enums have a data array
+{ 'enum': 'MyEnum' }
diff --git a/tests/qapi-schema/enum-missing-data.out b/tests/qapi-schema/enum-missing-data.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-wrong-data.err b/tests/qapi-schema/enum-wrong-data.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/enum-wrong-data.exit b/tests/qapi-schema/enum-wrong-data.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/enum-wrong-data.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/enum-wrong-data.json b/tests/qapi-schema/enum-wrong-data.json
new file mode 100644
index 0000000..61d25ec
--- /dev/null
+++ b/tests/qapi-schema/enum-wrong-data.json
@@ -0,0 +1,2 @@
+# FIXME: we should require that all qapi enums have an array for data
+{ 'enum': 'MyEnum', 'data': { 'value': 'str' } }
diff --git a/tests/qapi-schema/enum-wrong-data.out b/tests/qapi-schema/enum-wrong-data.out
new file mode 100644
index 0000000..28d2211
--- /dev/null
+++ b/tests/qapi-schema/enum-wrong-data.out
@@ -0,0 +1,3 @@
+[OrderedDict([('enum', 'MyEnum'), ('data', OrderedDict([('value', 'str')]))])]
+[{'enum_name': 'MyEnum', 'enum_values': OrderedDict([('value', 'str')])}]
+[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (4 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 05/19] qapi: Add some enum tests Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-23 14:23 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 07/19] qapi: Add some expr tests Eric Blake
` (13 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
The previous commit demonstrated that the generator had several
flaws with less-than-perfect enums:
- an enum that listed the same string twice (or two variant
strings that map to the same C enum) ended up generating an
invalid C enum
- because the generator adds a _MAX terminator to each enum,
the use of an enum member 'max' can also cause this clash
- if an enum omits 'data', the generator left a python stack
trace rather than a graceful message
- an enum used a non-array 'data' was silently accepted by
the parser
- an enum that used non-string members in the 'data' member
was silently accepted by the parser
Add check_enum to cover these situations, and update testcases
to match. While valid .json files won't trigger any of these
cases, we might as well be nicer to developers that make a typo
while trying to add new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 32 ++++++++++++++++++++++++++++----
tests/qapi-schema/enum-clash-member.err | 1 +
tests/qapi-schema/enum-clash-member.exit | 2 +-
tests/qapi-schema/enum-clash-member.json | 2 +-
tests/qapi-schema/enum-clash-member.out | 3 ---
tests/qapi-schema/enum-dict-member.err | 1 +
tests/qapi-schema/enum-dict-member.exit | 2 +-
tests/qapi-schema/enum-dict-member.json | 2 +-
tests/qapi-schema/enum-dict-member.out | 3 ---
tests/qapi-schema/enum-max-member.err | 1 +
tests/qapi-schema/enum-max-member.exit | 2 +-
tests/qapi-schema/enum-max-member.json | 4 ++--
tests/qapi-schema/enum-max-member.out | 3 ---
tests/qapi-schema/enum-missing-data.err | 7 +------
tests/qapi-schema/enum-missing-data.json | 2 +-
tests/qapi-schema/enum-wrong-data.err | 1 +
tests/qapi-schema/enum-wrong-data.exit | 2 +-
tests/qapi-schema/enum-wrong-data.json | 2 +-
tests/qapi-schema/enum-wrong-data.out | 3 ---
19 files changed, 43 insertions(+), 32 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 77d46aa..85aa8bf 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -2,7 +2,7 @@
# QAPI helper library
#
# Copyright IBM, Corp. 2011
-# Copyright (c) 2013 Red Hat Inc.
+# Copyright (c) 2013-2014 Red Hat Inc.
#
# Authors:
# Anthony Liguori <aliguori@us.ibm.com>
@@ -316,13 +316,37 @@ def check_union(expr, expr_info):
# Todo: add checking for values. Key is checked as above, value can be
# also checked here, but we need more functions to handle array case.
+def check_enum(expr, expr_info):
+ name = expr['enum']
+ members = expr.get('data')
+ values = { 'MAX': '(automatic)' }
+
+ if not isinstance(members, list):
+ raise QAPIExprError(expr_info,
+ "enum '%s' requires an array for 'data'" % name)
+ for member in members:
+ if not isinstance(member, basestring):
+ raise QAPIExprError(expr_info,
+ "enum '%s' member '%s' is not a string"
+ % (name, member))
+ key = _generate_enum_string(member)
+ if key in values:
+ raise QAPIExprError(expr_info,
+ "enum '%s' member '%s' clashes with '%s'"
+ % (name, member, values[key]))
+ values[key] = member
+
def check_exprs(schema):
for expr_elem in schema.exprs:
expr = expr_elem['expr']
+ info = expr_elem['info']
+
if expr.has_key('union'):
- check_union(expr, expr_elem['info'])
+ check_union(expr, info)
if expr.has_key('event'):
- check_event(expr, expr_elem['info'])
+ check_event(expr, info)
+ if expr.has_key('enum'):
+ check_enum(expr, info)
def parse_schema(input_file):
try:
@@ -336,7 +360,7 @@ def parse_schema(input_file):
for expr_elem in schema.exprs:
expr = expr_elem['expr']
if expr.has_key('enum'):
- add_enum(expr['enum'], expr['data'])
+ add_enum(expr['enum'], expr.get('data'))
elif expr.has_key('union'):
add_union(expr)
elif expr.has_key('type'):
diff --git a/tests/qapi-schema/enum-clash-member.err b/tests/qapi-schema/enum-clash-member.err
index e69de29..3731fc2 100644
--- a/tests/qapi-schema/enum-clash-member.err
+++ b/tests/qapi-schema/enum-clash-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-clash-member.json:2: enum 'MyEnum' member 'ONE' clashes with 'one'
diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-clash-member.exit
+++ b/tests/qapi-schema/enum-clash-member.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
index cb4b428..c668ff5 100644
--- a/tests/qapi-schema/enum-clash-member.json
+++ b/tests/qapi-schema/enum-clash-member.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject enums where members will clash in C switch
+# we reject enums where members will clash in C switch
{ 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
diff --git a/tests/qapi-schema/enum-clash-member.out b/tests/qapi-schema/enum-clash-member.out
index 0814459..e69de29 100644
--- a/tests/qapi-schema/enum-clash-member.out
+++ b/tests/qapi-schema/enum-clash-member.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', ['one', 'ONE'])])]
-[{'enum_name': 'MyEnum', 'enum_values': ['one', 'ONE']}]
-[]
diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
index e69de29..f5a8ffe 100644
--- a/tests/qapi-schema/enum-dict-member.err
+++ b/tests/qapi-schema/enum-dict-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-dict-member.json:2: enum 'MyEnum' member 'OrderedDict([('value', 'str')])' is not a string
diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-dict-member.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-dict-member.exit
+++ b/tests/qapi-schema/enum-dict-member.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
index de4d6bf..79672e0 100644
--- a/tests/qapi-schema/enum-dict-member.json
+++ b/tests/qapi-schema/enum-dict-member.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject any enum member that is not a string
+# we reject any enum member that is not a string
{ 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-dict-member.out
index 8b293f8..e69de29 100644
--- a/tests/qapi-schema/enum-dict-member.out
+++ b/tests/qapi-schema/enum-dict-member.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', [OrderedDict([('value', 'str')])])])]
-[{'enum_name': 'MyEnum', 'enum_values': [OrderedDict([('value', 'str')])]}]
-[]
diff --git a/tests/qapi-schema/enum-max-member.err b/tests/qapi-schema/enum-max-member.err
index e69de29..a6c5db1 100644
--- a/tests/qapi-schema/enum-max-member.err
+++ b/tests/qapi-schema/enum-max-member.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-max-member.json:3: enum 'MyEnum' member 'max' clashes with '(automatic)'
diff --git a/tests/qapi-schema/enum-max-member.exit b/tests/qapi-schema/enum-max-member.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-max-member.exit
+++ b/tests/qapi-schema/enum-max-member.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-max-member.json b/tests/qapi-schema/enum-max-member.json
index ea854c4..6af4662 100644
--- a/tests/qapi-schema/enum-max-member.json
+++ b/tests/qapi-schema/enum-max-member.json
@@ -1,3 +1,3 @@
-# FIXME: we should either reject user-supplied 'max', or munge the implicit
-# max value we generate at the end of an array
+# we reject user-supplied 'max' for clashing with implicit enum end
+# FIXME: should we instead munge the the implicit value to avoid the clash?
{ 'enum': 'MyEnum', 'data': [ 'max' ] }
diff --git a/tests/qapi-schema/enum-max-member.out b/tests/qapi-schema/enum-max-member.out
index c933044..e69de29 100644
--- a/tests/qapi-schema/enum-max-member.out
+++ b/tests/qapi-schema/enum-max-member.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', ['max'])])]
-[{'enum_name': 'MyEnum', 'enum_values': ['max']}]
-[]
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
index 1fec213..4b8b59e 100644
--- a/tests/qapi-schema/enum-missing-data.err
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -1,6 +1 @@
-Traceback (most recent call last):
- File "tests/qapi-schema/test-qapi.py", line 19, in <module>
- exprs = parse_schema(sys.argv[1])
- File "scripts/qapi.py", line 339, in parse_schema
- add_enum(expr['enum'], expr['data'])
-KeyError: 'data'
+tests/qapi-schema/enum-missing-data.json:2: enum 'MyEnum' requires an array for 'data'
diff --git a/tests/qapi-schema/enum-missing-data.json b/tests/qapi-schema/enum-missing-data.json
index 01f3f32..558fd35 100644
--- a/tests/qapi-schema/enum-missing-data.json
+++ b/tests/qapi-schema/enum-missing-data.json
@@ -1,2 +1,2 @@
-# FIXME: we should require that all QAPI enums have a data array
+# we require that all QAPI enums have a data array
{ 'enum': 'MyEnum' }
diff --git a/tests/qapi-schema/enum-wrong-data.err b/tests/qapi-schema/enum-wrong-data.err
index e69de29..0d14e80 100644
--- a/tests/qapi-schema/enum-wrong-data.err
+++ b/tests/qapi-schema/enum-wrong-data.err
@@ -0,0 +1 @@
+tests/qapi-schema/enum-wrong-data.json:2: enum 'MyEnum' requires an array for 'data'
diff --git a/tests/qapi-schema/enum-wrong-data.exit b/tests/qapi-schema/enum-wrong-data.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/enum-wrong-data.exit
+++ b/tests/qapi-schema/enum-wrong-data.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/enum-wrong-data.json b/tests/qapi-schema/enum-wrong-data.json
index 61d25ec..7b3e255 100644
--- a/tests/qapi-schema/enum-wrong-data.json
+++ b/tests/qapi-schema/enum-wrong-data.json
@@ -1,2 +1,2 @@
-# FIXME: we should require that all qapi enums have an array for data
+# we require that all qapi enums have an array for data
{ 'enum': 'MyEnum', 'data': { 'value': 'str' } }
diff --git a/tests/qapi-schema/enum-wrong-data.out b/tests/qapi-schema/enum-wrong-data.out
index 28d2211..e69de29 100644
--- a/tests/qapi-schema/enum-wrong-data.out
+++ b/tests/qapi-schema/enum-wrong-data.out
@@ -1,3 +0,0 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', OrderedDict([('value', 'str')]))])]
-[{'enum_name': 'MyEnum', 'enum_values': OrderedDict([('value', 'str')])}]
-[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 07/19] qapi: Add some expr tests
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (5 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-23 14:26 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions Eric Blake
` (12 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Demonstrate that the qapi generator doesn't deal well with
expressions that aren't up to par. Later patches will improve
the expected results as the generator is made stricter. Only
one of the added tests actually behaves sanely at rejecting
obvious problems.
Note that in some cases, we reject bad QAPI merely because our
pseudo-JSON parser does not yet know how to parse numbers. This
series does not address that, but when a later series adds support
for numeric defaults of integer fields, the testsuite will ensure
that we don't lose the error (and hopefully that the error
message quality is improved).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 4 +++-
tests/qapi-schema/bad-type-dict.err | 0
tests/qapi-schema/bad-type-dict.exit | 1 +
tests/qapi-schema/bad-type-dict.json | 2 ++
tests/qapi-schema/bad-type-dict.out | 3 +++
tests/qapi-schema/bad-type-int.err | 1 +
tests/qapi-schema/bad-type-int.exit | 1 +
tests/qapi-schema/bad-type-int.json | 3 +++
tests/qapi-schema/bad-type-int.out | 0
tests/qapi-schema/double-data.err | 1 +
tests/qapi-schema/double-data.exit | 1 +
tests/qapi-schema/double-data.json | 2 ++
tests/qapi-schema/double-data.out | 0
tests/qapi-schema/double-type.err | 0
tests/qapi-schema/double-type.exit | 1 +
tests/qapi-schema/double-type.json | 2 ++
tests/qapi-schema/double-type.out | 3 +++
tests/qapi-schema/missing-type.err | 0
tests/qapi-schema/missing-type.exit | 1 +
tests/qapi-schema/missing-type.json | 2 ++
tests/qapi-schema/missing-type.out | 3 +++
tests/qapi-schema/unknown-expr-key.err | 0
tests/qapi-schema/unknown-expr-key.exit | 1 +
tests/qapi-schema/unknown-expr-key.json | 2 ++
tests/qapi-schema/unknown-expr-key.out | 3 +++
25 files changed, 36 insertions(+), 1 deletion(-)
create mode 100644 tests/qapi-schema/bad-type-dict.err
create mode 100644 tests/qapi-schema/bad-type-dict.exit
create mode 100644 tests/qapi-schema/bad-type-dict.json
create mode 100644 tests/qapi-schema/bad-type-dict.out
create mode 100644 tests/qapi-schema/bad-type-int.err
create mode 100644 tests/qapi-schema/bad-type-int.exit
create mode 100644 tests/qapi-schema/bad-type-int.json
create mode 100644 tests/qapi-schema/bad-type-int.out
create mode 100644 tests/qapi-schema/double-data.err
create mode 100644 tests/qapi-schema/double-data.exit
create mode 100644 tests/qapi-schema/double-data.json
create mode 100644 tests/qapi-schema/double-data.out
create mode 100644 tests/qapi-schema/double-type.err
create mode 100644 tests/qapi-schema/double-type.exit
create mode 100644 tests/qapi-schema/double-type.json
create mode 100644 tests/qapi-schema/double-type.out
create mode 100644 tests/qapi-schema/missing-type.err
create mode 100644 tests/qapi-schema/missing-type.exit
create mode 100644 tests/qapi-schema/missing-type.json
create mode 100644 tests/qapi-schema/missing-type.out
create mode 100644 tests/qapi-schema/unknown-expr-key.err
create mode 100644 tests/qapi-schema/unknown-expr-key.exit
create mode 100644 tests/qapi-schema/unknown-expr-key.json
create mode 100644 tests/qapi-schema/unknown-expr-key.out
diff --git a/tests/Makefile b/tests/Makefile
index 6a4e5a6..753f7bd 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -198,7 +198,9 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
comments.json empty.json enum-empty.json enum-missing-data.json \
enum-wrong-data.json enum-int-member.json enum-dict-member.json \
enum-clash-member.json enum-max-member.json \
- funny-char.json indented-expr.json \
+ funny-char.json indented-expr.json missing-type.json \
+ double-type.json bad-type-int.json bad-type-dict.json \
+ double-data.json unknown-expr-key.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/bad-type-dict.err b/tests/qapi-schema/bad-type-dict.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-type-dict.exit b/tests/qapi-schema/bad-type-dict.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/bad-type-dict.json b/tests/qapi-schema/bad-type-dict.json
new file mode 100644
index 0000000..3c392a7
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with a metatype that is not a string
+{ 'command': { } }
diff --git a/tests/qapi-schema/bad-type-dict.out b/tests/qapi-schema/bad-type-dict.out
new file mode 100644
index 0000000..c62f1ed
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/bad-type-int.err b/tests/qapi-schema/bad-type-int.err
new file mode 100644
index 0000000..9808550
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-type-int.json:3:11: Stray "1"
diff --git a/tests/qapi-schema/bad-type-int.exit b/tests/qapi-schema/bad-type-int.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/bad-type-int.json b/tests/qapi-schema/bad-type-int.json
new file mode 100644
index 0000000..398879d
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.json
@@ -0,0 +1,3 @@
+# we reject an expression with a metatype that is not a string
+# FIXME: once the parser understands integer inputs, improve the error message
+{ 'type': 1, 'data': { } }
diff --git a/tests/qapi-schema/bad-type-int.out b/tests/qapi-schema/bad-type-int.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-data.err b/tests/qapi-schema/double-data.err
new file mode 100644
index 0000000..6f1a67b
--- /dev/null
+++ b/tests/qapi-schema/double-data.err
@@ -0,0 +1 @@
+tests/qapi-schema/double-data.json:2:39: Duplicate key "data"
diff --git a/tests/qapi-schema/double-data.exit b/tests/qapi-schema/double-data.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/double-data.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/double-data.json b/tests/qapi-schema/double-data.json
new file mode 100644
index 0000000..a94b7df
--- /dev/null
+++ b/tests/qapi-schema/double-data.json
@@ -0,0 +1,2 @@
+# we reject an expression with duplicate top-level keys
+{ 'type': 'bar', 'data': { }, 'data': { 'string': 'str'} }
diff --git a/tests/qapi-schema/double-data.out b/tests/qapi-schema/double-data.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-type.exit b/tests/qapi-schema/double-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/double-type.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/double-type.json b/tests/qapi-schema/double-type.json
new file mode 100644
index 0000000..6ca96b9
--- /dev/null
+++ b/tests/qapi-schema/double-type.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with ambiguous metatype
+{ 'command': 'foo', 'type': 'bar', 'data': { } }
diff --git a/tests/qapi-schema/double-type.out b/tests/qapi-schema/double-type.out
new file mode 100644
index 0000000..3e244f5
--- /dev/null
+++ b/tests/qapi-schema/double-type.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
+[]
+[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
diff --git a/tests/qapi-schema/missing-type.err b/tests/qapi-schema/missing-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/missing-type.exit b/tests/qapi-schema/missing-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/missing-type.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/missing-type.json b/tests/qapi-schema/missing-type.json
new file mode 100644
index 0000000..1696f5c
--- /dev/null
+++ b/tests/qapi-schema/missing-type.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with missing metatype
+{ 'data': { } }
diff --git a/tests/qapi-schema/missing-type.out b/tests/qapi-schema/missing-type.out
new file mode 100644
index 0000000..67fd4fa
--- /dev/null
+++ b/tests/qapi-schema/missing-type.out
@@ -0,0 +1,3 @@
+[OrderedDict([('data', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/unknown-expr-key.exit b/tests/qapi-schema/unknown-expr-key.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
new file mode 100644
index 0000000..1e9282d
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an expression with unknown top-level keys
+{ 'type': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
diff --git a/tests/qapi-schema/unknown-expr-key.out b/tests/qapi-schema/unknown-expr-key.out
new file mode 100644
index 0000000..c93f020
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.out
@@ -0,0 +1,3 @@
+[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
+[]
+[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (6 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 07/19] qapi: Add some expr tests Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-23 14:56 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 09/19] qapi: Add tests of redefined expressions Eric Blake
` (11 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
The previous commit demonstrated that the generator overlooked some
fairly basic broken expressions:
- missing metataype
- metatype key has a non-string value
- unknown key in relation to the metatype
- conflicting metatype (this patch treats the second metatype as an
unknown key of the first key visited, which is not necessarily the
first key the user typed)
Add check_keys to cover these situations, and update testcases to
match. A couple other tests (enum-missing-data, indented-expr) had
to change since the validation added here occurs so early.
While valid .json files won't trigger any of these cases, we might
as well be nicer to developers that make a typo while trying to add
new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 76 +++++++++++++++++++++++++--------
tests/qapi-schema/bad-type-dict.err | 1 +
tests/qapi-schema/bad-type-dict.exit | 2 +-
tests/qapi-schema/bad-type-dict.json | 2 +-
tests/qapi-schema/bad-type-dict.out | 3 --
tests/qapi-schema/double-type.err | 1 +
tests/qapi-schema/double-type.exit | 2 +-
tests/qapi-schema/double-type.json | 2 +-
tests/qapi-schema/double-type.out | 3 --
tests/qapi-schema/enum-missing-data.err | 2 +-
tests/qapi-schema/indented-expr.json | 4 +-
tests/qapi-schema/indented-expr.out | 2 +-
tests/qapi-schema/missing-type.err | 1 +
tests/qapi-schema/missing-type.exit | 2 +-
tests/qapi-schema/missing-type.json | 2 +-
tests/qapi-schema/missing-type.out | 3 --
tests/qapi-schema/unknown-expr-key.err | 1 +
tests/qapi-schema/unknown-expr-key.exit | 2 +-
tests/qapi-schema/unknown-expr-key.json | 2 +-
tests/qapi-schema/unknown-expr-key.out | 3 --
20 files changed, 75 insertions(+), 41 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 85aa8bf..8fbc45f 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -348,7 +348,29 @@ def check_exprs(schema):
if expr.has_key('enum'):
check_enum(expr, info)
+def check_keys(expr_elem, meta, required, optional=[]):
+ expr = expr_elem['expr']
+ info = expr_elem['info']
+ name = expr[meta]
+ if not isinstance(name, basestring):
+ raise QAPIExprError(info,
+ "%s key must have a string value" % meta)
+ expr_elem['name'] = name
+ required.append(meta)
+ for (key, value) in expr.items():
+ if not key in required and not key in optional:
+ raise QAPIExprError(info,
+ "%s '%s' has unknown key '%s'"
+ % (meta, name, key))
+ for key in required:
+ if not expr.has_key(key):
+ raise QAPIExprError(info,
+ "%s '%s' is missing key '%s'"
+ % (meta, name, key))
+
+
def parse_schema(input_file):
+ # First pass: read entire file into memory
try:
schema = QAPISchema(open(input_file, "r"))
except (QAPISchemaError, QAPIExprError), e:
@@ -357,24 +379,44 @@ def parse_schema(input_file):
exprs = []
- for expr_elem in schema.exprs:
- expr = expr_elem['expr']
- if expr.has_key('enum'):
- add_enum(expr['enum'], expr.get('data'))
- elif expr.has_key('union'):
- add_union(expr)
- elif expr.has_key('type'):
- add_struct(expr)
- exprs.append(expr)
-
- # Try again for hidden UnionKind enum
- for expr_elem in schema.exprs:
- expr = expr_elem['expr']
- if expr.has_key('union'):
- if not discriminator_find_enum_define(expr):
- add_enum('%sKind' % expr['union'])
-
try:
+ # Next pass: learn the types and check for valid expression keys. At
+ # this point, top-level 'include' has already been flattened.
+ for expr_elem in schema.exprs:
+ expr = expr_elem['expr']
+ if expr.has_key('enum'):
+ check_keys(expr_elem, 'enum', ['data'])
+ add_enum(expr['enum'], expr['data'])
+ elif expr.has_key('union'):
+ # Two styles of union, based on discriminator
+ discriminator = expr.get('discriminator')
+ if discriminator == {}:
+ check_keys(expr_elem, 'union', ['data', 'discriminator'])
+ else:
+ check_keys(expr_elem, 'union', ['data'],
+ ['base', 'discriminator'])
+ add_union(expr)
+ elif expr.has_key('type'):
+ check_keys(expr_elem, 'type', ['data'], ['base'])
+ add_struct(expr)
+ elif expr.has_key('command'):
+ check_keys(expr_elem, 'command', [],
+ ['data', 'returns', 'gen', 'success-response'])
+ elif expr.has_key('event'):
+ check_keys(expr_elem, 'event', [], ['data'])
+ else:
+ raise QAPIExprError(expr_elem['info'],
+ "expression is missing metatype")
+ exprs.append(expr)
+
+ # Try again for hidden UnionKind enum
+ for expr_elem in schema.exprs:
+ expr = expr_elem['expr']
+ if expr.has_key('union'):
+ if not discriminator_find_enum_define(expr):
+ add_enum('%sKind' % expr['union'])
+
+ # Final pass - validate that exprs make sense
check_exprs(schema)
except QAPIExprError, e:
print >>sys.stderr, e
diff --git a/tests/qapi-schema/bad-type-dict.err b/tests/qapi-schema/bad-type-dict.err
index e69de29..b7c73cc 100644
--- a/tests/qapi-schema/bad-type-dict.err
+++ b/tests/qapi-schema/bad-type-dict.err
@@ -0,0 +1 @@
+tests/qapi-schema/bad-type-dict.json:2: command key must have a string value
diff --git a/tests/qapi-schema/bad-type-dict.exit b/tests/qapi-schema/bad-type-dict.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/bad-type-dict.exit
+++ b/tests/qapi-schema/bad-type-dict.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/bad-type-dict.json b/tests/qapi-schema/bad-type-dict.json
index 3c392a7..2a91b24 100644
--- a/tests/qapi-schema/bad-type-dict.json
+++ b/tests/qapi-schema/bad-type-dict.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with a metatype that is not a string
+# we reject an expression with a metatype that is not a string
{ 'command': { } }
diff --git a/tests/qapi-schema/bad-type-dict.out b/tests/qapi-schema/bad-type-dict.out
index c62f1ed..e69de29 100644
--- a/tests/qapi-schema/bad-type-dict.out
+++ b/tests/qapi-schema/bad-type-dict.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', OrderedDict())])]
-[]
-[]
diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
index e69de29..394127b 100644
--- a/tests/qapi-schema/double-type.err
+++ b/tests/qapi-schema/double-type.err
@@ -0,0 +1 @@
+tests/qapi-schema/double-type.json:2: type 'bar' has unknown key 'command'
diff --git a/tests/qapi-schema/double-type.exit b/tests/qapi-schema/double-type.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/double-type.exit
+++ b/tests/qapi-schema/double-type.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/double-type.json b/tests/qapi-schema/double-type.json
index 6ca96b9..471623a 100644
--- a/tests/qapi-schema/double-type.json
+++ b/tests/qapi-schema/double-type.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with ambiguous metatype
+# we reject an expression with ambiguous metatype
{ 'command': 'foo', 'type': 'bar', 'data': { } }
diff --git a/tests/qapi-schema/double-type.out b/tests/qapi-schema/double-type.out
index 3e244f5..e69de29 100644
--- a/tests/qapi-schema/double-type.out
+++ b/tests/qapi-schema/double-type.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
-[]
-[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
diff --git a/tests/qapi-schema/enum-missing-data.err b/tests/qapi-schema/enum-missing-data.err
index 4b8b59e..62f4a7f 100644
--- a/tests/qapi-schema/enum-missing-data.err
+++ b/tests/qapi-schema/enum-missing-data.err
@@ -1 +1 @@
-tests/qapi-schema/enum-missing-data.json:2: enum 'MyEnum' requires an array for 'data'
+tests/qapi-schema/enum-missing-data.json:2: enum 'MyEnum' is missing key 'data'
diff --git a/tests/qapi-schema/indented-expr.json b/tests/qapi-schema/indented-expr.json
index d80af60..7115d31 100644
--- a/tests/qapi-schema/indented-expr.json
+++ b/tests/qapi-schema/indented-expr.json
@@ -1,2 +1,2 @@
-{ 'id' : 'eins' }
- { 'id' : 'zwei' }
+{ 'command' : 'eins' }
+ { 'command' : 'zwei' }
diff --git a/tests/qapi-schema/indented-expr.out b/tests/qapi-schema/indented-expr.out
index 98af89a..b5ce915 100644
--- a/tests/qapi-schema/indented-expr.out
+++ b/tests/qapi-schema/indented-expr.out
@@ -1,3 +1,3 @@
-[OrderedDict([('id', 'eins')]), OrderedDict([('id', 'zwei')])]
+[OrderedDict([('command', 'eins')]), OrderedDict([('command', 'zwei')])]
[]
[]
diff --git a/tests/qapi-schema/missing-type.err b/tests/qapi-schema/missing-type.err
index e69de29..19b7c49 100644
--- a/tests/qapi-schema/missing-type.err
+++ b/tests/qapi-schema/missing-type.err
@@ -0,0 +1 @@
+tests/qapi-schema/missing-type.json:2: expression is missing metatype
diff --git a/tests/qapi-schema/missing-type.exit b/tests/qapi-schema/missing-type.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/missing-type.exit
+++ b/tests/qapi-schema/missing-type.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/missing-type.json b/tests/qapi-schema/missing-type.json
index 1696f5c..ff5349d 100644
--- a/tests/qapi-schema/missing-type.json
+++ b/tests/qapi-schema/missing-type.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with missing metatype
+# we reject an expression with missing metatype
{ 'data': { } }
diff --git a/tests/qapi-schema/missing-type.out b/tests/qapi-schema/missing-type.out
index 67fd4fa..e69de29 100644
--- a/tests/qapi-schema/missing-type.out
+++ b/tests/qapi-schema/missing-type.out
@@ -1,3 +0,0 @@
-[OrderedDict([('data', OrderedDict())])]
-[]
-[]
diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
index e69de29..3b35508 100644
--- a/tests/qapi-schema/unknown-expr-key.err
+++ b/tests/qapi-schema/unknown-expr-key.err
@@ -0,0 +1 @@
+tests/qapi-schema/unknown-expr-key.json:2: type 'bar' has unknown key 'bogus'
diff --git a/tests/qapi-schema/unknown-expr-key.exit b/tests/qapi-schema/unknown-expr-key.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/unknown-expr-key.exit
+++ b/tests/qapi-schema/unknown-expr-key.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
index 1e9282d..ba7bdf3 100644
--- a/tests/qapi-schema/unknown-expr-key.json
+++ b/tests/qapi-schema/unknown-expr-key.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an expression with unknown top-level keys
+# we reject an expression with unknown top-level keys
{ 'type': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
diff --git a/tests/qapi-schema/unknown-expr-key.out b/tests/qapi-schema/unknown-expr-key.out
index c93f020..e69de29 100644
--- a/tests/qapi-schema/unknown-expr-key.out
+++ b/tests/qapi-schema/unknown-expr-key.out
@@ -1,3 +0,0 @@
-[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
-[]
-[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 09/19] qapi: Add tests of redefined expressions
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (7 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-24 11:24 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions Eric Blake
` (10 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Demonstrate that the qapi generator doesn't deal very well with
redefined expressions. At the parse level, they are silently
accepted; I'm not sure what would happen if we tried to go
further and use it in generated code, but the end result can't
be good. A later patch will tighten things up and adjust the
testsuite to match.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 3 ++-
tests/qapi-schema/redefined-builtin.err | 0
tests/qapi-schema/redefined-builtin.exit | 1 +
tests/qapi-schema/redefined-builtin.json | 2 ++
tests/qapi-schema/redefined-builtin.out | 3 +++
tests/qapi-schema/redefined-command.err | 0
tests/qapi-schema/redefined-command.exit | 1 +
tests/qapi-schema/redefined-command.json | 3 +++
tests/qapi-schema/redefined-command.out | 4 ++++
tests/qapi-schema/redefined-event.err | 0
tests/qapi-schema/redefined-event.exit | 1 +
tests/qapi-schema/redefined-event.json | 3 +++
tests/qapi-schema/redefined-event.out | 4 ++++
tests/qapi-schema/redefined-type.err | 0
tests/qapi-schema/redefined-type.exit | 1 +
tests/qapi-schema/redefined-type.json | 3 +++
tests/qapi-schema/redefined-type.out | 4 ++++
17 files changed, 32 insertions(+), 1 deletion(-)
create mode 100644 tests/qapi-schema/redefined-builtin.err
create mode 100644 tests/qapi-schema/redefined-builtin.exit
create mode 100644 tests/qapi-schema/redefined-builtin.json
create mode 100644 tests/qapi-schema/redefined-builtin.out
create mode 100644 tests/qapi-schema/redefined-command.err
create mode 100644 tests/qapi-schema/redefined-command.exit
create mode 100644 tests/qapi-schema/redefined-command.json
create mode 100644 tests/qapi-schema/redefined-command.out
create mode 100644 tests/qapi-schema/redefined-event.err
create mode 100644 tests/qapi-schema/redefined-event.exit
create mode 100644 tests/qapi-schema/redefined-event.json
create mode 100644 tests/qapi-schema/redefined-event.out
create mode 100644 tests/qapi-schema/redefined-type.err
create mode 100644 tests/qapi-schema/redefined-type.exit
create mode 100644 tests/qapi-schema/redefined-type.json
create mode 100644 tests/qapi-schema/redefined-type.out
diff --git a/tests/Makefile b/tests/Makefile
index 753f7bd..201e006 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -200,7 +200,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
enum-clash-member.json enum-max-member.json \
funny-char.json indented-expr.json missing-type.json \
double-type.json bad-type-int.json bad-type-dict.json \
- double-data.json unknown-expr-key.json \
+ double-data.json unknown-expr-key.json redefined-type.json \
+ redefined-command.json redefined-builtin.json redefined-event.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/redefined-builtin.err b/tests/qapi-schema/redefined-builtin.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-builtin.exit b/tests/qapi-schema/redefined-builtin.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-builtin.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-builtin.json b/tests/qapi-schema/redefined-builtin.json
new file mode 100644
index 0000000..a10050d
--- /dev/null
+++ b/tests/qapi-schema/redefined-builtin.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject types that duplicate builtin names
+{ 'type': 'size', 'data': { 'myint': 'size' } }
diff --git a/tests/qapi-schema/redefined-builtin.out b/tests/qapi-schema/redefined-builtin.out
new file mode 100644
index 0000000..b0a9aea
--- /dev/null
+++ b/tests/qapi-schema/redefined-builtin.out
@@ -0,0 +1,3 @@
+[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
+[]
+[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
diff --git a/tests/qapi-schema/redefined-command.err b/tests/qapi-schema/redefined-command.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-command.exit b/tests/qapi-schema/redefined-command.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-command.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-command.json b/tests/qapi-schema/redefined-command.json
new file mode 100644
index 0000000..d8c9975
--- /dev/null
+++ b/tests/qapi-schema/redefined-command.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject commands defined more than once
+{ 'command': 'foo', 'data': { 'one': 'str' } }
+{ 'command': 'foo', 'data': { '*two': 'str' } }
diff --git a/tests/qapi-schema/redefined-command.out b/tests/qapi-schema/redefined-command.out
new file mode 100644
index 0000000..44ddba6
--- /dev/null
+++ b/tests/qapi-schema/redefined-command.out
@@ -0,0 +1,4 @@
+[OrderedDict([('command', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
+ OrderedDict([('command', 'foo'), ('data', OrderedDict([('*two', 'str')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/redefined-event.err b/tests/qapi-schema/redefined-event.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-event.exit b/tests/qapi-schema/redefined-event.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-event.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-event.json b/tests/qapi-schema/redefined-event.json
new file mode 100644
index 0000000..152cce7
--- /dev/null
+++ b/tests/qapi-schema/redefined-event.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject duplicate events
+{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
+{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
diff --git a/tests/qapi-schema/redefined-event.out b/tests/qapi-schema/redefined-event.out
new file mode 100644
index 0000000..261b183
--- /dev/null
+++ b/tests/qapi-schema/redefined-event.out
@@ -0,0 +1,4 @@
+[OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))]),
+ OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/redefined-type.err b/tests/qapi-schema/redefined-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/redefined-type.exit b/tests/qapi-schema/redefined-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/redefined-type.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/redefined-type.json b/tests/qapi-schema/redefined-type.json
new file mode 100644
index 0000000..7972194
--- /dev/null
+++ b/tests/qapi-schema/redefined-type.json
@@ -0,0 +1,3 @@
+# FIXME: we should reject types defined more than once
+{ 'type': 'foo', 'data': { 'one': 'str' } }
+{ 'enum': 'foo', 'data': [ 'two' ] }
diff --git a/tests/qapi-schema/redefined-type.out b/tests/qapi-schema/redefined-type.out
new file mode 100644
index 0000000..b509e57
--- /dev/null
+++ b/tests/qapi-schema/redefined-type.out
@@ -0,0 +1,4 @@
+[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
+ OrderedDict([('enum', 'foo'), ('data', ['two'])])]
+[{'enum_name': 'foo', 'enum_values': ['two']}]
+[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))])]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (8 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 09/19] qapi: Add tests of redefined expressions Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-24 11:58 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 11/19] qapi: Add tests of type bypass Eric Blake
` (9 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
The previous commit demonstrated that the generator overlooked
duplicate expressions:
- a complex type reusing a built-in type name
- redeclaration of a type name, whether by the same or different
metatype
- redeclaration of a command or event
- lack of tracking of 'size' as a built-in type
Add a global array to track all known types and their source,
as well as enhancing check_exprs to also check for duplicate
events and commands. While valid .json files won't trigger any
of these cases, we might as well be nicer to developers that
make a typo while trying to add new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 71 +++++++++++++++++++++++++-------
tests/qapi-schema/redefined-builtin.err | 1 +
tests/qapi-schema/redefined-builtin.exit | 2 +-
tests/qapi-schema/redefined-builtin.json | 2 +-
tests/qapi-schema/redefined-builtin.out | 3 --
tests/qapi-schema/redefined-command.err | 1 +
tests/qapi-schema/redefined-command.exit | 2 +-
tests/qapi-schema/redefined-command.json | 2 +-
tests/qapi-schema/redefined-command.out | 4 --
tests/qapi-schema/redefined-event.err | 1 +
tests/qapi-schema/redefined-event.exit | 2 +-
tests/qapi-schema/redefined-event.json | 2 +-
tests/qapi-schema/redefined-event.out | 4 --
tests/qapi-schema/redefined-type.err | 1 +
tests/qapi-schema/redefined-type.exit | 2 +-
tests/qapi-schema/redefined-type.json | 2 +-
tests/qapi-schema/redefined-type.out | 4 --
17 files changed, 69 insertions(+), 37 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 8fbc45f..bf243fa 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -19,8 +19,15 @@ import sys
builtin_types = [
'str', 'int', 'number', 'bool',
'int8', 'int16', 'int32', 'int64',
- 'uint8', 'uint16', 'uint32', 'uint64'
+ 'uint8', 'uint16', 'uint32', 'uint64',
+ 'size'
]
+enum_types = []
+struct_types = []
+union_types = []
+all_types = {}
+commands = []
+events = ['MAX']
builtin_type_qtypes = {
'str': 'QTYPE_QSTRING',
@@ -248,8 +255,22 @@ def discriminator_find_enum_define(expr):
return find_enum(discriminator_type)
+def check_command(expr, expr_info):
+ global commands
+ name = expr['command']
+ if name in commands:
+ raise QAPIExprError(expr_info,
+ "command '%s' is already defined" % name)
+ commands.append(name)
+
def check_event(expr, expr_info):
+ global events
+ name = expr['event']
params = expr.get('data')
+ if name in events:
+ raise QAPIExprError(expr_info,
+ "event '%s' is already defined" % name)
+ events.append(name)
if params:
for argname, argentry, optional, structured in parse_args(params):
if structured:
@@ -347,6 +368,8 @@ def check_exprs(schema):
check_event(expr, info)
if expr.has_key('enum'):
check_enum(expr, info)
+ if expr.has_key('command'):
+ check_command(expr, info)
def check_keys(expr_elem, meta, required, optional=[]):
expr = expr_elem['expr']
@@ -370,6 +393,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
def parse_schema(input_file):
+ global all_types
+ exprs = []
+
# First pass: read entire file into memory
try:
schema = QAPISchema(open(input_file, "r"))
@@ -377,16 +403,17 @@ def parse_schema(input_file):
print >>sys.stderr, e
exit(1)
- exprs = []
-
try:
# Next pass: learn the types and check for valid expression keys. At
# this point, top-level 'include' has already been flattened.
+ for builtin in builtin_types:
+ all_types[builtin] = 'built-in'
for expr_elem in schema.exprs:
expr = expr_elem['expr']
+ info = expr_elem['info']
if expr.has_key('enum'):
check_keys(expr_elem, 'enum', ['data'])
- add_enum(expr['enum'], expr['data'])
+ add_enum(expr['enum'], info, expr['data'])
elif expr.has_key('union'):
# Two styles of union, based on discriminator
discriminator = expr.get('discriminator')
@@ -395,10 +422,10 @@ def parse_schema(input_file):
else:
check_keys(expr_elem, 'union', ['data'],
['base', 'discriminator'])
- add_union(expr)
+ add_union(expr, info)
elif expr.has_key('type'):
check_keys(expr_elem, 'type', ['data'], ['base'])
- add_struct(expr)
+ add_struct(expr, info)
elif expr.has_key('command'):
check_keys(expr_elem, 'command', [],
['data', 'returns', 'gen', 'success-response'])
@@ -414,7 +441,7 @@ def parse_schema(input_file):
expr = expr_elem['expr']
if expr.has_key('union'):
if not discriminator_find_enum_define(expr):
- add_enum('%sKind' % expr['union'])
+ add_enum('%sKind' % expr['union'], expr_elem['info'])
# Final pass - validate that exprs make sense
check_exprs(schema)
@@ -508,12 +535,15 @@ def type_name(name):
return c_list_type(name[0])
return name
-enum_types = []
-struct_types = []
-union_types = []
-
-def add_struct(definition):
+def add_struct(definition, info):
global struct_types
+ global all_types
+ name = definition['type']
+ if name in all_types:
+ raise QAPIExprError(info,
+ "%s '%s' is already defined"
+ %(all_types[name], name))
+ all_types[name] = 'struct'
struct_types.append(definition)
def find_struct(name):
@@ -523,8 +553,15 @@ def find_struct(name):
return struct
return None
-def add_union(definition):
+def add_union(definition, info):
global union_types
+ global all_types
+ name = definition['union']
+ if name in all_types:
+ raise QAPIExprError(info,
+ "%s '%s' is already defined"
+ %(all_types[name], name))
+ all_types[name] = 'union'
union_types.append(definition)
def find_union(name):
@@ -534,8 +571,14 @@ def find_union(name):
return union
return None
-def add_enum(name, enum_values = None):
+def add_enum(name, info, enum_values = None):
global enum_types
+ global all_types
+ if name in all_types:
+ raise QAPIExprError(info,
+ "%s '%s' is already defined"
+ %(all_types[name], name))
+ all_types[name] = 'enum'
enum_types.append({"enum_name": name, "enum_values": enum_values})
def find_enum(name):
diff --git a/tests/qapi-schema/redefined-builtin.err b/tests/qapi-schema/redefined-builtin.err
index e69de29..b275722 100644
--- a/tests/qapi-schema/redefined-builtin.err
+++ b/tests/qapi-schema/redefined-builtin.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-builtin.json:2: built-in 'size' is already defined
diff --git a/tests/qapi-schema/redefined-builtin.exit b/tests/qapi-schema/redefined-builtin.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-builtin.exit
+++ b/tests/qapi-schema/redefined-builtin.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-builtin.json b/tests/qapi-schema/redefined-builtin.json
index a10050d..df328cc 100644
--- a/tests/qapi-schema/redefined-builtin.json
+++ b/tests/qapi-schema/redefined-builtin.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject types that duplicate builtin names
+# we reject types that duplicate builtin names
{ 'type': 'size', 'data': { 'myint': 'size' } }
diff --git a/tests/qapi-schema/redefined-builtin.out b/tests/qapi-schema/redefined-builtin.out
index b0a9aea..e69de29 100644
--- a/tests/qapi-schema/redefined-builtin.out
+++ b/tests/qapi-schema/redefined-builtin.out
@@ -1,3 +0,0 @@
-[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
-[]
-[OrderedDict([('type', 'size'), ('data', OrderedDict([('myint', 'size')]))])]
diff --git a/tests/qapi-schema/redefined-command.err b/tests/qapi-schema/redefined-command.err
index e69de29..82ae256 100644
--- a/tests/qapi-schema/redefined-command.err
+++ b/tests/qapi-schema/redefined-command.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-command.json:3: command 'foo' is already defined
diff --git a/tests/qapi-schema/redefined-command.exit b/tests/qapi-schema/redefined-command.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-command.exit
+++ b/tests/qapi-schema/redefined-command.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-command.json b/tests/qapi-schema/redefined-command.json
index d8c9975..247e401 100644
--- a/tests/qapi-schema/redefined-command.json
+++ b/tests/qapi-schema/redefined-command.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject commands defined more than once
+# we reject commands defined more than once
{ 'command': 'foo', 'data': { 'one': 'str' } }
{ 'command': 'foo', 'data': { '*two': 'str' } }
diff --git a/tests/qapi-schema/redefined-command.out b/tests/qapi-schema/redefined-command.out
index 44ddba6..e69de29 100644
--- a/tests/qapi-schema/redefined-command.out
+++ b/tests/qapi-schema/redefined-command.out
@@ -1,4 +0,0 @@
-[OrderedDict([('command', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
- OrderedDict([('command', 'foo'), ('data', OrderedDict([('*two', 'str')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/redefined-event.err b/tests/qapi-schema/redefined-event.err
index e69de29..35429cb 100644
--- a/tests/qapi-schema/redefined-event.err
+++ b/tests/qapi-schema/redefined-event.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-event.json:3: event 'EVENT_A' is already defined
diff --git a/tests/qapi-schema/redefined-event.exit b/tests/qapi-schema/redefined-event.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-event.exit
+++ b/tests/qapi-schema/redefined-event.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-event.json b/tests/qapi-schema/redefined-event.json
index 152cce7..7717e91 100644
--- a/tests/qapi-schema/redefined-event.json
+++ b/tests/qapi-schema/redefined-event.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject duplicate events
+# we reject duplicate events
{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
{ 'event': 'EVENT_A', 'data': { 'myint': 'int' } }
diff --git a/tests/qapi-schema/redefined-event.out b/tests/qapi-schema/redefined-event.out
index 261b183..e69de29 100644
--- a/tests/qapi-schema/redefined-event.out
+++ b/tests/qapi-schema/redefined-event.out
@@ -1,4 +0,0 @@
-[OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))]),
- OrderedDict([('event', 'EVENT_A'), ('data', OrderedDict([('myint', 'int')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/redefined-type.err b/tests/qapi-schema/redefined-type.err
index e69de29..06ea78c 100644
--- a/tests/qapi-schema/redefined-type.err
+++ b/tests/qapi-schema/redefined-type.err
@@ -0,0 +1 @@
+tests/qapi-schema/redefined-type.json:3: struct 'foo' is already defined
diff --git a/tests/qapi-schema/redefined-type.exit b/tests/qapi-schema/redefined-type.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/redefined-type.exit
+++ b/tests/qapi-schema/redefined-type.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/redefined-type.json b/tests/qapi-schema/redefined-type.json
index 7972194..e6a5f24 100644
--- a/tests/qapi-schema/redefined-type.json
+++ b/tests/qapi-schema/redefined-type.json
@@ -1,3 +1,3 @@
-# FIXME: we should reject types defined more than once
+# we reject types defined more than once
{ 'type': 'foo', 'data': { 'one': 'str' } }
{ 'enum': 'foo', 'data': [ 'two' ] }
diff --git a/tests/qapi-schema/redefined-type.out b/tests/qapi-schema/redefined-type.out
index b509e57..e69de29 100644
--- a/tests/qapi-schema/redefined-type.out
+++ b/tests/qapi-schema/redefined-type.out
@@ -1,4 +0,0 @@
-[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))]),
- OrderedDict([('enum', 'foo'), ('data', ['two'])])]
-[{'enum_name': 'foo', 'enum_values': ['two']}]
-[OrderedDict([('type', 'foo'), ('data', OrderedDict([('one', 'str')]))])]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 11/19] qapi: Add tests of type bypass
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (9 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-24 16:10 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
` (8 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
For a few QMP commands, we are forced to pass an arbitrary type
without tracking it properly in QAPI. Among the existing clients,
this unnamed type was spelled 'dict', 'visitor', and '**'; this
patch standardizes on '**'. There is no difference to the generated
code. As the feature was previously undocumented, add some tests
and documentation on what we'd like to guarantee, although it will
take later patches to clean up test results.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
qapi-schema.json | 6 +++---
tests/Makefile | 1 +
tests/qapi-schema/type-bypass-bad-gen.err | 0
tests/qapi-schema/type-bypass-bad-gen.exit | 1 +
tests/qapi-schema/type-bypass-bad-gen.json | 2 ++
tests/qapi-schema/type-bypass-bad-gen.out | 3 +++
tests/qapi-schema/type-bypass-no-gen.err | 0
tests/qapi-schema/type-bypass-no-gen.exit | 1 +
tests/qapi-schema/type-bypass-no-gen.json | 2 ++
tests/qapi-schema/type-bypass-no-gen.out | 3 +++
tests/qapi-schema/type-bypass.err | 0
tests/qapi-schema/type-bypass.exit | 1 +
tests/qapi-schema/type-bypass.json | 2 ++
tests/qapi-schema/type-bypass.out | 3 +++
14 files changed, 22 insertions(+), 3 deletions(-)
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.err
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.json
create mode 100644 tests/qapi-schema/type-bypass-bad-gen.out
create mode 100644 tests/qapi-schema/type-bypass-no-gen.err
create mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
create mode 100644 tests/qapi-schema/type-bypass-no-gen.json
create mode 100644 tests/qapi-schema/type-bypass-no-gen.out
create mode 100644 tests/qapi-schema/type-bypass.err
create mode 100644 tests/qapi-schema/type-bypass.exit
create mode 100644 tests/qapi-schema/type-bypass.json
create mode 100644 tests/qapi-schema/type-bypass.out
diff --git a/qapi-schema.json b/qapi-schema.json
index 689b548..0b612bf 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1455,7 +1455,7 @@
##
{ 'command': 'qom-get',
'data': { 'path': 'str', 'property': 'str' },
- 'returns': 'visitor',
+ 'returns': '**',
'gen': 'no' }
##
@@ -1473,7 +1473,7 @@
# Since: 1.2
##
{ 'command': 'qom-set',
- 'data': { 'path': 'str', 'property': 'str', 'value': 'visitor' },
+ 'data': { 'path': 'str', 'property': 'str', 'value': '**' },
'gen': 'no' }
##
@@ -1850,7 +1850,7 @@
# Since: 2.0
##
{ 'command': 'object-add',
- 'data': {'qom-type': 'str', 'id': 'str', '*props': 'dict'},
+ 'data': {'qom-type': 'str', 'id': 'str', '*props': '**'},
'gen': 'no' }
##
diff --git a/tests/Makefile b/tests/Makefile
index 201e006..5e01952 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -202,6 +202,7 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
double-type.json bad-type-int.json bad-type-dict.json \
double-data.json unknown-expr-key.json redefined-type.json \
redefined-command.json redefined-builtin.json redefined-event.json \
+ type-bypass.json type-bypass-no-gen.json type-bypass-bad-gen.json \
missing-colon.json missing-comma-list.json \
missing-comma-object.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
diff --git a/tests/qapi-schema/type-bypass-bad-gen.err b/tests/qapi-schema/type-bypass-bad-gen.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/type-bypass-bad-gen.exit b/tests/qapi-schema/type-bypass-bad-gen.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-bad-gen.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/type-bypass-bad-gen.json b/tests/qapi-schema/type-bypass-bad-gen.json
new file mode 100644
index 0000000..6894526
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-bad-gen.json
@@ -0,0 +1,2 @@
+# FIXME: 'gen' should only appear with value 'no'
+{ 'command': 'foo', 'gen': 'whatever' }
diff --git a/tests/qapi-schema/type-bypass-bad-gen.out b/tests/qapi-schema/type-bypass-bad-gen.out
new file mode 100644
index 0000000..e678f2c
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-bad-gen.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('gen', 'whatever')])]
+[]
+[]
diff --git a/tests/qapi-schema/type-bypass-no-gen.err b/tests/qapi-schema/type-bypass-no-gen.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/type-bypass-no-gen.exit b/tests/qapi-schema/type-bypass-no-gen.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-no-gen.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/type-bypass-no-gen.json b/tests/qapi-schema/type-bypass-no-gen.json
new file mode 100644
index 0000000..72c817f
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-no-gen.json
@@ -0,0 +1,2 @@
+# FIXME: type bypass should only work with 'gen':'no'
+{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**' }
diff --git a/tests/qapi-schema/type-bypass-no-gen.out b/tests/qapi-schema/type-bypass-no-gen.out
new file mode 100644
index 0000000..8b2a9ac
--- /dev/null
+++ b/tests/qapi-schema/type-bypass-no-gen.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**')])]
+[]
+[]
diff --git a/tests/qapi-schema/type-bypass.err b/tests/qapi-schema/type-bypass.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/type-bypass.exit b/tests/qapi-schema/type-bypass.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/type-bypass.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/type-bypass.json b/tests/qapi-schema/type-bypass.json
new file mode 100644
index 0000000..fd65073
--- /dev/null
+++ b/tests/qapi-schema/type-bypass.json
@@ -0,0 +1,2 @@
+# Use of 'gen':'no' allows bypassing type system
+{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**', 'gen': 'no' }
diff --git a/tests/qapi-schema/type-bypass.out b/tests/qapi-schema/type-bypass.out
new file mode 100644
index 0000000..8957dcb
--- /dev/null
+++ b/tests/qapi-schema/type-bypass.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**'), ('gen', 'no')])]
+[]
+[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (10 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 11/19] qapi: Add tests of type bypass Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-25 7:34 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types Eric Blake
` (7 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Demonstrate that the qapi generator silently parses confusing
types, which may cause other errors later on. Later patches
will update the expected results as the generator is made stricter.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/Makefile | 8 ++++++--
tests/qapi-schema/data-array-empty.err | 0
tests/qapi-schema/data-array-empty.exit | 1 +
tests/qapi-schema/data-array-empty.json | 2 ++
tests/qapi-schema/data-array-empty.out | 3 +++
tests/qapi-schema/data-array-unknown.err | 0
tests/qapi-schema/data-array-unknown.exit | 1 +
tests/qapi-schema/data-array-unknown.json | 2 ++
tests/qapi-schema/data-array-unknown.out | 3 +++
tests/qapi-schema/data-int.err | 0
tests/qapi-schema/data-int.exit | 1 +
tests/qapi-schema/data-int.json | 2 ++
tests/qapi-schema/data-int.out | 3 +++
tests/qapi-schema/data-member-array-bad.err | 0
tests/qapi-schema/data-member-array-bad.exit | 1 +
tests/qapi-schema/data-member-array-bad.json | 2 ++
tests/qapi-schema/data-member-array-bad.out | 3 +++
tests/qapi-schema/data-member-array.err | 0
tests/qapi-schema/data-member-array.exit | 1 +
tests/qapi-schema/data-member-array.json | 4 ++++
tests/qapi-schema/data-member-array.out | 5 +++++
tests/qapi-schema/data-member-unknown.err | 0
tests/qapi-schema/data-member-unknown.exit | 1 +
tests/qapi-schema/data-member-unknown.json | 2 ++
tests/qapi-schema/data-member-unknown.out | 3 +++
tests/qapi-schema/data-unknown.err | 0
tests/qapi-schema/data-unknown.exit | 1 +
tests/qapi-schema/data-unknown.json | 2 ++
tests/qapi-schema/data-unknown.out | 3 +++
tests/qapi-schema/nested-struct-data.err | 0
tests/qapi-schema/nested-struct-data.exit | 1 +
tests/qapi-schema/nested-struct-data.json | 4 ++++
tests/qapi-schema/nested-struct-data.out | 3 +++
tests/qapi-schema/nested-struct-returns.err | 0
tests/qapi-schema/nested-struct-returns.exit | 1 +
tests/qapi-schema/nested-struct-returns.json | 3 +++
tests/qapi-schema/nested-struct-returns.out | 3 +++
tests/qapi-schema/returns-array-bad.err | 0
tests/qapi-schema/returns-array-bad.exit | 1 +
tests/qapi-schema/returns-array-bad.json | 2 ++
tests/qapi-schema/returns-array-bad.out | 3 +++
tests/qapi-schema/returns-int.err | 0
tests/qapi-schema/returns-int.exit | 1 +
tests/qapi-schema/returns-int.json | 2 ++
tests/qapi-schema/returns-int.out | 3 +++
tests/qapi-schema/returns-unknown.err | 0
tests/qapi-schema/returns-unknown.exit | 1 +
tests/qapi-schema/returns-unknown.json | 2 ++
tests/qapi-schema/returns-unknown.out | 3 +++
49 files changed, 85 insertions(+), 2 deletions(-)
create mode 100644 tests/qapi-schema/data-array-empty.err
create mode 100644 tests/qapi-schema/data-array-empty.exit
create mode 100644 tests/qapi-schema/data-array-empty.json
create mode 100644 tests/qapi-schema/data-array-empty.out
create mode 100644 tests/qapi-schema/data-array-unknown.err
create mode 100644 tests/qapi-schema/data-array-unknown.exit
create mode 100644 tests/qapi-schema/data-array-unknown.json
create mode 100644 tests/qapi-schema/data-array-unknown.out
create mode 100644 tests/qapi-schema/data-int.err
create mode 100644 tests/qapi-schema/data-int.exit
create mode 100644 tests/qapi-schema/data-int.json
create mode 100644 tests/qapi-schema/data-int.out
create mode 100644 tests/qapi-schema/data-member-array-bad.err
create mode 100644 tests/qapi-schema/data-member-array-bad.exit
create mode 100644 tests/qapi-schema/data-member-array-bad.json
create mode 100644 tests/qapi-schema/data-member-array-bad.out
create mode 100644 tests/qapi-schema/data-member-array.err
create mode 100644 tests/qapi-schema/data-member-array.exit
create mode 100644 tests/qapi-schema/data-member-array.json
create mode 100644 tests/qapi-schema/data-member-array.out
create mode 100644 tests/qapi-schema/data-member-unknown.err
create mode 100644 tests/qapi-schema/data-member-unknown.exit
create mode 100644 tests/qapi-schema/data-member-unknown.json
create mode 100644 tests/qapi-schema/data-member-unknown.out
create mode 100644 tests/qapi-schema/data-unknown.err
create mode 100644 tests/qapi-schema/data-unknown.exit
create mode 100644 tests/qapi-schema/data-unknown.json
create mode 100644 tests/qapi-schema/data-unknown.out
create mode 100644 tests/qapi-schema/nested-struct-data.err
create mode 100644 tests/qapi-schema/nested-struct-data.exit
create mode 100644 tests/qapi-schema/nested-struct-data.json
create mode 100644 tests/qapi-schema/nested-struct-data.out
create mode 100644 tests/qapi-schema/nested-struct-returns.err
create mode 100644 tests/qapi-schema/nested-struct-returns.exit
create mode 100644 tests/qapi-schema/nested-struct-returns.json
create mode 100644 tests/qapi-schema/nested-struct-returns.out
create mode 100644 tests/qapi-schema/returns-array-bad.err
create mode 100644 tests/qapi-schema/returns-array-bad.exit
create mode 100644 tests/qapi-schema/returns-array-bad.json
create mode 100644 tests/qapi-schema/returns-array-bad.out
create mode 100644 tests/qapi-schema/returns-int.err
create mode 100644 tests/qapi-schema/returns-int.exit
create mode 100644 tests/qapi-schema/returns-int.json
create mode 100644 tests/qapi-schema/returns-int.out
create mode 100644 tests/qapi-schema/returns-unknown.err
create mode 100644 tests/qapi-schema/returns-unknown.exit
create mode 100644 tests/qapi-schema/returns-unknown.json
create mode 100644 tests/qapi-schema/returns-unknown.out
diff --git a/tests/Makefile b/tests/Makefile
index 5e01952..6fe34f7 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -203,8 +203,12 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
double-data.json unknown-expr-key.json redefined-type.json \
redefined-command.json redefined-builtin.json redefined-event.json \
type-bypass.json type-bypass-no-gen.json type-bypass-bad-gen.json \
- missing-colon.json missing-comma-list.json \
- missing-comma-object.json non-objects.json \
+ data-array-empty.json data-array-unknown.json data-int.json \
+ data-unknown.json data-member-unknown.json data-member-array.json \
+ data-member-array-bad.json returns-array-bad.json returns-int.json \
+ returns-unknown.json missing-colon.json missing-comma-list.json \
+ missing-comma-object.json nested-struct-data.json \
+ nested-struct-returns.json non-objects.json \
qapi-schema-test.json quoted-structural-chars.json \
trailing-comma-list.json trailing-comma-object.json \
unclosed-list.json unclosed-object.json unclosed-string.json \
diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-array-empty.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
new file mode 100644
index 0000000..41b6c1e
--- /dev/null
+++ b/tests/qapi-schema/data-array-empty.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an array for data if it does not contain a known type
+{ 'command': 'oops', 'data': [ ] }
diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
new file mode 100644
index 0000000..67802be
--- /dev/null
+++ b/tests/qapi-schema/data-array-empty.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', [])])]
+[]
+[]
diff --git a/tests/qapi-schema/data-array-unknown.err b/tests/qapi-schema/data-array-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-array-unknown.exit b/tests/qapi-schema/data-array-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-array-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-array-unknown.json b/tests/qapi-schema/data-array-unknown.json
new file mode 100644
index 0000000..434fb5f
--- /dev/null
+++ b/tests/qapi-schema/data-array-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an array for data if it does not contain a known type
+{ 'command': 'oops', 'data': [ 'NoSuchType' ] }
diff --git a/tests/qapi-schema/data-array-unknown.out b/tests/qapi-schema/data-array-unknown.out
new file mode 100644
index 0000000..c3bc05e
--- /dev/null
+++ b/tests/qapi-schema/data-array-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', ['NoSuchType'])])]
+[]
+[]
diff --git a/tests/qapi-schema/data-int.err b/tests/qapi-schema/data-int.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-int.exit b/tests/qapi-schema/data-int.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-int.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-int.json b/tests/qapi-schema/data-int.json
new file mode 100644
index 0000000..37916e0
--- /dev/null
+++ b/tests/qapi-schema/data-int.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject commands where data is not an array or complex type
+{ 'command': 'oops', 'data': 'int' }
diff --git a/tests/qapi-schema/data-int.out b/tests/qapi-schema/data-int.out
new file mode 100644
index 0000000..e589a4f
--- /dev/null
+++ b/tests/qapi-schema/data-int.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', 'int')])]
+[]
+[]
diff --git a/tests/qapi-schema/data-member-array-bad.err b/tests/qapi-schema/data-member-array-bad.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-member-array-bad.exit b/tests/qapi-schema/data-member-array-bad.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-member-array-bad.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-member-array-bad.json b/tests/qapi-schema/data-member-array-bad.json
new file mode 100644
index 0000000..c954af1
--- /dev/null
+++ b/tests/qapi-schema/data-member-array-bad.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject data if it does not contain a valid array type
+{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
diff --git a/tests/qapi-schema/data-member-array-bad.out b/tests/qapi-schema/data-member-array-bad.out
new file mode 100644
index 0000000..0e00c41
--- /dev/null
+++ b/tests/qapi-schema/data-member-array-bad.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', [OrderedDict([('nested', 'str')])])]))])]
+[]
+[]
diff --git a/tests/qapi-schema/data-member-array.err b/tests/qapi-schema/data-member-array.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-member-array.exit b/tests/qapi-schema/data-member-array.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-member-array.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-member-array.json b/tests/qapi-schema/data-member-array.json
new file mode 100644
index 0000000..7cce276
--- /dev/null
+++ b/tests/qapi-schema/data-member-array.json
@@ -0,0 +1,4 @@
+# valid array members
+{ 'enum': 'abc', 'data': [ 'a', 'b', 'c' ] }
+{ 'type': 'def', 'data': { 'array': [ 'abc' ] } }
+{ 'command': 'okay', 'data': { 'member1': [ 'int' ], 'member2': [ 'def' ] } }
diff --git a/tests/qapi-schema/data-member-array.out b/tests/qapi-schema/data-member-array.out
new file mode 100644
index 0000000..8287120
--- /dev/null
+++ b/tests/qapi-schema/data-member-array.out
@@ -0,0 +1,5 @@
+[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
+ OrderedDict([('type', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
+ OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), ('member2', ['def'])]))])]
+[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
+[OrderedDict([('type', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
diff --git a/tests/qapi-schema/data-member-unknown.err b/tests/qapi-schema/data-member-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-member-unknown.exit b/tests/qapi-schema/data-member-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-member-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-member-unknown.json b/tests/qapi-schema/data-member-unknown.json
new file mode 100644
index 0000000..40e6252
--- /dev/null
+++ b/tests/qapi-schema/data-member-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject data if it does not contain a known type
+{ 'command': 'oops', 'data': { 'member': 'NoSuchType' } }
diff --git a/tests/qapi-schema/data-member-unknown.out b/tests/qapi-schema/data-member-unknown.out
new file mode 100644
index 0000000..36252a5
--- /dev/null
+++ b/tests/qapi-schema/data-member-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', 'NoSuchType')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/data-unknown.err b/tests/qapi-schema/data-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/data-unknown.exit b/tests/qapi-schema/data-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/data-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/data-unknown.json b/tests/qapi-schema/data-unknown.json
new file mode 100644
index 0000000..776bd34
--- /dev/null
+++ b/tests/qapi-schema/data-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject data if it does not contain a known type
+{ 'command': 'oops', 'data': 'NoSuchType' }
diff --git a/tests/qapi-schema/data-unknown.out b/tests/qapi-schema/data-unknown.out
new file mode 100644
index 0000000..2c60726
--- /dev/null
+++ b/tests/qapi-schema/data-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('data', 'NoSuchType')])]
+[]
+[]
diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/nested-struct-data.exit b/tests/qapi-schema/nested-struct-data.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-data.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/nested-struct-data.json b/tests/qapi-schema/nested-struct-data.json
new file mode 100644
index 0000000..0247c8c
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-data.json
@@ -0,0 +1,4 @@
+# FIXME: inline subtypes collide with our desired future use of defaults
+{ 'command': 'foo',
+ 'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' },
+ 'returns': {} }
diff --git a/tests/qapi-schema/nested-struct-data.out b/tests/qapi-schema/nested-struct-data.out
new file mode 100644
index 0000000..999cbb8
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-data.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('data', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')])), ('returns', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/nested-struct-returns.err b/tests/qapi-schema/nested-struct-returns.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/nested-struct-returns.exit b/tests/qapi-schema/nested-struct-returns.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-returns.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/nested-struct-returns.json b/tests/qapi-schema/nested-struct-returns.json
new file mode 100644
index 0000000..5a46840
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-returns.json
@@ -0,0 +1,3 @@
+# FIXME: inline subtypes collide with our desired future use of defaults
+{ 'command': 'foo',
+ 'returns': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }
diff --git a/tests/qapi-schema/nested-struct-returns.out b/tests/qapi-schema/nested-struct-returns.out
new file mode 100644
index 0000000..c53d23b
--- /dev/null
+++ b/tests/qapi-schema/nested-struct-returns.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'foo'), ('returns', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')]))])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-array-bad.err b/tests/qapi-schema/returns-array-bad.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-array-bad.exit b/tests/qapi-schema/returns-array-bad.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-array-bad.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-array-bad.json b/tests/qapi-schema/returns-array-bad.json
new file mode 100644
index 0000000..14882c1
--- /dev/null
+++ b/tests/qapi-schema/returns-array-bad.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject an array return that is not a single type
+{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
diff --git a/tests/qapi-schema/returns-array-bad.out b/tests/qapi-schema/returns-array-bad.out
new file mode 100644
index 0000000..eccad55
--- /dev/null
+++ b/tests/qapi-schema/returns-array-bad.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('returns', ['str', 'str'])])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-int.err b/tests/qapi-schema/returns-int.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-int.exit b/tests/qapi-schema/returns-int.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-int.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-int.json b/tests/qapi-schema/returns-int.json
new file mode 100644
index 0000000..7888fb1
--- /dev/null
+++ b/tests/qapi-schema/returns-int.json
@@ -0,0 +1,2 @@
+# It is okay (although not extensible) to return a non-dictionary
+{ 'command': 'okay', 'returns': 'int' }
diff --git a/tests/qapi-schema/returns-int.out b/tests/qapi-schema/returns-int.out
new file mode 100644
index 0000000..36b00a9
--- /dev/null
+++ b/tests/qapi-schema/returns-int.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'okay'), ('returns', 'int')])]
+[]
+[]
diff --git a/tests/qapi-schema/returns-unknown.err b/tests/qapi-schema/returns-unknown.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/returns-unknown.exit b/tests/qapi-schema/returns-unknown.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/returns-unknown.exit
@@ -0,0 +1 @@
+0
diff --git a/tests/qapi-schema/returns-unknown.json b/tests/qapi-schema/returns-unknown.json
new file mode 100644
index 0000000..61f20eb
--- /dev/null
+++ b/tests/qapi-schema/returns-unknown.json
@@ -0,0 +1,2 @@
+# FIXME: we should reject returns if it does not contain a known type
+{ 'command': 'oops', 'returns': 'NoSuchType' }
diff --git a/tests/qapi-schema/returns-unknown.out b/tests/qapi-schema/returns-unknown.out
new file mode 100644
index 0000000..a482c83
--- /dev/null
+++ b/tests/qapi-schema/returns-unknown.out
@@ -0,0 +1,3 @@
+[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
+[]
+[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (11 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-26 9:26 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass Eric Blake
` (6 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Now that we know every expression is valid with regards to
its keys, we can add further tests that those keys refer to
valid types. With this patch, all references to a type (the
'data': of command, type, union, and event, and the 'returns':
of command) must resolve to a builtin or another type declared
by the current qapi parse; this includes recursing into each
member of a data dictionary. Dealing with '**' and nested
sub-structs will be done in later patches.
Update the testsuite to match improved output.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 73 ++++++++++++++++++++++++++--
tests/qapi-schema/data-array-empty.err | 1 +
tests/qapi-schema/data-array-empty.exit | 2 +-
tests/qapi-schema/data-array-empty.json | 2 +-
tests/qapi-schema/data-array-empty.out | 3 --
tests/qapi-schema/data-array-unknown.err | 1 +
tests/qapi-schema/data-array-unknown.exit | 2 +-
tests/qapi-schema/data-array-unknown.json | 2 +-
tests/qapi-schema/data-array-unknown.out | 3 --
tests/qapi-schema/data-int.err | 1 +
tests/qapi-schema/data-int.exit | 2 +-
tests/qapi-schema/data-int.json | 2 +-
tests/qapi-schema/data-int.out | 3 --
tests/qapi-schema/data-member-array-bad.err | 1 +
tests/qapi-schema/data-member-array-bad.exit | 2 +-
tests/qapi-schema/data-member-array-bad.json | 2 +-
tests/qapi-schema/data-member-array-bad.out | 3 --
tests/qapi-schema/data-member-unknown.err | 1 +
tests/qapi-schema/data-member-unknown.exit | 2 +-
tests/qapi-schema/data-member-unknown.json | 2 +-
tests/qapi-schema/data-member-unknown.out | 3 --
tests/qapi-schema/data-unknown.err | 1 +
tests/qapi-schema/data-unknown.exit | 2 +-
tests/qapi-schema/data-unknown.json | 2 +-
tests/qapi-schema/data-unknown.out | 3 --
tests/qapi-schema/returns-array-bad.err | 1 +
tests/qapi-schema/returns-array-bad.exit | 2 +-
tests/qapi-schema/returns-array-bad.json | 2 +-
tests/qapi-schema/returns-array-bad.out | 3 --
tests/qapi-schema/returns-unknown.err | 1 +
tests/qapi-schema/returns-unknown.exit | 2 +-
tests/qapi-schema/returns-unknown.json | 2 +-
tests/qapi-schema/returns-unknown.out | 3 --
33 files changed, 93 insertions(+), 44 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index bf243fa..20c0ce9 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -255,6 +255,52 @@ def discriminator_find_enum_define(expr):
return find_enum(discriminator_type)
+def check_type(expr_info, source, data, allow_array = False,
+ allowed_names = [], allow_dict = True):
+ global all_types
+
+ if data is None:
+ return
+
+ if data == '**':
+ return
+
+ # Check if array type for data is okay
+ if isinstance(data, list):
+ if not allow_array:
+ raise QAPIExprError(expr_info,
+ "%s cannot be an array" % source)
+ if len(data) != 1 or not isinstance(data[0], basestring):
+ raise QAPIExprError(expr_info,
+ "%s: array type must contain single type name"
+ % source)
+ data = data[0]
+
+ # Check if type name for data is okay
+ if isinstance(data, basestring):
+ if not data in all_types:
+ raise QAPIExprError(expr_info,
+ "%s references unknown type '%s'"
+ % (source, data))
+ if not all_types[data] in allowed_names:
+ raise QAPIExprError(expr_info,
+ "%s cannot reference %s type '%s'"
+ % (source, all_types[data], data))
+ return
+
+ # data is a dictionary, check that each member is okay
+ if not isinstance(data, OrderedDict):
+ raise QAPIExprError(expr_info,
+ "%s should be a dictionary" % source)
+ if not allow_dict:
+ raise QAPIExprError(expr_info,
+ "%s should be a type name" % source)
+ for (key, value) in data.items():
+ check_type(expr_info, "member '%s' of %s" % (key, source), value,
+ allow_array=True,
+ allowed_names=['built-in', 'union', 'struct', 'enum'],
+ allow_dict=True)
+
def check_command(expr, expr_info):
global commands
name = expr['command']
@@ -262,6 +308,15 @@ def check_command(expr, expr_info):
raise QAPIExprError(expr_info,
"command '%s' is already defined" % name)
commands.append(name)
+ check_type(expr_info, "'data' for command '%s'" % name,
+ expr.get('data'), allow_array=True,
+ allowed_names=['union', 'struct'])
+ check_type(expr_info, "'base' for command '%s'" % name,
+ expr.get('base'), allowed_names=['struct'],
+ allow_dict=False)
+ check_type(expr_info, "'returns' for command '%s'" % name,
+ expr.get('returns'), allow_array=True,
+ allowed_names=['built-in', 'union', 'struct', 'enum'])
def check_event(expr, expr_info):
global events
@@ -271,6 +326,8 @@ def check_event(expr, expr_info):
raise QAPIExprError(expr_info,
"event '%s' is already defined" % name)
events.append(name)
+ check_type(expr_info, "'data' for event '%s'" % name,
+ expr.get('data'), allowed_names=['union', 'struct'])
if params:
for argname, argentry, optional, structured in parse_args(params):
if structured:
@@ -357,19 +414,27 @@ def check_enum(expr, expr_info):
% (name, member, values[key]))
values[key] = member
+def check_struct(expr, expr_info):
+ name = expr['type']
+ members = expr['data']
+
+ check_type(expr_info, "'data' for type '%s'" % name, members)
+
def check_exprs(schema):
for expr_elem in schema.exprs:
expr = expr_elem['expr']
info = expr_elem['info']
- if expr.has_key('union'):
- check_union(expr, info)
- if expr.has_key('event'):
- check_event(expr, info)
if expr.has_key('enum'):
check_enum(expr, info)
+ if expr.has_key('union'):
+ check_union(expr, info)
+ if expr.has_key('type'):
+ check_struct(expr, info)
if expr.has_key('command'):
check_command(expr, info)
+ if expr.has_key('event'):
+ check_event(expr, info)
def check_keys(expr_elem, meta, required, optional=[]):
expr = expr_elem['expr']
diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
index e69de29..94e046b 100644
--- a/tests/qapi-schema/data-array-empty.err
+++ b/tests/qapi-schema/data-array-empty.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-array-empty.json:2: 'data' for command 'oops': array type must contain single type name
diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-array-empty.exit
+++ b/tests/qapi-schema/data-array-empty.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
index 41b6c1e..fdd5612 100644
--- a/tests/qapi-schema/data-array-empty.json
+++ b/tests/qapi-schema/data-array-empty.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an array for data if it does not contain a known type
+# we reject an array for data if it does not contain a known type
{ 'command': 'oops', 'data': [ ] }
diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
index 67802be..e69de29 100644
--- a/tests/qapi-schema/data-array-empty.out
+++ b/tests/qapi-schema/data-array-empty.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', [])])]
-[]
-[]
diff --git a/tests/qapi-schema/data-array-unknown.err b/tests/qapi-schema/data-array-unknown.err
index e69de29..a66c2d6 100644
--- a/tests/qapi-schema/data-array-unknown.err
+++ b/tests/qapi-schema/data-array-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-array-unknown.json:2: 'data' for command 'oops' references unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/data-array-unknown.exit b/tests/qapi-schema/data-array-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-array-unknown.exit
+++ b/tests/qapi-schema/data-array-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-array-unknown.json b/tests/qapi-schema/data-array-unknown.json
index 434fb5f..9c75b96 100644
--- a/tests/qapi-schema/data-array-unknown.json
+++ b/tests/qapi-schema/data-array-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an array for data if it does not contain a known type
+# we reject an array for data if it does not contain a known type
{ 'command': 'oops', 'data': [ 'NoSuchType' ] }
diff --git a/tests/qapi-schema/data-array-unknown.out b/tests/qapi-schema/data-array-unknown.out
index c3bc05e..e69de29 100644
--- a/tests/qapi-schema/data-array-unknown.out
+++ b/tests/qapi-schema/data-array-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', ['NoSuchType'])])]
-[]
-[]
diff --git a/tests/qapi-schema/data-int.err b/tests/qapi-schema/data-int.err
index e69de29..e1d3ed5 100644
--- a/tests/qapi-schema/data-int.err
+++ b/tests/qapi-schema/data-int.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-int.json:2: 'data' for command 'oops' cannot reference built-in type 'int'
diff --git a/tests/qapi-schema/data-int.exit b/tests/qapi-schema/data-int.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-int.exit
+++ b/tests/qapi-schema/data-int.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-int.json b/tests/qapi-schema/data-int.json
index 37916e0..a334d92 100644
--- a/tests/qapi-schema/data-int.json
+++ b/tests/qapi-schema/data-int.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject commands where data is not an array or complex type
+# we reject commands where data is not an array or complex type
{ 'command': 'oops', 'data': 'int' }
diff --git a/tests/qapi-schema/data-int.out b/tests/qapi-schema/data-int.out
index e589a4f..e69de29 100644
--- a/tests/qapi-schema/data-int.out
+++ b/tests/qapi-schema/data-int.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', 'int')])]
-[]
-[]
diff --git a/tests/qapi-schema/data-member-array-bad.err b/tests/qapi-schema/data-member-array-bad.err
index e69de29..ac45442 100644
--- a/tests/qapi-schema/data-member-array-bad.err
+++ b/tests/qapi-schema/data-member-array-bad.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-member-array-bad.json:2: member 'member' of 'data' for command 'oops': array type must contain single type name
diff --git a/tests/qapi-schema/data-member-array-bad.exit b/tests/qapi-schema/data-member-array-bad.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-member-array-bad.exit
+++ b/tests/qapi-schema/data-member-array-bad.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-member-array-bad.json b/tests/qapi-schema/data-member-array-bad.json
index c954af1..b2ff144 100644
--- a/tests/qapi-schema/data-member-array-bad.json
+++ b/tests/qapi-schema/data-member-array-bad.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject data if it does not contain a valid array type
+# we reject data if it does not contain a valid array type
{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
diff --git a/tests/qapi-schema/data-member-array-bad.out b/tests/qapi-schema/data-member-array-bad.out
index 0e00c41..e69de29 100644
--- a/tests/qapi-schema/data-member-array-bad.out
+++ b/tests/qapi-schema/data-member-array-bad.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', [OrderedDict([('nested', 'str')])])]))])]
-[]
-[]
diff --git a/tests/qapi-schema/data-member-unknown.err b/tests/qapi-schema/data-member-unknown.err
index e69de29..89dce36 100644
--- a/tests/qapi-schema/data-member-unknown.err
+++ b/tests/qapi-schema/data-member-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-member-unknown.json:2: member 'member' of 'data' for command 'oops' references unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/data-member-unknown.exit b/tests/qapi-schema/data-member-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-member-unknown.exit
+++ b/tests/qapi-schema/data-member-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-member-unknown.json b/tests/qapi-schema/data-member-unknown.json
index 40e6252..342a41e 100644
--- a/tests/qapi-schema/data-member-unknown.json
+++ b/tests/qapi-schema/data-member-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject data if it does not contain a known type
+# we reject data if it does not contain a known type
{ 'command': 'oops', 'data': { 'member': 'NoSuchType' } }
diff --git a/tests/qapi-schema/data-member-unknown.out b/tests/qapi-schema/data-member-unknown.out
index 36252a5..e69de29 100644
--- a/tests/qapi-schema/data-member-unknown.out
+++ b/tests/qapi-schema/data-member-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', 'NoSuchType')]))])]
-[]
-[]
diff --git a/tests/qapi-schema/data-unknown.err b/tests/qapi-schema/data-unknown.err
index e69de29..d1a6086 100644
--- a/tests/qapi-schema/data-unknown.err
+++ b/tests/qapi-schema/data-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/data-unknown.json:2: 'data' for command 'oops' references unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/data-unknown.exit b/tests/qapi-schema/data-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/data-unknown.exit
+++ b/tests/qapi-schema/data-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/data-unknown.json b/tests/qapi-schema/data-unknown.json
index 776bd34..32aba43 100644
--- a/tests/qapi-schema/data-unknown.json
+++ b/tests/qapi-schema/data-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject data if it does not contain a known type
+# we reject data if it does not contain a known type
{ 'command': 'oops', 'data': 'NoSuchType' }
diff --git a/tests/qapi-schema/data-unknown.out b/tests/qapi-schema/data-unknown.out
index 2c60726..e69de29 100644
--- a/tests/qapi-schema/data-unknown.out
+++ b/tests/qapi-schema/data-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('data', 'NoSuchType')])]
-[]
-[]
diff --git a/tests/qapi-schema/returns-array-bad.err b/tests/qapi-schema/returns-array-bad.err
index e69de29..138095c 100644
--- a/tests/qapi-schema/returns-array-bad.err
+++ b/tests/qapi-schema/returns-array-bad.err
@@ -0,0 +1 @@
+tests/qapi-schema/returns-array-bad.json:2: 'returns' for command 'oops': array type must contain single type name
diff --git a/tests/qapi-schema/returns-array-bad.exit b/tests/qapi-schema/returns-array-bad.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/returns-array-bad.exit
+++ b/tests/qapi-schema/returns-array-bad.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/returns-array-bad.json b/tests/qapi-schema/returns-array-bad.json
index 14882c1..09b0b1f 100644
--- a/tests/qapi-schema/returns-array-bad.json
+++ b/tests/qapi-schema/returns-array-bad.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject an array return that is not a single type
+# we reject an array return that is not a single type
{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
diff --git a/tests/qapi-schema/returns-array-bad.out b/tests/qapi-schema/returns-array-bad.out
index eccad55..e69de29 100644
--- a/tests/qapi-schema/returns-array-bad.out
+++ b/tests/qapi-schema/returns-array-bad.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('returns', ['str', 'str'])])]
-[]
-[]
diff --git a/tests/qapi-schema/returns-unknown.err b/tests/qapi-schema/returns-unknown.err
index e69de29..962f27c 100644
--- a/tests/qapi-schema/returns-unknown.err
+++ b/tests/qapi-schema/returns-unknown.err
@@ -0,0 +1 @@
+tests/qapi-schema/returns-unknown.json:2: 'returns' for command 'oops' references unknown type 'NoSuchType'
diff --git a/tests/qapi-schema/returns-unknown.exit b/tests/qapi-schema/returns-unknown.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/returns-unknown.exit
+++ b/tests/qapi-schema/returns-unknown.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/returns-unknown.json b/tests/qapi-schema/returns-unknown.json
index 61f20eb..25bd498 100644
--- a/tests/qapi-schema/returns-unknown.json
+++ b/tests/qapi-schema/returns-unknown.json
@@ -1,2 +1,2 @@
-# FIXME: we should reject returns if it does not contain a known type
+# we reject returns if it does not contain a known type
{ 'command': 'oops', 'returns': 'NoSuchType' }
diff --git a/tests/qapi-schema/returns-unknown.out b/tests/qapi-schema/returns-unknown.out
index a482c83..e69de29 100644
--- a/tests/qapi-schema/returns-unknown.out
+++ b/tests/qapi-schema/returns-unknown.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
-[]
-[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (12 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types Eric Blake
@ 2014-09-19 22:24 ` Eric Blake
2014-09-29 8:38 ` Markus Armbruster
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 15/19] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
` (5 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:24 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
Now that we have a way to validate every type, we can also be
stricter about enforcing that callers that want to bypass
type safety in generated code. Prior to this patch, it didn't
matter what value was associated with the key 'gen', but it
looked odd that 'gen':'yes' could result in bypassing the
generated code. These changes also enforce the changes made
earlier in the series for documentation and consolidation of
using '**' as the wildcard type.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi.py | 21 ++++++++++++++++-----
tests/qapi-schema/type-bypass-bad-gen.err | 1 +
tests/qapi-schema/type-bypass-bad-gen.exit | 2 +-
tests/qapi-schema/type-bypass-bad-gen.json | 2 +-
tests/qapi-schema/type-bypass-bad-gen.out | 3 ---
tests/qapi-schema/type-bypass-no-gen.err | 1 +
tests/qapi-schema/type-bypass-no-gen.exit | 2 +-
tests/qapi-schema/type-bypass-no-gen.json | 2 +-
tests/qapi-schema/type-bypass-no-gen.out | 3 ---
9 files changed, 22 insertions(+), 15 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 20c0ce9..15972c6 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -256,13 +256,13 @@ def discriminator_find_enum_define(expr):
return find_enum(discriminator_type)
def check_type(expr_info, source, data, allow_array = False,
- allowed_names = [], allow_dict = True):
+ allowed_names = [], allow_dict = True, allow_star = False):
global all_types
if data is None:
return
- if data == '**':
+ if allow_star and data == '**':
return
# Check if array type for data is okay
@@ -278,6 +278,10 @@ def check_type(expr_info, source, data, allow_array = False,
# Check if type name for data is okay
if isinstance(data, basestring):
+ if data == '**':
+ raise QAPIExprError(expr_info,
+ "%s uses '**' but did not request 'gen':'no'"
+ % source)
if not data in all_types:
raise QAPIExprError(expr_info,
"%s references unknown type '%s'"
@@ -299,24 +303,27 @@ def check_type(expr_info, source, data, allow_array = False,
check_type(expr_info, "member '%s' of %s" % (key, source), value,
allow_array=True,
allowed_names=['built-in', 'union', 'struct', 'enum'],
- allow_dict=True)
+ allow_dict=True, allow_star=allow_star)
def check_command(expr, expr_info):
global commands
name = expr['command']
+ allow_star = expr.has_key('gen')
+
if name in commands:
raise QAPIExprError(expr_info,
"command '%s' is already defined" % name)
commands.append(name)
check_type(expr_info, "'data' for command '%s'" % name,
expr.get('data'), allow_array=True,
- allowed_names=['union', 'struct'])
+ allowed_names=['union', 'struct'], allow_star=allow_star)
check_type(expr_info, "'base' for command '%s'" % name,
expr.get('base'), allowed_names=['struct'],
allow_dict=False)
check_type(expr_info, "'returns' for command '%s'" % name,
expr.get('returns'), allow_array=True,
- allowed_names=['built-in', 'union', 'struct', 'enum'])
+ allowed_names=['built-in', 'union', 'struct', 'enum'],
+ allow_star=allow_star)
def check_event(expr, expr_info):
global events
@@ -450,6 +457,10 @@ def check_keys(expr_elem, meta, required, optional=[]):
raise QAPIExprError(info,
"%s '%s' has unknown key '%s'"
% (meta, name, key))
+ if (key == 'gen' or key == 'success-response') and value != 'no':
+ raise QAPIExprError(info,
+ "'%s' of %s '%s' should only have value 'no'"
+ % (key, meta, name))
for key in required:
if not expr.has_key(key):
raise QAPIExprError(info,
diff --git a/tests/qapi-schema/type-bypass-bad-gen.err b/tests/qapi-schema/type-bypass-bad-gen.err
index e69de29..c408364 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.err
+++ b/tests/qapi-schema/type-bypass-bad-gen.err
@@ -0,0 +1 @@
+tests/qapi-schema/type-bypass-bad-gen.json:2: 'gen' of command 'foo' should only have value 'no'
diff --git a/tests/qapi-schema/type-bypass-bad-gen.exit b/tests/qapi-schema/type-bypass-bad-gen.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.exit
+++ b/tests/qapi-schema/type-bypass-bad-gen.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/type-bypass-bad-gen.json b/tests/qapi-schema/type-bypass-bad-gen.json
index 6894526..5514b80 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.json
+++ b/tests/qapi-schema/type-bypass-bad-gen.json
@@ -1,2 +1,2 @@
-# FIXME: 'gen' should only appear with value 'no'
+# 'gen' should only appear with value 'no'
{ 'command': 'foo', 'gen': 'whatever' }
diff --git a/tests/qapi-schema/type-bypass-bad-gen.out b/tests/qapi-schema/type-bypass-bad-gen.out
index e678f2c..e69de29 100644
--- a/tests/qapi-schema/type-bypass-bad-gen.out
+++ b/tests/qapi-schema/type-bypass-bad-gen.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('gen', 'whatever')])]
-[]
-[]
diff --git a/tests/qapi-schema/type-bypass-no-gen.err b/tests/qapi-schema/type-bypass-no-gen.err
index e69de29..294ca2e 100644
--- a/tests/qapi-schema/type-bypass-no-gen.err
+++ b/tests/qapi-schema/type-bypass-no-gen.err
@@ -0,0 +1 @@
+tests/qapi-schema/type-bypass-no-gen.json:2: member 'arg' of 'data' for command 'unsafe' uses '**' but did not request 'gen':'no'
diff --git a/tests/qapi-schema/type-bypass-no-gen.exit b/tests/qapi-schema/type-bypass-no-gen.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/type-bypass-no-gen.exit
+++ b/tests/qapi-schema/type-bypass-no-gen.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/type-bypass-no-gen.json b/tests/qapi-schema/type-bypass-no-gen.json
index 72c817f..49b5742 100644
--- a/tests/qapi-schema/type-bypass-no-gen.json
+++ b/tests/qapi-schema/type-bypass-no-gen.json
@@ -1,2 +1,2 @@
-# FIXME: type bypass should only work with 'gen':'no'
+# type bypass only works with 'gen':'no'
{ 'command': 'unsafe', 'data': { 'arg': '**' }, 'returns': '**' }
diff --git a/tests/qapi-schema/type-bypass-no-gen.out b/tests/qapi-schema/type-bypass-no-gen.out
index 8b2a9ac..e69de29 100644
--- a/tests/qapi-schema/type-bypass-no-gen.out
+++ b/tests/qapi-schema/type-bypass-no-gen.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'unsafe'), ('data', OrderedDict([('arg', '**')])), ('returns', '**')])]
-[]
-[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 15/19] qapi: Merge UserDefTwo and UserDefNested in tests
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (13 preceding siblings ...)
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass Eric Blake
@ 2014-09-19 22:25 ` Eric Blake
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 16/19] qapi: Drop tests for inline subtypes Eric Blake
` (4 subsequent siblings)
19 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
In the testsuite, UserDefTwo and UserDefNested were identical
types other than the member names. Reduce code duplication by
having just one type, and choose names that also favor reuse.
This will also make it easier for a later patch to get rid of
nested inline subtypes in QAPI. When touching code to add new
allocations, also convert existing allocations to consistently
prefer typesafe g_new0 over g_malloc0.
Ensure that 'make check-qapi-schema check-unit' still passes.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qapi-schema/qapi-schema-test.json | 10 +----
tests/qapi-schema/qapi-schema-test.out | 6 +--
tests/test-qmp-commands.c | 32 +++++++--------
tests/test-qmp-input-strict.c | 19 +++++----
tests/test-qmp-input-visitor.c | 19 +++++----
tests/test-qmp-output-visitor.c | 50 +++++++++++------------
tests/test-visitor-serialization.c | 71 +++++++++++++++++----------------
7 files changed, 103 insertions(+), 104 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index ab4d3d9..2e13f09 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -15,16 +15,10 @@
'data': { 'string': 'str', '*enum1': 'EnumOne' } }
{ 'type': 'UserDefTwo',
- 'data': { 'string': 'str',
- 'dict': { 'string': 'str',
- 'dict': { 'userdef': 'UserDefOne', 'string': 'str' },
- '*dict2': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
-
-{ 'type': 'UserDefNested',
'data': { 'string0': 'str',
'dict1': { 'string1': 'str',
- 'dict2': { 'userdef1': 'UserDefOne', 'string2': 'str' },
- '*dict3': { 'userdef2': 'UserDefOne', 'string3': 'str' } } } }
+ 'dict2': { 'userdef': 'UserDefOne', 'string': 'str' },
+ '*dict3': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
# for testing unions
{ 'type': 'UserDefA',
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 95e9899..f94bbcc 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -2,8 +2,7 @@
OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string', 'str'), ('dict', OrderedDict([('string', 'str'), ('dict', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
- OrderedDict([('type', 'UserDefNested'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef1', 'UserDefOne'), ('string2', 'str')])), ('*dict3', OrderedDict([('userdef2', 'UserDefOne'), ('string3', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('union', 'UserDefUnion'), ('base', 'UserDefZero'), ('data', OrderedDict([('a', 'UserDefA'), ('b', 'UserDefB')]))]),
@@ -28,8 +27,7 @@
[OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string', 'str'), ('dict', OrderedDict([('string', 'str'), ('dict', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
- OrderedDict([('type', 'UserDefNested'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef1', 'UserDefOne'), ('string2', 'str')])), ('*dict3', OrderedDict([('userdef2', 'UserDefOne'), ('string3', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefUnionBase'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 554e222..9189cd2 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -32,13 +32,13 @@ UserDefTwo *qmp_user_def_cmd2(UserDefOne *ud1a,
ud1d->base->integer = has_udb1 ? ud1b->base->integer : 0;
ret = g_malloc0(sizeof(UserDefTwo));
- ret->string = strdup("blah1");
- ret->dict.string = strdup("blah2");
- ret->dict.dict.userdef = ud1c;
- ret->dict.dict.string = strdup("blah3");
- ret->dict.has_dict2 = true;
- ret->dict.dict2.userdef = ud1d;
- ret->dict.dict2.string = strdup("blah4");
+ ret->string0 = strdup("blah1");
+ ret->dict1.string1 = strdup("blah2");
+ ret->dict1.dict2.userdef = ud1c;
+ ret->dict1.dict2.string = strdup("blah3");
+ ret->dict1.has_dict3 = true;
+ ret->dict1.dict3.userdef = ud1d;
+ ret->dict1.dict3.string = strdup("blah4");
return ret;
}
@@ -120,15 +120,15 @@ static void test_dispatch_cmd_io(void)
ret = qobject_to_qdict(test_qmp_dispatch(req));
- assert(!strcmp(qdict_get_str(ret, "string"), "blah1"));
- ret_dict = qdict_get_qdict(ret, "dict");
- assert(!strcmp(qdict_get_str(ret_dict, "string"), "blah2"));
- ret_dict_dict = qdict_get_qdict(ret_dict, "dict");
+ assert(!strcmp(qdict_get_str(ret, "string0"), "blah1"));
+ ret_dict = qdict_get_qdict(ret, "dict1");
+ assert(!strcmp(qdict_get_str(ret_dict, "string1"), "blah2"));
+ ret_dict_dict = qdict_get_qdict(ret_dict, "dict2");
ret_dict_dict_userdef = qdict_get_qdict(ret_dict_dict, "userdef");
assert(qdict_get_int(ret_dict_dict_userdef, "integer") == 42);
assert(!strcmp(qdict_get_str(ret_dict_dict_userdef, "string"), "hello"));
assert(!strcmp(qdict_get_str(ret_dict_dict, "string"), "blah3"));
- ret_dict_dict2 = qdict_get_qdict(ret_dict, "dict2");
+ ret_dict_dict2 = qdict_get_qdict(ret_dict, "dict3");
ret_dict_dict2_userdef = qdict_get_qdict(ret_dict_dict2, "userdef");
assert(qdict_get_int(ret_dict_dict2_userdef, "integer") == 422);
assert(!strcmp(qdict_get_str(ret_dict_dict2_userdef, "string"), "hello2"));
@@ -192,7 +192,7 @@ static void test_dealloc_partial(void)
QmpInputVisitor *qiv;
ud2_dict = qdict_new();
- qdict_put_obj(ud2_dict, "string", QOBJECT(qstring_from_str(text)));
+ qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
qiv = qmp_input_visitor_new(QOBJECT(ud2_dict));
visit_type_UserDefTwo(qmp_input_get_visitor(qiv), &ud2, NULL, &err);
@@ -202,9 +202,9 @@ static void test_dealloc_partial(void)
/* verify partial success */
assert(ud2 != NULL);
- assert(ud2->string != NULL);
- assert(strcmp(ud2->string, text) == 0);
- assert(ud2->dict.dict.userdef == NULL);
+ assert(ud2->string0 != NULL);
+ assert(strcmp(ud2->string0, text) == 0);
+ assert(ud2->dict1.dict2.userdef == NULL);
/* confirm & release construction error */
assert(err != NULL);
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 0f77003..29d4f82 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -1,7 +1,7 @@
/*
* QMP Input Visitor unit-tests (strict mode).
*
- * Copyright (C) 2011-2012 Red Hat Inc.
+ * Copyright (C) 2011-2012, 2014 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -116,15 +116,18 @@ static void test_validate_struct(TestInputVisitorData *data,
static void test_validate_struct_nested(TestInputVisitorData *data,
const void *unused)
{
- UserDefNested *udp = NULL;
+ UserDefTwo *udp = NULL;
Error *err = NULL;
Visitor *v;
- v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string' }, 'string2': 'string2'}}}");
+ v = validate_test_init(data, "{ 'string0': 'string0', "
+ "'dict1': { 'string1': 'string1', "
+ "'dict2': { 'userdef': { 'integer': 42, "
+ "'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefNested(v, &udp, NULL, &err);
+ visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(!err);
- qapi_free_UserDefNested(udp);
+ qapi_free_UserDefTwo(udp);
}
static void test_validate_list(TestInputVisitorData *data,
@@ -207,15 +210,15 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
static void test_validate_fail_struct_nested(TestInputVisitorData *data,
const void *unused)
{
- UserDefNested *udp = NULL;
+ UserDefTwo *udp = NULL;
Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");
- visit_type_UserDefNested(v, &udp, NULL, &err);
+ visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(err);
- qapi_free_UserDefNested(udp);
+ qapi_free_UserDefTwo(udp);
}
static void test_validate_fail_list(TestInputVisitorData *data,
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 1c8e872..3e1da9d 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -1,7 +1,7 @@
/*
* QMP Input Visitor unit-tests.
*
- * Copyright (C) 2011 Red Hat Inc.
+ * Copyright (C) 2011, 2014 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -248,23 +248,26 @@ static void check_and_free_str(char *str, const char *cmp)
static void test_visitor_in_struct_nested(TestInputVisitorData *data,
const void *unused)
{
- UserDefNested *udp = NULL;
+ UserDefTwo *udp = NULL;
Error *err = NULL;
Visitor *v;
- v = visitor_input_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string' }, 'string2': 'string2'}}}");
+ v = visitor_input_test_init(data, "{ 'string0': 'string0', "
+ "'dict1': { 'string1': 'string1', "
+ "'dict2': { 'userdef': { 'integer': 42, "
+ "'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefNested(v, &udp, NULL, &err);
+ visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(!err);
check_and_free_str(udp->string0, "string0");
check_and_free_str(udp->dict1.string1, "string1");
- g_assert_cmpint(udp->dict1.dict2.userdef1->base->integer, ==, 42);
- check_and_free_str(udp->dict1.dict2.userdef1->string, "string");
- check_and_free_str(udp->dict1.dict2.string2, "string2");
+ g_assert_cmpint(udp->dict1.dict2.userdef->base->integer, ==, 42);
+ check_and_free_str(udp->dict1.dict2.userdef->string, "string");
+ check_and_free_str(udp->dict1.dict2.string, "string2");
g_assert(udp->dict1.has_dict3 == false);
- g_free(udp->dict1.dict2.userdef1);
+ g_free(udp->dict1.dict2.userdef);
g_free(udp);
}
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 74020de..6781a4f 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -1,7 +1,7 @@
/*
* QMP Output Visitor unit-tests.
*
- * Copyright (C) 2011 Red Hat Inc.
+ * Copyright (C) 2011, 2014 Red Hat Inc.
*
* Authors:
* Luiz Capitulino <lcapitulino@redhat.com>
@@ -234,7 +234,7 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
{
int64_t value = 42;
Error *err = NULL;
- UserDefNested *ud2;
+ UserDefTwo *ud2;
QObject *obj;
QDict *qdict, *dict1, *dict2, *dict3, *userdef;
const char *string = "user def string";
@@ -245,20 +245,20 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
ud2->string0 = g_strdup(strings[0]);
ud2->dict1.string1 = g_strdup(strings[1]);
- ud2->dict1.dict2.userdef1 = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict2.userdef1->string = g_strdup(string);
- ud2->dict1.dict2.userdef1->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict2.userdef1->base->integer = value;
- ud2->dict1.dict2.string2 = g_strdup(strings[2]);
+ ud2->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1.dict2.userdef->string = g_strdup(string);
+ ud2->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1.dict2.userdef->base->integer = value;
+ ud2->dict1.dict2.string = g_strdup(strings[2]);
ud2->dict1.has_dict3 = true;
- ud2->dict1.dict3.userdef2 = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict3.userdef2->string = g_strdup(string);
- ud2->dict1.dict3.userdef2->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict3.userdef2->base->integer = value;
- ud2->dict1.dict3.string3 = g_strdup(strings[3]);
+ ud2->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1.dict3.userdef->string = g_strdup(string);
+ ud2->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1.dict3.userdef->base->integer = value;
+ ud2->dict1.dict3.string = g_strdup(strings[3]);
- visit_type_UserDefNested(data->ov, &ud2, "unused", &err);
+ visit_type_UserDefTwo(data->ov, &ud2, "unused", &err);
g_assert(!err);
obj = qmp_output_get_qobject(data->qov);
@@ -275,22 +275,22 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
dict2 = qdict_get_qdict(dict1, "dict2");
g_assert_cmpint(qdict_size(dict2), ==, 2);
- g_assert_cmpstr(qdict_get_str(dict2, "string2"), ==, strings[2]);
- userdef = qdict_get_qdict(dict2, "userdef1");
+ g_assert_cmpstr(qdict_get_str(dict2, "string"), ==, strings[2]);
+ userdef = qdict_get_qdict(dict2, "userdef");
g_assert_cmpint(qdict_size(userdef), ==, 2);
g_assert_cmpint(qdict_get_int(userdef, "integer"), ==, value);
g_assert_cmpstr(qdict_get_str(userdef, "string"), ==, string);
dict3 = qdict_get_qdict(dict1, "dict3");
g_assert_cmpint(qdict_size(dict3), ==, 2);
- g_assert_cmpstr(qdict_get_str(dict3, "string3"), ==, strings[3]);
- userdef = qdict_get_qdict(dict3, "userdef2");
+ g_assert_cmpstr(qdict_get_str(dict3, "string"), ==, strings[3]);
+ userdef = qdict_get_qdict(dict3, "userdef");
g_assert_cmpint(qdict_size(userdef), ==, 2);
g_assert_cmpint(qdict_get_int(userdef, "integer"), ==, value);
g_assert_cmpstr(qdict_get_str(userdef, "string"), ==, string);
QDECREF(qdict);
- qapi_free_UserDefNested(ud2);
+ qapi_free_UserDefTwo(ud2);
}
static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
@@ -398,7 +398,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
const void *unused)
{
- UserDefNestedList *p, *head = NULL;
+ UserDefTwoList *p, *head = NULL;
const char string[] = "foo bar";
int i, max_count = 1024;
@@ -408,18 +408,18 @@ static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
p->value->string0 = g_strdup(string);
p->value->dict1.string1 = g_strdup(string);
- p->value->dict1.dict2.userdef1 = g_malloc0(sizeof(UserDefOne));
- p->value->dict1.dict2.userdef1->string = g_strdup(string);
- p->value->dict1.dict2.userdef1->base = g_new0(UserDefZero, 1);
- p->value->dict1.dict2.userdef1->base->integer = 42;
- p->value->dict1.dict2.string2 = g_strdup(string);
+ p->value->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
+ p->value->dict1.dict2.userdef->string = g_strdup(string);
+ p->value->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
+ p->value->dict1.dict2.userdef->base->integer = 42;
+ p->value->dict1.dict2.string = g_strdup(string);
p->value->dict1.has_dict3 = false;
p->next = head;
head = p;
}
- qapi_free_UserDefNestedList(head);
+ qapi_free_UserDefTwoList(head);
}
static void test_visitor_out_union(TestOutputVisitorData *data,
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 7ad1886..133f882 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -249,57 +249,57 @@ static void visit_struct(Visitor *v, void **native, Error **errp)
visit_type_TestStruct(v, (TestStruct **)native, NULL, errp);
}
-static UserDefNested *nested_struct_create(void)
+static UserDefTwo *nested_struct_create(void)
{
- UserDefNested *udnp = g_malloc0(sizeof(*udnp));
+ UserDefTwo *udnp = g_malloc0(sizeof(*udnp));
udnp->string0 = strdup("test_string0");
udnp->dict1.string1 = strdup("test_string1");
- udnp->dict1.dict2.userdef1 = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict2.userdef1->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict2.userdef1->base->integer = 42;
- udnp->dict1.dict2.userdef1->string = strdup("test_string");
- udnp->dict1.dict2.string2 = strdup("test_string2");
+ udnp->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1.dict2.userdef->base->integer = 42;
+ udnp->dict1.dict2.userdef->string = strdup("test_string");
+ udnp->dict1.dict2.string = strdup("test_string2");
udnp->dict1.has_dict3 = true;
- udnp->dict1.dict3.userdef2 = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict3.userdef2->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict3.userdef2->base->integer = 43;
- udnp->dict1.dict3.userdef2->string = strdup("test_string");
- udnp->dict1.dict3.string3 = strdup("test_string3");
+ udnp->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1.dict3.userdef->base->integer = 43;
+ udnp->dict1.dict3.userdef->string = strdup("test_string");
+ udnp->dict1.dict3.string = strdup("test_string3");
return udnp;
}
-static void nested_struct_compare(UserDefNested *udnp1, UserDefNested *udnp2)
+static void nested_struct_compare(UserDefTwo *udnp1, UserDefTwo *udnp2)
{
g_assert(udnp1);
g_assert(udnp2);
g_assert_cmpstr(udnp1->string0, ==, udnp2->string0);
g_assert_cmpstr(udnp1->dict1.string1, ==, udnp2->dict1.string1);
- g_assert_cmpint(udnp1->dict1.dict2.userdef1->base->integer, ==,
- udnp2->dict1.dict2.userdef1->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict2.userdef1->string, ==,
- udnp2->dict1.dict2.userdef1->string);
- g_assert_cmpstr(udnp1->dict1.dict2.string2, ==, udnp2->dict1.dict2.string2);
+ g_assert_cmpint(udnp1->dict1.dict2.userdef->base->integer, ==,
+ udnp2->dict1.dict2.userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1.dict2.userdef->string, ==,
+ udnp2->dict1.dict2.userdef->string);
+ g_assert_cmpstr(udnp1->dict1.dict2.string, ==, udnp2->dict1.dict2.string);
g_assert(udnp1->dict1.has_dict3 == udnp2->dict1.has_dict3);
- g_assert_cmpint(udnp1->dict1.dict3.userdef2->base->integer, ==,
- udnp2->dict1.dict3.userdef2->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict3.userdef2->string, ==,
- udnp2->dict1.dict3.userdef2->string);
- g_assert_cmpstr(udnp1->dict1.dict3.string3, ==, udnp2->dict1.dict3.string3);
+ g_assert_cmpint(udnp1->dict1.dict3.userdef->base->integer, ==,
+ udnp2->dict1.dict3.userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1.dict3.userdef->string, ==,
+ udnp2->dict1.dict3.userdef->string);
+ g_assert_cmpstr(udnp1->dict1.dict3.string, ==, udnp2->dict1.dict3.string);
}
-static void nested_struct_cleanup(UserDefNested *udnp)
+static void nested_struct_cleanup(UserDefTwo *udnp)
{
- qapi_free_UserDefNested(udnp);
+ qapi_free_UserDefTwo(udnp);
}
static void visit_nested_struct(Visitor *v, void **native, Error **errp)
{
- visit_type_UserDefNested(v, (UserDefNested **)native, NULL, errp);
+ visit_type_UserDefTwo(v, (UserDefTwo **)native, NULL, errp);
}
static void visit_nested_struct_list(Visitor *v, void **native, Error **errp)
{
- visit_type_UserDefNestedList(v, (UserDefNestedList **)native, NULL, errp);
+ visit_type_UserDefTwoList(v, (UserDefTwoList **)native, NULL, errp);
}
/* test cases */
@@ -715,13 +715,14 @@ static void test_nested_struct(gconstpointer opaque)
{
TestArgs *args = (TestArgs *) opaque;
const SerializeOps *ops = args->ops;
- UserDefNested *udnp = nested_struct_create();
- UserDefNested *udnp_copy = NULL;
+ UserDefTwo *udnp = nested_struct_create();
+ UserDefTwo *udnp_copy = NULL;
Error *err = NULL;
void *serialize_data;
-
+
ops->serialize(udnp, &serialize_data, visit_nested_struct, &err);
- ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct, &err);
+ ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct,
+ &err);
g_assert(err == NULL);
nested_struct_compare(udnp, udnp_copy);
@@ -737,13 +738,13 @@ static void test_nested_struct_list(gconstpointer opaque)
{
TestArgs *args = (TestArgs *) opaque;
const SerializeOps *ops = args->ops;
- UserDefNestedList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
+ UserDefTwoList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
Error *err = NULL;
void *serialize_data;
int i = 0;
for (i = 0; i < 8; i++) {
- tmp = g_malloc0(sizeof(UserDefNestedList));
+ tmp = g_malloc0(sizeof(UserDefTwoList));
tmp->value = nested_struct_create();
tmp->next = listp;
listp = tmp;
@@ -764,8 +765,8 @@ static void test_nested_struct_list(gconstpointer opaque)
listp_copy = listp_copy->next;
}
- qapi_free_UserDefNestedList(tmp);
- qapi_free_UserDefNestedList(tmp_copy);
+ qapi_free_UserDefTwoList(tmp);
+ qapi_free_UserDefTwoList(tmp_copy);
ops->cleanup(serialize_data);
g_free(args);
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 16/19] qapi: Drop tests for inline subtypes
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (14 preceding siblings ...)
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 15/19] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
@ 2014-09-19 22:25 ` Eric Blake
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 17/19] qapi: Drop inline subtype in query-version Eric Blake
` (3 subsequent siblings)
19 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline substructs conflicts with that goal.
This patch fixes the testsuite to avoid nested subtypes, by
breaking the nesting into explicit types. When touching code
to add new allocations, also convert existing allocations to
consistently prefer typesafe g_new0 over g_malloc0.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qapi-schema/qapi-schema-test.json | 12 ++++++--
tests/qapi-schema/qapi-schema-test.out | 8 +++--
tests/test-qmp-commands.c | 17 ++++++-----
tests/test-qmp-input-visitor.c | 14 +++++----
tests/test-qmp-output-visitor.c | 44 +++++++++++++++------------
tests/test-visitor-serialization.c | 53 ++++++++++++++++++---------------
6 files changed, 87 insertions(+), 61 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 2e13f09..1f922c1 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -14,11 +14,17 @@
'base': 'UserDefZero',
'data': { 'string': 'str', '*enum1': 'EnumOne' } }
+{ 'type': 'UserDefTwoDictDict',
+ 'data': { 'userdef': 'UserDefOne', 'string': 'str' } }
+
+{ 'type': 'UserDefTwoDict',
+ 'data': { 'string1': 'str',
+ 'dict2': 'UserDefTwoDictDict',
+ '*dict3': 'UserDefTwoDictDict' } }
+
{ 'type': 'UserDefTwo',
'data': { 'string0': 'str',
- 'dict1': { 'string1': 'str',
- 'dict2': { 'userdef': 'UserDefOne', 'string': 'str' },
- '*dict3': { 'userdef': 'UserDefOne', 'string': 'str' } } } }
+ 'dict1': 'UserDefTwoDict' } }
# for testing unions
{ 'type': 'UserDefA',
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index f94bbcc..1d7b53c 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -2,7 +2,9 @@
OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwoDictDict'), ('data', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]),
+ OrderedDict([('type', 'UserDefTwoDict'), ('data', OrderedDict([('string1', 'str'), ('dict2', 'UserDefTwoDictDict'), ('*dict3', 'UserDefTwoDictDict')]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', 'UserDefTwoDict')]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('union', 'UserDefUnion'), ('base', 'UserDefZero'), ('data', OrderedDict([('a', 'UserDefA'), ('b', 'UserDefB')]))]),
@@ -27,7 +29,9 @@
[OrderedDict([('type', 'NestedEnumsOne'), ('data', OrderedDict([('enum1', 'EnumOne'), ('*enum2', 'EnumOne'), ('enum3', 'EnumOne'), ('*enum4', 'EnumOne')]))]),
OrderedDict([('type', 'UserDefZero'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefOne'), ('base', 'UserDefZero'), ('data', OrderedDict([('string', 'str'), ('*enum1', 'EnumOne')]))]),
- OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', OrderedDict([('string1', 'str'), ('dict2', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')])), ('*dict3', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]))]))]),
+ OrderedDict([('type', 'UserDefTwoDictDict'), ('data', OrderedDict([('userdef', 'UserDefOne'), ('string', 'str')]))]),
+ OrderedDict([('type', 'UserDefTwoDict'), ('data', OrderedDict([('string1', 'str'), ('dict2', 'UserDefTwoDictDict'), ('*dict3', 'UserDefTwoDictDict')]))]),
+ OrderedDict([('type', 'UserDefTwo'), ('data', OrderedDict([('string0', 'str'), ('dict1', 'UserDefTwoDict')]))]),
OrderedDict([('type', 'UserDefA'), ('data', OrderedDict([('boolean', 'bool')]))]),
OrderedDict([('type', 'UserDefB'), ('data', OrderedDict([('integer', 'int')]))]),
OrderedDict([('type', 'UserDefUnionBase'), ('data', OrderedDict([('string', 'str'), ('enum1', 'EnumOne')]))]),
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 9189cd2..dc199d3 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -33,12 +33,15 @@ UserDefTwo *qmp_user_def_cmd2(UserDefOne *ud1a,
ret = g_malloc0(sizeof(UserDefTwo));
ret->string0 = strdup("blah1");
- ret->dict1.string1 = strdup("blah2");
- ret->dict1.dict2.userdef = ud1c;
- ret->dict1.dict2.string = strdup("blah3");
- ret->dict1.has_dict3 = true;
- ret->dict1.dict3.userdef = ud1d;
- ret->dict1.dict3.string = strdup("blah4");
+ ret->dict1 = g_malloc0(sizeof(UserDefTwoDict));
+ ret->dict1->string1 = strdup("blah2");
+ ret->dict1->dict2 = g_malloc0(sizeof(UserDefTwoDictDict));
+ ret->dict1->dict2->userdef = ud1c;
+ ret->dict1->dict2->string = strdup("blah3");
+ ret->dict1->dict3 = g_malloc0(sizeof(UserDefTwoDictDict));
+ ret->dict1->has_dict3 = true;
+ ret->dict1->dict3->userdef = ud1d;
+ ret->dict1->dict3->string = strdup("blah4");
return ret;
}
@@ -204,7 +207,7 @@ static void test_dealloc_partial(void)
assert(ud2 != NULL);
assert(ud2->string0 != NULL);
assert(strcmp(ud2->string0, text) == 0);
- assert(ud2->dict1.dict2.userdef == NULL);
+ assert(ud2->dict1 == NULL);
/* confirm & release construction error */
assert(err != NULL);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 3e1da9d..30a5441 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -261,13 +261,15 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
g_assert(!err);
check_and_free_str(udp->string0, "string0");
- check_and_free_str(udp->dict1.string1, "string1");
- g_assert_cmpint(udp->dict1.dict2.userdef->base->integer, ==, 42);
- check_and_free_str(udp->dict1.dict2.userdef->string, "string");
- check_and_free_str(udp->dict1.dict2.string, "string2");
- g_assert(udp->dict1.has_dict3 == false);
+ check_and_free_str(udp->dict1->string1, "string1");
+ g_assert_cmpint(udp->dict1->dict2->userdef->base->integer, ==, 42);
+ check_and_free_str(udp->dict1->dict2->userdef->string, "string");
+ check_and_free_str(udp->dict1->dict2->string, "string2");
+ g_assert(udp->dict1->has_dict3 == false);
- g_free(udp->dict1.dict2.userdef);
+ g_free(udp->dict1->dict2->userdef);
+ g_free(udp->dict1->dict2);
+ g_free(udp->dict1);
g_free(udp);
}
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 6781a4f..6c81fc7 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -244,19 +244,23 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
ud2 = g_malloc0(sizeof(*ud2));
ud2->string0 = g_strdup(strings[0]);
- ud2->dict1.string1 = g_strdup(strings[1]);
- ud2->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict2.userdef->string = g_strdup(string);
- ud2->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict2.userdef->base->integer = value;
- ud2->dict1.dict2.string = g_strdup(strings[2]);
+ ud2->dict1 = g_malloc0(sizeof(*ud2->dict1));
+ ud2->dict1->string1 = g_strdup(strings[1]);
- ud2->dict1.has_dict3 = true;
- ud2->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
- ud2->dict1.dict3.userdef->string = g_strdup(string);
- ud2->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
- ud2->dict1.dict3.userdef->base->integer = value;
- ud2->dict1.dict3.string = g_strdup(strings[3]);
+ ud2->dict1->dict2 = g_malloc0(sizeof(*ud2->dict1->dict2));
+ ud2->dict1->dict2->userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1->dict2->userdef->string = g_strdup(string);
+ ud2->dict1->dict2->userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1->dict2->userdef->base->integer = value;
+ ud2->dict1->dict2->string = g_strdup(strings[2]);
+
+ ud2->dict1->dict3 = g_malloc0(sizeof(*ud2->dict1->dict3));
+ ud2->dict1->has_dict3 = true;
+ ud2->dict1->dict3->userdef = g_malloc0(sizeof(UserDefOne));
+ ud2->dict1->dict3->userdef->string = g_strdup(string);
+ ud2->dict1->dict3->userdef->base = g_new0(UserDefZero, 1);
+ ud2->dict1->dict3->userdef->base->integer = value;
+ ud2->dict1->dict3->string = g_strdup(strings[3]);
visit_type_UserDefTwo(data->ov, &ud2, "unused", &err);
g_assert(!err);
@@ -407,13 +411,15 @@ static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
p->value = g_malloc0(sizeof(*p->value));
p->value->string0 = g_strdup(string);
- p->value->dict1.string1 = g_strdup(string);
- p->value->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
- p->value->dict1.dict2.userdef->string = g_strdup(string);
- p->value->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
- p->value->dict1.dict2.userdef->base->integer = 42;
- p->value->dict1.dict2.string = g_strdup(string);
- p->value->dict1.has_dict3 = false;
+ p->value->dict1 = g_malloc0(sizeof(UserDefTwoDict));
+ p->value->dict1->string1 = g_strdup(string);
+ p->value->dict1->dict2 = g_malloc0(sizeof(UserDefTwoDictDict));
+ p->value->dict1->dict2->userdef = g_malloc0(sizeof(UserDefOne));
+ p->value->dict1->dict2->userdef->string = g_strdup(string);
+ p->value->dict1->dict2->userdef->base = g_new0(UserDefZero, 1);
+ p->value->dict1->dict2->userdef->base->integer = 42;
+ p->value->dict1->dict2->string = g_strdup(string);
+ p->value->dict1->has_dict3 = false;
p->next = head;
head = p;
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 133f882..40169ff 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -253,18 +253,21 @@ static UserDefTwo *nested_struct_create(void)
{
UserDefTwo *udnp = g_malloc0(sizeof(*udnp));
udnp->string0 = strdup("test_string0");
- udnp->dict1.string1 = strdup("test_string1");
- udnp->dict1.dict2.userdef = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict2.userdef->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict2.userdef->base->integer = 42;
- udnp->dict1.dict2.userdef->string = strdup("test_string");
- udnp->dict1.dict2.string = strdup("test_string2");
- udnp->dict1.has_dict3 = true;
- udnp->dict1.dict3.userdef = g_malloc0(sizeof(UserDefOne));
- udnp->dict1.dict3.userdef->base = g_new0(UserDefZero, 1);
- udnp->dict1.dict3.userdef->base->integer = 43;
- udnp->dict1.dict3.userdef->string = strdup("test_string");
- udnp->dict1.dict3.string = strdup("test_string3");
+ udnp->dict1 = g_malloc0(sizeof(*udnp->dict1));
+ udnp->dict1->string1 = strdup("test_string1");
+ udnp->dict1->dict2 = g_malloc0(sizeof(*udnp->dict1->dict2));
+ udnp->dict1->dict2->userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1->dict2->userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1->dict2->userdef->base->integer = 42;
+ udnp->dict1->dict2->userdef->string = strdup("test_string");
+ udnp->dict1->dict2->string = strdup("test_string2");
+ udnp->dict1->dict3 = g_malloc0(sizeof(*udnp->dict1->dict3));
+ udnp->dict1->has_dict3 = true;
+ udnp->dict1->dict3->userdef = g_malloc0(sizeof(UserDefOne));
+ udnp->dict1->dict3->userdef->base = g_new0(UserDefZero, 1);
+ udnp->dict1->dict3->userdef->base->integer = 43;
+ udnp->dict1->dict3->userdef->string = strdup("test_string");
+ udnp->dict1->dict3->string = strdup("test_string3");
return udnp;
}
@@ -273,18 +276,20 @@ static void nested_struct_compare(UserDefTwo *udnp1, UserDefTwo *udnp2)
g_assert(udnp1);
g_assert(udnp2);
g_assert_cmpstr(udnp1->string0, ==, udnp2->string0);
- g_assert_cmpstr(udnp1->dict1.string1, ==, udnp2->dict1.string1);
- g_assert_cmpint(udnp1->dict1.dict2.userdef->base->integer, ==,
- udnp2->dict1.dict2.userdef->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict2.userdef->string, ==,
- udnp2->dict1.dict2.userdef->string);
- g_assert_cmpstr(udnp1->dict1.dict2.string, ==, udnp2->dict1.dict2.string);
- g_assert(udnp1->dict1.has_dict3 == udnp2->dict1.has_dict3);
- g_assert_cmpint(udnp1->dict1.dict3.userdef->base->integer, ==,
- udnp2->dict1.dict3.userdef->base->integer);
- g_assert_cmpstr(udnp1->dict1.dict3.userdef->string, ==,
- udnp2->dict1.dict3.userdef->string);
- g_assert_cmpstr(udnp1->dict1.dict3.string, ==, udnp2->dict1.dict3.string);
+ g_assert_cmpstr(udnp1->dict1->string1, ==, udnp2->dict1->string1);
+ g_assert_cmpint(udnp1->dict1->dict2->userdef->base->integer, ==,
+ udnp2->dict1->dict2->userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1->dict2->userdef->string, ==,
+ udnp2->dict1->dict2->userdef->string);
+ g_assert_cmpstr(udnp1->dict1->dict2->string, ==,
+ udnp2->dict1->dict2->string);
+ g_assert(udnp1->dict1->has_dict3 == udnp2->dict1->has_dict3);
+ g_assert_cmpint(udnp1->dict1->dict3->userdef->base->integer, ==,
+ udnp2->dict1->dict3->userdef->base->integer);
+ g_assert_cmpstr(udnp1->dict1->dict3->userdef->string, ==,
+ udnp2->dict1->dict3->userdef->string);
+ g_assert_cmpstr(udnp1->dict1->dict3->string, ==,
+ udnp2->dict1->dict3->string);
}
static void nested_struct_cleanup(UserDefTwo *udnp)
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 17/19] qapi: Drop inline subtype in query-version
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (15 preceding siblings ...)
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 16/19] qapi: Drop tests for inline subtypes Eric Blake
@ 2014-09-19 22:25 ` Eric Blake
2014-09-30 17:40 ` Markus Armbruster
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 18/19] qapi: Drop inline subtype in query-pci Eric Blake
` (2 subsequent siblings)
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline substructs conflicts with that goal.
This patch fixes one of only two commands relying on nested
subtypes, by breaking the nesting into an explicit type. The
QMP wire format is unaffected by this change.
Prefer the safer g_new0() while making the conversion.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
hmp.c | 2 +-
qapi/common.json | 26 +++++++++++++++++++-------
qmp.c | 9 +++++----
3 files changed, 25 insertions(+), 12 deletions(-)
diff --git a/hmp.c b/hmp.c
index 40a90da..23fc3da 100644
--- a/hmp.c
+++ b/hmp.c
@@ -55,7 +55,7 @@ void hmp_info_version(Monitor *mon, const QDict *qdict)
info = qmp_query_version(NULL);
monitor_printf(mon, "%" PRId64 ".%" PRId64 ".%" PRId64 "%s\n",
- info->qemu.major, info->qemu.minor, info->qemu.micro,
+ info->qemu->major, info->qemu->minor, info->qemu->micro,
info->package);
qapi_free_VersionInfo(info);
diff --git a/qapi/common.json b/qapi/common.json
index 4e9a21f..d007095 100644
--- a/qapi/common.json
+++ b/qapi/common.json
@@ -29,15 +29,28 @@
'DeviceNotActive', 'DeviceNotFound', 'KVMMissingCap' ] }
##
+# @VersionTriple
+#
+# A three-part version number.
+#
+# @qemu.major: The major version number.
+#
+# @qemu.minor: The minor version number.
+#
+# @qemu.micro: The micro version number.
+#
+# Since: 2.2
+##
+{ 'type': 'VersionTriple',
+ 'data': {'major': 'int', 'minor': 'int', 'micro': 'int'} }
+
+
+##
# @VersionInfo:
#
# A description of QEMU's version.
#
-# @qemu.major: The major version of QEMU
-#
-# @qemu.minor: The minor version of QEMU
-#
-# @qemu.micro: The micro version of QEMU. By current convention, a micro
+# @qemu: The version of QEMU. By current convention, a micro
# version of 50 signifies a development branch. A micro version
# greater than or equal to 90 signifies a release candidate for
# the next minor version. A micro version of less than 50
@@ -51,8 +64,7 @@
# Since: 0.14.0
##
{ 'type': 'VersionInfo',
- 'data': {'qemu': {'major': 'int', 'minor': 'int', 'micro': 'int'},
- 'package': 'str'} }
+ 'data': {'qemu': 'VersionTriple', 'package': 'str'} }
##
# @query-version:
diff --git a/qmp.c b/qmp.c
index c6767c4..24b658a 100644
--- a/qmp.c
+++ b/qmp.c
@@ -45,15 +45,16 @@ NameInfo *qmp_query_name(Error **errp)
VersionInfo *qmp_query_version(Error **errp)
{
- VersionInfo *info = g_malloc0(sizeof(*info));
+ VersionInfo *info = g_new0(VersionInfo, 1);
const char *version = QEMU_VERSION;
char *tmp;
- info->qemu.major = strtol(version, &tmp, 10);
+ info->qemu = g_new0(VersionTriple, 1);
+ info->qemu->major = strtol(version, &tmp, 10);
tmp++;
- info->qemu.minor = strtol(tmp, &tmp, 10);
+ info->qemu->minor = strtol(tmp, &tmp, 10);
tmp++;
- info->qemu.micro = strtol(tmp, &tmp, 10);
+ info->qemu->micro = strtol(tmp, &tmp, 10);
info->package = g_strdup(QEMU_PKGVERSION);
return info;
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 18/19] qapi: Drop inline subtype in query-pci
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (16 preceding siblings ...)
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 17/19] qapi: Drop inline subtype in query-version Eric Blake
@ 2014-09-19 22:25 ` Eric Blake
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 19/19] qapi: Drop support for inline subtypes Eric Blake
2014-09-26 15:42 ` [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
19 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline substructs conflicts with that goal.
This patch fixes one of only two commands relying on nested
subtypes, by breaking the nesting into an explicit type. The
QMP wire format is unaffected by this change.
Prefer the safer g_new0() while making the conversion, and reduce
some long lines.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
hmp.c | 26 ++++++++--------
hw/pci/pci.c | 42 ++++++++++++++------------
qapi-schema.json | 90 ++++++++++++++++++++++++++++++++++++++------------------
3 files changed, 98 insertions(+), 60 deletions(-)
diff --git a/hmp.c b/hmp.c
index 23fc3da..3d0c23f 100644
--- a/hmp.c
+++ b/hmp.c
@@ -553,14 +553,14 @@ static void hmp_info_pci_device(Monitor *mon, const PciDeviceInfo *dev)
dev->slot, dev->function);
monitor_printf(mon, " ");
- if (dev->class_info.has_desc) {
- monitor_printf(mon, "%s", dev->class_info.desc);
+ if (dev->class_info->has_desc) {
+ monitor_printf(mon, "%s", dev->class_info->desc);
} else {
- monitor_printf(mon, "Class %04" PRId64, dev->class_info.q_class);
+ monitor_printf(mon, "Class %04" PRId64, dev->class_info->q_class);
}
monitor_printf(mon, ": PCI device %04" PRIx64 ":%04" PRIx64 "\n",
- dev->id.vendor, dev->id.device);
+ dev->id->vendor, dev->id->device);
if (dev->has_irq) {
monitor_printf(mon, " IRQ %" PRId64 ".\n", dev->irq);
@@ -568,25 +568,25 @@ static void hmp_info_pci_device(Monitor *mon, const PciDeviceInfo *dev)
if (dev->has_pci_bridge) {
monitor_printf(mon, " BUS %" PRId64 ".\n",
- dev->pci_bridge->bus.number);
+ dev->pci_bridge->bus->number);
monitor_printf(mon, " secondary bus %" PRId64 ".\n",
- dev->pci_bridge->bus.secondary);
+ dev->pci_bridge->bus->secondary);
monitor_printf(mon, " subordinate bus %" PRId64 ".\n",
- dev->pci_bridge->bus.subordinate);
+ dev->pci_bridge->bus->subordinate);
monitor_printf(mon, " IO range [0x%04"PRIx64", 0x%04"PRIx64"]\n",
- dev->pci_bridge->bus.io_range->base,
- dev->pci_bridge->bus.io_range->limit);
+ dev->pci_bridge->bus->io_range->base,
+ dev->pci_bridge->bus->io_range->limit);
monitor_printf(mon,
" memory range [0x%08"PRIx64", 0x%08"PRIx64"]\n",
- dev->pci_bridge->bus.memory_range->base,
- dev->pci_bridge->bus.memory_range->limit);
+ dev->pci_bridge->bus->memory_range->base,
+ dev->pci_bridge->bus->memory_range->limit);
monitor_printf(mon, " prefetchable memory range "
"[0x%08"PRIx64", 0x%08"PRIx64"]\n",
- dev->pci_bridge->bus.prefetchable_range->base,
- dev->pci_bridge->bus.prefetchable_range->limit);
+ dev->pci_bridge->bus->prefetchable_range->base,
+ dev->pci_bridge->bus->prefetchable_range->limit);
}
for (region = dev->regions; region; region = region->next) {
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 6ce75aa..5922fd4 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -1441,24 +1441,26 @@ static PciBridgeInfo *qmp_query_pci_bridge(PCIDevice *dev, PCIBus *bus,
int bus_num)
{
PciBridgeInfo *info;
+ PciMemoryRange *range;
- info = g_malloc0(sizeof(*info));
+ info = g_new0(PciBridgeInfo, 1);
- info->bus.number = dev->config[PCI_PRIMARY_BUS];
- info->bus.secondary = dev->config[PCI_SECONDARY_BUS];
- info->bus.subordinate = dev->config[PCI_SUBORDINATE_BUS];
+ info->bus = g_new0(PciBusInfo, 1);
+ info->bus->number = dev->config[PCI_PRIMARY_BUS];
+ info->bus->secondary = dev->config[PCI_SECONDARY_BUS];
+ info->bus->subordinate = dev->config[PCI_SUBORDINATE_BUS];
- info->bus.io_range = g_malloc0(sizeof(*info->bus.io_range));
- info->bus.io_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_IO);
- info->bus.io_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_IO);
+ range = info->bus->io_range = g_new0(PciMemoryRange, 1);
+ range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_IO);
+ range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_IO);
- info->bus.memory_range = g_malloc0(sizeof(*info->bus.memory_range));
- info->bus.memory_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
- info->bus.memory_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
+ range = info->bus->memory_range = g_new0(PciMemoryRange, 1);
+ range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
+ range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_SPACE_MEMORY);
- info->bus.prefetchable_range = g_malloc0(sizeof(*info->bus.prefetchable_range));
- info->bus.prefetchable_range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
- info->bus.prefetchable_range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
+ range = info->bus->prefetchable_range = g_new0(PciMemoryRange, 1);
+ range->base = pci_bridge_get_base(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
+ range->limit = pci_bridge_get_limit(dev, PCI_BASE_ADDRESS_MEM_PREFETCH);
if (dev->config[PCI_SECONDARY_BUS] != 0) {
PCIBus *child_bus = pci_find_bus_nr(bus, dev->config[PCI_SECONDARY_BUS]);
@@ -1479,21 +1481,23 @@ static PciDeviceInfo *qmp_query_pci_device(PCIDevice *dev, PCIBus *bus,
uint8_t type;
int class;
- info = g_malloc0(sizeof(*info));
+ info = g_new0(PciDeviceInfo, 1);
info->bus = bus_num;
info->slot = PCI_SLOT(dev->devfn);
info->function = PCI_FUNC(dev->devfn);
+ info->class_info = g_new0(PciDeviceClass, 1);
class = pci_get_word(dev->config + PCI_CLASS_DEVICE);
- info->class_info.q_class = class;
+ info->class_info->q_class = class;
desc = get_class_desc(class);
if (desc->desc) {
- info->class_info.has_desc = true;
- info->class_info.desc = g_strdup(desc->desc);
+ info->class_info->has_desc = true;
+ info->class_info->desc = g_strdup(desc->desc);
}
- info->id.vendor = pci_get_word(dev->config + PCI_VENDOR_ID);
- info->id.device = pci_get_word(dev->config + PCI_DEVICE_ID);
+ info->id = g_new0(PciDeviceId, 1);
+ info->id->vendor = pci_get_word(dev->config + PCI_VENDOR_ID);
+ info->id->device = pci_get_word(dev->config + PCI_DEVICE_ID);
info->regions = qmp_query_pci_regions(dev);
info->qdev_id = g_strdup(dev->qdev.id ? dev->qdev.id : "");
diff --git a/qapi-schema.json b/qapi-schema.json
index 0b612bf..e00f91b 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -943,36 +943,75 @@
'*prefetch': 'bool', '*mem_type_64': 'bool' } }
##
+# @PciBusInfo:
+#
+# Information about a bus of a PCI Bridge device
+#
+# @number: primary bus interface number. This should be the number of the
+# bus the device resides on.
+#
+# @secondary: secondary bus interface number. This is the number of the
+# main bus for the bridge
+#
+# @subordinate: This is the highest number bus that resides below the
+# bridge.
+#
+# @io_range: The PIO range for all devices on this bridge
+#
+# @memory_range: The MMIO range for all devices on this bridge
+#
+# @prefetchable_range: The range of prefetchable MMIO for all devices on
+# this bridge
+#
+# Since: 2.2
+##
+{ 'type': 'PciBusInfo',
+ 'data': {'number': 'int', 'secondary': 'int', 'subordinate': 'int',
+ 'io_range': 'PciMemoryRange',
+ 'memory_range': 'PciMemoryRange',
+ 'prefetchable_range': 'PciMemoryRange' } }
+
+##
# @PciBridgeInfo:
#
# Information about a PCI Bridge device
#
-# @bus.number: primary bus interface number. This should be the number of the
-# bus the device resides on.
-#
-# @bus.secondary: secondary bus interface number. This is the number of the
-# main bus for the bridge
-#
-# @bus.subordinate: This is the highest number bus that resides below the
-# bridge.
-#
-# @bus.io_range: The PIO range for all devices on this bridge
-#
-# @bus.memory_range: The MMIO range for all devices on this bridge
-#
-# @bus.prefetchable_range: The range of prefetchable MMIO for all devices on
-# this bridge
+# @bus: information about the bus the device resides on
#
# @devices: a list of @PciDeviceInfo for each device on this bridge
#
# Since: 0.14.0
##
{ 'type': 'PciBridgeInfo',
- 'data': {'bus': { 'number': 'int', 'secondary': 'int', 'subordinate': 'int',
- 'io_range': 'PciMemoryRange',
- 'memory_range': 'PciMemoryRange',
- 'prefetchable_range': 'PciMemoryRange' },
- '*devices': ['PciDeviceInfo']} }
+ 'data': {'bus': 'PciBusInfo', '*devices': ['PciDeviceInfo']} }
+
+##
+# @PciDeviceClass:
+#
+# Information about the Class of a PCI device
+#
+# @desc: #optional a string description of the device's class
+#
+# @class: the class code of the device
+#
+# Since: 2.2
+##
+{ 'type': 'PciDeviceClass',
+ 'data': {'*desc': 'str', 'class': 'int'} }
+
+##
+# @PciDeviceId:
+#
+# Information about the Id of a PCI device
+#
+# @device: the PCI device id
+#
+# @vendor: the PCI vendor id
+#
+# Since: 2.2
+##
+{ 'type': 'PciDeviceId',
+ 'data': {'device': 'int', 'vendor': 'int'} }
##
# @PciDeviceInfo:
@@ -985,13 +1024,9 @@
#
# @function: the function of the slot used by the device
#
-# @class_info.desc: #optional a string description of the device's class
+# @class_info: the class of the device
#
-# @class_info.class: the class code of the device
-#
-# @id.device: the PCI device id
-#
-# @id.vendor: the PCI vendor id
+# @id: the PCI device id
#
# @irq: #optional if an IRQ is assigned to the device, the IRQ number
#
@@ -1008,8 +1043,7 @@
##
{ 'type': 'PciDeviceInfo',
'data': {'bus': 'int', 'slot': 'int', 'function': 'int',
- 'class_info': {'*desc': 'str', 'class': 'int'},
- 'id': {'device': 'int', 'vendor': 'int'},
+ 'class_info': 'PciDeviceClass', 'id': 'PciDeviceId',
'*irq': 'int', 'qdev_id': 'str', '*pci_bridge': 'PciBridgeInfo',
'regions': ['PciMemoryRegion']} }
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [Qemu-devel] [PATCH v4 19/19] qapi: Drop support for inline subtypes
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (17 preceding siblings ...)
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 18/19] qapi: Drop inline subtype in query-pci Eric Blake
@ 2014-09-19 22:25 ` Eric Blake
2014-09-30 17:47 ` Markus Armbruster
2014-09-26 15:42 ` [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
19 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-19 22:25 UTC (permalink / raw)
To: qemu-devel; +Cc: Luiz Capitulino, Fam Zheng, Markus Armbruster, wenchaoqemu
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline substructs conflicts with that goal.
Now that all commands have been fixed to avoid inline substructs,
nuke support for them, and turn it into a hard error. Update
the testsuite to reflect tighter parsing rules.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
scripts/qapi-commands.py | 8 +++---
scripts/qapi-event.py | 4 +--
scripts/qapi-types.py | 9 ++-----
scripts/qapi-visit.py | 37 ++++------------------------
scripts/qapi.py | 18 +++++---------
tests/qapi-schema/event-nest-struct.err | 2 +-
tests/qapi-schema/nested-struct-data.err | 1 +
tests/qapi-schema/nested-struct-data.exit | 2 +-
tests/qapi-schema/nested-struct-data.json | 2 +-
tests/qapi-schema/nested-struct-data.out | 3 ---
tests/qapi-schema/nested-struct-returns.err | 1 +
tests/qapi-schema/nested-struct-returns.exit | 2 +-
tests/qapi-schema/nested-struct-returns.json | 2 +-
tests/qapi-schema/nested-struct-returns.out | 3 ---
14 files changed, 26 insertions(+), 68 deletions(-)
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 053ba85..db81044 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -28,7 +28,7 @@ def type_visitor(name):
def generate_command_decl(name, args, ret_type):
arglist=""
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
argtype = c_type(argtype, is_param=True)
if optional:
arglist += "bool has_%s, " % c_var(argname)
@@ -53,7 +53,7 @@ def gen_sync_call(name, args, ret_type, indent=0):
retval=""
if ret_type:
retval = "retval = "
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
if optional:
arglist += "has_%s, " % c_var(argname)
arglist += "%s, " % (c_var(argname))
@@ -96,7 +96,7 @@ Visitor *v;
def gen_visitor_input_vars_decl(args):
ret = ""
push_indent()
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
if optional:
ret += mcgen('''
bool has_%(argname)s = false;
@@ -139,7 +139,7 @@ v = qapi_dealloc_get_visitor(md);
v = qmp_input_get_visitor(mi);
''')
- for argname, argtype, optional, structured in parse_args(args):
+ for argname, argtype, optional in parse_args(args):
if optional:
ret += mcgen('''
visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 601e307..47dc041 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -21,7 +21,7 @@ def _generate_event_api_name(event_name, params):
l = len(api_name)
if params:
- for argname, argentry, optional, structured in parse_args(params):
+ for argname, argentry, optional in parse_args(params):
if optional:
api_name += "bool has_%s,\n" % c_var(argname)
api_name += "".ljust(l)
@@ -93,7 +93,7 @@ def generate_event_implement(api_name, event_name, params):
""",
event_name = event_name)
- for argname, argentry, optional, structured in parse_args(params):
+ for argname, argentry, optional in parse_args(params):
if optional:
ret += mcgen("""
if (has_%(var)s) {
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index b463232..5d0e6e7 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -63,18 +63,13 @@ typedef struct %(name)sList
def generate_struct_fields(members):
ret = ''
- for argname, argentry, optional, structured in parse_args(members):
+ for argname, argentry, optional in parse_args(members):
if optional:
ret += mcgen('''
bool has_%(c_name)s;
''',
c_name=c_var(argname))
- if structured:
- push_indent()
- ret += generate_struct({ "field": argname, "data": argentry})
- pop_indent()
- else:
- ret += mcgen('''
+ ret += mcgen('''
%(c_type)s %(c_name)s;
''',
c_type=c_type(argentry), c_name=c_var(argname))
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index c129697..7674dd5 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -51,27 +51,6 @@ def generate_visit_struct_fields(name, field_prefix, fn_prefix, members, base =
else:
full_name = "%s_%s" % (name, fn_prefix)
- for argname, argentry, optional, structured in parse_args(members):
- if structured:
- if not fn_prefix:
- nested_fn_prefix = argname
- else:
- nested_fn_prefix = "%s_%s" % (fn_prefix, argname)
-
- nested_field_prefix = "%s%s." % (field_prefix, argname)
- ret += generate_visit_struct_fields(name, nested_field_prefix,
- nested_fn_prefix, argentry)
- ret += mcgen('''
-
-static void visit_type_%(full_name)s_field_%(c_name)s(Visitor *m, %(name)s **obj, Error **errp)
-{
-''',
- name=name, full_name=full_name, c_name=c_var(argname))
- ret += generate_visit_struct_body(full_name, argname, argentry)
- ret += mcgen('''
-}
-''')
-
if base:
ret += generate_visit_implicit_struct(base)
@@ -94,7 +73,7 @@ if (err) {
c_prefix=c_var(field_prefix),
type=type_name(base), c_name=c_var('base'))
- for argname, argentry, optional, structured in parse_args(members):
+ for argname, argentry, optional in parse_args(members):
if optional:
ret += mcgen('''
visit_optional(m, &(*obj)->%(c_prefix)shas_%(c_name)s, "%(name)s", &err);
@@ -104,18 +83,12 @@ if (!err && (*obj)->%(prefix)shas_%(c_name)s) {
c_name=c_var(argname), name=argname)
push_indent()
- if structured:
- ret += mcgen('''
-visit_type_%(full_name)s_field_%(c_name)s(m, obj, &err);
-''',
- full_name=full_name, c_name=c_var(argname))
- else:
- ret += mcgen('''
+ ret += mcgen('''
visit_type_%(type)s(m, &(*obj)->%(c_prefix)s%(c_name)s, "%(name)s", &err);
''',
- c_prefix=c_var(field_prefix), prefix=field_prefix,
- type=type_name(argentry), c_name=c_var(argname),
- name=argname)
+ c_prefix=c_var(field_prefix), prefix=field_prefix,
+ type=type_name(argentry), c_name=c_var(argname),
+ name=argname)
if optional:
pop_indent()
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 15972c6..17fca28 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -300,10 +300,12 @@ def check_type(expr_info, source, data, allow_array = False,
raise QAPIExprError(expr_info,
"%s should be a type name" % source)
for (key, value) in data.items():
+ # Todo: allow dictionaries to represent default values of
+ # an optional argument.
check_type(expr_info, "member '%s' of %s" % (key, source), value,
allow_array=True,
allowed_names=['built-in', 'union', 'struct', 'enum'],
- allow_dict=True, allow_star=allow_star)
+ allow_dict=False, allow_star=allow_star)
def check_command(expr, expr_info):
global commands
@@ -335,13 +337,6 @@ def check_event(expr, expr_info):
events.append(name)
check_type(expr_info, "'data' for event '%s'" % name,
expr.get('data'), allowed_names=['union', 'struct'])
- if params:
- for argname, argentry, optional, structured in parse_args(params):
- if structured:
- raise QAPIExprError(expr_info,
- "Nested structure define in event is not "
- "supported, event '%s', argname '%s'"
- % (expr['event'], argname))
def check_union(expr, expr_info):
name = expr['union']
@@ -537,13 +532,12 @@ def parse_args(typeinfo):
argname = member
argentry = typeinfo[member]
optional = False
- structured = False
if member.startswith('*'):
argname = member[1:]
optional = True
- if isinstance(argentry, OrderedDict):
- structured = True
- yield (argname, argentry, optional, structured)
+ # Todo: allow argentry to be OrderedDict, for providing the
+ # value of an optional argument.
+ yield (argname, argentry, optional)
def de_camel_case(name):
new_name = ''
diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
index 91bde1c..f17fbb7 100644
--- a/tests/qapi-schema/event-nest-struct.err
+++ b/tests/qapi-schema/event-nest-struct.err
@@ -1 +1 @@
-tests/qapi-schema/event-nest-struct.json:1: Nested structure define in event is not supported, event 'EVENT_A', argname 'a'
+tests/qapi-schema/event-nest-struct.json:1: member 'a' of 'data' for event 'EVENT_A' should be a type name
diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
index e69de29..1fc6687 100644
--- a/tests/qapi-schema/nested-struct-data.err
+++ b/tests/qapi-schema/nested-struct-data.err
@@ -0,0 +1 @@
+tests/qapi-schema/nested-struct-data.json:2: member 'a' of 'data' for command 'foo' should be a type name
diff --git a/tests/qapi-schema/nested-struct-data.exit b/tests/qapi-schema/nested-struct-data.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/nested-struct-data.exit
+++ b/tests/qapi-schema/nested-struct-data.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/nested-struct-data.json b/tests/qapi-schema/nested-struct-data.json
index 0247c8c..3d52d2b 100644
--- a/tests/qapi-schema/nested-struct-data.json
+++ b/tests/qapi-schema/nested-struct-data.json
@@ -1,4 +1,4 @@
-# FIXME: inline subtypes collide with our desired future use of defaults
+# inline subtypes collide with our desired future use of defaults
{ 'command': 'foo',
'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' },
'returns': {} }
diff --git a/tests/qapi-schema/nested-struct-data.out b/tests/qapi-schema/nested-struct-data.out
index 999cbb8..e69de29 100644
--- a/tests/qapi-schema/nested-struct-data.out
+++ b/tests/qapi-schema/nested-struct-data.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('data', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')])), ('returns', OrderedDict())])]
-[]
-[]
diff --git a/tests/qapi-schema/nested-struct-returns.err b/tests/qapi-schema/nested-struct-returns.err
index e69de29..f34bd92 100644
--- a/tests/qapi-schema/nested-struct-returns.err
+++ b/tests/qapi-schema/nested-struct-returns.err
@@ -0,0 +1 @@
+tests/qapi-schema/nested-struct-returns.json:2: member 'a' of 'returns' for command 'foo' should be a type name
diff --git a/tests/qapi-schema/nested-struct-returns.exit b/tests/qapi-schema/nested-struct-returns.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/nested-struct-returns.exit
+++ b/tests/qapi-schema/nested-struct-returns.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/nested-struct-returns.json b/tests/qapi-schema/nested-struct-returns.json
index 5a46840..d2cd047 100644
--- a/tests/qapi-schema/nested-struct-returns.json
+++ b/tests/qapi-schema/nested-struct-returns.json
@@ -1,3 +1,3 @@
-# FIXME: inline subtypes collide with our desired future use of defaults
+# inline subtypes collide with our desired future use of defaults
{ 'command': 'foo',
'returns': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }
diff --git a/tests/qapi-schema/nested-struct-returns.out b/tests/qapi-schema/nested-struct-returns.out
index c53d23b..e69de29 100644
--- a/tests/qapi-schema/nested-struct-returns.out
+++ b/tests/qapi-schema/nested-struct-returns.out
@@ -1,3 +0,0 @@
-[OrderedDict([('command', 'foo'), ('returns', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')]))])]
-[]
-[]
--
1.9.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations Eric Blake
@ 2014-09-22 12:37 ` Markus Armbruster
2014-09-22 16:53 ` Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-22 12:37 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Go into more details about the various types of valid expressions
> in a qapi schema, including tweaks to document fixes being done
> later in the current patch series.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> docs/qapi-code-gen.txt | 249 ++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 217 insertions(+), 32 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 8313ba6..3a79629 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -22,35 +22,136 @@ code are used.
> This file defines the types, commands, and events used by QMP. It should
> fully describe the interface used by QMP.
>
> -This file is designed to be loosely based on JSON although it's technically
> -executable Python. While dictionaries are used, they are parsed as
> -OrderedDicts so that ordering is preserved.
> +A QAPI file is designed to be loosely based on JSON although it's
> +technically executable Python. A valid QAPI schema consists of a list
> +of top-level expressions, with no commas between them. While
> +dictionaries are used, they are parsed as OrderedDicts so that
> +ordering is preserved; ordering doesn't matter for top-level
> +expressions, but does matter within 'data' members. QAPI input is
> +written using 'single quotes' instead of JSON's "double quotes" (in
> +contrast, QMP is strict JSON and only uses "double quotes"). As in
> +JSON, trailing commas are not permitted in arrays or dictionaries.
>
> -There are two basic syntaxes used, type definitions and command definitions.
> +Comments are allowed; anything between an unquoted # and the following
> +newline is ignored. Although there is not yet a documentation
> +generator, a form of stylized comments has developed for consistently
> +documenting details about an expression and when it was added to the
> +schema. The documentation is delimited between two lines of ##, then
> +the first line names the expression, an optional overview is provided,
> +then individual documentation about each member of 'data' is provided,
> +and finally, a 'Since: x.y.z' tag lists the release that introduced
> +the expression. Optional fields are tagged with the phrase
> +'#optional', often with their default value; and extensions added
> +after the expression was first released are also given a '(since
> +x.y.z)' comment. For example:
>
> -The first syntax defines a type and is represented by a dictionary. There are
> -three kinds of user-defined types that are supported: complex types,
> -enumeration types and union types.
> + ##
> + # @BlockStats:
> + #
> + # Statistics of a virtual block device or a block backing device.
> + #
> + # @device: #optional If the stats are for a virtual block device, the name
> + # corresponding to the virtual block device.
> + #
> + # @stats: A @BlockDeviceStats for the device.
> + #
> + # @parent: #optional This describes the file block device if it has one.
> + #
> + # @backing: #optional This describes the backing block device if it has one.
> + # (Since 2.0)
> + #
> + # Since: 0.14.0
> + ##
> + { 'type': 'BlockStats',
> + 'data': {'*device': 'str', 'stats': 'BlockDeviceStats',
> + '*parent': 'BlockStats',
> + '*backing': 'BlockStats'} }
>
> -Generally speaking, types definitions should always use CamelCase for the type
> -names. Command names should be all lower case with words separated by a hyphen.
> +The schema sets up a series of types, as well as commands and events
> +that will use those types. Forward references are allowed: the parser
> +scans in two passes, where the first pass learns all type names, and
> +the second validates the schema and generates the code. This allows
> +the definition of complex structs that can have mutually recursive
> +types, and allows for indefinite nesting of QMP that satisfies the
> +schema. A type name should not be defined more than once.
>
> +There are six top-level expressions recognized by the parser:
> +'include', 'command', 'type', 'enum', 'union', and 'event'. There are
> +several built-in types, such as 'int' and 'str'; additionally, the
> +top-level expressions can define complex types, enumeration types, and
> +union types. The 'command' expression can refer to existing types by
> +name, or list an anonymous type as a dictionary. Listing a type name
> +inside an array refers to a single-dimension array of that type;
> +multi-dimension arrays are not directly supported (although an array
> +of a complex struct that contains an array member is possible).
> +
> +Generally speaking, types definitions should always use CamelCase for
> +user-defined type names, while built-in types are lowercase. Command
> +names, and field names within a type, should be all lower case with
> +words separated by a hyphen. However, some existing older commands
> +and complex types use underscore; when extending such expressions,
> +consistency is preferred over blindly avoiding underscore. Event
> +names should be ALL_CAPS with words separated by underscore. The
> +special string '**' appears for some commands that manually perform
> +their own type checking rather than relying on the type-safe code
> +produced by the qapi code generators.
> +
> +Any command, type, or field name beginning with "x-" is marked
> +experimental, and may be withdrawn in a future release. Downstream
> +vendors may add extensions; such extensions should begin with a prefix
> +matching "__RFQDN_" (for the reverse-fully-qualified-domain-name of
> +the vendor), even if the rest of the command or field name uses dash
> +(example: __com.redhat_drive-mirror). Other than the dots used in
> +RFQDN of a downstream extension, all command, type, and field names
> +should begin with a letter, and contain only ASCII letters, numbers,
> +dash, and underscore. It is okay to reuse names that might collide
> +with programming languages; the generator will rename a field named
> +"default" in the QAPI to "q_default" in the generated C code.
> +
> +
> +=== Built-in Types ===
> +
> +The following types are built-in to the parser:
> + 'str' - arbitrary UTF-8 string
> + 'int' - 64-bit signed integer (although the C code may place further
> + restrictions on acceptable range)
> + 'number' - floating point number
> + 'bool' - JSON value of true or false
> + 'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
> + bit size
> + 'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
> + 'size' - like 'uint64', but allows scaled suffix from command line
> + visitor
>
> === Includes ===
>
> +Usage: { 'include': 'str' }
> +
> The QAPI schema definitions can be modularized using the 'include' directive:
>
> { 'include': 'path/to/file.json'}
>
> The directive is evaluated recursively, and include paths are relative to the
> -file using the directive. Multiple includes of the same file are safe.
> +file using the directive. Multiple includes of the same file are
> +safe. No other keys should appear in the expression, and the include
> +value should be a string.
> +
> +As a matter of style, it is a good idea to have all files be
> +self-contained, but at the moment, nothing prevents an included file
> +from making a forward reference to a type that is only introduced by
> +an outer file. The parser may be made stricter in the future to
> +prevent incomplete include files.
>
>
> === Complex types ===
>
> -A complex type is a dictionary containing a single key whose value is a
> -dictionary. This corresponds to a struct in C or an Object in JSON. An
> -example of a complex type is:
> +Usage: { 'type': 'str', 'data': 'dict', '*base': 'complex-type-name' }
> +
> +A complex type is a dictionary containing a single 'data' key whose
> +value is a dictionary. This corresponds to a struct in C or an Object
> +in JSON. Each value of the 'data' dictionary must be the name of a
> +complex, enum, union, or built-in type, or a one-element array
> +containing a type name. An example of a complex type is:
>
> { 'type': 'MyType',
> 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
> @@ -100,15 +201,42 @@ both fields like this:
> { "file": "/some/place/my-image",
> "backing": "/some/place/my-backing-file" }
>
> +
> === Enumeration types ===
>
> -An enumeration type is a dictionary containing a single key whose value is a
> -list of strings. An example enumeration is:
> +Usage: { 'enum': 'str', 'data': [ 'str' ] }
> +
> +An enumeration type is a dictionary containing a single 'data' key
> +whose value is a list of strings. An example enumeration is:
>
> { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }
>
> +Nothing prevents an empty enumeration, although it is probably not
> +useful. The list of strings should be lower case; if an enum name
> +represents multiple words, use '-' between words. The string 'max' is
> +not allowed as an enum value, and values should not be repeated.
> +
> +The enumeration values are passed as strings over the QMP protocol,
> +but are encoded as C enum integral values in generated code. While
> +the C code starts numbering at 0, it is better to use explicit
> +comparisons to enum values than implicit comparisons to 0; the C code
> +will also include a generated enum member ending in _MAX for tracking
> +the size of the enum, useful when using common functions for
> +converting between strings and enum values. Since the wire format
> +always passes by name, it is acceptable to reorder or add new
> +enumeration members in any location without breaking QMP clients;
> +however, removing enum values would break compatibility. For any
> +complex type that has a field that will only contain a finite set of
> +string values, using an enum type for that field is better than
> +open-coding the field to be type 'str'.
> +
> +
> === Union types ===
>
> +Usage: { 'union': 'str', 'data': 'dict', '*base': 'complex-type-name',
> + '*discriminator': 'enum-type-name' }
> +or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
> +
Suggest to split usage into simple union, flat union and anonymous
union, like this:
Usage: { 'union': 'str', 'data': 'dict', '*base': 'complex-type-name' }
or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
'*discriminator': 'enum-type-name' }
or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
> Union types are used to let the user choose between several different data
> types. A union type is defined using a dictionary as explained in the
> following paragraphs.
> @@ -153,8 +281,10 @@ And it looks like this on the wire:
>
> Flat union types avoid the nesting on the wire. They are used whenever a
> specific field of the base type is declared as the discriminator ('type' is
> -then no longer generated). The discriminator must be of enumeration type.
> -The above example can then be modified as follows:
> +then no longer generated). The discriminator must be of enumeration
> +type, and the keys of the 'data' dictionary must match the enumeration
> +keys (although not necessarily in the same order). The above example
> +can then be modified as follows:
>
> { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
> { 'type': 'BlockdevCommonOptions',
> @@ -200,23 +330,78 @@ This example allows using both of the following example objects:
>
> === Commands ===
>
> -Commands are defined by using a list containing three members. The first
> -member is the command name, the second member is a dictionary containing
> -arguments, and the third member is the return type.
> +Usage: { 'command': 'str', '*data': 'dict-or-complex-type-name',
> + '*returns': 'type',
> + '*gen': 'no', '*success-response': 'no' }
>
> -An example command is:
> +Commands are defined by using a dictionary containing several members,
> +where three members are most common.
Suggest paragraph break here.
> + The 'data' member is optional; if
> +absent, the command accepts an optional empty dictionary. If present,
> +it must be the string name of a complex type, a one-element array
> +containing the name of a complex type, or a dictionary that declares
> +an anonymous type with the same semantics as a 'type' expression, with
> +one exception noted below.
Suggest to say explicitly that 'data' specifies command arguments.
Suggest paragraph break here.
> + The 'returns' member is optional; if
> +absent, the command returns an empty dictionary. If present, it must
> +be the string name of a complex or built-in type, a one-element array
> +containing the name of a complex or built-in type, or a dictionary
> +that declares an anonymous type with the same semantics as a 'type'
> +expression, with one exception noted below.
> +
> +Although it is permitted to have the 'returns' member name a built-in
> +type, any command that does this cannot be extended to return
> +additional information in the future; thus, new commands should
> +strongly consider returning a dictionary-based type, even if it only
> +contains one field at the present. Besides, all commands return a
> +dictionary to report failure.
Suggest to say explicitly that 'returns' specifies the reply on success.
Suggest to replace the last sentence by a separate paragraph discussing
the reply on failure.
> +
> +Some example commands:
> +
> + { 'command': 'my-first-command',
> + 'data': { 'arg1': 'str', '*arg2': 'str' } }
> + { 'type': 'MyType', 'data': { '*value': 'str' } }
> + { 'command': 'my-second-command',
> + 'returns': [ 'MyType' ] }
> +
> +which would validate this QMP transaction:
> +
> + => { "execute": "my-first-command",
> + "arguments": { "arg1": "hello" } }
> + <= { "return": { } }
> + => { "execute": "my-second-command" }
> + <= { "return": [ { "value": "one" }, { } ] }
> +
> +In rare cases, QAPI cannot express a type-safe representation of a
> +corresponding QMP command. In these cases, if the command expression
> +includes the key 'gen' with value 'no', then the 'data' or 'returns'
The implementation actually ignores the value of 'gen', but specifying
it must be 'no' doesn't hurt.
> +member that intends to bypass generated type-safety and do its own
> +manual validation should use '**' rather than a valid type name.
> +Please try to avoid adding new commands that rely on this, and instead
> +use type-safe unions. For an example of bypass usage:
> +
> + { 'command': 'netdev_add',
> + 'data': {'type': 'str', 'id': 'str', '*props': '**'},
> + 'gen': 'no' }
> +
> +Normally, the QAPI schema is used to describe synchronous exchanges,
> +where a response is expected. But in some cases, the action of a
> +command is expected to change state in a way that a successful
> +response is not possible (a failure message still returns a
> +dictionary). In this case, the command expression should include the
> +optional key 'success-response' with value 'no'.
Learned something new here.
Suggest to change the parenthesis to something like (it still sends a
normal error reply on failure).
>
> - { 'command': 'my-command',
> - 'data': { 'arg1': 'str', '*arg2': 'str' },
> - 'returns': 'str' }
>
> === Events ===
>
> -Events are defined with the keyword 'event'. When 'data' is also specified,
> -additional info will be included in the event. Finally there will be C API
> -generated in qapi-event.h; when called by QEMU code, a message with timestamp
> -will be emitted on the wire. If timestamp is -1, it means failure to retrieve
> -host time.
> +Usage: { 'event': 'str', '*data': 'dict-or-complex-type-name' }
> +
> +Events are defined with the keyword 'event'. It is not allowed to
> +name an event 'MAX', since the generator also produces a C enumeration
> +of all event names with a generated _MAX value at the end.
One of the several places where the generator can thoughtlessly produce
clashing identifiers. You're documenting one of them, which is an
improvement of sorts.
> When
> +'data' is also specified, additional info will be included in the
> +event, with similar semantics to a 'type' expression. Finally there
> +will be C API generated in qapi-event.h; when called by QEMU code, a
> +message with timestamp will be emitted on the wire. If timestamp is
> +-1, it means failure to retrieve host time.
As far as I can tell, this can happen only if gettimeofday() fails.
Highly unlikely. But since the code could do it, the spec should
mention it.
>
> An example event is:
>
> @@ -311,7 +496,7 @@ Example:
> #ifndef EXAMPLE_QAPI_TYPES_H
> #define EXAMPLE_QAPI_TYPES_H
>
> -[Builtin types omitted...]
> +[Built-in types omitted...]
>
> typedef struct UserDefOne UserDefOne;
>
> @@ -324,7 +509,7 @@ Example:
> struct UserDefOneList *next;
> } UserDefOneList;
>
> -[Functions on builtin types omitted...]
> +[Functions on built-in types omitted...]
>
> struct UserDefOne
> {
> @@ -423,7 +608,7 @@ Example:
> #ifndef EXAMPLE_QAPI_VISIT_H
> #define EXAMPLE_QAPI_VISIT_H
>
> -[Visitors for builtin types omitted...]
> +[Visitors for built-in types omitted...]
>
> void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp);
> void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp);
Major improvement, thank you very much!
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile Eric Blake
@ 2014-09-22 12:40 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-22 12:40 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> tests/Makefile had a mix of TAB vs. 8-space indentation; given
> that it is a Makefile, TAB is more idiomatic even though in these
> particular cases the choice of whitespace didn't matter.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 03/19] qapi: Update docs given recent event, spacing fixes
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 03/19] qapi: Update docs given recent event, spacing fixes Eric Blake
@ 2014-09-22 12:40 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-22 12:40 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Commit 21cd70d added event support but didn't document what the
> generated code looks like. Commit 05dfb26 removed some unwanted
> spaces in the generated code, but didn't reflect those changes
> into the documentation. Finally, the docs start with a big
> disclaimer about QMP not using QAPI yet, which feels rather stale.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 05/19] qapi: Add some enum tests
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 05/19] qapi: Add some enum tests Eric Blake
@ 2014-09-22 12:43 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-22 12:43 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal well with enums
> that aren't up to par. Later patches will update the expected
> results as the generator is made stricter.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations
2014-09-22 12:37 ` Markus Armbruster
@ 2014-09-22 16:53 ` Eric Blake
0 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-22 16:53 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 3363 bytes --]
On 09/22/2014 06:37 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> === Union types ===
>>
>> +Usage: { 'union': 'str', 'data': 'dict', '*base': 'complex-type-name',
>> + '*discriminator': 'enum-type-name' }
>> +or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
>> +
>
> Suggest to split usage into simple union, flat union and anonymous
> union, like this:
>
> Usage: { 'union': 'str', 'data': 'dict', '*base': 'complex-type-name' }
^ simple
> or: { 'union': 'str', 'data': 'dict', 'base': 'complex-type-name',
> '*discriminator': 'enum-type-name' }
^ flat
> or: { 'union': 'str', 'data': 'dict', 'discriminator': {} }
^ anonymous
Except that someday, I'd like to have:
{ 'union': str', 'data': 'dict', 'discriminator': 'enum-type-name' }
which, when compared with the simple and flat versions, means that base
and discriminator are equally optional. But you are right that we don't
have that form yet, and that the code currently requires that if base is
specified, then discriminator is non-optional. I'll have to tweak this.
>> +
>> +In rare cases, QAPI cannot express a type-safe representation of a
>> +corresponding QMP command. In these cases, if the command expression
>> +includes the key 'gen' with value 'no', then the 'data' or 'returns'
>
> The implementation actually ignores the value of 'gen', but specifying
> it must be 'no' doesn't hurt.
Actually, see patch 14/19 later in the series, where I fix the code to
enforce that it must be 'no' :)
>
>> +member that intends to bypass generated type-safety and do its own
>> +manual validation should use '**' rather than a valid type name.
>> +Please try to avoid adding new commands that rely on this, and instead
>> +use type-safe unions. For an example of bypass usage:
>> +
>> + { 'command': 'netdev_add',
>> + 'data': {'type': 'str', 'id': 'str', '*props': '**'},
>> + 'gen': 'no' }
>> +
>> +Normally, the QAPI schema is used to describe synchronous exchanges,
>> +where a response is expected. But in some cases, the action of a
>> +command is expected to change state in a way that a successful
>> +response is not possible (a failure message still returns a
>> +dictionary). In this case, the command expression should include the
>> +optional key 'success-response' with value 'no'.
>
> Learned something new here.
To date, only qga uses this form.
>> +
>> +Events are defined with the keyword 'event'. It is not allowed to
>> +name an event 'MAX', since the generator also produces a C enumeration
>> +of all event names with a generated _MAX value at the end.
>
> One of the several places where the generator can thoughtlessly produce
> clashing identifiers. You're documenting one of them, which is an
> improvement of sorts.
I also enhance things later in the series to enforce this documentation :)
>
> Major improvement, thank you very much!
I've trimmed your other comments (such as suggested line breaks) because
I agree with them, and will incorporate them into either v5 (if the
series needs respinning) or a followup patch (if this is the only patch
that needs improvement).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 02/19] qapi: Ignore files created during make check
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 02/19] qapi: Ignore files created during make check Eric Blake
@ 2014-09-23 8:07 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-23 8:07 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> After an in-tree build and run of 'make check-{qapi-schema,unit}',
> I noticed some leftover files.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> Reviewed-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums Eric Blake
@ 2014-09-23 14:23 ` Markus Armbruster
2014-09-23 15:59 ` Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-23 14:23 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> The previous commit demonstrated that the generator had several
> flaws with less-than-perfect enums:
> - an enum that listed the same string twice (or two variant
> strings that map to the same C enum) ended up generating an
> invalid C enum
> - because the generator adds a _MAX terminator to each enum,
> the use of an enum member 'max' can also cause this clash
> - if an enum omits 'data', the generator left a python stack
> trace rather than a graceful message
> - an enum used a non-array 'data' was silently accepted by
> the parser
> - an enum that used non-string members in the 'data' member
> was silently accepted by the parser
>
> Add check_enum to cover these situations, and update testcases
> to match. While valid .json files won't trigger any of these
> cases, we might as well be nicer to developers that make a typo
> while trying to add new QAPI code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 32 ++++++++++++++++++++++++++++----
> tests/qapi-schema/enum-clash-member.err | 1 +
> tests/qapi-schema/enum-clash-member.exit | 2 +-
> tests/qapi-schema/enum-clash-member.json | 2 +-
> tests/qapi-schema/enum-clash-member.out | 3 ---
> tests/qapi-schema/enum-dict-member.err | 1 +
> tests/qapi-schema/enum-dict-member.exit | 2 +-
> tests/qapi-schema/enum-dict-member.json | 2 +-
> tests/qapi-schema/enum-dict-member.out | 3 ---
> tests/qapi-schema/enum-max-member.err | 1 +
> tests/qapi-schema/enum-max-member.exit | 2 +-
> tests/qapi-schema/enum-max-member.json | 4 ++--
> tests/qapi-schema/enum-max-member.out | 3 ---
> tests/qapi-schema/enum-missing-data.err | 7 +------
> tests/qapi-schema/enum-missing-data.json | 2 +-
> tests/qapi-schema/enum-wrong-data.err | 1 +
> tests/qapi-schema/enum-wrong-data.exit | 2 +-
> tests/qapi-schema/enum-wrong-data.json | 2 +-
> tests/qapi-schema/enum-wrong-data.out | 3 ---
> 19 files changed, 43 insertions(+), 32 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 77d46aa..85aa8bf 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -2,7 +2,7 @@
> # QAPI helper library
> #
> # Copyright IBM, Corp. 2011
> -# Copyright (c) 2013 Red Hat Inc.
> +# Copyright (c) 2013-2014 Red Hat Inc.
> #
> # Authors:
> # Anthony Liguori <aliguori@us.ibm.com>
> @@ -316,13 +316,37 @@ def check_union(expr, expr_info):
> # Todo: add checking for values. Key is checked as above, value can be
> # also checked here, but we need more functions to handle array case.
>
> +def check_enum(expr, expr_info):
> + name = expr['enum']
> + members = expr.get('data')
> + values = { 'MAX': '(automatic)' }
> +
> + if not isinstance(members, list):
> + raise QAPIExprError(expr_info,
> + "enum '%s' requires an array for 'data'" % name)
> + for member in members:
> + if not isinstance(member, basestring):
> + raise QAPIExprError(expr_info,
> + "enum '%s' member '%s' is not a string"
> + % (name, member))
> + key = _generate_enum_string(member)
> + if key in values:
> + raise QAPIExprError(expr_info,
> + "enum '%s' member '%s' clashes with '%s'"
> + % (name, member, values[key]))
> + values[key] = member
> +
> def check_exprs(schema):
> for expr_elem in schema.exprs:
> expr = expr_elem['expr']
> + info = expr_elem['info']
> +
> if expr.has_key('union'):
> - check_union(expr, expr_elem['info'])
> + check_union(expr, info)
> if expr.has_key('event'):
> - check_event(expr, expr_elem['info'])
> + check_event(expr, info)
> + if expr.has_key('enum'):
> + check_enum(expr, info)
>
> def parse_schema(input_file):
> try:
> @@ -336,7 +360,7 @@ def parse_schema(input_file):
> for expr_elem in schema.exprs:
> expr = expr_elem['expr']
> if expr.has_key('enum'):
> - add_enum(expr['enum'], expr['data'])
> + add_enum(expr['enum'], expr.get('data'))
> elif expr.has_key('union'):
> add_union(expr)
> elif expr.has_key('type'):
> diff --git a/tests/qapi-schema/enum-clash-member.err b/tests/qapi-schema/enum-clash-member.err
> index e69de29..3731fc2 100644
> --- a/tests/qapi-schema/enum-clash-member.err
> +++ b/tests/qapi-schema/enum-clash-member.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/enum-clash-member.json:2: enum 'MyEnum' member 'ONE' clashes with 'one'
> diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/enum-clash-member.exit
> +++ b/tests/qapi-schema/enum-clash-member.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
> index cb4b428..c668ff5 100644
> --- a/tests/qapi-schema/enum-clash-member.json
> +++ b/tests/qapi-schema/enum-clash-member.json
> @@ -1,2 +1,2 @@
> -# FIXME: we should reject enums where members will clash in C switch
> +# we reject enums where members will clash in C switch
> { 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
Actually in PATCH 05 already: "clash in C switch". I guess you mean we
generate an enum with clashing enumeration constants. In the test case,
we'd generate MY_ENUM_ONE (I think) both for 'one' and 'ONE'. Correct?
> diff --git a/tests/qapi-schema/enum-clash-member.out b/tests/qapi-schema/enum-clash-member.out
> index 0814459..e69de29 100644
> --- a/tests/qapi-schema/enum-clash-member.out
> +++ b/tests/qapi-schema/enum-clash-member.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('enum', 'MyEnum'), ('data', ['one', 'ONE'])])]
> -[{'enum_name': 'MyEnum', 'enum_values': ['one', 'ONE']}]
> -[]
> diff --git a/tests/qapi-schema/enum-dict-member.err b/tests/qapi-schema/enum-dict-member.err
> index e69de29..f5a8ffe 100644
> --- a/tests/qapi-schema/enum-dict-member.err
> +++ b/tests/qapi-schema/enum-dict-member.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/enum-dict-member.json:2: enum 'MyEnum' member 'OrderedDict([('value', 'str')])' is not a string
Error message is in terms of implementation instead of source. Since
this is merely a development tool, it'll do. Same elsewhere, and I'm
not going to flag it. Precedents in master quite possible.
> diff --git a/tests/qapi-schema/enum-dict-member.exit b/tests/qapi-schema/enum-dict-member.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/enum-dict-member.exit
> +++ b/tests/qapi-schema/enum-dict-member.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/enum-dict-member.json b/tests/qapi-schema/enum-dict-member.json
> index de4d6bf..79672e0 100644
> --- a/tests/qapi-schema/enum-dict-member.json
> +++ b/tests/qapi-schema/enum-dict-member.json
> @@ -1,2 +1,2 @@
> -# FIXME: we should reject any enum member that is not a string
> +# we reject any enum member that is not a string
> { 'enum': 'MyEnum', 'data': [ { 'value': 'str' } ] }
> diff --git a/tests/qapi-schema/enum-dict-member.out b/tests/qapi-schema/enum-dict-member.out
> index 8b293f8..e69de29 100644
> --- a/tests/qapi-schema/enum-dict-member.out
> +++ b/tests/qapi-schema/enum-dict-member.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('enum', 'MyEnum'), ('data', [OrderedDict([('value', 'str')])])])]
> -[{'enum_name': 'MyEnum', 'enum_values': [OrderedDict([('value', 'str')])]}]
> -[]
> diff --git a/tests/qapi-schema/enum-max-member.err b/tests/qapi-schema/enum-max-member.err
> index e69de29..a6c5db1 100644
> --- a/tests/qapi-schema/enum-max-member.err
> +++ b/tests/qapi-schema/enum-max-member.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/enum-max-member.json:3: enum 'MyEnum' member 'max' clashes with '(automatic)'
> diff --git a/tests/qapi-schema/enum-max-member.exit b/tests/qapi-schema/enum-max-member.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/enum-max-member.exit
> +++ b/tests/qapi-schema/enum-max-member.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/enum-max-member.json b/tests/qapi-schema/enum-max-member.json
> index ea854c4..6af4662 100644
> --- a/tests/qapi-schema/enum-max-member.json
> +++ b/tests/qapi-schema/enum-max-member.json
> @@ -1,3 +1,3 @@
> -# FIXME: we should either reject user-supplied 'max', or munge the implicit
> -# max value we generate at the end of an array
> +# we reject user-supplied 'max' for clashing with implicit enum end
> +# FIXME: should we instead munge the the implicit value to avoid the clash?
Or pick an identifier for the max member so that it cannot clash with
the ones we generate for the user's members.
The generator picks identifiers pretty much thoughtlessly in general.
If something clashes, the C compiler spits it out, and you get to fiddle
with the .json.
[...]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 07/19] qapi: Add some expr tests
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 07/19] qapi: Add some expr tests Eric Blake
@ 2014-09-23 14:26 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-23 14:26 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal well with
> expressions that aren't up to par. Later patches will improve
> the expected results as the generator is made stricter. Only
> one of the added tests actually behaves sanely at rejecting
> obvious problems.
>
> Note that in some cases, we reject bad QAPI merely because our
> pseudo-JSON parser does not yet know how to parse numbers. This
> series does not address that, but when a later series adds support
> for numeric defaults of integer fields, the testsuite will ensure
> that we don't lose the error (and hopefully that the error
> message quality is improved).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions Eric Blake
@ 2014-09-23 14:56 ` Markus Armbruster
2014-09-23 16:11 ` Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-23 14:56 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> The previous commit demonstrated that the generator overlooked some
> fairly basic broken expressions:
> - missing metataype
> - metatype key has a non-string value
> - unknown key in relation to the metatype
> - conflicting metatype (this patch treats the second metatype as an
> unknown key of the first key visited, which is not necessarily the
> first key the user typed)
The whole thing is a Saturday afternoon hack, with extra hacks bolted on
left & right.
> Add check_keys to cover these situations, and update testcases to
> match. A couple other tests (enum-missing-data, indented-expr) had
> to change since the validation added here occurs so early.
>
> While valid .json files won't trigger any of these cases, we might
> as well be nicer to developers that make a typo while trying to add
> new QAPI code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 76 +++++++++++++++++++++++++--------
> tests/qapi-schema/bad-type-dict.err | 1 +
> tests/qapi-schema/bad-type-dict.exit | 2 +-
> tests/qapi-schema/bad-type-dict.json | 2 +-
> tests/qapi-schema/bad-type-dict.out | 3 --
> tests/qapi-schema/double-type.err | 1 +
> tests/qapi-schema/double-type.exit | 2 +-
> tests/qapi-schema/double-type.json | 2 +-
> tests/qapi-schema/double-type.out | 3 --
> tests/qapi-schema/enum-missing-data.err | 2 +-
> tests/qapi-schema/indented-expr.json | 4 +-
> tests/qapi-schema/indented-expr.out | 2 +-
> tests/qapi-schema/missing-type.err | 1 +
> tests/qapi-schema/missing-type.exit | 2 +-
> tests/qapi-schema/missing-type.json | 2 +-
> tests/qapi-schema/missing-type.out | 3 --
> tests/qapi-schema/unknown-expr-key.err | 1 +
> tests/qapi-schema/unknown-expr-key.exit | 2 +-
> tests/qapi-schema/unknown-expr-key.json | 2 +-
> tests/qapi-schema/unknown-expr-key.out | 3 --
> 20 files changed, 75 insertions(+), 41 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 85aa8bf..8fbc45f 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -348,7 +348,29 @@ def check_exprs(schema):
> if expr.has_key('enum'):
> check_enum(expr, info)
>
> +def check_keys(expr_elem, meta, required, optional=[]):
> + expr = expr_elem['expr']
> + info = expr_elem['info']
> + name = expr[meta]
Caller ensures expr[meta] exists. Okay.
> + if not isinstance(name, basestring):
I'm a Python noob: why basestring and not str?
> + raise QAPIExprError(info,
> + "%s key must have a string value" % meta)
> + expr_elem['name'] = name
Where is this used?
> + required.append(meta)
Ugly side effect. To avoid, either make a new list here
required = required + [ meta ]
or do nothing here and...
> + for (key, value) in expr.items():
> + if not key in required and not key in optional:
... add "and key != meta" to this condition.
> + raise QAPIExprError(info,
> + "%s '%s' has unknown key '%s'"
> + % (meta, name, key))
> + for key in required:
> + if not expr.has_key(key):
> + raise QAPIExprError(info,
> + "%s '%s' is missing key '%s'"
> + % (meta, name, key))
> +
> +
> def parse_schema(input_file):
> + # First pass: read entire file into memory
> try:
> schema = QAPISchema(open(input_file, "r"))
> except (QAPISchemaError, QAPIExprError), e:
> @@ -357,24 +379,44 @@ def parse_schema(input_file):
>
> exprs = []
>
> - for expr_elem in schema.exprs:
> - expr = expr_elem['expr']
> - if expr.has_key('enum'):
> - add_enum(expr['enum'], expr.get('data'))
> - elif expr.has_key('union'):
> - add_union(expr)
> - elif expr.has_key('type'):
> - add_struct(expr)
> - exprs.append(expr)
> -
> - # Try again for hidden UnionKind enum
> - for expr_elem in schema.exprs:
> - expr = expr_elem['expr']
> - if expr.has_key('union'):
> - if not discriminator_find_enum_define(expr):
> - add_enum('%sKind' % expr['union'])
> -
> try:
> + # Next pass: learn the types and check for valid expression keys. At
> + # this point, top-level 'include' has already been flattened.
> + for expr_elem in schema.exprs:
> + expr = expr_elem['expr']
> + if expr.has_key('enum'):
> + check_keys(expr_elem, 'enum', ['data'])
> + add_enum(expr['enum'], expr['data'])
> + elif expr.has_key('union'):
> + # Two styles of union, based on discriminator
> + discriminator = expr.get('discriminator')
> + if discriminator == {}:
> + check_keys(expr_elem, 'union', ['data', 'discriminator'])
> + else:
> + check_keys(expr_elem, 'union', ['data'],
> + ['base', 'discriminator'])
> + add_union(expr)
> + elif expr.has_key('type'):
> + check_keys(expr_elem, 'type', ['data'], ['base'])
> + add_struct(expr)
> + elif expr.has_key('command'):
> + check_keys(expr_elem, 'command', [],
> + ['data', 'returns', 'gen', 'success-response'])
> + elif expr.has_key('event'):
> + check_keys(expr_elem, 'event', [], ['data'])
> + else:
> + raise QAPIExprError(expr_elem['info'],
> + "expression is missing metatype")
> + exprs.append(expr)
> +
> + # Try again for hidden UnionKind enum
> + for expr_elem in schema.exprs:
> + expr = expr_elem['expr']
> + if expr.has_key('union'):
> + if not discriminator_find_enum_define(expr):
> + add_enum('%sKind' % expr['union'])
> +
> + # Final pass - validate that exprs make sense
> check_exprs(schema)
> except QAPIExprError, e:
> print >>sys.stderr, e
This hunk is easier to review with whitespace ignored:
@@ -356,13 +356,34 @@
exprs = []
- for expr_elem in schema.exprs:
+ try:
+ # Next pass: learn the types and check for valid expression keys. At
+ # this point, top-level 'include' has already been flattened.
+ for expr_elem in schema.exprs:
expr = expr_elem['expr']
if expr.has_key('enum'):
- add_enum(expr['enum'], expr.get('data'))
+ check_keys(expr_elem, 'enum', ['data'])
+ add_enum(expr['enum'], expr['data'])
elif expr.has_key('union'):
+ # Two styles of union, based on discriminator
+ discriminator = expr.get('discriminator')
+ if discriminator == {}:
+ check_keys(expr_elem, 'union', ['data', 'discriminator'])
+ else:
+ check_keys(expr_elem, 'union', ['data'],
+ ['base', 'discriminator'])
add_union(expr)
elif expr.has_key('type'):
+ check_keys(expr_elem, 'type', ['data'], ['base'])
add_struct(expr)
+ elif expr.has_key('command'):
+ check_keys(expr_elem, 'command', [],
+ ['data', 'returns', 'gen', 'success-response'])
+ elif expr.has_key('event'):
+ check_keys(expr_elem, 'event', [], ['data'])
+ else:
+ raise QAPIExprError(expr_elem['info'],
+ "expression is missing metatype")
exprs.append(expr)
# Try again for hidden UnionKind enum
@@ -372,7 +393,7 @@
if not discriminator_find_enum_define(expr):
add_enum('%sKind' % expr['union'])
- try:
+ # Final pass - validate that exprs make sense
check_exprs(schema)
except QAPIExprError, e:
print >>sys.stderr, e
Looks good to me. The tests, too.
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums
2014-09-23 14:23 ` Markus Armbruster
@ 2014-09-23 15:59 ` Eric Blake
2014-09-24 7:46 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-23 15:59 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 4819 bytes --]
On 09/23/2014 08:23 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> The previous commit demonstrated that the generator had several
>> flaws with less-than-perfect enums:
>> - an enum that listed the same string twice (or two variant
>> strings that map to the same C enum) ended up generating an
>> invalid C enum
>> - because the generator adds a _MAX terminator to each enum,
>> the use of an enum member 'max' can also cause this clash
>> - if an enum omits 'data', the generator left a python stack
>> trace rather than a graceful message
>> - an enum used a non-array 'data' was silently accepted by
>> the parser
>> - an enum that used non-string members in the 'data' member
>> was silently accepted by the parser
>>
>> Add check_enum to cover these situations, and update testcases
>> to match. While valid .json files won't trigger any of these
>> cases, we might as well be nicer to developers that make a typo
>> while trying to add new QAPI code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> --- a/tests/qapi-schema/enum-clash-member.err
>> +++ b/tests/qapi-schema/enum-clash-member.err
>> @@ -0,0 +1 @@
>> +tests/qapi-schema/enum-clash-member.json:2: enum 'MyEnum' member 'ONE' clashes with 'one'
>> diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
>> index 573541a..d00491f 100644
>> --- a/tests/qapi-schema/enum-clash-member.exit
>> +++ b/tests/qapi-schema/enum-clash-member.exit
>> @@ -1 +1 @@
>> -0
>> +1
>> diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
>> index cb4b428..c668ff5 100644
>> --- a/tests/qapi-schema/enum-clash-member.json
>> +++ b/tests/qapi-schema/enum-clash-member.json
>> @@ -1,2 +1,2 @@
>> -# FIXME: we should reject enums where members will clash in C switch
>> +# we reject enums where members will clash in C switch
>> { 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
>
> Actually in PATCH 05 already: "clash in C switch". I guess you mean we
> generate an enum with clashing enumeration constants. In the test case,
> we'd generate MY_ENUM_ONE (I think) both for 'one' and 'ONE'. Correct?
Correct; the generated C code would include an invalid switch statement
with two repetitions of the same spelling of a case label. In patch 5,
the test case demonstrates that the parser was silently accepting code
that resulted in a clash in the generated C code; this patch updates
both qapi.py to make it a hard error, and the testsuite to change from
(accidental) pass to (intentional) error detection, so that we no longer
have to worry about the issue in the generated C code.
The same principle applies throughout my series - I first introduced new
tests in isolation for existing pre-patch behavior, with FIXME comments
where the tests expose bogus behavior, then in later patches fix the
parser to reject bogus behavior and update the test to match that it now
covers the new error message.
>> +++ b/tests/qapi-schema/enum-dict-member.err
>> @@ -0,0 +1 @@
>> +tests/qapi-schema/enum-dict-member.json:2: enum 'MyEnum' member 'OrderedDict([('value', 'str')])' is not a string
>
> Error message is in terms of implementation instead of source. Since
> this is merely a development tool, it'll do. Same elsewhere, and I'm
> not going to flag it. Precedents in master quite possible.
Yeah, I couldn't figure a way to get back at the original text typed by
the user. I'm open to suggestions, but I'm (obviously) okay with
leaving it as is.
>> +++ b/tests/qapi-schema/enum-max-member.json
>> @@ -1,3 +1,3 @@
>> -# FIXME: we should either reject user-supplied 'max', or munge the implicit
>> -# max value we generate at the end of an array
>> +# we reject user-supplied 'max' for clashing with implicit enum end
>> +# FIXME: should we instead munge the the implicit value to avoid the clash?
>
> Or pick an identifier for the max member so that it cannot clash with
> the ones we generate for the user's members.
>
> The generator picks identifiers pretty much thoughtlessly in general.
> If something clashes, the C compiler spits it out, and you get to fiddle
> with the .json.
Well, hopefully by hoisting the error message away from C compilation
time (late) to qapi.py runtime (early), we have made it easier for
anyone actually needing the name 'max' to identify what still needs
fixing. At any rate, I'm always a fan of erroring out as early as
possible, rather than waiting for an obscure crash later on in the C
compilation :)
>
> [...]
>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
Thanks.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-23 14:56 ` Markus Armbruster
@ 2014-09-23 16:11 ` Eric Blake
2014-09-24 7:34 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-23 16:11 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 2932 bytes --]
On 09/23/2014 08:56 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> The previous commit demonstrated that the generator overlooked some
>> fairly basic broken expressions:
>> - missing metataype
>> - metatype key has a non-string value
>> - unknown key in relation to the metatype
>> - conflicting metatype (this patch treats the second metatype as an
>> unknown key of the first key visited, which is not necessarily the
>> first key the user typed)
>
> The whole thing is a Saturday afternoon hack, with extra hacks bolted on
> left & right.
And this series is a sequence of MY Saturday afternoon hacks in cleaning
it up :)
>
>> Add check_keys to cover these situations, and update testcases to
>> match. A couple other tests (enum-missing-data, indented-expr) had
>> to change since the validation added here occurs so early.
>>
>> While valid .json files won't trigger any of these cases, we might
>> as well be nicer to developers that make a typo while trying to add
>> new QAPI code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
>> +def check_keys(expr_elem, meta, required, optional=[]):
>> + expr = expr_elem['expr']
>> + info = expr_elem['info']
>> + name = expr[meta]
>
> Caller ensures expr[meta] exists. Okay.
>
>> + if not isinstance(name, basestring):
>
> I'm a Python noob: why basestring and not str?
Me too. No clue. Copy and paste from existing code.
http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
>
>> + raise QAPIExprError(info,
>> + "%s key must have a string value" % meta)
>> + expr_elem['name'] = name
>
> Where is this used?
Hmm, I know I used it at one point in my series (to be able to print the
name of an expression in check_type added in 13/19, without having to
repeat the dance of if enum: expr['enum'] elif union: expr['union'] etc.
in multiple places). Although in my refactoring, I may have eliminated
the need for it after all. If it's not in use at the end of the series,
I can drop it.
>
>> + required.append(meta)
>
> Ugly side effect. To avoid, either make a new list here
>
> required = required + [ meta ]
>
> or do nothing here and...
>
>> + for (key, value) in expr.items():
>> + if not key in required and not key in optional:
>
> ... add "and key != meta" to this condition.
Shows my inexperience in python. Sure, I can fix. Looks like I get to
spin a v5.
>
> This hunk is easier to review with whitespace ignored:
Indeed. Alas, python is a stickler about whitespace reindentation.
Since I have to respin, I'll probably split into two pieces (one with a
no-op change that reindents, the other for the improvements).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-23 16:11 ` Eric Blake
@ 2014-09-24 7:34 ` Markus Armbruster
2014-09-24 9:25 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 7:34 UTC (permalink / raw)
To: Eric Blake
Cc: Kevin Wolf, Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 09/23/2014 08:56 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> The previous commit demonstrated that the generator overlooked some
>>> fairly basic broken expressions:
>>> - missing metataype
>>> - metatype key has a non-string value
>>> - unknown key in relation to the metatype
>>> - conflicting metatype (this patch treats the second metatype as an
>>> unknown key of the first key visited, which is not necessarily the
>>> first key the user typed)
>>
>> The whole thing is a Saturday afternoon hack, with extra hacks bolted on
>> left & right.
>
> And this series is a sequence of MY Saturday afternoon hacks in cleaning
> it up :)
Much appreciated!
>>> Add check_keys to cover these situations, and update testcases to
>>> match. A couple other tests (enum-missing-data, indented-expr) had
>>> to change since the validation added here occurs so early.
>>>
>>> While valid .json files won't trigger any of these cases, we might
>>> as well be nicer to developers that make a typo while trying to add
>>> new QAPI code.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>
>>> +def check_keys(expr_elem, meta, required, optional=[]):
>>> + expr = expr_elem['expr']
>>> + info = expr_elem['info']
>>> + name = expr[meta]
>>
>> Caller ensures expr[meta] exists. Okay.
>>
>>> + if not isinstance(name, basestring):
>>
>> I'm a Python noob: why basestring and not str?
>
> Me too. No clue. Copy and paste from existing code.
> http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
Yes. It's our only use of basestring. Other places use isinstance(FOO,
str).
The basestring use comes from Kevin's commit b35284e. Kevin, why
basestring and not str?
>>
>>> + raise QAPIExprError(info,
>>> + "%s key must have a string value" % meta)
>>> + expr_elem['name'] = name
>>
>> Where is this used?
>
> Hmm, I know I used it at one point in my series (to be able to print the
> name of an expression in check_type added in 13/19, without having to
> repeat the dance of if enum: expr['enum'] elif union: expr['union'] etc.
> in multiple places). Although in my refactoring, I may have eliminated
> the need for it after all. If it's not in use at the end of the series,
> I can drop it.
A quick grep comes up empty.
>>> + required.append(meta)
>>
>> Ugly side effect. To avoid, either make a new list here
>>
>> required = required + [ meta ]
>>
>> or do nothing here and...
>>
>>> + for (key, value) in expr.items():
>>> + if not key in required and not key in optional:
>>
>> ... add "and key != meta" to this condition.
>
> Shows my inexperience in python. Sure, I can fix. Looks like I get to
> spin a v5.
>
>>
>> This hunk is easier to review with whitespace ignored:
>
> Indeed. Alas, python is a stickler about whitespace reindentation.
> Since I have to respin, I'll probably split into two pieces (one with a
> no-op change that reindents, the other for the improvements).
Not necessary for me, I'm comfy with telling Emacs to hide whitespace
differences :)
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums
2014-09-23 15:59 ` Eric Blake
@ 2014-09-24 7:46 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 7:46 UTC (permalink / raw)
To: Eric Blake; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 09/23/2014 08:23 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> The previous commit demonstrated that the generator had several
>>> flaws with less-than-perfect enums:
>>> - an enum that listed the same string twice (or two variant
>>> strings that map to the same C enum) ended up generating an
>>> invalid C enum
>>> - because the generator adds a _MAX terminator to each enum,
>>> the use of an enum member 'max' can also cause this clash
>>> - if an enum omits 'data', the generator left a python stack
>>> trace rather than a graceful message
>>> - an enum used a non-array 'data' was silently accepted by
>>> the parser
>>> - an enum that used non-string members in the 'data' member
>>> was silently accepted by the parser
>>>
>>> Add check_enum to cover these situations, and update testcases
>>> to match. While valid .json files won't trigger any of these
>>> cases, we might as well be nicer to developers that make a typo
>>> while trying to add new QAPI code.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> ---
>
>>> --- a/tests/qapi-schema/enum-clash-member.err
>>> +++ b/tests/qapi-schema/enum-clash-member.err
>>> @@ -0,0 +1 @@
>>> +tests/qapi-schema/enum-clash-member.json:2: enum 'MyEnum' member 'ONE' clashes with 'one'
>>> diff --git a/tests/qapi-schema/enum-clash-member.exit b/tests/qapi-schema/enum-clash-member.exit
>>> index 573541a..d00491f 100644
>>> --- a/tests/qapi-schema/enum-clash-member.exit
>>> +++ b/tests/qapi-schema/enum-clash-member.exit
>>> @@ -1 +1 @@
>>> -0
>>> +1
>>> diff --git a/tests/qapi-schema/enum-clash-member.json b/tests/qapi-schema/enum-clash-member.json
>>> index cb4b428..c668ff5 100644
>>> --- a/tests/qapi-schema/enum-clash-member.json
>>> +++ b/tests/qapi-schema/enum-clash-member.json
>>> @@ -1,2 +1,2 @@
>>> -# FIXME: we should reject enums where members will clash in C switch
>>> +# we reject enums where members will clash in C switch
>>> { 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
>>
>> Actually in PATCH 05 already: "clash in C switch". I guess you mean we
>> generate an enum with clashing enumeration constants. In the test case,
>> we'd generate MY_ENUM_ONE (I think) both for 'one' and 'ONE'. Correct?
>
> Correct; the generated C code would include an invalid switch statement
> with two repetitions of the same spelling of a case label.
Actually, it generates a broken enum definition.
For instance,
{ 'enum': 'MyEnum', 'data': [ 'one', 'ONE' ] }
produces
typedef enum BadEnum
{
BAD_ENUM_ONE = 0,
BAD_ENUM_ONE = 1,
BAD_ENUM_MAX = 2,
} BadEnum;
Drop "in C switch" from the comment?
> In patch 5,
> the test case demonstrates that the parser was silently accepting code
> that resulted in a clash in the generated C code; this patch updates
> both qapi.py to make it a hard error, and the testsuite to change from
> (accidental) pass to (intentional) error detection, so that we no longer
> have to worry about the issue in the generated C code.
>
> The same principle applies throughout my series - I first introduced new
> tests in isolation for existing pre-patch behavior, with FIXME comments
> where the tests expose bogus behavior, then in later patches fix the
> parser to reject bogus behavior and update the test to match that it now
> covers the new error message.
That's exactly how I like it done.
>>> +++ b/tests/qapi-schema/enum-dict-member.err
>>> @@ -0,0 +1 @@
>>> +tests/qapi-schema/enum-dict-member.json:2: enum 'MyEnum' member
>>> OrderedDict([('value', 'str')])' is not a string
>>
>> Error message is in terms of implementation instead of source. Since
>> this is merely a development tool, it'll do. Same elsewhere, and I'm
>> not going to flag it. Precedents in master quite possible.
>
> Yeah, I couldn't figure a way to get back at the original text typed by
> the user. I'm open to suggestions, but I'm (obviously) okay with
> leaving it as is.
Let's leave it as is for now.
>>> +++ b/tests/qapi-schema/enum-max-member.json
>>> @@ -1,3 +1,3 @@
>>> -# FIXME: we should either reject user-supplied 'max', or munge the implicit
>>> -# max value we generate at the end of an array
>>> +# we reject user-supplied 'max' for clashing with implicit enum end
>>> +# FIXME: should we instead munge the the implicit value to avoid the clash?
>>
>> Or pick an identifier for the max member so that it cannot clash with
>> the ones we generate for the user's members.
>>
>> The generator picks identifiers pretty much thoughtlessly in general.
>> If something clashes, the C compiler spits it out, and you get to fiddle
>> with the .json.
>
> Well, hopefully by hoisting the error message away from C compilation
> time (late) to qapi.py runtime (early), we have made it easier for
> anyone actually needing the name 'max' to identify what still needs
> fixing. At any rate, I'm always a fan of erroring out as early as
> possible, rather than waiting for an obscure crash later on in the C
> compilation :)
One saturday of happy hacking, followed by years of chipping away at the
mess...
>> [...]
>>
>> Reviewed-by: Markus Armbruster <armbru@redhat.com>
>
> Thanks.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-24 7:34 ` Markus Armbruster
@ 2014-09-24 9:25 ` Kevin Wolf
2014-09-24 11:14 ` Markus Armbruster
2014-09-26 9:15 ` Markus Armbruster
0 siblings, 2 replies; 62+ messages in thread
From: Kevin Wolf @ 2014-09-24 9:25 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, Luiz Capitulino, qemu-devel, wenchaoqemu
Am 24.09.2014 um 09:34 hat Markus Armbruster geschrieben:
> Eric Blake <eblake@redhat.com> writes:
>
> > On 09/23/2014 08:56 AM, Markus Armbruster wrote:
> >> Eric Blake <eblake@redhat.com> writes:
> >>> Add check_keys to cover these situations, and update testcases to
> >>> match. A couple other tests (enum-missing-data, indented-expr) had
> >>> to change since the validation added here occurs so early.
> >>>
> >>> While valid .json files won't trigger any of these cases, we might
> >>> as well be nicer to developers that make a typo while trying to add
> >>> new QAPI code.
> >>>
> >>> Signed-off-by: Eric Blake <eblake@redhat.com>
> >>
> >
> >>> +def check_keys(expr_elem, meta, required, optional=[]):
> >>> + expr = expr_elem['expr']
> >>> + info = expr_elem['info']
> >>> + name = expr[meta]
> >>
> >> Caller ensures expr[meta] exists. Okay.
> >>
> >>> + if not isinstance(name, basestring):
> >>
> >> I'm a Python noob: why basestring and not str?
> >
> > Me too. No clue. Copy and paste from existing code.
> > http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
>
> Yes. It's our only use of basestring. Other places use isinstance(FOO,
> str).
>
> The basestring use comes from Kevin's commit b35284e. Kevin, why
> basestring and not str?
Do I look as if I knew what I'm doing when I write Python code? :-)
Apparently basestring is the superclass for ASCII and Unicode strings. I
seem to dimly remember that I did indeed get a Unicode string somewhere
(even though probably no non-ASCII characters in it) and it caused
trouble. Might well have been here.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-24 9:25 ` Kevin Wolf
@ 2014-09-24 11:14 ` Markus Armbruster
2014-09-26 9:15 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 11:14 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Kevin Wolf <kwolf@redhat.com> writes:
> Am 24.09.2014 um 09:34 hat Markus Armbruster geschrieben:
>> Eric Blake <eblake@redhat.com> writes:
>>
>> > On 09/23/2014 08:56 AM, Markus Armbruster wrote:
>> >> Eric Blake <eblake@redhat.com> writes:
>> >>> Add check_keys to cover these situations, and update testcases to
>> >>> match. A couple other tests (enum-missing-data, indented-expr) had
>> >>> to change since the validation added here occurs so early.
>> >>>
>> >>> While valid .json files won't trigger any of these cases, we might
>> >>> as well be nicer to developers that make a typo while trying to add
>> >>> new QAPI code.
>> >>>
>> >>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> >>
>> >
>> >>> +def check_keys(expr_elem, meta, required, optional=[]):
>> >>> + expr = expr_elem['expr']
>> >>> + info = expr_elem['info']
>> >>> + name = expr[meta]
>> >>
>> >> Caller ensures expr[meta] exists. Okay.
>> >>
>> >>> + if not isinstance(name, basestring):
>> >>
>> >> I'm a Python noob: why basestring and not str?
>> >
>> > Me too. No clue. Copy and paste from existing code.
>> > http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
>>
>> Yes. It's our only use of basestring. Other places use isinstance(FOO,
>> str).
>>
>> The basestring use comes from Kevin's commit b35284e. Kevin, why
>> basestring and not str?
>
> Do I look as if I knew what I'm doing when I write Python code? :-)
>
> Apparently basestring is the superclass for ASCII and Unicode strings. I
> seem to dimly remember that I did indeed get a Unicode string somewhere
> (even though probably no non-ASCII characters in it) and it caused
> trouble. Might well have been here.
Aha. These links apply:
https://stackoverflow.com/questions/11301138/how-to-check-if-variable-is-string-with-python-2-and-3-compatibility
https://en.wikipedia.org/wiki/File:Pieter_Bruegel_the_Elder_%281568%29_The_Blind_Leading_the_Blind.jpg
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 09/19] qapi: Add tests of redefined expressions
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 09/19] qapi: Add tests of redefined expressions Eric Blake
@ 2014-09-24 11:24 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 11:24 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator doesn't deal very well with
> redefined expressions. At the parse level, they are silently
> accepted; I'm not sure what would happen if we tried to go
> further and use it in generated code, but the end result can't
> be good. A later patch will tighten things up and adjust the
> testsuite to match.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions Eric Blake
@ 2014-09-24 11:58 ` Markus Armbruster
2014-09-24 14:10 ` Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 11:58 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> The previous commit demonstrated that the generator overlooked
> duplicate expressions:
> - a complex type reusing a built-in type name
> - redeclaration of a type name, whether by the same or different
> metatype
> - redeclaration of a command or event
> - lack of tracking of 'size' as a built-in type
>
> Add a global array to track all known types and their source,
> as well as enhancing check_exprs to also check for duplicate
> events and commands. While valid .json files won't trigger any
> of these cases, we might as well be nicer to developers that
> make a typo while trying to add new QAPI code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 71 +++++++++++++++++++++++++-------
> tests/qapi-schema/redefined-builtin.err | 1 +
> tests/qapi-schema/redefined-builtin.exit | 2 +-
> tests/qapi-schema/redefined-builtin.json | 2 +-
> tests/qapi-schema/redefined-builtin.out | 3 --
> tests/qapi-schema/redefined-command.err | 1 +
> tests/qapi-schema/redefined-command.exit | 2 +-
> tests/qapi-schema/redefined-command.json | 2 +-
> tests/qapi-schema/redefined-command.out | 4 --
> tests/qapi-schema/redefined-event.err | 1 +
> tests/qapi-schema/redefined-event.exit | 2 +-
> tests/qapi-schema/redefined-event.json | 2 +-
> tests/qapi-schema/redefined-event.out | 4 --
> tests/qapi-schema/redefined-type.err | 1 +
> tests/qapi-schema/redefined-type.exit | 2 +-
> tests/qapi-schema/redefined-type.json | 2 +-
> tests/qapi-schema/redefined-type.out | 4 --
> 17 files changed, 69 insertions(+), 37 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 8fbc45f..bf243fa 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -19,8 +19,15 @@ import sys
> builtin_types = [
> 'str', 'int', 'number', 'bool',
> 'int8', 'int16', 'int32', 'int64',
> - 'uint8', 'uint16', 'uint32', 'uint64'
> + 'uint8', 'uint16', 'uint32', 'uint64',
> + 'size'
> ]
I figure we want a blank line here.
Adding 'size' should have the following effects:
* Type sizeList is generated into qapi-types.h
* Declaration of qapi_free_sizeList() is generated into qapi-types.h,
definition into qapi-types.c
* generate_visit_anon_union() no longer fails an assertion when it runs
into a member of type 'size'
* Declaration of visit_type_sizeList() is generated into qapi-visit.h,
definition into qapi-visit.c
Make sense.
How can we be sure we now got all built-in types covered? Documentation
says yes, but it cannot be trusted. I figure the best evidence we have
is c_type(). Looks good.
Aside: have a look at how it recognizes event names: "name ==
name.upper()". Ugh! I guess this patch lets us clean it up to "name in
events".
I think you should add 'size' to builtin_type_qtypes[], too.
> +enum_types = []
> +struct_types = []
> +union_types = []
Semi-related code motion. Okay.
> +all_types = {}
> +commands = []
> +events = ['MAX']
>
> builtin_type_qtypes = {
> 'str': 'QTYPE_QSTRING',
> @@ -248,8 +255,22 @@ def discriminator_find_enum_define(expr):
>
> return find_enum(discriminator_type)
>
> +def check_command(expr, expr_info):
> + global commands
> + name = expr['command']
> + if name in commands:
> + raise QAPIExprError(expr_info,
> + "command '%s' is already defined" % name)
> + commands.append(name)
> +
This rejects duplicate commands. Good.
> def check_event(expr, expr_info):
> + global events
> + name = expr['event']
> params = expr.get('data')
> + if name in events:
> + raise QAPIExprError(expr_info,
> + "event '%s' is already defined" % name)
> + events.append(name)
> if params:
> for argname, argentry, optional, structured in parse_args(params):
> if structured:
This rejects duplicate events. Good.
> @@ -347,6 +368,8 @@ def check_exprs(schema):
> check_event(expr, info)
> if expr.has_key('enum'):
> check_enum(expr, info)
> + if expr.has_key('command'):
> + check_command(expr, info)
>
> def check_keys(expr_elem, meta, required, optional=[]):
> expr = expr_elem['expr']
> @@ -370,6 +393,9 @@ def check_keys(expr_elem, meta, required, optional=[]):
>
>
> def parse_schema(input_file):
> + global all_types
> + exprs = []
> +
> # First pass: read entire file into memory
> try:
> schema = QAPISchema(open(input_file, "r"))
> @@ -377,16 +403,17 @@ def parse_schema(input_file):
> print >>sys.stderr, e
> exit(1)
>
> - exprs = []
> -
> try:
> # Next pass: learn the types and check for valid expression keys. At
> # this point, top-level 'include' has already been flattened.
> + for builtin in builtin_types:
> + all_types[builtin] = 'built-in'
> for expr_elem in schema.exprs:
> expr = expr_elem['expr']
> + info = expr_elem['info']
> if expr.has_key('enum'):
> check_keys(expr_elem, 'enum', ['data'])
> - add_enum(expr['enum'], expr['data'])
> + add_enum(expr['enum'], info, expr['data'])
> elif expr.has_key('union'):
> # Two styles of union, based on discriminator
> discriminator = expr.get('discriminator')
> @@ -395,10 +422,10 @@ def parse_schema(input_file):
> else:
> check_keys(expr_elem, 'union', ['data'],
> ['base', 'discriminator'])
> - add_union(expr)
> + add_union(expr, info)
> elif expr.has_key('type'):
> check_keys(expr_elem, 'type', ['data'], ['base'])
> - add_struct(expr)
> + add_struct(expr, info)
> elif expr.has_key('command'):
> check_keys(expr_elem, 'command', [],
> ['data', 'returns', 'gen', 'success-response'])
> @@ -414,7 +441,7 @@ def parse_schema(input_file):
> expr = expr_elem['expr']
> if expr.has_key('union'):
> if not discriminator_find_enum_define(expr):
> - add_enum('%sKind' % expr['union'])
> + add_enum('%sKind' % expr['union'], expr_elem['info'])
>
> # Final pass - validate that exprs make sense
> check_exprs(schema)
> @@ -508,12 +535,15 @@ def type_name(name):
> return c_list_type(name[0])
> return name
>
> -enum_types = []
> -struct_types = []
> -union_types = []
> -
> -def add_struct(definition):
> +def add_struct(definition, info):
> global struct_types
> + global all_types
> + name = definition['type']
> + if name in all_types:
> + raise QAPIExprError(info,
> + "%s '%s' is already defined"
> + %(all_types[name], name))
> + all_types[name] = 'struct'
> struct_types.append(definition)
>
> def find_struct(name):
> @@ -523,8 +553,15 @@ def find_struct(name):
> return struct
> return None
>
> -def add_union(definition):
> +def add_union(definition, info):
> global union_types
> + global all_types
> + name = definition['union']
> + if name in all_types:
> + raise QAPIExprError(info,
> + "%s '%s' is already defined"
> + %(all_types[name], name))
> + all_types[name] = 'union'
> union_types.append(definition)
>
> def find_union(name):
> @@ -534,8 +571,14 @@ def find_union(name):
> return union
> return None
>
> -def add_enum(name, enum_values = None):
> +def add_enum(name, info, enum_values = None):
> global enum_types
> + global all_types
> + if name in all_types:
> + raise QAPIExprError(info,
> + "%s '%s' is already defined"
> + %(all_types[name], name))
> + all_types[name] = 'enum'
> enum_types.append({"enum_name": name, "enum_values": enum_values})
>
> def find_enum(name):
These hunks reject duplicate types of any kind: enum, complex
(a.k.a. struct), union.
We have separate name spaces for events, commands and types. Works for
me. A single name space would work for me, too.
[Tests snipped, they look good...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions
2014-09-24 11:58 ` Markus Armbruster
@ 2014-09-24 14:10 ` Eric Blake
2014-09-24 15:29 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-24 14:10 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 882 bytes --]
On 09/24/2014 05:58 AM, Markus Armbruster wrote:
>
> We have separate name spaces for events, commands and types. Works for
> me. A single name space would work for me, too.
I thought about that too. Our conventions are that commands are
all-lower-case, events are ALL_UPPER, and user-defined types are
CamelCase - so a single namespace would not have any collisions, except
for one case: built-in types like 'int' are also all lower, which
collides with commands. But I see no technical reason why we wouldn't
be able to generate C code for a command named 'int'. I can go either
way, but should probably add a test for a .json file that does
{'command':'int'} to test which way we go. Preferences on whether that
should be allowed or forbidden?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions
2014-09-24 14:10 ` Eric Blake
@ 2014-09-24 15:29 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 15:29 UTC (permalink / raw)
To: Eric Blake; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 09/24/2014 05:58 AM, Markus Armbruster wrote:
>
>>
>> We have separate name spaces for events, commands and types. Works for
>> me. A single name space would work for me, too.
>
> I thought about that too. Our conventions are that commands are
> all-lower-case, events are ALL_UPPER, and user-defined types are
> CamelCase - so a single namespace would not have any collisions, except
> for one case: built-in types like 'int' are also all lower, which
> collides with commands. But I see no technical reason why we wouldn't
> be able to generate C code for a command named 'int'. I can go either
> way, but should probably add a test for a .json file that does
> {'command':'int'} to test which way we go. Preferences on whether that
> should be allowed or forbidden?
With a single name space, 'command': 'int' should collide with the
built-in type.
With separate name spaces, it should in theory just work. No big deal
if it doesn't due to generator sloppiness or something.
As I said, I'm fine both with keeping the current separate name space,
and with switching to a single name space. Either way, documentation
would be nice.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 11/19] qapi: Add tests of type bypass
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 11/19] qapi: Add tests of type bypass Eric Blake
@ 2014-09-24 16:10 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-24 16:10 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> For a few QMP commands, we are forced to pass an arbitrary type
> without tracking it properly in QAPI. Among the existing clients,
> this unnamed type was spelled 'dict', 'visitor', and '**'; this
> patch standardizes on '**'. There is no difference to the generated
> code. As the feature was previously undocumented, add some tests
> and documentation on what we'd like to guarantee, although it will
> take later patches to clean up test results.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
@ 2014-09-25 7:34 ` Markus Armbruster
2014-09-25 8:06 ` Markus Armbruster
2014-09-25 13:54 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
0 siblings, 2 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-25 7:34 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Demonstrate that the qapi generator silently parses confusing
> types, which may cause other errors later on. Later patches
> will update the expected results as the generator is made stricter.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> tests/Makefile | 8 ++++++--
> tests/qapi-schema/data-array-empty.err | 0
> tests/qapi-schema/data-array-empty.exit | 1 +
> tests/qapi-schema/data-array-empty.json | 2 ++
[Twelve new tests...]
> create mode 100644 tests/qapi-schema/returns-unknown.err
> create mode 100644 tests/qapi-schema/returns-unknown.exit
> create mode 100644 tests/qapi-schema/returns-unknown.json
> create mode 100644 tests/qapi-schema/returns-unknown.out
Holy moly!
> diff --git a/tests/Makefile b/tests/Makefile
> index 5e01952..6fe34f7 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -203,8 +203,12 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
> double-data.json unknown-expr-key.json redefined-type.json \
> redefined-command.json redefined-builtin.json redefined-event.json \
> type-bypass.json type-bypass-no-gen.json type-bypass-bad-gen.json \
> - missing-colon.json missing-comma-list.json \
> - missing-comma-object.json non-objects.json \
> + data-array-empty.json data-array-unknown.json data-int.json \
> + data-unknown.json data-member-unknown.json data-member-array.json \
> + data-member-array-bad.json returns-array-bad.json returns-int.json \
> + returns-unknown.json missing-colon.json missing-comma-list.json \
> + missing-comma-object.json nested-struct-data.json \
> + nested-struct-returns.json non-objects.json \
> qapi-schema-test.json quoted-structural-chars.json \
> trailing-comma-list.json trailing-comma-object.json \
> unclosed-list.json unclosed-object.json unclosed-string.json \
> diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-empty.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
> new file mode 100644
> index 0000000..41b6c1e
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-empty.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject an array for data if it does not contain a known type
> +{ 'command': 'oops', 'data': [ ] }
Do we want to permit anything but a complex type for 'data'?
For what it's worth, docs/qmp/qmp-spec.txt specifies:
2.3 Issuing Commands
--------------------
The format for command execution is:
{ "execute": json-string, "arguments": json-object, "id": json-value }
Where,
- The "execute" member identifies the command to be executed by the Server
- The "arguments" member is used to pass any arguments required for the
execution of the command, it is optional when no arguments are required
- The "id" member is a transaction identification associated with the
command execution, it is optional and will be part of the response if
provided
The QAPI schema's 'data' is "arguments" on the wire.
Semantically, 'data' of a complex type / json-object "arguments" is an
ordered list of named parameters. Makes obvious sense.
'data' of list type / json-array "arguments" is an ordered list of
unnamed parameters. Makes sense, but it isn't how QMP works. Or C for
that matter. Do we really want to support this in QAPI?
If yes, then 'data': [] means the same thing as 'data': {} or no 'data':
no arguments.
Aside: discussion of list types in qapi-code-gen.txt is spread over the
places that use them. I feel giving them their own section on the same
level as complex types etc. would make sense.
'data' of built-in or enumeration type / json-number or json-string
"arguments" could be regarded as a single unnamed parameter. Even if we
want unnamed parameters, the question remains whether we want two
syntactic forms for a single unnamed parameter, boxed in a [ ] and
unboxed. I doubt it.
One kind of types left to discuss: unions. I figure the semantics of a
simple or flat union type are obvious enough, so we can discuss whether
we want them. Anonymous unions are a different matter, because they
boil down to a single parameter that need not be json-object.
> diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
> new file mode 100644
> index 0000000..67802be
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-empty.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', [])])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/data-array-unknown.err b/tests/qapi-schema/data-array-unknown.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-array-unknown.exit b/tests/qapi-schema/data-array-unknown.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-unknown.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-array-unknown.json b/tests/qapi-schema/data-array-unknown.json
> new file mode 100644
> index 0000000..434fb5f
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-unknown.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject an array for data if it does not contain a known type
> +{ 'command': 'oops', 'data': [ 'NoSuchType' ] }
> diff --git a/tests/qapi-schema/data-array-unknown.out b/tests/qapi-schema/data-array-unknown.out
> new file mode 100644
> index 0000000..c3bc05e
> --- /dev/null
> +++ b/tests/qapi-schema/data-array-unknown.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', ['NoSuchType'])])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/data-int.err b/tests/qapi-schema/data-int.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-int.exit b/tests/qapi-schema/data-int.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-int.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-int.json b/tests/qapi-schema/data-int.json
> new file mode 100644
> index 0000000..37916e0
> --- /dev/null
> +++ b/tests/qapi-schema/data-int.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject commands where data is not an array or complex type
> +{ 'command': 'oops', 'data': 'int' }
> diff --git a/tests/qapi-schema/data-int.out b/tests/qapi-schema/data-int.out
> new file mode 100644
> index 0000000..e589a4f
> --- /dev/null
> +++ b/tests/qapi-schema/data-int.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', 'int')])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/data-member-array-bad.err b/tests/qapi-schema/data-member-array-bad.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-member-array-bad.exit b/tests/qapi-schema/data-member-array-bad.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-array-bad.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-member-array-bad.json b/tests/qapi-schema/data-member-array-bad.json
> new file mode 100644
> index 0000000..c954af1
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-array-bad.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject data if it does not contain a valid array type
> +{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
I'm probably just suffering from temporary denseness here... why is this
example problematic? To me, it looks like a single argument 'member' of
type "array of complex type with a single member 'nested' of
builtin-type 'str'".
> diff --git a/tests/qapi-schema/data-member-array-bad.out b/tests/qapi-schema/data-member-array-bad.out
> new file mode 100644
> index 0000000..0e00c41
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-array-bad.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', [OrderedDict([('nested', 'str')])])]))])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/data-member-array.err b/tests/qapi-schema/data-member-array.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-member-array.exit b/tests/qapi-schema/data-member-array.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-array.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-member-array.json b/tests/qapi-schema/data-member-array.json
> new file mode 100644
> index 0000000..7cce276
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-array.json
> @@ -0,0 +1,4 @@
> +# valid array members
> +{ 'enum': 'abc', 'data': [ 'a', 'b', 'c' ] }
> +{ 'type': 'def', 'data': { 'array': [ 'abc' ] } }
> +{ 'command': 'okay', 'data': { 'member1': [ 'int' ], 'member2': [ 'def' ] } }
> diff --git a/tests/qapi-schema/data-member-array.out b/tests/qapi-schema/data-member-array.out
> new file mode 100644
> index 0000000..8287120
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-array.out
> @@ -0,0 +1,5 @@
> +[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
> + OrderedDict([('type', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
> + OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), ('member2', ['def'])]))])]
> +[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
> +[OrderedDict([('type', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
> diff --git a/tests/qapi-schema/data-member-unknown.err b/tests/qapi-schema/data-member-unknown.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-member-unknown.exit b/tests/qapi-schema/data-member-unknown.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-unknown.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-member-unknown.json b/tests/qapi-schema/data-member-unknown.json
> new file mode 100644
> index 0000000..40e6252
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-unknown.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject data if it does not contain a known type
> +{ 'command': 'oops', 'data': { 'member': 'NoSuchType' } }
> diff --git a/tests/qapi-schema/data-member-unknown.out b/tests/qapi-schema/data-member-unknown.out
> new file mode 100644
> index 0000000..36252a5
> --- /dev/null
> +++ b/tests/qapi-schema/data-member-unknown.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', OrderedDict([('member', 'NoSuchType')]))])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/data-unknown.err b/tests/qapi-schema/data-unknown.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/data-unknown.exit b/tests/qapi-schema/data-unknown.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/data-unknown.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/data-unknown.json b/tests/qapi-schema/data-unknown.json
> new file mode 100644
> index 0000000..776bd34
> --- /dev/null
> +++ b/tests/qapi-schema/data-unknown.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject data if it does not contain a known type
> +{ 'command': 'oops', 'data': 'NoSuchType' }
> diff --git a/tests/qapi-schema/data-unknown.out b/tests/qapi-schema/data-unknown.out
> new file mode 100644
> index 0000000..2c60726
> --- /dev/null
> +++ b/tests/qapi-schema/data-unknown.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('data', 'NoSuchType')])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/nested-struct-data.err b/tests/qapi-schema/nested-struct-data.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/nested-struct-data.exit b/tests/qapi-schema/nested-struct-data.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/nested-struct-data.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/nested-struct-data.json b/tests/qapi-schema/nested-struct-data.json
> new file mode 100644
> index 0000000..0247c8c
> --- /dev/null
> +++ b/tests/qapi-schema/nested-struct-data.json
> @@ -0,0 +1,4 @@
> +# FIXME: inline subtypes collide with our desired future use of defaults
> +{ 'command': 'foo',
> + 'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' },
> + 'returns': {} }
> diff --git a/tests/qapi-schema/nested-struct-data.out b/tests/qapi-schema/nested-struct-data.out
> new file mode 100644
> index 0000000..999cbb8
> --- /dev/null
> +++ b/tests/qapi-schema/nested-struct-data.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'foo'), ('data', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')])), ('returns', OrderedDict())])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/nested-struct-returns.err b/tests/qapi-schema/nested-struct-returns.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/nested-struct-returns.exit b/tests/qapi-schema/nested-struct-returns.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/nested-struct-returns.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/nested-struct-returns.json b/tests/qapi-schema/nested-struct-returns.json
> new file mode 100644
> index 0000000..5a46840
> --- /dev/null
> +++ b/tests/qapi-schema/nested-struct-returns.json
> @@ -0,0 +1,3 @@
> +# FIXME: inline subtypes collide with our desired future use of defaults
> +{ 'command': 'foo',
> + 'returns': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }
> diff --git a/tests/qapi-schema/nested-struct-returns.out b/tests/qapi-schema/nested-struct-returns.out
> new file mode 100644
> index 0000000..c53d23b
> --- /dev/null
> +++ b/tests/qapi-schema/nested-struct-returns.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'foo'), ('returns', OrderedDict([('a', OrderedDict([('string', 'str'), ('integer', 'int')])), ('b', 'str')]))])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/returns-array-bad.err b/tests/qapi-schema/returns-array-bad.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/returns-array-bad.exit b/tests/qapi-schema/returns-array-bad.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/returns-array-bad.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/returns-array-bad.json b/tests/qapi-schema/returns-array-bad.json
> new file mode 100644
> index 0000000..14882c1
> --- /dev/null
> +++ b/tests/qapi-schema/returns-array-bad.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject an array return that is not a single type
> +{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
Do we want to permit anything but a complex type for 'returns'?
For what it's worth, docs/qmp/qmp-spec.txt specifies:
2.4.1 success
-------------
The format of a success response is:
{ "return": json-object, "id": json-value }
Where,
- The "return" member contains the command returned data, which is defined
in a per-command basis or an empty json-object if the command does not
return data
- The "id" member contains the transaction identification associated
with the command execution if issued by the Client
The QAPI schema's 'returns' becomes "return" on the wire. We suck.
qmp-spec.txt is *wrong*! We actually use json-array in addition to
json-object.
Similar argument on types wanted as for 'data' / "arguments" above. I
think we should permit exactly the same QAPI types, plus lists.
> diff --git a/tests/qapi-schema/returns-array-bad.out b/tests/qapi-schema/returns-array-bad.out
> new file mode 100644
> index 0000000..eccad55
> --- /dev/null
> +++ b/tests/qapi-schema/returns-array-bad.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('returns', ['str', 'str'])])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/returns-int.err b/tests/qapi-schema/returns-int.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/returns-int.exit b/tests/qapi-schema/returns-int.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/returns-int.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/returns-int.json b/tests/qapi-schema/returns-int.json
> new file mode 100644
> index 0000000..7888fb1
> --- /dev/null
> +++ b/tests/qapi-schema/returns-int.json
> @@ -0,0 +1,2 @@
> +# It is okay (although not extensible) to return a non-dictionary
> +{ 'command': 'okay', 'returns': 'int' }
> diff --git a/tests/qapi-schema/returns-int.out b/tests/qapi-schema/returns-int.out
> new file mode 100644
> index 0000000..36b00a9
> --- /dev/null
> +++ b/tests/qapi-schema/returns-int.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'okay'), ('returns', 'int')])]
> +[]
> +[]
> diff --git a/tests/qapi-schema/returns-unknown.err b/tests/qapi-schema/returns-unknown.err
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/returns-unknown.exit b/tests/qapi-schema/returns-unknown.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/returns-unknown.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/returns-unknown.json b/tests/qapi-schema/returns-unknown.json
> new file mode 100644
> index 0000000..61f20eb
> --- /dev/null
> +++ b/tests/qapi-schema/returns-unknown.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject returns if it does not contain a known type
> +{ 'command': 'oops', 'returns': 'NoSuchType' }
> diff --git a/tests/qapi-schema/returns-unknown.out b/tests/qapi-schema/returns-unknown.out
> new file mode 100644
> index 0000000..a482c83
> --- /dev/null
> +++ b/tests/qapi-schema/returns-unknown.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
> +[]
> +[]
scripts/qapi* is a sick joke when it comes to semantic analysis.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-25 7:34 ` Markus Armbruster
@ 2014-09-25 8:06 ` Markus Armbruster
2014-09-25 14:00 ` Eric Blake
2014-09-25 13:54 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
1 sibling, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-25 8:06 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> Demonstrate that the qapi generator silently parses confusing
>> types, which may cause other errors later on. Later patches
>> will update the expected results as the generator is made stricter.
[...]
>> diff --git a/tests/qapi-schema/returns-array-bad.json
>> b/tests/qapi-schema/returns-array-bad.json
>> new file mode 100644
>> index 0000000..14882c1
>> --- /dev/null
>> +++ b/tests/qapi-schema/returns-array-bad.json
>> @@ -0,0 +1,2 @@
>> +# FIXME: we should reject an array return that is not a single type
>> +{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
Yes, we want this test, and your remaining tests of 'returns' are fine,
too.
> Do we want to permit anything but a complex type for 'returns'?
>
> For what it's worth, docs/qmp/qmp-spec.txt specifies:
>
> 2.4.1 success
> -------------
>
> The format of a success response is:
>
> { "return": json-object, "id": json-value }
>
> Where,
>
> - The "return" member contains the command returned data, which is defined
> in a per-command basis or an empty json-object if the command does not
> return data
> - The "id" member contains the transaction identification associated
> with the command execution if issued by the Client
>
> The QAPI schema's 'returns' becomes "return" on the wire. We suck.
>
> qmp-spec.txt is *wrong*! We actually use json-array in addition to
> json-object.
Actually, we use json-int and json-str as well:
query-migrate-cache-size, ringbuf-read, human-monitor-command.
> Similar argument on types wanted as for 'data' / "arguments" above. I
> think we should permit exactly the same QAPI types, plus lists.
The similarity to 'data' just isn't there. Separate analysis needed.
Any QAPI types that don't make sense, other than list with length != 1?
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-25 7:34 ` Markus Armbruster
2014-09-25 8:06 ` Markus Armbruster
@ 2014-09-25 13:54 ` Eric Blake
2014-09-25 16:12 ` Markus Armbruster
1 sibling, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-25 13:54 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 5908 bytes --]
On 09/25/2014 01:34 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Demonstrate that the qapi generator silently parses confusing
>> types, which may cause other errors later on. Later patches
>> will update the expected results as the generator is made stricter.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> ---
>> tests/Makefile | 8 ++++++--
>> tests/qapi-schema/data-array-empty.err | 0
>> tests/qapi-schema/data-array-empty.exit | 1 +
>> tests/qapi-schema/data-array-empty.json | 2 ++
> [Twelve new tests...]
>> create mode 100644 tests/qapi-schema/returns-unknown.err
>> create mode 100644 tests/qapi-schema/returns-unknown.exit
>> create mode 100644 tests/qapi-schema/returns-unknown.json
>> create mode 100644 tests/qapi-schema/returns-unknown.out
>
> Holy moly!
Yeah, this series cleans up a lot of cruft, which means a lot of testing.
>> +++ b/tests/qapi-schema/data-array-empty.json
>> @@ -0,0 +1,2 @@
>> +# FIXME: we should reject an array for data if it does not contain a known type
>> +{ 'command': 'oops', 'data': [ ] }
>
> Do we want to permit anything but a complex type for 'data'?
Oh, good question. Probably not (I just tested, and nothing already
does that). I'll tighten it in v5 (mostly doc changes, plus a one-liner
in 13/19 to remove allow_array=True when calling check_type for a
command data member).
>
> For what it's worth, docs/qmp/qmp-spec.txt specifies:
Ooh, I probably ought to skim that file when making my doc improvements
on the front end of the series.
>
> 'data' of list type / json-array "arguments" is an ordered list of
> unnamed parameters. Makes sense, but it isn't how QMP works. Or C for
> that matter. Do we really want to support this in QAPI?
No existing command takes a non-dict for "arguments", and the generator
probably chokes on it.
>
> If yes, then 'data': [] means the same thing as 'data': {} or no 'data':
> no arguments.
>
> Aside: discussion of list types in qapi-code-gen.txt is spread over the
> places that use them. I feel giving them their own section on the same
> level as complex types etc. would make sense.
Good idea, will do in v5.
>
> 'data' of built-in or enumeration type / json-number or json-string
> "arguments" could be regarded as a single unnamed parameter. Even if we
> want unnamed parameters, the question remains whether we want two
> syntactic forms for a single unnamed parameter, boxed in a [ ] and
> unboxed. I doubt it.
No. We don't (patch 13/19 already forbids them, with no violators
found). It's not extensible (well, maybe it is by having some way to
mark a dict so that at most one of its keys is the default key to be
implied when used in a non-dict form, and all other keys being optional,
but that's ugly).
>
> One kind of types left to discuss: unions. I figure the semantics of a
> simple or flat union type are obvious enough, so we can discuss whether
> we want them. Anonymous unions are a different matter, because they
> boil down to a single parameter that need not be json-object.
Oooh, I didn't even consider anon unions. We absolutely need to support
{ 'command': 'foo', 'data': 'FlatUnion' } (blockdev-add, anyone), but
you are probably right that we don't want to support { 'command': 'foo',
'data': 'AnonUnion'}, because it would allow "arguments" to be a
non-dictionary (unless the AnonUnion has only a dict branch, but then
why make it a union?). I'll have to make qapi.py be smarter about
regular vs. anon unions - it might be easier by using an actual
different keyword for anon unions, because they are so different in
nature. (Generated code will be the same, just a tweak to the qapi
representation and to qapi.py). I'll play with that for v5.
>> +++ b/tests/qapi-schema/data-member-array-bad.json
>> @@ -0,0 +1,2 @@
>> +# FIXME: we should reject data if it does not contain a valid array type
>> +{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
>
> I'm probably just suffering from temporary denseness here... why is this
> example problematic? To me, it looks like a single argument 'member' of
> type "array of complex type with a single member 'nested' of
> builtin-type 'str'".
The generator does not have a way to produce a List of an unnamed type.
All lists are of named types (or rather, every creation of a named type
generates code for both that type and its list counterpart). Maybe we
eventually want to support an array of an anonymous type, but the
generator doesn't handle it now. So it was easier to forbid it when
writing 13/19.
>> +# FIXME: we should reject an array return that is not a single type
>> +{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
>
> Do we want to permit anything but a complex type for 'returns'?
Sadly, yes. We have existing commands that do just that. I already
documented that new commands should avoid it (it's not extensible).
>
> The QAPI schema's 'returns' becomes "return" on the wire. We suck.
We could search-and-replace the schema, but why bother. Yeah, the
discrepancy is a bit annoying; on the other hand, it makes it easy to
tell schema apart from on-the-wire samples, at least for commands that
return something :)
>
> qmp-spec.txt is *wrong*! We actually use json-array in addition to
> json-object.
Yep, added to my list of doc improvements for v5.
>> +++ b/tests/qapi-schema/returns-unknown.out
>> @@ -0,0 +1,3 @@
>> +[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
>> +[]
>> +[]
>
> scripts/qapi* is a sick joke when it comes to semantic analysis.
That gets a lot better in 13/19 :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-25 8:06 ` Markus Armbruster
@ 2014-09-25 14:00 ` Eric Blake
2014-09-25 16:19 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-25 14:00 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 1513 bytes --]
On 09/25/2014 02:06 AM, Markus Armbruster wrote:
>>
>> The QAPI schema's 'returns' becomes "return" on the wire. We suck.
>>
>> qmp-spec.txt is *wrong*! We actually use json-array in addition to
>> json-object.
>
> Actually, we use json-int and json-str as well:
> query-migrate-cache-size, ringbuf-read, human-monitor-command.
>
>> Similar argument on types wanted as for 'data' / "arguments" above. I
>> think we should permit exactly the same QAPI types, plus lists.
>
> The similarity to 'data' just isn't there. Separate analysis needed.
Correct. 'data' and 'returns' are different beasts when it comes to
acceptable types. And different still from the acceptable type of each
member of a dictionary. But my check_type function in 13/19 is flexible
enough to cover all the cases.
>
> Any QAPI types that don't make sense, other than list with length != 1?
Return of an anon union isn't used yet, but _might_ make sense (as the
only feasible way of changing existing commands that return an array or
primitive extensible to instead return a dict) - except that back-compat
demands that we can't return a dict in place of a primitive unless the
arguments of the command are also enhanced (that is, older callers are
not expecting a dict, so we can't return a dict unless the caller
witnesses they are new enough by explicitly asking for a dict return).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-25 13:54 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
@ 2014-09-25 16:12 ` Markus Armbruster
2014-09-25 16:32 ` Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-25 16:12 UTC (permalink / raw)
To: Eric Blake; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 09/25/2014 01:34 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Demonstrate that the qapi generator silently parses confusing
>>> types, which may cause other errors later on. Later patches
>>> will update the expected results as the generator is made stricter.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>> ---
>>> tests/Makefile | 8 ++++++--
>>> tests/qapi-schema/data-array-empty.err | 0
>>> tests/qapi-schema/data-array-empty.exit | 1 +
>>> tests/qapi-schema/data-array-empty.json | 2 ++
>> [Twelve new tests...]
>>> create mode 100644 tests/qapi-schema/returns-unknown.err
>>> create mode 100644 tests/qapi-schema/returns-unknown.exit
>>> create mode 100644 tests/qapi-schema/returns-unknown.json
>>> create mode 100644 tests/qapi-schema/returns-unknown.out
>>
>> Holy moly!
>
> Yeah, this series cleans up a lot of cruft, which means a lot of testing.
Very much appreciated.
>>> +++ b/tests/qapi-schema/data-array-empty.json
>>> @@ -0,0 +1,2 @@
>>> +# FIXME: we should reject an array for data if it does not contain
>>> a known type
>>> +{ 'command': 'oops', 'data': [ ] }
>>
>> Do we want to permit anything but a complex type for 'data'?
>
> Oh, good question. Probably not (I just tested, and nothing already
> does that). I'll tighten it in v5 (mostly doc changes, plus a one-liner
> in 13/19 to remove allow_array=True when calling check_type for a
> command data member).
Yes, please.
>> For what it's worth, docs/qmp/qmp-spec.txt specifies:
>
> Ooh, I probably ought to skim that file when making my doc improvements
> on the front end of the series.
>
>> 'data' of list type / json-array "arguments" is an ordered list of
>> unnamed parameters. Makes sense, but it isn't how QMP works. Or C for
>> that matter. Do we really want to support this in QAPI?
>
> No existing command takes a non-dict for "arguments", and the generator
> probably chokes on it.
Let's stick to dict arguments.
>> If yes, then 'data': [] means the same thing as 'data': {} or no 'data':
>> no arguments.
>>
>> Aside: discussion of list types in qapi-code-gen.txt is spread over the
>> places that use them. I feel giving them their own section on the same
>> level as complex types etc. would make sense.
>
> Good idea, will do in v5.
>
>>
>> 'data' of built-in or enumeration type / json-number or json-string
>> "arguments" could be regarded as a single unnamed parameter. Even if we
>> want unnamed parameters, the question remains whether we want two
>> syntactic forms for a single unnamed parameter, boxed in a [ ] and
>> unboxed. I doubt it.
>
> No. We don't (patch 13/19 already forbids them, with no violators
> found). It's not extensible (well, maybe it is by having some way to
> mark a dict so that at most one of its keys is the default key to be
> implied when used in a non-dict form, and all other keys being optional,
> but that's ugly).
Agreed.
>> One kind of types left to discuss: unions. I figure the semantics of a
>> simple or flat union type are obvious enough, so we can discuss whether
>> we want them. Anonymous unions are a different matter, because they
>> boil down to a single parameter that need not be json-object.
>
> Oooh, I didn't even consider anon unions. We absolutely need to support
> { 'command': 'foo', 'data': 'FlatUnion' } (blockdev-add, anyone), but
> you are probably right that we don't want to support { 'command': 'foo',
> 'data': 'AnonUnion'}, because it would allow "arguments" to be a
> non-dictionary (unless the AnonUnion has only a dict branch, but then
> why make it a union?). I'll have to make qapi.py be smarter about
> regular vs. anon unions - it might be easier by using an actual
> different keyword for anon unions, because they are so different in
> nature. (Generated code will be the same, just a tweak to the qapi
> representation and to qapi.py). I'll play with that for v5.
Okay :)
>>> +++ b/tests/qapi-schema/data-member-array-bad.json
>>> @@ -0,0 +1,2 @@
>>> +# FIXME: we should reject data if it does not contain a valid array type
>>> +{ 'command': 'oops', 'data': { 'member': [ { 'nested': 'str' } ] } }
>>
>> I'm probably just suffering from temporary denseness here... why is this
>> example problematic? To me, it looks like a single argument 'member' of
>> type "array of complex type with a single member 'nested' of
>> builtin-type 'str'".
>
> The generator does not have a way to produce a List of an unnamed type.
> All lists are of named types (or rather, every creation of a named type
> generates code for both that type and its list counterpart). Maybe we
> eventually want to support an array of an anonymous type, but the
> generator doesn't handle it now. So it was easier to forbid it when
> writing 13/19.
I see. We already accepted restricting nested structs (see series
subject), and this is just one instance.
>>> +# FIXME: we should reject an array return that is not a single type
>>> +{ 'command': 'oops', 'returns': [ 'str', 'str' ] }
>>
>> Do we want to permit anything but a complex type for 'returns'?
>
> Sadly, yes. We have existing commands that do just that. I already
> documented that new commands should avoid it (it's not extensible).
If we care, we can whitelist the existing offenders, and reject new
offenders.
>> The QAPI schema's 'returns' becomes "return" on the wire. We suck.
>
> We could search-and-replace the schema, but why bother. Yeah, the
> discrepancy is a bit annoying; on the other hand, it makes it easy to
> tell schema apart from on-the-wire samples, at least for commands that
> return something :)
>
>>
>> qmp-spec.txt is *wrong*! We actually use json-array in addition to
>> json-object.
>
> Yep, added to my list of doc improvements for v5.
>
>
>>> +++ b/tests/qapi-schema/returns-unknown.out
>>> @@ -0,0 +1,3 @@
>>> +[OrderedDict([('command', 'oops'), ('returns', 'NoSuchType')])]
>>> +[]
>>> +[]
>>
>> scripts/qapi* is a sick joke when it comes to semantic analysis.
>
> That gets a lot better in 13/19 :)
Will review as soon as I can!
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-25 14:00 ` Eric Blake
@ 2014-09-25 16:19 ` Markus Armbruster
2015-03-23 15:33 ` [Qemu-devel] RFC: 'alternate' qapi top-level expression [was: [PATCH v4 12/19] qapi: Add some type check tests] Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-25 16:19 UTC (permalink / raw)
To: Eric Blake; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 09/25/2014 02:06 AM, Markus Armbruster wrote:
>
>>>
>>> The QAPI schema's 'returns' becomes "return" on the wire. We suck.
>>>
>>> qmp-spec.txt is *wrong*! We actually use json-array in addition to
>>> json-object.
>>
>> Actually, we use json-int and json-str as well:
>> query-migrate-cache-size, ringbuf-read, human-monitor-command.
>>
>>> Similar argument on types wanted as for 'data' / "arguments" above. I
>>> think we should permit exactly the same QAPI types, plus lists.
>>
>> The similarity to 'data' just isn't there. Separate analysis needed.
>
> Correct. 'data' and 'returns' are different beasts when it comes to
> acceptable types. And different still from the acceptable type of each
> member of a dictionary. But my check_type function in 13/19 is flexible
> enough to cover all the cases.
>
>>
>> Any QAPI types that don't make sense, other than list with length != 1?
>
> Return of an anon union isn't used yet, but _might_ make sense (as the
> only feasible way of changing existing commands that return an array or
> primitive extensible to instead return a dict) -
Good point.
> except that back-compat
> demands that we can't return a dict in place of a primitive unless the
> arguments of the command are also enhanced (that is, older callers are
> not expecting a dict, so we can't return a dict unless the caller
> witnesses they are new enough by explicitly asking for a dict return).
I think we can keep things simple for now and reject anonymous unions.
We can always relax the check when we run into a use.
You're giving the generator a good shove from "god knows what it
accepts, but as long as you stick to stuff that is being used already,
probably generates something that works" towards "if it accepts it, it
works". I like it.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests
2014-09-25 16:12 ` Markus Armbruster
@ 2014-09-25 16:32 ` Eric Blake
0 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-25 16:32 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 689 bytes --]
On 09/25/2014 10:12 AM, Markus Armbruster wrote:
>>> Do we want to permit anything but a complex type for 'returns'?
>>
>> Sadly, yes. We have existing commands that do just that. I already
>> documented that new commands should avoid it (it's not extensible).
>
> If we care, we can whitelist the existing offenders, and reject new
> offenders.
Also a good idea. The whitelist may grow over time, but forcing a
developer to modify the whitelist calls attention to their action :)
I'll add that to my v5 queue. Thanks again for a thought-provoking review.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-24 9:25 ` Kevin Wolf
2014-09-24 11:14 ` Markus Armbruster
@ 2014-09-26 9:15 ` Markus Armbruster
2014-09-26 9:25 ` Kevin Wolf
1 sibling, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-26 9:15 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Kevin Wolf <kwolf@redhat.com> writes:
> Am 24.09.2014 um 09:34 hat Markus Armbruster geschrieben:
>> Eric Blake <eblake@redhat.com> writes:
>>
>> > On 09/23/2014 08:56 AM, Markus Armbruster wrote:
>> >> Eric Blake <eblake@redhat.com> writes:
>> >>> Add check_keys to cover these situations, and update testcases to
>> >>> match. A couple other tests (enum-missing-data, indented-expr) had
>> >>> to change since the validation added here occurs so early.
>> >>>
>> >>> While valid .json files won't trigger any of these cases, we might
>> >>> as well be nicer to developers that make a typo while trying to add
>> >>> new QAPI code.
>> >>>
>> >>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> >>
>> >
>> >>> +def check_keys(expr_elem, meta, required, optional=[]):
>> >>> + expr = expr_elem['expr']
>> >>> + info = expr_elem['info']
>> >>> + name = expr[meta]
>> >>
>> >> Caller ensures expr[meta] exists. Okay.
>> >>
>> >>> + if not isinstance(name, basestring):
>> >>
>> >> I'm a Python noob: why basestring and not str?
>> >
>> > Me too. No clue. Copy and paste from existing code.
>> > http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
>>
>> Yes. It's our only use of basestring. Other places use isinstance(FOO,
>> str).
>>
>> The basestring use comes from Kevin's commit b35284e. Kevin, why
>> basestring and not str?
>
> Do I look as if I knew what I'm doing when I write Python code? :-)
>
> Apparently basestring is the superclass for ASCII and Unicode strings. I
> seem to dimly remember that I did indeed get a Unicode string somewhere
> (even though probably no non-ASCII characters in it) and it caused
> trouble. Might well have been here.
I think there are two sane ways forward:
1. Declare QAPI schemas to be ASCII only for now. Replace the one
instance of basestring by plain str.
2. Fix the code to work with both ASCII and Unicode strings even in
Python 2. Package six could be used.
I'm voting for 1.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-26 9:15 ` Markus Armbruster
@ 2014-09-26 9:25 ` Kevin Wolf
2014-09-26 11:40 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2014-09-26 9:25 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Am 26.09.2014 um 11:15 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 24.09.2014 um 09:34 hat Markus Armbruster geschrieben:
> >> Eric Blake <eblake@redhat.com> writes:
> >>
> >> > On 09/23/2014 08:56 AM, Markus Armbruster wrote:
> >> >> Eric Blake <eblake@redhat.com> writes:
> >> >>> Add check_keys to cover these situations, and update testcases to
> >> >>> match. A couple other tests (enum-missing-data, indented-expr) had
> >> >>> to change since the validation added here occurs so early.
> >> >>>
> >> >>> While valid .json files won't trigger any of these cases, we might
> >> >>> as well be nicer to developers that make a typo while trying to add
> >> >>> new QAPI code.
> >> >>>
> >> >>> Signed-off-by: Eric Blake <eblake@redhat.com>
> >> >>
> >> >
> >> >>> +def check_keys(expr_elem, meta, required, optional=[]):
> >> >>> + expr = expr_elem['expr']
> >> >>> + info = expr_elem['info']
> >> >>> + name = expr[meta]
> >> >>
> >> >> Caller ensures expr[meta] exists. Okay.
> >> >>
> >> >>> + if not isinstance(name, basestring):
> >> >>
> >> >> I'm a Python noob: why basestring and not str?
> >> >
> >> > Me too. No clue. Copy and paste from existing code.
> >> > http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
> >>
> >> Yes. It's our only use of basestring. Other places use isinstance(FOO,
> >> str).
> >>
> >> The basestring use comes from Kevin's commit b35284e. Kevin, why
> >> basestring and not str?
> >
> > Do I look as if I knew what I'm doing when I write Python code? :-)
> >
> > Apparently basestring is the superclass for ASCII and Unicode strings. I
> > seem to dimly remember that I did indeed get a Unicode string somewhere
> > (even though probably no non-ASCII characters in it) and it caused
> > trouble. Might well have been here.
>
> I think there are two sane ways forward:
>
> 1. Declare QAPI schemas to be ASCII only for now. Replace the one
> instance of basestring by plain str.
>
> 2. Fix the code to work with both ASCII and Unicode strings even in
> Python 2. Package six could be used.
>
> I'm voting for 1.
As I said, the problem is probably not about the actual value of the
string (I doubt we use anything but ASCII at the moment), but about its
type. We may need to convert something somewhere. If you think it's
worth the effort of finding that place, go ahead.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types Eric Blake
@ 2014-09-26 9:26 ` Markus Armbruster
2014-09-29 8:27 ` Markus Armbruster
2014-09-29 14:35 ` Eric Blake
0 siblings, 2 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-26 9:26 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Now that we know every expression is valid with regards to
> its keys, we can add further tests that those keys refer to
> valid types. With this patch, all references to a type (the
> 'data': of command, type, union, and event, and the 'returns':
> of command) must resolve to a builtin or another type declared
> by the current qapi parse; this includes recursing into each
> member of a data dictionary. Dealing with '**' and nested
> sub-structs will be done in later patches.
>
> Update the testsuite to match improved output.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 73 ++++++++++++++++++++++++++--
> tests/qapi-schema/data-array-empty.err | 1 +
> tests/qapi-schema/data-array-empty.exit | 2 +-
> tests/qapi-schema/data-array-empty.json | 2 +-
> tests/qapi-schema/data-array-empty.out | 3 --
> tests/qapi-schema/data-array-unknown.err | 1 +
> tests/qapi-schema/data-array-unknown.exit | 2 +-
> tests/qapi-schema/data-array-unknown.json | 2 +-
> tests/qapi-schema/data-array-unknown.out | 3 --
> tests/qapi-schema/data-int.err | 1 +
> tests/qapi-schema/data-int.exit | 2 +-
> tests/qapi-schema/data-int.json | 2 +-
> tests/qapi-schema/data-int.out | 3 --
> tests/qapi-schema/data-member-array-bad.err | 1 +
> tests/qapi-schema/data-member-array-bad.exit | 2 +-
> tests/qapi-schema/data-member-array-bad.json | 2 +-
> tests/qapi-schema/data-member-array-bad.out | 3 --
> tests/qapi-schema/data-member-unknown.err | 1 +
> tests/qapi-schema/data-member-unknown.exit | 2 +-
> tests/qapi-schema/data-member-unknown.json | 2 +-
> tests/qapi-schema/data-member-unknown.out | 3 --
> tests/qapi-schema/data-unknown.err | 1 +
> tests/qapi-schema/data-unknown.exit | 2 +-
> tests/qapi-schema/data-unknown.json | 2 +-
> tests/qapi-schema/data-unknown.out | 3 --
> tests/qapi-schema/returns-array-bad.err | 1 +
> tests/qapi-schema/returns-array-bad.exit | 2 +-
> tests/qapi-schema/returns-array-bad.json | 2 +-
> tests/qapi-schema/returns-array-bad.out | 3 --
> tests/qapi-schema/returns-unknown.err | 1 +
> tests/qapi-schema/returns-unknown.exit | 2 +-
> tests/qapi-schema/returns-unknown.json | 2 +-
> tests/qapi-schema/returns-unknown.out | 3 --
> 33 files changed, 93 insertions(+), 44 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index bf243fa..20c0ce9 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -255,6 +255,52 @@ def discriminator_find_enum_define(expr):
>
> return find_enum(discriminator_type)
>
> +def check_type(expr_info, source, data, allow_array = False,
> + allowed_names = [], allow_dict = True):
> + global all_types
> +
> + if data is None:
> + return
> +
> + if data == '**':
> + return
> +
> + # Check if array type for data is okay
> + if isinstance(data, list):
> + if not allow_array:
> + raise QAPIExprError(expr_info,
> + "%s cannot be an array" % source)
> + if len(data) != 1 or not isinstance(data[0], basestring):
> + raise QAPIExprError(expr_info,
> + "%s: array type must contain single type name"
> + % source)
> + data = data[0]
> +
> + # Check if type name for data is okay
> + if isinstance(data, basestring):
> + if not data in all_types:
> + raise QAPIExprError(expr_info,
> + "%s references unknown type '%s'"
> + % (source, data))
> + if not all_types[data] in allowed_names:
> + raise QAPIExprError(expr_info,
> + "%s cannot reference %s type '%s'"
> + % (source, all_types[data], data))
> + return
> +
> + # data is a dictionary, check that each member is okay
> + if not isinstance(data, OrderedDict):
> + raise QAPIExprError(expr_info,
> + "%s should be a dictionary" % source)
> + if not allow_dict:
> + raise QAPIExprError(expr_info,
> + "%s should be a type name" % source)
> + for (key, value) in data.items():
> + check_type(expr_info, "member '%s' of %s" % (key, source), value,
> + allow_array=True,
> + allowed_names=['built-in', 'union', 'struct', 'enum'],
> + allow_dict=True)
Hmm, allowed_names isn't about allowed names, it's about allowed
meta-types. Rename?
> +
> def check_command(expr, expr_info):
> global commands
> name = expr['command']
> @@ -262,6 +308,15 @@ def check_command(expr, expr_info):
> raise QAPIExprError(expr_info,
> "command '%s' is already defined" % name)
> commands.append(name)
> + check_type(expr_info, "'data' for command '%s'" % name,
> + expr.get('data'), allow_array=True,
> + allowed_names=['union', 'struct'])
> + check_type(expr_info, "'base' for command '%s'" % name,
> + expr.get('base'), allowed_names=['struct'],
> + allow_dict=False)
> + check_type(expr_info, "'returns' for command '%s'" % name,
> + expr.get('returns'), allow_array=True,
> + allowed_names=['built-in', 'union', 'struct', 'enum'])
>
Nicely done.
> def check_event(expr, expr_info):
> global events
> @@ -271,6 +326,8 @@ def check_event(expr, expr_info):
> raise QAPIExprError(expr_info,
> "event '%s' is already defined" % name)
> events.append(name)
> + check_type(expr_info, "'data' for event '%s'" % name,
> + expr.get('data'), allowed_names=['union', 'struct'])
> if params:
> for argname, argentry, optional, structured in parse_args(params):
> if structured:
> @@ -357,19 +414,27 @@ def check_enum(expr, expr_info):
> % (name, member, values[key]))
> values[key] = member
>
> +def check_struct(expr, expr_info):
> + name = expr['type']
> + members = expr['data']
> +
> + check_type(expr_info, "'data' for type '%s'" % name, members)
> +
> def check_exprs(schema):
> for expr_elem in schema.exprs:
> expr = expr_elem['expr']
> info = expr_elem['info']
>
> - if expr.has_key('union'):
> - check_union(expr, info)
> - if expr.has_key('event'):
> - check_event(expr, info)
> if expr.has_key('enum'):
> check_enum(expr, info)
> + if expr.has_key('union'):
> + check_union(expr, info)
> + if expr.has_key('type'):
> + check_struct(expr, info)
> if expr.has_key('command'):
> check_command(expr, info)
> + if expr.has_key('event'):
> + check_event(expr, info)
Not related to this patch: this chain of ifs bothers me. The conditions
should be exclusive, because each expression must have a well-defined
meta-type. If I remember correctly, you actually enforce this elsewhere
in your series. Do we want to make it obvious in the code here? elif
instead of if, perhaps?
>
> def check_keys(expr_elem, meta, required, optional=[]):
> expr = expr_elem['expr']
> diff --git a/tests/qapi-schema/data-array-empty.err b/tests/qapi-schema/data-array-empty.err
> index e69de29..94e046b 100644
> --- a/tests/qapi-schema/data-array-empty.err
> +++ b/tests/qapi-schema/data-array-empty.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/data-array-empty.json:2: 'data' for command 'oops': array type must contain single type name
> diff --git a/tests/qapi-schema/data-array-empty.exit b/tests/qapi-schema/data-array-empty.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/data-array-empty.exit
> +++ b/tests/qapi-schema/data-array-empty.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/data-array-empty.json b/tests/qapi-schema/data-array-empty.json
> index 41b6c1e..fdd5612 100644
> --- a/tests/qapi-schema/data-array-empty.json
> +++ b/tests/qapi-schema/data-array-empty.json
> @@ -1,2 +1,2 @@
> -# FIXME: we should reject an array for data if it does not contain a known type
> +# we reject an array for data if it does not contain a known type
> { 'command': 'oops', 'data': [ ] }
> diff --git a/tests/qapi-schema/data-array-empty.out b/tests/qapi-schema/data-array-empty.out
> index 67802be..e69de29 100644
> --- a/tests/qapi-schema/data-array-empty.out
> +++ b/tests/qapi-schema/data-array-empty.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('command', 'oops'), ('data', [])])]
> -[]
> -[]
> diff --git a/tests/qapi-schema/data-array-unknown.err b/tests/qapi-schema/data-array-unknown.err
> index e69de29..a66c2d6 100644
> --- a/tests/qapi-schema/data-array-unknown.err
> +++ b/tests/qapi-schema/data-array-unknown.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/data-array-unknown.json:2: 'data' for command 'oops' references unknown type 'NoSuchType'
The wording "references type" somehow unduly excites my "reference type"
synapses :)
Perhaps something like "Type 'NoSuchType' is unknown" would suffice.
Entirely up to you; feel free to keep the wording as is.
> diff --git a/tests/qapi-schema/data-array-unknown.exit b/tests/qapi-schema/data-array-unknown.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/data-array-unknown.exit
> +++ b/tests/qapi-schema/data-array-unknown.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/data-array-unknown.json b/tests/qapi-schema/data-array-unknown.json
> index 434fb5f..9c75b96 100644
> --- a/tests/qapi-schema/data-array-unknown.json
> +++ b/tests/qapi-schema/data-array-unknown.json
> @@ -1,2 +1,2 @@
> -# FIXME: we should reject an array for data if it does not contain a known type
> +# we reject an array for data if it does not contain a known type
> { 'command': 'oops', 'data': [ 'NoSuchType' ] }
> diff --git a/tests/qapi-schema/data-array-unknown.out b/tests/qapi-schema/data-array-unknown.out
> index c3bc05e..e69de29 100644
> --- a/tests/qapi-schema/data-array-unknown.out
> +++ b/tests/qapi-schema/data-array-unknown.out
> @@ -1,3 +0,0 @@
> -[OrderedDict([('command', 'oops'), ('data', ['NoSuchType'])])]
> -[]
> -[]
> diff --git a/tests/qapi-schema/data-int.err b/tests/qapi-schema/data-int.err
> index e69de29..e1d3ed5 100644
> --- a/tests/qapi-schema/data-int.err
> +++ b/tests/qapi-schema/data-int.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/data-int.json:2: 'data' for command 'oops' cannot reference built-in type 'int'
Perhaps something like "'data' for command 'oops' can't be of built-in
type 'int'", or maybe positive instead of negative: "a command's 'data'
must be of complex type".
Again, entirely up to you.
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions
2014-09-26 9:25 ` Kevin Wolf
@ 2014-09-26 11:40 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-26 11:40 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Kevin Wolf <kwolf@redhat.com> writes:
> Am 26.09.2014 um 11:15 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 24.09.2014 um 09:34 hat Markus Armbruster geschrieben:
>> >> Eric Blake <eblake@redhat.com> writes:
>> >>
>> >> > On 09/23/2014 08:56 AM, Markus Armbruster wrote:
>> >> >> Eric Blake <eblake@redhat.com> writes:
>> >> >>> Add check_keys to cover these situations, and update testcases to
>> >> >>> match. A couple other tests (enum-missing-data, indented-expr) had
>> >> >>> to change since the validation added here occurs so early.
>> >> >>>
>> >> >>> While valid .json files won't trigger any of these cases, we might
>> >> >>> as well be nicer to developers that make a typo while trying to add
>> >> >>> new QAPI code.
>> >> >>>
>> >> >>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> >> >>
>> >> >
>> >> >>> +def check_keys(expr_elem, meta, required, optional=[]):
>> >> >>> + expr = expr_elem['expr']
>> >> >>> + info = expr_elem['info']
>> >> >>> + name = expr[meta]
>> >> >>
>> >> >> Caller ensures expr[meta] exists. Okay.
>> >> >>
>> >> >>> + if not isinstance(name, basestring):
>> >> >>
>> >> >> I'm a Python noob: why basestring and not str?
>> >> >
>> >> > Me too. No clue. Copy and paste from existing code.
>> >> > http://git.qemu.org/?p=qemu.git;a=blob;f=scripts/qapi.py;h=77d46aa;hb=769188d3b#l361
>> >>
>> >> Yes. It's our only use of basestring. Other places use isinstance(FOO,
>> >> str).
>> >>
>> >> The basestring use comes from Kevin's commit b35284e. Kevin, why
>> >> basestring and not str?
>> >
>> > Do I look as if I knew what I'm doing when I write Python code? :-)
>> >
>> > Apparently basestring is the superclass for ASCII and Unicode strings. I
>> > seem to dimly remember that I did indeed get a Unicode string somewhere
>> > (even though probably no non-ASCII characters in it) and it caused
>> > trouble. Might well have been here.
>>
>> I think there are two sane ways forward:
>>
>> 1. Declare QAPI schemas to be ASCII only for now. Replace the one
>> instance of basestring by plain str.
>>
>> 2. Fix the code to work with both ASCII and Unicode strings even in
>> Python 2. Package six could be used.
>>
>> I'm voting for 1.
>
> As I said, the problem is probably not about the actual value of the
> string (I doubt we use anything but ASCII at the moment), but about its
> type. We may need to convert something somewhere. If you think it's
> worth the effort of finding that place, go ahead.
A value always has a concrete type. I'd expect all string values in the
qapi scripts to be of type str.
We have to mess with Python 2's basestring only where we need to cope
with string values that may be either of type str or of type unicode.
Since there should not be any values of type unicode, I don't see why
we'd want to do the extra work required for 2.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
` (18 preceding siblings ...)
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 19/19] qapi: Drop support for inline subtypes Eric Blake
@ 2014-09-26 15:42 ` Eric Blake
19 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-26 15:42 UTC (permalink / raw)
To: qemu-devel; +Cc: Fam Zheng, Markus Armbruster, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]
On 09/19/2014 04:24 PM, Eric Blake wrote:
> After dragging my feet due to some libvirt work, I've finally
> revived this patch series as promised.
>
> v3 was here:
> https://lists.gnu.org/archive/html/qemu-devel/2014-08/msg00725.html
>
> v4:
> update commit message style [Markus]
> add docs (patches 3-4 are new) [Markus]
> lots of new tests (spread among 5, 7, 9, 11, 12)
> add comments to all new tests [Markus]
> rework check for valid expressions (patches 8, 10 are pretty much
> new, later patches rebased and simplified as a result)
> rebase to master
> fix tests to match file name (broken test in v3 patch 7 now fixed
> in patch 12)
> drop expr_name (v3 patch 8); series now validates expressions
> earlier (patch 8)
> consolidate '**' magic, and reject 'gen':'yes' in commands (patch
> 14 is new)
> probably other stuff - it's been more than a month, so reviewers
> will probably want to start from scratch anyways :)
I'll be posting a v5 to address Markus' useful feedback, but as that may
take me a while to get to, I've spun off patches 1-3 into their own
trivial series:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05357.html
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types
2014-09-26 9:26 ` Markus Armbruster
@ 2014-09-29 8:27 ` Markus Armbruster
2014-09-29 14:26 ` Eric Blake
2014-09-29 14:35 ` Eric Blake
1 sibling, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-29 8:27 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> Now that we know every expression is valid with regards to
>> its keys, we can add further tests that those keys refer to
>> valid types. With this patch, all references to a type (the
>> 'data': of command, type, union, and event, and the 'returns':
>> of command) must resolve to a builtin or another type declared
>> by the current qapi parse; this includes recursing into each
>> member of a data dictionary. Dealing with '**' and nested
>> sub-structs will be done in later patches.
>>
>> Update the testsuite to match improved output.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
[...]
>> @@ -262,6 +308,15 @@ def check_command(expr, expr_info):
>> raise QAPIExprError(expr_info,
>> "command '%s' is already defined" % name)
>> commands.append(name)
>> + check_type(expr_info, "'data' for command '%s'" % name,
>> + expr.get('data'), allow_array=True,
>> + allowed_names=['union', 'struct'])
>> + check_type(expr_info, "'base' for command '%s'" % name,
>> + expr.get('base'), allowed_names=['struct'],
>> + allow_dict=False)
>> + check_type(expr_info, "'returns' for command '%s'" % name,
>> + expr.get('returns'), allow_array=True,
>> + allowed_names=['built-in', 'union', 'struct', 'enum'])
>>
>
> Nicely done.
Wait a sec! What's a command's 'base'?
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass Eric Blake
@ 2014-09-29 8:38 ` Markus Armbruster
2014-09-29 14:33 ` Eric Blake
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2014-09-29 8:38 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> Now that we have a way to validate every type, we can also be
> stricter about enforcing that callers that want to bypass
> type safety in generated code. Prior to this patch, it didn't
> matter what value was associated with the key 'gen', but it
> looked odd that 'gen':'yes' could result in bypassing the
> generated code. These changes also enforce the changes made
> earlier in the series for documentation and consolidation of
> using '**' as the wildcard type.
Perhaps it's worth mentioning that the schema has always used 'gen':
'no'.
That said, 'no' is silly. The natural JSON for a flag is false / true!
Since you're touching it anyway, consider making it an optional boolean
defaulting to false. Same for the other silly 'no': success-response.
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> scripts/qapi.py | 21 ++++++++++++++++-----
> tests/qapi-schema/type-bypass-bad-gen.err | 1 +
> tests/qapi-schema/type-bypass-bad-gen.exit | 2 +-
> tests/qapi-schema/type-bypass-bad-gen.json | 2 +-
> tests/qapi-schema/type-bypass-bad-gen.out | 3 ---
> tests/qapi-schema/type-bypass-no-gen.err | 1 +
> tests/qapi-schema/type-bypass-no-gen.exit | 2 +-
> tests/qapi-schema/type-bypass-no-gen.json | 2 +-
> tests/qapi-schema/type-bypass-no-gen.out | 3 ---
> 9 files changed, 22 insertions(+), 15 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 20c0ce9..15972c6 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -256,13 +256,13 @@ def discriminator_find_enum_define(expr):
> return find_enum(discriminator_type)
>
> def check_type(expr_info, source, data, allow_array = False,
> - allowed_names = [], allow_dict = True):
> + allowed_names = [], allow_dict = True, allow_star = False):
> global all_types
>
> if data is None:
> return
>
> - if data == '**':
> + if allow_star and data == '**':
> return
>
> # Check if array type for data is okay
> @@ -278,6 +278,10 @@ def check_type(expr_info, source, data, allow_array = False,
>
> # Check if type name for data is okay
> if isinstance(data, basestring):
> + if data == '**':
> + raise QAPIExprError(expr_info,
> + "%s uses '**' but did not request 'gen':'no'"
> + % source)
> if not data in all_types:
> raise QAPIExprError(expr_info,
> "%s references unknown type '%s'"
I'm blind: I can't see where this error gets suppressed when we have
'gen': 'no'.
> @@ -299,24 +303,27 @@ def check_type(expr_info, source, data, allow_array = False,
> check_type(expr_info, "member '%s' of %s" % (key, source), value,
> allow_array=True,
> allowed_names=['built-in', 'union', 'struct', 'enum'],
> - allow_dict=True)
> + allow_dict=True, allow_star=allow_star)
>
allow_star applies recursively. Correct, because...
> def check_command(expr, expr_info):
> global commands
> name = expr['command']
> + allow_star = expr.has_key('gen')
> +
> if name in commands:
> raise QAPIExprError(expr_info,
> "command '%s' is already defined" % name)
> commands.append(name)
> check_type(expr_info, "'data' for command '%s'" % name,
> expr.get('data'), allow_array=True,
> - allowed_names=['union', 'struct'])
> + allowed_names=['union', 'struct'], allow_star=allow_star)
> check_type(expr_info, "'base' for command '%s'" % name,
> expr.get('base'), allowed_names=['struct'],
> allow_dict=False)
> check_type(expr_info, "'returns' for command '%s'" % name,
> expr.get('returns'), allow_array=True,
> - allowed_names=['built-in', 'union', 'struct', 'enum'])
> + allowed_names=['built-in', 'union', 'struct', 'enum'],
> + allow_star=allow_star)
>
... it applies exactly to a command's 'data' and 'returns' when it has
'gen': 'no'. Good.
> def check_event(expr, expr_info):
> global events
> @@ -450,6 +457,10 @@ def check_keys(expr_elem, meta, required, optional=[]):
> raise QAPIExprError(info,
> "%s '%s' has unknown key '%s'"
> % (meta, name, key))
> + if (key == 'gen' or key == 'success-response') and value != 'no':
> + raise QAPIExprError(info,
> + "'%s' of %s '%s' should only have value 'no'"
> + % (key, meta, name))
> for key in required:
> if not expr.has_key(key):
> raise QAPIExprError(info,
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types
2014-09-29 8:27 ` Markus Armbruster
@ 2014-09-29 14:26 ` Eric Blake
0 siblings, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-29 14:26 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]
On 09/29/2014 02:27 AM, Markus Armbruster wrote:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Now that we know every expression is valid with regards to
>>> its keys, we can add further tests that those keys refer to
>>> valid types. With this patch, all references to a type (the
>>> 'data': of command, type, union, and event, and the 'returns':
>>> of command) must resolve to a builtin or another type declared
>>> by the current qapi parse; this includes recursing into each
>>> member of a data dictionary. Dealing with '**' and nested
>>> sub-structs will be done in later patches.
>>>
>>> Update the testsuite to match improved output.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
> [...]
>>> @@ -262,6 +308,15 @@ def check_command(expr, expr_info):
>>> raise QAPIExprError(expr_info,
>>> "command '%s' is already defined" % name)
>>> commands.append(name)
>>> + check_type(expr_info, "'data' for command '%s'" % name,
>>> + expr.get('data'), allow_array=True,
>>> + allowed_names=['union', 'struct'])
>>> + check_type(expr_info, "'base' for command '%s'" % name,
>>> + expr.get('base'), allowed_names=['struct'],
>>> + allow_dict=False)
>>
>> Nicely done.
>
> Wait a sec! What's a command's 'base'?
Blech. You're right - only 'union' and 'struct' have a base. And since
an earlier patch already filtered out 'base' as a non-allowed key for
'command, this check_type is dead code. All the more reason for me to
spin v5 :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass
2014-09-29 8:38 ` Markus Armbruster
@ 2014-09-29 14:33 ` Eric Blake
2014-09-29 16:35 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2014-09-29 14:33 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 5302 bytes --]
On 09/29/2014 02:38 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Now that we have a way to validate every type, we can also be
>> stricter about enforcing that callers that want to bypass
>> type safety in generated code. Prior to this patch, it didn't
>> matter what value was associated with the key 'gen', but it
>> looked odd that 'gen':'yes' could result in bypassing the
>> generated code. These changes also enforce the changes made
>> earlier in the series for documentation and consolidation of
>> using '**' as the wildcard type.
>
> Perhaps it's worth mentioning that the schema has always used 'gen':
> 'no'.
>
> That said, 'no' is silly. The natural JSON for a flag is false / true!
> Since you're touching it anyway, consider making it an optional boolean
> defaulting to false. Same for the other silly 'no': success-response.
I'd love to, except that the .json parser does not yet allow literal
true/false JSON keywords. Fam had a patch back in May that would fix
that; maybe I'll fold in his patch to my v5 as a prereq patch:
https://lists.gnu.org/archive/html/qemu-devel/2014-05/msg03916.html
>> @@ -256,13 +256,13 @@ def discriminator_find_enum_define(expr):
>> return find_enum(discriminator_type)
>>
>> def check_type(expr_info, source, data, allow_array = False,
>> - allowed_names = [], allow_dict = True):
>> + allowed_names = [], allow_dict = True, allow_star = False):
>> global all_types
>>
>> if data is None:
>> return
>>
>> - if data == '**':
>> + if allow_star and data == '**':
>> return
[1]
>>
>> # Check if array type for data is okay
>> @@ -278,6 +278,10 @@ def check_type(expr_info, source, data, allow_array = False,
>>
>> # Check if type name for data is okay
>> if isinstance(data, basestring):
>> + if data == '**':
>> + raise QAPIExprError(expr_info,
>> + "%s uses '**' but did not request 'gen':'no'"
>> + % source)
>> if not data in all_types:
>> raise QAPIExprError(expr_info,
>> "%s references unknown type '%s'"
>
> I'm blind: I can't see where this error gets suppressed when we have
> 'gen': 'no'.
'gen':'no' triggers the caller to pass allow_star=True to check_type
[2]; then at point [1] we short-circuit and exit if '**' was passed.
Therefore, if we get here and still have '**', then allow_star is still
false, which means 'gen':'no' was not passed and it is user error.
>
>> @@ -299,24 +303,27 @@ def check_type(expr_info, source, data, allow_array = False,
>> check_type(expr_info, "member '%s' of %s" % (key, source), value,
>> allow_array=True,
>> allowed_names=['built-in', 'union', 'struct', 'enum'],
>> - allow_dict=True)
>> + allow_dict=True, allow_star=allow_star)
>>
>
> allow_star applies recursively. Correct, because...
>
>> def check_command(expr, expr_info):
>> global commands
>> name = expr['command']
>> + allow_star = expr.has_key('gen')
>> +
[2] The other trick to note here is that this only checks if 'gen' is a
key; but at [3], which is run earlier, we further required that 'gen'
can only appear if it has value 'no'.
>> if name in commands:
>> raise QAPIExprError(expr_info,
>> "command '%s' is already defined" % name)
>> commands.append(name)
>> check_type(expr_info, "'data' for command '%s'" % name,
>> expr.get('data'), allow_array=True,
>> - allowed_names=['union', 'struct'])
>> + allowed_names=['union', 'struct'], allow_star=allow_star)
>> check_type(expr_info, "'base' for command '%s'" % name,
>> expr.get('base'), allowed_names=['struct'],
>> allow_dict=False)
>> check_type(expr_info, "'returns' for command '%s'" % name,
>> expr.get('returns'), allow_array=True,
>> - allowed_names=['built-in', 'union', 'struct', 'enum'])
>> + allowed_names=['built-in', 'union', 'struct', 'enum'],
>> + allow_star=allow_star)
>>
>
> ... it applies exactly to a command's 'data' and 'returns' when it has
> 'gen': 'no'. Good.
>
>> def check_event(expr, expr_info):
>> global events
>> @@ -450,6 +457,10 @@ def check_keys(expr_elem, meta, required, optional=[]):
>> raise QAPIExprError(info,
>> "%s '%s' has unknown key '%s'"
>> % (meta, name, key))
>> + if (key == 'gen' or key == 'success-response') and value != 'no':
>> + raise QAPIExprError(info,
>> + "'%s' of %s '%s' should only have value 'no'"
>> + % (key, meta, name))
[3]
>> for key in required:
>> if not expr.has_key(key):
>> raise QAPIExprError(info,
> [...]
>
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types
2014-09-26 9:26 ` Markus Armbruster
2014-09-29 8:27 ` Markus Armbruster
@ 2014-09-29 14:35 ` Eric Blake
1 sibling, 0 replies; 62+ messages in thread
From: Eric Blake @ 2014-09-29 14:35 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 2824 bytes --]
On 09/26/2014 03:26 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Now that we know every expression is valid with regards to
>> its keys, we can add further tests that those keys refer to
>> valid types. With this patch, all references to a type (the
>> 'data': of command, type, union, and event, and the 'returns':
>> of command) must resolve to a builtin or another type declared
>> by the current qapi parse; this includes recursing into each
>> member of a data dictionary. Dealing with '**' and nested
>> sub-structs will be done in later patches.
>>
>> + for (key, value) in data.items():
>> + check_type(expr_info, "member '%s' of %s" % (key, source), value,
>> + allow_array=True,
>> + allowed_names=['built-in', 'union', 'struct', 'enum'],
>> + allow_dict=True)
>
> Hmm, allowed_names isn't about allowed names, it's about allowed
> meta-types. Rename?
Will do.
>> def check_exprs(schema):
>> for expr_elem in schema.exprs:
>> expr = expr_elem['expr']
>> info = expr_elem['info']
>>
>> - if expr.has_key('union'):
>> - check_union(expr, info)
>> - if expr.has_key('event'):
>> - check_event(expr, info)
>> if expr.has_key('enum'):
>> check_enum(expr, info)
>> + if expr.has_key('union'):
>> + check_union(expr, info)
>> + if expr.has_key('type'):
>> + check_struct(expr, info)
>> if expr.has_key('command'):
>> check_command(expr, info)
>> + if expr.has_key('event'):
>> + check_event(expr, info)
>
> Not related to this patch: this chain of ifs bothers me. The conditions
> should be exclusive, because each expression must have a well-defined
> meta-type. If I remember correctly, you actually enforce this elsewhere
> in your series. Do we want to make it obvious in the code here? elif
> instead of if, perhaps?
Yes, earlier in the series guarantees that by this point, the conditions
are now exclusive. It also bothers me that different points in the file
are iterating over the 5 key types in different order, I'll clean that
up in v5.
>> +++ b/tests/qapi-schema/data-array-unknown.err
>> @@ -0,0 +1 @@
>> +tests/qapi-schema/data-array-unknown.json:2: 'data' for command 'oops' references unknown type 'NoSuchType'
>
> The wording "references type" somehow unduly excites my "reference type"
> synapses :)
>
> Perhaps something like "Type 'NoSuchType' is unknown" would suffice.
> Entirely up to you; feel free to keep the wording as is.
I'll play with it.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass
2014-09-29 14:33 ` Eric Blake
@ 2014-09-29 16:35 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-29 16:35 UTC (permalink / raw)
To: Eric Blake; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
Eric Blake <eblake@redhat.com> writes:
> On 09/29/2014 02:38 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Now that we have a way to validate every type, we can also be
>>> stricter about enforcing that callers that want to bypass
>>> type safety in generated code. Prior to this patch, it didn't
>>> matter what value was associated with the key 'gen', but it
>>> looked odd that 'gen':'yes' could result in bypassing the
>>> generated code. These changes also enforce the changes made
>>> earlier in the series for documentation and consolidation of
>>> using '**' as the wildcard type.
>>
>> Perhaps it's worth mentioning that the schema has always used 'gen':
>> 'no'.
>>
>> That said, 'no' is silly. The natural JSON for a flag is false / true!
>> Since you're touching it anyway, consider making it an optional boolean
>> defaulting to false. Same for the other silly 'no': success-response.
>
> I'd love to, except that the .json parser does not yet allow literal
> true/false JSON keywords. Fam had a patch back in May that would fix
> that; maybe I'll fold in his patch to my v5 as a prereq patch:
>
> https://lists.gnu.org/archive/html/qemu-devel/2014-05/msg03916.html
Your choice. I certainly don't want the silly 'no' block your series.
>>> @@ -256,13 +256,13 @@ def discriminator_find_enum_define(expr):
>>> return find_enum(discriminator_type)
>>>
>>> def check_type(expr_info, source, data, allow_array = False,
>>> - allowed_names = [], allow_dict = True):
>>> + allowed_names = [], allow_dict = True, allow_star = False):
>>> global all_types
>>>
>>> if data is None:
>>> return
>>>
>>> - if data == '**':
>>> + if allow_star and data == '**':
>>> return
>
> [1]
>
>>>
>>> # Check if array type for data is okay
>>> @@ -278,6 +278,10 @@ def check_type(expr_info, source, data, allow_array = False,
>>>
>>> # Check if type name for data is okay
>>> if isinstance(data, basestring):
>>> + if data == '**':
>>> + raise QAPIExprError(expr_info,
>>> + "%s uses '**' but did not request 'gen':'no'"
>>> + % source)
>>> if not data in all_types:
>>> raise QAPIExprError(expr_info,
>>> "%s references unknown type '%s'"
>>
>> I'm blind: I can't see where this error gets suppressed when we have
>> 'gen': 'no'.
>
> 'gen':'no' triggers the caller to pass allow_star=True to check_type
> [2]; then at point [1] we short-circuit and exit if '**' was passed.
> Therefore, if we get here and still have '**', then allow_star is still
> false, which means 'gen':'no' was not passed and it is user error.
Got it, thanks!
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 17/19] qapi: Drop inline subtype in query-version
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 17/19] qapi: Drop inline subtype in query-version Eric Blake
@ 2014-09-30 17:40 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-30 17:40 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> A future patch will be using a 'name':{dictionary} entry in the
> QAPI schema to specify a default value for an optional argument;
> but existing use of inline substructs conflicts with that goal.
> This patch fixes one of only two commands relying on nested
> subtypes, by breaking the nesting into an explicit type. The
> QMP wire format is unaffected by this change.
"subtype" makes me think of subtype polymorphism. I'd stick to
"substruct", or "nested struct".
Suggest to mention nested structs are unboxed, but explicit types are
boxed.
> Prefer the safer g_new0() while making the conversion.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
> ---
> hmp.c | 2 +-
> qapi/common.json | 26 +++++++++++++++++++-------
> qmp.c | 9 +++++----
> 3 files changed, 25 insertions(+), 12 deletions(-)
>
> diff --git a/hmp.c b/hmp.c
> index 40a90da..23fc3da 100644
> --- a/hmp.c
> +++ b/hmp.c
> @@ -55,7 +55,7 @@ void hmp_info_version(Monitor *mon, const QDict *qdict)
> info = qmp_query_version(NULL);
>
> monitor_printf(mon, "%" PRId64 ".%" PRId64 ".%" PRId64 "%s\n",
> - info->qemu.major, info->qemu.minor, info->qemu.micro,
> + info->qemu->major, info->qemu->minor, info->qemu->micro,
> info->package);
>
> qapi_free_VersionInfo(info);
> diff --git a/qapi/common.json b/qapi/common.json
> index 4e9a21f..d007095 100644
> --- a/qapi/common.json
> +++ b/qapi/common.json
> @@ -29,15 +29,28 @@
> 'DeviceNotActive', 'DeviceNotFound', 'KVMMissingCap' ] }
>
> ##
> +# @VersionTriple
> +#
> +# A three-part version number.
> +#
> +# @qemu.major: The major version number.
> +#
> +# @qemu.minor: The minor version number.
> +#
> +# @qemu.micro: The micro version number.
> +#
> +# Since: 2.2
> +##
> +{ 'type': 'VersionTriple',
> + 'data': {'major': 'int', 'minor': 'int', 'micro': 'int'} }
> +
> +
> +##
> # @VersionInfo:
> #
> # A description of QEMU's version.
> #
> -# @qemu.major: The major version of QEMU
> -#
> -# @qemu.minor: The minor version of QEMU
> -#
> -# @qemu.micro: The micro version of QEMU. By current convention, a micro
> +# @qemu: The version of QEMU. By current convention, a micro
> # version of 50 signifies a development branch. A micro version
> # greater than or equal to 90 signifies a release candidate for
> # the next minor version. A micro version of less than 50
> @@ -51,8 +64,7 @@
> # Since: 0.14.0
> ##
> { 'type': 'VersionInfo',
> - 'data': {'qemu': {'major': 'int', 'minor': 'int', 'micro': 'int'},
> - 'package': 'str'} }
> + 'data': {'qemu': 'VersionTriple', 'package': 'str'} }
>
> ##
> # @query-version:
> diff --git a/qmp.c b/qmp.c
> index c6767c4..24b658a 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -45,15 +45,16 @@ NameInfo *qmp_query_name(Error **errp)
>
> VersionInfo *qmp_query_version(Error **errp)
> {
> - VersionInfo *info = g_malloc0(sizeof(*info));
> + VersionInfo *info = g_new0(VersionInfo, 1);
> const char *version = QEMU_VERSION;
> char *tmp;
>
> - info->qemu.major = strtol(version, &tmp, 10);
> + info->qemu = g_new0(VersionTriple, 1);
> + info->qemu->major = strtol(version, &tmp, 10);
> tmp++;
> - info->qemu.minor = strtol(tmp, &tmp, 10);
> + info->qemu->minor = strtol(tmp, &tmp, 10);
> tmp++;
> - info->qemu.micro = strtol(tmp, &tmp, 10);
> + info->qemu->micro = strtol(tmp, &tmp, 10);
> info->package = g_strdup(QEMU_PKGVERSION);
>
> return info;
Here you can see the boxing.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] [PATCH v4 19/19] qapi: Drop support for inline subtypes
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 19/19] qapi: Drop support for inline subtypes Eric Blake
@ 2014-09-30 17:47 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2014-09-30 17:47 UTC (permalink / raw)
To: Eric Blake; +Cc: Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> A future patch will be using a 'name':{dictionary} entry in the
> QAPI schema to specify a default value for an optional argument;
> but existing use of inline substructs conflicts with that goal.
> Now that all commands have been fixed to avoid inline substructs,
Suggest s/fixed/changed/
> nuke support for them, and turn it into a hard error. Update
> the testsuite to reflect tighter parsing rules.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Patch looks good.
^ permalink raw reply [flat|nested] 62+ messages in thread
* [Qemu-devel] RFC: 'alternate' qapi top-level expression [was: [PATCH v4 12/19] qapi: Add some type check tests]
2014-09-25 16:19 ` Markus Armbruster
@ 2015-03-23 15:33 ` Eric Blake
2015-03-23 19:28 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Eric Blake @ 2015-03-23 15:33 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Luiz Capitulino, Fam Zheng, qemu-devel, wenchaoqemu
[-- Attachment #1: Type: text/plain, Size: 3415 bytes --]
[revisiting this series, finally!]
On 09/25/2014 10:19 AM, Markus Armbruster wrote:
>> Return of an anon union isn't used yet, but _might_ make sense (as the
>> only feasible way of changing existing commands that return an array or
>> primitive extensible to instead return a dict) -
>
> Good point.
>
>> except that back-compat
>> demands that we can't return a dict in place of a primitive unless the
>> arguments of the command are also enhanced (that is, older callers are
>> not expecting a dict, so we can't return a dict unless the caller
>> witnesses they are new enough by explicitly asking for a dict return).
>
> I think we can keep things simple for now and reject anonymous unions.
> We can always relax the check when we run into a use.
In trying to code up what it would take to easily reject anonymous
unions from a return type, I'm realizing that it would be smarter to
quit mixing anonymous unions in with other unions.
Refresher course: on the wire, both a regular union:
QAPI:
{ 'type': 'Type1', 'data': { 'value': 'int' } }
{ 'union': 'Foo', 'data': { 'a': 'Type1', 'b': 'Type2' } }
{ 'command': 'bar', 'data': 'Foo' }
Wire:
{ "execute": "bar", "arguments": { "type": "a",
"data": { "value": 1 } } }
and a flat union:
QAPI:
{ 'type': 'Type1', 'data': { 'value': 'int' } }
{ 'enum': 'Enum', 'data': [ 'a', 'b' ] }
{ 'type': 'Base', { 'switch': 'Enum' } }
{ 'union': 'Foo', 'base': 'Base', 'discriminator': 'switch',
'data': { 'a': 'Type1', 'b': 'Type2' } }
{ 'command': 'bar', 'data': 'Foo' }
Wire:
{ "execute": "bar", "arguments": { "switch": "a",
"value": 1 } }
happen to guarantee a top-level dictionary (plain unions always have a
two-element dictionary, flat unions are required to have a base type
which is itself a dictionary). But an anonymous union is explicitly
allowing a multi-type variable, where the determination of which branch
of the union is made by the type of the variable. Furthermore, we do
not allow two branches to have the same type, so at least one branch
must be a non-dictionary; but as _all_ QMP commands currently take a
dictionary for the "arguments" key, we do not want to allow:
QAPI:
{ 'type': 'Type1', 'data': { 'value': 'int' } }
{ 'union': 'Foo', 'discriminator': {},
'data': { 'a': 'Type1', 'b': 'int' } }
{ 'command': 'bar', 'data': 'Foo' }
Wire:
{ "execute": "bar", "arguments": 1 }
Tracking all three qapi expressions as union types is making the
generator code a bit verbose, especially since the code generation for
all three is so distinct.
Proposal: I am proposing that we convert to an alternate syntax for what
we now term as anonymous unions. It will not have any impact to the
wire representation of QMP, only to the qapi code generators. The
proposal is simple: instead of using "'union':'Name',
'discriminator':{}", we instead use "'alternate': 'Foo'" when declaring
a type as an anonymous union (which, for obvious reasons, I would then
update the documentation to call an "alternate" instead of an "anonymous
union").
I'll go ahead and propose the patches (I've already done the bulk of the
conversion work, to prove that not many files were affected).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [Qemu-devel] RFC: 'alternate' qapi top-level expression [was: [PATCH v4 12/19] qapi: Add some type check tests]
2015-03-23 15:33 ` [Qemu-devel] RFC: 'alternate' qapi top-level expression [was: [PATCH v4 12/19] qapi: Add some type check tests] Eric Blake
@ 2015-03-23 19:28 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2015-03-23 19:28 UTC (permalink / raw)
To: Eric Blake
Cc: Kevin Wolf, Fam Zheng, qemu-devel, wenchaoqemu, Luiz Capitulino
Cc'ing Kevin as fair punishment for is previous work on QAPI code
generation in general, and union types in particular.
Eric Blake <eblake@redhat.com> writes:
> [revisiting this series, finally!]
>
> On 09/25/2014 10:19 AM, Markus Armbruster wrote:
>
>>> Return of an anon union isn't used yet, but _might_ make sense (as the
>>> only feasible way of changing existing commands that return an array or
>>> primitive extensible to instead return a dict) -
>>
>> Good point.
>>
>>> except that back-compat
>>> demands that we can't return a dict in place of a primitive unless the
>>> arguments of the command are also enhanced (that is, older callers are
>>> not expecting a dict, so we can't return a dict unless the caller
>>> witnesses they are new enough by explicitly asking for a dict return).
>>
>> I think we can keep things simple for now and reject anonymous unions.
>> We can always relax the check when we run into a use.
>
> In trying to code up what it would take to easily reject anonymous
> unions from a return type, I'm realizing that it would be smarter to
> quit mixing anonymous unions in with other unions.
>
> Refresher course: on the wire, both a regular union:
>
> QAPI:
> { 'type': 'Type1', 'data': { 'value': 'int' } }
> { 'union': 'Foo', 'data': { 'a': 'Type1', 'b': 'Type2' } }
> { 'command': 'bar', 'data': 'Foo' }
> Wire:
> { "execute": "bar", "arguments": { "type": "a",
> "data": { "value": 1 } } }
>
> and a flat union:
>
> QAPI:
> { 'type': 'Type1', 'data': { 'value': 'int' } }
> { 'enum': 'Enum', 'data': [ 'a', 'b' ] }
> { 'type': 'Base', { 'switch': 'Enum' } }
> { 'union': 'Foo', 'base': 'Base', 'discriminator': 'switch',
> 'data': { 'a': 'Type1', 'b': 'Type2' } }
> { 'command': 'bar', 'data': 'Foo' }
> Wire:
> { "execute": "bar", "arguments": { "switch": "a",
> "value": 1 } }
>
> happen to guarantee a top-level dictionary (plain unions always have a
> two-element dictionary, flat unions are required to have a base type
> which is itself a dictionary).
Yes.
> But an anonymous union is explicitly
> allowing a multi-type variable, where the determination of which branch
> of the union is made by the type of the variable. Furthermore, we do
> not allow two branches to have the same type,
Even more severe: the JSON types have to be different! Thus, at most
one can be a complex or union type, and at most one can be a string or
enum type.
> so at least one branch
> must be a non-dictionary;
Correct.
> but as _all_ QMP commands currently take a
> dictionary for the "arguments" key, we do not want to allow:
>
> QAPI:
> { 'type': 'Type1', 'data': { 'value': 'int' } }
> { 'union': 'Foo', 'discriminator': {},
> 'data': { 'a': 'Type1', 'b': 'int' } }
> { 'command': 'bar', 'data': 'Foo' }
> Wire:
> { "execute": "bar", "arguments": 1 }
Yes, let's stay out of the generalization tar pits and stick to named
parameters, i.e. JSON object arguments.
> Tracking all three qapi expressions as union types is making the
> generator code a bit verbose, especially since the code generation for
> all three is so distinct.
>
>
> Proposal: I am proposing that we convert to an alternate syntax for what
> we now term as anonymous unions. It will not have any impact to the
> wire representation of QMP, only to the qapi code generators. The
> proposal is simple: instead of using "'union':'Name',
> 'discriminator':{}", we instead use "'alternate': 'Foo'" when declaring
> a type as an anonymous union (which, for obvious reasons, I would then
> update the documentation to call an "alternate" instead of an "anonymous
> union").
This separates tagged and untagged unions more clearly: reserve 'union'
for the two kinds of tagged unions, switch the untagged union to
'alternate'.
I don't mind. Kevin, any objections?
> I'll go ahead and propose the patches (I've already done the bulk of the
> conversion work, to prove that not many files were affected).
I'll gladly review.
^ permalink raw reply [flat|nested] 62+ messages in thread
end of thread, other threads:[~2015-03-23 19:29 UTC | newest]
Thread overview: 62+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-19 22:24 [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 01/19] qapi: Consistent whitespace in tests/Makefile Eric Blake
2014-09-22 12:40 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 02/19] qapi: Ignore files created during make check Eric Blake
2014-09-23 8:07 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 03/19] qapi: Update docs given recent event, spacing fixes Eric Blake
2014-09-22 12:40 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 04/19] qapi: Document type-safety considerations Eric Blake
2014-09-22 12:37 ` Markus Armbruster
2014-09-22 16:53 ` Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 05/19] qapi: Add some enum tests Eric Blake
2014-09-22 12:43 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 06/19] qapi: Better error messages for bad enums Eric Blake
2014-09-23 14:23 ` Markus Armbruster
2014-09-23 15:59 ` Eric Blake
2014-09-24 7:46 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 07/19] qapi: Add some expr tests Eric Blake
2014-09-23 14:26 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 08/19] qapi: Better error messages for bad expressions Eric Blake
2014-09-23 14:56 ` Markus Armbruster
2014-09-23 16:11 ` Eric Blake
2014-09-24 7:34 ` Markus Armbruster
2014-09-24 9:25 ` Kevin Wolf
2014-09-24 11:14 ` Markus Armbruster
2014-09-26 9:15 ` Markus Armbruster
2014-09-26 9:25 ` Kevin Wolf
2014-09-26 11:40 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 09/19] qapi: Add tests of redefined expressions Eric Blake
2014-09-24 11:24 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 10/19] qapi: Better error messages for duplicated expressions Eric Blake
2014-09-24 11:58 ` Markus Armbruster
2014-09-24 14:10 ` Eric Blake
2014-09-24 15:29 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 11/19] qapi: Add tests of type bypass Eric Blake
2014-09-24 16:10 ` Markus Armbruster
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
2014-09-25 7:34 ` Markus Armbruster
2014-09-25 8:06 ` Markus Armbruster
2014-09-25 14:00 ` Eric Blake
2014-09-25 16:19 ` Markus Armbruster
2015-03-23 15:33 ` [Qemu-devel] RFC: 'alternate' qapi top-level expression [was: [PATCH v4 12/19] qapi: Add some type check tests] Eric Blake
2015-03-23 19:28 ` Markus Armbruster
2014-09-25 13:54 ` [Qemu-devel] [PATCH v4 12/19] qapi: Add some type check tests Eric Blake
2014-09-25 16:12 ` Markus Armbruster
2014-09-25 16:32 ` Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 13/19] qapi: More rigourous checking of types Eric Blake
2014-09-26 9:26 ` Markus Armbruster
2014-09-29 8:27 ` Markus Armbruster
2014-09-29 14:26 ` Eric Blake
2014-09-29 14:35 ` Eric Blake
2014-09-19 22:24 ` [Qemu-devel] [PATCH v4 14/19] qapi: More rigorous checking for type safety bypass Eric Blake
2014-09-29 8:38 ` Markus Armbruster
2014-09-29 14:33 ` Eric Blake
2014-09-29 16:35 ` Markus Armbruster
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 15/19] qapi: Merge UserDefTwo and UserDefNested in tests Eric Blake
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 16/19] qapi: Drop tests for inline subtypes Eric Blake
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 17/19] qapi: Drop inline subtype in query-version Eric Blake
2014-09-30 17:40 ` Markus Armbruster
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 18/19] qapi: Drop inline subtype in query-pci Eric Blake
2014-09-19 22:25 ` [Qemu-devel] [PATCH v4 19/19] qapi: Drop support for inline subtypes Eric Blake
2014-09-30 17:47 ` Markus Armbruster
2014-09-26 15:42 ` [Qemu-devel] [PATCH v4 00/19] drop qapi nested structs Eric Blake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.