* [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
@ 2015-11-06 6:35 Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 01/30] qapi: Use generated TestStruct machinery in tests Eric Blake
` (30 more replies)
0 siblings, 31 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru
No pending prerequisites; based on qemu.git master
Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv9c
and will soon be part of my branch with the rest of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi
v10 notes:
Split several patches, redo the middle patches from Markus to be
back in the order they were first posted, some fallout change to
my patches due to the nicer pattern of minimizing conditionals
inside .check(), by instead calling .check_clash() as needed.
Change data->err magic in tests to instead use a new helper
error_free_or_abort(). Add a patch that would prevent qapi
case-insensitive clashes.
I am redoing my subset boundaries slightly: patches 23-27 of
v9 (updating the alternate layout) will be delayed to subset D,
and 2 other patches previously posted in subset D are now here
(turning qapi clash checking into actual error messages), so
the subject line of this cover letter is slightly different.
Hopefully, we are converging on something that will be ready
for a pull request, especially for the earlier patches of this
subset.
Backport diff in relation to v9:
001/30:[----] [--] 'qapi: Use generated TestStruct machinery in tests'
002/30:[----] [--] 'qapi: Strengthen test of TestStructList'
003/30:[down] 'qobject: Protect against use-after-free in qobject_decref()'
004/30:[down] 'qapi: Share test_init code in test-qmp-input*'
005/30:[0001] [FC] 'qapi: Plug leaks in test-qmp-*'
006/30:[down] 'qapi: Simplify non-error testing in test-qmp-*'
007/30:[down] 'qapi: Simplify error cleanup in test-qmp-*'
008/30:[----] [--] 'qapi: More tests of alternate output'
009/30:[0010] [FC] 'qapi: Test failure in middle of array parse'
010/30:[0025] [FC] 'qapi: More tests of input arrays'
011/30:[----] [--] 'qapi: Provide nicer array names in introspection'
012/30:[0014] [FC] 'qapi-introspect: Document lack of sorting'
013/30:[----] [--] 'qapi: Track simple union tag in object.local_members'
014/30:[----] [--] 'qapi-types: Consolidate gen_struct() and gen_union()'
015/30:[----] [--] 'qapi-types: Simplify gen_struct_field[s]'
016/30:[0005] [FC] 'qapi: Drop obsolete tag value collision assertions'
017/30:[0002] [FC] 'qapi: Simplify QAPISchemaObjectTypeMember.check()'
018/30:[down] 'qapi: Clean up after previous commit'
019/30:[----] [-C] 'qapi: Fix up commit 7618b91's clash sanity checking change'
020/30:[0002] [FC] 'qapi: Eliminate QAPISchemaObjectType.check() variable members'
021/30:[----] [--] 'qapi: Factor out QAPISchemaObjectTypeMember.check_clash()'
022/30:[----] [-C] 'qapi: Simplify QAPISchemaObjectTypeVariants.check()'
023/30:[0023] [FC] 'qapi: Check for qapi collisions of flat union branches'
024/30:[0013] [FC] 'qapi: Factor out QAPISchemaObjectType.check_clash()'
025/30:[down] 'qapi: Hoist tag collision check to Variants.check()'
026/30:[----] [--] 'qapi: Remove outdated tests related to QMP/branch collisions'
027/30:[down] 'qapi: Track owner of each object member'
028/30:[down] 'qapi: Detect collisions in C member names'
029/30:[down] 'cpu: Convert CpuInfo into flat union'
030/30:[down] 'qapi: Forbid case-insensitive clashes'
v9 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg00652.html
More patches added, and several reorganized. Lots of new patches
from Markus, although not in the order originally proposed.
The first 8 patches are fairly straightforward, and could probably
be taken as-is. Patch 9 is a rewrite of v8 4/17, but in the opposite
direction (document that no sorting is done, rather than attempting
to sort), so it may need further fine-tuning. Patches 12-21
represents a fusion of Markus' and my attempts to rewrite v5 7/17
into a more-reviewable set of patches, and caused further churn
later in the series.
Patch 23 still uses tag_member.type == None; I ran out of time to
work on Markus' idea of providing an instance of QAPISchemaBuiltinType
to fill the role for 'qtype_code' without being exposed through .json
files and without breaking the invariant of a valid member.type after
check(), and wanted to get the rest of the series started under revew.
So I may need a followup patch or even a v10 of the later half of
this series after exploring that idea more.
v8 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg06674.html
Minor changes when rebasing to latest, improved commit messages in
a few places, plus the addition of a few new patches. Markus started
reviewing v7, but hadn't gotten very far.
v7 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04112.html
Patches 1-3 of the previous round have moved to subset B; patch 7
is gone, and a couple of new patches are present in this round. The
latest version of subset B made it much easier to reason about
collision detection (namely, tag values can't collide with QMP values
thanks to a named rather than anonymous union in the C type), and I
like how things turned out. I suspect the hardest part of the review
will be patches 5/14 and 7/14, although none of this has really
had much review in any earlier versions.
v6 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg01980.html
Add some patches and rebase onto work on subset B. Rearrange some
patches from v5 (this set includes 17-20, 23, 25-27). Backport diff
gets a bit confused by one patch title changing.
Not much direct review comments, although some of the changes here
are updated based on comments made on other patches in the v5 series.
Subset D (and more?) will come later.
In v5:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg05410.html
I _did_ rearrange patches to try and group related features:
1-2: Groundwork cleanups
3-5: Add more test cases
6-16: Front-end cleanups
17-18: Introspection output cleanups
19-20: 'alternate' type cleanups
21-29: qapi visitor cleanups
30-45: qapi-ify netdev_add
46: add qapi shorthand for flat unions
Lots of fixes based on additional testing, and rebased to
track other changes that happened in the meantime. The series
is huge; I can split off smaller portions as requested.
In v4:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02580.html
add some more clean up patches
rebase to Markus' recent work
pull in part of Zoltán's work to make netdev_add a flat union,
further enhancing it to be introspectible
I might be able to rearrange some of these patches, or separate
it into smaller independent series, if requested; but I'm
posting now to get review started.
In v3:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02059.html
redo cleanup of dealloc of partial struct
add patches to make all visit_type_*() avoid leaks on failure
add patches to allow boxed command arguments and events
In v2:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00900.html
rebase to Markus' v3 series
rework how comments are emitted for fields inherited from base
additional patches added for deleting colliding 'void *data'
documentation updates to match code changes
v1 was here:
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05266.html
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05325.html
Eric Blake (23):
qapi: Use generated TestStruct machinery in tests
qapi: Strengthen test of TestStructList
qobject: Protect against use-after-free in qobject_decref()
qapi: Share test_init code in test-qmp-input*
qapi: Plug leaks in test-qmp-*
qapi: Simplify non-error testing in test-qmp-*
qapi: Simplify error cleanup in test-qmp-*
qapi: More tests of alternate output
qapi: Test failure in middle of array parse
qapi: More tests of input arrays
qapi: Provide nicer array names in introspection
qapi-introspect: Document lack of sorting
qapi: Track simple union tag in object.local_members
qapi-types: Consolidate gen_struct() and gen_union()
qapi-types: Simplify gen_struct_field[s]
qapi: Check for qapi collisions of flat union branches
qapi: Factor out QAPISchemaObjectType.check_clash()
qapi: Hoist tag collision check to Variants.check()
qapi: Remove outdated tests related to QMP/branch collisions
qapi: Track owner of each object member
qapi: Detect collisions in C member names
cpu: Convert CpuInfo into flat union
qapi: Forbid case-insensitive clashes
Markus Armbruster (7):
qapi: Drop obsolete tag value collision assertions
qapi: Simplify QAPISchemaObjectTypeMember.check()
qapi: Clean up after previous commit
qapi: Fix up commit 7618b91's clash sanity checking change
qapi: Eliminate QAPISchemaObjectType.check() variable members
qapi: Factor out QAPISchemaObjectTypeMember.check_clash()
qapi: Simplify QAPISchemaObjectTypeVariants.check()
cpus.c | 31 ++-
docs/qapi-code-gen.txt | 31 ++-
hmp.c | 30 ++-
include/qapi/qmp/qobject.h | 1 +
qapi-schema.json | 120 +++++++--
qapi/introspect.json | 17 +-
scripts/qapi-introspect.py | 8 +-
scripts/qapi-types.py | 68 ++----
scripts/qapi-visit.py | 9 +-
scripts/qapi.py | 135 ++++++----
tests/Makefile | 4 +-
tests/qapi-schema/args-case-clash.err | 1 +
...{union-clash-type.exit => args-case-clash.exit} | 0
tests/qapi-schema/args-case-clash.json | 5 +
.../{union-clash-type.out => args-case-clash.out} | 0
tests/qapi-schema/args-name-clash.err | 1 +
tests/qapi-schema/args-name-clash.exit | 2 +-
tests/qapi-schema/args-name-clash.json | 6 +-
tests/qapi-schema/args-name-clash.out | 6 -
tests/qapi-schema/flat-union-clash-branch.err | 0
tests/qapi-schema/flat-union-clash-branch.exit | 1 -
tests/qapi-schema/flat-union-clash-branch.json | 18 --
tests/qapi-schema/flat-union-clash-branch.out | 14 --
tests/qapi-schema/flat-union-clash-type.err | 1 -
tests/qapi-schema/flat-union-clash-type.exit | 1 -
tests/qapi-schema/flat-union-clash-type.json | 14 --
tests/qapi-schema/flat-union-clash-type.out | 0
tests/qapi-schema/qapi-schema-test.json | 6 +-
tests/qapi-schema/qapi-schema-test.out | 15 +-
tests/qapi-schema/union-clash-data.out | 1 +
tests/qapi-schema/union-clash-type.err | 1 -
tests/qapi-schema/union-clash-type.json | 9 -
tests/qapi-schema/union-empty.out | 1 +
tests/test-qmp-commands.c | 8 +-
tests/test-qmp-common.h | 22 ++
tests/test-qmp-event.c | 1 +
tests/test-qmp-input-strict.c | 131 +++-------
tests/test-qmp-input-visitor.c | 272 +++++++++------------
tests/test-qmp-output-visitor.c | 151 +++---------
tests/test-visitor-serialization.c | 76 ++----
40 files changed, 570 insertions(+), 648 deletions(-)
create mode 100644 tests/qapi-schema/args-case-clash.err
rename tests/qapi-schema/{union-clash-type.exit => args-case-clash.exit} (100%)
create mode 100644 tests/qapi-schema/args-case-clash.json
rename tests/qapi-schema/{union-clash-type.out => args-case-clash.out} (100%)
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.err
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.json
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.out
delete mode 100644 tests/qapi-schema/flat-union-clash-type.err
delete mode 100644 tests/qapi-schema/flat-union-clash-type.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-type.json
delete mode 100644 tests/qapi-schema/flat-union-clash-type.out
delete mode 100644 tests/qapi-schema/union-clash-type.err
delete mode 100644 tests/qapi-schema/union-clash-type.json
create mode 100644 tests/test-qmp-common.h
--
2.4.3
^ permalink raw reply [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 01/30] qapi: Use generated TestStruct machinery in tests
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 02/30] qapi: Strengthen test of TestStructList Eric Blake
` (29 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Commit d88f5fd and friends first introduced the various test-qmp-*
tests in 2011, with duplicated hand-rolled TestStruct machinery,
to make sure the qapi visitor interface was tested. Later, commit
4f193e3 in 2013 added a .json file for further testing use by the
files, but without consolidating any of the existing hand-rolled
visitors. And with four copies, subtle differences have crept in,
between the tests themselves (mainly whitespace differences, but
also a question of whether to use NULL or "TestStruct" when
calling visit_start_struct()) and from what the generator produces
(the hand-rolled versions did not cater to partially-allocated
objects, because they did not have a deallocation usage).
Of course, just because the visitor interface is tested does not
mean it is a sane interface; and future patches will be changing
some of the visitor contracts. Rather than having to duplicate
the cleanup work in each copy of the TestStruct visitor, and keep
each hand-rolled copy in sync with what the generator supplies, we
might as well just test what the generator should give us in the
first place.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: no change
v8: improve commit message
v7: rebase on top of subset B v9; defer unrelated trailing whitespace
cleanups to later in series
v6: new patch
---
tests/qapi-schema/qapi-schema-test.json | 6 +++-
tests/qapi-schema/qapi-schema-test.out | 5 +++
tests/test-qmp-input-strict.c | 35 --------------------
tests/test-qmp-input-visitor.c | 34 -------------------
tests/test-qmp-output-visitor.c | 58 ---------------------------------
tests/test-visitor-serialization.c | 34 -------------------
6 files changed, 10 insertions(+), 162 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 48e104b..44638da 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -3,6 +3,9 @@
# This file is a stress test of supported qapi constructs that must
# parse and compile correctly.
+{ 'struct': 'TestStruct',
+ 'data': { 'integer': 'int', 'boolean': 'bool', 'string': 'str' } }
+
# for testing enums
{ 'struct': 'NestedEnumsOne',
'data': { 'enum1': 'EnumOne', # Intentional forward reference
@@ -46,7 +49,8 @@
# dummy struct to force generation of array types not otherwise mentioned
{ 'struct': 'ForceArrays',
- 'data': { 'unused1':['UserDefOne'], 'unused2':['UserDefTwo'] } }
+ 'data': { 'unused1':['UserDefOne'], 'unused2':['UserDefTwo'],
+ 'unused3':['TestStruct'] } }
# for testing unions
# Among other things, test that a name collision between branches does
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index a7e9aab..e20a823 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -92,6 +92,7 @@ object EventStructOne
object ForceArrays
member unused1: UserDefOneList optional=False
member unused2: UserDefTwoList optional=False
+ member unused3: TestStructList optional=False
enum MyEnum []
object NestedEnumsOne
member enum1: EnumOne optional=False
@@ -100,6 +101,10 @@ object NestedEnumsOne
member enum4: EnumOne optional=True
enum QEnumTwo ['value1', 'value2']
prefix QENUM_TWO
+object TestStruct
+ member integer: int optional=False
+ member boolean: bool optional=False
+ member string: str optional=False
object UserDefA
member boolean: bool optional=False
member a_b: int optional=True
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 53a7693..b44184f 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -89,41 +89,6 @@ static Visitor *validate_test_init_raw(TestInputVisitorData *data,
return v;
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, "TestStruct", name, sizeof(TestStruct),
- &err);
- if (err) {
- goto out;
- }
-
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static void test_validate_struct(TestInputVisitorData *data,
const void *unused)
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index de65982..3f6bc4d 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -185,40 +185,6 @@ static void test_visitor_in_enum(TestInputVisitorData *data,
data->qiv = NULL;
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, "TestStruct", name, sizeof(TestStruct),
- &err);
- if (err) {
- goto out;
- }
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static void test_visitor_in_struct(TestInputVisitorData *data,
const void *unused)
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 09d0dd8..baf58dc 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -166,41 +166,6 @@ static void test_visitor_out_enum_errors(TestOutputVisitorData *data,
}
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, "TestStruct", name, sizeof(TestStruct),
- &err);
- if (err) {
- goto out;
- }
-
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static void test_visitor_out_struct(TestOutputVisitorData *data,
const void *unused)
@@ -314,29 +279,6 @@ static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
}
}
-typedef struct TestStructList
-{
- union {
- TestStruct *value;
- uint64_t padding;
- };
- struct TestStructList *next;
-} TestStructList;
-
-static void visit_type_TestStructList(Visitor *v, TestStructList **obj,
- const char *name, Error **errp)
-{
- GenericList *i, **head = (GenericList **)obj;
-
- visit_start_list(v, name, errp);
-
- for (*head = i = visit_next_list(v, head, errp); i; i = visit_next_list(v, &i, errp)) {
- TestStructList *native_i = (TestStructList *)i;
- visit_type_TestStruct(v, &native_i->value, NULL, errp);
- }
-
- visit_end_list(v, errp);
-}
static void test_visitor_out_list(TestOutputVisitorData *data,
const void *unused)
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 634563b..c024e5e 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -186,40 +186,6 @@ static void visit_primitive_list(Visitor *v, void **native, Error **errp)
}
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, NULL, name, sizeof(TestStruct), &err);
- if (err) {
- goto out;
- }
-
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static TestStruct *struct_create(void)
{
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 02/30] qapi: Strengthen test of TestStructList
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 01/30] qapi: Use generated TestStruct machinery in tests Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 03/30] qobject: Protect against use-after-free in qobject_decref() Eric Blake
` (28 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Make each list element different, to ensure that order is
preserved, and use the generated free function instead of
hand-rolling our own to ensure (under valgrind) that the
list is properly cleaned.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: no change
v8: no change
v7: new patch
---
tests/test-qmp-output-visitor.c | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index baf58dc..9364843 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -283,7 +283,7 @@ static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
static void test_visitor_out_list(TestOutputVisitorData *data,
const void *unused)
{
- char *value_str = (char *) "list value";
+ const char *value_str = "list value";
TestStructList *p, *head = NULL;
const int max_items = 10;
bool value_bool = true;
@@ -294,12 +294,13 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
QList *qlist;
int i;
+ /* Build the list in reverse order... */
for (i = 0; i < max_items; i++) {
p = g_malloc0(sizeof(*p));
p->value = g_malloc0(sizeof(*p->value));
- p->value->integer = value_int;
+ p->value->integer = value_int + (max_items - i - 1);
p->value->boolean = value_bool;
- p->value->string = value_str;
+ p->value->string = g_strdup(value_str);
p->next = head;
head = p;
@@ -315,6 +316,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
qlist = qobject_to_qlist(obj);
g_assert(!qlist_empty(qlist));
+ /* ...and ensure that the visitor sees it in order */
i = 0;
QLIST_FOREACH_ENTRY(qlist, entry) {
QDict *qdict;
@@ -322,7 +324,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
g_assert(qobject_type(entry->value) == QTYPE_QDICT);
qdict = qobject_to_qdict(entry->value);
g_assert_cmpint(qdict_size(qdict), ==, 3);
- g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, value_int);
+ g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, value_int + i);
g_assert_cmpint(qdict_get_bool(qdict, "boolean"), ==, value_bool);
g_assert_cmpstr(qdict_get_str(qdict, "string"), ==, value_str);
i++;
@@ -330,13 +332,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
g_assert_cmpint(i, ==, max_items);
QDECREF(qlist);
-
- for (p = head; p;) {
- TestStructList *tmp = p->next;
- g_free(p->value);
- g_free(p);
- p = tmp;
- }
+ qapi_free_TestStructList(head);
}
static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 03/30] qobject: Protect against use-after-free in qobject_decref()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 01/30] qapi: Use generated TestStruct machinery in tests Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 02/30] qapi: Strengthen test of TestStructList Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 04/30] qapi: Share test_init code in test-qmp-input* Eric Blake
` (27 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Luiz Capitulino
Adding an assertion to qobject_decref() will ensure that a
programming error causing use-after-free will result in
immediate failure (provided no other thread has started
using the memory) instead of silently attempting to wrap
refcnt around and leaving the problem to potentially bite
later at a harder point to diagnose.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: new patch
---
include/qapi/qmp/qobject.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h
index c856f55..4b96ed5 100644
--- a/include/qapi/qmp/qobject.h
+++ b/include/qapi/qmp/qobject.h
@@ -90,6 +90,7 @@ static inline void qobject_incref(QObject *obj)
*/
static inline void qobject_decref(QObject *obj)
{
+ assert(!obj || obj->refcnt);
if (obj && --obj->refcnt == 0) {
assert(obj->type != NULL);
assert(obj->type->destroy != NULL);
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 04/30] qapi: Share test_init code in test-qmp-input*
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (2 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 03/30] qobject: Protect against use-after-free in qobject_decref() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 15:17 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-* Eric Blake
` (26 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Rather than duplicate the body of two functions just to
decide between qobject_from_jsonv() and qobject_from_json(),
exploit the fact that qobject_from_jsonv() intentionally
takes 'va_list *' instead of the more common 'va_list', and
that qobject_from_json() just calls qobject_from_jsonv(,NULL).
For each file, our two existing init functions then become
thin wrappers around a new internal function, and future
updates to initialization don't have to be duplicated.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: new patch
---
tests/test-qmp-input-strict.c | 48 ++++++++++++++++++++---------------------
tests/test-qmp-input-visitor.c | 49 ++++++++++++++++++++----------------------
2 files changed, 46 insertions(+), 51 deletions(-)
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index b44184f..77151de 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -40,9 +40,27 @@ static void validate_teardown(TestInputVisitorData *data,
}
}
-/* This is provided instead of a test setup function so that the JSON
- string used by the tests are kept in the test functions (and not
- int main()) */
+/* The various test_init functions are provided instead of a test setup
+ function so that the JSON string used by the tests are kept in the test
+ functions (and not int main()). */
+static Visitor *validate_test_init_internal(TestInputVisitorData *data,
+ const char *json_string,
+ va_list *ap)
+{
+ Visitor *v;
+
+ data->obj = qobject_from_jsonv(json_string, ap);
+ g_assert(data->obj);
+
+ data->qiv = qmp_input_visitor_new_strict(data->obj);
+ g_assert(data->qiv);
+
+ v = qmp_input_get_visitor(data->qiv);
+ g_assert(v);
+
+ return v;
+}
+
static GCC_FMT_ATTR(2, 3)
Visitor *validate_test_init(TestInputVisitorData *data,
const char *json_string, ...)
@@ -51,17 +69,8 @@ Visitor *validate_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- data->obj = qobject_from_jsonv(json_string, &ap);
+ v = validate_test_init_internal(data, json_string, &ap);
va_end(ap);
-
- g_assert(data->obj != NULL);
-
- data->qiv = qmp_input_visitor_new_strict(data->obj);
- g_assert(data->qiv != NULL);
-
- v = qmp_input_get_visitor(data->qiv);
- g_assert(v != NULL);
-
return v;
}
@@ -75,18 +84,7 @@ Visitor *validate_test_init(TestInputVisitorData *data,
static Visitor *validate_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- Visitor *v;
-
- data->obj = qobject_from_json(json_string);
- g_assert(data->obj != NULL);
-
- data->qiv = qmp_input_visitor_new_strict(data->obj);
- g_assert(data->qiv != NULL);
-
- v = qmp_input_get_visitor(data->qiv);
- g_assert(v != NULL);
-
- return v;
+ return validate_test_init_internal(data, json_string, NULL);
}
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 3f6bc4d..933243d 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -36,9 +36,27 @@ static void visitor_input_teardown(TestInputVisitorData *data,
}
}
-/* This is provided instead of a test setup function so that the JSON
- string used by the tests are kept in the test functions (and not
- int main()) */
+/* The various test_init functions are provided instead of a test setup
+ function so that the JSON string used by the tests are kept in the test
+ functions (and not int main()). */
+static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
+ const char *json_string,
+ va_list *ap)
+{
+ Visitor *v;
+
+ data->obj = qobject_from_jsonv(json_string, ap);
+ g_assert(data->obj);
+
+ data->qiv = qmp_input_visitor_new(data->obj);
+ g_assert(data->qiv);
+
+ v = qmp_input_get_visitor(data->qiv);
+ g_assert(v);
+
+ return v;
+}
+
static GCC_FMT_ATTR(2, 3)
Visitor *visitor_input_test_init(TestInputVisitorData *data,
const char *json_string, ...)
@@ -47,17 +65,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
va_list ap;
va_start(ap, json_string);
- data->obj = qobject_from_jsonv(json_string, &ap);
+ v = visitor_input_test_init_internal(data, json_string, &ap);
va_end(ap);
-
- g_assert(data->obj != NULL);
-
- data->qiv = qmp_input_visitor_new(data->obj);
- g_assert(data->qiv != NULL);
-
- v = qmp_input_get_visitor(data->qiv);
- g_assert(v != NULL);
-
return v;
}
@@ -71,19 +80,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
const char *json_string)
{
- Visitor *v;
-
- data->obj = qobject_from_json(json_string);
-
- g_assert(data->obj != NULL);
-
- data->qiv = qmp_input_visitor_new(data->obj);
- g_assert(data->qiv != NULL);
-
- v = qmp_input_get_visitor(data->qiv);
- g_assert(v != NULL);
-
- return v;
+ return visitor_input_test_init_internal(data, json_string, NULL);
}
static void test_visitor_in_int(TestInputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-*
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (3 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 04/30] qapi: Share test_init code in test-qmp-input* Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 15:21 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing " Eric Blake
` (25 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Make valgrind happy with the current state of the tests, so that
it is easier to see if future patches introduce new memory problems
without being drowned in noise. Many of the leaks were due to
calling a second init without tearing down the data from an earlier
visit. But since teardown is already idempotent, and we already
register teardown as part of input_visitor_test_add(), it is nicer
to just make init() safe to call multiple times than it is to have
to make all tests call teardown.
Another common leak was forgetting to clean up an error object,
after testing that an error was raised.
Another leak was in test_visitor_in_struct_nested(), failing to
clean the base member of UserDefTwo. Cleaning that up left
check_and_free_str() as dead code (since using the qapi_free_*
takes care of recursion, and we don't want double frees).
A final leak was in test_visitor_out_any(), which was reassigning
the qobj local variable to a subset of the overall structure
needing freeing; it did not result in a use-after-free, but
was not cleaning up all the qdict.
test-qmp-event and test-qmp-commands were already clean.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: improve commit message, split out refactor of test init so
that both init and init_raw benefit from change
v9: move earlier in series (was 13/17)
v8: no change
v7: no change
v6: make init repeatable rather than adding teardown everywhere,
fix additional leak with UserDefTwo base, plug additional files
---
tests/test-qmp-input-strict.c | 9 +++++++++
tests/test-qmp-input-visitor.c | 41 +++++++----------------------------------
tests/test-qmp-output-visitor.c | 3 ++-
3 files changed, 18 insertions(+), 35 deletions(-)
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 77151de..95cd8e1 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -49,6 +49,8 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
{
Visitor *v;
+ validate_teardown(data, NULL);
+
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
@@ -191,6 +193,7 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
visit_type_TestStruct(v, &p, NULL, &err);
g_assert(err);
+ error_free(err);
if (p) {
g_free(p->string);
}
@@ -208,6 +211,7 @@ static void test_validate_fail_struct_nested(TestInputVisitorData *data,
visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefTwo(udp);
}
@@ -222,6 +226,7 @@ static void test_validate_fail_list(TestInputVisitorData *data,
visit_type_UserDefOneList(v, &head, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefOneList(head);
}
@@ -237,6 +242,7 @@ static void test_validate_fail_union_native_list(TestInputVisitorData *data,
visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefNativeListUnion(tmp);
}
@@ -251,6 +257,7 @@ static void test_validate_fail_union_flat(TestInputVisitorData *data,
visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefFlatUnion(tmp);
}
@@ -266,6 +273,7 @@ static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
visit_type_UserDefFlatUnion2(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefFlatUnion2(tmp);
}
@@ -280,6 +288,7 @@ static void test_validate_fail_alternate(TestInputVisitorData *data,
visit_type_UserDefAlternate(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefAlternate(tmp);
}
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 933243d..8ecbc21 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -45,6 +45,8 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
{
Visitor *v;
+ visitor_input_teardown(data, NULL);
+
data->obj = qobject_from_jsonv(json_string, ap);
g_assert(data->obj);
@@ -174,12 +176,7 @@ static void test_visitor_in_enum(TestInputVisitorData *data,
visit_type_EnumOne(v, &res, NULL, &err);
g_assert(!err);
g_assert_cmpint(i, ==, res);
-
- visitor_input_teardown(data, NULL);
}
-
- data->obj = NULL;
- data->qiv = NULL;
}
@@ -202,12 +199,6 @@ static void test_visitor_in_struct(TestInputVisitorData *data,
g_free(p);
}
-static void check_and_free_str(char *str, const char *cmp)
-{
- g_assert_cmpstr(str, ==, cmp);
- g_free(str);
-}
-
static void test_visitor_in_struct_nested(TestInputVisitorData *data,
const void *unused)
{
@@ -223,17 +214,14 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(!err);
- check_and_free_str(udp->string0, "string0");
- check_and_free_str(udp->dict1->string1, "string1");
+ g_assert_cmpstr(udp->string0, ==, "string0");
+ g_assert_cmpstr(udp->dict1->string1, ==, "string1");
g_assert_cmpint(udp->dict1->dict2->userdef->integer, ==, 42);
- check_and_free_str(udp->dict1->dict2->userdef->string, "string");
- check_and_free_str(udp->dict1->dict2->string, "string2");
+ g_assert_cmpstr(udp->dict1->dict2->userdef->string, ==, "string");
+ g_assert_cmpstr(udp->dict1->dict2->string, ==, "string2");
g_assert(udp->dict1->has_dict3 == false);
- g_free(udp->dict1->dict2->userdef);
- g_free(udp->dict1->dict2);
- g_free(udp->dict1);
- g_free(udp);
+ qapi_free_UserDefTwo(udp);
}
static void test_visitor_in_list(TestInputVisitorData *data,
@@ -343,14 +331,12 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_I);
g_assert_cmpint(tmp->u.i, ==, 42);
qapi_free_UserDefAlternate(tmp);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "'string'");
visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_S);
g_assert_cmpstr(tmp->u.s, ==, "string");
qapi_free_UserDefAlternate(tmp);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "false");
visit_type_UserDefAlternate(v, &tmp, NULL, &err);
@@ -358,7 +344,6 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_UserDefAlternate(tmp);
- visitor_input_teardown(data, NULL);
}
static void test_visitor_in_alternate_number(TestInputVisitorData *data,
@@ -381,7 +366,6 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrBool(asb);
- visitor_input_teardown(data, NULL);
/* FIXME: Order of alternate should not affect semantics; asn should
* parse the same as ans */
@@ -393,35 +377,30 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrNum(asn);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltNumStr(v, &ans, NULL, &error_abort);
g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
g_assert_cmpfloat(ans->u.n, ==, 42);
qapi_free_AltNumStr(ans);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltStrInt(v, &asi, NULL, &error_abort);
g_assert_cmpint(asi->type, ==, ALT_STR_INT_KIND_I);
g_assert_cmpint(asi->u.i, ==, 42);
qapi_free_AltStrInt(asi);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltIntNum(v, &ain, NULL, &error_abort);
g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_I);
g_assert_cmpint(ain->u.i, ==, 42);
qapi_free_AltIntNum(ain);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltNumInt(v, &ani, NULL, &error_abort);
g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_I);
g_assert_cmpint(ani->u.i, ==, 42);
qapi_free_AltNumInt(ani);
- visitor_input_teardown(data, NULL);
/* Parsing a double */
@@ -431,21 +410,18 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrBool(asb);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrNum(v, &asn, NULL, &error_abort);
g_assert_cmpint(asn->type, ==, ALT_STR_NUM_KIND_N);
g_assert_cmpfloat(asn->u.n, ==, 42.5);
qapi_free_AltStrNum(asn);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltNumStr(v, &ans, NULL, &error_abort);
g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
g_assert_cmpfloat(ans->u.n, ==, 42.5);
qapi_free_AltNumStr(ans);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrInt(v, &asi, NULL, &err);
@@ -453,21 +429,18 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrInt(asi);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltIntNum(v, &ain, NULL, &error_abort);
g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_N);
g_assert_cmpfloat(ain->u.n, ==, 42.5);
qapi_free_AltIntNum(ain);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltNumInt(v, &ani, NULL, &error_abort);
g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_N);
g_assert_cmpfloat(ani->u.n, ==, 42.5);
qapi_free_AltNumInt(ani);
- visitor_input_teardown(data, NULL);
}
static void test_native_list_integer_helper(TestInputVisitorData *data,
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 9364843..8606111 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -391,6 +391,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qobj = QOBJECT(qdict);
visit_type_any(data->ov, &qobj, NULL, &err);
g_assert(!err);
+ qobject_decref(qobj);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
qdict = qobject_to_qdict(obj);
@@ -411,7 +412,6 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
g_assert(qstring);
g_assert_cmpstr(qstring_get_str(qstring), ==, "foo");
qobject_decref(obj);
- qobject_decref(qobj);
}
static void test_visitor_out_union_flat(TestOutputVisitorData *data,
@@ -463,6 +463,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
qapi_free_UserDefAlternate(tmp);
+ qobject_decref(arg);
}
static void test_visitor_out_empty(TestOutputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing in test-qmp-*
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (4 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-* Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 15:36 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup " Eric Blake
` (24 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
By using &error_abort, we can avoid a local err variable in
situations where we expect success. It also has the nice
effect that if the test breaks, the error message from
error_abort tends to be nicer than that of g_assert().
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: split into two pieces
v9: move earlier in series (was 14/17), reword commit message
v8: no change
v7: pick up whitespace changes dropped from earlier commit
v6: new patch
---
tests/test-qmp-input-strict.c | 31 +++++---------------
tests/test-qmp-input-visitor.c | 59 ++++++++++----------------------------
tests/test-qmp-output-visitor.c | 56 +++++++++---------------------------
tests/test-visitor-serialization.c | 42 +++++++++++----------------
4 files changed, 53 insertions(+), 135 deletions(-)
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 95cd8e1..6226397 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -94,13 +94,11 @@ static void test_validate_struct(TestInputVisitorData *data,
const void *unused)
{
TestStruct *p = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
- visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(!err);
+ visit_type_TestStruct(v, &p, NULL, &error_abort);
g_free(p->string);
g_free(p);
}
@@ -109,7 +107,6 @@ static void test_validate_struct_nested(TestInputVisitorData *data,
const void *unused)
{
UserDefTwo *udp = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'string0': 'string0', "
@@ -117,8 +114,7 @@ static void test_validate_struct_nested(TestInputVisitorData *data,
"'dict2': { 'userdef': { 'integer': 42, "
"'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefTwo(v, &udp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefTwo(v, &udp, NULL, &error_abort);
qapi_free_UserDefTwo(udp);
}
@@ -126,13 +122,11 @@ static void test_validate_list(TestInputVisitorData *data,
const void *unused)
{
UserDefOneList *head = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44 } ]");
- visit_type_UserDefOneList(v, &head, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefOneList(v, &head, NULL, &error_abort);
qapi_free_UserDefOneList(head);
}
@@ -141,12 +135,10 @@ static void test_validate_union_native_list(TestInputVisitorData *data,
{
UserDefNativeListUnion *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data, "{ 'type': 'integer', 'data' : [ 1, 2 ] }");
- visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefNativeListUnion(v, &tmp, NULL, &error_abort);
qapi_free_UserDefNativeListUnion(tmp);
}
@@ -155,7 +147,6 @@ static void test_validate_union_flat(TestInputVisitorData *data,
{
UserDefFlatUnion *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data,
"{ 'enum1': 'value1', "
@@ -163,8 +154,7 @@ static void test_validate_union_flat(TestInputVisitorData *data,
"'string': 'str', "
"'boolean': true }");
- visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefFlatUnion(v, &tmp, NULL, &error_abort);
qapi_free_UserDefFlatUnion(tmp);
}
@@ -173,12 +163,10 @@ static void test_validate_alternate(TestInputVisitorData *data,
{
UserDefAlternate *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data, "42");
- visit_type_UserDefAlternate(v, &tmp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
qapi_free_UserDefAlternate(tmp);
}
@@ -296,16 +284,11 @@ static void do_test_validate_qmp_introspect(TestInputVisitorData *data,
const char *schema_json)
{
SchemaInfoList *schema = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init_raw(data, schema_json);
- visit_type_SchemaInfoList(v, &schema, NULL, &err);
- if (err) {
- fprintf(stderr, "%s", error_get_pretty(err));
- }
- g_assert(!err);
+ visit_type_SchemaInfoList(v, &schema, NULL, &error_abort);
g_assert(schema);
qapi_free_SchemaInfoList(schema);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 8ecbc21..46566bc 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -89,13 +89,11 @@ static void test_visitor_in_int(TestInputVisitorData *data,
const void *unused)
{
int64_t res = 0, value = -42;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "%" PRId64, value);
- visit_type_int(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_int(v, &res, NULL, &error_abort);
g_assert_cmpint(res, ==, value);
}
@@ -120,14 +118,12 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
static void test_visitor_in_bool(TestInputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
bool res = false;
Visitor *v;
v = visitor_input_test_init(data, "true");
- visit_type_bool(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_bool(v, &res, NULL, &error_abort);
g_assert_cmpint(res, ==, true);
}
@@ -135,13 +131,11 @@ static void test_visitor_in_number(TestInputVisitorData *data,
const void *unused)
{
double res = 0, value = 3.14;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "%f", value);
- visit_type_number(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_number(v, &res, NULL, &error_abort);
g_assert_cmpfloat(res, ==, value);
}
@@ -149,13 +143,11 @@ static void test_visitor_in_string(TestInputVisitorData *data,
const void *unused)
{
char *res = NULL, *value = (char *) "Q E M U";
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "%s", value);
- visit_type_str(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_str(v, &res, NULL, &error_abort);
g_assert_cmpstr(res, ==, value);
g_free(res);
@@ -164,7 +156,6 @@ static void test_visitor_in_string(TestInputVisitorData *data,
static void test_visitor_in_enum(TestInputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
Visitor *v;
EnumOne i;
@@ -173,8 +164,7 @@ static void test_visitor_in_enum(TestInputVisitorData *data,
v = visitor_input_test_init(data, "%s", EnumOne_lookup[i]);
- visit_type_EnumOne(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_EnumOne(v, &res, NULL, &error_abort);
g_assert_cmpint(i, ==, res);
}
}
@@ -184,13 +174,11 @@ static void test_visitor_in_struct(TestInputVisitorData *data,
const void *unused)
{
TestStruct *p = NULL;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
- visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(!err);
+ visit_type_TestStruct(v, &p, NULL, &error_abort);
g_assert_cmpint(p->integer, ==, -42);
g_assert(p->boolean == true);
g_assert_cmpstr(p->string, ==, "foo");
@@ -203,7 +191,6 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
const void *unused)
{
UserDefTwo *udp = NULL;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "{ 'string0': 'string0', "
@@ -211,8 +198,7 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
"'dict2': { 'userdef': { 'integer': 42, "
"'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefTwo(v, &udp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefTwo(v, &udp, NULL, &error_abort);
g_assert_cmpstr(udp->string0, ==, "string0");
g_assert_cmpstr(udp->dict1->string1, ==, "string1");
@@ -228,14 +214,12 @@ static void test_visitor_in_list(TestInputVisitorData *data,
const void *unused)
{
UserDefOneList *item, *head = NULL;
- Error *err = NULL;
Visitor *v;
int i;
v = visitor_input_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44 } ]");
- visit_type_UserDefOneList(v, &head, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefOneList(v, &head, NULL, &error_abort);
g_assert(head != NULL);
for (i = 0, item = head; item; item = item->next, i++) {
@@ -253,7 +237,6 @@ static void test_visitor_in_any(TestInputVisitorData *data,
const void *unused)
{
QObject *res = NULL;
- Error *err = NULL;
Visitor *v;
QInt *qint;
QBool *qbool;
@@ -262,16 +245,14 @@ static void test_visitor_in_any(TestInputVisitorData *data,
QObject *qobj;
v = visitor_input_test_init(data, "-42");
- visit_type_any(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_any(v, &res, NULL, &error_abort);
qint = qobject_to_qint(res);
g_assert(qint);
g_assert_cmpint(qint_get_int(qint), ==, -42);
qobject_decref(res);
v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
- visit_type_any(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_any(v, &res, NULL, &error_abort);
qdict = qobject_to_qdict(res);
g_assert(qdict && qdict_size(qdict) == 3);
qobj = qdict_get(qdict, "integer");
@@ -296,7 +277,6 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,
const void *unused)
{
Visitor *v;
- Error *err = NULL;
UserDefFlatUnion *tmp;
UserDefUnionBase *base;
@@ -306,8 +286,7 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,
"'string': 'str', "
"'boolean': true }");
- visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefFlatUnion(v, &tmp, NULL, &error_abort);
g_assert_cmpint(tmp->enum1, ==, ENUM_ONE_VALUE1);
g_assert_cmpstr(tmp->string, ==, "str");
g_assert_cmpint(tmp->integer, ==, 41);
@@ -448,7 +427,6 @@ static void test_native_list_integer_helper(TestInputVisitorData *data,
UserDefNativeListUnionKind kind)
{
UserDefNativeListUnion *cvalue = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -465,8 +443,7 @@ static void test_native_list_integer_helper(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, kind);
@@ -611,7 +588,6 @@ static void test_visitor_in_native_list_bool(TestInputVisitorData *data,
{
UserDefNativeListUnion *cvalue = NULL;
boolList *elem = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -628,8 +604,7 @@ static void test_visitor_in_native_list_bool(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, USER_DEF_NATIVE_LIST_UNION_KIND_BOOLEAN);
@@ -647,7 +622,6 @@ static void test_visitor_in_native_list_string(TestInputVisitorData *data,
{
UserDefNativeListUnion *cvalue = NULL;
strList *elem = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -663,8 +637,7 @@ static void test_visitor_in_native_list_string(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, USER_DEF_NATIVE_LIST_UNION_KIND_STRING);
@@ -686,7 +659,6 @@ static void test_visitor_in_native_list_number(TestInputVisitorData *data,
{
UserDefNativeListUnion *cvalue = NULL;
numberList *elem = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -702,8 +674,7 @@ static void test_visitor_in_native_list_number(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, USER_DEF_NATIVE_LIST_UNION_KIND_NUMBER);
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 8606111..0164984 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -45,11 +45,9 @@ static void test_visitor_out_int(TestOutputVisitorData *data,
const void *unused)
{
int64_t value = -42;
- Error *err = NULL;
QObject *obj;
- visit_type_int(data->ov, &value, NULL, &err);
- g_assert(!err);
+ visit_type_int(data->ov, &value, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -62,12 +60,10 @@ static void test_visitor_out_int(TestOutputVisitorData *data,
static void test_visitor_out_bool(TestOutputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
bool value = true;
QObject *obj;
- visit_type_bool(data->ov, &value, NULL, &err);
- g_assert(!err);
+ visit_type_bool(data->ov, &value, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -81,11 +77,9 @@ static void test_visitor_out_number(TestOutputVisitorData *data,
const void *unused)
{
double value = 3.14;
- Error *err = NULL;
QObject *obj;
- visit_type_number(data->ov, &value, NULL, &err);
- g_assert(!err);
+ visit_type_number(data->ov, &value, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -99,11 +93,9 @@ static void test_visitor_out_string(TestOutputVisitorData *data,
const void *unused)
{
char *string = (char *) "Q E M U";
- Error *err = NULL;
QObject *obj;
- visit_type_str(data->ov, &string, NULL, &err);
- g_assert(!err);
+ visit_type_str(data->ov, &string, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -117,12 +109,10 @@ static void test_visitor_out_no_string(TestOutputVisitorData *data,
const void *unused)
{
char *string = NULL;
- Error *err = NULL;
QObject *obj;
/* A null string should return "" */
- visit_type_str(data->ov, &string, NULL, &err);
- g_assert(!err);
+ visit_type_str(data->ov, &string, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -135,13 +125,11 @@ static void test_visitor_out_no_string(TestOutputVisitorData *data,
static void test_visitor_out_enum(TestOutputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
QObject *obj;
EnumOne i;
for (i = 0; i < ENUM_ONE_MAX; i++) {
- visit_type_EnumOne(data->ov, &i, "unused", &err);
- g_assert(!err);
+ visit_type_EnumOne(data->ov, &i, "unused", &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -174,12 +162,10 @@ static void test_visitor_out_struct(TestOutputVisitorData *data,
.boolean = false,
.string = (char *) "foo"};
TestStruct *p = &test_struct;
- Error *err = NULL;
QObject *obj;
QDict *qdict;
- visit_type_TestStruct(data->ov, &p, NULL, &err);
- g_assert(!err);
+ visit_type_TestStruct(data->ov, &p, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -198,7 +184,6 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
const void *unused)
{
int64_t value = 42;
- Error *err = NULL;
UserDefTwo *ud2;
QObject *obj;
QDict *qdict, *dict1, *dict2, *dict3, *userdef;
@@ -225,8 +210,7 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
ud2->dict1->dict3->userdef->integer = value;
ud2->dict1->dict3->string = g_strdup(strings[3]);
- visit_type_UserDefTwo(data->ov, &ud2, "unused", &err);
- g_assert(!err);
+ visit_type_UserDefTwo(data->ov, &ud2, "unused", &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -288,7 +272,6 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
const int max_items = 10;
bool value_bool = true;
int value_int = 10;
- Error *err = NULL;
QListEntry *entry;
QObject *obj;
QList *qlist;
@@ -306,8 +289,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
head = p;
}
- visit_type_TestStructList(data->ov, &head, NULL, &err);
- g_assert(!err);
+ visit_type_TestStructList(data->ov, &head, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -367,7 +349,6 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
const void *unused)
{
QObject *qobj;
- Error *err = NULL;
QInt *qint;
QBool *qbool;
QString *qstring;
@@ -375,8 +356,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
QObject *obj;
qobj = QOBJECT(qint_from_int(-42));
- visit_type_any(data->ov, &qobj, NULL, &err);
- g_assert(!err);
+ visit_type_any(data->ov, &qobj, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QINT);
@@ -389,8 +369,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qdict_put(qdict, "boolean", qbool_from_bool(true));
qdict_put(qdict, "string", qstring_from_str("foo"));
qobj = QOBJECT(qdict);
- visit_type_any(data->ov, &qobj, NULL, &err);
- g_assert(!err);
+ visit_type_any(data->ov, &qobj, NULL, &error_abort);
qobject_decref(qobj);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -420,8 +399,6 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,
QObject *arg;
QDict *qdict;
- Error *err = NULL;
-
UserDefFlatUnion *tmp = g_malloc0(sizeof(UserDefFlatUnion));
tmp->enum1 = ENUM_ONE_VALUE1;
tmp->string = g_strdup("str");
@@ -429,8 +406,7 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,
tmp->integer = 41;
tmp->u.value1->boolean = true;
- visit_type_UserDefFlatUnion(data->ov, &tmp, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefFlatUnion(data->ov, &tmp, NULL, &error_abort);
arg = qmp_output_get_qobject(data->qov);
g_assert(qobject_type(arg) == QTYPE_QDICT);
@@ -449,14 +425,12 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
const void *unused)
{
QObject *arg;
- Error *err = NULL;
UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
tmp->type = USER_DEF_ALTERNATE_KIND_I;
tmp->u.i = 42;
- visit_type_UserDefAlternate(data->ov, &tmp, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
arg = qmp_output_get_qobject(data->qov);
g_assert(qobject_type(arg) == QTYPE_QINT);
@@ -697,14 +671,12 @@ static void test_native_list(TestOutputVisitorData *data,
UserDefNativeListUnionKind kind)
{
UserDefNativeListUnion *cvalue = g_new0(UserDefNativeListUnion, 1);
- Error *err = NULL;
QObject *obj;
cvalue->type = kind;
init_native_list(cvalue);
- visit_type_UserDefNativeListUnion(data->ov, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(data->ov, &cvalue, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
check_native_list(obj, cvalue->type);
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index c024e5e..9f67f9e 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -302,14 +302,13 @@ static void test_primitives(gconstpointer opaque)
const SerializeOps *ops = args->ops;
PrimitiveType *pt = args->test_data;
PrimitiveType *pt_copy = g_malloc0(sizeof(*pt_copy));
- Error *err = NULL;
void *serialize_data;
pt_copy->type = pt->type;
- ops->serialize(pt, &serialize_data, visit_primitive_type, &err);
- ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type, &err);
+ ops->serialize(pt, &serialize_data, visit_primitive_type, &error_abort);
+ ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type,
+ &error_abort);
- g_assert(err == NULL);
g_assert(pt_copy != NULL);
if (pt->type == PTYPE_STRING) {
g_assert_cmpstr(pt->value.string, ==, pt_copy->value.string);
@@ -345,7 +344,6 @@ static void test_primitive_lists(gconstpointer opaque)
PrimitiveList pl = { .value = { NULL } };
PrimitiveList pl_copy = { .value = { NULL } };
PrimitiveList *pl_copy_ptr = &pl_copy;
- Error *err = NULL;
void *serialize_data;
void *cur_head = NULL;
int i;
@@ -492,10 +490,11 @@ static void test_primitive_lists(gconstpointer opaque)
}
}
- ops->serialize((void **)&pl, &serialize_data, visit_primitive_list, &err);
- ops->deserialize((void **)&pl_copy_ptr, serialize_data, visit_primitive_list, &err);
+ ops->serialize((void **)&pl, &serialize_data, visit_primitive_list,
+ &error_abort);
+ ops->deserialize((void **)&pl_copy_ptr, serialize_data,
+ visit_primitive_list, &error_abort);
- g_assert(err == NULL);
i = 0;
/* compare our deserialized list of primitives to the original */
@@ -652,10 +651,8 @@ static void test_primitive_lists(gconstpointer opaque)
g_assert_cmpint(i, ==, 33);
ops->cleanup(serialize_data);
- dealloc_helper(&pl, visit_primitive_list, &err);
- g_assert(!err);
- dealloc_helper(&pl_copy, visit_primitive_list, &err);
- g_assert(!err);
+ dealloc_helper(&pl, visit_primitive_list, &error_abort);
+ dealloc_helper(&pl_copy, visit_primitive_list, &error_abort);
g_free(args);
}
@@ -665,13 +662,12 @@ static void test_struct(gconstpointer opaque)
const SerializeOps *ops = args->ops;
TestStruct *ts = struct_create();
TestStruct *ts_copy = NULL;
- Error *err = NULL;
void *serialize_data;
- ops->serialize(ts, &serialize_data, visit_struct, &err);
- ops->deserialize((void **)&ts_copy, serialize_data, visit_struct, &err);
+ ops->serialize(ts, &serialize_data, visit_struct, &error_abort);
+ ops->deserialize((void **)&ts_copy, serialize_data, visit_struct,
+ &error_abort);
- g_assert(err == NULL);
struct_compare(ts, ts_copy);
struct_cleanup(ts);
@@ -687,14 +683,12 @@ static void test_nested_struct(gconstpointer opaque)
const SerializeOps *ops = args->ops;
UserDefTwo *udnp = nested_struct_create();
UserDefTwo *udnp_copy = NULL;
- Error *err = NULL;
void *serialize_data;
- ops->serialize(udnp, &serialize_data, visit_nested_struct, &err);
+ ops->serialize(udnp, &serialize_data, visit_nested_struct, &error_abort);
ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct,
- &err);
+ &error_abort);
- g_assert(err == NULL);
nested_struct_compare(udnp, udnp_copy);
nested_struct_cleanup(udnp);
@@ -709,7 +703,6 @@ static void test_nested_struct_list(gconstpointer opaque)
TestArgs *args = (TestArgs *) opaque;
const SerializeOps *ops = args->ops;
UserDefTwoList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
- Error *err = NULL;
void *serialize_data;
int i = 0;
@@ -720,11 +713,10 @@ static void test_nested_struct_list(gconstpointer opaque)
listp = tmp;
}
- ops->serialize(listp, &serialize_data, visit_nested_struct_list, &err);
+ ops->serialize(listp, &serialize_data, visit_nested_struct_list,
+ &error_abort);
ops->deserialize((void **)&listp_copy, serialize_data,
- visit_nested_struct_list, &err);
-
- g_assert(err == NULL);
+ visit_nested_struct_list, &error_abort);
tmp = listp;
tmp_copy = listp_copy;
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup in test-qmp-*
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (5 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing " Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 15:40 ` Markus Armbruster
2015-11-06 17:04 ` [Qemu-devel] [PATCH] fixup! " Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 08/30] qapi: More tests of alternate output Eric Blake
` (23 subsequent siblings)
30 siblings, 2 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
By moving err into data, we can let test teardown take care
of cleaning up any collected error; it also gives us fewer
lines of code between repeated tests where init runs teardown
on our behalf.
Rather than duplicate code between .c files, I added a new
test-qmp-common.h. I debated about putting
error_free_or_abort() in error.h, but it seems like something
that is only useful for tests.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: split in two pieces
v9: move earlier in series (was 14/17), reword commit message
v8: no change
v7: pick up whitespace changes dropped from earlier commit
v6: new patch
---
tests/test-qmp-commands.c | 8 ++++----
tests/test-qmp-common.h | 22 ++++++++++++++++++++++
tests/test-qmp-event.c | 1 +
tests/test-qmp-input-strict.c | 22 ++++++++--------------
tests/test-qmp-input-visitor.c | 27 ++++++++-------------------
5 files changed, 43 insertions(+), 37 deletions(-)
create mode 100644 tests/test-qmp-common.h
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index f23d8ea..9f65fc2 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -1,12 +1,13 @@
#include <glib.h>
#include "qemu-common.h"
+#include "test-qmp-common.h"
#include "qapi/qmp/types.h"
#include "test-qmp-commands.h"
#include "qapi/qmp/dispatch.h"
#include "qemu/module.h"
#include "qapi/qmp-input-visitor.h"
-#include "tests/test-qapi-types.h"
-#include "tests/test-qapi-visit.h"
+#include "test-qapi-types.h"
+#include "test-qapi-visit.h"
void qmp_user_def_cmd(Error **errp)
{
@@ -225,8 +226,7 @@ static void test_dealloc_partial(void)
assert(ud2->dict1 == NULL);
/* confirm & release construction error */
- assert(err != NULL);
- error_free(err);
+ error_free_or_abort(&err);
/* tear down partial object */
qapi_free_UserDefTwo(ud2);
diff --git a/tests/test-qmp-common.h b/tests/test-qmp-common.h
new file mode 100644
index 0000000..043f49c
--- /dev/null
+++ b/tests/test-qmp-common.h
@@ -0,0 +1,22 @@
+/*
+ * Code common to qmp/qapi unit-tests.
+ *
+ * Copyright (C) 2015 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef TEST_QMP_COMMON_H__
+#define TEST_QMP_COMMON_H__
+
+/* Expect an error, abort() if there is none. */
+static inline void error_free_or_abort(Error **errp)
+{
+ g_assert(*errp);
+ error_free(*errp);
+ *errp = NULL;
+}
+
+#endif
diff --git a/tests/test-qmp-event.c b/tests/test-qmp-event.c
index 035c65c..c0ac3ea 100644
--- a/tests/test-qmp-event.c
+++ b/tests/test-qmp-event.c
@@ -22,6 +22,7 @@
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qobject.h"
#include "qapi/qmp-event.h"
+#include "test-qmp-common.h"
typedef struct TestEventData {
QDict *expect;
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 6226397..7e15b09 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -15,6 +15,7 @@
#include <stdarg.h>
#include "qemu-common.h"
+#include "test-qmp-common.h"
#include "qapi/qmp-input-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
@@ -180,8 +181,7 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
v = validate_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo', 'extra': 42 }");
visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
if (p) {
g_free(p->string);
}
@@ -198,8 +198,7 @@ static void test_validate_fail_struct_nested(TestInputVisitorData *data,
v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");
visit_type_UserDefTwo(v, &udp, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
qapi_free_UserDefTwo(udp);
}
@@ -213,8 +212,7 @@ static void test_validate_fail_list(TestInputVisitorData *data,
v = validate_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44, 'extra': 'ggg' } ]");
visit_type_UserDefOneList(v, &head, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
qapi_free_UserDefOneList(head);
}
@@ -229,8 +227,7 @@ static void test_validate_fail_union_native_list(TestInputVisitorData *data,
"{ 'type': 'integer', 'data' : [ 'string' ] }");
visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
qapi_free_UserDefNativeListUnion(tmp);
}
@@ -244,8 +241,7 @@ static void test_validate_fail_union_flat(TestInputVisitorData *data,
v = validate_test_init(data, "{ 'string': 'c', 'integer': 41, 'boolean': true }");
visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
qapi_free_UserDefFlatUnion(tmp);
}
@@ -260,8 +256,7 @@ static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
v = validate_test_init(data, "{ 'integer': 42, 'string': 'c', 'string1': 'd', 'string2': 'e' }");
visit_type_UserDefFlatUnion2(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
qapi_free_UserDefFlatUnion2(tmp);
}
@@ -275,8 +270,7 @@ static void test_validate_fail_alternate(TestInputVisitorData *data,
v = validate_test_init(data, "3.14");
visit_type_UserDefAlternate(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
qapi_free_UserDefAlternate(tmp);
}
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 46566bc..2b59b6f 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -14,6 +14,7 @@
#include <stdarg.h>
#include "qemu-common.h"
+#include "test-qmp-common.h"
#include "qapi/qmp-input-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
@@ -111,8 +112,7 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
v = visitor_input_test_init(data, "%f", DBL_MAX);
visit_type_int(v, &res, NULL, &err);
- g_assert(err);
- error_free(err);
+ error_free_or_abort(&err);
}
static void test_visitor_in_bool(TestInputVisitorData *data,
@@ -319,9 +319,7 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
v = visitor_input_test_init(data, "false");
visit_type_UserDefAlternate(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ error_free_or_abort(&err);
qapi_free_UserDefAlternate(tmp);
}
@@ -341,9 +339,7 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
v = visitor_input_test_init(data, "42");
visit_type_AltStrBool(v, &asb, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ error_free_or_abort(&err);
qapi_free_AltStrBool(asb);
/* FIXME: Order of alternate should not affect semantics; asn should
@@ -352,9 +348,7 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
visit_type_AltStrNum(v, &asn, NULL, &err);
/* FIXME g_assert_cmpint(asn->type, == ALT_STR_NUM_KIND_N); */
/* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */
- g_assert(err);
- error_free(err);
- err = NULL;
+ error_free_or_abort(&err);
qapi_free_AltStrNum(asn);
v = visitor_input_test_init(data, "42");
@@ -385,9 +379,7 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrBool(v, &asb, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ error_free_or_abort(&err);
qapi_free_AltStrBool(asb);
v = visitor_input_test_init(data, "42.5");
@@ -404,9 +396,7 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrInt(v, &asi, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ error_free_or_abort(&err);
qapi_free_AltStrInt(asi);
v = visitor_input_test_init(data, "42.5");
@@ -713,12 +703,11 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
v = visitor_input_test_init(data, "{ 'integer': false, 'boolean': 'foo', 'string': -42 }");
visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(err);
+ error_free_or_abort(&err);
/* FIXME - a failed parse should not leave a partially-allocated p
* for us to clean up; this could cause callers to leak memory. */
g_assert(p->string == NULL);
- error_free(err);
g_free(p->string);
g_free(p);
}
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 08/30] qapi: More tests of alternate output
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (6 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup " Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 09/30] qapi: Test failure in middle of array parse Eric Blake
` (22 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
The testsuite was only covering that we could output the 'int'
branch of an alternate (no additional allocation/cleanup required).
Add a test of the 'str' branch, to make sure that things still
work even when a branch involves allocation.
Update to modern style of g_new0() over g_malloc0() while
touching it.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: improve commit message
v9: new patch, split off of 10/17
---
tests/test-qmp-output-visitor.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 0164984..0d0c859 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -425,8 +425,9 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
const void *unused)
{
QObject *arg;
+ UserDefAlternate *tmp;
- UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
+ tmp = g_new0(UserDefAlternate, 1);
tmp->type = USER_DEF_ALTERNATE_KIND_I;
tmp->u.i = 42;
@@ -438,6 +439,19 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
qapi_free_UserDefAlternate(tmp);
qobject_decref(arg);
+
+ tmp = g_new0(UserDefAlternate, 1);
+ tmp->type = USER_DEF_ALTERNATE_KIND_S;
+ tmp->u.s = g_strdup("hello");
+
+ visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
+ arg = qmp_output_get_qobject(data->qov);
+
+ g_assert(qobject_type(arg) == QTYPE_QSTRING);
+ g_assert_cmpstr(qstring_get_str(qobject_to_qstring(arg)), ==, "hello");
+
+ qapi_free_UserDefAlternate(tmp);
+ qobject_decref(arg);
}
static void test_visitor_out_empty(TestOutputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 09/30] qapi: Test failure in middle of array parse
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (7 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 08/30] qapi: More tests of alternate output Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 10/30] qapi: More tests of input arrays Eric Blake
` (21 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Our generated list visitors have the same problem as has been
mentioned elsewhere (see commit 2f52e20): they allocate data
even on failure. An upcoming patch will correct things to
provide saner guarantees, but first we need to expose the
behavior in the testsuite to ensure we aren't introducing any
memory usage bugs.
There are more test cases throughout the test-qmp-input-* tests
that already deal with partial allocation; a later commit will
clean up all visit_type_FOO(), without marking all of the tests
with FIXME at this time.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: rebase to earlier changes, more commit message text
v9: move earlier in series (was 15/17)
v8: no change
v7: no change
v6: rebase onto earlier gen_err_check() and testsuite improvements
---
scripts/qapi-visit.py | 4 ++++
tests/test-qmp-input-visitor.c | 10 +++++++++-
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index f40c3c7..3ef5c16 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -138,6 +138,10 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
def gen_visit_list(name, element_type):
+ # FIXME: if *obj is NULL on entry, and the first visit_next_list()
+ # assigns to *obj, while a later one fails, we should clean up *obj
+ # rather than leaving it non-NULL. As currently written, the caller must
+ # call qapi_free_FOOList() to avoid a memory leak of the partial FOOList.
return mcgen('''
void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error **errp)
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 2b59b6f..b9b0c31 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -699,8 +699,10 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
TestStruct *p = NULL;
Error *err = NULL;
Visitor *v;
+ strList *q = NULL;
- v = visitor_input_test_init(data, "{ 'integer': false, 'boolean': 'foo', 'string': -42 }");
+ v = visitor_input_test_init(data, "{ 'integer': false, 'boolean': 'foo', "
+ "'string': -42 }");
visit_type_TestStruct(v, &p, NULL, &err);
error_free_or_abort(&err);
@@ -710,6 +712,12 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
g_free(p->string);
g_free(p);
+
+ v = visitor_input_test_init(data, "[ '1', '2', false, '3' ]");
+ visit_type_strList(v, &q, NULL, &err);
+ error_free_or_abort(&err);
+ assert(q);
+ qapi_free_strList(q);
}
int main(int argc, char **argv)
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 10/30] qapi: More tests of input arrays
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (8 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 09/30] qapi: Test failure in middle of array parse Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 11/30] qapi: Provide nicer array names in introspection Eric Blake
` (20 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Our testsuite had no coverage of empty arrays, nor of what
happens when the input does not match the expected type.
Useful to have, especially if we start changing the visitor
contracts.
I did not think it worth duplicating these additions to
test-qmp-input-strict; since all strict mode does is add
the ability to reject JSON input that has more keys than
what the visitor expects, yet the additions in this patch
error out earlier than that point regardless of whether
strict mode was requested.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: rebase to earlier changes, more in commit message
v9: move earlier in series (was 16/17)
v8: no change
v7: no change
v6: new patch
---
tests/test-qmp-input-visitor.c | 52 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 52 insertions(+)
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index b9b0c31..6a94683 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -231,6 +231,12 @@ static void test_visitor_in_list(TestInputVisitorData *data,
}
qapi_free_UserDefOneList(head);
+ head = NULL;
+
+ /* An empty list is valid */
+ v = visitor_input_test_init(data, "[]");
+ visit_type_UserDefOneList(v, &head, NULL, &error_abort);
+ g_assert(!head);
}
static void test_visitor_in_any(TestInputVisitorData *data,
@@ -720,6 +726,50 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
qapi_free_strList(q);
}
+static void test_visitor_in_wrong_type(TestInputVisitorData *data,
+ const void *unused)
+{
+ TestStruct *p = NULL;
+ Visitor *v;
+ strList *q = NULL;
+ int64_t i;
+ Error *err = NULL;
+
+ /* Make sure arrays and structs cannot be confused */
+
+ v = visitor_input_test_init(data, "[]");
+ visit_type_TestStruct(v, &p, NULL, &err);
+ error_free_or_abort(&err);
+ g_assert(!p);
+
+ v = visitor_input_test_init(data, "{}");
+ visit_type_strList(v, &q, NULL, &err);
+ error_free_or_abort(&err);
+ assert(!q);
+
+ /* Make sure primitives and struct cannot be confused */
+
+ v = visitor_input_test_init(data, "1");
+ visit_type_TestStruct(v, &p, NULL, &err);
+ error_free_or_abort(&err);
+ g_assert(!p);
+
+ v = visitor_input_test_init(data, "{}");
+ visit_type_int(v, &i, NULL, &err);
+ error_free_or_abort(&err);
+
+ /* Make sure primitives and arrays cannot be confused */
+
+ v = visitor_input_test_init(data, "1");
+ visit_type_strList(v, &q, NULL, &err);
+ error_free_or_abort(&err);
+ assert(!q);
+
+ v = visitor_input_test_init(data, "[]");
+ visit_type_int(v, &i, NULL, &err);
+ error_free_or_abort(&err);
+}
+
int main(int argc, char **argv)
{
TestInputVisitorData in_visitor_data;
@@ -752,6 +802,8 @@ int main(int argc, char **argv)
&in_visitor_data, test_visitor_in_alternate);
input_visitor_test_add("/visitor/input/errors",
&in_visitor_data, test_visitor_in_errors);
+ input_visitor_test_add("/visitor/input/wrong-type",
+ &in_visitor_data, test_visitor_in_wrong_type);
input_visitor_test_add("/visitor/input/alternate-number",
&in_visitor_data, test_visitor_in_alternate_number);
input_visitor_test_add("/visitor/input/native_list/int",
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 11/30] qapi: Provide nicer array names in introspection
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (9 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 10/30] qapi: More tests of input arrays Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting Eric Blake
` (19 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
For the sake of humans reading introspection output, it is nice
to have the name of implicit array types be recognizable as
arrays of the underlying type. However, while this patch allows
humans to skip from a command with return type "[123]" straight
to the definition of type "123" without having to first inspect
type "[123]", document that this shortcut should not be taken by
client apps.
This makes the resulting introspection string slightly larger by
default (just over 200 bytes), but it's in the noise (less than
0.3% of the overall 70k size of 'query-qmp-capabilities').
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: no change
v8: commit message stats on the size increase
v7: no change
v6: no change
---
docs/qapi-code-gen.txt | 7 +++++--
scripts/qapi-introspect.py | 8 +++++---
2 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 4d8c2fc..ba29bc6 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -662,11 +662,14 @@ Example: the SchemaInfo for BlockRef from section Alternate types
The SchemaInfo for an array type has meta-type "array", and variant
member "element-type", which names the array's element type. Array
-types are implicitly defined.
+types are implicitly defined. For convenience, the array's name may
+resemble the element type; however, clients should examine member
+"element-type" instead of making assumptions based on parsing member
+"name".
Example: the SchemaInfo for ['str']
- { "name": "strList", "meta-type": "array",
+ { "name": "[str]", "meta-type": "array",
"element-type": "str" }
The SchemaInfo for an enumeration type has meta-type "enum" and
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
index c0dad66..64f2cd0 100644
--- a/scripts/qapi-introspect.py
+++ b/scripts/qapi-introspect.py
@@ -107,10 +107,12 @@ const char %(c_name)s[] = %(c_string)s;
# characters.
if isinstance(typ, QAPISchemaBuiltinType):
return typ.name
+ if isinstance(typ, QAPISchemaArrayType):
+ return '[' + self._use_type(typ.element_type) + ']'
return self._name(typ.name)
def _gen_json(self, name, mtype, obj):
- if mtype != 'command' and mtype != 'event' and mtype != 'builtin':
+ if mtype not in ('command', 'event', 'builtin', 'array'):
name = self._name(name)
obj['name'] = name
obj['meta-type'] = mtype
@@ -136,8 +138,8 @@ const char %(c_name)s[] = %(c_string)s;
self._gen_json(name, 'enum', {'values': values})
def visit_array_type(self, name, info, element_type):
- self._gen_json(name, 'array',
- {'element-type': self._use_type(element_type)})
+ element = self._use_type(element_type)
+ self._gen_json('[' + element + ']', 'array', {'element-type': element})
def visit_object_type_flat(self, name, info, members, variants):
obj = {'members': [self._gen_member(m) for m in members]}
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (10 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 11/30] qapi: Provide nicer array names in introspection Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 15:52 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 13/30] qapi: Track simple union tag in object.local_members Eric Blake
` (18 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
qapi-code-gen.txt already claims that types, commands, and
events share a common namespace; set this in stone by further
documenting that our introspection output will never have
collisions with the same name tied to more than one meta-type.
Our largest QMP enum currently has 125 values, our largest
object type has 27 members, and the mean for each is less than
10. These sizes are small enough that the per-element overhead
of O(log n) binary searching probably outweighs the speed
possible with direct O(n) linear searching (a better algorithm
with more overhead will only beat a leaner naive algorithm only
as you scale to larger input sizes).
Arguably, the overall SchemaInfo array could be sorted by name;
there, we currently have 531 entities, large enough for a binary
search to be faster than linear. However, remember that we have
mutually-recursive types, which means there is no topological
ordering that will allow clients to learn all information about
that type in a single linear pass; thus clients will want to do
random access over the data, and they will probably read the
introspection output into a hashtable for O(1) lookup rather
than O(log n) binary searching, at which point, pre-sorting our
introspection output doesn't help the client.
It doesn't help that sorting can be subjective if you introduce
locales into the mix (I'm not experienced enough with Python
to know for sure, but at least it looks like it defaults to
sorting in the C locale even when run under a different locale).
And while our current introspection output is deterministic
(because we visit entities in a sorted order), we may want
to change that order in the future (such as using OrderedDict
to stick to .json declaration order).
For these reasons, we simply document that clients should not
rely on any particular order of items in introspection output.
And since it is now a documented part of the contract, we have
the freedom to later rearrange output if needed, without
worrying about breaking well-written clients that were already
aware that they had to do linear searches.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: wording touchups based on review
v9: retitle; and swing in the opposite direction: give the user
no guarantee about order
v8: no change
v7: tweak commit wording
v6: no change from v5
---
docs/qapi-code-gen.txt | 19 +++++++++++++++----
qapi/introspect.json | 17 ++++++++++++-----
2 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index ba29bc6..f9fa6f3 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -516,6 +516,10 @@ query-qmp-schema. QGA currently doesn't support introspection.
query-qmp-schema returns a JSON array of SchemaInfo objects. These
objects together describe the wire ABI, as defined in the QAPI schema.
+There is no specified order to the SchemaInfo objects returned; a
+client must search for a particular name throughout the entire array
+to learn more about that name, but is at least guaranteed that there
+will be no collisions between type, command, and event names.
However, the SchemaInfo can't reflect all the rules and restrictions
that apply to QMP. It's interface introspection (figuring out what's
@@ -596,7 +600,9 @@ any. Each element is a JSON object with members "name" (the member's
name), "type" (the name of its type), and optionally "default". The
member is optional if "default" is present. Currently, "default" can
only have value null. Other values are reserved for future
-extensions.
+extensions. The "members" array is in no particular order; clients
+must search the entire object when learning whether a particular
+member is supported.
Example: the SchemaInfo for MyType from section Struct types
@@ -610,7 +616,9 @@ Example: the SchemaInfo for MyType from section Struct types
"variants" is a JSON array describing the object's variant members.
Each element is a JSON object with members "case" (the value of type
tag this element applies to) and "type" (the name of an object type
-that provides the variant members for this type tag value).
+that provides the variant members for this type tag value). The
+"variants" array is in no particular order, and is not guaranteed to
+list cases in the same order as the corresponding "tag" enum type.
Example: the SchemaInfo for flat union BlockdevOptions from section
Union types
@@ -651,7 +659,8 @@ Union types
The SchemaInfo for an alternate type has meta-type "alternate", and
variant member "members". "members" is a JSON array. Each element is
a JSON object with member "type", which names a type. Values of the
-alternate type conform to exactly one of its member types.
+alternate type conform to exactly one of its member types. There is
+no guarantee on the order in which "members" will be listed.
Example: the SchemaInfo for BlockRef from section Alternate types
@@ -673,7 +682,9 @@ Example: the SchemaInfo for ['str']
"element-type": "str" }
The SchemaInfo for an enumeration type has meta-type "enum" and
-variant member "values".
+variant member "values". The values are listed in no particular
+order; clients must search the entire enum when learning whether a
+particular value is supported.
Example: the SchemaInfo for MyEnum from section Enumeration types
diff --git a/qapi/introspect.json b/qapi/introspect.json
index cc50dc6..e7c4c3e 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -25,6 +25,10 @@
# Returns: array of @SchemaInfo, where each element describes an
# entity in the ABI: command, event, type, ...
#
+# The order of the various SchemaInfo is unspecified; however, all
+# names are guaranteed to be unique (no name will be duplicated with
+# different meta-types).
+#
# Note: the QAPI schema is also used to help define *internal*
# interfaces, by defining QAPI types. These are not part of the QMP
# wire ABI, and therefore not returned by this command.
@@ -78,7 +82,8 @@
# Commands and events have the name defined in the QAPI schema.
# Unlike command and event names, type names are not part of
# the wire ABI. Consequently, type names are meaningless
-# strings here.
+# strings here, although they are still guaranteed unique
+# regardless of @meta-type.
#
# All references to other SchemaInfo are by name.
#
@@ -130,7 +135,7 @@
#
# Additional SchemaInfo members for meta-type 'enum'.
#
-# @values: the enumeration type's values.
+# @values: the enumeration type's values, in no particular order.
#
# Values of this type are JSON string on the wire.
#
@@ -158,14 +163,16 @@
#
# Additional SchemaInfo members for meta-type 'object'.
#
-# @members: the object type's (non-variant) members.
+# @members: the object type's (non-variant) members, in no particular order.
#
# @tag: #optional the name of the member serving as type tag.
# An element of @members with this name must exist.
#
# @variants: #optional variant members, i.e. additional members that
# depend on the type tag's value. Present exactly when
-# @tag is present.
+# @tag is present. The variants are in no particular order,
+# and may even differ from the order of the values of the
+# enum type of the @tag.
#
# Values of this type are JSON object on the wire.
#
@@ -219,7 +226,7 @@
#
# Additional SchemaInfo members for meta-type 'alternate'.
#
-# @members: the alternate type's members.
+# @members: the alternate type's members, in no particular order.
# The members' wire encoding is distinct, see
# docs/qapi-code-gen.txt section Alternate types.
#
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 13/30] qapi: Track simple union tag in object.local_members
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (11 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 14/30] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
` (17 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: improve commit message, add comments, tweak how alternates
check tag_member
v8: new patch
---
scripts/qapi-types.py | 5 ++++-
scripts/qapi-visit.py | 5 ++++-
scripts/qapi.py | 16 +++++++++++-----
tests/qapi-schema/qapi-schema-test.out | 2 ++
tests/qapi-schema/union-clash-data.out | 1 +
tests/qapi-schema/union-empty.out | 1 +
6 files changed, 23 insertions(+), 7 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index b37900f..946afab 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -269,7 +269,10 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_object_type(self, name, info, base, members, variants):
self._fwdecl += gen_fwd_object_or_array(name)
if variants:
- assert not members # not implemented
+ if members:
+ # Members other than variants.tag_member not implemented
+ assert len(members) == 1
+ assert members[0] == variants.tag_member
self.decl += gen_union(name, base, variants)
else:
self.decl += gen_struct(name, base, members)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3ef5c16..94cd113 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -364,7 +364,10 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
def visit_object_type(self, name, info, base, members, variants):
self.decl += gen_visit_decl(name)
if variants:
- assert not members # not implemented
+ if members:
+ # Members other than variants.tag_member not implemented
+ assert len(members) == 1
+ assert members[0] == variants.tag_member
self.defn += gen_visit_union(name, base, variants)
else:
self.defn += gen_visit_struct(name, base, members)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 7c50cc4..a814e20 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -957,6 +957,9 @@ class QAPISchemaArrayType(QAPISchemaType):
class QAPISchemaObjectType(QAPISchemaType):
def __init__(self, name, info, base, local_members, variants):
+ # struct has local_members, optional base, and no variants
+ # flat union has base, variants, and no local_members
+ # simple union has local_members, variants, and no base
QAPISchemaType.__init__(self, name, info)
assert base is None or isinstance(base, str)
for m in local_members:
@@ -1048,10 +1051,11 @@ class QAPISchemaObjectTypeVariants(object):
self.variants = variants
def check(self, schema, members, seen):
- if self.tag_name:
+ # seen is non-empty for unions, empty for alternates
+ if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
- else:
- self.tag_member.check(schema, members, seen)
+ if seen:
+ assert self.tag_member in seen.itervalues()
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
vseen = dict(seen)
@@ -1085,6 +1089,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
+ self.variants.tag_member.check(schema, [], {})
self.variants.check(schema, [], {})
def json_type(self):
@@ -1270,13 +1275,14 @@ class QAPISchema(object):
if tag_name:
variants = [self._make_variant(key, value)
for (key, value) in data.iteritems()]
+ members = []
else:
variants = [self._make_simple_variant(key, value, info)
for (key, value) in data.iteritems()]
tag_member = self._make_implicit_tag(name, info, variants)
+ members = [tag_member]
self._def_entity(
- QAPISchemaObjectType(name, info, base,
- self._make_members(OrderedDict(), info),
+ QAPISchemaObjectType(name, info, base, members,
QAPISchemaObjectTypeVariants(tag_name,
tag_member,
variants)))
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index e20a823..786024e 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -132,6 +132,7 @@ object UserDefFlatUnion2
case value2: UserDefB
case value3: UserDefA
object UserDefNativeListUnion
+ member type: UserDefNativeListUnionKind optional=False
case integer: :obj-intList-wrapper
case s8: :obj-int8List-wrapper
case s16: :obj-int16List-wrapper
@@ -187,6 +188,7 @@ object __org.qemu_x-Struct
object __org.qemu_x-Struct2
member array: __org.qemu_x-Union1List optional=False
object __org.qemu_x-Union1
+ member type: __org.qemu_x-Union1Kind optional=False
case __org.qemu_x-branch: :obj-str-wrapper
enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch']
object __org.qemu_x-Union2
diff --git a/tests/qapi-schema/union-clash-data.out b/tests/qapi-schema/union-clash-data.out
index 6277239..cea8551 100644
--- a/tests/qapi-schema/union-clash-data.out
+++ b/tests/qapi-schema/union-clash-data.out
@@ -2,5 +2,6 @@ object :empty
object :obj-int-wrapper
member data: int optional=False
object TestUnion
+ member type: TestUnionKind optional=False
case data: :obj-int-wrapper
enum TestUnionKind ['data']
diff --git a/tests/qapi-schema/union-empty.out b/tests/qapi-schema/union-empty.out
index 8b5a7bf..9c89fd1 100644
--- a/tests/qapi-schema/union-empty.out
+++ b/tests/qapi-schema/union-empty.out
@@ -1,3 +1,4 @@
object :empty
object Union
+ member type: UnionKind optional=False
enum UnionKind []
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 14/30] qapi-types: Consolidate gen_struct() and gen_union()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (12 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 13/30] qapi: Track simple union tag in object.local_members Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 15/30] qapi-types: Simplify gen_struct_field[s] Eric Blake
` (16 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
These two methods are now close enough that we can finally merge
them, relying on the fact that simple unions now provide a
reasonable local_members. Change gen_struct() to gen_object()
that handles all forms of QAPISchemaObjectType, and rename and
shrink gen_union() to gen_variants() to handle the portion of
gen_object() needed when variants are present.
gen_struct_fields() now has a single caller, so it no longer
needs an optional parameter; however, I did not choose to inline
it into the caller.
No difference to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: rebase to earlier changes
v8: new patch
---
scripts/qapi-types.py | 37 +++++++++++--------------------------
1 file changed, 11 insertions(+), 26 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 946afab..403768b 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -51,7 +51,7 @@ def gen_struct_field(member):
return ret
-def gen_struct_fields(local_members, base=None):
+def gen_struct_fields(local_members, base):
ret = ''
if base:
@@ -70,7 +70,7 @@ def gen_struct_fields(local_members, base=None):
return ret
-def gen_struct(name, base, members):
+def gen_object(name, base, members, variants):
ret = mcgen('''
struct %(c_name)s {
@@ -79,11 +79,14 @@ struct %(c_name)s {
ret += gen_struct_fields(members, base)
+ if variants:
+ ret += gen_variants(variants)
+
# Make sure that all structs have at least one field; this avoids
# potential issues with attempting to malloc space for zero-length
# structs in C, and also incompatibility with C++ (where an empty
# struct is size 1).
- if not (base and base.members) and not members:
+ if not (base and base.members) and not members and not variants:
ret += mcgen('''
char qapi_dummy_field_for_empty_struct;
''')
@@ -140,17 +143,7 @@ const int %(c_name)s_qtypes[QTYPE_MAX] = {
return ret
-def gen_union(name, base, variants):
- ret = mcgen('''
-
-struct %(c_name)s {
-''',
- c_name=c_name(name))
- if base:
- ret += gen_struct_fields([], base)
- else:
- ret += gen_struct_field(variants.tag_member)
-
+def gen_variants(variants):
# FIXME: What purpose does data serve, besides preventing a union that
# has a branch named 'data'? We use it in qapi-visit.py to decide
# whether to bypass the switch statement if visiting the discriminator
@@ -159,11 +152,11 @@ struct %(c_name)s {
# should not be any data leaks even without a data pointer. Or, if
# 'data' is merely added to guarantee we don't have an empty union,
# shouldn't we enforce that at .json parse time?
- ret += mcgen('''
+ ret = mcgen('''
union { /* union tag is @%(c_name)s */
void *data;
''',
- c_name=c_name(variants.tag_member.name))
+ c_name=c_name(variants.tag_member.name))
for var in variants.variants:
# Ugly special case for simple union TODO get rid of it
@@ -176,7 +169,6 @@ struct %(c_name)s {
ret += mcgen('''
} u;
-};
''')
return ret
@@ -268,14 +260,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_object_type(self, name, info, base, members, variants):
self._fwdecl += gen_fwd_object_or_array(name)
- if variants:
- if members:
- # Members other than variants.tag_member not implemented
- assert len(members) == 1
- assert members[0] == variants.tag_member
- self.decl += gen_union(name, base, variants)
- else:
- self.decl += gen_struct(name, base, members)
+ self.decl += gen_object(name, base, members, variants)
if base:
self.decl += gen_upcast(name, base)
self._gen_type_cleanup(name)
@@ -283,7 +268,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_alternate_type(self, name, info, variants):
self._fwdecl += gen_fwd_object_or_array(name)
self._fwdefn += gen_alternate_qtypes(name, variants)
- self.decl += gen_union(name, None, variants)
+ self.decl += gen_object(name, None, [variants.tag_member], variants)
self.decl += gen_alternate_qtypes_decl(name)
self._gen_type_cleanup(name)
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 15/30] qapi-types: Simplify gen_struct_field[s]
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (13 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 14/30] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 16/30] qapi: Drop obsolete tag value collision assertions Eric Blake
` (15 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Simplify gen_struct_fields() back to a single iteration over a
list of fields (like it was prior to commit f87ab7f9), by moving
the generated comments to gen_object(). Then, inline
gen_struct_field() into its only caller.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: new patch
---
scripts/qapi-types.py | 40 +++++++++++++++-------------------------
1 file changed, 15 insertions(+), 25 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 403768b..2f2f7df 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -36,48 +36,38 @@ struct %(c_name)s {
c_name=c_name(name), c_type=element_type.c_type())
-def gen_struct_field(member):
+def gen_struct_fields(members):
ret = ''
-
- if member.optional:
- ret += mcgen('''
+ for memb in members:
+ if memb.optional:
+ ret += mcgen('''
bool has_%(c_name)s;
''',
- c_name=c_name(member.name))
- ret += mcgen('''
+ c_name=c_name(memb.name))
+ ret += mcgen('''
%(c_type)s %(c_name)s;
''',
- c_type=member.type.c_type(), c_name=c_name(member.name))
+ c_type=memb.type.c_type(), c_name=c_name(memb.name))
return ret
-def gen_struct_fields(local_members, base):
- ret = ''
+def gen_object(name, base, members, variants):
+ ret = mcgen('''
+
+struct %(c_name)s {
+''',
+ c_name=c_name(name))
if base:
ret += mcgen('''
/* Members inherited from %(c_name)s: */
''',
c_name=base.c_name())
- for memb in base.members:
- ret += gen_struct_field(memb)
+ ret += gen_struct_fields(base.members)
ret += mcgen('''
/* Own members: */
''')
-
- for memb in local_members:
- ret += gen_struct_field(memb)
- return ret
-
-
-def gen_object(name, base, members, variants):
- ret = mcgen('''
-
-struct %(c_name)s {
-''',
- c_name=c_name(name))
-
- ret += gen_struct_fields(members, base)
+ ret += gen_struct_fields(members)
if variants:
ret += gen_variants(variants)
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 16/30] qapi: Drop obsolete tag value collision assertions
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (14 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 15/30] qapi-types: Simplify gen_struct_field[s] Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check() Eric Blake
` (14 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
Union tag values can't clash with member names in generated C anymore
since commit e4ba22b, but QAPISchemaObjectTypeVariants.check() still
asserts they don't. Drop it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-1-git-send-email-armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a814e20..1ac870d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1058,8 +1058,7 @@ class QAPISchemaObjectTypeVariants(object):
assert self.tag_member in seen.itervalues()
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
- vseen = dict(seen)
- v.check(schema, self.tag_member.type, vseen)
+ v.check(schema, self.tag_member.type, {})
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (15 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 16/30] qapi: Drop obsolete tag value collision assertions Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 12:31 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 18/30] qapi: Clean up after previous commit Eric Blake
` (13 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
QAPISchemaObjectTypeMember.check() currently does four things:
1. Compute self.type
2. Accumulate members in all_members
Only one caller cares: QAPISchemaObjectType.check() uses it to
compute self.members. The other callers pass a throw-away
accumulator.
3. Accumulate a map from names to members in seen
Only one caller cares: QAPISchemaObjectType.check() uses it to
compute its local variable seen, for self.variants.check(), which
uses it to compute self.variants.tag_member from
self.variants.tag_name. The other callers pass a throw-away
accumulator.
4. Check for collisions
This piggyback on 3: before adding a new entry, we assert it's new.
Only one caller cares: QAPISchemaObjectType.check() uses it to
assert non-variant members don't clash.
Simplify QAPISchemaObjectType.check(): move 2.-4. to
QAPISchemaObjectType.check(), and drop parameters all_members and
seen.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-2-git-send-email-armbru@redhat.com>
[rebase to earlier changes that moved tag_member.check() of
alternate types]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 1ac870d..a4350b2 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -990,7 +990,10 @@ class QAPISchemaObjectType(QAPISchemaType):
assert c_name(m.name) not in seen
seen[m.name] = m
for m in self.local_members:
- m.check(schema, members, seen)
+ m.check(schema)
+ assert m.name not in seen
+ seen[m.name] = m
+ members.append(m)
if self.variants:
self.variants.check(schema, members, seen)
self.members = members
@@ -1027,12 +1030,9 @@ class QAPISchemaObjectTypeMember(object):
self.type = None
self.optional = optional
- def check(self, schema, all_members, seen):
- assert self.name not in seen
+ def check(self, schema):
self.type = schema.lookup_type(self._type_name)
assert self.type
- all_members.append(self)
- seen[self.name] = self
class QAPISchemaObjectTypeVariants(object):
@@ -1066,7 +1066,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
def check(self, schema, tag_type, seen):
- QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ QAPISchemaObjectTypeMember.check(self, schema)
assert self.name in tag_type.values
# This function exists to support ugly simple union special cases
@@ -1088,7 +1088,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
- self.variants.tag_member.check(schema, [], {})
+ self.variants.tag_member.check(schema)
self.variants.check(schema, [], {})
def json_type(self):
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 18/30] qapi: Clean up after previous commit
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (16 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 19/30] qapi: Fix up commit 7618b91's clash sanity checking change Eric Blake
` (12 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
QAPISchemaObjectTypeVariants.check() parameter members and
QAPISchemaObjectTypeVariant.check() parameter seen are no longer used,
drop them.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-3-git-send-email-armbru@redhat.com>
[rebase to earlier changes that moved tag_member.check() of
alternate types]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a4350b2..0bf8235 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -995,7 +995,7 @@ class QAPISchemaObjectType(QAPISchemaType):
seen[m.name] = m
members.append(m)
if self.variants:
- self.variants.check(schema, members, seen)
+ self.variants.check(schema, seen)
self.members = members
def is_implicit(self):
@@ -1050,22 +1050,21 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
- def check(self, schema, members, seen):
- # seen is non-empty for unions, empty for alternates
+ def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
if seen:
assert self.tag_member in seen.itervalues()
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
- v.check(schema, self.tag_member.type, {})
+ v.check(schema, self.tag_member.type)
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
- def check(self, schema, tag_type, seen):
+ def check(self, schema, tag_type):
QAPISchemaObjectTypeMember.check(self, schema)
assert self.name in tag_type.values
@@ -1089,7 +1088,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
def check(self, schema):
self.variants.tag_member.check(schema)
- self.variants.check(schema, [], {})
+ self.variants.check(schema, {})
def json_type(self):
return 'value'
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 19/30] qapi: Fix up commit 7618b91's clash sanity checking change
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (17 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 18/30] qapi: Clean up after previous commit Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 20/30] qapi: Eliminate QAPISchemaObjectType.check() variable members Eric Blake
` (11 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
This hunk
@@ -964,6 +965,7 @@ class QAPISchemaObjectType(QAPISchemaType):
members = []
seen = {}
for m in members:
+ assert c_name(m.name) not in seen
seen[m.name] = m
for m in self.local_members:
m.check(schema, members, seen)
is plainly broken.
Asserting the members inherited from base don't clash is somewhat
redundant, because self.base.check() just checked that. But it
doesn't hurt.
The idea to use c_name(m.name) instead of m.name for collision
checking is sound, because we need to catch clashes between the m.name
and between the c_name(m.name), and when two m.name clash, then their
c_name() also clash.
However, using c_name(m.name) instead of m.name in one of several
places doesn't work. See the very next line.
Keep the assertion, but drop the c_name() for now. A future commit
will bring it back.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-4-git-send-email-armbru@redhat.com>
[change TABs in commit message to space]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 0bf8235..86d2adc 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -987,7 +987,7 @@ class QAPISchemaObjectType(QAPISchemaType):
members = []
seen = {}
for m in members:
- assert c_name(m.name) not in seen
+ assert m.name not in seen
seen[m.name] = m
for m in self.local_members:
m.check(schema)
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 20/30] qapi: Eliminate QAPISchemaObjectType.check() variable members
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (18 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 19/30] qapi: Fix up commit 7618b91's clash sanity checking change Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 21/30] qapi: Factor out QAPISchemaObjectTypeMember.check_clash() Eric Blake
` (10 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
We can use seen.values() instead if we make it an OrderedDict.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-5-git-send-email-armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 86d2adc..44d08c1 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -977,26 +977,22 @@ class QAPISchemaObjectType(QAPISchemaType):
if self.members:
return
self.members = False # mark as being checked
+ seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
assert isinstance(self.base, QAPISchemaObjectType)
assert not self.base.variants # not implemented
self.base.check(schema)
- members = list(self.base.members)
- else:
- members = []
- seen = {}
- for m in members:
- assert m.name not in seen
- seen[m.name] = m
+ for m in self.base.members:
+ assert m.name not in seen
+ seen[m.name] = m
for m in self.local_members:
m.check(schema)
assert m.name not in seen
seen[m.name] = m
- members.append(m)
if self.variants:
self.variants.check(schema, seen)
- self.members = members
+ self.members = seen.values()
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 21/30] qapi: Factor out QAPISchemaObjectTypeMember.check_clash()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (19 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 20/30] qapi: Eliminate QAPISchemaObjectType.check() variable members Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check() Eric Blake
` (9 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
While there, stick in a TODO change key of seen from QAPI name to C
name. Can't do it right away, because it would fail the assertion for
tests/qapi-schema/args-has-clash.json.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-6-git-send-email-armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 44d08c1..2a73b2b 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -984,12 +984,10 @@ class QAPISchemaObjectType(QAPISchemaType):
assert not self.base.variants # not implemented
self.base.check(schema)
for m in self.base.members:
- assert m.name not in seen
- seen[m.name] = m
+ m.check_clash(seen)
for m in self.local_members:
m.check(schema)
- assert m.name not in seen
- seen[m.name] = m
+ m.check_clash(seen)
if self.variants:
self.variants.check(schema, seen)
self.members = seen.values()
@@ -1030,6 +1028,11 @@ class QAPISchemaObjectTypeMember(object):
self.type = schema.lookup_type(self._type_name)
assert self.type
+ def check_clash(self, seen):
+ # TODO change key of seen from QAPI name to C name
+ assert self.name not in seen
+ seen[self.name] = self
+
class QAPISchemaObjectTypeVariants(object):
def __init__(self, tag_name, tag_member, variants):
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (20 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 21/30] qapi: Factor out QAPISchemaObjectTypeMember.check_clash() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 12:38 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches Eric Blake
` (8 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
From: Markus Armbruster <armbru@redhat.com>
Reduce the ugly flat union / simple union conditional by doing just
the essential work here, namely setting self.tag_member.
Move the rest to callers.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-7-git-send-email-armbru@redhat.com>
[rebase to earlier changes that moved tag_member.check() of
alternate types, and tweak commit title and wording]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: redo closer to Markus' original proposal
v9: new patch
---
scripts/qapi.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 2a73b2b..e057408 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -988,9 +988,10 @@ class QAPISchemaObjectType(QAPISchemaType):
for m in self.local_members:
m.check(schema)
m.check_clash(seen)
+ self.members = seen.values()
if self.variants:
self.variants.check(schema, seen)
- self.members = seen.values()
+ assert self.variants.tag_member in self.members
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
@@ -1052,8 +1053,6 @@ class QAPISchemaObjectTypeVariants(object):
def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
- if seen:
- assert self.tag_member in seen.itervalues()
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
v.check(schema, self.tag_member.type)
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (21 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 12:56 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash() Eric Blake
` (7 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any qapi member names that would
conflict with the non-variant qapi members already present
from the union's base type (see flat-union-clash-member.json).
We want QAPISchemaObjectType.check() to make the same check,
so we can later reduce some of the ad hoc checks.
We already ensure that all branches of a flat union are qapi
structs with no variants, at which point those members appear
in the same JSON object as all non-variant members. And we
already have a map 'seen' of all non-variant members. All
we need is a new QAPISchemaObjectTypeVariants.check_clash(),
which clones the seen map then checks for clashes with each
member of the variant's qapi type.
Note that the clone of seen inside Variants.check_clash()
resembles the one we just removed from Variants.check(); the
difference here is that we are now checking for clashes
among the qapi members of the variant type, rather than for
a single clash with the variant tag name itself.
In general, a type used as a branch of a flat union cannot
also be the base type of the flat union, so even though we are
adding a call to variant.type.check() in order to populate
variant.type.members, this is merely a case of gaining
topological sorting of how types are visited (and type.check()
is already set up to allow multiple calls due to base types).
For simple unions, the same code happens to work by design,
because of our synthesized wrapper classes (however, the
wrapper has a single member 'data' which will never collide
with the one non-variant member 'type', so it doesn't really
matter).
There is no impact to alternates, which intentionally do not
need to call variants.check_clash() (there, at most one of
the variant's branches will be an ObjectType, and even if one
exists, we are not inlining the qapi members of that object
into a parent object, the way we do for unions).
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: create new Variants.check_clash() rather than piggybacking
on .check()
v9: new patch, split off from v8 7/17
---
scripts/qapi.py | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index e057408..4c56935 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -992,6 +992,7 @@ class QAPISchemaObjectType(QAPISchemaType):
if self.variants:
self.variants.check(schema, seen)
assert self.variants.tag_member in self.members
+ self.variants.check_clash(schema, seen)
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
@@ -1057,6 +1058,17 @@ class QAPISchemaObjectTypeVariants(object):
for v in self.variants:
v.check(schema, self.tag_member.type)
+ def check_clash(self, schema, seen):
+ for v in self.variants:
+ # Reset seen map for each variant, since qapi names from one
+ # branch do not affect another branch
+ vseen = dict(seen)
+ assert isinstance(v.type, QAPISchemaObjectType)
+ assert not v.type.variants # not implemented
+ v.type.check(schema)
+ for m in v.type.members:
+ m.check_clash(vseen)
+
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (22 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 13:00 ` Markus Armbruster
2015-11-09 14:49 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check() Eric Blake
` (6 subsequent siblings)
30 siblings, 2 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Consolidate two common sequences of clash detection into a
new QAPISchemaObjectType.check_clash() helper method.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: rebase on new Variants.check_clash()
v9: new patch, split off from v8 7/17
---
scripts/qapi.py | 19 ++++++++-----------
1 file changed, 8 insertions(+), 11 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 4c56935..6d8c4c7 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
- assert isinstance(self.base, QAPISchemaObjectType)
- assert not self.base.variants # not implemented
- self.base.check(schema)
- for m in self.base.members:
- m.check_clash(seen)
+ self.base.check_clash(schema, seen)
for m in self.local_members:
m.check(schema)
m.check_clash(seen)
@@ -994,6 +990,12 @@ class QAPISchemaObjectType(QAPISchemaType):
assert self.variants.tag_member in self.members
self.variants.check_clash(schema, seen)
+ def check_clash(self, schema, seen):
+ self.check(schema)
+ assert not self.variants # not implemented
+ for m in self.members:
+ m.check_clash(seen)
+
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
return self.name[0] == ':'
@@ -1062,12 +1064,7 @@ class QAPISchemaObjectTypeVariants(object):
for v in self.variants:
# Reset seen map for each variant, since qapi names from one
# branch do not affect another branch
- vseen = dict(seen)
- assert isinstance(v.type, QAPISchemaObjectType)
- assert not v.type.variants # not implemented
- v.type.check(schema)
- for m in v.type.members:
- m.check_clash(vseen)
+ v.type.check_clash(schema, dict(seen))
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check()
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (23 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 13:07 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 26/30] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
` (5 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Checking that a given QAPISchemaObjectTypeVariant.name is a
member of the corresponding QAPISchemaEnumType of the owning
QAPISchemaObjectTypeVariants.tag_member ensures that there are
no collisions in the generated C union for those tag values
(since the enum itself should have no collisions).
However, this check was the only thing that Variant.check() was
doing beyond the work of the superclass ObjectTypeMember.check(),
and resulted in a difference of the .check() signatures just to
pass the enum type down.
Simplify things by instead doing the tag name check as part of
Variants.check(), at which point we can rely on inheritance
instead of overriding Variant.check().
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: new patch
---
scripts/qapi.py | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6d8c4c7..798df51 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1056,9 +1056,11 @@ class QAPISchemaObjectTypeVariants(object):
def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
- assert isinstance(self.tag_member.type, QAPISchemaEnumType)
+ tag_type = self.tag_member.type
+ assert isinstance(tag_type, QAPISchemaEnumType)
for v in self.variants:
- v.check(schema, self.tag_member.type)
+ v.check(schema)
+ assert v.name in tag_type.values
def check_clash(self, schema, seen):
for v in self.variants:
@@ -1071,10 +1073,6 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
- def check(self, schema, tag_type):
- QAPISchemaObjectTypeMember.check(self, schema)
- assert self.name in tag_type.values
-
# This function exists to support ugly simple union special cases
# TODO get rid of them, and drop the function
def simple_union_type(self):
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 26/30] qapi: Remove outdated tests related to QMP/branch collisions
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (24 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check() Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member Eric Blake
` (4 subsequent siblings)
30 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Now that branches are in a separate C namespace, we can remove
the restrictions in the parser that claim a branch name would
collide with QMP, and delete the negative tests that are no
longer problematic. A separate patch can then add positive
tests to qapi-schema-test to test that any corner cases will
compile correctly.
This reverts the scripts/qapi.py portion of commit 7b2a5c2,
now that the assertions that it plugged are no longer possible.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: no change
v9: no change
v8: retitle and update commit message, finish reversion of 7b2a5c2
v7: new patch (also temporarily appeared in subset B v10)
---
scripts/qapi.py | 11 ++---------
tests/Makefile | 3 ---
tests/qapi-schema/flat-union-clash-branch.err | 0
tests/qapi-schema/flat-union-clash-branch.exit | 1 -
tests/qapi-schema/flat-union-clash-branch.json | 18 ------------------
tests/qapi-schema/flat-union-clash-branch.out | 14 --------------
tests/qapi-schema/flat-union-clash-type.err | 1 -
tests/qapi-schema/flat-union-clash-type.exit | 1 -
tests/qapi-schema/flat-union-clash-type.json | 14 --------------
tests/qapi-schema/flat-union-clash-type.out | 0
tests/qapi-schema/union-clash-type.err | 1 -
tests/qapi-schema/union-clash-type.exit | 1 -
tests/qapi-schema/union-clash-type.json | 9 ---------
tests/qapi-schema/union-clash-type.out | 0
14 files changed, 2 insertions(+), 72 deletions(-)
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.err
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.json
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.out
delete mode 100644 tests/qapi-schema/flat-union-clash-type.err
delete mode 100644 tests/qapi-schema/flat-union-clash-type.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-type.json
delete mode 100644 tests/qapi-schema/flat-union-clash-type.out
delete mode 100644 tests/qapi-schema/union-clash-type.err
delete mode 100644 tests/qapi-schema/union-clash-type.exit
delete mode 100644 tests/qapi-schema/union-clash-type.json
delete mode 100644 tests/qapi-schema/union-clash-type.out
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 798df51..b914785 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -548,8 +548,7 @@ def check_union(expr, expr_info):
base = expr.get('base')
discriminator = expr.get('discriminator')
members = expr['data']
- values = {'MAX': '(automatic)', 'KIND': '(automatic)',
- 'TYPE': '(automatic)'}
+ values = {'MAX': '(automatic)'}
# Two types of unions, determined by discriminator.
@@ -607,19 +606,13 @@ def check_union(expr, expr_info):
" of branch '%s'" % key)
# If the discriminator names an enum type, then all members
- # of 'data' must also be members of the enum type, which in turn
- # must not collide with the discriminator name.
+ # of 'data' must also be members of the enum type.
if enum_define:
if key not in enum_define['enum_values']:
raise QAPIExprError(expr_info,
"Discriminator value '%s' is not found in "
"enum '%s'" %
(key, enum_define["enum_name"]))
- if discriminator in enum_define['enum_values']:
- raise QAPIExprError(expr_info,
- "Discriminator name '%s' collides with "
- "enum value in '%s'" %
- (discriminator, enum_define["enum_name"]))
# Otherwise, check for conflicts in the generated enum
else:
diff --git a/tests/Makefile b/tests/Makefile
index 92969e8..c84c6cb 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -277,9 +277,7 @@ qapi-schema += flat-union-bad-base.json
qapi-schema += flat-union-bad-discriminator.json
qapi-schema += flat-union-base-any.json
qapi-schema += flat-union-base-union.json
-qapi-schema += flat-union-clash-branch.json
qapi-schema += flat-union-clash-member.json
-qapi-schema += flat-union-clash-type.json
qapi-schema += flat-union-empty.json
qapi-schema += flat-union-inline.json
qapi-schema += flat-union-int-branch.json
@@ -341,7 +339,6 @@ qapi-schema += union-bad-branch.json
qapi-schema += union-base-no-discriminator.json
qapi-schema += union-clash-branches.json
qapi-schema += union-clash-data.json
-qapi-schema += union-clash-type.json
qapi-schema += union-empty.json
qapi-schema += union-invalid-base.json
qapi-schema += union-max.json
diff --git a/tests/qapi-schema/flat-union-clash-branch.err b/tests/qapi-schema/flat-union-clash-branch.err
deleted file mode 100644
index e69de29..0000000
diff --git a/tests/qapi-schema/flat-union-clash-branch.exit b/tests/qapi-schema/flat-union-clash-branch.exit
deleted file mode 100644
index 573541a..0000000
--- a/tests/qapi-schema/flat-union-clash-branch.exit
+++ /dev/null
@@ -1 +0,0 @@
-0
diff --git a/tests/qapi-schema/flat-union-clash-branch.json b/tests/qapi-schema/flat-union-clash-branch.json
deleted file mode 100644
index e593336..0000000
--- a/tests/qapi-schema/flat-union-clash-branch.json
+++ /dev/null
@@ -1,18 +0,0 @@
-# Flat union branch name collision
-# FIXME: this parses, but then fails to compile due to a duplicate 'c_d'
-# (one from the base member, the other from the branch name). We should
-# either reject the collision at parse time, or munge the generated branch
-# name to allow this to compile.
-{ 'enum': 'TestEnum',
- 'data': [ 'base', 'c-d' ] }
-{ 'struct': 'Base',
- 'data': { 'enum1': 'TestEnum', '*c_d': 'str' } }
-{ 'struct': 'Branch1',
- 'data': { 'string': 'str' } }
-{ 'struct': 'Branch2',
- 'data': { 'value': 'int' } }
-{ 'union': 'TestUnion',
- 'base': 'Base',
- 'discriminator': 'enum1',
- 'data': { 'base': 'Branch1',
- 'c-d': 'Branch2' } }
diff --git a/tests/qapi-schema/flat-union-clash-branch.out b/tests/qapi-schema/flat-union-clash-branch.out
deleted file mode 100644
index 8e0da73..0000000
--- a/tests/qapi-schema/flat-union-clash-branch.out
+++ /dev/null
@@ -1,14 +0,0 @@
-object :empty
-object Base
- member enum1: TestEnum optional=False
- member c_d: str optional=True
-object Branch1
- member string: str optional=False
-object Branch2
- member value: int optional=False
-enum TestEnum ['base', 'c-d']
-object TestUnion
- base Base
- tag enum1
- case base: Branch1
- case c-d: Branch2
diff --git a/tests/qapi-schema/flat-union-clash-type.err b/tests/qapi-schema/flat-union-clash-type.err
deleted file mode 100644
index b44dd40..0000000
--- a/tests/qapi-schema/flat-union-clash-type.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/flat-union-clash-type.json:11: Discriminator name 'type' collides with enum value in 'TestEnum'
diff --git a/tests/qapi-schema/flat-union-clash-type.exit b/tests/qapi-schema/flat-union-clash-type.exit
deleted file mode 100644
index d00491f..0000000
--- a/tests/qapi-schema/flat-union-clash-type.exit
+++ /dev/null
@@ -1 +0,0 @@
-1
diff --git a/tests/qapi-schema/flat-union-clash-type.json b/tests/qapi-schema/flat-union-clash-type.json
deleted file mode 100644
index 8f710f0..0000000
--- a/tests/qapi-schema/flat-union-clash-type.json
+++ /dev/null
@@ -1,14 +0,0 @@
-# Flat union branch 'type'
-# Reject this, because we would have a clash in generated C, between the
-# outer tag 'type' and the branch name 'type' within the union.
-# TODO: We could munge the generated C branch name to let it compile.
-{ 'enum': 'TestEnum',
- 'data': [ 'type' ] }
-{ 'struct': 'Base',
- 'data': { 'type': 'TestEnum' } }
-{ 'struct': 'Branch1',
- 'data': { 'string': 'str' } }
-{ 'union': 'TestUnion',
- 'base': 'Base',
- 'discriminator': 'type',
- 'data': { 'type': 'Branch1' } }
diff --git a/tests/qapi-schema/flat-union-clash-type.out b/tests/qapi-schema/flat-union-clash-type.out
deleted file mode 100644
index e69de29..0000000
diff --git a/tests/qapi-schema/union-clash-type.err b/tests/qapi-schema/union-clash-type.err
deleted file mode 100644
index a5dead1..0000000
--- a/tests/qapi-schema/union-clash-type.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/union-clash-type.json:8: Union 'TestUnion' member 'kind' clashes with '(automatic)'
diff --git a/tests/qapi-schema/union-clash-type.exit b/tests/qapi-schema/union-clash-type.exit
deleted file mode 100644
index d00491f..0000000
--- a/tests/qapi-schema/union-clash-type.exit
+++ /dev/null
@@ -1 +0,0 @@
-1
diff --git a/tests/qapi-schema/union-clash-type.json b/tests/qapi-schema/union-clash-type.json
deleted file mode 100644
index cfc256b..0000000
--- a/tests/qapi-schema/union-clash-type.json
+++ /dev/null
@@ -1,9 +0,0 @@
-# Union branch 'type'
-# Reject this, because we would have a clash in generated C, between the
-# simple union's implicit tag member 'kind' and the branch name 'kind'
-# within the union.
-# TODO: Even when the generated C is switched to use 'type' rather than
-# 'kind', to match the QMP spelling, the collision should still be detected.
-# Or, we could munge the branch name to allow compilation.
-{ 'union': 'TestUnion',
- 'data': { 'kind': 'int', 'type': 'str' } }
diff --git a/tests/qapi-schema/union-clash-type.out b/tests/qapi-schema/union-clash-type.out
deleted file mode 100644
index e69de29..0000000
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (25 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 26/30] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 14:26 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names Eric Blake
` (3 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (argument of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner(), plus a
tweak to report event data separately from command arguments.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10 (now in subset C): rebase to latest, set alternate tag_member
owner from alternate
v9 (now in subset D): rebase to earlier changes, hoist 'role' to top
of class, split out _pretty_helper(), manage owner when tag_member
appears as part of local_members for unions
v8: don't munge implicit type names [except for event data], and
instead make describe() create nicer messages. Add set_owner(), and
use field 'role' instead of method _describe()
v7: total rewrite: rework implicit object names, assign owner
when initializing owner type rather than when creating member
python object
v6: rebase on new lazy array creation and simple union 'type'
motion; tweak commit message
---
scripts/qapi.py | 44 +++++++++++++++++++++++++++++++---
tests/qapi-schema/qapi-schema-test.out | 8 +++----
2 files changed, 45 insertions(+), 7 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index b914785..b3af973 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -957,8 +957,10 @@ class QAPISchemaObjectType(QAPISchemaType):
assert base is None or isinstance(base, str)
for m in local_members:
assert isinstance(m, QAPISchemaObjectTypeMember)
- assert (variants is None or
- isinstance(variants, QAPISchemaObjectTypeVariants))
+ m.set_owner(name)
+ if variants is not None:
+ assert isinstance(variants, QAPISchemaObjectTypeVariants)
+ variants.set_owner(name)
self._base_name = base
self.base = None
self.local_members = local_members
@@ -1012,6 +1014,8 @@ class QAPISchemaObjectType(QAPISchemaType):
class QAPISchemaObjectTypeMember(object):
+ role = 'member'
+
def __init__(self, name, typ, optional):
assert isinstance(name, str)
assert isinstance(typ, str)
@@ -1020,8 +1024,14 @@ class QAPISchemaObjectTypeMember(object):
self._type_name = typ
self.type = None
self.optional = optional
+ self.owner = None
+
+ def set_owner(self, name):
+ assert not self.owner
+ self.owner = name
def check(self, schema):
+ assert self.owner
self.type = schema.lookup_type(self._type_name)
assert self.type
@@ -1030,6 +1040,25 @@ class QAPISchemaObjectTypeMember(object):
assert self.name not in seen
seen[self.name] = self
+ def _pretty_owner(self):
+ # See QAPISchema._make_implicit_object_type() - reverse the
+ # mapping there to create a nice human-readable description
+ owner = self.owner
+ if owner.startswith(':obj-'):
+ owner = owner[5:]
+ if owner.endswith('-arg'):
+ return '(argument of %s)' % owner[:-4]
+ elif owner.endswith('-data'):
+ return '(data of %s)' % owner[:-5]
+ else:
+ assert owner.endswith('-wrapper')
+ return '(branch of %s)' % owner[:-8]
+ else:
+ return '(%s of %s)' % (self.role, owner)
+
+ def describe(self):
+ return "'%s' %s" % (self.name, self._pretty_owner())
+
class QAPISchemaObjectTypeVariants(object):
def __init__(self, tag_name, tag_member, variants):
@@ -1046,6 +1075,10 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
+ def set_owner(self, name):
+ for v in self.variants:
+ v.set_owner(name)
+
def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
@@ -1063,6 +1096,8 @@ class QAPISchemaObjectTypeVariants(object):
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
+ role = 'branch'
+
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
@@ -1082,9 +1117,11 @@ class QAPISchemaAlternateType(QAPISchemaType):
QAPISchemaType.__init__(self, name, info)
assert isinstance(variants, QAPISchemaObjectTypeVariants)
assert not variants.tag_name
+ variants.set_owner(name)
self.variants = variants
def check(self, schema):
+ self.variants.tag_member.set_owner(self.name)
self.variants.tag_member.check(schema)
self.variants.check(schema, {})
@@ -1212,6 +1249,7 @@ class QAPISchema(object):
def _make_implicit_object_type(self, name, info, role, members):
if not members:
return None
+ # See also QAPISchemaObjectTypeMember.describe()
name = ':obj-%s-%s' % (name, role)
if not self.lookup_entity(name, QAPISchemaObjectType):
self._def_entity(QAPISchemaObjectType(name, info, None,
@@ -1315,7 +1353,7 @@ class QAPISchema(object):
data = expr.get('data')
if isinstance(data, OrderedDict):
data = self._make_implicit_object_type(
- name, info, 'arg', self._make_members(data, info))
+ name, info, 'data', self._make_members(data, info))
self._def_entity(QAPISchemaEvent(name, info, data))
def _def_exprs(self):
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 786024e..f78ef04 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -1,9 +1,9 @@
object :empty
-object :obj-EVENT_C-arg
+object :obj-EVENT_C-data
member a: int optional=True
member b: UserDefOne optional=True
member c: str optional=False
-object :obj-EVENT_D-arg
+object :obj-EVENT_D-data
member a: EventStructOne optional=False
member b: str optional=False
member c: str optional=True
@@ -79,8 +79,8 @@ alternate AltStrNum
enum AltStrNumKind ['s', 'n']
event EVENT_A None
event EVENT_B None
-event EVENT_C :obj-EVENT_C-arg
-event EVENT_D :obj-EVENT_D-arg
+event EVENT_C :obj-EVENT_C-data
+event EVENT_D :obj-EVENT_D-data
object Empty1
object Empty2
base Empty1
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (26 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 15:17 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union Eric Blake
` (2 subsequent siblings)
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Detect attempts to declare two object members that would result
in the same C member name, by keying the 'seen' dictionary off
of the C name rather than the qapi name. It also requires passing
info through the check_clash() methods.
This addresses a TODO and fixes the previously-broken
args-name-clash test. The resulting error message demonstrates
the utility of the .describe() method added previously. No change
to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10 (now in subset C): rebase to latest; update commit message
v9 (now in subset D): rebase to earlier changes, now only one test
affected
v8: rebase to earlier changes
v7: split out error reporting prep and member.c_name() addition
v6: rebase to earlier testsuite and info improvements
---
scripts/qapi.py | 31 +++++++++++++++++++------------
tests/qapi-schema/args-name-clash.err | 1 +
tests/qapi-schema/args-name-clash.exit | 2 +-
tests/qapi-schema/args-name-clash.json | 6 +++---
tests/qapi-schema/args-name-clash.out | 6 ------
5 files changed, 24 insertions(+), 22 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index b3af973..3a359cb 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -975,21 +975,24 @@ class QAPISchemaObjectType(QAPISchemaType):
seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
- self.base.check_clash(schema, seen)
+ self.base.check_clash(schema, self.info, seen)
for m in self.local_members:
m.check(schema)
- m.check_clash(seen)
+ m.check_clash(self.info, seen)
self.members = seen.values()
if self.variants:
self.variants.check(schema, seen)
assert self.variants.tag_member in self.members
- self.variants.check_clash(schema, seen)
+ self.variants.check_clash(schema, self.info, seen)
- def check_clash(self, schema, seen):
+ # Check that the members of this type do not cause duplicate JSON fields,
+ # and update seen to track the members seen so far. Report any errors
+ # on behalf of info, which is not necessarily self.info
+ def check_clash(self, schema, info, seen):
self.check(schema)
assert not self.variants # not implemented
for m in self.members:
- m.check_clash(seen)
+ m.check_clash(info, seen)
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
@@ -1035,10 +1038,13 @@ class QAPISchemaObjectTypeMember(object):
self.type = schema.lookup_type(self._type_name)
assert self.type
- def check_clash(self, seen):
- # TODO change key of seen from QAPI name to C name
- assert self.name not in seen
- seen[self.name] = self
+ def check_clash(self, info, seen):
+ name = c_name(self.name)
+ if name in seen:
+ raise QAPIExprError(info,
+ "%s collides with %s"
+ % (self.describe(), seen[name].describe()))
+ seen[name] = self
def _pretty_owner(self):
# See QAPISchema._make_implicit_object_type() - reverse the
@@ -1081,18 +1087,19 @@ class QAPISchemaObjectTypeVariants(object):
def check(self, schema, seen):
if self.tag_name: # flat union
- self.tag_member = seen[self.tag_name]
+ self.tag_member = seen[c_name(self.tag_name)]
+ assert self.tag_name == self.tag_member.name
tag_type = self.tag_member.type
assert isinstance(tag_type, QAPISchemaEnumType)
for v in self.variants:
v.check(schema)
assert v.name in tag_type.values
- def check_clash(self, schema, seen):
+ def check_clash(self, schema, info, seen):
for v in self.variants:
# Reset seen map for each variant, since qapi names from one
# branch do not affect another branch
- v.type.check_clash(schema, dict(seen))
+ v.type.check_clash(schema, info, dict(seen))
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
diff --git a/tests/qapi-schema/args-name-clash.err b/tests/qapi-schema/args-name-clash.err
index e69de29..2735217 100644
--- a/tests/qapi-schema/args-name-clash.err
+++ b/tests/qapi-schema/args-name-clash.err
@@ -0,0 +1 @@
+tests/qapi-schema/args-name-clash.json:5: 'a_b' (argument of oops) collides with 'a-b' (argument of oops)
diff --git a/tests/qapi-schema/args-name-clash.exit b/tests/qapi-schema/args-name-clash.exit
index 573541a..d00491f 100644
--- a/tests/qapi-schema/args-name-clash.exit
+++ b/tests/qapi-schema/args-name-clash.exit
@@ -1 +1 @@
-0
+1
diff --git a/tests/qapi-schema/args-name-clash.json b/tests/qapi-schema/args-name-clash.json
index 9e8f889..3fe4ea5 100644
--- a/tests/qapi-schema/args-name-clash.json
+++ b/tests/qapi-schema/args-name-clash.json
@@ -1,5 +1,5 @@
# C member name collision
-# FIXME - This parses, but fails to compile, because the C struct is given
-# two 'a_b' members. Either reject this at parse time, or munge the C names
-# to avoid the collision.
+# Reject members that clash when mapped to C names (we would have two 'a_b'
+# members). It would also be possible to munge the C names to avoid the
+# collision, but unlikely to be worth the effort.
{ 'command': 'oops', 'data': { 'a-b': 'str', 'a_b': 'str' } }
diff --git a/tests/qapi-schema/args-name-clash.out b/tests/qapi-schema/args-name-clash.out
index 9b2f6e4..e69de29 100644
--- a/tests/qapi-schema/args-name-clash.out
+++ b/tests/qapi-schema/args-name-clash.out
@@ -1,6 +0,0 @@
-object :empty
-object :obj-oops-arg
- member a-b: str optional=False
- member a_b: str optional=False
-command oops :obj-oops-arg -> None
- gen=True success_response=True
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (27 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 15:22 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 30/30] qapi: Forbid case-insensitive clashes Eric Blake
2015-11-06 16:03 ` [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Markus Armbruster
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: Paolo Bonzini, armbru, Luiz Capitulino
When qapi type CpuInfo was originally created for 0.14, we had
no notion of a flat union, and instead just listed a bunch of
optional fields with documentation about the mutually-exclusive
choice of which instruction pointer field(s) would be provided
for a given architecture. But now that we have flat unions and
introspection, it is better to segregate off which fields will
be provided according to the actual architecture. With this in
place, we no longer need the fields to be optional, because the
choice of the discriminator serves that role.
This has an additional benefit: the old all-in-one struct was
the only place in the code base that had a case-sensitive
naming of members 'pc' vs. 'PC'. Separating these spellings
into different branches of the flat union will allow us to add
restrictions against future case-insensitive collisions, and
make it possible for a future qemu to decide whether to enable
case-insensitive QMP.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: new patch
---
cpus.c | 31 ++++++++------
hmp.c | 30 +++++++++-----
qapi-schema.json | 120 ++++++++++++++++++++++++++++++++++++++++++++++---------
3 files changed, 139 insertions(+), 42 deletions(-)
diff --git a/cpus.c b/cpus.c
index c6a5d0e..1c8078d 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1526,22 +1526,29 @@ CpuInfoList *qmp_query_cpus(Error **errp)
info->value->qom_path = object_get_canonical_path(OBJECT(cpu));
info->value->thread_id = cpu->thread_id;
#if defined(TARGET_I386)
- info->value->has_pc = true;
- info->value->pc = env->eip + env->segs[R_CS].base;
+ info->value->arch = CPU_INFO_ARCH_X86;
+ info->value->u.x86 = g_new0(CpuInfoX86, 1);
+ info->value->u.x86->pc = env->eip + env->segs[R_CS].base;
#elif defined(TARGET_PPC)
- info->value->has_nip = true;
- info->value->nip = env->nip;
+ info->value->arch = CPU_INFO_ARCH_PPC;
+ info->value->u.ppc = g_new0(CpuInfoPpc, 1);
+ info->value->u.ppc->nip = env->nip;
#elif defined(TARGET_SPARC)
- info->value->has_pc = true;
- info->value->pc = env->pc;
- info->value->has_npc = true;
- info->value->npc = env->npc;
+ info->value->arch = CPU_INFO_ARCH_SPARC;
+ info->value->u.sparc = g_new0(CpuInfoSPARC, 1);
+ info->value->u.sparc->pc = env->pc;
+ info->value->u.sparc->npc = env->npc;
#elif defined(TARGET_MIPS)
- info->value->has_PC = true;
- info->value->PC = env->active_tc.PC;
+ info->value->arch = CPU_INFO_ARCH_MIPS;
+ info->value->u.mips = g_new0(CpuInfoMips, 1);
+ info->value->u.mips->PC = env->active_tc.PC;
#elif defined(TARGET_TRICORE)
- info->value->has_PC = true;
- info->value->PC = env->PC;
+ info->value->arch = CPU_INFO_ARCH_TRICORE;
+ info->value->u.tricore = g_new0(CpuInfoTricore, 1);
+ info->value->u.tricore->PC = env->PC;
+#else
+ info->value->arch = CPU_INFO_ARCH_OTHER;
+ info->value->u.other = g_new0(CpuInfoOther, 1);
#endif
/* XXX: waiting for the qapi to support GSList */
diff --git a/hmp.c b/hmp.c
index a15d00c..1307016 100644
--- a/hmp.c
+++ b/hmp.c
@@ -310,17 +310,25 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)
monitor_printf(mon, "%c CPU #%" PRId64 ":", active, cpu->value->CPU);
- if (cpu->value->has_pc) {
- monitor_printf(mon, " pc=0x%016" PRIx64, cpu->value->pc);
- }
- if (cpu->value->has_nip) {
- monitor_printf(mon, " nip=0x%016" PRIx64, cpu->value->nip);
- }
- if (cpu->value->has_npc) {
- monitor_printf(mon, " npc=0x%016" PRIx64, cpu->value->npc);
- }
- if (cpu->value->has_PC) {
- monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->PC);
+ switch (cpu->value->arch) {
+ case CPU_INFO_ARCH_X86:
+ monitor_printf(mon, " pc=0x%016" PRIx64, cpu->value->u.x86->pc);
+ break;
+ case CPU_INFO_ARCH_PPC:
+ monitor_printf(mon, " nip=0x%016" PRIx64, cpu->value->u.ppc->nip);
+ break;
+ case CPU_INFO_ARCH_SPARC:
+ monitor_printf(mon, " pc=0x%016" PRIx64, cpu->value->u.sparc->pc);
+ monitor_printf(mon, " npc=0x%016" PRIx64, cpu->value->u.sparc->npc);
+ break;
+ case CPU_INFO_ARCH_MIPS:
+ monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->u.mips->PC);
+ break;
+ case CPU_INFO_ARCH_TRICORE:
+ monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->u.tricore->PC);
+ break;
+ default:
+ break;
}
if (cpu->value->halted) {
diff --git a/qapi-schema.json b/qapi-schema.json
index 702b7b5..dd497ea 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -728,43 +728,125 @@
{ 'command': 'query-mice', 'returns': ['MouseInfo'] }
##
-# @CpuInfo:
+# @CpuInfoArch:
#
-# Information about a virtual CPU
+# An enumeration of cpu types that enable additional information during
+# @query-cpus.
+#
+# Since: 2.5
+##
+{ 'enum': 'CpuInfoArch',
+ 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] }
+
+##
+# @CpuInfoBase:
+#
+# Common information about a virtual CPU
#
# @CPU: the index of the virtual CPU
#
-# @current: this only exists for backwards compatible and should be ignored
+# @current: this only exists for backwards compatibility and should be ignored
#
# @halted: true if the virtual CPU is in the halt state. Halt usually refers
# to a processor specific low power mode.
#
# @qom_path: path to the CPU object in the QOM tree (since 2.4)
#
-# @pc: #optional If the target is i386 or x86_64, this is the 64-bit instruction
-# pointer.
-# If the target is Sparc, this is the PC component of the
-# instruction pointer.
-#
-# @nip: #optional If the target is PPC, the instruction pointer
-#
-# @npc: #optional If the target is Sparc, the NPC component of the instruction
-# pointer
-#
-# @PC: #optional If the target is MIPS, the instruction pointer
-#
# @thread_id: ID of the underlying host thread
#
+# @arch: architecture of the cpu, which determines which additional fields
+# will be listed (since 2.5)
+#
# Since: 0.14.0
#
# Notes: @halted is a transient state that changes frequently. By the time the
# data is sent to the client, the guest may no longer be halted.
##
-{ 'struct': 'CpuInfo',
+{ 'struct': 'CpuInfoBase',
'data': {'CPU': 'int', 'current': 'bool', 'halted': 'bool',
- 'qom_path': 'str',
- '*pc': 'int', '*nip': 'int', '*npc': 'int', '*PC': 'int',
- 'thread_id': 'int'} }
+ 'qom_path': 'str', 'thread_id': 'int', 'arch': 'CpuInfoArch' } }
+
+##
+# @CpuInfo:
+#
+# Information about a virtual CPU
+#
+# Since: 0.14.0
+##
+{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch',
+ 'data': { 'x86': 'CpuInfoX86',
+ 'sparc': 'CpuInfoSparc',
+ 'ppc': 'CpuInfoPpc',
+ 'mips': 'CpuInfoMips',
+ 'tricore': 'CpuInfoTricore',
+ 'other': 'CpuInfoOther' } }
+
+##
+# @CpuInfoX86:
+#
+# Additional information about a virtual i386 or x86_64 CPU
+#
+# @pc: the 64-bit instruction pointer
+#
+# Since 2.5
+##
+{ 'struct': 'CpuInfoX86', 'data': { 'pc': 'int' } }
+
+##
+# @CpuInfoSparc:
+#
+# Additional information about a virtual Sparc CPU
+#
+# @pc: the PC component of the instruction pointer
+#
+# @npc: the NPC component of the instruction pointer
+#
+# Since 2.5
+##
+{ 'struct': 'CpuInfoSparc', 'data': { 'pc': 'int', 'npc': 'int' } }
+
+##
+# @CpuInfoPpc:
+#
+# Additional information about a virtual PPC CPU
+#
+# @nip: the instruction pointer
+#
+# Since 2.5
+##
+{ 'struct': 'CpuInfoPpc', 'data': { 'nip': 'int' } }
+
+##
+# @CpuInfoMips:
+#
+# Additional information about a virtual MIPS CPU
+#
+# @PC: the instruction pointer
+#
+# Since 2.5
+##
+{ 'struct': 'CpuInfoMips', 'data': { 'PC': 'int' } }
+
+##
+# @CpuInfoMips:
+#
+# Additional information about a virtual Tricore CPU
+#
+# @PC: the instruction pointer
+#
+# Since 2.5
+##
+{ 'struct': 'CpuInfoTricore', 'data': { 'PC': 'int' } }
+
+##
+# @CpuInfoOther:
+#
+# No additional information is available about the virtual CPU
+#
+# Since 2.5
+#
+##
+{ 'struct': 'CpuInfoOther', 'data': { } }
##
# @query-cpus:
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH v10 30/30] qapi: Forbid case-insensitive clashes
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (28 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union Eric Blake
@ 2015-11-06 6:35 ` Eric Blake
2015-11-09 15:42 ` Markus Armbruster
2015-11-06 16:03 ` [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Markus Armbruster
30 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 6:35 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
We have toyed on list with the idea of a future extension to
QMP of teaching it to be case-insensitive (the user could
request command 'Quit' instead of 'quit', or could spell a
struct field as 'CPU' instead of 'cpu'). But for that to be
a practical extension, we cannot break backwards compatibility
with any existing struct that was already relying on case
sensitivity. Fortunately, after the previous patch cleaned
up CpuInfo, there are no such existing qapi structs.
Another benefit of enforcing a restriction against
case-insensitive is that we no longer have to worry about the
case of enum values that could be distinguished by case if
mapped by c_name(), but which cannot be distinguished when
mapped to C as ALL_CAPS by camel_to_uppper(). Having the
generator look for case collisions up front will prevent
developers from worrying whether different munging rules
for member names compared to enum values as a discriminator
will cause any problems in qapi unions.
Of course, if we never implement a case-insensitive QMP
extension, or find a legitimate reason to need qapi members
that differ only by case, we could always relax this
restriction. But it is easier to relax later than it is to
wish we had the restriction in place earlier.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v10: new patch
---
docs/qapi-code-gen.txt | 5 +++++
scripts/qapi.py | 4 ++--
tests/Makefile | 1 +
tests/qapi-schema/args-case-clash.err | 1 +
tests/qapi-schema/args-case-clash.exit | 1 +
tests/qapi-schema/args-case-clash.json | 5 +++++
tests/qapi-schema/args-case-clash.out | 0
7 files changed, 15 insertions(+), 2 deletions(-)
create mode 100644 tests/qapi-schema/args-case-clash.err
create mode 100644 tests/qapi-schema/args-case-clash.exit
create mode 100644 tests/qapi-schema/args-case-clash.json
create mode 100644 tests/qapi-schema/args-case-clash.out
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index f9fa6f3..13cec10 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -116,6 +116,11 @@ names should be ALL_CAPS with words separated by underscore. Field
names cannot start with 'has-' or 'has_', as this is reserved for
tracking optional fields.
+For now, Client JSON Protocol is case-sensitive, but future extensions
+may allow for case-insensitive recognition of command and event names,
+or of member field names. As such, the generator rejects attempts to
+create entities that only differ by case.
+
Any name (command, event, type, field, or enum value) beginning with
"x-" is marked experimental, and may be withdrawn or changed
incompatibly in a future release. Downstream vendors may add
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 3a359cb..7e7ad6e 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1039,7 +1039,7 @@ class QAPISchemaObjectTypeMember(object):
assert self.type
def check_clash(self, info, seen):
- name = c_name(self.name)
+ name = c_name(self.name).upper()
if name in seen:
raise QAPIExprError(info,
"%s collides with %s"
@@ -1087,7 +1087,7 @@ class QAPISchemaObjectTypeVariants(object):
def check(self, schema, seen):
if self.tag_name: # flat union
- self.tag_member = seen[c_name(self.tag_name)]
+ self.tag_member = seen[c_name(self.tag_name).upper()]
assert self.tag_name == self.tag_member.name
tag_type = self.tag_member.type
assert isinstance(tag_type, QAPISchemaEnumType)
diff --git a/tests/Makefile b/tests/Makefile
index c84c6cb..d1c6817 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -239,6 +239,7 @@ qapi-schema += args-alternate.json
qapi-schema += args-any.json
qapi-schema += args-array-empty.json
qapi-schema += args-array-unknown.json
+qapi-schema += args-case-clash.json
qapi-schema += args-int.json
qapi-schema += args-invalid.json
qapi-schema += args-member-array-bad.json
diff --git a/tests/qapi-schema/args-case-clash.err b/tests/qapi-schema/args-case-clash.err
new file mode 100644
index 0000000..5495ab8
--- /dev/null
+++ b/tests/qapi-schema/args-case-clash.err
@@ -0,0 +1 @@
+tests/qapi-schema/args-case-clash.json:5: 'A' (argument of oops) collides with 'a' (argument of oops)
diff --git a/tests/qapi-schema/args-case-clash.exit b/tests/qapi-schema/args-case-clash.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/args-case-clash.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/args-case-clash.json b/tests/qapi-schema/args-case-clash.json
new file mode 100644
index 0000000..55ae488
--- /dev/null
+++ b/tests/qapi-schema/args-case-clash.json
@@ -0,0 +1,5 @@
+# C member name collision
+# Reject members that clash case-insensitively (our mapping to C names
+# preserves case, but allowing these members now would prevent a future
+# relaxing of QMP to be case-insensitive).
+{ 'command': 'oops', 'data': { 'a': 'str', 'A': 'int' } }
diff --git a/tests/qapi-schema/args-case-clash.out b/tests/qapi-schema/args-case-clash.out
new file mode 100644
index 0000000..e69de29
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 04/30] qapi: Share test_init code in test-qmp-input*
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 04/30] qapi: Share test_init code in test-qmp-input* Eric Blake
@ 2015-11-06 15:17 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 15:17 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Rather than duplicate the body of two functions just to
> decide between qobject_from_jsonv() and qobject_from_json(),
> exploit the fact that qobject_from_jsonv() intentionally
> takes 'va_list *' instead of the more common 'va_list', and
> that qobject_from_json() just calls qobject_from_jsonv(,NULL).
Maximizes the common code. Relies on qobject_from_jsonv() treating a
null ap argument specially. Okay, but I would've left the
initialization of data->obj alone, to avoid the less than obvious null
argument.
Two trivial remarks inline.
> For each file, our two existing init functions then become
> thin wrappers around a new internal function, and future
> updates to initialization don't have to be duplicated.
>
> Suggested-by: Markus Armbruster <armbru@redhat.com>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: new patch
> ---
> tests/test-qmp-input-strict.c | 48 ++++++++++++++++++++---------------------
> tests/test-qmp-input-visitor.c | 49 ++++++++++++++++++++----------------------
> 2 files changed, 46 insertions(+), 51 deletions(-)
>
> diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
> index b44184f..77151de 100644
> --- a/tests/test-qmp-input-strict.c
> +++ b/tests/test-qmp-input-strict.c
> @@ -40,9 +40,27 @@ static void validate_teardown(TestInputVisitorData *data,
> }
> }
>
> -/* This is provided instead of a test setup function so that the JSON
> - string used by the tests are kept in the test functions (and not
> - int main()) */
> +/* The various test_init functions are provided instead of a test setup
> + function so that the JSON string used by the tests are kept in the test
> + functions (and not int main()). */
Since you're touching it: "not in main()".
> +static Visitor *validate_test_init_internal(TestInputVisitorData *data,
> + const char *json_string,
> + va_list *ap)
> +{
> + Visitor *v;
> +
> + data->obj = qobject_from_jsonv(json_string, ap);
> + g_assert(data->obj);
> +
> + data->qiv = qmp_input_visitor_new_strict(data->obj);
> + g_assert(data->qiv);
> +
> + v = qmp_input_get_visitor(data->qiv);
> + g_assert(v);
> +
> + return v;
> +}
> +
> static GCC_FMT_ATTR(2, 3)
> Visitor *validate_test_init(TestInputVisitorData *data,
> const char *json_string, ...)
> @@ -51,17 +69,8 @@ Visitor *validate_test_init(TestInputVisitorData *data,
> va_list ap;
>
> va_start(ap, json_string);
> - data->obj = qobject_from_jsonv(json_string, &ap);
> + v = validate_test_init_internal(data, json_string, &ap);
> va_end(ap);
> -
> - g_assert(data->obj != NULL);
> -
> - data->qiv = qmp_input_visitor_new_strict(data->obj);
> - g_assert(data->qiv != NULL);
> -
> - v = qmp_input_get_visitor(data->qiv);
> - g_assert(v != NULL);
> -
> return v;
> }
>
> @@ -75,18 +84,7 @@ Visitor *validate_test_init(TestInputVisitorData *data,
> static Visitor *validate_test_init_raw(TestInputVisitorData *data,
> const char *json_string)
> {
> - Visitor *v;
> -
> - data->obj = qobject_from_json(json_string);
> - g_assert(data->obj != NULL);
> -
> - data->qiv = qmp_input_visitor_new_strict(data->obj);
> - g_assert(data->qiv != NULL);
> -
> - v = qmp_input_get_visitor(data->qiv);
> - g_assert(v != NULL);
> -
> - return v;
> + return validate_test_init_internal(data, json_string, NULL);
> }
>
>
> diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
> index 3f6bc4d..933243d 100644
> --- a/tests/test-qmp-input-visitor.c
> +++ b/tests/test-qmp-input-visitor.c
> @@ -36,9 +36,27 @@ static void visitor_input_teardown(TestInputVisitorData *data,
> }
> }
>
> -/* This is provided instead of a test setup function so that the JSON
> - string used by the tests are kept in the test functions (and not
> - int main()) */
> +/* The various test_init functions are provided instead of a test setup
> + function so that the JSON string used by the tests are kept in the test
> + functions (and not int main()). */
Likewise.
> +static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
> + const char *json_string,
> + va_list *ap)
> +{
> + Visitor *v;
> +
> + data->obj = qobject_from_jsonv(json_string, ap);
> + g_assert(data->obj);
> +
> + data->qiv = qmp_input_visitor_new(data->obj);
> + g_assert(data->qiv);
> +
> + v = qmp_input_get_visitor(data->qiv);
> + g_assert(v);
> +
> + return v;
> +}
> +
> static GCC_FMT_ATTR(2, 3)
> Visitor *visitor_input_test_init(TestInputVisitorData *data,
> const char *json_string, ...)
> @@ -47,17 +65,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> va_list ap;
>
> va_start(ap, json_string);
> - data->obj = qobject_from_jsonv(json_string, &ap);
> + v = visitor_input_test_init_internal(data, json_string, &ap);
> va_end(ap);
> -
> - g_assert(data->obj != NULL);
> -
> - data->qiv = qmp_input_visitor_new(data->obj);
> - g_assert(data->qiv != NULL);
> -
> - v = qmp_input_get_visitor(data->qiv);
> - g_assert(v != NULL);
> -
> return v;
> }
>
> @@ -71,19 +80,7 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
> static Visitor *visitor_input_test_init_raw(TestInputVisitorData *data,
> const char *json_string)
> {
> - Visitor *v;
> -
> - data->obj = qobject_from_json(json_string);
> -
> - g_assert(data->obj != NULL);
> -
> - data->qiv = qmp_input_visitor_new(data->obj);
> - g_assert(data->qiv != NULL);
> -
> - v = qmp_input_get_visitor(data->qiv);
> - g_assert(v != NULL);
> -
> - return v;
> + return visitor_input_test_init_internal(data, json_string, NULL);
> }
>
> static void test_visitor_in_int(TestInputVisitorData *data,
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-*
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-* Eric Blake
@ 2015-11-06 15:21 ` Markus Armbruster
2015-11-06 15:49 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 15:21 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Make valgrind happy with the current state of the tests, so that
> it is easier to see if future patches introduce new memory problems
> without being drowned in noise. Many of the leaks were due to
> calling a second init without tearing down the data from an earlier
> visit. But since teardown is already idempotent, and we already
> register teardown as part of input_visitor_test_add(), it is nicer
> to just make init() safe to call multiple times than it is to have
> to make all tests call teardown.
>
> Another common leak was forgetting to clean up an error object,
> after testing that an error was raised.
>
> Another leak was in test_visitor_in_struct_nested(), failing to
> clean the base member of UserDefTwo. Cleaning that up left
> check_and_free_str() as dead code (since using the qapi_free_*
> takes care of recursion, and we don't want double frees).
>
> A final leak was in test_visitor_out_any(), which was reassigning
> the qobj local variable to a subset of the overall structure
> needing freeing; it did not result in a use-after-free, but
> was not cleaning up all the qdict.
>
> test-qmp-event and test-qmp-commands were already clean.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: improve commit message, split out refactor of test init so
> that both init and init_raw benefit from change
> v9: move earlier in series (was 13/17)
> v8: no change
> v7: no change
> v6: make init repeatable rather than adding teardown everywhere,
> fix additional leak with UserDefTwo base, plug additional files
> ---
> tests/test-qmp-input-strict.c | 9 +++++++++
> tests/test-qmp-input-visitor.c | 41 +++++++----------------------------------
> tests/test-qmp-output-visitor.c | 3 ++-
> 3 files changed, 18 insertions(+), 35 deletions(-)
>
> diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
> index 77151de..95cd8e1 100644
> --- a/tests/test-qmp-input-strict.c
> +++ b/tests/test-qmp-input-strict.c
> @@ -49,6 +49,8 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
> {
> Visitor *v;
>
> + validate_teardown(data, NULL);
> +
> data->obj = qobject_from_jsonv(json_string, ap);
> g_assert(data->obj);
>
> @@ -191,6 +193,7 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
>
> visit_type_TestStruct(v, &p, NULL, &err);
> g_assert(err);
> + error_free(err);
v9 had /* FIXME: visitor should not allocate p when returning error */
here. Did you drop it intentionally?
If you put it back, please mention it in the commit message.
> if (p) {
> g_free(p->string);
> }
[...]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing in test-qmp-*
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing " Eric Blake
@ 2015-11-06 15:36 ` Markus Armbruster
2015-11-06 15:54 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 15:36 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> By using &error_abort, we can avoid a local err variable in
> situations where we expect success. It also has the nice
> effect that if the test breaks, the error message from
> error_abort tends to be nicer than that of g_assert().
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
[Boring mechanical changes snipped...]
> diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
> index c024e5e..9f67f9e 100644
> --- a/tests/test-visitor-serialization.c
> +++ b/tests/test-visitor-serialization.c
> @@ -302,14 +302,13 @@ static void test_primitives(gconstpointer opaque)
> const SerializeOps *ops = args->ops;
> PrimitiveType *pt = args->test_data;
> PrimitiveType *pt_copy = g_malloc0(sizeof(*pt_copy));
> - Error *err = NULL;
> void *serialize_data;
>
> pt_copy->type = pt->type;
> - ops->serialize(pt, &serialize_data, visit_primitive_type, &err);
> - ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type, &err);
> + ops->serialize(pt, &serialize_data, visit_primitive_type, &error_abort);
> + ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type,
> + &error_abort);
>
> - g_assert(err == NULL);
This looks like a (very minor) bug fix / cleanup: you're not supposed to
pass the same &err to multiple functions without checking and clearing
it in between, because the second failure trips assert(*errp == NULL) in
error_setv(). Harmless here, but it's nice to get rid of a bad example.
Several more below.
> g_assert(pt_copy != NULL);
> if (pt->type == PTYPE_STRING) {
> g_assert_cmpstr(pt->value.string, ==, pt_copy->value.string);
> @@ -345,7 +344,6 @@ static void test_primitive_lists(gconstpointer opaque)
> PrimitiveList pl = { .value = { NULL } };
> PrimitiveList pl_copy = { .value = { NULL } };
> PrimitiveList *pl_copy_ptr = &pl_copy;
> - Error *err = NULL;
> void *serialize_data;
> void *cur_head = NULL;
> int i;
> @@ -492,10 +490,11 @@ static void test_primitive_lists(gconstpointer opaque)
> }
> }
>
> - ops->serialize((void **)&pl, &serialize_data, visit_primitive_list, &err);
> - ops->deserialize((void **)&pl_copy_ptr, serialize_data, visit_primitive_list, &err);
> + ops->serialize((void **)&pl, &serialize_data, visit_primitive_list,
> + &error_abort);
> + ops->deserialize((void **)&pl_copy_ptr, serialize_data,
> + visit_primitive_list, &error_abort);
>
> - g_assert(err == NULL);
> i = 0;
>
> /* compare our deserialized list of primitives to the original */
> @@ -652,10 +651,8 @@ static void test_primitive_lists(gconstpointer opaque)
> g_assert_cmpint(i, ==, 33);
>
> ops->cleanup(serialize_data);
> - dealloc_helper(&pl, visit_primitive_list, &err);
> - g_assert(!err);
> - dealloc_helper(&pl_copy, visit_primitive_list, &err);
> - g_assert(!err);
> + dealloc_helper(&pl, visit_primitive_list, &error_abort);
> + dealloc_helper(&pl_copy, visit_primitive_list, &error_abort);
> g_free(args);
> }
>
> @@ -665,13 +662,12 @@ static void test_struct(gconstpointer opaque)
> const SerializeOps *ops = args->ops;
> TestStruct *ts = struct_create();
> TestStruct *ts_copy = NULL;
> - Error *err = NULL;
> void *serialize_data;
>
> - ops->serialize(ts, &serialize_data, visit_struct, &err);
> - ops->deserialize((void **)&ts_copy, serialize_data, visit_struct, &err);
> + ops->serialize(ts, &serialize_data, visit_struct, &error_abort);
> + ops->deserialize((void **)&ts_copy, serialize_data, visit_struct,
> + &error_abort);
>
> - g_assert(err == NULL);
> struct_compare(ts, ts_copy);
>
> struct_cleanup(ts);
> @@ -687,14 +683,12 @@ static void test_nested_struct(gconstpointer opaque)
> const SerializeOps *ops = args->ops;
> UserDefTwo *udnp = nested_struct_create();
> UserDefTwo *udnp_copy = NULL;
> - Error *err = NULL;
> void *serialize_data;
>
> - ops->serialize(udnp, &serialize_data, visit_nested_struct, &err);
> + ops->serialize(udnp, &serialize_data, visit_nested_struct, &error_abort);
> ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct,
> - &err);
> + &error_abort);
>
> - g_assert(err == NULL);
> nested_struct_compare(udnp, udnp_copy);
>
> nested_struct_cleanup(udnp);
> @@ -709,7 +703,6 @@ static void test_nested_struct_list(gconstpointer opaque)
> TestArgs *args = (TestArgs *) opaque;
> const SerializeOps *ops = args->ops;
> UserDefTwoList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
> - Error *err = NULL;
> void *serialize_data;
> int i = 0;
>
> @@ -720,11 +713,10 @@ static void test_nested_struct_list(gconstpointer opaque)
> listp = tmp;
> }
>
> - ops->serialize(listp, &serialize_data, visit_nested_struct_list, &err);
> + ops->serialize(listp, &serialize_data, visit_nested_struct_list,
> + &error_abort);
> ops->deserialize((void **)&listp_copy, serialize_data,
> - visit_nested_struct_list, &err);
> -
> - g_assert(err == NULL);
> + visit_nested_struct_list, &error_abort);
>
> tmp = listp;
> tmp_copy = listp_copy;
Suggest to note the cleanup in the commit message.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup in test-qmp-*
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup " Eric Blake
@ 2015-11-06 15:40 ` Markus Armbruster
2015-11-06 15:59 ` Eric Blake
2015-11-06 17:04 ` [Qemu-devel] [PATCH] fixup! " Eric Blake
1 sibling, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 15:40 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> By moving err into data, we can let test teardown take care
> of cleaning up any collected error; it also gives us fewer
> lines of code between repeated tests where init runs teardown
> on our behalf.
I think this paragraph is no longer valid: you aren't moving err
anywhere in this version.
> Rather than duplicate code between .c files, I added a new
> test-qmp-common.h. I debated about putting
> error_free_or_abort() in error.h, but it seems like something
> that is only useful for tests.
Maybe, maybe not. I'd accept it into error.h.
> Signed-off-by: Eric Blake <eblake@redhat.com>
Patch looks okay.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-*
2015-11-06 15:21 ` Markus Armbruster
@ 2015-11-06 15:49 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 15:49 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]
On 11/06/2015 08:21 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Make valgrind happy with the current state of the tests, so that
>> it is easier to see if future patches introduce new memory problems
>> without being drowned in noise. Many of the leaks were due to
>> calling a second init without tearing down the data from an earlier
>> visit. But since teardown is already idempotent, and we already
>> register teardown as part of input_visitor_test_add(), it is nicer
>> to just make init() safe to call multiple times than it is to have
>> to make all tests call teardown.
>>
>> @@ -191,6 +193,7 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
>>
>> visit_type_TestStruct(v, &p, NULL, &err);
>> g_assert(err);
>> + error_free(err);
>
> v9 had /* FIXME: visitor should not allocate p when returning error */
> here. Did you drop it intentionally?
>
Yes, I dropped it intentionally.
> If you put it back, please mention it in the commit message.
Rather, I decided that it was sufficient to just fix the bug later
(patch already queued, but not posted), and minimize the churn (no need
to mark things up just to pull down the markings later):
http://repo.or.cz/qemu/ericb.git/commitdiff/ac584b53a
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting Eric Blake
@ 2015-11-06 15:52 ` Markus Armbruster
2015-11-09 20:56 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 15:52 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> qapi-code-gen.txt already claims that types, commands, and
> events share a common namespace; set this in stone by further
> documenting that our introspection output will never have
> collisions with the same name tied to more than one meta-type.
>
> Our largest QMP enum currently has 125 values, our largest
> object type has 27 members, and the mean for each is less than
> 10. These sizes are small enough that the per-element overhead
> of O(log n) binary searching probably outweighs the speed
> possible with direct O(n) linear searching (a better algorithm
> with more overhead will only beat a leaner naive algorithm only
> as you scale to larger input sizes).
>
> Arguably, the overall SchemaInfo array could be sorted by name;
> there, we currently have 531 entities, large enough for a binary
> search to be faster than linear. However, remember that we have
> mutually-recursive types, which means there is no topological
> ordering that will allow clients to learn all information about
> that type in a single linear pass; thus clients will want to do
> random access over the data, and they will probably read the
> introspection output into a hashtable for O(1) lookup rather
> than O(log n) binary searching, at which point, pre-sorting our
> introspection output doesn't help the client.
>
> It doesn't help that sorting can be subjective if you introduce
> locales into the mix (I'm not experienced enough with Python
> to know for sure, but at least it looks like it defaults to
> sorting in the C locale even when run under a different locale).
> And while our current introspection output is deterministic
> (because we visit entities in a sorted order), we may want
> to change that order in the future (such as using OrderedDict
> to stick to .json declaration order).
>
> For these reasons, we simply document that clients should not
> rely on any particular order of items in introspection output.
> And since it is now a documented part of the contract, we have
> the freedom to later rearrange output if needed, without
> worrying about breaking well-written clients that were already
> aware that they had to do linear searches.
Well, any kind of search that doesn't rely on sorting. Suggest to drop
the "that were already..." clause. Happy to do that on commit.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing in test-qmp-*
2015-11-06 15:36 ` Markus Armbruster
@ 2015-11-06 15:54 ` Eric Blake
2015-11-06 16:24 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 15:54 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2271 bytes --]
On 11/06/2015 08:36 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> By using &error_abort, we can avoid a local err variable in
>> situations where we expect success. It also has the nice
>> effect that if the test breaks, the error message from
>> error_abort tends to be nicer than that of g_assert().
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
> [Boring mechanical changes snipped...]
>> pt_copy->type = pt->type;
>> - ops->serialize(pt, &serialize_data, visit_primitive_type, &err);
>> - ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type, &err);
>> + ops->serialize(pt, &serialize_data, visit_primitive_type, &error_abort);
>> + ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type,
>> + &error_abort);
>>
>> - g_assert(err == NULL);
>
> This looks like a (very minor) bug fix / cleanup: you're not supposed to
> pass the same &err to multiple functions without checking and clearing
> it in between, because the second failure trips assert(*errp == NULL) in
> error_setv(). Harmless here, but it's nice to get rid of a bad example.
Harmless here because we are asserting that err is still NULL after the
chain (which means it was NULL at all points during the chain). But I
agree that it is nice to get rid of poor practice, and that adding a
paragraph to the commit message to point it out would be a nice idea.
>
> Suggest to note the cleanup in the commit message.
We may be close enough to take the series without needing a v11; if
that's the case, and you are the one squashing in the change, how about
this text:
This patch has an additional bonus of fixing several call sites that
were passing &err to two different functions without checking it in
between. In general that is unsafe practice; because if the first
function sets an error, the second function could abort() if it tries to
set a different error. We got away with it because we were asserting
that err was NULL through the entire chain, but switching to
&error_abort avoids the questionable practice up front.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup in test-qmp-*
2015-11-06 15:40 ` Markus Armbruster
@ 2015-11-06 15:59 ` Eric Blake
2015-11-06 16:23 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 15:59 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1610 bytes --]
On 11/06/2015 08:40 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> By moving err into data, we can let test teardown take care
>> of cleaning up any collected error; it also gives us fewer
>> lines of code between repeated tests where init runs teardown
>> on our behalf.
>
> I think this paragraph is no longer valid: you aren't moving err
> anywhere in this version.
D'oh. I scrubbed the code, but not the commit message. At least it was
a faithful split of the v9 version of the commit, before I reworked the
mechanism :)
How about:
We have several tests that perform multiple sub-actions that are
expected to fail. Asserting that an error occurred, then clearing it up
to prepare for the next action, turned into enough boilerplate that it
was sometimes forgotten, risking memory leak or invalidating the efforts
of the second action (passing a non-NULL err into a function is
generally a bad idea). Encapsulate the boilerplate into a single helper
function error_free_or_abort(), and consistently use it.
>
>> Rather than duplicate code between .c files, I added a new
>> test-qmp-common.h. I debated about putting
>> error_free_or_abort() in error.h, but it seems like something
>> that is only useful for tests.
>
> Maybe, maybe not. I'd accept it into error.h.
Do you want me to post a fixup patch that relocates it into error.h?
>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> Patch looks okay.
>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
` (29 preceding siblings ...)
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 30/30] qapi: Forbid case-insensitive clashes Eric Blake
@ 2015-11-06 16:03 ` Markus Armbruster
2015-11-06 16:08 ` Eric Blake
30 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 16:03 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel
Eric Blake <eblake@redhat.com> writes:
> No pending prerequisites; based on qemu.git master
>
> Also available as a tag at this location:
> git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv9c
>
> and will soon be part of my branch with the rest of the v5 series, at:
> http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi
>
> v10 notes:
> Split several patches, redo the middle patches from Markus to be
> back in the order they were first posted, some fallout change to
> my patches due to the nicer pattern of minimizing conditionals
> inside .check(), by instead calling .check_clash() as needed.
> Change data->err magic in tests to instead use a new helper
> error_free_or_abort(). Add a patch that would prevent qapi
> case-insensitive clashes.
>
> I am redoing my subset boundaries slightly: patches 23-27 of
> v9 (updating the alternate layout) will be delayed to subset D,
> and 2 other patches previously posted in subset D are now here
> (turning qapi clash checking into actual error messages), so
> the subject line of this cover letter is slightly different.
>
> Hopefully, we are converging on something that will be ready
> for a pull request, especially for the earlier patches of this
> subset.
I guess you mean PATCH 01-12. I had a few questions, but the most
likely outcome seems to be minor touchups I could apply in my tree.
I'm okay with trying to get more patches in, but let's get these out of
the way meanwhile.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-06 16:03 ` [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Markus Armbruster
@ 2015-11-06 16:08 ` Eric Blake
2015-11-09 9:59 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-06 16:08 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel
[-- Attachment #1: Type: text/plain, Size: 2018 bytes --]
On 11/06/2015 09:03 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> No pending prerequisites; based on qemu.git master
>>
>> Also available as a tag at this location:
>> git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv9c
Make that the qapi-cleanupv10c label.
>>
>> and will soon be part of my branch with the rest of the v5 series, at:
>> http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi
>>
>> v10 notes:
>> Split several patches, redo the middle patches from Markus to be
>> back in the order they were first posted, some fallout change to
>> my patches due to the nicer pattern of minimizing conditionals
>> inside .check(), by instead calling .check_clash() as needed.
>> Change data->err magic in tests to instead use a new helper
>> error_free_or_abort(). Add a patch that would prevent qapi
>> case-insensitive clashes.
>>
>> I am redoing my subset boundaries slightly: patches 23-27 of
>> v9 (updating the alternate layout) will be delayed to subset D,
>> and 2 other patches previously posted in subset D are now here
>> (turning qapi clash checking into actual error messages), so
>> the subject line of this cover letter is slightly different.
>>
>> Hopefully, we are converging on something that will be ready
>> for a pull request, especially for the earlier patches of this
>> subset.
>
> I guess you mean PATCH 01-12. I had a few questions, but the most
> likely outcome seems to be minor touchups I could apply in my tree.
>
> I'm okay with trying to get more patches in, but let's get these out of
> the way meanwhile.
Yes, 01-12 seems like a good first set, if you want to make those
touchups (I've supplied some potential text improvements in reply to
some of your comments); I'm happy, as always, to take a peek over your
staging repo to double check what you are prepping for the pull request.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup in test-qmp-*
2015-11-06 15:59 ` Eric Blake
@ 2015-11-06 16:23 ` Markus Armbruster
2015-11-06 16:32 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 16:23 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/06/2015 08:40 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> By moving err into data, we can let test teardown take care
>>> of cleaning up any collected error; it also gives us fewer
>>> lines of code between repeated tests where init runs teardown
>>> on our behalf.
>>
>> I think this paragraph is no longer valid: you aren't moving err
>> anywhere in this version.
>
> D'oh. I scrubbed the code, but not the commit message. At least it was
> a faithful split of the v9 version of the commit, before I reworked the
> mechanism :)
>
> How about:
>
> We have several tests that perform multiple sub-actions that are
> expected to fail. Asserting that an error occurred, then clearing it up
> to prepare for the next action, turned into enough boilerplate that it
> was sometimes forgotten,
If you got suitable commit SHAs handy, lets insert some here.
> risking memory leak or invalidating the efforts
> of the second action (passing a non-NULL err into a function is
> generally a bad idea). Encapsulate the boilerplate into a single helper
> function error_free_or_abort(), and consistently use it.
Works for me.
>>> Rather than duplicate code between .c files, I added a new
>>> test-qmp-common.h. I debated about putting
>>> error_free_or_abort() in error.h, but it seems like something
>>> that is only useful for tests.
>>
>> Maybe, maybe not. I'd accept it into error.h.
>
> Do you want me to post a fixup patch that relocates it into error.h?
I guess the result would be slightly simpler, so go ahead.
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> Patch looks okay.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing in test-qmp-*
2015-11-06 15:54 ` Eric Blake
@ 2015-11-06 16:24 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-06 16:24 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/06/2015 08:36 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> By using &error_abort, we can avoid a local err variable in
>>> situations where we expect success. It also has the nice
>>> effect that if the test breaks, the error message from
>>> error_abort tends to be nicer than that of g_assert().
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>> [Boring mechanical changes snipped...]
>
>>> pt_copy->type = pt->type;
>>> - ops->serialize(pt, &serialize_data, visit_primitive_type, &err);
>>> - ops->deserialize((void **)&pt_copy, serialize_data,
>>> visit_primitive_type, &err);
>>> + ops->serialize(pt, &serialize_data, visit_primitive_type, &error_abort);
>>> + ops->deserialize((void **)&pt_copy, serialize_data,
>>> visit_primitive_type,
>>> + &error_abort);
>>>
>>> - g_assert(err == NULL);
>>
>> This looks like a (very minor) bug fix / cleanup: you're not supposed to
>> pass the same &err to multiple functions without checking and clearing
>> it in between, because the second failure trips assert(*errp == NULL) in
>> error_setv(). Harmless here, but it's nice to get rid of a bad example.
>
> Harmless here because we are asserting that err is still NULL after the
> chain (which means it was NULL at all points during the chain). But I
> agree that it is nice to get rid of poor practice, and that adding a
> paragraph to the commit message to point it out would be a nice idea.
>
>>
>> Suggest to note the cleanup in the commit message.
>
> We may be close enough to take the series without needing a v11; if
> that's the case, and you are the one squashing in the change, how about
> this text:
>
> This patch has an additional bonus of fixing several call sites that
> were passing &err to two different functions without checking it in
> between. In general that is unsafe practice; because if the first
> function sets an error, the second function could abort() if it tries to
> set a different error. We got away with it because we were asserting
> that err was NULL through the entire chain, but switching to
> &error_abort avoids the questionable practice up front.
Works for me.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup in test-qmp-*
2015-11-06 16:23 ` Markus Armbruster
@ 2015-11-06 16:32 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 16:32 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]
On 11/06/2015 09:23 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 11/06/2015 08:40 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> By moving err into data, we can let test teardown take care
>>>> of cleaning up any collected error; it also gives us fewer
>>>> lines of code between repeated tests where init runs teardown
>>>> on our behalf.
>>>
>>> I think this paragraph is no longer valid: you aren't moving err
>>> anywhere in this version.
>>
>> D'oh. I scrubbed the code, but not the commit message. At least it was
>> a faithful split of the v9 version of the commit, before I reworked the
>> mechanism :)
>>
>> How about:
>>
>> We have several tests that perform multiple sub-actions that are
>> expected to fail. Asserting that an error occurred, then clearing it up
>> to prepare for the next action, turned into enough boilerplate that it
>> was sometimes forgotten,
>
> If you got suitable commit SHAs handy, lets insert some here.
A number of them were introduced in d88f5fd (look at
test-qmp-input-visitor.c:test_validate_fail_struct_nested(), for
example), and only barely cleaned up in 5/30 of this series. I'm not
finding other commits off-hand, though.
>
>> risking memory leak or invalidating the efforts
>> of the second action (passing a non-NULL err into a function is
>> generally a bad idea). Encapsulate the boilerplate into a single helper
>> function error_free_or_abort(), and consistently use it.
>
> Works for me.
>
>>>> Rather than duplicate code between .c files, I added a new
>>>> test-qmp-common.h. I debated about putting
>>>> error_free_or_abort() in error.h, but it seems like something
>>>> that is only useful for tests.
>>>
>>> Maybe, maybe not. I'd accept it into error.h.
>>
>> Do you want me to post a fixup patch that relocates it into error.h?
>
> I guess the result would be slightly simpler, so go ahead.
Coming up soon.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* [Qemu-devel] [PATCH] fixup! qapi: Simplify error cleanup in test-qmp-*
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup " Eric Blake
2015-11-06 15:40 ` Markus Armbruster
@ 2015-11-06 17:04 ` Eric Blake
1 sibling, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-06 17:04 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
[As part of the fixup, replace the old commit message with this]
We have several tests that perform multiple sub-actions that are
expected to fail. Asserting that an error occurred, then clearing
it up to prepare for the next action, turned into enough
boilerplate that it was sometimes forgotten (for example, a number
of tests added to test-qmp-input-visitor.c in d88f5fd leaked err).
Worse, if an error is not reset to NULL, we risk invalidating
later use of that error (passing a non-NULL err into a function
is generally a bad idea). Encapsulate the boilerplate into a
single helper function error_free_or_abort(), and consistently
use it.
The new function is added into error.c for use everywhere,
although it is anticipated that testsuites will be the main
client.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
Drop test-qmp-common.h, and move to error.h instead
---
include/qapi/error.h | 9 +++++++++
tests/test-qmp-commands.c | 5 ++---
tests/test-qmp-common.h | 22 ----------------------
tests/test-qmp-event.c | 1 -
tests/test-qmp-input-strict.c | 1 -
tests/test-qmp-input-visitor.c | 1 -
util/error.c | 7 +++++++
7 files changed, 18 insertions(+), 28 deletions(-)
delete mode 100644 tests/test-qmp-common.h
diff --git a/include/qapi/error.h b/include/qapi/error.h
index c69dddb..faad466 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -30,6 +30,10 @@
* Handle an error without reporting it (just for completeness):
* error_free(err);
*
+ * Assert than an expected error occurred, but clean it up without
+ * reporting it (primarily useful in testsuites):
+ * error_free_or_abort(&err);
+ *
* Pass an existing error to the caller:
* error_propagate(errp, err);
* where Error **errp is a parameter, by convention the last one.
@@ -190,6 +194,11 @@ Error *error_copy(const Error *err);
void error_free(Error *err);
/*
+ * Convenience function to assert that *@errp is set, then silently free it.
+ */
+void error_free_or_abort(Error **errp);
+
+/*
* Convenience function to error_report() and free @err.
*/
void error_report_err(Error *);
diff --git a/tests/test-qmp-commands.c b/tests/test-qmp-commands.c
index 9f65fc2..888fb5f 100644
--- a/tests/test-qmp-commands.c
+++ b/tests/test-qmp-commands.c
@@ -1,13 +1,12 @@
#include <glib.h>
#include "qemu-common.h"
-#include "test-qmp-common.h"
#include "qapi/qmp/types.h"
#include "test-qmp-commands.h"
#include "qapi/qmp/dispatch.h"
#include "qemu/module.h"
#include "qapi/qmp-input-visitor.h"
-#include "test-qapi-types.h"
-#include "test-qapi-visit.h"
+#include "tests/test-qapi-types.h"
+#include "tests/test-qapi-visit.h"
void qmp_user_def_cmd(Error **errp)
{
diff --git a/tests/test-qmp-common.h b/tests/test-qmp-common.h
deleted file mode 100644
index 043f49c..0000000
--- a/tests/test-qmp-common.h
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Code common to qmp/qapi unit-tests.
- *
- * Copyright (C) 2015 Red Hat, Inc.
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-
-#ifndef TEST_QMP_COMMON_H__
-#define TEST_QMP_COMMON_H__
-
-/* Expect an error, abort() if there is none. */
-static inline void error_free_or_abort(Error **errp)
-{
- g_assert(*errp);
- error_free(*errp);
- *errp = NULL;
-}
-
-#endif
diff --git a/tests/test-qmp-event.c b/tests/test-qmp-event.c
index c0ac3ea..035c65c 100644
--- a/tests/test-qmp-event.c
+++ b/tests/test-qmp-event.c
@@ -22,7 +22,6 @@
#include "qapi/qmp/qint.h"
#include "qapi/qmp/qobject.h"
#include "qapi/qmp-event.h"
-#include "test-qmp-common.h"
typedef struct TestEventData {
QDict *expect;
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 7e15b09..201221c 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -15,7 +15,6 @@
#include <stdarg.h>
#include "qemu-common.h"
-#include "test-qmp-common.h"
#include "qapi/qmp-input-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 2b59b6f..ff4e7c9 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -14,7 +14,6 @@
#include <stdarg.h>
#include "qemu-common.h"
-#include "test-qmp-common.h"
#include "qapi/qmp-input-visitor.h"
#include "test-qapi-types.h"
#include "test-qapi-visit.h"
diff --git a/util/error.c b/util/error.c
index 8b86490..80c89a2 100644
--- a/util/error.c
+++ b/util/error.c
@@ -220,6 +220,13 @@ void error_free(Error *err)
}
}
+void error_free_or_abort(Error **errp)
+{
+ assert(errp && *errp);
+ error_free(*errp);
+ *errp = NULL;
+}
+
void error_propagate(Error **dst_errp, Error *local_err)
{
if (!local_err) {
--
2.4.3
^ permalink raw reply related [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-06 16:08 ` Eric Blake
@ 2015-11-09 9:59 ` Markus Armbruster
2015-11-09 14:43 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 9:59 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel
Eric Blake <eblake@redhat.com> writes:
> On 11/06/2015 09:03 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
[...]
>>> Hopefully, we are converging on something that will be ready
>>> for a pull request, especially for the earlier patches of this
>>> subset.
>>
>> I guess you mean PATCH 01-12. I had a few questions, but the most
>> likely outcome seems to be minor touchups I could apply in my tree.
>>
>> I'm okay with trying to get more patches in, but let's get these out of
>> the way meanwhile.
>
> Yes, 01-12 seems like a good first set, if you want to make those
> touchups (I've supplied some potential text improvements in reply to
> some of your comments); I'm happy, as always, to take a peek over your
> staging repo to double check what you are prepping for the pull request.
Please have a look at my qapi-next branch.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check()
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check() Eric Blake
@ 2015-11-09 12:31 ` Markus Armbruster
2015-11-09 14:44 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 12:31 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> From: Markus Armbruster <armbru@redhat.com>
>
> QAPISchemaObjectTypeMember.check() currently does four things:
>
> 1. Compute self.type
>
> 2. Accumulate members in all_members
>
> Only one caller cares: QAPISchemaObjectType.check() uses it to
> compute self.members. The other callers pass a throw-away
> accumulator.
>
> 3. Accumulate a map from names to members in seen
>
> Only one caller cares: QAPISchemaObjectType.check() uses it to
> compute its local variable seen, for self.variants.check(), which
> uses it to compute self.variants.tag_member from
> self.variants.tag_name. The other callers pass a throw-away
> accumulator.
>
> 4. Check for collisions
>
> This piggyback on 3: before adding a new entry, we assert it's new.
piggybacks (typo is mine)
>
> Only one caller cares: QAPISchemaObjectType.check() uses it to
> assert non-variant members don't clash.
>
> Simplify QAPISchemaObjectType.check(): move 2.-4. to
> QAPISchemaObjectType.check(), and drop parameters all_members and
> seen.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Message-Id: <1446559499-26984-2-git-send-email-armbru@redhat.com>
> [rebase to earlier changes that moved tag_member.check() of
> alternate types]
> Signed-off-by: Eric Blake <eblake@redhat.com>
Patch is as good as ever ;)
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check()
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check() Eric Blake
@ 2015-11-09 12:38 ` Markus Armbruster
2015-11-10 5:04 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 12:38 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> From: Markus Armbruster <armbru@redhat.com>
>
> Reduce the ugly flat union / simple union conditional by doing just
> the essential work here, namely setting self.tag_member.
> Move the rest to callers.
>
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> Message-Id: <1446559499-26984-7-git-send-email-armbru@redhat.com>
> [rebase to earlier changes that moved tag_member.check() of
> alternate types, and tweak commit title and wording]
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: redo closer to Markus' original proposal
> v9: new patch
> ---
> scripts/qapi.py | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 2a73b2b..e057408 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -988,9 +988,10 @@ class QAPISchemaObjectType(QAPISchemaType):
> for m in self.local_members:
> m.check(schema)
> m.check_clash(seen)
> + self.members = seen.values()
> if self.variants:
> self.variants.check(schema, seen)
> - self.members = seen.values()
> + assert self.variants.tag_member in self.members
>
> def is_implicit(self):
> # See QAPISchema._make_implicit_object_type()
> @@ -1052,8 +1053,6 @@ class QAPISchemaObjectTypeVariants(object):
> def check(self, schema, seen):
> if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
My patch has:
- if self.tag_name: # flat union
+ if not self.tag_member: # flat union
self.tag_member = seen[self.tag_name]
Any particular reason for dropping it?
I like my change, because I feel it makes the assignment's purpose more
obvious: ensure .tag_member is set.
> - if seen:
> - assert self.tag_member in seen.itervalues()
> assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> for v in self.variants:
> v.check(schema, self.tag_member.type)
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches Eric Blake
@ 2015-11-09 12:56 ` Markus Armbruster
2015-11-09 15:13 ` Markus Armbruster
2015-11-10 5:16 ` Eric Blake
0 siblings, 2 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 12:56 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Right now, our ad hoc parser ensures that we cannot have a
> flat union that introduces any qapi member names that would
> conflict with the non-variant qapi members already present
> from the union's base type (see flat-union-clash-member.json).
> We want QAPISchemaObjectType.check() to make the same check,
> so we can later reduce some of the ad hoc checks.
>
> We already ensure that all branches of a flat union are qapi
> structs with no variants, at which point those members appear
> in the same JSON object as all non-variant members. And we
> already have a map 'seen' of all non-variant members. All
> we need is a new QAPISchemaObjectTypeVariants.check_clash(),
> which clones the seen map then checks for clashes with each
> member of the variant's qapi type.
>
> Note that the clone of seen inside Variants.check_clash()
> resembles the one we just removed from Variants.check(); the
> difference here is that we are now checking for clashes
> among the qapi members of the variant type, rather than for
> a single clash with the variant tag name itself.
>
> In general, a type used as a branch of a flat union cannot
> also be the base type of the flat union, so even though we are
> adding a call to variant.type.check() in order to populate
> variant.type.members, this is merely a case of gaining
> topological sorting of how types are visited (and type.check()
> is already set up to allow multiple calls due to base types).
Yes, a type cannot contain itself, neither as base nor as variant.
We have tests covering attempts to do the former
(struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
see, we don't have tests covering the latter. Do we catch it?
> For simple unions, the same code happens to work by design,
> because of our synthesized wrapper classes (however, the
> wrapper has a single member 'data' which will never collide
> with the one non-variant member 'type', so it doesn't really
> matter).
>
> There is no impact to alternates, which intentionally do not
> need to call variants.check_clash() (there, at most one of
> the variant's branches will be an ObjectType, and even if one
> exists, we are not inlining the qapi members of that object
> into a parent object, the way we do for unions).
>
> No change to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Patch looks good.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash() Eric Blake
@ 2015-11-09 13:00 ` Markus Armbruster
2015-11-09 17:36 ` Eric Blake
2015-11-09 14:49 ` Markus Armbruster
1 sibling, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 13:00 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Consolidate two common sequences of clash detection into a
> new QAPISchemaObjectType.check_clash() helper method.
>
> No change to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: rebase on new Variants.check_clash()
> v9: new patch, split off from v8 7/17
> ---
> scripts/qapi.py | 19 ++++++++-----------
> 1 file changed, 8 insertions(+), 11 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 4c56935..6d8c4c7 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
> seen = OrderedDict()
> if self._base_name:
> self.base = schema.lookup_type(self._base_name)
> - assert isinstance(self.base, QAPISchemaObjectType)
This assertion is lost.
> - assert not self.base.variants # not implemented
> - self.base.check(schema)
> - for m in self.base.members:
> - m.check_clash(seen)
> + self.base.check_clash(schema, seen)
> for m in self.local_members:
> m.check(schema)
> m.check_clash(seen)
> @@ -994,6 +990,12 @@ class QAPISchemaObjectType(QAPISchemaType):
> assert self.variants.tag_member in self.members
> self.variants.check_clash(schema, seen)
>
> + def check_clash(self, schema, seen):
> + self.check(schema)
> + assert not self.variants # not implemented
> + for m in self.members:
> + m.check_clash(seen)
> +
> def is_implicit(self):
> # See QAPISchema._make_implicit_object_type()
> return self.name[0] == ':'
> @@ -1062,12 +1064,7 @@ class QAPISchemaObjectTypeVariants(object):
> for v in self.variants:
> # Reset seen map for each variant, since qapi names from one
> # branch do not affect another branch
> - vseen = dict(seen)
> - assert isinstance(v.type, QAPISchemaObjectType)
This assertion is lost.
> - assert not v.type.variants # not implemented
> - v.type.check(schema)
> - for m in v.type.members:
> - m.check_clash(vseen)
> + v.type.check_clash(schema, dict(seen))
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check()
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check() Eric Blake
@ 2015-11-09 13:07 ` Markus Armbruster
2015-11-10 5:33 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 13:07 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Checking that a given QAPISchemaObjectTypeVariant.name is a
> member of the corresponding QAPISchemaEnumType of the owning
> QAPISchemaObjectTypeVariants.tag_member ensures that there are
> no collisions in the generated C union for those tag values
> (since the enum itself should have no collisions).
>
> However, this check was the only thing that Variant.check() was
> doing beyond the work of the superclass ObjectTypeMember.check(),
> and resulted in a difference of the .check() signatures just to
> pass the enum type down.
>
> Simplify things by instead doing the tag name check as part of
> Variants.check(), at which point we can rely on inheritance
> instead of overriding Variant.check().
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: new patch
> ---
> scripts/qapi.py | 10 ++++------
> 1 file changed, 4 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 6d8c4c7..798df51 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -1056,9 +1056,11 @@ class QAPISchemaObjectTypeVariants(object):
> def check(self, schema, seen):
> if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
> - assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> + tag_type = self.tag_member.type
> + assert isinstance(tag_type, QAPISchemaEnumType)
> for v in self.variants:
> - v.check(schema, self.tag_member.type)
> + v.check(schema)
> + assert v.name in tag_type.values
Two changes squashed together:
* Move the assertion from QAPISchemaObjectTypeVariant.check(),
* Capture self.tag_member.type in tag_type
The second part makes the patch slightly less obvious. Matter of taste.
>
> def check_clash(self, schema, seen):
> for v in self.variants:
> @@ -1071,10 +1073,6 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> def __init__(self, name, typ):
> QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
>
> - def check(self, schema, tag_type):
> - QAPISchemaObjectTypeMember.check(self, schema)
> - assert self.name in tag_type.values
> -
> # This function exists to support ugly simple union special cases
> # TODO get rid of them, and drop the function
> def simple_union_type(self):
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member Eric Blake
@ 2015-11-09 14:26 ` Markus Armbruster
2015-11-11 0:17 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 14:26 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Future commits will migrate semantic checking away from parsing
> and over to the various QAPISchema*.check() methods. But to
> report an error message about an incorrect semantic use of a
> member of an object type, it helps to know which type, command,
> or event owns the member. In particular, when a member is
> inherited from a base type, it is desirable to associate the
> member name with the base type (and not the type calling
> member.check()).
>
> Rather than packing additional information into the seen array
> passed to each member.check() (as in seen[m.name] = {'member':m,
> 'owner':type}), it is easier to have each member track the name
> of the owner type in the first place (keeping things simpler
> with the existing seen[m.name] = m). The new member.owner field
> is set via a new set_owner() method, called when registering
> the members and variants arrays with an object or variant type.
> Track only a name, and not the actual type object, to avoid
> creating a circular python reference chain.
>
> Note that Variants.set_owner() method does not set the owner
> for the tag_member field; this field is set earlier either as
> part of an object's non-variant members, or explicitly by
> alternates.
>
> The source information is intended for human consumption in
> error messages, and a new describe() method is added to access
> the resulting information. For example, given the qapi:
> { 'command': 'foo', 'data': { 'string': 'str' } }
> an implementation of visit_command() that calls
> arg_type.members[0].describe()
> will see "'string' (argument of foo)".
>
> To make the human-readable name of implicit types work without
> duplicating efforts, the describe() method has to reverse the
> name of implicit types, via the helper _pretty_owner(), plus a
> tweak to report event data separately from command arguments.
>
> No change to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10 (now in subset C): rebase to latest, set alternate tag_member
> owner from alternate
> v9 (now in subset D): rebase to earlier changes, hoist 'role' to top
> of class, split out _pretty_helper(), manage owner when tag_member
> appears as part of local_members for unions
> v8: don't munge implicit type names [except for event data], and
> instead make describe() create nicer messages. Add set_owner(), and
> use field 'role' instead of method _describe()
> v7: total rewrite: rework implicit object names, assign owner
> when initializing owner type rather than when creating member
> python object
> v6: rebase on new lazy array creation and simple union 'type'
> motion; tweak commit message
> ---
> scripts/qapi.py | 44 +++++++++++++++++++++++++++++++---
> tests/qapi-schema/qapi-schema-test.out | 8 +++----
> 2 files changed, 45 insertions(+), 7 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index b914785..b3af973 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -957,8 +957,10 @@ class QAPISchemaObjectType(QAPISchemaType):
> assert base is None or isinstance(base, str)
> for m in local_members:
> assert isinstance(m, QAPISchemaObjectTypeMember)
> - assert (variants is None or
> - isinstance(variants, QAPISchemaObjectTypeVariants))
> + m.set_owner(name)
> + if variants is not None:
> + assert isinstance(variants, QAPISchemaObjectTypeVariants)
> + variants.set_owner(name)
> self._base_name = base
> self.base = None
> self.local_members = local_members
> @@ -1012,6 +1014,8 @@ class QAPISchemaObjectType(QAPISchemaType):
>
>
> class QAPISchemaObjectTypeMember(object):
> + role = 'member'
> +
> def __init__(self, name, typ, optional):
> assert isinstance(name, str)
> assert isinstance(typ, str)
> @@ -1020,8 +1024,14 @@ class QAPISchemaObjectTypeMember(object):
> self._type_name = typ
> self.type = None
> self.optional = optional
> + self.owner = None
> +
> + def set_owner(self, name):
> + assert not self.owner
> + self.owner = name
>
> def check(self, schema):
> + assert self.owner
> self.type = schema.lookup_type(self._type_name)
> assert self.type
>
> @@ -1030,6 +1040,25 @@ class QAPISchemaObjectTypeMember(object):
> assert self.name not in seen
> seen[self.name] = self
>
> + def _pretty_owner(self):
> + # See QAPISchema._make_implicit_object_type() - reverse the
> + # mapping there to create a nice human-readable description
This comment confused me briefly. It applies to the following
conditionals if-part, but not its else-part. Move it into the if-part?
> + owner = self.owner
> + if owner.startswith(':obj-'):
> + owner = owner[5:]
> + if owner.endswith('-arg'):
> + return '(argument of %s)' % owner[:-4]
> + elif owner.endswith('-data'):
> + return '(data of %s)' % owner[:-5]
> + else:
> + assert owner.endswith('-wrapper')
> + return '(branch of %s)' % owner[:-8]
See last remark.
> + else:
> + return '(%s of %s)' % (self.role, owner)
> +
> + def describe(self):
> + return "'%s' %s" % (self.name, self._pretty_owner())
> +
>
> class QAPISchemaObjectTypeVariants(object):
> def __init__(self, tag_name, tag_member, variants):
> @@ -1046,6 +1075,10 @@ class QAPISchemaObjectTypeVariants(object):
> self.tag_member = tag_member
> self.variants = variants
>
> + def set_owner(self, name):
> + for v in self.variants:
> + v.set_owner(name)
> +
> def check(self, schema, seen):
> if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
> @@ -1063,6 +1096,8 @@ class QAPISchemaObjectTypeVariants(object):
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> + role = 'branch'
> +
> def __init__(self, name, typ):
> QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
>
> @@ -1082,9 +1117,11 @@ class QAPISchemaAlternateType(QAPISchemaType):
> QAPISchemaType.__init__(self, name, info)
> assert isinstance(variants, QAPISchemaObjectTypeVariants)
> assert not variants.tag_name
> + variants.set_owner(name)
> self.variants = variants
>
> def check(self, schema):
> + self.variants.tag_member.set_owner(self.name)
> self.variants.tag_member.check(schema)
> self.variants.check(schema, {})
>
Odd: all other .set_owner() calls are in .__init__() or .set_owner().
Can we move this one to __init__() for consistency?
I think we can: __init__() requires its variants argument to have a
tag_member (it even asserts not variants.tag_name).
> @@ -1212,6 +1249,7 @@ class QAPISchema(object):
> def _make_implicit_object_type(self, name, info, role, members):
> if not members:
> return None
> + # See also QAPISchemaObjectTypeMember.describe()
Should this point to ._pretty_owner() instead?
> name = ':obj-%s-%s' % (name, role)
> if not self.lookup_entity(name, QAPISchemaObjectType):
> self._def_entity(QAPISchemaObjectType(name, info, None,
> @@ -1315,7 +1353,7 @@ class QAPISchema(object):
> data = expr.get('data')
> if isinstance(data, OrderedDict):
> data = self._make_implicit_object_type(
> - name, info, 'arg', self._make_members(data, info))
> + name, info, 'data', self._make_members(data, info))
This is necessary only to make .pretty_owner() say 'data of EVENT_NAME'
instead of 'argument of EVENT_NAME'. Do we really need different
wording for commands and events?
I'd make it say 'parameter of' for both commands and events. I
habitually use "parameter" for formal parameters, and "argument" for
actual arguments. Guess I've worked with the C standard too much...
For what it's worth, the QAPI schema language uses 'data' for both (and
many other things, too), and the introspection schema uses 'arg-type'
for both.
> self._def_entity(QAPISchemaEvent(name, info, data))
>
> def _def_exprs(self):
> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> index 786024e..f78ef04 100644
> --- a/tests/qapi-schema/qapi-schema-test.out
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -1,9 +1,9 @@
> object :empty
> -object :obj-EVENT_C-arg
> +object :obj-EVENT_C-data
> member a: int optional=True
> member b: UserDefOne optional=True
> member c: str optional=False
> -object :obj-EVENT_D-arg
> +object :obj-EVENT_D-data
> member a: EventStructOne optional=False
> member b: str optional=False
> member c: str optional=True
> @@ -79,8 +79,8 @@ alternate AltStrNum
> enum AltStrNumKind ['s', 'n']
> event EVENT_A None
> event EVENT_B None
> -event EVENT_C :obj-EVENT_C-arg
> -event EVENT_D :obj-EVENT_D-arg
> +event EVENT_C :obj-EVENT_C-data
> +event EVENT_D :obj-EVENT_D-data
> object Empty1
> object Empty2
> base Empty1
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-09 9:59 ` Markus Armbruster
@ 2015-11-09 14:43 ` Eric Blake
2015-11-09 18:42 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-09 14:43 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1192 bytes --]
On 11/09/2015 02:59 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 11/06/2015 09:03 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
> [...]
>>>> Hopefully, we are converging on something that will be ready
>>>> for a pull request, especially for the earlier patches of this
>>>> subset.
>>>
>>> I guess you mean PATCH 01-12. I had a few questions, but the most
>>> likely outcome seems to be minor touchups I could apply in my tree.
>>>
>>> I'm okay with trying to get more patches in, but let's get these out of
>>> the way meanwhile.
>>
>> Yes, 01-12 seems like a good first set, if you want to make those
>> touchups (I've supplied some potential text improvements in reply to
>> some of your comments); I'm happy, as always, to take a peek over your
>> staging repo to double check what you are prepping for the pull request.
>
> Please have a look at my qapi-next branch.
Close, but commit 5f72bb85 on that branch is missing the one-line summary:
qapi: Simplify error cleanup in test-qmp-*
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check()
2015-11-09 12:31 ` Markus Armbruster
@ 2015-11-09 14:44 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-09 14:44 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 601 bytes --]
On 11/09/2015 05:31 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> From: Markus Armbruster <armbru@redhat.com>
>>
>> QAPISchemaObjectTypeMember.check() currently does four things:
>>
>> 4. Check for collisions
>>
>> This piggyback on 3: before adding a new entry, we assert it's new.
>
> piggybacks (typo is mine)
What's more, I noticed it in v9, but in redoing v10 to go closer to your
original series, I reverted my touchup. Oh well :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash() Eric Blake
2015-11-09 13:00 ` Markus Armbruster
@ 2015-11-09 14:49 ` Markus Armbruster
2015-11-10 5:32 ` Eric Blake
1 sibling, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 14:49 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Consolidate two common sequences of clash detection into a
> new QAPISchemaObjectType.check_clash() helper method.
>
> No change to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: rebase on new Variants.check_clash()
> v9: new patch, split off from v8 7/17
> ---
> scripts/qapi.py | 19 ++++++++-----------
> 1 file changed, 8 insertions(+), 11 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 4c56935..6d8c4c7 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
> seen = OrderedDict()
> if self._base_name:
> self.base = schema.lookup_type(self._base_name)
> - assert isinstance(self.base, QAPISchemaObjectType)
> - assert not self.base.variants # not implemented
> - self.base.check(schema)
> - for m in self.base.members:
> - m.check_clash(seen)
> + self.base.check_clash(schema, seen)
> for m in self.local_members:
> m.check(schema)
> m.check_clash(seen)
> @@ -994,6 +990,12 @@ class QAPISchemaObjectType(QAPISchemaType):
> assert self.variants.tag_member in self.members
> self.variants.check_clash(schema, seen)
>
> + def check_clash(self, schema, seen):
> + self.check(schema)
Do we want to hide this .check() inside .check_clash()?
QAPISchemaObjectTypeMember.check() doesn't. I think the two better
behave the same.
> + assert not self.variants # not implemented
> + for m in self.members:
> + m.check_clash(seen)
> +
> def is_implicit(self):
> # See QAPISchema._make_implicit_object_type()
> return self.name[0] == ':'
> @@ -1062,12 +1064,7 @@ class QAPISchemaObjectTypeVariants(object):
> for v in self.variants:
> # Reset seen map for each variant, since qapi names from one
> # branch do not affect another branch
> - vseen = dict(seen)
> - assert isinstance(v.type, QAPISchemaObjectType)
> - assert not v.type.variants # not implemented
> - v.type.check(schema)
> - for m in v.type.members:
> - m.check_clash(vseen)
> + v.type.check_clash(schema, dict(seen))
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-09 12:56 ` Markus Armbruster
@ 2015-11-09 15:13 ` Markus Armbruster
2015-11-10 5:18 ` Eric Blake
2015-11-10 5:16 ` Eric Blake
1 sibling, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 15:13 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> Right now, our ad hoc parser ensures that we cannot have a
>> flat union that introduces any qapi member names that would
>> conflict with the non-variant qapi members already present
>> from the union's base type (see flat-union-clash-member.json).
>> We want QAPISchemaObjectType.check() to make the same check,
>> so we can later reduce some of the ad hoc checks.
>>
>> We already ensure that all branches of a flat union are qapi
>> structs with no variants, at which point those members appear
>> in the same JSON object as all non-variant members. And we
>> already have a map 'seen' of all non-variant members. All
>> we need is a new QAPISchemaObjectTypeVariants.check_clash(),
>> which clones the seen map then checks for clashes with each
>> member of the variant's qapi type.
>>
>> Note that the clone of seen inside Variants.check_clash()
>> resembles the one we just removed from Variants.check(); the
>> difference here is that we are now checking for clashes
>> among the qapi members of the variant type, rather than for
>> a single clash with the variant tag name itself.
>>
>> In general, a type used as a branch of a flat union cannot
>> also be the base type of the flat union, so even though we are
>> adding a call to variant.type.check() in order to populate
>> variant.type.members, this is merely a case of gaining
>> topological sorting of how types are visited (and type.check()
>> is already set up to allow multiple calls due to base types).
>
> Yes, a type cannot contain itself, neither as base nor as variant.
>
> We have tests covering attempts to do the former
> (struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
> see, we don't have tests covering the latter. Do we catch it?
>
>> For simple unions, the same code happens to work by design,
>> because of our synthesized wrapper classes (however, the
>> wrapper has a single member 'data' which will never collide
>> with the one non-variant member 'type', so it doesn't really
>> matter).
>>
>> There is no impact to alternates, which intentionally do not
>> need to call variants.check_clash() (there, at most one of
>> the variant's branches will be an ObjectType, and even if one
>> exists, we are not inlining the qapi members of that object
>> into a parent object, the way we do for unions).
Yes. QAPISchemaObjectTypeVariants.check_clash() checks for each
variant's members clashing with other members in the same name space.
For alternates, there are no such other members.
That said, should we add a comment to QAPISchemaAlternateType.check()?
Perhaps:
# Not calling self.variant.check_clash(), because there's
# nothing to clash with.
>> No change to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> Patch looks good.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names Eric Blake
@ 2015-11-09 15:17 ` Markus Armbruster
2015-11-11 0:34 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 15:17 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Detect attempts to declare two object members that would result
> in the same C member name, by keying the 'seen' dictionary off
> of the C name rather than the qapi name. It also requires passing
> info through the check_clash() methods.
>
> This addresses a TODO and fixes the previously-broken
> args-name-clash test. The resulting error message demonstrates
> the utility of the .describe() method added previously. No change
> to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10 (now in subset C): rebase to latest; update commit message
> v9 (now in subset D): rebase to earlier changes, now only one test
> affected
> v8: rebase to earlier changes
> v7: split out error reporting prep and member.c_name() addition
> v6: rebase to earlier testsuite and info improvements
> ---
> scripts/qapi.py | 31 +++++++++++++++++++------------
> tests/qapi-schema/args-name-clash.err | 1 +
> tests/qapi-schema/args-name-clash.exit | 2 +-
> tests/qapi-schema/args-name-clash.json | 6 +++---
> tests/qapi-schema/args-name-clash.out | 6 ------
> 5 files changed, 24 insertions(+), 22 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index b3af973..3a359cb 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -975,21 +975,24 @@ class QAPISchemaObjectType(QAPISchemaType):
> seen = OrderedDict()
> if self._base_name:
> self.base = schema.lookup_type(self._base_name)
> - self.base.check_clash(schema, seen)
> + self.base.check_clash(schema, self.info, seen)
Note that if this one reports an error for self.info, something went
wrong. We first run self.base.check(), which we survive only when
self.base doesn't have clashing members. Would be easier to see if
self.base.check() wasn't hiding in self.base.check_clash().
> for m in self.local_members:
> m.check(schema)
> - m.check_clash(seen)
> + m.check_clash(self.info, seen)
> self.members = seen.values()
> if self.variants:
> self.variants.check(schema, seen)
> assert self.variants.tag_member in self.members
> - self.variants.check_clash(schema, seen)
> + self.variants.check_clash(schema, self.info, seen)
>
> - def check_clash(self, schema, seen):
> + # Check that the members of this type do not cause duplicate JSON fields,
> + # and update seen to track the members seen so far. Report any errors
> + # on behalf of info, which is not necessarily self.info
Do we actually need info != self.info? If yes, test case? If no,
should the comment be adjusted?
> + def check_clash(self, schema, info, seen):
> self.check(schema)
> assert not self.variants # not implemented
> for m in self.members:
> - m.check_clash(seen)
> + m.check_clash(info, seen)
>
> def is_implicit(self):
> # See QAPISchema._make_implicit_object_type()
> @@ -1035,10 +1038,13 @@ class QAPISchemaObjectTypeMember(object):
> self.type = schema.lookup_type(self._type_name)
> assert self.type
>
> - def check_clash(self, seen):
> - # TODO change key of seen from QAPI name to C name
> - assert self.name not in seen
> - seen[self.name] = self
> + def check_clash(self, info, seen):
> + name = c_name(self.name)
I'd call it cname. Matter of taste, of course.
> + if name in seen:
> + raise QAPIExprError(info,
> + "%s collides with %s"
> + % (self.describe(), seen[name].describe()))
> + seen[name] = self
>
> def _pretty_owner(self):
> # See QAPISchema._make_implicit_object_type() - reverse the
> @@ -1081,18 +1087,19 @@ class QAPISchemaObjectTypeVariants(object):
>
> def check(self, schema, seen):
> if self.tag_name: # flat union
> - self.tag_member = seen[self.tag_name]
> + self.tag_member = seen[c_name(self.tag_name)]
> + assert self.tag_name == self.tag_member.name
> tag_type = self.tag_member.type
> assert isinstance(tag_type, QAPISchemaEnumType)
> for v in self.variants:
> v.check(schema)
> assert v.name in tag_type.values
>
> - def check_clash(self, schema, seen):
> + def check_clash(self, schema, info, seen):
> for v in self.variants:
> # Reset seen map for each variant, since qapi names from one
> # branch do not affect another branch
> - v.type.check_clash(schema, dict(seen))
> + v.type.check_clash(schema, info, dict(seen))
Since we can't inherit variant members, info is always the owning object
type's info.
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> diff --git a/tests/qapi-schema/args-name-clash.err b/tests/qapi-schema/args-name-clash.err
> index e69de29..2735217 100644
> --- a/tests/qapi-schema/args-name-clash.err
> +++ b/tests/qapi-schema/args-name-clash.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/args-name-clash.json:5: 'a_b' (argument of oops) collides with 'a-b' (argument of oops)
> diff --git a/tests/qapi-schema/args-name-clash.exit b/tests/qapi-schema/args-name-clash.exit
> index 573541a..d00491f 100644
> --- a/tests/qapi-schema/args-name-clash.exit
> +++ b/tests/qapi-schema/args-name-clash.exit
> @@ -1 +1 @@
> -0
> +1
> diff --git a/tests/qapi-schema/args-name-clash.json b/tests/qapi-schema/args-name-clash.json
> index 9e8f889..3fe4ea5 100644
> --- a/tests/qapi-schema/args-name-clash.json
> +++ b/tests/qapi-schema/args-name-clash.json
> @@ -1,5 +1,5 @@
> # C member name collision
> -# FIXME - This parses, but fails to compile, because the C struct is given
> -# two 'a_b' members. Either reject this at parse time, or munge the C names
> -# to avoid the collision.
> +# Reject members that clash when mapped to C names (we would have two 'a_b'
> +# members). It would also be possible to munge the C names to avoid the
> +# collision, but unlikely to be worth the effort.
I'd drop the second sentence.
> { 'command': 'oops', 'data': { 'a-b': 'str', 'a_b': 'str' } }
> diff --git a/tests/qapi-schema/args-name-clash.out b/tests/qapi-schema/args-name-clash.out
> index 9b2f6e4..e69de29 100644
> --- a/tests/qapi-schema/args-name-clash.out
> +++ b/tests/qapi-schema/args-name-clash.out
> @@ -1,6 +0,0 @@
> -object :empty
> -object :obj-oops-arg
> - member a-b: str optional=False
> - member a_b: str optional=False
> -command oops :obj-oops-arg -> None
> - gen=True success_response=True
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union Eric Blake
@ 2015-11-09 15:22 ` Markus Armbruster
2015-11-11 2:50 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 15:22 UTC (permalink / raw)
To: Eric Blake; +Cc: Paolo Bonzini, qemu-devel, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> When qapi type CpuInfo was originally created for 0.14, we had
> no notion of a flat union, and instead just listed a bunch of
> optional fields with documentation about the mutually-exclusive
> choice of which instruction pointer field(s) would be provided
> for a given architecture. But now that we have flat unions and
> introspection, it is better to segregate off which fields will
> be provided according to the actual architecture. With this in
> place, we no longer need the fields to be optional, because the
> choice of the discriminator serves that role.
>
> This has an additional benefit: the old all-in-one struct was
> the only place in the code base that had a case-sensitive
> naming of members 'pc' vs. 'PC'. Separating these spellings
> into different branches of the flat union will allow us to add
> restrictions against future case-insensitive collisions, and
> make it possible for a future qemu to decide whether to enable
> case-insensitive QMP.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v10: new patch
> ---
> cpus.c | 31 ++++++++------
> hmp.c | 30 +++++++++-----
> qapi-schema.json | 120 ++++++++++++++++++++++++++++++++++++++++++++++---------
> 3 files changed, 139 insertions(+), 42 deletions(-)
>
> diff --git a/cpus.c b/cpus.c
> index c6a5d0e..1c8078d 100644
> --- a/cpus.c
> +++ b/cpus.c
> @@ -1526,22 +1526,29 @@ CpuInfoList *qmp_query_cpus(Error **errp)
> info->value->qom_path = object_get_canonical_path(OBJECT(cpu));
> info->value->thread_id = cpu->thread_id;
> #if defined(TARGET_I386)
> - info->value->has_pc = true;
> - info->value->pc = env->eip + env->segs[R_CS].base;
> + info->value->arch = CPU_INFO_ARCH_X86;
> + info->value->u.x86 = g_new0(CpuInfoX86, 1);
> + info->value->u.x86->pc = env->eip + env->segs[R_CS].base;
> #elif defined(TARGET_PPC)
> - info->value->has_nip = true;
> - info->value->nip = env->nip;
> + info->value->arch = CPU_INFO_ARCH_PPC;
> + info->value->u.ppc = g_new0(CpuInfoPpc, 1);
> + info->value->u.ppc->nip = env->nip;
> #elif defined(TARGET_SPARC)
> - info->value->has_pc = true;
> - info->value->pc = env->pc;
> - info->value->has_npc = true;
> - info->value->npc = env->npc;
> + info->value->arch = CPU_INFO_ARCH_SPARC;
> + info->value->u.sparc = g_new0(CpuInfoSPARC, 1);
> + info->value->u.sparc->pc = env->pc;
> + info->value->u.sparc->npc = env->npc;
> #elif defined(TARGET_MIPS)
> - info->value->has_PC = true;
> - info->value->PC = env->active_tc.PC;
> + info->value->arch = CPU_INFO_ARCH_MIPS;
> + info->value->u.mips = g_new0(CpuInfoMips, 1);
> + info->value->u.mips->PC = env->active_tc.PC;
> #elif defined(TARGET_TRICORE)
> - info->value->has_PC = true;
> - info->value->PC = env->PC;
> + info->value->arch = CPU_INFO_ARCH_TRICORE;
> + info->value->u.tricore = g_new0(CpuInfoTricore, 1);
> + info->value->u.tricore->PC = env->PC;
> +#else
> + info->value->arch = CPU_INFO_ARCH_OTHER;
> + info->value->u.other = g_new0(CpuInfoOther, 1);
> #endif
>
> /* XXX: waiting for the qapi to support GSList */
> diff --git a/hmp.c b/hmp.c
> index a15d00c..1307016 100644
> --- a/hmp.c
> +++ b/hmp.c
> @@ -310,17 +310,25 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)
>
> monitor_printf(mon, "%c CPU #%" PRId64 ":", active, cpu->value->CPU);
>
> - if (cpu->value->has_pc) {
> - monitor_printf(mon, " pc=0x%016" PRIx64, cpu->value->pc);
> - }
> - if (cpu->value->has_nip) {
> - monitor_printf(mon, " nip=0x%016" PRIx64, cpu->value->nip);
> - }
> - if (cpu->value->has_npc) {
> - monitor_printf(mon, " npc=0x%016" PRIx64, cpu->value->npc);
> - }
> - if (cpu->value->has_PC) {
> - monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->PC);
> + switch (cpu->value->arch) {
> + case CPU_INFO_ARCH_X86:
> + monitor_printf(mon, " pc=0x%016" PRIx64, cpu->value->u.x86->pc);
> + break;
> + case CPU_INFO_ARCH_PPC:
> + monitor_printf(mon, " nip=0x%016" PRIx64, cpu->value->u.ppc->nip);
> + break;
> + case CPU_INFO_ARCH_SPARC:
> + monitor_printf(mon, " pc=0x%016" PRIx64, cpu->value->u.sparc->pc);
> + monitor_printf(mon, " npc=0x%016" PRIx64, cpu->value->u.sparc->npc);
> + break;
> + case CPU_INFO_ARCH_MIPS:
> + monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->u.mips->PC);
> + break;
> + case CPU_INFO_ARCH_TRICORE:
> + monitor_printf(mon, " PC=0x%016" PRIx64, cpu->value->u.tricore->PC);
> + break;
> + default:
> + break;
> }
>
> if (cpu->value->halted) {
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 702b7b5..dd497ea 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -728,43 +728,125 @@
> { 'command': 'query-mice', 'returns': ['MouseInfo'] }
>
> ##
> -# @CpuInfo:
> +# @CpuInfoArch:
> #
> -# Information about a virtual CPU
> +# An enumeration of cpu types that enable additional information during
> +# @query-cpus.
> +#
> +# Since: 2.5
> +##
> +{ 'enum': 'CpuInfoArch',
> + 'data': ['x86', 'sparc', 'ppc', 'mips', 'tricore', 'other' ] }
> +
> +##
> +# @CpuInfoBase:
> +#
> +# Common information about a virtual CPU
> #
> # @CPU: the index of the virtual CPU
> #
> -# @current: this only exists for backwards compatible and should be ignored
> +# @current: this only exists for backwards compatibility and should be ignored
> #
> # @halted: true if the virtual CPU is in the halt state. Halt usually refers
> # to a processor specific low power mode.
> #
> # @qom_path: path to the CPU object in the QOM tree (since 2.4)
> #
> -# @pc: #optional If the target is i386 or x86_64, this is the 64-bit instruction
> -# pointer.
> -# If the target is Sparc, this is the PC component of the
> -# instruction pointer.
> -#
> -# @nip: #optional If the target is PPC, the instruction pointer
> -#
> -# @npc: #optional If the target is Sparc, the NPC component of the instruction
> -# pointer
> -#
> -# @PC: #optional If the target is MIPS, the instruction pointer
> -#
> # @thread_id: ID of the underlying host thread
> #
> +# @arch: architecture of the cpu, which determines which additional fields
> +# will be listed (since 2.5)
> +#
> # Since: 0.14.0
> #
> # Notes: @halted is a transient state that changes frequently. By the time the
> # data is sent to the client, the guest may no longer be halted.
> ##
> -{ 'struct': 'CpuInfo',
> +{ 'struct': 'CpuInfoBase',
> 'data': {'CPU': 'int', 'current': 'bool', 'halted': 'bool',
> - 'qom_path': 'str',
> - '*pc': 'int', '*nip': 'int', '*npc': 'int', '*PC': 'int',
> - 'thread_id': 'int'} }
> + 'qom_path': 'str', 'thread_id': 'int', 'arch': 'CpuInfoArch' } }
> +
> +##
> +# @CpuInfo:
> +#
> +# Information about a virtual CPU
> +#
> +# Since: 0.14.0
> +##
> +{ 'union': 'CpuInfo', 'base': 'CpuInfoBase', 'discriminator': 'arch',
> + 'data': { 'x86': 'CpuInfoX86',
> + 'sparc': 'CpuInfoSparc',
> + 'ppc': 'CpuInfoPpc',
> + 'mips': 'CpuInfoMips',
> + 'tricore': 'CpuInfoTricore',
> + 'other': 'CpuInfoOther' } }
> +
> +##
> +# @CpuInfoX86:
> +#
> +# Additional information about a virtual i386 or x86_64 CPU
> +#
> +# @pc: the 64-bit instruction pointer
> +#
> +# Since 2.5
> +##
> +{ 'struct': 'CpuInfoX86', 'data': { 'pc': 'int' } }
> +
> +##
> +# @CpuInfoSparc:
> +#
> +# Additional information about a virtual Sparc CPU
> +#
> +# @pc: the PC component of the instruction pointer
> +#
> +# @npc: the NPC component of the instruction pointer
> +#
> +# Since 2.5
> +##
> +{ 'struct': 'CpuInfoSparc', 'data': { 'pc': 'int', 'npc': 'int' } }
> +
> +##
> +# @CpuInfoPpc:
> +#
> +# Additional information about a virtual PPC CPU
> +#
> +# @nip: the instruction pointer
> +#
> +# Since 2.5
> +##
> +{ 'struct': 'CpuInfoPpc', 'data': { 'nip': 'int' } }
> +
> +##
> +# @CpuInfoMips:
> +#
> +# Additional information about a virtual MIPS CPU
> +#
> +# @PC: the instruction pointer
> +#
> +# Since 2.5
> +##
> +{ 'struct': 'CpuInfoMips', 'data': { 'PC': 'int' } }
> +
> +##
> +# @CpuInfoMips:
> +#
> +# Additional information about a virtual Tricore CPU
> +#
> +# @PC: the instruction pointer
> +#
> +# Since 2.5
> +##
> +{ 'struct': 'CpuInfoTricore', 'data': { 'PC': 'int' } }
> +
> +##
> +# @CpuInfoOther:
> +#
> +# No additional information is available about the virtual CPU
> +#
> +# Since 2.5
> +#
> +##
> +{ 'struct': 'CpuInfoOther', 'data': { } }
>
> ##
> # @query-cpus:
CpuInfo is only used as return type of query-cpus.
If I understand the change correctly, the value of query-cpus is not
changed at all. Its introspection, however, is.
Do we need this to make 2.5?
Any other messy optionals that should really be (flat) unions?
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 30/30] qapi: Forbid case-insensitive clashes
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 30/30] qapi: Forbid case-insensitive clashes Eric Blake
@ 2015-11-09 15:42 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 15:42 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> We have toyed on list with the idea of a future extension to
> QMP of teaching it to be case-insensitive (the user could
> request command 'Quit' instead of 'quit', or could spell a
> struct field as 'CPU' instead of 'cpu'). But for that to be
> a practical extension, we cannot break backwards compatibility
> with any existing struct that was already relying on case
> sensitivity. Fortunately, after the previous patch cleaned
> up CpuInfo, there are no such existing qapi structs.
>
> Another benefit of enforcing a restriction against
> case-insensitive is that we no longer have to worry about the
> case of enum values that could be distinguished by case if
> mapped by c_name(), but which cannot be distinguished when
> mapped to C as ALL_CAPS by camel_to_uppper(). Having the
camel_to_upper()
> generator look for case collisions up front will prevent
> developers from worrying whether different munging rules
> for member names compared to enum values as a discriminator
> will cause any problems in qapi unions.
>
> Of course, if we never implement a case-insensitive QMP
> extension, or find a legitimate reason to need qapi members
> that differ only by case, we could always relax this
> restriction. But it is easier to relax later than it is to
> wish we had the restriction in place earlier.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
I think "make QMP case-insensitive" is a very weak justification for
anything.
Your second reason is much stronger: forbidding case-insensitive clashes
lets the generators shout without fear of introducing clashes in
generated code.
However, since camel_to_upper() does more than just shout, forbidding
case-insensitive clashes protects against clashes in generated code only
for names where camel_to_upper() just shouts. It does for lower case
names, i.e. almost all of them.
The commit would make more sense if we first defanged c_enum_const():
make it mangle the const_name part like c_name(const_name).upper().
This is independent of the argument I'm having with Dan on how the
prefix should be mangled.
One more good argument for forbidding case-insensitive clashes: an
interface that sports names differing only in case is a bad interface.
> ---
> v10: new patch
> ---
> docs/qapi-code-gen.txt | 5 +++++
> scripts/qapi.py | 4 ++--
> tests/Makefile | 1 +
> tests/qapi-schema/args-case-clash.err | 1 +
> tests/qapi-schema/args-case-clash.exit | 1 +
> tests/qapi-schema/args-case-clash.json | 5 +++++
> tests/qapi-schema/args-case-clash.out | 0
> 7 files changed, 15 insertions(+), 2 deletions(-)
> create mode 100644 tests/qapi-schema/args-case-clash.err
> create mode 100644 tests/qapi-schema/args-case-clash.exit
> create mode 100644 tests/qapi-schema/args-case-clash.json
> create mode 100644 tests/qapi-schema/args-case-clash.out
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index f9fa6f3..13cec10 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -116,6 +116,11 @@ names should be ALL_CAPS with words separated by underscore. Field
> names cannot start with 'has-' or 'has_', as this is reserved for
> tracking optional fields.
>
> +For now, Client JSON Protocol is case-sensitive, but future extensions
> +may allow for case-insensitive recognition of command and event names,
> +or of member field names. As such, the generator rejects attempts to
> +create entities that only differ by case.
> +
This suggests we're actually planning to make QMP case-insensitive.
Let's avoid that. Perhaps:
Since we don't want interfaces with different names that differ only
in case, the generator rejects such names.
> Any name (command, event, type, field, or enum value) beginning with
> "x-" is marked experimental, and may be withdrawn or changed
> incompatibly in a future release. Downstream vendors may add
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 3a359cb..7e7ad6e 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -1039,7 +1039,7 @@ class QAPISchemaObjectTypeMember(object):
> assert self.type
>
> def check_clash(self, info, seen):
> - name = c_name(self.name)
> + name = c_name(self.name).upper()
> if name in seen:
> raise QAPIExprError(info,
> "%s collides with %s"
> @@ -1087,7 +1087,7 @@ class QAPISchemaObjectTypeVariants(object):
>
> def check(self, schema, seen):
> if self.tag_name: # flat union
> - self.tag_member = seen[c_name(self.tag_name)]
> + self.tag_member = seen[c_name(self.tag_name).upper()]
> assert self.tag_name == self.tag_member.name
> tag_type = self.tag_member.type
> assert isinstance(tag_type, QAPISchemaEnumType)
If we ever acquire more lookups in seen, we'll need a lookup function,
so we don't duplicate c_name(NAME).upper(). But this will do for now.
> diff --git a/tests/Makefile b/tests/Makefile
> index c84c6cb..d1c6817 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -239,6 +239,7 @@ qapi-schema += args-alternate.json
> qapi-schema += args-any.json
> qapi-schema += args-array-empty.json
> qapi-schema += args-array-unknown.json
> +qapi-schema += args-case-clash.json
> qapi-schema += args-int.json
> qapi-schema += args-invalid.json
> qapi-schema += args-member-array-bad.json
> diff --git a/tests/qapi-schema/args-case-clash.err b/tests/qapi-schema/args-case-clash.err
> new file mode 100644
> index 0000000..5495ab8
> --- /dev/null
> +++ b/tests/qapi-schema/args-case-clash.err
> @@ -0,0 +1 @@
> +tests/qapi-schema/args-case-clash.json:5: 'A' (argument of oops) collides with 'a' (argument of oops)
> diff --git a/tests/qapi-schema/args-case-clash.exit b/tests/qapi-schema/args-case-clash.exit
> new file mode 100644
> index 0000000..d00491f
> --- /dev/null
> +++ b/tests/qapi-schema/args-case-clash.exit
> @@ -0,0 +1 @@
> +1
> diff --git a/tests/qapi-schema/args-case-clash.json b/tests/qapi-schema/args-case-clash.json
> new file mode 100644
> index 0000000..55ae488
> --- /dev/null
> +++ b/tests/qapi-schema/args-case-clash.json
> @@ -0,0 +1,5 @@
> +# C member name collision
> +# Reject members that clash case-insensitively (our mapping to C names
> +# preserves case, but allowing these members now would prevent a future
> +# relaxing of QMP to be case-insensitive).
Again, let's not suggest we're planning to make QMP case-insensitive.
> +{ 'command': 'oops', 'data': { 'a': 'str', 'A': 'int' } }
> diff --git a/tests/qapi-schema/args-case-clash.out b/tests/qapi-schema/args-case-clash.out
> new file mode 100644
> index 0000000..e69de29
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-09 13:00 ` Markus Armbruster
@ 2015-11-09 17:36 ` Eric Blake
2015-11-09 19:11 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-09 17:36 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2066 bytes --]
On 11/09/2015 06:00 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Consolidate two common sequences of clash detection into a
>> new QAPISchemaObjectType.check_clash() helper method.
>>
>> No change to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
>> seen = OrderedDict()
>> if self._base_name:
>> self.base = schema.lookup_type(self._base_name)
>> - assert isinstance(self.base, QAPISchemaObjectType)
>
> This assertion is lost.
>
>> - assert not self.base.variants # not implemented
>> - self.base.check(schema)
>> - for m in self.base.members:
>> - m.check_clash(seen)
>> + self.base.check_clash(schema, seen)
Directly lost, but indirectly still present. The new code is calling
QAPISchemaObjectType.check_clash(), which won't exist unless self.base
is a QAPISchemaObjectType. Folding the assert into the refactored
function makes no sense (the condition isinstance(self,
QAPISchemaObjectType) would always be true), and leaving the assert
prior to calling self.base.check_clash() adds no real protection against
programming bugs.
>> @@ -1062,12 +1064,7 @@ class QAPISchemaObjectTypeVariants(object):
>> for v in self.variants:
>> # Reset seen map for each variant, since qapi names from one
>> # branch do not affect another branch
>> - vseen = dict(seen)
>> - assert isinstance(v.type, QAPISchemaObjectType)
>
> This assertion is lost.
>
>> - assert not v.type.variants # not implemented
>> - v.type.check(schema)
>> - for m in v.type.members:
>> - m.check_clash(vseen)
>> + v.type.check_clash(schema, dict(seen))
Same explanation.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-09 14:43 ` Eric Blake
@ 2015-11-09 18:42 ` Markus Armbruster
2015-11-10 11:57 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 18:42 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 02:59 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> On 11/06/2015 09:03 AM, Markus Armbruster wrote:
>>>> Eric Blake <eblake@redhat.com> writes:
>> [...]
>>>>> Hopefully, we are converging on something that will be ready
>>>>> for a pull request, especially for the earlier patches of this
>>>>> subset.
>>>>
>>>> I guess you mean PATCH 01-12. I had a few questions, but the most
>>>> likely outcome seems to be minor touchups I could apply in my tree.
>>>>
>>>> I'm okay with trying to get more patches in, but let's get these out of
>>>> the way meanwhile.
>>>
>>> Yes, 01-12 seems like a good first set, if you want to make those
>>> touchups (I've supplied some potential text improvements in reply to
>>> some of your comments); I'm happy, as always, to take a peek over your
>>> staging repo to double check what you are prepping for the pull request.
>>
>> Please have a look at my qapi-next branch.
>
> Close, but commit 5f72bb85 on that branch is missing the one-line summary:
>
> qapi: Simplify error cleanup in test-qmp-*
Fixed & sent pull request, thanks!
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-09 17:36 ` Eric Blake
@ 2015-11-09 19:11 ` Markus Armbruster
2015-11-10 5:22 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-09 19:11 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 06:00 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Consolidate two common sequences of clash detection into a
>>> new QAPISchemaObjectType.check_clash() helper method.
>>>
>>> No change to generated code.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>
>>> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
>>> seen = OrderedDict()
>>> if self._base_name:
>>> self.base = schema.lookup_type(self._base_name)
>>> - assert isinstance(self.base, QAPISchemaObjectType)
>>
>> This assertion is lost.
>>
>>> - assert not self.base.variants # not implemented
>>> - self.base.check(schema)
>>> - for m in self.base.members:
>>> - m.check_clash(seen)
>>> + self.base.check_clash(schema, seen)
>
> Directly lost, but indirectly still present. The new code is calling
> QAPISchemaObjectType.check_clash(), which won't exist unless self.base
> is a QAPISchemaObjectType.
or a QAPISchemaObjectTypeMember, or a QAPISchemaObjectVariants, or
whatever else acquires the method in the future.
> Folding the assert into the refactored
> function makes no sense (the condition isinstance(self,
> QAPISchemaObjectType) would always be true),
Correct.
> and leaving the assert
> prior to calling self.base.check_clash() adds no real protection against
> programming bugs.
Maybe, but the isinstance(self.base, QAPISchemaObjectType) will come
right back anyway when we move the "'base' for FOO cannot use BAR type"
check from the old semantic analysis into the check methods. Until
then, it makes sense at least as a place holder.
>>> @@ -1062,12 +1064,7 @@ class QAPISchemaObjectTypeVariants(object):
>>> for v in self.variants:
>>> # Reset seen map for each variant, since qapi names from one
>>> # branch do not affect another branch
>>> - vseen = dict(seen)
>>> - assert isinstance(v.type, QAPISchemaObjectType)
>>
>> This assertion is lost.
>>
>>> - assert not v.type.variants # not implemented
>>> - v.type.check(schema)
>>> - for m in v.type.members:
>>> - m.check_clash(vseen)
>>> + v.type.check_clash(schema, dict(seen))
>
> Same explanation.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting
2015-11-06 15:52 ` Markus Armbruster
@ 2015-11-09 20:56 ` Eric Blake
2015-11-10 7:36 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-09 20:56 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1066 bytes --]
On 11/06/2015 08:52 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> qapi-code-gen.txt already claims that types, commands, and
>> events share a common namespace; set this in stone by further
>> documenting that our introspection output will never have
>> collisions with the same name tied to more than one meta-type.
>>
>> For these reasons, we simply document that clients should not
>> rely on any particular order of items in introspection output.
>> And since it is now a documented part of the contract, we have
>> the freedom to later rearrange output if needed, without
>> worrying about breaking well-written clients that were already
>> aware that they had to do linear searches.
>
> Well, any kind of search that doesn't rely on sorting. Suggest to drop
> the "that were already..." clause. Happy to do that on commit.
I don't see that phrase dropped in your v1 pull request. Worth fixing?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check()
2015-11-09 12:38 ` Markus Armbruster
@ 2015-11-10 5:04 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-10 5:04 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]
On 11/09/2015 05:38 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> From: Markus Armbruster <armbru@redhat.com>
>>
>> Reduce the ugly flat union / simple union conditional by doing just
>> the essential work here, namely setting self.tag_member.
>> Move the rest to callers.
>>
>> @@ -1052,8 +1053,6 @@ class QAPISchemaObjectTypeVariants(object):
>> def check(self, schema, seen):
>> if self.tag_name: # flat union
>> self.tag_member = seen[self.tag_name]
>
> My patch has:
>
> - if self.tag_name: # flat union
> + if not self.tag_member: # flat union
> self.tag_member = seen[self.tag_name]
>
> Any particular reason for dropping it?
Not really; I couldn't find anything in later patches that cared either way.
>
> I like my change, because I feel it makes the assignment's purpose more
> obvious: ensure .tag_member is set.
Sure, we'll go with your approach.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-09 12:56 ` Markus Armbruster
2015-11-09 15:13 ` Markus Armbruster
@ 2015-11-10 5:16 ` Eric Blake
2015-11-10 8:30 ` Markus Armbruster
2015-11-10 23:37 ` Eric Blake
1 sibling, 2 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-10 5:16 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2759 bytes --]
On 11/09/2015 05:56 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Right now, our ad hoc parser ensures that we cannot have a
>> flat union that introduces any qapi member names that would
>> conflict with the non-variant qapi members already present
>> from the union's base type (see flat-union-clash-member.json).
>> We want QAPISchemaObjectType.check() to make the same check,
>> so we can later reduce some of the ad hoc checks.
>>
>> In general, a type used as a branch of a flat union cannot
>> also be the base type of the flat union, so even though we are
>> adding a call to variant.type.check() in order to populate
>> variant.type.members, this is merely a case of gaining
>> topological sorting of how types are visited (and type.check()
>> is already set up to allow multiple calls due to base types).
>
> Yes, a type cannot contain itself, neither as base nor as variant.
>
> We have tests covering attempts to do the former
> (struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
> see, we don't have tests covering the latter. Do we catch it?
Yes, at least by virtue of the ad hoc tests: attempting to reuse a base
type of the flat union as a variant member will cause the qapi members
of the base type to appear more than once in the JSON object (that is,
the checks that reject flat-union-clash-member.json would also reject
this scenario). To test:
diff --git i/tests/qapi-schema/qapi-schema-test.json
w/tests/qapi-schema/qapi-schema-test.json
index 44638da..16b2ffb 100644
--- i/tests/qapi-schema/qapi-schema-test.json
+++ w/tests/qapi-schema/qapi-schema-test.json
@@ -67,7 +67,7 @@
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
- 'value3' : 'UserDefB' } }
+ 'value3' : 'UserDefUnionBase' } }
{ 'struct': 'UserDefUnionBase',
'base': 'UserDefZero',
GEN tests/test-qapi-types.h
/home/eblake/qemu/tests/qapi-schema/qapi-schema-test.json:65: Member
name 'string' of branch 'value3' clashes with base 'UserDefUnionBase'
/home/eblake/qemu/tests/Makefile:415: recipe for target
'tests/test-qapi-types.h' failed
But you have me curious if this collision is still caught when the ad
hoc tests are gone. If so, great; if not, I'll add a test here. (I'll
know later when I get through rebasing to all of your comments.)
>> No change to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> Patch looks good.
Yay; it's nice to see results after all our mental gymnastics over how
collision testing should work.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply related [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-09 15:13 ` Markus Armbruster
@ 2015-11-10 5:18 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-10 5:18 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1544 bytes --]
On 11/09/2015 08:13 AM, Markus Armbruster wrote:
> Markus Armbruster <armbru@redhat.com> writes:
>
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Right now, our ad hoc parser ensures that we cannot have a
>>> flat union that introduces any qapi member names that would
>>> conflict with the non-variant qapi members already present
>>> from the union's base type (see flat-union-clash-member.json).
>>> We want QAPISchemaObjectType.check() to make the same check,
>>> so we can later reduce some of the ad hoc checks.
>>>
>>> There is no impact to alternates, which intentionally do not
>>> need to call variants.check_clash() (there, at most one of
>>> the variant's branches will be an ObjectType, and even if one
>>> exists, we are not inlining the qapi members of that object
>>> into a parent object, the way we do for unions).
>
> Yes. QAPISchemaObjectTypeVariants.check_clash() checks for each
> variant's members clashing with other members in the same name space.
> For alternates, there are no such other members.
>
> That said, should we add a comment to QAPISchemaAlternateType.check()?
> Perhaps:
>
> # Not calling self.variant.check_clash(), because there's
> # nothing to clash with.
Sure; if there's another reason for a respin, I can add it here;
otherwise, it's already slated to be added in my respin of the patches
for redoing how alternates are laid out.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-09 19:11 ` Markus Armbruster
@ 2015-11-10 5:22 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-10 5:22 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1481 bytes --]
On 11/09/2015 12:11 PM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 11/09/2015 06:00 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> Consolidate two common sequences of clash detection into a
>>>> new QAPISchemaObjectType.check_clash() helper method.
>>>>
>>>> No change to generated code.
>>>>
>>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
>>>> seen = OrderedDict()
>>>> if self._base_name:
>>>> self.base = schema.lookup_type(self._base_name)
>>>> - assert isinstance(self.base, QAPISchemaObjectType)
>>>
>>> This assertion is lost.
>>>
>>
>> Directly lost, but indirectly still present. The new code is calling
>> QAPISchemaObjectType.check_clash(), which won't exist unless self.base
>> is a QAPISchemaObjectType.
>
> or a QAPISchemaObjectTypeMember, or a QAPISchemaObjectVariants, or
> whatever else acquires the method in the future.
>
>
> Maybe, but the isinstance(self.base, QAPISchemaObjectType) will come
> right back anyway when we move the "'base' for FOO cannot use BAR type"
> check from the old semantic analysis into the check methods. Until
> then, it makes sense at least as a place holder.
Good point, I'll add it back in.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-09 14:49 ` Markus Armbruster
@ 2015-11-10 5:32 ` Eric Blake
2015-11-10 9:15 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-10 5:32 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2750 bytes --]
On 11/09/2015 07:49 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Consolidate two common sequences of clash detection into a
>> new QAPISchemaObjectType.check_clash() helper method.
>>
>> No change to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
>> seen = OrderedDict()
>> if self._base_name:
>> self.base = schema.lookup_type(self._base_name)
>> - assert isinstance(self.base, QAPISchemaObjectType)
>> - assert not self.base.variants # not implemented
>> - self.base.check(schema)
>> - for m in self.base.members:
>> - m.check_clash(seen)
>> + self.base.check_clash(schema, seen)
>> for m in self.local_members:
>> m.check(schema)
>> m.check_clash(seen)
>> @@ -994,6 +990,12 @@ class QAPISchemaObjectType(QAPISchemaType):
>> assert self.variants.tag_member in self.members
>> self.variants.check_clash(schema, seen)
>>
>> + def check_clash(self, schema, seen):
>> + self.check(schema)
>
> Do we want to hide this .check() inside .check_clash()?
>
> QAPISchemaObjectTypeMember.check() doesn't. I think the two better
> behave the same.
>
>> + assert not self.variants # not implemented
>> + for m in self.members:
>> + m.check_clash(seen)
The self.check(schema) call is necessary to get self.members populated.
We cannot iterate over self.members if the type has not had check()
called; this is true for both callers of type.check_clash()
(ObjectType.check(), and Variants.check_clash()).
You are correct that neither Member.check() nor Member.check_clash()
call a form of type.check() - but that's because at that level, there is
no need to populate a type.members list.
On the other hand, we've been arguing that check() should populate
everything after construction prior to anything else being run; and not
running Variant.type.check() during Variants.check() of flat unions
feels like we may have a hole (a flat union will have to inline its
types to the overall JSON object, and inlining types requires access to
type.members - but as written, we aren't populating them until
Variants.check_clash()). I can play with hoisting the type.check() out
of type.check_clash() and instead keep base.check() in type.check(), and
add variant.type.check() in Variants.check() (but only for unions, not
for alternates), if you are interested.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check()
2015-11-09 13:07 ` Markus Armbruster
@ 2015-11-10 5:33 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-10 5:33 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]
On 11/09/2015 06:07 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Checking that a given QAPISchemaObjectTypeVariant.name is a
>> member of the corresponding QAPISchemaEnumType of the owning
>> QAPISchemaObjectTypeVariants.tag_member ensures that there are
>> no collisions in the generated C union for those tag values
>> (since the enum itself should have no collisions).
>>
>> However, this check was the only thing that Variant.check() was
>> doing beyond the work of the superclass ObjectTypeMember.check(),
>> and resulted in a difference of the .check() signatures just to
>> pass the enum type down.
>>
>> Simplify things by instead doing the tag name check as part of
>> Variants.check(), at which point we can rely on inheritance
>> instead of overriding Variant.check().
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> - assert isinstance(self.tag_member.type, QAPISchemaEnumType)
>> + tag_type = self.tag_member.type
>> + assert isinstance(tag_type, QAPISchemaEnumType)
>> for v in self.variants:
>> - v.check(schema, self.tag_member.type)
>> + v.check(schema)
>> + assert v.name in tag_type.values
>
> Two changes squashed together:
>
> * Move the assertion from QAPISchemaObjectTypeVariant.check(),
>
> * Capture self.tag_member.type in tag_type
>
> The second part makes the patch slightly less obvious. Matter of taste.
I'm dropping the tag_type temporary member. Once I played more with
making qtype_code a qapi builtin type, it became much more elegant to
revisit this code (the use of tag_type here was for when my later
patches were trying to key off of tag_type==None as a witness of
alternates).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting
2015-11-09 20:56 ` Eric Blake
@ 2015-11-10 7:36 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-10 7:36 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/06/2015 08:52 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> qapi-code-gen.txt already claims that types, commands, and
>>> events share a common namespace; set this in stone by further
>>> documenting that our introspection output will never have
>>> collisions with the same name tied to more than one meta-type.
>>>
>
>>> For these reasons, we simply document that clients should not
>>> rely on any particular order of items in introspection output.
>>> And since it is now a documented part of the contract, we have
>>> the freedom to later rearrange output if needed, without
>>> worrying about breaking well-written clients that were already
>>> aware that they had to do linear searches.
>>
>> Well, any kind of search that doesn't rely on sorting. Suggest to drop
>> the "that were already..." clause. Happy to do that on commit.
>
> I don't see that phrase dropped in your v1 pull request. Worth fixing?
Fixed in PULL v2, thanks!
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-10 5:16 ` Eric Blake
@ 2015-11-10 8:30 ` Markus Armbruster
2015-11-10 13:24 ` Eric Blake
2015-11-10 23:37 ` Eric Blake
1 sibling, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-10 8:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 05:56 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Right now, our ad hoc parser ensures that we cannot have a
>>> flat union that introduces any qapi member names that would
>>> conflict with the non-variant qapi members already present
>>> from the union's base type (see flat-union-clash-member.json).
>>> We want QAPISchemaObjectType.check() to make the same check,
>>> so we can later reduce some of the ad hoc checks.
>>>
>
>>> In general, a type used as a branch of a flat union cannot
>>> also be the base type of the flat union, so even though we are
>>> adding a call to variant.type.check() in order to populate
>>> variant.type.members, this is merely a case of gaining
>>> topological sorting of how types are visited (and type.check()
>>> is already set up to allow multiple calls due to base types).
>>
>> Yes, a type cannot contain itself, neither as base nor as variant.
>>
>> We have tests covering attempts to do the former
>> (struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
Actually, these are just local, unpublished tests. They both make
check_member_clash() recurse infinitely.
# Direct inheritance loop
# FIXME triggers infinite recursion
{ 'struct': 'Loopy', 'base': 'Loopy',
'data': {} }
# we reject a loop in base classes
{ 'struct': 'Base1', 'base': 'Base2', 'data': {} }
{ 'struct': 'Base2', 'base': 'Base1', 'data': {} }
The latter is actually yours, proposed as base-cycle.json in
Subject: qapi: Detect collisions in C member names
Message-Id: <1442872682-6523-17-git-send-email-eblake@redhat.com>
If I disable the recursive call, the cycle detection in
QAPISchemaObjectType.check() is reached, and works.
Completing the move of clash detection to check() methods should improve
things from "accidental infinite recursion" to "intentional assertion
failure", because it should get rid of check_member_clash() and should
not break the cycle detection.
Then we can turn the assertion into a proper error message, and add the
tests.
>> see, we don't have tests covering the latter. Do we catch it?
>
> Yes, at least by virtue of the ad hoc tests: attempting to reuse a base
> type of the flat union as a variant member will cause the qapi members
> of the base type to appear more than once in the JSON object (that is,
> the checks that reject flat-union-clash-member.json would also reject
> this scenario). To test:
>
> diff --git i/tests/qapi-schema/qapi-schema-test.json
> w/tests/qapi-schema/qapi-schema-test.json
> index 44638da..16b2ffb 100644
> --- i/tests/qapi-schema/qapi-schema-test.json
> +++ w/tests/qapi-schema/qapi-schema-test.json
> @@ -67,7 +67,7 @@
> 'discriminator': 'enum1',
> 'data': { 'value1' : 'UserDefA',
> 'value2' : 'UserDefB',
> - 'value3' : 'UserDefB' } }
> + 'value3' : 'UserDefUnionBase' } }
>
> { 'struct': 'UserDefUnionBase',
> 'base': 'UserDefZero',
>
> GEN tests/test-qapi-types.h
> /home/eblake/qemu/tests/qapi-schema/qapi-schema-test.json:65: Member
> name 'string' of branch 'value3' clashes with base 'UserDefUnionBase'
> /home/eblake/qemu/tests/Makefile:415: recipe for target
> 'tests/test-qapi-types.h' failed
>
> But you have me curious if this collision is still caught when the ad
> hoc tests are gone. If so, great; if not, I'll add a test here. (I'll
> know later when I get through rebasing to all of your comments.)
>
>>> No change to generated code.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> Patch looks good.
>
> Yay; it's nice to see results after all our mental gymnastics over how
> collision testing should work.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-10 5:32 ` Eric Blake
@ 2015-11-10 9:15 ` Markus Armbruster
2015-11-10 13:19 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-10 9:15 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 07:49 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Consolidate two common sequences of clash detection into a
>>> new QAPISchemaObjectType.check_clash() helper method.
>>>
>>> No change to generated code.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>
>>> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
>>> seen = OrderedDict()
>>> if self._base_name:
>>> self.base = schema.lookup_type(self._base_name)
>>> - assert isinstance(self.base, QAPISchemaObjectType)
>>> - assert not self.base.variants # not implemented
>>> - self.base.check(schema)
>>> - for m in self.base.members:
>>> - m.check_clash(seen)
>>> + self.base.check_clash(schema, seen)
>>> for m in self.local_members:
>>> m.check(schema)
>>> m.check_clash(seen)
>>> @@ -994,6 +990,12 @@ class QAPISchemaObjectType(QAPISchemaType):
>>> assert self.variants.tag_member in self.members
>>> self.variants.check_clash(schema, seen)
>>>
>>> + def check_clash(self, schema, seen):
>>> + self.check(schema)
>>
>> Do we want to hide this .check() inside .check_clash()?
>>
>> QAPISchemaObjectTypeMember.check() doesn't. I think the two better
>> behave the same.
>>
>>> + assert not self.variants # not implemented
>>> + for m in self.members:
>>> + m.check_clash(seen)
>
> The self.check(schema) call is necessary to get self.members populated.
> We cannot iterate over self.members if the type has not had check()
> called; this is true for both callers of type.check_clash()
> (ObjectType.check(), and Variants.check_clash()).
Yes.
We have a common protocol for QAPISchemaFOO objects, namely that certain
instance variables and methods are only valid after .check().
> You are correct that neither Member.check() nor Member.check_clash()
> call a form of type.check() - but that's because at that level, there is
> no need to populate a type.members list.
>
> On the other hand, we've been arguing that check() should populate
> everything after construction prior to anything else being run; and not
> running Variant.type.check() during Variants.check() of flat unions
> feels like we may have a hole (a flat union will have to inline its
> types to the overall JSON object, and inlining types requires access to
> type.members - but as written, we aren't populating them until
> Variants.check_clash()). I can play with hoisting the type.check() out
> of type.check_clash() and instead keep base.check() in type.check(), and
> add variant.type.check() in Variants.check() (but only for unions, not
> for alternates), if you are interested.
My "qapi: Factor out QAPISchemaObjectTypeMember.check_clash()" added
QAPISchemaObjectTypeMember.check_clash() without changing the common
protocol. The new QAPISchemaObjectTypeMember.check_clash() is merely a
helper for QAPISchemaObjectType.check().
Your
Gcc: nnml:mail.redhat.xlst.qemu-devel
From: Markus Armbruster <armbru@redhat.com>
--text follows this line--
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 07:49 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Consolidate two common sequences of clash detection into a
>>> new QAPISchemaObjectType.check_clash() helper method.
>>>
>>> No change to generated code.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>
>>> @@ -980,11 +980,7 @@ class QAPISchemaObjectType(QAPISchemaType):
>>> seen = OrderedDict()
>>> if self._base_name:
>>> self.base = schema.lookup_type(self._base_name)
>>> - assert isinstance(self.base, QAPISchemaObjectType)
>>> - assert not self.base.variants # not implemented
>>> - self.base.check(schema)
>>> - for m in self.base.members:
>>> - m.check_clash(seen)
>>> + self.base.check_clash(schema, seen)
>>> for m in self.local_members:
>>> m.check(schema)
>>> m.check_clash(seen)
>>> @@ -994,6 +990,12 @@ class QAPISchemaObjectType(QAPISchemaType):
>>> assert self.variants.tag_member in self.members
>>> self.variants.check_clash(schema, seen)
>>>
>>> + def check_clash(self, schema, seen):
>>> + self.check(schema)
>>
>> Do we want to hide this .check() inside .check_clash()?
>>
>> QAPISchemaObjectTypeMember.check() doesn't. I think the two better
>> behave the same.
>>
>>> + assert not self.variants # not implemented
>>> + for m in self.members:
>>> + m.check_clash(seen)
>
> The self.check(schema) call is necessary to get self.members populated.
> We cannot iterate over self.members if the type has not had check()
> called; this is true for both callers of type.check_clash()
> (ObjectType.check(), and Variants.check_clash()).
Yes.
We have a common protocol for QAPISchemaFOO objects, namely that certain
instance variables and methods are only valid after .check().
> You are correct that neither Member.check() nor Member.check_clash()
> call a form of type.check() - but that's because at that level, there is
> no need to populate a type.members list.
>
> On the other hand, we've been arguing that check() should populate
> everything after construction prior to anything else being run; and not
> running Variant.type.check() during Variants.check() of flat unions
> feels like we may have a hole (a flat union will have to inline its
> types to the overall JSON object, and inlining types requires access to
> type.members - but as written, we aren't populating them until
> Variants.check_clash()). I can play with hoisting the type.check() out
> of type.check_clash() and instead keep base.check() in type.check(), and
> add variant.type.check() in Variants.check() (but only for unions, not
> for alternates), if you are interested.
My "qapi: Factor out QAPISchemaObjectTypeMember.check_clash()" adds
QAPISchemaObjectTypeMember.check_clash() without changing the common
protocol. The new QAPISchemaObjectTypeMember.check_clash() is merely a
helper for QAPISchemaObjectType.check().
The two .check_clash() you add (one in this patch, one in the previous
one) are different: both contain calls of QAPISchemaObjectType.check().
I feel the .check() calls are too important to be buried deep like that.
I'd stick to prior practice and put the .check() calls right into
.check(). Obviously, the .check_clash() methods may only called after
.check() then, but that's nothing new.
Fixup for your previous patch:
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 4c56935..357127d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1065,7 +1065,6 @@ class QAPISchemaObjectTypeVariants(object):
vseen = dict(seen)
assert isinstance(v.type, QAPISchemaObjectType)
assert not v.type.variants # not implemented
- v.type.check(schema)
for m in v.type.members:
m.check_clash(vseen)
@@ -1077,6 +1076,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def check(self, schema, tag_type):
QAPISchemaObjectTypeMember.check(self, schema)
assert self.name in tag_type.values
+ self.type.check(schema)
# This function exists to support ugly simple union special cases
# TODO get rid of them, and drop the function
This patch with conflicts resolved and the same idea worked in then
becomes:
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 357127d..f443ec5 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -981,10 +981,8 @@ class QAPISchemaObjectType(QAPISchemaType):
if self._base_name:
self.base = schema.lookup_type(self._base_name)
assert isinstance(self.base, QAPISchemaObjectType)
- assert not self.base.variants # not implemented
self.base.check(schema)
- for m in self.base.members:
- m.check_clash(seen)
+ self.base.check_clash(schema, seen)
for m in self.local_members:
m.check(schema)
m.check_clash(seen)
@@ -994,6 +992,11 @@ class QAPISchemaObjectType(QAPISchemaType):
assert self.variants.tag_member in self.members
self.variants.check_clash(schema, seen)
+ def check_clash(self, schema, seen):
+ assert not self.variants # not implemented
+ for m in self.members:
+ m.check_clash(seen)
+
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
return self.name[0] == ':'
@@ -1062,11 +1065,8 @@ class QAPISchemaObjectTypeVariants(object):
for v in self.variants:
# Reset seen map for each variant, since qapi names from one
# branch do not affect another branch
- vseen = dict(seen)
assert isinstance(v.type, QAPISchemaObjectType)
- assert not v.type.variants # not implemented
- for m in v.type.members:
- m.check_clash(vseen)
+ v.type.check_clash(schema, dict(seen))
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
Your two check_clash() are then simple helpers, just like mine.
^ permalink raw reply related [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-09 18:42 ` Markus Armbruster
@ 2015-11-10 11:57 ` Markus Armbruster
2015-11-11 22:48 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-10 11:57 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel
Markus Armbruster <armbru@redhat.com> writes:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 11/09/2015 02:59 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> On 11/06/2015 09:03 AM, Markus Armbruster wrote:
>>>>> Eric Blake <eblake@redhat.com> writes:
>>> [...]
>>>>>> Hopefully, we are converging on something that will be ready
>>>>>> for a pull request, especially for the earlier patches of this
>>>>>> subset.
>>>>>
>>>>> I guess you mean PATCH 01-12. I had a few questions, but the most
>>>>> likely outcome seems to be minor touchups I could apply in my tree.
>>>>>
>>>>> I'm okay with trying to get more patches in, but let's get these out of
>>>>> the way meanwhile.
>>>>
>>>> Yes, 01-12 seems like a good first set, if you want to make those
>>>> touchups (I've supplied some potential text improvements in reply to
>>>> some of your comments); I'm happy, as always, to take a peek over your
>>>> staging repo to double check what you are prepping for the pull request.
>>>
>>> Please have a look at my qapi-next branch.
>>
>> Close, but commit 5f72bb85 on that branch is missing the one-line summary:
>>
>> qapi: Simplify error cleanup in test-qmp-*
>
> Fixed & sent pull request, thanks!
Pulled.
I think I'll close the QAPI floodgates for 2.5 now. Bug fixes are of
course exempted, and if we find something that impacts ABI, I'm willing
to consider patches. Work on our backlog can continue uninterrupted;
I'm happy to collect patches that are ready, and will take care of
getting them into master once 2.6 opens.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-10 9:15 ` Markus Armbruster
@ 2015-11-10 13:19 ` Eric Blake
2015-11-10 14:43 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-10 13:19 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 4234 bytes --]
On 11/10/2015 02:15 AM, Markus Armbruster wrote:
>> On the other hand, we've been arguing that check() should populate
>> everything after construction prior to anything else being run; and not
>> running Variant.type.check() during Variants.check() of flat unions
>> feels like we may have a hole (a flat union will have to inline its
>> types to the overall JSON object, and inlining types requires access to
>> type.members - but as written, we aren't populating them until
>> Variants.check_clash()). I can play with hoisting the type.check() out
>> of type.check_clash() and instead keep base.check() in type.check(), and
>> add variant.type.check() in Variants.check() (but only for unions, not
>> for alternates), if you are interested.
>
> My "qapi: Factor out QAPISchemaObjectTypeMember.check_clash()" adds
> QAPISchemaObjectTypeMember.check_clash() without changing the common
> protocol. The new QAPISchemaObjectTypeMember.check_clash() is merely a
> helper for QAPISchemaObjectType.check().
>
> The two .check_clash() you add (one in this patch, one in the previous
> one) are different: both contain calls of QAPISchemaObjectType.check().
>
> I feel the .check() calls are too important to be buried deep like that.
> I'd stick to prior practice and put the .check() calls right into
> .check(). Obviously, the .check_clash() methods may only called after
> .check() then, but that's nothing new.
>
> Fixup for your previous patch:
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 4c56935..357127d 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -1065,7 +1065,6 @@ class QAPISchemaObjectTypeVariants(object):
> vseen = dict(seen)
> assert isinstance(v.type, QAPISchemaObjectType)
> assert not v.type.variants # not implemented
> - v.type.check(schema)
> for m in v.type.members:
> m.check_clash(vseen)
>
> @@ -1077,6 +1076,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> def check(self, schema, tag_type):
> QAPISchemaObjectTypeMember.check(self, schema)
> assert self.name in tag_type.values
> + self.type.check(schema)
>
Won't quite work. You are right that we must call
self.type.check(schema) for variants used by a union; but calling it for
ALL variants used by an alternate is wrong, because self.type for at
least one branch of an alternate will not be an instance of
QAPISchemaObjectType. However, I'm currently testing whether it is safe
to check to just blindly check an object branch of an alternate, if
present (and that should not lead to cycles, since alternates have no
base class and since we don't allow one alternate type as a variant of
another alternate), in which case the fixup for 23/30 is more like:
diff --git i/scripts/qapi.py w/scripts/qapi.py
index a005c87..25fa642 100644
--- i/scripts/qapi.py
+++ w/scripts/qapi.py
@@ -1065,7 +1065,6 @@ class QAPISchemaObjectTypeVariants(object):
vseen = dict(seen)
assert isinstance(v.type, QAPISchemaObjectType)
assert not v.type.variants # not implemented
- v.type.check(schema)
for m in v.type.members:
m.check_clash(vseen)
@@ -1077,6 +1076,8 @@ class
QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def check(self, schema, tag_type):
QAPISchemaObjectTypeMember.check(self, schema)
assert self.name in tag_type.values
+ if isinstance(self.type, QAPISchemaObjectType):
+ self.type.check(schema)
# This function exists to support ugly simple union special cases
# TODO get rid of them, and drop the function
@@ -1098,6 +1099,8 @@ class QAPISchemaAlternateType(QAPISchemaType):
def check(self, schema):
self.variants.tag_member.check(schema)
+ # Not calling self.variants.check_clash(), because there's
+ # nothing to clash with
self.variants.check(schema, {})
def json_type(self):
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply related [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-10 8:30 ` Markus Armbruster
@ 2015-11-10 13:24 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-10 13:24 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 3394 bytes --]
On 11/10/2015 01:30 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 11/09/2015 05:56 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> Right now, our ad hoc parser ensures that we cannot have a
>>>> flat union that introduces any qapi member names that would
>>>> conflict with the non-variant qapi members already present
>>>> from the union's base type (see flat-union-clash-member.json).
>>>> We want QAPISchemaObjectType.check() to make the same check,
>>>> so we can later reduce some of the ad hoc checks.
>>>>
>>
>>>> In general, a type used as a branch of a flat union cannot
>>>> also be the base type of the flat union, so even though we are
>>>> adding a call to variant.type.check() in order to populate
>>>> variant.type.members, this is merely a case of gaining
>>>> topological sorting of how types are visited (and type.check()
>>>> is already set up to allow multiple calls due to base types).
>>>
>>> Yes, a type cannot contain itself, neither as base nor as variant.
>>>
>>> We have tests covering attempts to do the former
>>> (struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
>
> Actually, these are just local, unpublished tests. They both make
> check_member_clash() recurse infinitely.
>
> # Direct inheritance loop
> # FIXME triggers infinite recursion
> { 'struct': 'Loopy', 'base': 'Loopy',
> 'data': {} }
Okay, I should add that into my pending patch that cleans up base loops,
>
> # we reject a loop in base classes
> { 'struct': 'Base1', 'base': 'Base2', 'data': {} }
> { 'struct': 'Base2', 'base': 'Base1', 'data': {} }
>
> The latter is actually yours, proposed as base-cycle.json in
> Subject: qapi: Detect collisions in C member names
> Message-Id: <1442872682-6523-17-git-send-email-eblake@redhat.com>
and yes, that one is still in my queue for subset D:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg07001.html
and I may indeed at a test for reusing the base type of a flat union as
one of the branches of the same union, depending on whether it uncovers
anything different.
>
> If I disable the recursive call, the cycle detection in
> QAPISchemaObjectType.check() is reached, and works.
>
> Completing the move of clash detection to check() methods should improve
> things from "accidental infinite recursion" to "intentional assertion
> failure", because it should get rid of check_member_clash() and should
> not break the cycle detection.
>
> Then we can turn the assertion into a proper error message, and add the
> tests.
Yep, that's what is pending in my queue, just further out than subset C.
Doesn't matter if it misses 2.5 (the bug is real, but is only triggered
by bad .json code, and we aren't going to add any bad .json code between
now and 2.5).
>> But you have me curious if this collision is still caught when the ad
>> hoc tests are gone. If so, great; if not, I'll add a test here. (I'll
>> know later when I get through rebasing to all of your comments.)
Still true - I'm still plowing through earlier patches before deciding
if my 'qapi: Detect base class loops' also needs to detect flat union loops.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash()
2015-11-10 13:19 ` Eric Blake
@ 2015-11-10 14:43 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-10 14:43 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/10/2015 02:15 AM, Markus Armbruster wrote:
>
>>> On the other hand, we've been arguing that check() should populate
>>> everything after construction prior to anything else being run; and not
>>> running Variant.type.check() during Variants.check() of flat unions
>>> feels like we may have a hole (a flat union will have to inline its
>>> types to the overall JSON object, and inlining types requires access to
>>> type.members - but as written, we aren't populating them until
>>> Variants.check_clash()). I can play with hoisting the type.check() out
>>> of type.check_clash() and instead keep base.check() in type.check(), and
>>> add variant.type.check() in Variants.check() (but only for unions, not
>>> for alternates), if you are interested.
>>
>> My "qapi: Factor out QAPISchemaObjectTypeMember.check_clash()" adds
>> QAPISchemaObjectTypeMember.check_clash() without changing the common
>> protocol. The new QAPISchemaObjectTypeMember.check_clash() is merely a
>> helper for QAPISchemaObjectType.check().
>>
>> The two .check_clash() you add (one in this patch, one in the previous
>> one) are different: both contain calls of QAPISchemaObjectType.check().
>>
>> I feel the .check() calls are too important to be buried deep like that.
>> I'd stick to prior practice and put the .check() calls right into
>> .check(). Obviously, the .check_clash() methods may only called after
>> .check() then, but that's nothing new.
>>
>> Fixup for your previous patch:
>>
>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>> index 4c56935..357127d 100644
>> --- a/scripts/qapi.py
>> +++ b/scripts/qapi.py
>> @@ -1065,7 +1065,6 @@ class QAPISchemaObjectTypeVariants(object):
>> vseen = dict(seen)
>> assert isinstance(v.type, QAPISchemaObjectType)
>> assert not v.type.variants # not implemented
>> - v.type.check(schema)
>> for m in v.type.members:
>> m.check_clash(vseen)
>>
>> @@ -1077,6 +1076,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
>> def check(self, schema, tag_type):
>> QAPISchemaObjectTypeMember.check(self, schema)
>> assert self.name in tag_type.values
>> + self.type.check(schema)
>>
>
> Won't quite work. You are right that we must call
> self.type.check(schema) for variants used by a union; but calling it for
> ALL variants used by an alternate is wrong, because self.type for at
> least one branch of an alternate will not be an instance of
> QAPISchemaObjectType. However, I'm currently testing whether it is safe
> to check to just blindly check an object branch of an alternate, if
> present (and that should not lead to cycles, since alternates have no
> base class and since we don't allow one alternate type as a variant of
> another alternate), in which case the fixup for 23/30 is more like:
>
> diff --git i/scripts/qapi.py w/scripts/qapi.py
> index a005c87..25fa642 100644
> --- i/scripts/qapi.py
> +++ w/scripts/qapi.py
> @@ -1065,7 +1065,6 @@ class QAPISchemaObjectTypeVariants(object):
> vseen = dict(seen)
> assert isinstance(v.type, QAPISchemaObjectType)
> assert not v.type.variants # not implemented
> - v.type.check(schema)
> for m in v.type.members:
> m.check_clash(vseen)
>
> @@ -1077,6 +1076,8 @@ class
> QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> def check(self, schema, tag_type):
> QAPISchemaObjectTypeMember.check(self, schema)
> assert self.name in tag_type.values
> + if isinstance(self.type, QAPISchemaObjectType):
> + self.type.check(schema)
>
> # This function exists to support ugly simple union special cases
> # TODO get rid of them, and drop the function
> @@ -1098,6 +1099,8 @@ class QAPISchemaAlternateType(QAPISchemaType):
>
> def check(self, schema):
> self.variants.tag_member.check(schema)
> + # Not calling self.variants.check_clash(), because there's
> + # nothing to clash with
> self.variants.check(schema, {})
>
> def json_type(self):
Makes sense to me.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-10 5:16 ` Eric Blake
2015-11-10 8:30 ` Markus Armbruster
@ 2015-11-10 23:37 ` Eric Blake
2015-11-11 9:50 ` Markus Armbruster
1 sibling, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-10 23:37 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1507 bytes --]
On 11/09/2015 10:16 PM, Eric Blake wrote:
>> We have tests covering attempts to do the former
>> (struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
>> see, we don't have tests covering the latter. Do we catch it?
>
> Yes, at least by virtue of the ad hoc tests: attempting to reuse a base
> type of the flat union as a variant member will cause the qapi members
> of the base type to appear more than once in the JSON object (that is,
> the checks that reject flat-union-clash-member.json would also reject
> this scenario). To test:
>
> diff --git i/tests/qapi-schema/qapi-schema-test.json
> w/tests/qapi-schema/qapi-schema-test.json
> index 44638da..16b2ffb 100644
> --- i/tests/qapi-schema/qapi-schema-test.json
> +++ w/tests/qapi-schema/qapi-schema-test.json
> @@ -67,7 +67,7 @@
> 'discriminator': 'enum1',
> 'data': { 'value1' : 'UserDefA',
> 'value2' : 'UserDefB',
> - 'value3' : 'UserDefB' } }
> + 'value3' : 'UserDefUnionBase' } }
Another test I just tried is creating a flat union with:
empty -> base -> union
then use empty as one of the union branches. In that case, there is no
conflict (although base is included twice, neither inclusion adds
members to the JSON object; and the inclusion is not circular so things
compile just fine). So probably not worth adding a test for it.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member
2015-11-09 14:26 ` Markus Armbruster
@ 2015-11-11 0:17 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-11 0:17 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 4070 bytes --]
On 11/09/2015 07:26 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Future commits will migrate semantic checking away from parsing
>> and over to the various QAPISchema*.check() methods. But to
>> report an error message about an incorrect semantic use of a
>> member of an object type, it helps to know which type, command,
>> or event owns the member. In particular, when a member is
>> inherited from a base type, it is desirable to associate the
>> member name with the base type (and not the type calling
>> member.check()).
>>
>>
>> + def _pretty_owner(self):
>> + # See QAPISchema._make_implicit_object_type() - reverse the
>> + # mapping there to create a nice human-readable description
>
> This comment confused me briefly. It applies to the following
> conditionals if-part, but not its else-part. Move it into the if-part?
Done.
>> @@ -1082,9 +1117,11 @@ class QAPISchemaAlternateType(QAPISchemaType):
>> QAPISchemaType.__init__(self, name, info)
>> assert isinstance(variants, QAPISchemaObjectTypeVariants)
>> assert not variants.tag_name
>> + variants.set_owner(name)
>> self.variants = variants
>>
>> def check(self, schema):
>> + self.variants.tag_member.set_owner(self.name)
>> self.variants.tag_member.check(schema)
>> self.variants.check(schema, {})
>>
>
> Odd: all other .set_owner() calls are in .__init__() or .set_owner().
> Can we move this one to __init__() for consistency?
Yes, done.
>
> I think we can: __init__() requires its variants argument to have a
> tag_member (it even asserts not variants.tag_name).
>
>> @@ -1212,6 +1249,7 @@ class QAPISchema(object):
>> def _make_implicit_object_type(self, name, info, role, members):
>> if not members:
>> return None
>> + # See also QAPISchemaObjectTypeMember.describe()
>
> Should this point to ._pretty_owner() instead?
Good call.
>
>> name = ':obj-%s-%s' % (name, role)
>> if not self.lookup_entity(name, QAPISchemaObjectType):
>> self._def_entity(QAPISchemaObjectType(name, info, None,
>> @@ -1315,7 +1353,7 @@ class QAPISchema(object):
>> data = expr.get('data')
>> if isinstance(data, OrderedDict):
>> data = self._make_implicit_object_type(
>> - name, info, 'arg', self._make_members(data, info))
>> + name, info, 'data', self._make_members(data, info))
>
> This is necessary only to make .pretty_owner() say 'data of EVENT_NAME'
> instead of 'argument of EVENT_NAME'. Do we really need different
> wording for commands and events?
>
> I'd make it say 'parameter of' for both commands and events. I
> habitually use "parameter" for formal parameters, and "argument" for
> actual arguments. Guess I've worked with the C standard too much...
'parameter' sounds better for both, at which point the messages no
longer differ,...
>
> For what it's worth, the QAPI schema language uses 'data' for both (and
> many other things, too), and the introspection schema uses 'arg-type'
> for both.
>
>> self._def_entity(QAPISchemaEvent(name, info, data))
>>
>> def _def_exprs(self):
>> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
>> index 786024e..f78ef04 100644
>> --- a/tests/qapi-schema/qapi-schema-test.out
>> +++ b/tests/qapi-schema/qapi-schema-test.out
>> @@ -1,9 +1,9 @@
>> object :empty
>> -object :obj-EVENT_C-arg
>> +object :obj-EVENT_C-data
>> member a: int optional=True
>> member b: UserDefOne optional=True
>> member c: str optional=False
>> -object :obj-EVENT_D-arg
>> +object :obj-EVENT_D-data
...so I no longer need to rename the implicit struct for events to allow
the differentiation. v11 is thus a smaller patch :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names
2015-11-09 15:17 ` Markus Armbruster
@ 2015-11-11 0:34 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-11 0:34 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 4651 bytes --]
On 11/09/2015 08:17 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Detect attempts to declare two object members that would result
>> in the same C member name, by keying the 'seen' dictionary off
>> of the C name rather than the qapi name. It also requires passing
>> info through the check_clash() methods.
>>
>> This addresses a TODO and fixes the previously-broken
>> args-name-clash test. The resulting error message demonstrates
>> the utility of the .describe() method added previously. No change
>> to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> +++ b/scripts/qapi.py
>> @@ -975,21 +975,24 @@ class QAPISchemaObjectType(QAPISchemaType):
>> seen = OrderedDict()
>> if self._base_name:
>> self.base = schema.lookup_type(self._base_name)
>> - self.base.check_clash(schema, seen)
>> + self.base.check_clash(schema, self.info, seen)
>
> Note that if this one reports an error for self.info, something went
> wrong. We first run self.base.check(), which we survive only when
> self.base doesn't have clashing members. Would be easier to see if
> self.base.check() wasn't hiding in self.base.check_clash().
Hmm. I could pass None as a way of asserting that it won't fail. But I
think that's a bit more confusing, so I'll probably just leave it alone.
>> - def check_clash(self, schema, seen):
>> + # Check that the members of this type do not cause duplicate JSON fields,
>> + # and update seen to track the members seen so far. Report any errors
>> + # on behalf of info, which is not necessarily self.info
>
> Do we actually need info != self.info? If yes, test case? If no,
> should the comment be adjusted?
The comment is correct. For a flat union, if we have a variant
associated with type 'Foo', and type 'Foo' has no clashes itself, but
one of its members duplicates a member already present from the base of
union, then we are calling Foo.check_clash(schema, Union.info, seen), so
that the error message points to the location of Union (rather than Foo)
as the place in the .json file introducing the clash.
flat-union-clash-member.json tests this; it's just that we first have to
remove ad hoc testing before we can trigger this case. It's on my
queue, just not in this particular posting of subset C:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg07000.html
>> @@ -1035,10 +1038,13 @@ class QAPISchemaObjectTypeMember(object):
>> self.type = schema.lookup_type(self._type_name)
>> assert self.type
>>
>> - def check_clash(self, seen):
>> - # TODO change key of seen from QAPI name to C name
>> - assert self.name not in seen
>> - seen[self.name] = self
>> + def check_clash(self, info, seen):
>> + name = c_name(self.name)
>
> I'd call it cname. Matter of taste, of course.
Sure, that works. I first tried calling it c_name, but then python
thought I was trying to redefine the global c_name().
>>
>> - def check_clash(self, schema, seen):
>> + def check_clash(self, schema, info, seen):
>> for v in self.variants:
>> # Reset seen map for each variant, since qapi names from one
>> # branch do not affect another branch
>> - v.type.check_clash(schema, dict(seen))
>> + v.type.check_clash(schema, info, dict(seen))
>
> Since we can't inherit variant members, info is always the owning object
> type's info.
Exactly. We want to output the passed-in info, and NOT v.type.info, so
that errors are reported on behalf of the union that owns the variant,
rather than at the location of the variant's type which independently
passed .check() and therefore has no internal collisions.
And someday we might be able to inherit from objects with variants, but
that's a lot further down the pipeline (if at all).
>> +++ b/tests/qapi-schema/args-name-clash.json
>> @@ -1,5 +1,5 @@
>> # C member name collision
>> -# FIXME - This parses, but fails to compile, because the C struct is given
>> -# two 'a_b' members. Either reject this at parse time, or munge the C names
>> -# to avoid the collision.
>> +# Reject members that clash when mapped to C names (we would have two 'a_b'
>> +# members). It would also be possible to munge the C names to avoid the
>> +# collision, but unlikely to be worth the effort.
>
> I'd drop the second sentence.
Done.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union
2015-11-09 15:22 ` Markus Armbruster
@ 2015-11-11 2:50 ` Eric Blake
2015-11-11 10:19 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-11 2:50 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Paolo Bonzini, qemu-devel, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 3424 bytes --]
On 11/09/2015 08:22 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> When qapi type CpuInfo was originally created for 0.14, we had
>> no notion of a flat union, and instead just listed a bunch of
>> optional fields with documentation about the mutually-exclusive
>> choice of which instruction pointer field(s) would be provided
>> for a given architecture. But now that we have flat unions and
>> introspection, it is better to segregate off which fields will
>> be provided according to the actual architecture. With this in
>> place, we no longer need the fields to be optional, because the
>> choice of the discriminator serves that role.
>>
>> This has an additional benefit: the old all-in-one struct was
>> the only place in the code base that had a case-sensitive
>> naming of members 'pc' vs. 'PC'. Separating these spellings
>> into different branches of the flat union will allow us to add
>> restrictions against future case-insensitive collisions, and
>> make it possible for a future qemu to decide whether to enable
>> case-insensitive QMP.
>> ##
>> # @query-cpus:
>
> CpuInfo is only used as return type of query-cpus.
>
> If I understand the change correctly, the value of query-cpus is not
> changed at all. Its introspection, however, is.
Almost. The output of query-cpus gains a new 'arch':'FOO' string per
cpu, saying which branch of the flat union is in use. But as it is
output-only, adding a new output field shouldn't break any existing
clients (because QMP already documents that clients should ignore new
fields); and since the struct is output-only, it can't break any input
fields.
I probably can spell that out better in the commit message, though.
Oh, and I need to update qmp-commands.hx, as part of my v11 spin.
>
> Do we need this to make 2.5?
It's true that the introspection will change (instead of seeing flat
optional members, you now have to chase down variants). But I don't
think it is pressing enough to rush into 2.5; the change is
backwards-compatible no matter when we do it, and introspection clients
are going to have to get used to the idea of chasing down different
paths to find all members of a struct (that is, if we don't change this
until 2.6, I'm sure it won't be the last case of introspection changing
while keeping QMP wire format back-compatible as formerly non-variant
fields become variant).
>
> Any other messy optionals that should really be (flat) unions?
>
Possibly.
/me goes and audits all 0.14 interfaces...
add_client is an input command, but it would be nicer as a flat union
(we can't do that until commands can take unions; but that happens as
part of my pending queue for netdev_add), so that's not going to make 2.5.
client_migrate_info looks odd, since it requires 'protocol':'str' to be
the value 'spice'. Probably several functions that take or produce a
finite set of strings that should be using enums, but conversion from
'str' to 'enum' is backwards-compatible according to QMP wire format.
Doing it sooner rather than later makes introspection nicer.
But I didn't spot any other obvious commands from that far back with
mutually-exclusive optional parameters, and certainly nothing else with
case clashes.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches
2015-11-10 23:37 ` Eric Blake
@ 2015-11-11 9:50 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-11 9:50 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 10:16 PM, Eric Blake wrote:
>
>>> We have tests covering attempts to do the former
>>> (struct-cycle-direct.json, struct-cycle-indirect.json). As far as I can
>>> see, we don't have tests covering the latter. Do we catch it?
>>
>> Yes, at least by virtue of the ad hoc tests: attempting to reuse a base
>> type of the flat union as a variant member will cause the qapi members
>> of the base type to appear more than once in the JSON object (that is,
>> the checks that reject flat-union-clash-member.json would also reject
>> this scenario). To test:
>>
>> diff --git i/tests/qapi-schema/qapi-schema-test.json
>> w/tests/qapi-schema/qapi-schema-test.json
>> index 44638da..16b2ffb 100644
>> --- i/tests/qapi-schema/qapi-schema-test.json
>> +++ w/tests/qapi-schema/qapi-schema-test.json
>> @@ -67,7 +67,7 @@
>> 'discriminator': 'enum1',
>> 'data': { 'value1' : 'UserDefA',
>> 'value2' : 'UserDefB',
>> - 'value3' : 'UserDefB' } }
>> + 'value3' : 'UserDefUnionBase' } }
>
> Another test I just tried is creating a flat union with:
>
> empty -> base -> union
>
> then use empty as one of the union branches. In that case, there is no
> conflict (although base is included twice, neither inclusion adds
> members to the JSON object; and the inclusion is not circular so things
> compile just fine). So probably not worth adding a test for it.
Member name clashes occur because:
1. Two separately defined members happen to have a clashing name.
Whether the members are local or inherited, variant or non-variant
doesn't matter.
2. The same type gets spliced in twice, and conflict.
By "spliced", I mean the members get included. A bit like unboxed,
but also "unwrapped".
Base types and variant types get spliced in. Variant types don't
conflict with each other. Empty type can't conflict (outlawing it
anyway would be pointless). Therefore, the only conflicting splice
is a non-empty type occuring both as (base type of the) base type and
as (base type of a) variant type.
We detect this kind of clash exactly like 1. That's fine, because
the resulting error messages are plainly good enough.
3. Type contains itself.
The type gets spliced into itself, or has itself as unboxed member.
We don't unbox object members so far. That leaves the spliced ones:
base types and variants. Thus, a type contains itself if it's its
own base or one of its own variants, or a direct or indirect base of
either.
We detect this kind of clash separately, when we guard against
infinite check() recursion. Reporting it right there will be
easiest.
We want to cover all this with tests. I guess we're lacking only 3.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union
2015-11-11 2:50 ` Eric Blake
@ 2015-11-11 10:19 ` Markus Armbruster
2015-11-11 15:40 ` Eric Blake
0 siblings, 1 reply; 85+ messages in thread
From: Markus Armbruster @ 2015-11-11 10:19 UTC (permalink / raw)
To: Eric Blake; +Cc: Paolo Bonzini, qemu-devel, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> On 11/09/2015 08:22 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> When qapi type CpuInfo was originally created for 0.14, we had
>>> no notion of a flat union, and instead just listed a bunch of
>>> optional fields with documentation about the mutually-exclusive
>>> choice of which instruction pointer field(s) would be provided
>>> for a given architecture. But now that we have flat unions and
>>> introspection, it is better to segregate off which fields will
>>> be provided according to the actual architecture. With this in
>>> place, we no longer need the fields to be optional, because the
>>> choice of the discriminator serves that role.
>>>
>>> This has an additional benefit: the old all-in-one struct was
>>> the only place in the code base that had a case-sensitive
>>> naming of members 'pc' vs. 'PC'. Separating these spellings
>>> into different branches of the flat union will allow us to add
>>> restrictions against future case-insensitive collisions, and
>>> make it possible for a future qemu to decide whether to enable
>>> case-insensitive QMP.
>
>>> ##
>>> # @query-cpus:
>>
>> CpuInfo is only used as return type of query-cpus.
>>
>> If I understand the change correctly, the value of query-cpus is not
>> changed at all. Its introspection, however, is.
>
> Almost. The output of query-cpus gains a new 'arch':'FOO' string per
> cpu, saying which branch of the flat union is in use. But as it is
> output-only, adding a new output field shouldn't break any existing
> clients (because QMP already documents that clients should ignore new
> fields); and since the struct is output-only, it can't break any input
> fields.
Understood.
> I probably can spell that out better in the commit message, though.
Yes, please
> Oh, and I need to update qmp-commands.hx, as part of my v11 spin.
Yes, please.
>> Do we need this to make 2.5?
>
> It's true that the introspection will change (instead of seeing flat
> optional members, you now have to chase down variants). But I don't
> think it is pressing enough to rush into 2.5; the change is
> backwards-compatible no matter when we do it, and introspection clients
> are going to have to get used to the idea of chasing down different
> paths to find all members of a struct (that is, if we don't change this
> until 2.6, I'm sure it won't be the last case of introspection changing
> while keeping QMP wire format back-compatible as formerly non-variant
> fields become variant).
We can mess with the schema as long as the QMP wire format stays
compatible.
With introspection, schema changes get exposed in QMP that weren't
exposed before. Begs the question whether the ABI promise extends to
the introspection value.
I think we don't want to extend it, at least for now. In other words,
we refuse to put additional constraints on schema changes to keep the
introspection value stable. Makes introspection a bit harder to use.
Not ideal, but better than committing to constraints we don't even fully
understand, yet.
I think we should spell this out introspection documentation.
Our only example so far is converting between optional members and flat
unions. Can we think of others? Hmm, you found one below.
>> Any other messy optionals that should really be (flat) unions?
>>
>
> Possibly.
> /me goes and audits all 0.14 interfaces...
>
> add_client is an input command, but it would be nicer as a flat union
> (we can't do that until commands can take unions; but that happens as
> part of my pending queue for netdev_add), so that's not going to make 2.5.
Since the command parameters are just an object type, this is merely
another case of converting between optional members and flat unions.
> client_migrate_info looks odd, since it requires 'protocol':'str' to be
> the value 'spice'. Probably several functions that take or produce a
> finite set of strings that should be using enums, but conversion from
> 'str' to 'enum' is backwards-compatible according to QMP wire format.
> Doing it sooner rather than later makes introspection nicer.
This is another example: converting between string and enum. Going from
string to enum only adds information, and should be easy enough for
clients. Going the other way removes information, and probably should
not be done.
> But I didn't spot any other obvious commands from that far back with
> mutually-exclusive optional parameters, and certainly nothing else with
> case clashes.
Okay. I guess finding all the examples that matter may take a few
development cycles. Perhaps we want to commit to some additional
compatibility promises for introspection then.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union
2015-11-11 10:19 ` Markus Armbruster
@ 2015-11-11 15:40 ` Eric Blake
2015-11-11 17:00 ` Markus Armbruster
0 siblings, 1 reply; 85+ messages in thread
From: Eric Blake @ 2015-11-11 15:40 UTC (permalink / raw)
To: Markus Armbruster; +Cc: Paolo Bonzini, qemu-devel, Luiz Capitulino
[-- Attachment #1: Type: text/plain, Size: 2516 bytes --]
On 11/11/2015 03:19 AM, Markus Armbruster wrote:
>>> Do we need this to make 2.5?
>>
>> It's true that the introspection will change (instead of seeing flat
>> optional members, you now have to chase down variants). But I don't
>> think it is pressing enough to rush into 2.5; the change is
>> backwards-compatible no matter when we do it, and introspection clients
>> are going to have to get used to the idea of chasing down different
>> paths to find all members of a struct (that is, if we don't change this
>> until 2.6, I'm sure it won't be the last case of introspection changing
>> while keeping QMP wire format back-compatible as formerly non-variant
>> fields become variant).
>
> We can mess with the schema as long as the QMP wire format stays
> compatible.
>
> With introspection, schema changes get exposed in QMP that weren't
> exposed before. Begs the question whether the ABI promise extends to
> the introspection value.
>
> I think we don't want to extend it, at least for now. In other words,
> we refuse to put additional constraints on schema changes to keep the
> introspection value stable. Makes introspection a bit harder to use.
> Not ideal, but better than committing to constraints we don't even fully
> understand, yet.
>
> I think we should spell this out introspection documentation.
Sounds like I need to prep a doc patch for 2.5 then.
>
> Our only example so far is converting between optional members and flat
> unions. Can we think of others? Hmm, you found one below.
Converting from optional non-variant to variant, converting from string
to enum, converting from single type to alternate. Might be other
conversions, and the set of conversions for input types may differ from
those of output conversions.
> Okay. I guess finding all the examples that matter may take a few
> development cycles. Perhaps we want to commit to some additional
> compatibility promises for introspection then.
Maybe, but until clients are no longer worried about older clients, it
may be several years before taking advantage of whatever we later
document (as it is, upstream libvirt is just now bumping minimum qemu
support up to 0.12 [thanks to RHEL 6] and discarding older qemu from
RHEL 5 - quite a few years behind upstream qemu 2.5, although it is
finding a lot of code in libvirt that is no longer necessary).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union
2015-11-11 15:40 ` Eric Blake
@ 2015-11-11 17:00 ` Markus Armbruster
0 siblings, 0 replies; 85+ messages in thread
From: Markus Armbruster @ 2015-11-11 17:00 UTC (permalink / raw)
To: Eric Blake; +Cc: Paolo Bonzini, qemu-devel, Luiz Capitulino
Eric Blake <eblake@redhat.com> writes:
> On 11/11/2015 03:19 AM, Markus Armbruster wrote:
>
>>>> Do we need this to make 2.5?
>>>
>>> It's true that the introspection will change (instead of seeing flat
>>> optional members, you now have to chase down variants). But I don't
>>> think it is pressing enough to rush into 2.5; the change is
>>> backwards-compatible no matter when we do it, and introspection clients
>>> are going to have to get used to the idea of chasing down different
>>> paths to find all members of a struct (that is, if we don't change this
>>> until 2.6, I'm sure it won't be the last case of introspection changing
>>> while keeping QMP wire format back-compatible as formerly non-variant
>>> fields become variant).
>>
>> We can mess with the schema as long as the QMP wire format stays
>> compatible.
>>
>> With introspection, schema changes get exposed in QMP that weren't
>> exposed before. Begs the question whether the ABI promise extends to
>> the introspection value.
>>
>> I think we don't want to extend it, at least for now. In other words,
>> we refuse to put additional constraints on schema changes to keep the
>> introspection value stable. Makes introspection a bit harder to use.
>> Not ideal, but better than committing to constraints we don't even fully
>> understand, yet.
>>
>> I think we should spell this out introspection documentation.
>
> Sounds like I need to prep a doc patch for 2.5 then.
Yes, please!
>> Our only example so far is converting between optional members and flat
>> unions. Can we think of others? Hmm, you found one below.
>
> Converting from optional non-variant to variant, converting from string
> to enum, converting from single type to alternate. Might be other
> conversions, and the set of conversions for input types may differ from
> those of output conversions.
>
>> Okay. I guess finding all the examples that matter may take a few
>> development cycles. Perhaps we want to commit to some additional
>> compatibility promises for introspection then.
>
> Maybe, but until clients are no longer worried about older clients, it
> may be several years before taking advantage of whatever we later
> document
That's life.
I fully expect the consumers of QMP introspection to get a few things
wrong initially, requiring their users to upgrade to fixed versions.
That's life, too.
> (as it is, upstream libvirt is just now bumping minimum qemu
> support up to 0.12 [thanks to RHEL 6] and discarding older qemu from
> RHEL 5 - quite a few years behind upstream qemu 2.5, although it is
> finding a lot of code in libvirt that is no longer necessary).
In my opinion, attempting to support old versions of QEMU is a complete
waste of resources unless you actually test them to stop the bit-rot.
RHEL-6 0.12 is by now quite different from upstream 0.12. Unlike
upstream 0.12, it's actually still relevant and useful today.
I encourage you to identify relevant upstream and downstream versions,
keep them tested, and drop support for the rest.
^ permalink raw reply [flat|nested] 85+ messages in thread
* Re: [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C')
2015-11-10 11:57 ` Markus Armbruster
@ 2015-11-11 22:48 ` Eric Blake
0 siblings, 0 replies; 85+ messages in thread
From: Eric Blake @ 2015-11-11 22:48 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel
[-- Attachment #1: Type: text/plain, Size: 962 bytes --]
On 11/10/2015 04:57 AM, Markus Armbruster wrote:
> I think I'll close the QAPI floodgates for 2.5 now. Bug fixes are of
> course exempted, and if we find something that impacts ABI, I'm willing
> to consider patches. Work on our backlog can continue uninterrupted;
> I'm happy to collect patches that are ready, and will take care of
> getting them into master once 2.6 opens.
I think I found something that ought to go into 2.5: we have nothing in
current query-qmp-introspection that shows the set of valid 'ErrorClass'
enum values that a client can expect on error (this is because no
existing commands have a reference to the type). While we are unlikely
to be adding new ErrorClass values any time soon, this feels like
something that should be exposed over the wire.
I'm trying to play with ideas on how best to expose it.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 85+ messages in thread
end of thread, other threads:[~2015-11-11 22:48 UTC | newest]
Thread overview: 85+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-06 6:35 [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 01/30] qapi: Use generated TestStruct machinery in tests Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 02/30] qapi: Strengthen test of TestStructList Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 03/30] qobject: Protect against use-after-free in qobject_decref() Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 04/30] qapi: Share test_init code in test-qmp-input* Eric Blake
2015-11-06 15:17 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 05/30] qapi: Plug leaks in test-qmp-* Eric Blake
2015-11-06 15:21 ` Markus Armbruster
2015-11-06 15:49 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 06/30] qapi: Simplify non-error testing " Eric Blake
2015-11-06 15:36 ` Markus Armbruster
2015-11-06 15:54 ` Eric Blake
2015-11-06 16:24 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 07/30] qapi: Simplify error cleanup " Eric Blake
2015-11-06 15:40 ` Markus Armbruster
2015-11-06 15:59 ` Eric Blake
2015-11-06 16:23 ` Markus Armbruster
2015-11-06 16:32 ` Eric Blake
2015-11-06 17:04 ` [Qemu-devel] [PATCH] fixup! " Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 08/30] qapi: More tests of alternate output Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 09/30] qapi: Test failure in middle of array parse Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 10/30] qapi: More tests of input arrays Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 11/30] qapi: Provide nicer array names in introspection Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 12/30] qapi-introspect: Document lack of sorting Eric Blake
2015-11-06 15:52 ` Markus Armbruster
2015-11-09 20:56 ` Eric Blake
2015-11-10 7:36 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 13/30] qapi: Track simple union tag in object.local_members Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 14/30] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 15/30] qapi-types: Simplify gen_struct_field[s] Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 16/30] qapi: Drop obsolete tag value collision assertions Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 17/30] qapi: Simplify QAPISchemaObjectTypeMember.check() Eric Blake
2015-11-09 12:31 ` Markus Armbruster
2015-11-09 14:44 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 18/30] qapi: Clean up after previous commit Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 19/30] qapi: Fix up commit 7618b91's clash sanity checking change Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 20/30] qapi: Eliminate QAPISchemaObjectType.check() variable members Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 21/30] qapi: Factor out QAPISchemaObjectTypeMember.check_clash() Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 22/30] qapi: Simplify QAPISchemaObjectTypeVariants.check() Eric Blake
2015-11-09 12:38 ` Markus Armbruster
2015-11-10 5:04 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 23/30] qapi: Check for qapi collisions of flat union branches Eric Blake
2015-11-09 12:56 ` Markus Armbruster
2015-11-09 15:13 ` Markus Armbruster
2015-11-10 5:18 ` Eric Blake
2015-11-10 5:16 ` Eric Blake
2015-11-10 8:30 ` Markus Armbruster
2015-11-10 13:24 ` Eric Blake
2015-11-10 23:37 ` Eric Blake
2015-11-11 9:50 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 24/30] qapi: Factor out QAPISchemaObjectType.check_clash() Eric Blake
2015-11-09 13:00 ` Markus Armbruster
2015-11-09 17:36 ` Eric Blake
2015-11-09 19:11 ` Markus Armbruster
2015-11-10 5:22 ` Eric Blake
2015-11-09 14:49 ` Markus Armbruster
2015-11-10 5:32 ` Eric Blake
2015-11-10 9:15 ` Markus Armbruster
2015-11-10 13:19 ` Eric Blake
2015-11-10 14:43 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 25/30] qapi: Hoist tag collision check to Variants.check() Eric Blake
2015-11-09 13:07 ` Markus Armbruster
2015-11-10 5:33 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 26/30] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 27/30] qapi: Track owner of each object member Eric Blake
2015-11-09 14:26 ` Markus Armbruster
2015-11-11 0:17 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 28/30] qapi: Detect collisions in C member names Eric Blake
2015-11-09 15:17 ` Markus Armbruster
2015-11-11 0:34 ` Eric Blake
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 29/30] cpu: Convert CpuInfo into flat union Eric Blake
2015-11-09 15:22 ` Markus Armbruster
2015-11-11 2:50 ` Eric Blake
2015-11-11 10:19 ` Markus Armbruster
2015-11-11 15:40 ` Eric Blake
2015-11-11 17:00 ` Markus Armbruster
2015-11-06 6:35 ` [Qemu-devel] [PATCH v10 30/30] qapi: Forbid case-insensitive clashes Eric Blake
2015-11-09 15:42 ` Markus Armbruster
2015-11-06 16:03 ` [Qemu-devel] [PATCH v10 00/30] qapi member collision (post-introspection cleanups, subset C') Markus Armbruster
2015-11-06 16:08 ` Eric Blake
2015-11-09 9:59 ` Markus Armbruster
2015-11-09 14:43 ` Eric Blake
2015-11-09 18:42 ` Markus Armbruster
2015-11-10 11:57 ` Markus Armbruster
2015-11-11 22:48 ` Eric Blake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.