* [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C)
@ 2015-10-28 17:14 Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 01/17] qapi: Use generated TestStruct machinery in tests Eric Blake
` (16 more replies)
0 siblings, 17 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru
Pending prerequisite: Markus' qapi-next branch
git://repo.or.cz/qemu/armbru.git qapi-next
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg06417.html
Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv8c
as well as part of my branch with the rest of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi
v8 notes:
Minor changes when rebasing to latest, improved commit messages in
a few places, plus the addition of a few new patches. Markus started
reviewing v7, but hadn't gotten very far.
001/17:[----] [--] 'qapi: Use generated TestStruct machinery in tests'
002/17:[----] [--] 'qapi: Strengthen test of TestStructList'
003/17:[----] [--] 'qapi: Provide nicer array names in introspection'
004/17:[----] [--] 'qapi-introspect: Guarantee particular sorting'
005/17:[down] 'qapi: Track simple union tag in object.local_members'
006/17:[down] 'qapi-types: Consolidate gen_struct() and gen_union()'
007/17:[0007] [FC] 'qapi: Rework collision assertions'
008/17:[down] 'qapi: Remove outdated tests related to QMP/branch collisions'
009/17:[down] 'qapi: Add positive tests to qapi-schema-test'
010/17:[----] [-C] 'qapi: Simplify visiting of alternate types'
011/17:[----] [--] 'qapi: Fix alternates that accept 'number' but not 'int''
012/17:[----] [--] 'qapi: Remove dead visitor code'
013/17:[----] [--] 'qapi: Plug leaks in test-qmp-*'
014/17:[----] [-C] 'qapi: Simplify error testing in test-qmp-*'
015/17:[----] [--] 'qapi: Test failure in middle of array parse'
016/17:[----] [--] 'qapi: More tests of input arrays'
017/17:[----] [--] 'qapi: Simplify visits of optional fields'
v7 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04112.html
Patches 1-3 of the previous round have moved to subset B; patch 7
is gone, and a couple of new patches are present in this round. The
latest version of subset B made it much easier to reason about
collision detection (namely, tag values can't collide with QMP values
thanks to a named rather than anonymous union in the C type), and I
like how things turned out. I suspect the hardest part of the review
will be patches 5/14 and 7/14, although none of this has really
had much review in any earlier versions.
v6 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg01980.html
Add some patches and rebase onto work on subset B. Rearrange some
patches from v5 (this set includes 17-20, 23, 25-27). Backport diff
gets a bit confused by one patch title changing.
Not much direct review comments, although some of the changes here
are updated based on comments made on other patches in the v5 series.
Subset D (and more?) will come later.
In v5:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg05410.html
I _did_ rearrange patches to try and group related features:
1-2: Groundwork cleanups
3-5: Add more test cases
6-16: Front-end cleanups
17-18: Introspection output cleanups
19-20: 'alternate' type cleanups
21-29: qapi visitor cleanups
30-45: qapi-ify netdev_add
46: add qapi shorthand for flat unions
Lots of fixes based on additional testing, and rebased to
track other changes that happened in the meantime. The series
is huge; I can split off smaller portions as requested.
In v4:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02580.html
add some more clean up patches
rebase to Markus' recent work
pull in part of Zoltán's work to make netdev_add a flat union,
further enhancing it to be introspectible
I might be able to rearrange some of these patches, or separate
it into smaller independent series, if requested; but I'm
posting now to get review started.
In v3:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02059.html
redo cleanup of dealloc of partial struct
add patches to make all visit_type_*() avoid leaks on failure
add patches to allow boxed command arguments and events
In v2:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00900.html
rebase to Markus' v3 series
rework how comments are emitted for fields inherited from base
additional patches added for deleting colliding 'void *data'
documentation updates to match code changes
v1 was here:
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05266.html
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05325.html
Eric Blake (17):
qapi: Use generated TestStruct machinery in tests
qapi: Strengthen test of TestStructList
qapi: Provide nicer array names in introspection
qapi-introspect: Guarantee particular sorting
qapi: Track simple union tag in object.local_members
qapi-types: Consolidate gen_struct() and gen_union()
qapi: Rework collision assertions
qapi: Remove outdated tests related to QMP/branch collisions
qapi: Add positive tests to qapi-schema-test
qapi: Simplify visiting of alternate types
qapi: Fix alternates that accept 'number' but not 'int'
qapi: Remove dead visitor code
qapi: Plug leaks in test-qmp-*
qapi: Simplify error testing in test-qmp-*
qapi: Test failure in middle of array parse
qapi: More tests of input arrays
qapi: Simplify visits of optional fields
docs/qapi-code-gen.txt | 31 ++-
include/qapi/visitor-impl.h | 27 ++-
include/qapi/visitor.h | 22 +-
qapi/introspect.json | 22 +-
qapi/opts-visitor.c | 2 +-
qapi/qapi-visit-core.c | 141 +++++--------
qapi/qmp-input-visitor.c | 11 +-
qapi/string-input-visitor.c | 3 +-
scripts/qapi-introspect.py | 17 +-
scripts/qapi-types.py | 70 ++-----
scripts/qapi-visit.py | 27 ++-
scripts/qapi.py | 116 ++++++-----
tests/Makefile | 3 -
tests/qapi-schema/alternate-empty.out | 1 -
tests/qapi-schema/flat-union-clash-branch.err | 0
tests/qapi-schema/flat-union-clash-branch.exit | 1 -
tests/qapi-schema/flat-union-clash-branch.json | 18 --
tests/qapi-schema/flat-union-clash-branch.out | 14 --
tests/qapi-schema/flat-union-clash-type.err | 1 -
tests/qapi-schema/flat-union-clash-type.exit | 1 -
tests/qapi-schema/flat-union-clash-type.json | 14 --
tests/qapi-schema/flat-union-clash-type.out | 0
tests/qapi-schema/qapi-schema-test.json | 26 ++-
tests/qapi-schema/qapi-schema-test.out | 46 ++++-
tests/qapi-schema/union-clash-data.out | 1 +
tests/qapi-schema/union-clash-type.err | 1 -
tests/qapi-schema/union-clash-type.exit | 1 -
tests/qapi-schema/union-clash-type.json | 9 -
tests/qapi-schema/union-clash-type.out | 0
tests/qapi-schema/union-empty.out | 1 +
tests/test-qmp-input-strict.c | 108 +++-------
tests/test-qmp-input-visitor.c | 268 +++++++++++--------------
tests/test-qmp-output-visitor.c | 153 ++++----------
tests/test-visitor-serialization.c | 76 ++-----
34 files changed, 503 insertions(+), 729 deletions(-)
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.err
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.json
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.out
delete mode 100644 tests/qapi-schema/flat-union-clash-type.err
delete mode 100644 tests/qapi-schema/flat-union-clash-type.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-type.json
delete mode 100644 tests/qapi-schema/flat-union-clash-type.out
delete mode 100644 tests/qapi-schema/union-clash-type.err
delete mode 100644 tests/qapi-schema/union-clash-type.exit
delete mode 100644 tests/qapi-schema/union-clash-type.json
delete mode 100644 tests/qapi-schema/union-clash-type.out
--
2.4.3
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 01/17] qapi: Use generated TestStruct machinery in tests
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 02/17] qapi: Strengthen test of TestStructList Eric Blake
` (15 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Commit d88f5fd and friends first introduced the various test-qmp-*
tests in 2011, with duplicated hand-rolled TestStruct machinery,
to make sure the qapi visitor interface was tested. Later, commit
4f193e3 in 2013 added a .json file for further testing use by the
files, but without consolidating any of the existing hand-rolled
visitors. And with four copies, subtle differences have crept in,
between the tests themselves (mainly whitespace differences, but
also a question of whether to use NULL or "TestStruct" when
calling visit_start_struct()) and from what the generator produces
(the hand-rolled versions did not cater to partially-allocated
objects, because they did not have a deallocation usage).
Of course, just because the visitor interface is tested does not
mean it is a sane interface; and future patches will be changing
some of the visitor contracts. Rather than having to duplicate
the cleanup work in each copy of the TestStruct visitor, and keep
each hand-rolled copy in sync with what the generator supplies, we
might as well just test what the generator should give us in the
first place.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: improve commit message
v7: rebase on top of subset B v9; defer unrelated trailing whitespace
cleanups to later in series
v6: new patch
---
tests/qapi-schema/qapi-schema-test.json | 6 +++-
tests/qapi-schema/qapi-schema-test.out | 5 +++
tests/test-qmp-input-strict.c | 35 --------------------
tests/test-qmp-input-visitor.c | 34 -------------------
tests/test-qmp-output-visitor.c | 58 ---------------------------------
tests/test-visitor-serialization.c | 34 -------------------
6 files changed, 10 insertions(+), 162 deletions(-)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 48e104b..44638da 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -3,6 +3,9 @@
# This file is a stress test of supported qapi constructs that must
# parse and compile correctly.
+{ 'struct': 'TestStruct',
+ 'data': { 'integer': 'int', 'boolean': 'bool', 'string': 'str' } }
+
# for testing enums
{ 'struct': 'NestedEnumsOne',
'data': { 'enum1': 'EnumOne', # Intentional forward reference
@@ -46,7 +49,8 @@
# dummy struct to force generation of array types not otherwise mentioned
{ 'struct': 'ForceArrays',
- 'data': { 'unused1':['UserDefOne'], 'unused2':['UserDefTwo'] } }
+ 'data': { 'unused1':['UserDefOne'], 'unused2':['UserDefTwo'],
+ 'unused3':['TestStruct'] } }
# for testing unions
# Among other things, test that a name collision between branches does
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index a7e9aab..e20a823 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -92,6 +92,7 @@ object EventStructOne
object ForceArrays
member unused1: UserDefOneList optional=False
member unused2: UserDefTwoList optional=False
+ member unused3: TestStructList optional=False
enum MyEnum []
object NestedEnumsOne
member enum1: EnumOne optional=False
@@ -100,6 +101,10 @@ object NestedEnumsOne
member enum4: EnumOne optional=True
enum QEnumTwo ['value1', 'value2']
prefix QENUM_TWO
+object TestStruct
+ member integer: int optional=False
+ member boolean: bool optional=False
+ member string: str optional=False
object UserDefA
member boolean: bool optional=False
member a_b: int optional=True
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 53a7693..b44184f 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -89,41 +89,6 @@ static Visitor *validate_test_init_raw(TestInputVisitorData *data,
return v;
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, "TestStruct", name, sizeof(TestStruct),
- &err);
- if (err) {
- goto out;
- }
-
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static void test_validate_struct(TestInputVisitorData *data,
const void *unused)
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index de65982..3f6bc4d 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -185,40 +185,6 @@ static void test_visitor_in_enum(TestInputVisitorData *data,
data->qiv = NULL;
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, "TestStruct", name, sizeof(TestStruct),
- &err);
- if (err) {
- goto out;
- }
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static void test_visitor_in_struct(TestInputVisitorData *data,
const void *unused)
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 09d0dd8..baf58dc 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -166,41 +166,6 @@ static void test_visitor_out_enum_errors(TestOutputVisitorData *data,
}
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, "TestStruct", name, sizeof(TestStruct),
- &err);
- if (err) {
- goto out;
- }
-
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static void test_visitor_out_struct(TestOutputVisitorData *data,
const void *unused)
@@ -314,29 +279,6 @@ static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
}
}
-typedef struct TestStructList
-{
- union {
- TestStruct *value;
- uint64_t padding;
- };
- struct TestStructList *next;
-} TestStructList;
-
-static void visit_type_TestStructList(Visitor *v, TestStructList **obj,
- const char *name, Error **errp)
-{
- GenericList *i, **head = (GenericList **)obj;
-
- visit_start_list(v, name, errp);
-
- for (*head = i = visit_next_list(v, head, errp); i; i = visit_next_list(v, &i, errp)) {
- TestStructList *native_i = (TestStructList *)i;
- visit_type_TestStruct(v, &native_i->value, NULL, errp);
- }
-
- visit_end_list(v, errp);
-}
static void test_visitor_out_list(TestOutputVisitorData *data,
const void *unused)
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index 634563b..c024e5e 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -186,40 +186,6 @@ static void visit_primitive_list(Visitor *v, void **native, Error **errp)
}
}
-typedef struct TestStruct
-{
- int64_t integer;
- bool boolean;
- char *string;
-} TestStruct;
-
-static void visit_type_TestStruct(Visitor *v, TestStruct **obj,
- const char *name, Error **errp)
-{
- Error *err = NULL;
-
- visit_start_struct(v, (void **)obj, NULL, name, sizeof(TestStruct), &err);
- if (err) {
- goto out;
- }
-
- visit_type_int(v, &(*obj)->integer, "integer", &err);
- if (err) {
- goto out_end;
- }
- visit_type_bool(v, &(*obj)->boolean, "boolean", &err);
- if (err) {
- goto out_end;
- }
- visit_type_str(v, &(*obj)->string, "string", &err);
-
-out_end:
- error_propagate(errp, err);
- err = NULL;
- visit_end_struct(v, &err);
-out:
- error_propagate(errp, err);
-}
static TestStruct *struct_create(void)
{
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 02/17] qapi: Strengthen test of TestStructList
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 01/17] qapi: Use generated TestStruct machinery in tests Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 03/17] qapi: Provide nicer array names in introspection Eric Blake
` (14 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Make each list element different, to ensure that order is
preserved, and use the generated free function instead of
hand-rolling our own to ensure (under valgrind) that the
list is properly cleaned.
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: new patch
---
tests/test-qmp-output-visitor.c | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index baf58dc..9364843 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -283,7 +283,7 @@ static void test_visitor_out_struct_errors(TestOutputVisitorData *data,
static void test_visitor_out_list(TestOutputVisitorData *data,
const void *unused)
{
- char *value_str = (char *) "list value";
+ const char *value_str = "list value";
TestStructList *p, *head = NULL;
const int max_items = 10;
bool value_bool = true;
@@ -294,12 +294,13 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
QList *qlist;
int i;
+ /* Build the list in reverse order... */
for (i = 0; i < max_items; i++) {
p = g_malloc0(sizeof(*p));
p->value = g_malloc0(sizeof(*p->value));
- p->value->integer = value_int;
+ p->value->integer = value_int + (max_items - i - 1);
p->value->boolean = value_bool;
- p->value->string = value_str;
+ p->value->string = g_strdup(value_str);
p->next = head;
head = p;
@@ -315,6 +316,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
qlist = qobject_to_qlist(obj);
g_assert(!qlist_empty(qlist));
+ /* ...and ensure that the visitor sees it in order */
i = 0;
QLIST_FOREACH_ENTRY(qlist, entry) {
QDict *qdict;
@@ -322,7 +324,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
g_assert(qobject_type(entry->value) == QTYPE_QDICT);
qdict = qobject_to_qdict(entry->value);
g_assert_cmpint(qdict_size(qdict), ==, 3);
- g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, value_int);
+ g_assert_cmpint(qdict_get_int(qdict, "integer"), ==, value_int + i);
g_assert_cmpint(qdict_get_bool(qdict, "boolean"), ==, value_bool);
g_assert_cmpstr(qdict_get_str(qdict, "string"), ==, value_str);
i++;
@@ -330,13 +332,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
g_assert_cmpint(i, ==, max_items);
QDECREF(qlist);
-
- for (p = head; p;) {
- TestStructList *tmp = p->next;
- g_free(p->value);
- g_free(p);
- p = tmp;
- }
+ qapi_free_TestStructList(head);
}
static void test_visitor_out_list_qapi_free(TestOutputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 03/17] qapi: Provide nicer array names in introspection
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 01/17] qapi: Use generated TestStruct machinery in tests Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 02/17] qapi: Strengthen test of TestStructList Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting Eric Blake
` (13 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
For the sake of humans reading introspection output, it is nice
to have the name of implicit array types be recognizable as
arrays of the underlying type. However, while this patch allows
humans to skip from a command with return type "[123]" straight
to the definition of type "123" without having to first inspect
type "[123]", document that this shortcut should not be taken by
client apps.
This makes the resulting introspection string slightly larger by
default (just over 200 bytes), but it's in the noise (less than
0.3% of the overall 70k size of 'query-qmp-capabilities').
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: commit message stats on the size increase
v7: no change
v6: no change
---
docs/qapi-code-gen.txt | 7 +++++--
scripts/qapi-introspect.py | 8 +++++---
2 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 4d8c2fc..ba29bc6 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -662,11 +662,14 @@ Example: the SchemaInfo for BlockRef from section Alternate types
The SchemaInfo for an array type has meta-type "array", and variant
member "element-type", which names the array's element type. Array
-types are implicitly defined.
+types are implicitly defined. For convenience, the array's name may
+resemble the element type; however, clients should examine member
+"element-type" instead of making assumptions based on parsing member
+"name".
Example: the SchemaInfo for ['str']
- { "name": "strList", "meta-type": "array",
+ { "name": "[str]", "meta-type": "array",
"element-type": "str" }
The SchemaInfo for an enumeration type has meta-type "enum" and
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
index c0dad66..64f2cd0 100644
--- a/scripts/qapi-introspect.py
+++ b/scripts/qapi-introspect.py
@@ -107,10 +107,12 @@ const char %(c_name)s[] = %(c_string)s;
# characters.
if isinstance(typ, QAPISchemaBuiltinType):
return typ.name
+ if isinstance(typ, QAPISchemaArrayType):
+ return '[' + self._use_type(typ.element_type) + ']'
return self._name(typ.name)
def _gen_json(self, name, mtype, obj):
- if mtype != 'command' and mtype != 'event' and mtype != 'builtin':
+ if mtype not in ('command', 'event', 'builtin', 'array'):
name = self._name(name)
obj['name'] = name
obj['meta-type'] = mtype
@@ -136,8 +138,8 @@ const char %(c_name)s[] = %(c_string)s;
self._gen_json(name, 'enum', {'values': values})
def visit_array_type(self, name, info, element_type):
- self._gen_json(name, 'array',
- {'element-type': self._use_type(element_type)})
+ element = self._use_type(element_type)
+ self._gen_json('[' + element + ']', 'array', {'element-type': element})
def visit_object_type_flat(self, name, info, members, variants):
obj = {'members': [self._gen_member(m) for m in members]}
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (2 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 03/17] qapi: Provide nicer array names in introspection Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-30 11:20 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members Eric Blake
` (12 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Sorting the values of an enum makes it easier to look up whether
a particular value is present, via binary rather than linear search
(probably most visible with QKeyCode, which has grown over
several releases). Additionally, QMP clients need not know which
C value is associated with an enum name, so sorting is an
effective way to hide that non-ABI aspect of qapi.
While we are at it, it is also easy to sort the members and
variants of objects, to allow for a similar binary search (although
less compelling, as any struct with that many members should
arguably be broken into hierarchical substructs), and equally
valid since JSON objects have no specific order in which keys must
appear. There is no trivial or obvious way to sort the types of
an alternate, so that is left unchanged.
However, the overall SchemaInfo array remains unsorted. It might
make sense to sort with 'meta-type' as a primary key and 'name'
as a secondary key, but it is not obvious that this will provide
benefits to end-user clients (we allow mutually recursive types,
so there is no posible topological sorting where a single pass
over the array could resolve all types, and while binary searches
could be made possible by sorting, it would be even more efficient
for clients to read the array into a hashtable for O(1) rather
than O(log n) random-access lookups, at which point pre-sorting is
wasted effort).
Document these conventions, so that clients will know what can
and cannot be relied on.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
TODO: should the documentation mention that the sorting is done
in the C locale? Is there anything required to ensure that python
sorts sanely (ie. that the choice of locale while building
doesn't cause inadvertent sorting differences such as turning on
case-insensitivity)?
v8: no change
v7: tweak commit wording
v6: no change from v5
---
docs/qapi-code-gen.txt | 21 +++++++++++++++++----
qapi/introspect.json | 22 +++++++++++++++++-----
scripts/qapi-introspect.py | 9 ++++++---
3 files changed, 40 insertions(+), 12 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index ba29bc6..163f547 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -516,6 +516,13 @@ query-qmp-schema. QGA currently doesn't support introspection.
query-qmp-schema returns a JSON array of SchemaInfo objects. These
objects together describe the wire ABI, as defined in the QAPI schema.
+There is no specified order to the SchemaInfo objects returned; a
+client must search for a particular name and meta-type throughout the
+entire array to learn more about that name. For now, the name for
+each SchemaInfo is unique thanks to qapi naming conventions; however
+this may be changed in the future (such as allowing a command and an
+event with the same name), so it is important that the client check
+for the desired meta-type.
However, the SchemaInfo can't reflect all the rules and restrictions
that apply to QMP. It's interface introspection (figuring out what's
@@ -596,7 +603,8 @@ any. Each element is a JSON object with members "name" (the member's
name), "type" (the name of its type), and optionally "default". The
member is optional if "default" is present. Currently, "default" can
only have value null. Other values are reserved for future
-extensions.
+extensions. The "members" array is sorted by "name", so that clients
+can use a binary search to see if a particular member is supported.
Example: the SchemaInfo for MyType from section Struct types
@@ -610,7 +618,9 @@ Example: the SchemaInfo for MyType from section Struct types
"variants" is a JSON array describing the object's variant members.
Each element is a JSON object with members "case" (the value of type
tag this element applies to) and "type" (the name of an object type
-that provides the variant members for this type tag value).
+that provides the variant members for this type tag value). The
+"variants" array is sorted by "case", so it appears in the same
+order as the enum type matching "tag".
Example: the SchemaInfo for flat union BlockdevOptions from section
Union types
@@ -651,7 +661,8 @@ Union types
The SchemaInfo for an alternate type has meta-type "alternate", and
variant member "members". "members" is a JSON array. Each element is
a JSON object with member "type", which names a type. Values of the
-alternate type conform to exactly one of its member types.
+alternate type conform to exactly one of its member types. There is
+no guarantee on the order in which "members" will be listed.
Example: the SchemaInfo for BlockRef from section Alternate types
@@ -673,7 +684,9 @@ Example: the SchemaInfo for ['str']
"element-type": "str" }
The SchemaInfo for an enumeration type has meta-type "enum" and
-variant member "values".
+variant member "values". The values are listed in sorted order,
+so clients can use a binary search to see if a particular value
+is present.
Example: the SchemaInfo for MyEnum from section Enumeration types
diff --git a/qapi/introspect.json b/qapi/introspect.json
index cc50dc6..71632af 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -25,6 +25,11 @@
# Returns: array of @SchemaInfo, where each element describes an
# entity in the ABI: command, event, type, ...
#
+# The order of the various SchemaInfo is unspecified. For now, the
+# name of each SchemaInfo is unique regardless of meta-type, but to be
+# safe, clients should check that a given name has the expected
+# meta-type.
+#
# Note: the QAPI schema is also used to help define *internal*
# interfaces, by defining QAPI types. These are not part of the QMP
# wire ABI, and therefore not returned by this command.
@@ -78,7 +83,8 @@
# Commands and events have the name defined in the QAPI schema.
# Unlike command and event names, type names are not part of
# the wire ABI. Consequently, type names are meaningless
-# strings here.
+# strings here. Although all names are currently unique
+# regardless of @meta-type, clients should not rely on this.
#
# All references to other SchemaInfo are by name.
#
@@ -130,7 +136,9 @@
#
# Additional SchemaInfo members for meta-type 'enum'.
#
-# @values: the enumeration type's values.
+# @values: the enumeration type's values. The values are sorted, so
+# clients can use a binary search to see if a particular value
+# is present.
#
# Values of this type are JSON string on the wire.
#
@@ -158,14 +166,18 @@
#
# Additional SchemaInfo members for meta-type 'object'.
#
-# @members: the object type's (non-variant) members.
+# @members: the object type's (non-variant) members. The members are
+# sorted by name, so clients can use a binary search to see
+# if a given member is present.
#
# @tag: #optional the name of the member serving as type tag.
# An element of @members with this name must exist.
#
# @variants: #optional variant members, i.e. additional members that
# depend on the type tag's value. Present exactly when
-# @tag is present.
+# @tag is present. The variants are sorted by case, which
+# means they appear in the same order as the values of the
+# enum type of the @tag.
#
# Values of this type are JSON object on the wire.
#
@@ -219,7 +231,7 @@
#
# Additional SchemaInfo members for meta-type 'alternate'.
#
-# @members: the alternate type's members.
+# @members: the alternate type's members, in no particular order.
# The members' wire encoding is distinct, see
# docs/qapi-code-gen.txt section Alternate types.
#
diff --git a/scripts/qapi-introspect.py b/scripts/qapi-introspect.py
index 64f2cd0..6a5a843 100644
--- a/scripts/qapi-introspect.py
+++ b/scripts/qapi-introspect.py
@@ -10,6 +10,7 @@
# See the COPYING file in the top-level directory.
from qapi import *
+from operator import attrgetter
# Caveman's json.dumps() replacement (we're stuck at Python 2.4)
@@ -126,7 +127,8 @@ const char %(c_name)s[] = %(c_string)s;
def _gen_variants(self, tag_name, variants):
return {'tag': tag_name,
- 'variants': [self._gen_variant(v) for v in variants]}
+ 'variants': [self._gen_variant(v) for v in
+ sorted(variants, key=attrgetter('name'))]}
def _gen_variant(self, variant):
return {'case': variant.name, 'type': self._use_type(variant.type)}
@@ -135,14 +137,15 @@ const char %(c_name)s[] = %(c_string)s;
self._gen_json(name, 'builtin', {'json-type': json_type})
def visit_enum_type(self, name, info, values, prefix):
- self._gen_json(name, 'enum', {'values': values})
+ self._gen_json(name, 'enum', {'values': sorted(values)})
def visit_array_type(self, name, info, element_type):
element = self._use_type(element_type)
self._gen_json('[' + element + ']', 'array', {'element-type': element})
def visit_object_type_flat(self, name, info, members, variants):
- obj = {'members': [self._gen_member(m) for m in members]}
+ obj = {'members': [self._gen_member(m) for m in
+ sorted(members, key=attrgetter('name'))]}
if variants:
obj.update(self._gen_variants(variants.tag_member.name,
variants.variants))
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (3 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-30 12:54 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
` (11 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base).
The various front end entities then map as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients. No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: new patch
---
scripts/qapi-types.py | 4 +++-
scripts/qapi-visit.py | 4 +++-
scripts/qapi.py | 14 ++++++++++----
tests/qapi-schema/qapi-schema-test.out | 2 ++
tests/qapi-schema/union-clash-data.out | 1 +
tests/qapi-schema/union-empty.out | 1 +
6 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index b37900f..a6bf95d 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -269,7 +269,9 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_object_type(self, name, info, base, members, variants):
self._fwdecl += gen_fwd_object_or_array(name)
if variants:
- assert not members # not implemented
+ if members:
+ assert len(members) == 1
+ assert members[0] == variants.tag_member
self.decl += gen_union(name, base, variants)
else:
self.decl += gen_struct(name, base, members)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index f40c3c7..318b8e6 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -360,7 +360,9 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
def visit_object_type(self, name, info, base, members, variants):
self.decl += gen_visit_decl(name)
if variants:
- assert not members # not implemented
+ if members:
+ assert len(members) == 1
+ assert members[0] == variants.tag_member
self.defn += gen_visit_union(name, base, variants)
else:
self.defn += gen_visit_struct(name, base, members)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 7c50cc4..84ac151 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -957,6 +957,9 @@ class QAPISchemaArrayType(QAPISchemaType):
class QAPISchemaObjectType(QAPISchemaType):
def __init__(self, name, info, base, local_members, variants):
+ # struct has local_members, optional base, and no variants
+ # flat union has base, variants, and no local_members
+ # simple union has local_members, variants, and no base
QAPISchemaType.__init__(self, name, info)
assert base is None or isinstance(base, str)
for m in local_members:
@@ -1048,9 +1051,11 @@ class QAPISchemaObjectTypeVariants(object):
self.variants = variants
def check(self, schema, members, seen):
- if self.tag_name:
+ if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
- else:
+ elif seen: # simple union
+ assert self.tag_member in seen.itervalues()
+ else: # alternate
self.tag_member.check(schema, members, seen)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
@@ -1270,13 +1275,14 @@ class QAPISchema(object):
if tag_name:
variants = [self._make_variant(key, value)
for (key, value) in data.iteritems()]
+ members = []
else:
variants = [self._make_simple_variant(key, value, info)
for (key, value) in data.iteritems()]
tag_member = self._make_implicit_tag(name, info, variants)
+ members = [tag_member]
self._def_entity(
- QAPISchemaObjectType(name, info, base,
- self._make_members(OrderedDict(), info),
+ QAPISchemaObjectType(name, info, base, members,
QAPISchemaObjectTypeVariants(tag_name,
tag_member,
variants)))
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index e20a823..786024e 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -132,6 +132,7 @@ object UserDefFlatUnion2
case value2: UserDefB
case value3: UserDefA
object UserDefNativeListUnion
+ member type: UserDefNativeListUnionKind optional=False
case integer: :obj-intList-wrapper
case s8: :obj-int8List-wrapper
case s16: :obj-int16List-wrapper
@@ -187,6 +188,7 @@ object __org.qemu_x-Struct
object __org.qemu_x-Struct2
member array: __org.qemu_x-Union1List optional=False
object __org.qemu_x-Union1
+ member type: __org.qemu_x-Union1Kind optional=False
case __org.qemu_x-branch: :obj-str-wrapper
enum __org.qemu_x-Union1Kind ['__org.qemu_x-branch']
object __org.qemu_x-Union2
diff --git a/tests/qapi-schema/union-clash-data.out b/tests/qapi-schema/union-clash-data.out
index 6277239..cea8551 100644
--- a/tests/qapi-schema/union-clash-data.out
+++ b/tests/qapi-schema/union-clash-data.out
@@ -2,5 +2,6 @@ object :empty
object :obj-int-wrapper
member data: int optional=False
object TestUnion
+ member type: TestUnionKind optional=False
case data: :obj-int-wrapper
enum TestUnionKind ['data']
diff --git a/tests/qapi-schema/union-empty.out b/tests/qapi-schema/union-empty.out
index 8b5a7bf..9c89fd1 100644
--- a/tests/qapi-schema/union-empty.out
+++ b/tests/qapi-schema/union-empty.out
@@ -1,3 +1,4 @@
object :empty
object Union
+ member type: UnionKind optional=False
enum UnionKind []
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (4 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-30 13:01 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions Eric Blake
` (10 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
These two methods are now close enough that we can finally merge
them, relying on the fact that simple unions now provide a
reasonable local_members. Change gen_struct() to gen_object()
that handles all forms of QAPISchemaObjectType, and rename and
shrink gen_union() to gen_variants() to handle the portion of
gen_object() needed when variants are present.
gen_struct_fields() now has a single caller, so it no longer
needs an optional parameter; however, I did not choose to inline
it into the caller.
No difference to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: new patch
---
scripts/qapi-types.py | 36 +++++++++++-------------------------
1 file changed, 11 insertions(+), 25 deletions(-)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index a6bf95d..403768b 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -51,7 +51,7 @@ def gen_struct_field(member):
return ret
-def gen_struct_fields(local_members, base=None):
+def gen_struct_fields(local_members, base):
ret = ''
if base:
@@ -70,7 +70,7 @@ def gen_struct_fields(local_members, base=None):
return ret
-def gen_struct(name, base, members):
+def gen_object(name, base, members, variants):
ret = mcgen('''
struct %(c_name)s {
@@ -79,11 +79,14 @@ struct %(c_name)s {
ret += gen_struct_fields(members, base)
+ if variants:
+ ret += gen_variants(variants)
+
# Make sure that all structs have at least one field; this avoids
# potential issues with attempting to malloc space for zero-length
# structs in C, and also incompatibility with C++ (where an empty
# struct is size 1).
- if not (base and base.members) and not members:
+ if not (base and base.members) and not members and not variants:
ret += mcgen('''
char qapi_dummy_field_for_empty_struct;
''')
@@ -140,17 +143,7 @@ const int %(c_name)s_qtypes[QTYPE_MAX] = {
return ret
-def gen_union(name, base, variants):
- ret = mcgen('''
-
-struct %(c_name)s {
-''',
- c_name=c_name(name))
- if base:
- ret += gen_struct_fields([], base)
- else:
- ret += gen_struct_field(variants.tag_member)
-
+def gen_variants(variants):
# FIXME: What purpose does data serve, besides preventing a union that
# has a branch named 'data'? We use it in qapi-visit.py to decide
# whether to bypass the switch statement if visiting the discriminator
@@ -159,11 +152,11 @@ struct %(c_name)s {
# should not be any data leaks even without a data pointer. Or, if
# 'data' is merely added to guarantee we don't have an empty union,
# shouldn't we enforce that at .json parse time?
- ret += mcgen('''
+ ret = mcgen('''
union { /* union tag is @%(c_name)s */
void *data;
''',
- c_name=c_name(variants.tag_member.name))
+ c_name=c_name(variants.tag_member.name))
for var in variants.variants:
# Ugly special case for simple union TODO get rid of it
@@ -176,7 +169,6 @@ struct %(c_name)s {
ret += mcgen('''
} u;
-};
''')
return ret
@@ -268,13 +260,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_object_type(self, name, info, base, members, variants):
self._fwdecl += gen_fwd_object_or_array(name)
- if variants:
- if members:
- assert len(members) == 1
- assert members[0] == variants.tag_member
- self.decl += gen_union(name, base, variants)
- else:
- self.decl += gen_struct(name, base, members)
+ self.decl += gen_object(name, base, members, variants)
if base:
self.decl += gen_upcast(name, base)
self._gen_type_cleanup(name)
@@ -282,7 +268,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_alternate_type(self, name, info, variants):
self._fwdecl += gen_fwd_object_or_array(name)
self._fwdefn += gen_alternate_qtypes(name, variants)
- self.decl += gen_union(name, None, variants)
+ self.decl += gen_object(name, None, [variants.tag_member], variants)
self.decl += gen_alternate_qtypes_decl(name)
self._gen_type_cleanup(name)
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (5 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-11-02 15:37 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 08/17] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
` (9 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Now that we have separate namespaces for QMP vs. tag values,
we can simplify how the QAPISchema*.check() methods check for
collisions. Each QAPISchemaObjectTypeMember check() call is
given a single set of names it must not collide with; this set
is either the QMP names (when this member is used by an
ObjectType) or the case names (when this member is used by an
ObjectTypeVariants). We no longer need an all_members
parameter, as it can be computed by seen.values(). When used
by a union, QAPISchemaObjectTypeVariant must also perform a
QMP collision check for each member of its corresponding type.
The new ObjectType.check_qmp() is an idempotent subset of
check(), and can be called multiple times over different seen
sets (useful, since the members of one type can be applied
into more than one other location via inheritance or flat
union variants).
The code needs a temporary hack of passing a 'union' flag
through Variants.check(), since we do not inline the branches
of an alternate type into a parent QMP object. A later patch
will rework how alternates are laid out, by adding a new
subclass, and that will allow us to drop the extra parameter.
There are no changes to generated code.
Future patches will enhance testsuite coverage, improve error
message quality on actual collisions, and move collision
checks out of ad hoc parse code into the check() methods.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: rebase to earlier patches, defer positive test additions to
later in series
v7: new patch, although it is a much cleaner implementation of
what was attempted by subset B v8 15/18
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03042.html
---
scripts/qapi.py | 55 +++++++++++++++++++++++++++++++++----------------------
1 file changed, 33 insertions(+), 22 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 84ac151..c571709 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -972,28 +972,28 @@ class QAPISchemaObjectType(QAPISchemaType):
self.variants = variants
self.members = None
+ # Finish construction, and validate that all members are usable
def check(self, schema):
assert self.members is not False # not running in cycles
if self.members:
return
self.members = False # mark as being checked
+ seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
- assert isinstance(self.base, QAPISchemaObjectType)
- assert not self.base.variants # not implemented
- self.base.check(schema)
- members = list(self.base.members)
- else:
- members = []
- seen = {}
- for m in members:
- assert c_name(m.name) not in seen
- seen[m.name] = m
+ self.base.check_qmp(schema, seen)
for m in self.local_members:
- m.check(schema, members, seen)
+ m.check(schema, seen)
if self.variants:
- self.variants.check(schema, members, seen)
- self.members = members
+ self.variants.check(schema, seen)
+ self.members = seen.values()
+
+ # Check that this type does not introduce QMP collisions into seen
+ def check_qmp(self, schema, seen):
+ self.check(schema)
+ assert not self.variants # not implemented
+ for m in self.members:
+ m.check(schema, seen)
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
@@ -1027,11 +1027,13 @@ class QAPISchemaObjectTypeMember(object):
self.type = None
self.optional = optional
- def check(self, schema, all_members, seen):
+ def check(self, schema, seen):
+ # seen is a map of names we must not collide with (either QMP
+ # names, when called by ObjectType, or case names, when called
+ # by Variant). This method is safe to call over multiple 'seen'.
assert self.name not in seen
self.type = schema.lookup_type(self._type_name)
assert self.type
- all_members.append(self)
seen[self.name] = self
@@ -1050,26 +1052,35 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
- def check(self, schema, members, seen):
+ # TODO drop union once alternates can be distinguished by tag_member
+ def check(self, schema, seen, union=True):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
+ assert self.tag_member
elif seen: # simple union
assert self.tag_member in seen.itervalues()
else: # alternate
- self.tag_member.check(schema, members, seen)
+ self.tag_member.check(schema, seen)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
+ cases = OrderedDict()
for v in self.variants:
- vseen = dict(seen)
- v.check(schema, self.tag_member.type, vseen)
+ # Reset seen array for each variant, since QMP names from one
+ # branch do not affect another branch, nor add to all_members
+ v.check(schema, self.tag_member.type, dict(seen), cases, union)
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
- def check(self, schema, tag_type, seen):
- QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ # TODO drop union once alternates can be distinguished by tag_type
+ def check(self, schema, tag_type, seen, cases, union):
+ # cases is case names we must not collide with
+ QAPISchemaObjectTypeMember.check(self, schema, cases)
assert self.name in tag_type.values
+ if union:
+ # seen is QMP names our members must not collide with
+ self.type.check_qmp(schema, seen)
# This function exists to support ugly simple union special cases
# TODO get rid of them, and drop the function
@@ -1090,7 +1101,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
- self.variants.check(schema, [], {})
+ self.variants.check(schema, {}, False)
def json_type(self):
return 'value'
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 08/17] qapi: Remove outdated tests related to QMP/branch collisions
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (6 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-11-03 16:32 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test Eric Blake
` (8 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Now that branches are in a separate C namespace, we can remove
the restrictions in the parser that claim a branch name would
collide with QMP, and delete the negative tests that are no
longer problematic. A separate patch can then add positive
tests to qapi-schema-test to test that any corner cases will
compile correctly.
This reverts the scripts/qapi.py portion of commit 7b2a5c2,
now that the assertions that it plugged are no longer possible.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: retitle and update commit message, finish reversion of 7b2a5c2
v7: new patch (also temporarily appeared in subset B v10)
---
scripts/qapi.py | 11 ++---------
tests/Makefile | 3 ---
tests/qapi-schema/flat-union-clash-branch.err | 0
tests/qapi-schema/flat-union-clash-branch.exit | 1 -
tests/qapi-schema/flat-union-clash-branch.json | 18 ------------------
tests/qapi-schema/flat-union-clash-branch.out | 14 --------------
tests/qapi-schema/flat-union-clash-type.err | 1 -
tests/qapi-schema/flat-union-clash-type.exit | 1 -
tests/qapi-schema/flat-union-clash-type.json | 14 --------------
tests/qapi-schema/flat-union-clash-type.out | 0
tests/qapi-schema/union-clash-type.err | 1 -
tests/qapi-schema/union-clash-type.exit | 1 -
tests/qapi-schema/union-clash-type.json | 9 ---------
tests/qapi-schema/union-clash-type.out | 0
14 files changed, 2 insertions(+), 72 deletions(-)
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.err
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.json
delete mode 100644 tests/qapi-schema/flat-union-clash-branch.out
delete mode 100644 tests/qapi-schema/flat-union-clash-type.err
delete mode 100644 tests/qapi-schema/flat-union-clash-type.exit
delete mode 100644 tests/qapi-schema/flat-union-clash-type.json
delete mode 100644 tests/qapi-schema/flat-union-clash-type.out
delete mode 100644 tests/qapi-schema/union-clash-type.err
delete mode 100644 tests/qapi-schema/union-clash-type.exit
delete mode 100644 tests/qapi-schema/union-clash-type.json
delete mode 100644 tests/qapi-schema/union-clash-type.out
diff --git a/scripts/qapi.py b/scripts/qapi.py
index c571709..a0639e4 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -548,8 +548,7 @@ def check_union(expr, expr_info):
base = expr.get('base')
discriminator = expr.get('discriminator')
members = expr['data']
- values = {'MAX': '(automatic)', 'KIND': '(automatic)',
- 'TYPE': '(automatic)'}
+ values = {'MAX': '(automatic)'}
# Two types of unions, determined by discriminator.
@@ -607,19 +606,13 @@ def check_union(expr, expr_info):
" of branch '%s'" % key)
# If the discriminator names an enum type, then all members
- # of 'data' must also be members of the enum type, which in turn
- # must not collide with the discriminator name.
+ # of 'data' must also be members of the enum type.
if enum_define:
if key not in enum_define['enum_values']:
raise QAPIExprError(expr_info,
"Discriminator value '%s' is not found in "
"enum '%s'" %
(key, enum_define["enum_name"]))
- if discriminator in enum_define['enum_values']:
- raise QAPIExprError(expr_info,
- "Discriminator name '%s' collides with "
- "enum value in '%s'" %
- (discriminator, enum_define["enum_name"]))
# Otherwise, check for conflicts in the generated enum
else:
diff --git a/tests/Makefile b/tests/Makefile
index f448b5a..30879ba 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -277,9 +277,7 @@ qapi-schema += flat-union-bad-base.json
qapi-schema += flat-union-bad-discriminator.json
qapi-schema += flat-union-base-any.json
qapi-schema += flat-union-base-union.json
-qapi-schema += flat-union-clash-branch.json
qapi-schema += flat-union-clash-member.json
-qapi-schema += flat-union-clash-type.json
qapi-schema += flat-union-empty.json
qapi-schema += flat-union-inline.json
qapi-schema += flat-union-int-branch.json
@@ -341,7 +339,6 @@ qapi-schema += union-bad-branch.json
qapi-schema += union-base-no-discriminator.json
qapi-schema += union-clash-branches.json
qapi-schema += union-clash-data.json
-qapi-schema += union-clash-type.json
qapi-schema += union-empty.json
qapi-schema += union-invalid-base.json
qapi-schema += union-max.json
diff --git a/tests/qapi-schema/flat-union-clash-branch.err b/tests/qapi-schema/flat-union-clash-branch.err
deleted file mode 100644
index e69de29..0000000
diff --git a/tests/qapi-schema/flat-union-clash-branch.exit b/tests/qapi-schema/flat-union-clash-branch.exit
deleted file mode 100644
index 573541a..0000000
--- a/tests/qapi-schema/flat-union-clash-branch.exit
+++ /dev/null
@@ -1 +0,0 @@
-0
diff --git a/tests/qapi-schema/flat-union-clash-branch.json b/tests/qapi-schema/flat-union-clash-branch.json
deleted file mode 100644
index e593336..0000000
--- a/tests/qapi-schema/flat-union-clash-branch.json
+++ /dev/null
@@ -1,18 +0,0 @@
-# Flat union branch name collision
-# FIXME: this parses, but then fails to compile due to a duplicate 'c_d'
-# (one from the base member, the other from the branch name). We should
-# either reject the collision at parse time, or munge the generated branch
-# name to allow this to compile.
-{ 'enum': 'TestEnum',
- 'data': [ 'base', 'c-d' ] }
-{ 'struct': 'Base',
- 'data': { 'enum1': 'TestEnum', '*c_d': 'str' } }
-{ 'struct': 'Branch1',
- 'data': { 'string': 'str' } }
-{ 'struct': 'Branch2',
- 'data': { 'value': 'int' } }
-{ 'union': 'TestUnion',
- 'base': 'Base',
- 'discriminator': 'enum1',
- 'data': { 'base': 'Branch1',
- 'c-d': 'Branch2' } }
diff --git a/tests/qapi-schema/flat-union-clash-branch.out b/tests/qapi-schema/flat-union-clash-branch.out
deleted file mode 100644
index 8e0da73..0000000
--- a/tests/qapi-schema/flat-union-clash-branch.out
+++ /dev/null
@@ -1,14 +0,0 @@
-object :empty
-object Base
- member enum1: TestEnum optional=False
- member c_d: str optional=True
-object Branch1
- member string: str optional=False
-object Branch2
- member value: int optional=False
-enum TestEnum ['base', 'c-d']
-object TestUnion
- base Base
- tag enum1
- case base: Branch1
- case c-d: Branch2
diff --git a/tests/qapi-schema/flat-union-clash-type.err b/tests/qapi-schema/flat-union-clash-type.err
deleted file mode 100644
index b44dd40..0000000
--- a/tests/qapi-schema/flat-union-clash-type.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/flat-union-clash-type.json:11: Discriminator name 'type' collides with enum value in 'TestEnum'
diff --git a/tests/qapi-schema/flat-union-clash-type.exit b/tests/qapi-schema/flat-union-clash-type.exit
deleted file mode 100644
index d00491f..0000000
--- a/tests/qapi-schema/flat-union-clash-type.exit
+++ /dev/null
@@ -1 +0,0 @@
-1
diff --git a/tests/qapi-schema/flat-union-clash-type.json b/tests/qapi-schema/flat-union-clash-type.json
deleted file mode 100644
index 8f710f0..0000000
--- a/tests/qapi-schema/flat-union-clash-type.json
+++ /dev/null
@@ -1,14 +0,0 @@
-# Flat union branch 'type'
-# Reject this, because we would have a clash in generated C, between the
-# outer tag 'type' and the branch name 'type' within the union.
-# TODO: We could munge the generated C branch name to let it compile.
-{ 'enum': 'TestEnum',
- 'data': [ 'type' ] }
-{ 'struct': 'Base',
- 'data': { 'type': 'TestEnum' } }
-{ 'struct': 'Branch1',
- 'data': { 'string': 'str' } }
-{ 'union': 'TestUnion',
- 'base': 'Base',
- 'discriminator': 'type',
- 'data': { 'type': 'Branch1' } }
diff --git a/tests/qapi-schema/flat-union-clash-type.out b/tests/qapi-schema/flat-union-clash-type.out
deleted file mode 100644
index e69de29..0000000
diff --git a/tests/qapi-schema/union-clash-type.err b/tests/qapi-schema/union-clash-type.err
deleted file mode 100644
index a5dead1..0000000
--- a/tests/qapi-schema/union-clash-type.err
+++ /dev/null
@@ -1 +0,0 @@
-tests/qapi-schema/union-clash-type.json:8: Union 'TestUnion' member 'kind' clashes with '(automatic)'
diff --git a/tests/qapi-schema/union-clash-type.exit b/tests/qapi-schema/union-clash-type.exit
deleted file mode 100644
index d00491f..0000000
--- a/tests/qapi-schema/union-clash-type.exit
+++ /dev/null
@@ -1 +0,0 @@
-1
diff --git a/tests/qapi-schema/union-clash-type.json b/tests/qapi-schema/union-clash-type.json
deleted file mode 100644
index cfc256b..0000000
--- a/tests/qapi-schema/union-clash-type.json
+++ /dev/null
@@ -1,9 +0,0 @@
-# Union branch 'type'
-# Reject this, because we would have a clash in generated C, between the
-# simple union's implicit tag member 'kind' and the branch name 'kind'
-# within the union.
-# TODO: Even when the generated C is switched to use 'type' rather than
-# 'kind', to match the QMP spelling, the collision should still be detected.
-# Or, we could munge the branch name to allow compilation.
-{ 'union': 'TestUnion',
- 'data': { 'kind': 'int', 'type': 'str' } }
diff --git a/tests/qapi-schema/union-clash-type.out b/tests/qapi-schema/union-clash-type.out
deleted file mode 100644
index e69de29..0000000
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (7 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 08/17] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-11-03 16:43 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types Eric Blake
` (7 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Add positive tests to qapi-schema-test for things that were
made possible by recent patches but which caused compile errors
due to collisions prior to that point.
This includes:
Use of a member 'base' in a struct with a base class
Use of a member name ending in 'Kind' or 'List'
Use of a type name starting with 'has_'
Use of a type named 'u'
Use of a union branch name of 'u'
Use of a union branch name starting with 'has_'
Use of a union branch name of 'type'
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: new, but collects portions of subset B v10 patches 2, 3, and
16 and subset C v7 patch 6 that were deferred to later.
It might be worth dropping or simplifying this patch, depending
on how many corner cases we actually want to test.
---
tests/qapi-schema/qapi-schema-test.json | 20 ++++++++++++++++++++
tests/qapi-schema/qapi-schema-test.out | 31 +++++++++++++++++++++++++++++++
2 files changed, 51 insertions(+)
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 44638da..4354604 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -47,6 +47,10 @@
'data': { 'string0': 'str',
'dict1': 'UserDefTwoDict' } }
+# ensure that we don't have an artificial collision on 'base'
+{ 'struct': 'UserDefThree',
+ 'base': 'UserDefOne', 'data': { 'base': 'str' } }
+
# dummy struct to force generation of array types not otherwise mentioned
{ 'struct': 'ForceArrays',
'data': { 'unused1':['UserDefOne'], 'unused2':['UserDefTwo'],
@@ -113,6 +117,22 @@
'sizes': ['size'],
'any': ['any'] } }
+# Even though 'u' and 'has_*' are forbidden as struct member names, they
+# should still be valid as a type or union branch name. And although
+# '*Kind' and '*List' are forbidden as type names, they should not be
+# forbidden as a member or branch name. Flat union branches do not
+# collide with base members.
+{ 'enum': 'EnumName', 'data': [ 'value1', 'has_a', 'u', 'type' ] }
+{ 'struct': 'has_a', 'data': { 'MyKind': 'int', 'MyList': ['int'],
+ 'value1': 'EnumName' } }
+{ 'union': 'u', 'data': { 'u': 'uint8', 'myKind': 'has_a',
+ 'myList': 'has_a', 'has_a': 'has_a' } }
+{ 'union': 'UnionName', 'base': 'has_a', 'discriminator': 'value1',
+ 'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero',
+ 'u': 'UserDefZero', 'type': 'UserDefZero' } }
+{ 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool',
+ 'myKind': 'has_a' } }
+
# testing commands
{ 'command': 'user_def_cmd', 'data': {} }
{ 'command': 'user_def_cmd1', 'data': {'ud1a': 'UserDefOne'} }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 786024e..29c46da 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -22,6 +22,8 @@ object :obj-guest-get-time-arg
member b: int optional=True
object :obj-guest-sync-arg
member arg: any optional=False
+object :obj-has_a-wrapper
+ member data: has_a optional=False
object :obj-int16List-wrapper
member data: int16List optional=False
object :obj-int32List-wrapper
@@ -46,6 +48,8 @@ object :obj-uint32List-wrapper
member data: uint32List optional=False
object :obj-uint64List-wrapper
member data: uint64List optional=False
+object :obj-uint8-wrapper
+ member data: uint8 optional=False
object :obj-uint8List-wrapper
member data: uint8List optional=False
object :obj-user_def_cmd1-arg
@@ -57,6 +61,11 @@ alternate AltIntNum
case i: int
case n: number
enum AltIntNumKind ['i', 'n']
+alternate AltName
+ case type: int
+ case u: bool
+ case myKind: has_a
+enum AltNameKind ['type', 'u', 'myKind']
alternate AltNumInt
case n: number
case i: int
@@ -84,6 +93,7 @@ event EVENT_D :obj-EVENT_D-arg
object Empty1
object Empty2
base Empty1
+enum EnumName ['value1', 'has_a', 'u', 'type']
enum EnumOne ['value1', 'value2', 'value3']
object EventStructOne
member struct1: UserDefOne optional=False
@@ -105,6 +115,13 @@ object TestStruct
member integer: int optional=False
member boolean: bool optional=False
member string: str optional=False
+object UnionName
+ base has_a
+ tag value1
+ case value1: UserDefZero
+ case has_a: UserDefZero
+ case u: UserDefZero
+ case type: UserDefZero
object UserDefA
member boolean: bool optional=False
member a_b: int optional=True
@@ -158,6 +175,9 @@ object UserDefOptions
member u16: uint16List optional=True
member i64x: int optional=True
member u64x: uint64 optional=True
+object UserDefThree
+ base UserDefOne
+ member base: str optional=False
object UserDefTwo
member string0: str optional=False
member dict1: UserDefTwoDict optional=False
@@ -201,6 +221,17 @@ command guest-get-time :obj-guest-get-time-arg -> int
gen=True success_response=True
command guest-sync :obj-guest-sync-arg -> any
gen=True success_response=True
+object has_a
+ member MyKind: int optional=False
+ member MyList: intList optional=False
+ member value1: EnumName optional=False
+object u
+ member type: uKind optional=False
+ case u: :obj-uint8-wrapper
+ case myKind: :obj-has_a-wrapper
+ case myList: :obj-has_a-wrapper
+ case has_a: :obj-has_a-wrapper
+enum uKind ['u', 'myKind', 'myList', 'has_a']
command user_def_cmd None -> None
gen=True success_response=True
command user_def_cmd1 :obj-user_def_cmd1-arg -> None
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (8 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-11-03 18:30 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 11/17] qapi: Fix alternates that accept 'number' but not 'int' Eric Blake
` (6 subsequent siblings)
16 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Previously, working with alternates required two enums, and
some indirection: for type Foo, we created Foo_qtypes[] which
maps each qtype to a member of FooKind_lookup[], then use
FooKind_lookup[] like we do for other union types.
This has a subtle bug: since the values of FooKind_lookup
start at zero, all entries of Foo_qtypes that were not
explicitly initialized map to the same branch of the union as
the first member of the alternate, rather than triggering a
failure in visit_get_next_type(). Fortunately, the bug
seldom bites; the very next thing the input visitor does is
try to parse the incoming JSON with the wrong parser, which
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, it IS observable in one case: the behavior of an
alternate that contains a 'number' member but no 'int' member
differs according to whether the 'number' was first in the
qapi definition, and when the input being parsed is an integer;
this is because the 'number' parser accepts QTYPE_QINT in
addition to the expected QTYPE_QFLOAT. A later patch will worry
about fixing alternates to parse all inputs that a non-alternate
'number' would accept, for now it is still marked FIXME.
This patch fixes the validation bug by deleting the indirection,
and modifying get_next_type() to directly return a qtype code.
There is no longer a need to generate an implicit FooKind array
associated with the alternate type (since the QMP wire format
never uses the stringized counterparts of the C union member
names); that also means we no longer have a collision with an
alternate branch named 'max'. Next, the generated visitor is
fixed to properly detect unexpected qtypes in the switch
statement. This is done via the use of a new
QAPISchemaAlternateTypeTag subclass and the use of a new
member.c_type() method when producing qapi-types. The new
subtype also allows us to clean up a TODO left in the previous
commit.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. If that gets too confusing, we
could reintroduce FooKind, but initialize it differently than
most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type. But without a current client, I
didn't see the point of doing it now.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: rebase onto earlier changes, rework how subtype makes things work
v6: rebase onto tag_member subclass, testsuite, gen_err_check(),
and info improvements
---
docs/qapi-code-gen.txt | 3 ---
include/qapi/visitor-impl.h | 3 ++-
include/qapi/visitor.h | 8 +++++-
qapi/qapi-visit-core.c | 4 +--
qapi/qmp-input-visitor.c | 4 +--
scripts/qapi-types.py | 36 +--------------------------
scripts/qapi-visit.py | 12 +++++----
scripts/qapi.py | 43 ++++++++++++++++++++++-----------
tests/qapi-schema/alternate-empty.out | 1 -
tests/qapi-schema/qapi-schema-test.json | 2 +-
tests/qapi-schema/qapi-schema-test.out | 10 +-------
tests/test-qmp-input-visitor.c | 33 +++++++++++++------------
tests/test-qmp-output-visitor.c | 21 ++++++++++++----
13 files changed, 86 insertions(+), 94 deletions(-)
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 163f547..0c1bd36 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -383,9 +383,6 @@ where each branch of the union names a QAPI type. For example:
'data': { 'definition': 'BlockdevOptions',
'reference': 'str' } }
-Just like for a simple union, an implicit C enum 'NameKind' is created
-to enumerate the branches for the alternate 'Name'.
-
Unlike a union, the discriminator string is never passed on the wire
for the Client JSON Protocol. Instead, the value's JSON type serves
as an implicit discriminator, which in turn means that an alternate
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 8c0ba57..6d95b36 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -32,7 +32,8 @@ struct Visitor
void (*type_enum)(Visitor *v, int *obj, const char * const strings[],
const char *kind, const char *name, Error **errp);
- void (*get_next_type)(Visitor *v, int *kind, const int *qobjects,
+ /* May be NULL; most useful for input visitors. */
+ void (*get_next_type)(Visitor *v, qtype_code *type,
const char *name, Error **errp);
void (*type_int)(Visitor *v, int64_t *obj, const char *name, Error **errp);
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index cfc19a6..b765993 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -41,7 +41,13 @@ GenericList *visit_next_list(Visitor *v, GenericList **list, Error **errp);
void visit_end_list(Visitor *v, Error **errp);
void visit_optional(Visitor *v, bool *present, const char *name,
Error **errp);
-void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
+
+/**
+ * Determine the qtype of the item @name in the current object visit.
+ * For input visitors, set *@type to the correct qtype of a qapi
+ * alternate type; for other visitors, leave *@type unchanged.
+ */
+void visit_get_next_type(Visitor *v, qtype_code *type,
const char *name, Error **errp);
void visit_type_enum(Visitor *v, int *obj, const char * const strings[],
const char *kind, const char *name, Error **errp);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 59ed506..3f24daa 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -81,11 +81,11 @@ void visit_optional(Visitor *v, bool *present, const char *name,
}
}
-void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
+void visit_get_next_type(Visitor *v, qtype_code *type,
const char *name, Error **errp)
{
if (v->get_next_type) {
- v->get_next_type(v, obj, qtypes, name, errp);
+ v->get_next_type(v, type, name, errp);
}
}
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 5dd9ed5..803ffad 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -208,7 +208,7 @@ static void qmp_input_end_list(Visitor *v, Error **errp)
qmp_input_pop(qiv, errp);
}
-static void qmp_input_get_next_type(Visitor *v, int *kind, const int *qobjects,
+static void qmp_input_get_next_type(Visitor *v, qtype_code *type,
const char *name, Error **errp)
{
QmpInputVisitor *qiv = to_qiv(v);
@@ -218,7 +218,7 @@ static void qmp_input_get_next_type(Visitor *v, int *kind, const int *qobjects,
error_setg(errp, QERR_MISSING_PARAMETER, name ? name : "null");
return;
}
- *kind = qobjects[qobject_type(qobj)];
+ *type = qobject_type(qobj);
}
static void qmp_input_type_int(Visitor *v, int64_t *obj, const char *name,
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 403768b..3fd07fd 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -47,7 +47,7 @@ def gen_struct_field(member):
ret += mcgen('''
%(c_type)s %(c_name)s;
''',
- c_type=member.type.c_type(), c_name=c_name(member.name))
+ c_type=member.c_type(), c_name=c_name(member.name))
return ret
@@ -111,38 +111,6 @@ static inline %(base)s *qapi_%(c_name)s_base(const %(c_name)s *obj)
c_name=c_name(name), base=base.c_name())
-def gen_alternate_qtypes_decl(name):
- return mcgen('''
-
-extern const int %(c_name)s_qtypes[];
-''',
- c_name=c_name(name))
-
-
-def gen_alternate_qtypes(name, variants):
- ret = mcgen('''
-
-const int %(c_name)s_qtypes[QTYPE_MAX] = {
-''',
- c_name=c_name(name))
-
- for var in variants.variants:
- qtype = var.type.alternate_qtype()
- assert qtype
-
- ret += mcgen('''
- [%(qtype)s] = %(enum_const)s,
-''',
- qtype=qtype,
- enum_const=c_enum_const(variants.tag_member.type.name,
- var.name))
-
- ret += mcgen('''
-};
-''')
- return ret
-
-
def gen_variants(variants):
# FIXME: What purpose does data serve, besides preventing a union that
# has a branch named 'data'? We use it in qapi-visit.py to decide
@@ -267,9 +235,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
def visit_alternate_type(self, name, info, variants):
self._fwdecl += gen_fwd_object_or_array(name)
- self._fwdefn += gen_alternate_qtypes(name, variants)
self.decl += gen_object(name, None, [variants.tag_member], variants)
- self.decl += gen_alternate_qtypes_decl(name)
self._gen_type_cleanup(name)
# If you link code generated from multiple schemata, you want only one
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 318b8e6..2091a0f 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -189,7 +189,7 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
if (err) {
goto out;
}
- visit_get_next_type(v, (int*) &(*obj)->type, %(c_name)s_qtypes, name, &err);
+ visit_get_next_type(v, &(*obj)->type, name, &err);
if (err) {
goto out_obj;
}
@@ -203,14 +203,14 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
visit_type_%(c_type)s(v, &(*obj)->u.%(c_name)s, name, &err);
break;
''',
- case=c_enum_const(variants.tag_member.type.name,
- var.name),
+ case=var.type.alternate_qtype(),
c_type=var.type.c_name(),
c_name=c_name(var.name))
ret += mcgen('''
default:
- abort();
+ error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+ "%(name)s");
}
out_obj:
error_propagate(errp, err);
@@ -219,7 +219,8 @@ out_obj:
out:
error_propagate(errp, err);
}
-''')
+''',
+ name=name)
return ret
@@ -425,6 +426,7 @@ fdef.write(mcgen('''
fdecl.write(mcgen('''
#include "qapi/visitor.h"
+#include "qapi/qmp/qerror.h"
#include "%(prefix)sqapi-types.h"
''',
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a0639e4..bd74470 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -627,15 +627,15 @@ def check_union(expr, expr_info):
def check_alternate(expr, expr_info):
name = expr['alternate']
members = expr['data']
- values = {'MAX': '(automatic)'}
+ values = {}
types_seen = {}
# Check every branch
for (key, value) in members.items():
check_name(expr_info, "Member of alternate '%s'" % name, key)
- # Check for conflicts in the generated enum
- c_key = camel_to_upper(key)
+ # Check for conflicts in the branch names
+ c_key = c_name(key)
if c_key in values:
raise QAPIExprError(expr_info,
"Alternate '%s' member '%s' clashes with '%s'"
@@ -1029,13 +1029,17 @@ class QAPISchemaObjectTypeMember(object):
assert self.type
seen[self.name] = self
+ def c_type(self):
+ return self.type.c_type()
+
class QAPISchemaObjectTypeVariants(object):
def __init__(self, tag_name, tag_member, variants):
# Flat unions pass tag_name but not tag_member.
# Simple unions and alternates pass tag_member but not tag_name.
# After check(), tag_member is always set, and tag_name remains
- # a reliable witness of being used by a flat union.
+ # a reliable witness of being used by a flat union, and
+ # tag_member.type being None is a reliable witness of an alternate.
assert bool(tag_member) != bool(tag_name)
assert (isinstance(tag_name, str) or
isinstance(tag_member, QAPISchemaObjectTypeMember))
@@ -1045,8 +1049,7 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
- # TODO drop union once alternates can be distinguished by tag_member
- def check(self, schema, seen, union=True):
+ def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
assert self.tag_member
@@ -1054,25 +1057,25 @@ class QAPISchemaObjectTypeVariants(object):
assert self.tag_member in seen.itervalues()
else: # alternate
self.tag_member.check(schema, seen)
- assert isinstance(self.tag_member.type, QAPISchemaEnumType)
+ if not isinstance(self.tag_member, QAPISchemaAlternateTypeTag):
+ assert isinstance(self.tag_member.type, QAPISchemaEnumType)
cases = OrderedDict()
for v in self.variants:
# Reset seen array for each variant, since QMP names from one
# branch do not affect another branch, nor add to all_members
- v.check(schema, self.tag_member.type, dict(seen), cases, union)
+ v.check(schema, self.tag_member.type, dict(seen), cases)
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
- # TODO drop union once alternates can be distinguished by tag_type
- def check(self, schema, tag_type, seen, cases, union):
+ def check(self, schema, tag_type, seen, cases):
# cases is case names we must not collide with
QAPISchemaObjectTypeMember.check(self, schema, cases)
- assert self.name in tag_type.values
- if union:
+ if tag_type:
# seen is QMP names our members must not collide with
+ assert self.name in tag_type.values
self.type.check_qmp(schema, seen)
# This function exists to support ugly simple union special cases
@@ -1094,7 +1097,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
- self.variants.check(schema, {}, False)
+ self.variants.check(schema, {})
def json_type(self):
return 'value'
@@ -1103,6 +1106,18 @@ class QAPISchemaAlternateType(QAPISchemaType):
visitor.visit_alternate_type(self.name, self.info, self.variants)
+class QAPISchemaAlternateTypeTag(QAPISchemaObjectTypeMember):
+ def __init__(self):
+ QAPISchemaObjectTypeMember.__init__(self, 'type', '', False)
+
+ def check(self, schema, seen):
+ assert len(seen) == 0
+ seen[self.name] = self
+
+ def c_type(self):
+ return 'qtype_code'
+
+
class QAPISchemaCommand(QAPISchemaEntity):
def __init__(self, name, info, arg_type, ret_type, gen, success_response):
QAPISchemaEntity.__init__(self, name, info)
@@ -1296,7 +1311,7 @@ class QAPISchema(object):
data = expr['data']
variants = [self._make_variant(key, value)
for (key, value) in data.iteritems()]
- tag_member = self._make_implicit_tag(name, info, variants)
+ tag_member = QAPISchemaAlternateTypeTag()
self._def_entity(
QAPISchemaAlternateType(name, info,
QAPISchemaObjectTypeVariants(None,
diff --git a/tests/qapi-schema/alternate-empty.out b/tests/qapi-schema/alternate-empty.out
index 0f153b6..9b010d8 100644
--- a/tests/qapi-schema/alternate-empty.out
+++ b/tests/qapi-schema/alternate-empty.out
@@ -1,4 +1,3 @@
object :empty
alternate Alt
case i: int
-enum AltKind ['i']
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 4354604..4513d94 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -131,7 +131,7 @@
'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero',
'u': 'UserDefZero', 'type': 'UserDefZero' } }
{ 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool',
- 'myKind': 'has_a' } }
+ 'myKind': 'has_a', 'max': 'str' } }
# testing commands
{ 'command': 'user_def_cmd', 'data': {} }
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 29c46da..eda71b9 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -60,32 +60,26 @@ object :obj-user_def_cmd2-arg
alternate AltIntNum
case i: int
case n: number
-enum AltIntNumKind ['i', 'n']
alternate AltName
case type: int
case u: bool
case myKind: has_a
-enum AltNameKind ['type', 'u', 'myKind']
+ case max: str
alternate AltNumInt
case n: number
case i: int
-enum AltNumIntKind ['n', 'i']
alternate AltNumStr
case n: number
case s: str
-enum AltNumStrKind ['n', 's']
alternate AltStrBool
case s: str
case b: bool
-enum AltStrBoolKind ['s', 'b']
alternate AltStrInt
case s: str
case i: int
-enum AltStrIntKind ['s', 'i']
alternate AltStrNum
case s: str
case n: number
-enum AltStrNumKind ['s', 'n']
event EVENT_A None
event EVENT_B None
event EVENT_C :obj-EVENT_C-arg
@@ -129,7 +123,6 @@ alternate UserDefAlternate
case uda: UserDefA
case s: str
case i: int
-enum UserDefAlternateKind ['uda', 's', 'i']
object UserDefB
member intb: int optional=False
member a-b: bool optional=True
@@ -198,7 +191,6 @@ event __ORG.QEMU_X-EVENT __org.qemu_x-Struct
alternate __org.qemu_x-Alt
case __org.qemu_x-branch: str
case b: __org.qemu_x-Base
-enum __org.qemu_x-AltKind ['__org.qemu_x-branch', 'b']
object __org.qemu_x-Base
member __org.qemu_x-member1: __org.qemu_x-Enum optional=False
enum __org.qemu_x-Enum ['__org.qemu_x-value']
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 3f6bc4d..552fc74 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -343,14 +343,14 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
v = visitor_input_test_init(data, "42");
visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
- g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_I);
+ g_assert_cmpint(tmp->type, ==, QTYPE_QINT);
g_assert_cmpint(tmp->u.i, ==, 42);
qapi_free_UserDefAlternate(tmp);
visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "'string'");
visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
- g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_S);
+ g_assert_cmpint(tmp->type, ==, QTYPE_QSTRING);
g_assert_cmpstr(tmp->u.s, ==, "string");
qapi_free_UserDefAlternate(tmp);
visitor_input_teardown(data, NULL);
@@ -386,11 +386,10 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
qapi_free_AltStrBool(asb);
visitor_input_teardown(data, NULL);
- /* FIXME: Order of alternate should not affect semantics; asn should
- * parse the same as ans */
+ /* FIXME: integer should parse as number */
v = visitor_input_test_init(data, "42");
visit_type_AltStrNum(v, &asn, NULL, &err);
- /* FIXME g_assert_cmpint(asn->type, == ALT_STR_NUM_KIND_N); */
+ /* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */
/* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */
g_assert(err);
error_free(err);
@@ -398,30 +397,34 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
qapi_free_AltStrNum(asn);
visitor_input_teardown(data, NULL);
+ /* FIXME: integer should parse as number */
v = visitor_input_test_init(data, "42");
- visit_type_AltNumStr(v, &ans, NULL, &error_abort);
- g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
- g_assert_cmpfloat(ans->u.n, ==, 42);
+ visit_type_AltNumStr(v, &ans, NULL, &err);
+ /* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */
+ /* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */
+ g_assert(err);
+ error_free(err);
+ err = NULL;
qapi_free_AltNumStr(ans);
visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltStrInt(v, &asi, NULL, &error_abort);
- g_assert_cmpint(asi->type, ==, ALT_STR_INT_KIND_I);
+ g_assert_cmpint(asi->type, ==, QTYPE_QINT);
g_assert_cmpint(asi->u.i, ==, 42);
qapi_free_AltStrInt(asi);
visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltIntNum(v, &ain, NULL, &error_abort);
- g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_I);
+ g_assert_cmpint(ain->type, ==, QTYPE_QINT);
g_assert_cmpint(ain->u.i, ==, 42);
qapi_free_AltIntNum(ain);
visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltNumInt(v, &ani, NULL, &error_abort);
- g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_I);
+ g_assert_cmpint(ani->type, ==, QTYPE_QINT);
g_assert_cmpint(ani->u.i, ==, 42);
qapi_free_AltNumInt(ani);
visitor_input_teardown(data, NULL);
@@ -438,14 +441,14 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrNum(v, &asn, NULL, &error_abort);
- g_assert_cmpint(asn->type, ==, ALT_STR_NUM_KIND_N);
+ g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(asn->u.n, ==, 42.5);
qapi_free_AltStrNum(asn);
visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltNumStr(v, &ans, NULL, &error_abort);
- g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
+ g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ans->u.n, ==, 42.5);
qapi_free_AltNumStr(ans);
visitor_input_teardown(data, NULL);
@@ -460,14 +463,14 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
v = visitor_input_test_init(data, "42.5");
visit_type_AltIntNum(v, &ain, NULL, &error_abort);
- g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_N);
+ g_assert_cmpint(ain->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ain->u.n, ==, 42.5);
qapi_free_AltIntNum(ain);
visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltNumInt(v, &ani, NULL, &error_abort);
- g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_N);
+ g_assert_cmpint(ani->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ani->u.n, ==, 42.5);
qapi_free_AltNumInt(ani);
visitor_input_teardown(data, NULL);
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 9364843..695db32 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -449,20 +449,31 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
const void *unused)
{
QObject *arg;
- Error *err = NULL;
+ UserDefAlternate *tmp;
- UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
- tmp->type = USER_DEF_ALTERNATE_KIND_I;
+ tmp = g_new0(UserDefAlternate, 1);
+ tmp->type = QTYPE_QINT;
tmp->u.i = 42;
- visit_type_UserDefAlternate(data->ov, &tmp, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
arg = qmp_output_get_qobject(data->qov);
g_assert(qobject_type(arg) == QTYPE_QINT);
g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
qapi_free_UserDefAlternate(tmp);
+
+ tmp = g_malloc0(sizeof(UserDefAlternate));
+ tmp->type = QTYPE_QSTRING;
+ tmp->u.s = g_strdup("hello");
+
+ visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
+ arg = qmp_output_get_qobject(data->qov);
+
+ g_assert(qobject_type(arg) == QTYPE_QSTRING);
+ g_assert_cmpstr(qstring_get_str(qobject_to_qstring(arg)), ==, "hello");
+
+ qapi_free_UserDefAlternate(tmp);
}
static void test_visitor_out_empty(TestOutputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 11/17] qapi: Fix alternates that accept 'number' but not 'int'
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (9 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 12/17] qapi: Remove dead visitor code Eric Blake
` (5 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
The QMP input visitor allows integral values to be assigned by
promotion to a QTYPE_QFLOAT. However, when parsing an alternate,
we did not take this into account, such that an alternate that
accepts 'number' but not 'int' would reject integral values.
With this patch, we now have the following desirable table:
alternate has case selected for
'int' 'number' QTYPE_QINT QTYPE_QFLOAT
no no error error
no yes 'number' 'number'
yes no 'int' error
yes yes 'int' 'number'
While it is unlikely that we will ever use 'number' in an
alternate other than in the testsuite, it never hurts to be
more precise in what we allow.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: rebase to named .u union
v6: rebase onto earlier testsuite and gen_err_check() improvements
---
include/qapi/visitor-impl.h | 2 +-
include/qapi/visitor.h | 3 ++-
qapi/qapi-visit-core.c | 4 ++--
qapi/qmp-input-visitor.c | 4 ++++
scripts/qapi-visit.py | 9 +++++++--
tests/test-qmp-input-visitor.c | 20 ++++++--------------
6 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 6d95b36..1d09b7b 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -33,7 +33,7 @@ struct Visitor
void (*type_enum)(Visitor *v, int *obj, const char * const strings[],
const char *kind, const char *name, Error **errp);
/* May be NULL; most useful for input visitors. */
- void (*get_next_type)(Visitor *v, qtype_code *type,
+ void (*get_next_type)(Visitor *v, qtype_code *type, bool promote_int,
const char *name, Error **errp);
void (*type_int)(Visitor *v, int64_t *obj, const char *name, Error **errp);
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index b765993..baea594 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -46,8 +46,9 @@ void visit_optional(Visitor *v, bool *present, const char *name,
* Determine the qtype of the item @name in the current object visit.
* For input visitors, set *@type to the correct qtype of a qapi
* alternate type; for other visitors, leave *@type unchanged.
+ * If @promote_int, treat integers as QTYPE_FLOAT.
*/
-void visit_get_next_type(Visitor *v, qtype_code *type,
+void visit_get_next_type(Visitor *v, qtype_code *type, bool promote_int,
const char *name, Error **errp);
void visit_type_enum(Visitor *v, int *obj, const char * const strings[],
const char *kind, const char *name, Error **errp);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 3f24daa..884fe94 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -81,11 +81,11 @@ void visit_optional(Visitor *v, bool *present, const char *name,
}
}
-void visit_get_next_type(Visitor *v, qtype_code *type,
+void visit_get_next_type(Visitor *v, qtype_code *type, bool promote_int,
const char *name, Error **errp)
{
if (v->get_next_type) {
- v->get_next_type(v, type, name, errp);
+ v->get_next_type(v, type, promote_int, name, errp);
}
}
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 803ffad..5310db5 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -209,6 +209,7 @@ static void qmp_input_end_list(Visitor *v, Error **errp)
}
static void qmp_input_get_next_type(Visitor *v, qtype_code *type,
+ bool promote_int,
const char *name, Error **errp)
{
QmpInputVisitor *qiv = to_qiv(v);
@@ -219,6 +220,9 @@ static void qmp_input_get_next_type(Visitor *v, qtype_code *type,
return;
}
*type = qobject_type(qobj);
+ if (promote_int && *type == QTYPE_QINT) {
+ *type = QTYPE_QFLOAT;
+ }
}
static void qmp_input_type_int(Visitor *v, int64_t *obj, const char *name,
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 2091a0f..948016d 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -179,6 +179,11 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s *obj, const char *name, Error
def gen_visit_alternate(name, variants):
+ promote_int = 'true'
+ for var in variants.variants:
+ if var.type.alternate_qtype() == 'QTYPE_QINT':
+ promote_int = 'false'
+
ret = mcgen('''
void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error **errp)
@@ -189,13 +194,13 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
if (err) {
goto out;
}
- visit_get_next_type(v, &(*obj)->type, name, &err);
+ visit_get_next_type(v, &(*obj)->type, %(promote_int)s, name, &err);
if (err) {
goto out_obj;
}
switch ((*obj)->type) {
''',
- c_name=c_name(name))
+ c_name=c_name(name), promote_int=promote_int)
for var in variants.variants:
ret += mcgen('''
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 552fc74..5a5014b 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -386,25 +386,17 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
qapi_free_AltStrBool(asb);
visitor_input_teardown(data, NULL);
- /* FIXME: integer should parse as number */
v = visitor_input_test_init(data, "42");
- visit_type_AltStrNum(v, &asn, NULL, &err);
- /* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */
- /* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */
- g_assert(err);
- error_free(err);
- err = NULL;
+ visit_type_AltStrNum(v, &asn, NULL, &error_abort);
+ g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT);
+ g_assert_cmpfloat(asn->u.n, ==, 42);
qapi_free_AltStrNum(asn);
visitor_input_teardown(data, NULL);
- /* FIXME: integer should parse as number */
v = visitor_input_test_init(data, "42");
- visit_type_AltNumStr(v, &ans, NULL, &err);
- /* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */
- /* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */
- g_assert(err);
- error_free(err);
- err = NULL;
+ visit_type_AltNumStr(v, &ans, NULL, &error_abort);
+ g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT);
+ g_assert_cmpfloat(ans->u.n, ==, 42);
qapi_free_AltNumStr(ans);
visitor_input_teardown(data, NULL);
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 12/17] qapi: Remove dead visitor code
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (10 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 11/17] qapi: Fix alternates that accept 'number' but not 'int' Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 13/17] qapi: Plug leaks in test-qmp-* Eric Blake
` (4 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Commit cbc95538 removed unused start_handle() and end_handle(),
but forgot got remove their declarations.
Commit 4e27e819 introduced optional visitor callbacks for all
sorts of int types, but except for type_uint64 and type_size,
none of them have ever been supplied (the generic implementation
based on using type_int then bounds-checking works just fine).
In the interest of simplicity, it's easier to make the visitor
callback interface not have to worry about the other sizes.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: no change
v6: no change
---
include/qapi/visitor-impl.h | 19 +++----
include/qapi/visitor.h | 3 -
qapi/qapi-visit-core.c | 131 +++++++++++++++++---------------------------
3 files changed, 58 insertions(+), 95 deletions(-)
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 1d09b7b..370935a 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -1,7 +1,7 @@
/*
* Core Definitions for QAPI Visitor implementations
*
- * Copyright (C) 2012 Red Hat, Inc.
+ * Copyright (C) 2012, 2015 Red Hat, Inc.
*
* Author: Paolo Bonizni <pbonzini@redhat.com>
*
@@ -48,18 +48,15 @@ struct Visitor
void (*optional)(Visitor *v, bool *present, const char *name,
Error **errp);
- void (*type_uint8)(Visitor *v, uint8_t *obj, const char *name, Error **errp);
- void (*type_uint16)(Visitor *v, uint16_t *obj, const char *name, Error **errp);
- void (*type_uint32)(Visitor *v, uint32_t *obj, const char *name, Error **errp);
- void (*type_uint64)(Visitor *v, uint64_t *obj, const char *name, Error **errp);
- void (*type_int8)(Visitor *v, int8_t *obj, const char *name, Error **errp);
- void (*type_int16)(Visitor *v, int16_t *obj, const char *name, Error **errp);
- void (*type_int32)(Visitor *v, int32_t *obj, const char *name, Error **errp);
- void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error **errp);
- /* visit_type_size() falls back to (*type_uint64)() if type_size is unset */
- void (*type_size)(Visitor *v, uint64_t *obj, const char *name, Error **errp);
bool (*start_union)(Visitor *v, bool data_present, Error **errp);
void (*end_union)(Visitor *v, bool data_present, Error **errp);
+
+ /* Only required to visit uint64 differently than (*type_int)(). */
+ void (*type_uint64)(Visitor *v, uint64_t *obj, const char *name,
+ Error **errp);
+ /* Only required to visit sizes differently than (*type_uint64)(). */
+ void (*type_size)(Visitor *v, uint64_t *obj, const char *name,
+ Error **errp);
};
void input_type_enum(Visitor *v, int *obj, const char * const strings[],
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index baea594..67ddd83 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -27,9 +27,6 @@ typedef struct GenericList
struct GenericList *next;
} GenericList;
-void visit_start_handle(Visitor *v, void **obj, const char *kind,
- const char *name, Error **errp);
-void visit_end_handle(Visitor *v, Error **errp);
void visit_start_struct(Visitor *v, void **obj, const char *kind,
const char *name, size_t size, Error **errp);
void visit_end_struct(Visitor *v, Error **errp);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 884fe94..cbf7780 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -104,57 +104,48 @@ void visit_type_uint8(Visitor *v, uint8_t *obj, const char *name, Error **errp)
{
int64_t value;
- if (v->type_uint8) {
- v->type_uint8(v, obj, name, errp);
- } else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- if (value < 0 || value > UINT8_MAX) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name ? name : "null", "uint8_t");
- return;
- }
- *obj = value;
+ value = *obj;
+ v->type_int(v, &value, name, errp);
+ if (value < 0 || value > UINT8_MAX) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
+ name ? name : "null", "uint8_t");
+ return;
}
+ *obj = value;
}
-void visit_type_uint16(Visitor *v, uint16_t *obj, const char *name, Error **errp)
+void visit_type_uint16(Visitor *v, uint16_t *obj, const char *name,
+ Error **errp)
{
int64_t value;
- if (v->type_uint16) {
- v->type_uint16(v, obj, name, errp);
- } else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- if (value < 0 || value > UINT16_MAX) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name ? name : "null", "uint16_t");
- return;
- }
- *obj = value;
+ value = *obj;
+ v->type_int(v, &value, name, errp);
+ if (value < 0 || value > UINT16_MAX) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
+ name ? name : "null", "uint16_t");
+ return;
}
+ *obj = value;
}
-void visit_type_uint32(Visitor *v, uint32_t *obj, const char *name, Error **errp)
+void visit_type_uint32(Visitor *v, uint32_t *obj, const char *name,
+ Error **errp)
{
int64_t value;
- if (v->type_uint32) {
- v->type_uint32(v, obj, name, errp);
- } else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- if (value < 0 || value > UINT32_MAX) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name ? name : "null", "uint32_t");
- return;
- }
- *obj = value;
+ value = *obj;
+ v->type_int(v, &value, name, errp);
+ if (value < 0 || value > UINT32_MAX) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
+ name ? name : "null", "uint32_t");
+ return;
}
+ *obj = value;
}
-void visit_type_uint64(Visitor *v, uint64_t *obj, const char *name, Error **errp)
+void visit_type_uint64(Visitor *v, uint64_t *obj, const char *name,
+ Error **errp)
{
int64_t value;
@@ -171,77 +162,55 @@ void visit_type_int8(Visitor *v, int8_t *obj, const char *name, Error **errp)
{
int64_t value;
- if (v->type_int8) {
- v->type_int8(v, obj, name, errp);
- } else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- if (value < INT8_MIN || value > INT8_MAX) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name ? name : "null", "int8_t");
- return;
- }
- *obj = value;
+ value = *obj;
+ v->type_int(v, &value, name, errp);
+ if (value < INT8_MIN || value > INT8_MAX) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
+ name ? name : "null", "int8_t");
+ return;
}
+ *obj = value;
}
void visit_type_int16(Visitor *v, int16_t *obj, const char *name, Error **errp)
{
int64_t value;
- if (v->type_int16) {
- v->type_int16(v, obj, name, errp);
- } else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- if (value < INT16_MIN || value > INT16_MAX) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name ? name : "null", "int16_t");
- return;
- }
- *obj = value;
+ value = *obj;
+ v->type_int(v, &value, name, errp);
+ if (value < INT16_MIN || value > INT16_MAX) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
+ name ? name : "null", "int16_t");
+ return;
}
+ *obj = value;
}
void visit_type_int32(Visitor *v, int32_t *obj, const char *name, Error **errp)
{
int64_t value;
- if (v->type_int32) {
- v->type_int32(v, obj, name, errp);
- } else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- if (value < INT32_MIN || value > INT32_MAX) {
- error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
- name ? name : "null", "int32_t");
- return;
- }
- *obj = value;
+ value = *obj;
+ v->type_int(v, &value, name, errp);
+ if (value < INT32_MIN || value > INT32_MAX) {
+ error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
+ name ? name : "null", "int32_t");
+ return;
}
+ *obj = value;
}
void visit_type_int64(Visitor *v, int64_t *obj, const char *name, Error **errp)
{
- if (v->type_int64) {
- v->type_int64(v, obj, name, errp);
- } else {
- v->type_int(v, obj, name, errp);
- }
+ v->type_int(v, obj, name, errp);
}
void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp)
{
- int64_t value;
-
if (v->type_size) {
v->type_size(v, obj, name, errp);
- } else if (v->type_uint64) {
- v->type_uint64(v, obj, name, errp);
} else {
- value = *obj;
- v->type_int(v, &value, name, errp);
- *obj = value;
+ visit_type_uint64(v, obj, name, errp);
}
}
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 13/17] qapi: Plug leaks in test-qmp-*
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (11 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 12/17] qapi: Remove dead visitor code Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 14/17] qapi: Simplify error testing " Eric Blake
` (3 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Make valgrind happy with the current state of the tests, so that
it is easier to see if future patches introduce new memory problems
without being drowned in noise. Many of the leaks were due to
calling a second init without tearing down the data from an earlier
visit. But since teardown is already idempotent, and we already
register teardown as part of input_visitor_test_add(), it is nicer
to just make init() safe to call multiple times than it is to have
to make all tests call teardown.
Another common leak was forgetting to clean up an error object,
after testing that an error was raised.
Another leak was in test_visitor_in_struct_nested(), failing to
clean the base member of UserDefTwo. Cleaning that up left
check_and_free_str() as dead code (since using the qapi_free_*
takes care of recursion, and we don't want double frees).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: no change
v6: make init repeatable rather than adding teardown everywhere,
fix additional leak with UserDefTwo base, plug additional files
---
tests/test-qmp-input-strict.c | 10 ++++++++++
tests/test-qmp-input-visitor.c | 41 +++++++----------------------------------
tests/test-qmp-output-visitor.c | 4 +++-
3 files changed, 20 insertions(+), 35 deletions(-)
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index b44184f..910e2f9 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -77,6 +77,8 @@ static Visitor *validate_test_init_raw(TestInputVisitorData *data,
{
Visitor *v;
+ validate_teardown(data, NULL);
+
data->obj = qobject_from_json(json_string);
g_assert(data->obj != NULL);
@@ -193,6 +195,8 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
visit_type_TestStruct(v, &p, NULL, &err);
g_assert(err);
+ error_free(err);
+ /* FIXME: visitor should not allocate p when returning error */
if (p) {
g_free(p->string);
}
@@ -210,6 +214,7 @@ static void test_validate_fail_struct_nested(TestInputVisitorData *data,
visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefTwo(udp);
}
@@ -224,6 +229,7 @@ static void test_validate_fail_list(TestInputVisitorData *data,
visit_type_UserDefOneList(v, &head, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefOneList(head);
}
@@ -239,6 +245,7 @@ static void test_validate_fail_union_native_list(TestInputVisitorData *data,
visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefNativeListUnion(tmp);
}
@@ -253,6 +260,7 @@ static void test_validate_fail_union_flat(TestInputVisitorData *data,
visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefFlatUnion(tmp);
}
@@ -268,6 +276,7 @@ static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
visit_type_UserDefFlatUnion2(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefFlatUnion2(tmp);
}
@@ -282,6 +291,7 @@ static void test_validate_fail_alternate(TestInputVisitorData *data,
visit_type_UserDefAlternate(v, &tmp, NULL, &err);
g_assert(err);
+ error_free(err);
qapi_free_UserDefAlternate(tmp);
}
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 5a5014b..00b80c0 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -46,6 +46,8 @@ Visitor *visitor_input_test_init(TestInputVisitorData *data,
Visitor *v;
va_list ap;
+ visitor_input_teardown(data, NULL);
+
va_start(ap, json_string);
data->obj = qobject_from_jsonv(json_string, &ap);
va_end(ap);
@@ -177,12 +179,7 @@ static void test_visitor_in_enum(TestInputVisitorData *data,
visit_type_EnumOne(v, &res, NULL, &err);
g_assert(!err);
g_assert_cmpint(i, ==, res);
-
- visitor_input_teardown(data, NULL);
}
-
- data->obj = NULL;
- data->qiv = NULL;
}
@@ -205,12 +202,6 @@ static void test_visitor_in_struct(TestInputVisitorData *data,
g_free(p);
}
-static void check_and_free_str(char *str, const char *cmp)
-{
- g_assert_cmpstr(str, ==, cmp);
- g_free(str);
-}
-
static void test_visitor_in_struct_nested(TestInputVisitorData *data,
const void *unused)
{
@@ -226,17 +217,14 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
visit_type_UserDefTwo(v, &udp, NULL, &err);
g_assert(!err);
- check_and_free_str(udp->string0, "string0");
- check_and_free_str(udp->dict1->string1, "string1");
+ g_assert_cmpstr(udp->string0, ==, "string0");
+ g_assert_cmpstr(udp->dict1->string1, ==, "string1");
g_assert_cmpint(udp->dict1->dict2->userdef->integer, ==, 42);
- check_and_free_str(udp->dict1->dict2->userdef->string, "string");
- check_and_free_str(udp->dict1->dict2->string, "string2");
+ g_assert_cmpstr(udp->dict1->dict2->userdef->string, ==, "string");
+ g_assert_cmpstr(udp->dict1->dict2->string, ==, "string2");
g_assert(udp->dict1->has_dict3 == false);
- g_free(udp->dict1->dict2->userdef);
- g_free(udp->dict1->dict2);
- g_free(udp->dict1);
- g_free(udp);
+ qapi_free_UserDefTwo(udp);
}
static void test_visitor_in_list(TestInputVisitorData *data,
@@ -346,14 +334,12 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
g_assert_cmpint(tmp->type, ==, QTYPE_QINT);
g_assert_cmpint(tmp->u.i, ==, 42);
qapi_free_UserDefAlternate(tmp);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "'string'");
visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
g_assert_cmpint(tmp->type, ==, QTYPE_QSTRING);
g_assert_cmpstr(tmp->u.s, ==, "string");
qapi_free_UserDefAlternate(tmp);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "false");
visit_type_UserDefAlternate(v, &tmp, NULL, &err);
@@ -361,7 +347,6 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_UserDefAlternate(tmp);
- visitor_input_teardown(data, NULL);
}
static void test_visitor_in_alternate_number(TestInputVisitorData *data,
@@ -384,42 +369,36 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrBool(asb);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltStrNum(v, &asn, NULL, &error_abort);
g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(asn->u.n, ==, 42);
qapi_free_AltStrNum(asn);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltNumStr(v, &ans, NULL, &error_abort);
g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ans->u.n, ==, 42);
qapi_free_AltNumStr(ans);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltStrInt(v, &asi, NULL, &error_abort);
g_assert_cmpint(asi->type, ==, QTYPE_QINT);
g_assert_cmpint(asi->u.i, ==, 42);
qapi_free_AltStrInt(asi);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltIntNum(v, &ain, NULL, &error_abort);
g_assert_cmpint(ain->type, ==, QTYPE_QINT);
g_assert_cmpint(ain->u.i, ==, 42);
qapi_free_AltIntNum(ain);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42");
visit_type_AltNumInt(v, &ani, NULL, &error_abort);
g_assert_cmpint(ani->type, ==, QTYPE_QINT);
g_assert_cmpint(ani->u.i, ==, 42);
qapi_free_AltNumInt(ani);
- visitor_input_teardown(data, NULL);
/* Parsing a double */
@@ -429,21 +408,18 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrBool(asb);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrNum(v, &asn, NULL, &error_abort);
g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(asn->u.n, ==, 42.5);
qapi_free_AltStrNum(asn);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltNumStr(v, &ans, NULL, &error_abort);
g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ans->u.n, ==, 42.5);
qapi_free_AltNumStr(ans);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltStrInt(v, &asi, NULL, &err);
@@ -451,21 +427,18 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
error_free(err);
err = NULL;
qapi_free_AltStrInt(asi);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltIntNum(v, &ain, NULL, &error_abort);
g_assert_cmpint(ain->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ain->u.n, ==, 42.5);
qapi_free_AltIntNum(ain);
- visitor_input_teardown(data, NULL);
v = visitor_input_test_init(data, "42.5");
visit_type_AltNumInt(v, &ani, NULL, &error_abort);
g_assert_cmpint(ani->type, ==, QTYPE_QFLOAT);
g_assert_cmpfloat(ani->u.n, ==, 42.5);
qapi_free_AltNumInt(ani);
- visitor_input_teardown(data, NULL);
}
static void test_native_list_integer_helper(TestInputVisitorData *data,
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 695db32..42458d7 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -391,6 +391,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qobj = QOBJECT(qdict);
visit_type_any(data->ov, &qobj, NULL, &err);
g_assert(!err);
+ qobject_decref(qobj);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
qdict = qobject_to_qdict(obj);
@@ -411,7 +412,6 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
g_assert(qstring);
g_assert_cmpstr(qstring_get_str(qstring), ==, "foo");
qobject_decref(obj);
- qobject_decref(qobj);
}
static void test_visitor_out_union_flat(TestOutputVisitorData *data,
@@ -462,6 +462,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
qapi_free_UserDefAlternate(tmp);
+ qobject_decref(arg);
tmp = g_malloc0(sizeof(UserDefAlternate));
tmp->type = QTYPE_QSTRING;
@@ -474,6 +475,7 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
g_assert_cmpstr(qstring_get_str(qobject_to_qstring(arg)), ==, "hello");
qapi_free_UserDefAlternate(tmp);
+ qobject_decref(arg);
}
static void test_visitor_out_empty(TestOutputVisitorData *data,
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 14/17] qapi: Simplify error testing in test-qmp-*
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (12 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 13/17] qapi: Plug leaks in test-qmp-* Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 15/17] qapi: Test failure in middle of array parse Eric Blake
` (2 subsequent siblings)
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
By using &error_abort, we can avoid a local err variable in
situations where we expect success.
By moving err into data, we can let test teardown take care
of cleaning up any collected error (and allowing for fewer
lines of code between repeated tests where init runs teardown
on our behalf).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: pick up whitespace changes dropped from earlier commit
v6: new patch
---
tests/test-qmp-input-strict.c | 77 +++++++++-------------------
tests/test-qmp-input-visitor.c | 101 ++++++++++++-------------------------
tests/test-qmp-output-visitor.c | 52 +++++--------------
tests/test-visitor-serialization.c | 42 +++++++--------
4 files changed, 86 insertions(+), 186 deletions(-)
diff --git a/tests/test-qmp-input-strict.c b/tests/test-qmp-input-strict.c
index 910e2f9..f8da75c 100644
--- a/tests/test-qmp-input-strict.c
+++ b/tests/test-qmp-input-strict.c
@@ -26,6 +26,7 @@
typedef struct TestInputVisitorData {
QObject *obj;
QmpInputVisitor *qiv;
+ Error *err;
} TestInputVisitorData;
static void validate_teardown(TestInputVisitorData *data,
@@ -34,6 +35,9 @@ static void validate_teardown(TestInputVisitorData *data,
qobject_decref(data->obj);
data->obj = NULL;
+ error_free(data->err);
+ data->err = NULL;
+
if (data->qiv) {
qmp_input_visitor_cleanup(data->qiv);
data->qiv = NULL;
@@ -96,13 +100,11 @@ static void test_validate_struct(TestInputVisitorData *data,
const void *unused)
{
TestStruct *p = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
- visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(!err);
+ visit_type_TestStruct(v, &p, NULL, &error_abort);
g_free(p->string);
g_free(p);
}
@@ -111,7 +113,6 @@ static void test_validate_struct_nested(TestInputVisitorData *data,
const void *unused)
{
UserDefTwo *udp = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'string0': 'string0', "
@@ -119,8 +120,7 @@ static void test_validate_struct_nested(TestInputVisitorData *data,
"'dict2': { 'userdef': { 'integer': 42, "
"'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefTwo(v, &udp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefTwo(v, &udp, NULL, &error_abort);
qapi_free_UserDefTwo(udp);
}
@@ -128,13 +128,11 @@ static void test_validate_list(TestInputVisitorData *data,
const void *unused)
{
UserDefOneList *head = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44 } ]");
- visit_type_UserDefOneList(v, &head, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefOneList(v, &head, NULL, &error_abort);
qapi_free_UserDefOneList(head);
}
@@ -143,12 +141,10 @@ static void test_validate_union_native_list(TestInputVisitorData *data,
{
UserDefNativeListUnion *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data, "{ 'type': 'integer', 'data' : [ 1, 2 ] }");
- visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefNativeListUnion(v, &tmp, NULL, &error_abort);
qapi_free_UserDefNativeListUnion(tmp);
}
@@ -157,7 +153,6 @@ static void test_validate_union_flat(TestInputVisitorData *data,
{
UserDefFlatUnion *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data,
"{ 'enum1': 'value1', "
@@ -165,8 +160,7 @@ static void test_validate_union_flat(TestInputVisitorData *data,
"'string': 'str', "
"'boolean': true }");
- visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefFlatUnion(v, &tmp, NULL, &error_abort);
qapi_free_UserDefFlatUnion(tmp);
}
@@ -175,12 +169,10 @@ static void test_validate_alternate(TestInputVisitorData *data,
{
UserDefAlternate *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data, "42");
- visit_type_UserDefAlternate(v, &tmp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
qapi_free_UserDefAlternate(tmp);
}
@@ -188,14 +180,12 @@ static void test_validate_fail_struct(TestInputVisitorData *data,
const void *unused)
{
TestStruct *p = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo', 'extra': 42 }");
- visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_TestStruct(v, &p, NULL, &data->err);
+ g_assert(data->err);
/* FIXME: visitor should not allocate p when returning error */
if (p) {
g_free(p->string);
@@ -207,14 +197,12 @@ static void test_validate_fail_struct_nested(TestInputVisitorData *data,
const void *unused)
{
UserDefTwo *udp = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'string0': 'string0', 'dict1': { 'string1': 'string1', 'dict2': { 'userdef1': { 'integer': 42, 'string': 'string', 'extra': [42, 23, {'foo':'bar'}] }, 'string2': 'string2'}}}");
- visit_type_UserDefTwo(v, &udp, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_UserDefTwo(v, &udp, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefTwo(udp);
}
@@ -222,14 +210,12 @@ static void test_validate_fail_list(TestInputVisitorData *data,
const void *unused)
{
UserDefOneList *head = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44, 'extra': 'ggg' } ]");
- visit_type_UserDefOneList(v, &head, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_UserDefOneList(v, &head, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefOneList(head);
}
@@ -237,15 +223,13 @@ static void test_validate_fail_union_native_list(TestInputVisitorData *data,
const void *unused)
{
UserDefNativeListUnion *tmp = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data,
"{ 'type': 'integer', 'data' : [ 'string' ] }");
- visit_type_UserDefNativeListUnion(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_UserDefNativeListUnion(v, &tmp, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefNativeListUnion(tmp);
}
@@ -253,14 +237,12 @@ static void test_validate_fail_union_flat(TestInputVisitorData *data,
const void *unused)
{
UserDefFlatUnion *tmp = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init(data, "{ 'string': 'c', 'integer': 41, 'boolean': true }");
- visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_UserDefFlatUnion(v, &tmp, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefFlatUnion(tmp);
}
@@ -268,15 +250,13 @@ static void test_validate_fail_union_flat_no_discrim(TestInputVisitorData *data,
const void *unused)
{
UserDefFlatUnion2 *tmp = NULL;
- Error *err = NULL;
Visitor *v;
/* test situation where discriminator field ('enum1' here) is missing */
v = validate_test_init(data, "{ 'integer': 42, 'string': 'c', 'string1': 'd', 'string2': 'e' }");
- visit_type_UserDefFlatUnion2(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_UserDefFlatUnion2(v, &tmp, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefFlatUnion2(tmp);
}
@@ -285,13 +265,11 @@ static void test_validate_fail_alternate(TestInputVisitorData *data,
{
UserDefAlternate *tmp = NULL;
Visitor *v;
- Error *err = NULL;
v = validate_test_init(data, "3.14");
- visit_type_UserDefAlternate(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_UserDefAlternate(v, &tmp, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefAlternate(tmp);
}
@@ -299,16 +277,11 @@ static void do_test_validate_qmp_introspect(TestInputVisitorData *data,
const char *schema_json)
{
SchemaInfoList *schema = NULL;
- Error *err = NULL;
Visitor *v;
v = validate_test_init_raw(data, schema_json);
- visit_type_SchemaInfoList(v, &schema, NULL, &err);
- if (err) {
- fprintf(stderr, "%s", error_get_pretty(err));
- }
- g_assert(!err);
+ visit_type_SchemaInfoList(v, &schema, NULL, &error_abort);
g_assert(schema);
qapi_free_SchemaInfoList(schema);
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 00b80c0..4553210 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -22,6 +22,7 @@
typedef struct TestInputVisitorData {
QObject *obj;
QmpInputVisitor *qiv;
+ Error *err;
} TestInputVisitorData;
static void visitor_input_teardown(TestInputVisitorData *data,
@@ -30,6 +31,9 @@ static void visitor_input_teardown(TestInputVisitorData *data,
qobject_decref(data->obj);
data->obj = NULL;
+ error_free(data->err);
+ data->err = NULL;
+
if (data->qiv) {
qmp_input_visitor_cleanup(data->qiv);
data->qiv = NULL;
@@ -92,13 +96,11 @@ static void test_visitor_in_int(TestInputVisitorData *data,
const void *unused)
{
int64_t res = 0, value = -42;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "%" PRId64, value);
- visit_type_int(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_int(v, &res, NULL, &error_abort);
g_assert_cmpint(res, ==, value);
}
@@ -106,7 +108,6 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
const void *unused)
{
int64_t res = 0;
- Error *err = NULL;
Visitor *v;
/* this will overflow a Qint/int64, so should be deserialized into
@@ -115,22 +116,19 @@ static void test_visitor_in_int_overflow(TestInputVisitorData *data,
*/
v = visitor_input_test_init(data, "%f", DBL_MAX);
- visit_type_int(v, &res, NULL, &err);
- g_assert(err);
- error_free(err);
+ visit_type_int(v, &res, NULL, &data->err);
+ g_assert(data->err);
}
static void test_visitor_in_bool(TestInputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
bool res = false;
Visitor *v;
v = visitor_input_test_init(data, "true");
- visit_type_bool(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_bool(v, &res, NULL, &error_abort);
g_assert_cmpint(res, ==, true);
}
@@ -138,13 +136,11 @@ static void test_visitor_in_number(TestInputVisitorData *data,
const void *unused)
{
double res = 0, value = 3.14;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "%f", value);
- visit_type_number(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_number(v, &res, NULL, &error_abort);
g_assert_cmpfloat(res, ==, value);
}
@@ -152,13 +148,11 @@ static void test_visitor_in_string(TestInputVisitorData *data,
const void *unused)
{
char *res = NULL, *value = (char *) "Q E M U";
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "%s", value);
- visit_type_str(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_str(v, &res, NULL, &error_abort);
g_assert_cmpstr(res, ==, value);
g_free(res);
@@ -167,7 +161,6 @@ static void test_visitor_in_string(TestInputVisitorData *data,
static void test_visitor_in_enum(TestInputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
Visitor *v;
EnumOne i;
@@ -176,8 +169,7 @@ static void test_visitor_in_enum(TestInputVisitorData *data,
v = visitor_input_test_init(data, "%s", EnumOne_lookup[i]);
- visit_type_EnumOne(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_EnumOne(v, &res, NULL, &error_abort);
g_assert_cmpint(i, ==, res);
}
}
@@ -187,13 +179,11 @@ static void test_visitor_in_struct(TestInputVisitorData *data,
const void *unused)
{
TestStruct *p = NULL;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
- visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(!err);
+ visit_type_TestStruct(v, &p, NULL, &error_abort);
g_assert_cmpint(p->integer, ==, -42);
g_assert(p->boolean == true);
g_assert_cmpstr(p->string, ==, "foo");
@@ -206,7 +196,6 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
const void *unused)
{
UserDefTwo *udp = NULL;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "{ 'string0': 'string0', "
@@ -214,8 +203,7 @@ static void test_visitor_in_struct_nested(TestInputVisitorData *data,
"'dict2': { 'userdef': { 'integer': 42, "
"'string': 'string' }, 'string': 'string2'}}}");
- visit_type_UserDefTwo(v, &udp, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefTwo(v, &udp, NULL, &error_abort);
g_assert_cmpstr(udp->string0, ==, "string0");
g_assert_cmpstr(udp->dict1->string1, ==, "string1");
@@ -231,14 +219,12 @@ static void test_visitor_in_list(TestInputVisitorData *data,
const void *unused)
{
UserDefOneList *item, *head = NULL;
- Error *err = NULL;
Visitor *v;
int i;
v = visitor_input_test_init(data, "[ { 'string': 'string0', 'integer': 42 }, { 'string': 'string1', 'integer': 43 }, { 'string': 'string2', 'integer': 44 } ]");
- visit_type_UserDefOneList(v, &head, NULL, &err);
- g_assert(!err);
+ visit_type_UserDefOneList(v, &head, NULL, &error_abort);
g_assert(head != NULL);
for (i = 0, item = head; item; item = item->next, i++) {
@@ -256,7 +242,6 @@ static void test_visitor_in_any(TestInputVisitorData *data,
const void *unused)
{
QObject *res = NULL;
- Error *err = NULL;
Visitor *v;
QInt *qint;
QBool *qbool;
@@ -265,16 +250,14 @@ static void test_visitor_in_any(TestInputVisitorData *data,
QObject *qobj;
v = visitor_input_test_init(data, "-42");
- visit_type_any(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_any(v, &res, NULL, &error_abort);
qint = qobject_to_qint(res);
g_assert(qint);
g_assert_cmpint(qint_get_int(qint), ==, -42);
qobject_decref(res);
v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }");
- visit_type_any(v, &res, NULL, &err);
- g_assert(!err);
+ visit_type_any(v, &res, NULL, &error_abort);
qdict = qobject_to_qdict(res);
g_assert(qdict && qdict_size(qdict) == 3);
qobj = qdict_get(qdict, "integer");
@@ -299,7 +282,6 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,
const void *unused)
{
Visitor *v;
- Error *err = NULL;
UserDefFlatUnion *tmp;
UserDefUnionBase *base;
@@ -309,8 +291,7 @@ static void test_visitor_in_union_flat(TestInputVisitorData *data,
"'string': 'str', "
"'boolean': true }");
- visit_type_UserDefFlatUnion(v, &tmp, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefFlatUnion(v, &tmp, NULL, &error_abort);
g_assert_cmpint(tmp->enum1, ==, ENUM_ONE_VALUE1);
g_assert_cmpstr(tmp->string, ==, "str");
g_assert_cmpint(tmp->integer, ==, 41);
@@ -326,7 +307,6 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
const void *unused)
{
Visitor *v;
- Error *err = NULL;
UserDefAlternate *tmp;
v = visitor_input_test_init(data, "42");
@@ -342,10 +322,8 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
qapi_free_UserDefAlternate(tmp);
v = visitor_input_test_init(data, "false");
- visit_type_UserDefAlternate(v, &tmp, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ visit_type_UserDefAlternate(v, &tmp, NULL, &data->err);
+ g_assert(data->err);
qapi_free_UserDefAlternate(tmp);
}
@@ -353,7 +331,6 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
const void *unused)
{
Visitor *v;
- Error *err = NULL;
AltStrBool *asb;
AltStrNum *asn;
AltNumStr *ans;
@@ -364,10 +341,8 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
/* Parsing an int */
v = visitor_input_test_init(data, "42");
- visit_type_AltStrBool(v, &asb, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ visit_type_AltStrBool(v, &asb, NULL, &data->err);
+ g_assert(data->err);
qapi_free_AltStrBool(asb);
v = visitor_input_test_init(data, "42");
@@ -403,10 +378,8 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
/* Parsing a double */
v = visitor_input_test_init(data, "42.5");
- visit_type_AltStrBool(v, &asb, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ visit_type_AltStrBool(v, &asb, NULL, &data->err);
+ g_assert(data->err);
qapi_free_AltStrBool(asb);
v = visitor_input_test_init(data, "42.5");
@@ -422,10 +395,8 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
qapi_free_AltNumStr(ans);
v = visitor_input_test_init(data, "42.5");
- visit_type_AltStrInt(v, &asi, NULL, &err);
- g_assert(err);
- error_free(err);
- err = NULL;
+ visit_type_AltStrInt(v, &asi, NULL, &data->err);
+ g_assert(data->err);
qapi_free_AltStrInt(asi);
v = visitor_input_test_init(data, "42.5");
@@ -446,7 +417,6 @@ static void test_native_list_integer_helper(TestInputVisitorData *data,
UserDefNativeListUnionKind kind)
{
UserDefNativeListUnion *cvalue = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -463,8 +433,7 @@ static void test_native_list_integer_helper(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, kind);
@@ -609,7 +578,6 @@ static void test_visitor_in_native_list_bool(TestInputVisitorData *data,
{
UserDefNativeListUnion *cvalue = NULL;
boolList *elem = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -626,8 +594,7 @@ static void test_visitor_in_native_list_bool(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, USER_DEF_NATIVE_LIST_UNION_KIND_BOOLEAN);
@@ -645,7 +612,6 @@ static void test_visitor_in_native_list_string(TestInputVisitorData *data,
{
UserDefNativeListUnion *cvalue = NULL;
strList *elem = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -661,8 +627,7 @@ static void test_visitor_in_native_list_string(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, USER_DEF_NATIVE_LIST_UNION_KIND_STRING);
@@ -684,7 +649,6 @@ static void test_visitor_in_native_list_number(TestInputVisitorData *data,
{
UserDefNativeListUnion *cvalue = NULL;
numberList *elem = NULL;
- Error *err = NULL;
Visitor *v;
GString *gstr_list = g_string_new("");
GString *gstr_union = g_string_new("");
@@ -700,8 +664,7 @@ static void test_visitor_in_native_list_number(TestInputVisitorData *data,
gstr_list->str);
v = visitor_input_test_init_raw(data, gstr_union->str);
- visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(v, &cvalue, NULL, &error_abort);
g_assert(cvalue != NULL);
g_assert_cmpint(cvalue->type, ==, USER_DEF_NATIVE_LIST_UNION_KIND_NUMBER);
@@ -734,18 +697,16 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
const void *unused)
{
TestStruct *p = NULL;
- Error *err = NULL;
Visitor *v;
v = visitor_input_test_init(data, "{ 'integer': false, 'boolean': 'foo', 'string': -42 }");
- visit_type_TestStruct(v, &p, NULL, &err);
- g_assert(err);
+ visit_type_TestStruct(v, &p, NULL, &data->err);
+ g_assert(data->err);
/* FIXME - a failed parse should not leave a partially-allocated p
* for us to clean up; this could cause callers to leak memory. */
g_assert(p->string == NULL);
- error_free(err);
g_free(p->string);
g_free(p);
}
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 42458d7..9d3bfca 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -45,11 +45,9 @@ static void test_visitor_out_int(TestOutputVisitorData *data,
const void *unused)
{
int64_t value = -42;
- Error *err = NULL;
QObject *obj;
- visit_type_int(data->ov, &value, NULL, &err);
- g_assert(!err);
+ visit_type_int(data->ov, &value, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -62,12 +60,10 @@ static void test_visitor_out_int(TestOutputVisitorData *data,
static void test_visitor_out_bool(TestOutputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
bool value = true;
QObject *obj;
- visit_type_bool(data->ov, &value, NULL, &err);
- g_assert(!err);
+ visit_type_bool(data->ov, &value, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -81,11 +77,9 @@ static void test_visitor_out_number(TestOutputVisitorData *data,
const void *unused)
{
double value = 3.14;
- Error *err = NULL;
QObject *obj;
- visit_type_number(data->ov, &value, NULL, &err);
- g_assert(!err);
+ visit_type_number(data->ov, &value, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -99,11 +93,9 @@ static void test_visitor_out_string(TestOutputVisitorData *data,
const void *unused)
{
char *string = (char *) "Q E M U";
- Error *err = NULL;
QObject *obj;
- visit_type_str(data->ov, &string, NULL, &err);
- g_assert(!err);
+ visit_type_str(data->ov, &string, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -117,12 +109,10 @@ static void test_visitor_out_no_string(TestOutputVisitorData *data,
const void *unused)
{
char *string = NULL;
- Error *err = NULL;
QObject *obj;
/* A null string should return "" */
- visit_type_str(data->ov, &string, NULL, &err);
- g_assert(!err);
+ visit_type_str(data->ov, &string, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -135,13 +125,11 @@ static void test_visitor_out_no_string(TestOutputVisitorData *data,
static void test_visitor_out_enum(TestOutputVisitorData *data,
const void *unused)
{
- Error *err = NULL;
QObject *obj;
EnumOne i;
for (i = 0; i < ENUM_ONE_MAX; i++) {
- visit_type_EnumOne(data->ov, &i, "unused", &err);
- g_assert(!err);
+ visit_type_EnumOne(data->ov, &i, "unused", &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -174,12 +162,10 @@ static void test_visitor_out_struct(TestOutputVisitorData *data,
.boolean = false,
.string = (char *) "foo"};
TestStruct *p = &test_struct;
- Error *err = NULL;
QObject *obj;
QDict *qdict;
- visit_type_TestStruct(data->ov, &p, NULL, &err);
- g_assert(!err);
+ visit_type_TestStruct(data->ov, &p, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -198,7 +184,6 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
const void *unused)
{
int64_t value = 42;
- Error *err = NULL;
UserDefTwo *ud2;
QObject *obj;
QDict *qdict, *dict1, *dict2, *dict3, *userdef;
@@ -225,8 +210,7 @@ static void test_visitor_out_struct_nested(TestOutputVisitorData *data,
ud2->dict1->dict3->userdef->integer = value;
ud2->dict1->dict3->string = g_strdup(strings[3]);
- visit_type_UserDefTwo(data->ov, &ud2, "unused", &err);
- g_assert(!err);
+ visit_type_UserDefTwo(data->ov, &ud2, "unused", &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -288,7 +272,6 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
const int max_items = 10;
bool value_bool = true;
int value_int = 10;
- Error *err = NULL;
QListEntry *entry;
QObject *obj;
QList *qlist;
@@ -306,8 +289,7 @@ static void test_visitor_out_list(TestOutputVisitorData *data,
head = p;
}
- visit_type_TestStructList(data->ov, &head, NULL, &err);
- g_assert(!err);
+ visit_type_TestStructList(data->ov, &head, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -367,7 +349,6 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
const void *unused)
{
QObject *qobj;
- Error *err = NULL;
QInt *qint;
QBool *qbool;
QString *qstring;
@@ -375,8 +356,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
QObject *obj;
qobj = QOBJECT(qint_from_int(-42));
- visit_type_any(data->ov, &qobj, NULL, &err);
- g_assert(!err);
+ visit_type_any(data->ov, &qobj, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
g_assert(qobject_type(obj) == QTYPE_QINT);
@@ -389,8 +369,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data,
qdict_put(qdict, "boolean", qbool_from_bool(true));
qdict_put(qdict, "string", qstring_from_str("foo"));
qobj = QOBJECT(qdict);
- visit_type_any(data->ov, &qobj, NULL, &err);
- g_assert(!err);
+ visit_type_any(data->ov, &qobj, NULL, &error_abort);
qobject_decref(qobj);
obj = qmp_output_get_qobject(data->qov);
g_assert(obj != NULL);
@@ -420,8 +399,6 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,
QObject *arg;
QDict *qdict;
- Error *err = NULL;
-
UserDefFlatUnion *tmp = g_malloc0(sizeof(UserDefFlatUnion));
tmp->enum1 = ENUM_ONE_VALUE1;
tmp->string = g_strdup("str");
@@ -429,8 +406,7 @@ static void test_visitor_out_union_flat(TestOutputVisitorData *data,
tmp->integer = 41;
tmp->u.value1->boolean = true;
- visit_type_UserDefFlatUnion(data->ov, &tmp, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefFlatUnion(data->ov, &tmp, NULL, &error_abort);
arg = qmp_output_get_qobject(data->qov);
g_assert(qobject_type(arg) == QTYPE_QDICT);
@@ -709,14 +685,12 @@ static void test_native_list(TestOutputVisitorData *data,
UserDefNativeListUnionKind kind)
{
UserDefNativeListUnion *cvalue = g_new0(UserDefNativeListUnion, 1);
- Error *err = NULL;
QObject *obj;
cvalue->type = kind;
init_native_list(cvalue);
- visit_type_UserDefNativeListUnion(data->ov, &cvalue, NULL, &err);
- g_assert(err == NULL);
+ visit_type_UserDefNativeListUnion(data->ov, &cvalue, NULL, &error_abort);
obj = qmp_output_get_qobject(data->qov);
check_native_list(obj, cvalue->type);
diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c
index c024e5e..9f67f9e 100644
--- a/tests/test-visitor-serialization.c
+++ b/tests/test-visitor-serialization.c
@@ -302,14 +302,13 @@ static void test_primitives(gconstpointer opaque)
const SerializeOps *ops = args->ops;
PrimitiveType *pt = args->test_data;
PrimitiveType *pt_copy = g_malloc0(sizeof(*pt_copy));
- Error *err = NULL;
void *serialize_data;
pt_copy->type = pt->type;
- ops->serialize(pt, &serialize_data, visit_primitive_type, &err);
- ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type, &err);
+ ops->serialize(pt, &serialize_data, visit_primitive_type, &error_abort);
+ ops->deserialize((void **)&pt_copy, serialize_data, visit_primitive_type,
+ &error_abort);
- g_assert(err == NULL);
g_assert(pt_copy != NULL);
if (pt->type == PTYPE_STRING) {
g_assert_cmpstr(pt->value.string, ==, pt_copy->value.string);
@@ -345,7 +344,6 @@ static void test_primitive_lists(gconstpointer opaque)
PrimitiveList pl = { .value = { NULL } };
PrimitiveList pl_copy = { .value = { NULL } };
PrimitiveList *pl_copy_ptr = &pl_copy;
- Error *err = NULL;
void *serialize_data;
void *cur_head = NULL;
int i;
@@ -492,10 +490,11 @@ static void test_primitive_lists(gconstpointer opaque)
}
}
- ops->serialize((void **)&pl, &serialize_data, visit_primitive_list, &err);
- ops->deserialize((void **)&pl_copy_ptr, serialize_data, visit_primitive_list, &err);
+ ops->serialize((void **)&pl, &serialize_data, visit_primitive_list,
+ &error_abort);
+ ops->deserialize((void **)&pl_copy_ptr, serialize_data,
+ visit_primitive_list, &error_abort);
- g_assert(err == NULL);
i = 0;
/* compare our deserialized list of primitives to the original */
@@ -652,10 +651,8 @@ static void test_primitive_lists(gconstpointer opaque)
g_assert_cmpint(i, ==, 33);
ops->cleanup(serialize_data);
- dealloc_helper(&pl, visit_primitive_list, &err);
- g_assert(!err);
- dealloc_helper(&pl_copy, visit_primitive_list, &err);
- g_assert(!err);
+ dealloc_helper(&pl, visit_primitive_list, &error_abort);
+ dealloc_helper(&pl_copy, visit_primitive_list, &error_abort);
g_free(args);
}
@@ -665,13 +662,12 @@ static void test_struct(gconstpointer opaque)
const SerializeOps *ops = args->ops;
TestStruct *ts = struct_create();
TestStruct *ts_copy = NULL;
- Error *err = NULL;
void *serialize_data;
- ops->serialize(ts, &serialize_data, visit_struct, &err);
- ops->deserialize((void **)&ts_copy, serialize_data, visit_struct, &err);
+ ops->serialize(ts, &serialize_data, visit_struct, &error_abort);
+ ops->deserialize((void **)&ts_copy, serialize_data, visit_struct,
+ &error_abort);
- g_assert(err == NULL);
struct_compare(ts, ts_copy);
struct_cleanup(ts);
@@ -687,14 +683,12 @@ static void test_nested_struct(gconstpointer opaque)
const SerializeOps *ops = args->ops;
UserDefTwo *udnp = nested_struct_create();
UserDefTwo *udnp_copy = NULL;
- Error *err = NULL;
void *serialize_data;
- ops->serialize(udnp, &serialize_data, visit_nested_struct, &err);
+ ops->serialize(udnp, &serialize_data, visit_nested_struct, &error_abort);
ops->deserialize((void **)&udnp_copy, serialize_data, visit_nested_struct,
- &err);
+ &error_abort);
- g_assert(err == NULL);
nested_struct_compare(udnp, udnp_copy);
nested_struct_cleanup(udnp);
@@ -709,7 +703,6 @@ static void test_nested_struct_list(gconstpointer opaque)
TestArgs *args = (TestArgs *) opaque;
const SerializeOps *ops = args->ops;
UserDefTwoList *listp = NULL, *tmp, *tmp_copy, *listp_copy = NULL;
- Error *err = NULL;
void *serialize_data;
int i = 0;
@@ -720,11 +713,10 @@ static void test_nested_struct_list(gconstpointer opaque)
listp = tmp;
}
- ops->serialize(listp, &serialize_data, visit_nested_struct_list, &err);
+ ops->serialize(listp, &serialize_data, visit_nested_struct_list,
+ &error_abort);
ops->deserialize((void **)&listp_copy, serialize_data,
- visit_nested_struct_list, &err);
-
- g_assert(err == NULL);
+ visit_nested_struct_list, &error_abort);
tmp = listp;
tmp_copy = listp_copy;
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 15/17] qapi: Test failure in middle of array parse
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (13 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 14/17] qapi: Simplify error testing " Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 16/17] qapi: More tests of input arrays Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 17/17] qapi: Simplify visits of optional fields Eric Blake
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Our generated list visitors have the same problem as has been
mentioned elsewhere (see commit 2f52e20): they allocate data
even on failure. An upcoming patch will correct things to
provide saner guarantees, but first we need to expose the
behavior in the testsuite to ensure we aren't introducing any
memory usage bugs.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: no change
v6: rebase onto earlier gen_err_check() and testsuite improvements
---
scripts/qapi-visit.py | 4 ++++
tests/test-qmp-input-visitor.c | 10 ++++++++++
2 files changed, 14 insertions(+)
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 948016d..4152f40 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -138,6 +138,10 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
def gen_visit_list(name, element_type):
+ # FIXME: if *obj is NULL on entry, and the first visit_next_list()
+ # assigns to *obj, while a later one fails, we should clean up *obj
+ # rather than leaving it non-NULL. As currently written, the caller must
+ # call qapi_free_FOOList() to avoid a memory leak of the partial FOOList.
return mcgen('''
void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error **errp)
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index 4553210..df0bceb 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -698,6 +698,7 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
{
TestStruct *p = NULL;
Visitor *v;
+ strList *q = NULL;
v = visitor_input_test_init(data, "{ 'integer': false, 'boolean': 'foo', 'string': -42 }");
@@ -709,6 +710,15 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
g_free(p->string);
g_free(p);
+
+ v = visitor_input_test_init(data, "[ '1', '2', false, '3' ]");
+ /* FIXME - a failed parse should not leave a partially-allocated
+ * array for us to clean up; this could cause callers to leak
+ * memory. */
+ visit_type_strList(v, &q, NULL, &data->err);
+ assert(q);
+ assert(data->err);
+ qapi_free_strList(q);
}
int main(int argc, char **argv)
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 16/17] qapi: More tests of input arrays
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (14 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 15/17] qapi: Test failure in middle of array parse Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 17/17] qapi: Simplify visits of optional fields Eric Blake
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Our testsuite had no coverage of empty arrays, nor of what
happens when the input does not match the expected type.
Useful to have, especially if we start changing the visitor
contracts.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: no change
v6: new patch
---
tests/test-qmp-input-visitor.c | 51 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
index df0bceb..b93f698 100644
--- a/tests/test-qmp-input-visitor.c
+++ b/tests/test-qmp-input-visitor.c
@@ -236,6 +236,12 @@ static void test_visitor_in_list(TestInputVisitorData *data,
}
qapi_free_UserDefOneList(head);
+ head = NULL;
+
+ /* An empty list is valid */
+ v = visitor_input_test_init(data, "[]");
+ visit_type_UserDefOneList(v, &head, NULL, &error_abort);
+ g_assert(!head);
}
static void test_visitor_in_any(TestInputVisitorData *data,
@@ -721,6 +727,49 @@ static void test_visitor_in_errors(TestInputVisitorData *data,
qapi_free_strList(q);
}
+static void test_visitor_in_wrong_type(TestInputVisitorData *data,
+ const void *unused)
+{
+ TestStruct *p = NULL;
+ Visitor *v;
+ strList *q = NULL;
+ int64_t i;
+
+ /* Make sure arrays and structs cannot be confused */
+
+ v = visitor_input_test_init(data, "[]");
+ visit_type_TestStruct(v, &p, NULL, &data->err);
+ g_assert(data->err);
+ g_assert(!p);
+
+ v = visitor_input_test_init(data, "{}");
+ visit_type_strList(v, &q, NULL, &data->err);
+ assert(data->err);
+ assert(!q);
+
+ /* Make sure primitives and struct cannot be confused */
+
+ v = visitor_input_test_init(data, "1");
+ visit_type_TestStruct(v, &p, NULL, &data->err);
+ g_assert(data->err);
+ g_assert(!p);
+
+ v = visitor_input_test_init(data, "{}");
+ visit_type_int(v, &i, NULL, &data->err);
+ assert(data->err);
+
+ /* Make sure primitives and arrays cannot be confused */
+
+ v = visitor_input_test_init(data, "1");
+ visit_type_strList(v, &q, NULL, &data->err);
+ assert(data->err);
+ assert(!q);
+
+ v = visitor_input_test_init(data, "[]");
+ visit_type_int(v, &i, NULL, &data->err);
+ assert(data->err);
+}
+
int main(int argc, char **argv)
{
TestInputVisitorData in_visitor_data;
@@ -753,6 +802,8 @@ int main(int argc, char **argv)
&in_visitor_data, test_visitor_in_alternate);
input_visitor_test_add("/visitor/input/errors",
&in_visitor_data, test_visitor_in_errors);
+ input_visitor_test_add("/visitor/input/wrong-type",
+ &in_visitor_data, test_visitor_in_wrong_type);
input_visitor_test_add("/visitor/input/alternate-number",
&in_visitor_data, test_visitor_in_alternate_number);
input_visitor_test_add("/visitor/input/native_list/int",
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8 17/17] qapi: Simplify visits of optional fields
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
` (15 preceding siblings ...)
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 16/17] qapi: More tests of input arrays Eric Blake
@ 2015-10-28 17:14 ` Eric Blake
16 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-28 17:14 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
None of the visitor callbacks would set an error when testing
if an optional field was present; make this part of the interface
contract by eliminating the errp argument. Then, for less code,
reflect the determined boolean value back to the caller instead
of making the caller read the boolean after the fact.
The resulting generated code has a nice diff:
|- visit_optional(v, &has_fdset_id, "fdset-id", &err);
|- if (err) {
|- goto out;
|- }
|- if (has_fdset_id) {
|+ if (visit_optional(v, &has_fdset_id, "fdset-id")) {
| visit_type_int(v, &fdset_id, "fdset-id", &err);
| if (err) {
| goto out;
| }
| }
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v8: no change
v7: rebase to no member.c_name()
v6: rebase onto earlier testsuite and gen_err_check() improvements
---
include/qapi/visitor-impl.h | 5 ++---
include/qapi/visitor.h | 10 ++++++++--
qapi/opts-visitor.c | 2 +-
qapi/qapi-visit-core.c | 6 +++---
qapi/qmp-input-visitor.c | 3 +--
qapi/string-input-visitor.c | 3 +--
scripts/qapi.py | 7 +------
7 files changed, 17 insertions(+), 19 deletions(-)
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 370935a..fd2e905 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -44,9 +44,8 @@ struct Visitor
void (*type_any)(Visitor *v, QObject **obj, const char *name,
Error **errp);
- /* May be NULL */
- void (*optional)(Visitor *v, bool *present, const char *name,
- Error **errp);
+ /* May be NULL; most useful for input visitors. */
+ void (*optional)(Visitor *v, bool *present, const char *name);
bool (*start_union)(Visitor *v, bool data_present, Error **errp);
void (*end_union)(Visitor *v, bool data_present, Error **errp);
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 67ddd83..e52ad39 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -36,8 +36,14 @@ void visit_end_implicit_struct(Visitor *v, Error **errp);
void visit_start_list(Visitor *v, const char *name, Error **errp);
GenericList *visit_next_list(Visitor *v, GenericList **list, Error **errp);
void visit_end_list(Visitor *v, Error **errp);
-void visit_optional(Visitor *v, bool *present, const char *name,
- Error **errp);
+
+/**
+ * Check if an optional member @name of an object needs visiting.
+ * For input visitors, set *@present according to whether the
+ * corresponding visit_type_*() needs calling; for other visitors,
+ * leave *@present unchanged. Return *@present for convenience.
+ */
+bool visit_optional(Visitor *v, bool *present, const char *name);
/**
* Determine the qtype of the item @name in the current object visit.
diff --git a/qapi/opts-visitor.c b/qapi/opts-visitor.c
index cd10392..ef5fb8b 100644
--- a/qapi/opts-visitor.c
+++ b/qapi/opts-visitor.c
@@ -488,7 +488,7 @@ opts_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp)
static void
-opts_optional(Visitor *v, bool *present, const char *name, Error **errp)
+opts_optional(Visitor *v, bool *present, const char *name)
{
OptsVisitor *ov = DO_UPCAST(OptsVisitor, visitor, v);
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index cbf7780..2594147 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -73,12 +73,12 @@ void visit_end_union(Visitor *v, bool data_present, Error **errp)
}
}
-void visit_optional(Visitor *v, bool *present, const char *name,
- Error **errp)
+bool visit_optional(Visitor *v, bool *present, const char *name)
{
if (v->optional) {
- v->optional(v, present, name, errp);
+ v->optional(v, present, name);
}
+ return *present;
}
void visit_get_next_type(Visitor *v, qtype_code *type, bool promote_int,
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index 5310db5..f714dfc 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -300,8 +300,7 @@ static void qmp_input_type_any(Visitor *v, QObject **obj, const char *name,
*obj = qobj;
}
-static void qmp_input_optional(Visitor *v, bool *present, const char *name,
- Error **errp)
+static void qmp_input_optional(Visitor *v, bool *present, const char *name)
{
QmpInputVisitor *qiv = to_qiv(v);
QObject *qobj = qmp_input_get_object(qiv, name, true);
diff --git a/qapi/string-input-visitor.c b/qapi/string-input-visitor.c
index bbd6a54..dee780a 100644
--- a/qapi/string-input-visitor.c
+++ b/qapi/string-input-visitor.c
@@ -299,8 +299,7 @@ static void parse_type_number(Visitor *v, double *obj, const char *name,
*obj = val;
}
-static void parse_optional(Visitor *v, bool *present, const char *name,
- Error **errp)
+static void parse_optional(Visitor *v, bool *present, const char *name)
{
StringInputVisitor *siv = DO_UPCAST(StringInputVisitor, visitor, v);
diff --git a/scripts/qapi.py b/scripts/qapi.py
index bd74470..0eb5d9a 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1619,15 +1619,10 @@ def gen_visit_fields(members, prefix='', need_cast=False, skiperr=False):
for memb in members:
if memb.optional:
ret += mcgen('''
- visit_optional(v, &%(prefix)shas_%(c_name)s, "%(name)s", %(errp)s);
+ if (visit_optional(v, &%(prefix)shas_%(c_name)s, "%(name)s")) {
''',
prefix=prefix, c_name=c_name(memb.name),
name=memb.name, errp=errparg)
- ret += gen_err_check(skiperr=skiperr)
- ret += mcgen('''
- if (%(prefix)shas_%(c_name)s) {
-''',
- prefix=prefix, c_name=c_name(memb.name))
push_indent()
# Ugly: sometimes we need to cast away const
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting Eric Blake
@ 2015-10-30 11:20 ` Markus Armbruster
2015-10-30 15:41 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-10-30 11:20 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
For now, only high-level review.
The main cost of sorting is interface complexity: we need to specify
which things are sorted, and what the sorting order is (see your TODO
below). Once we've done so, we can't go back.
There's also implementation complexity, but your patch shows it's low
enough to be ignored.
Cost needs be justified by benefits. Let's have a look at the benefits.
Eric Blake <eblake@redhat.com> writes:
> Sorting the values of an enum makes it easier to look up whether
> a particular value is present, via binary rather than linear search
> (probably most visible with QKeyCode, which has grown over
> several releases).
Making the interface guarantee "sorted" saves a client that wants it
sorted (say for binary search) the trouble of sorting itself. We sort
at QEMU compile time instead of client run time.
My qmp-introspect.c has 48 SchemaInfoEnum, ranging from one to 125
members. Histogram:
#enums with this #members
6 1
11 2
8 3
9 4
2 5
1 6
3 7
1 8
2 9
1 15
1 19
1 27
1 43
1 125
Mean is less than eight. Median is *three*.
Sorting these member arrays in the client won't have a noticeable
performance impact. Heck, I suspect even using linear search wouldn't!
Therefore, we can't really claim client performance as a benefit.
We might be able to claim reduced client complexity as a benefit, but
I'm rather skeptical. When your search space has a dozen members, you
better stick to simple search techniques. Linear search in array or
list and binary search in array look adequate, hash table already
approaches overkill, and anything fancier definitely is. Of these, only
binary search profits from a "sorted" guarantee. But when you got data
in an array already, all it takes to sort it is a call to qsort() and a
comparison function. Ten straightforward lines of code, tops. Less in
a language that isn't quite as primitive as C. Not much of a benefit,
I'm afraid.
> Additionally, QMP clients need not know which
> C value is associated with an enum name, so sorting is an
> effective way to hide that non-ABI aspect of qapi.
Sorting indeed hides the implementation detail of how enumerations are
encoded in the server. However, I can't see what would tempt clients
into relying on it. I can see for type names, and that's why we hide
them (commit 1a9a507).
One more potential benefit: when the schema changes in a way that
affects only order in introspection, sorting can hide the changes from
clients. Can't think of a practical use of it, though.
> While we are at it, it is also easy to sort the members and
> variants of objects, to allow for a similar binary search (although
> less compelling, as any struct with that many members should
> arguably be broken into hierarchical substructs), and equally
> valid since JSON objects have no specific order in which keys must
> appear.
My qmp-introspect.c has 48 SchemaInfoObject, ranging from zero to 27
members. Histogram:
#objs with this #members
3 0
102 2
58 3
28 4
14 5
17 6
15 7
3 8
2 9
7 10
2 11
3 12
1 13
2 14
1 15
1 16
1 27
Mean is 9.5, median is 9.
The variants are also enums, and therefore can't be any worse than
enums.
Again, client performance is hardly a benefit, and client complexity
isn't particularly convincing, either.
> There is no trivial or obvious way to sort the types of
> an alternate, so that is left unchanged.
In the schema, an alternate's member has a name, a type and nothing
else, so that's what we could use to sort. The name isn't visible in
introspection, and the type is masked. Sorting by either hides schema
change from the client, but isn't of much use to the client otherwise.
If you want to let the client use binary search without having to sort
itself, you obviously have to sort by masked type.
My qmp-introspect.c has *two* alternate types, each with two members.
> However, the overall SchemaInfo array remains unsorted. It might
> make sense to sort with 'meta-type' as a primary key and 'name'
> as a secondary key, but it is not obvious that this will provide
> benefits to end-user clients (we allow mutually recursive types,
> so there is no posible topological sorting where a single pass
> over the array could resolve all types, and while binary searches
> could be made possible by sorting, it would be even more efficient
> for clients to read the array into a hashtable for O(1) rather
> than O(log n) random-access lookups, at which point pre-sorting is
> wasted effort).
My qmp-introspect.c has:
2 alternate
55 array
5 builtin
126 command
48 enum
35 event
260 object
-------------
531 total
Since they all share the same entity name space, I'd use a single hash
table to map from name to SchemaInfo, just like qapi.py's QAPISchema
does.
Since we visit stuff in a defined order, qmp-introspect.c's order is
stable. The order just isn't particularly useful for clients, let alone
specified.
> Document these conventions, so that clients will know what can
> and cannot be relied on.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> TODO: should the documentation mention that the sorting is done
> in the C locale?
I'd assume C locale because we're sorting plain ASCII strings. But
spelling it out can't hurt.
> Is there anything required to ensure that python
> sorts sanely (ie. that the choice of locale while building
> doesn't cause inadvertent sorting differences such as turning on
> case-insensitivity)?
Good question.
> v8: no change
> v7: tweak commit wording
> v6: no change from v5
> ---
> docs/qapi-code-gen.txt | 21 +++++++++++++++++----
> qapi/introspect.json | 22 +++++++++++++++++-----
> scripts/qapi-introspect.py | 9 ++++++---
> 3 files changed, 40 insertions(+), 12 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index ba29bc6..163f547 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -516,6 +516,13 @@ query-qmp-schema. QGA currently doesn't support introspection.
>
> query-qmp-schema returns a JSON array of SchemaInfo objects. These
> objects together describe the wire ABI, as defined in the QAPI schema.
> +There is no specified order to the SchemaInfo objects returned; a
> +client must search for a particular name and meta-type throughout the
> +entire array to learn more about that name. For now, the name for
> +each SchemaInfo is unique thanks to qapi naming conventions; however
> +this may be changed in the future (such as allowing a command and an
> +event with the same name), so it is important that the client check
> +for the desired meta-type.
I feel separate name spaces aren't necessary and would be confusing, and
I'd be prepared to cast "single entity name space" in stone now.
>
> However, the SchemaInfo can't reflect all the rules and restrictions
> that apply to QMP. It's interface introspection (figuring out what's
> @@ -596,7 +603,8 @@ any. Each element is a JSON object with members "name" (the member's
> name), "type" (the name of its type), and optionally "default". The
> member is optional if "default" is present. Currently, "default" can
> only have value null. Other values are reserved for future
> -extensions.
> +extensions. The "members" array is sorted by "name", so that clients
> +can use a binary search to see if a particular member is supported.
>
> Example: the SchemaInfo for MyType from section Struct types
>
[...]
This all boils down to whether the increase in interface specification
complexity is justified by the benefits. The cost is small, but I'm
having a hard time seeing the benefits, to be honest.
Am I missing something?
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members Eric Blake
@ 2015-10-30 12:54 ` Markus Armbruster
2015-10-30 16:32 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-10-30 12:54 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> We were previously creating all unions with an empty list for
> local_members. However, it will make it easier to unify struct
> and union generation if we include the generated tag member in
> local_members. That way, we can have a common code pattern:
> visit the base (if any), visit the local members (if any), visit
> the variants (if any). The local_members of a flat union
> remains empty (because the discriminator is already visited as
> part of the base).
Keeping the implicit tag implicit by not including it in local_members
was a conscious design decision, but if including it makes unifying
struct and union into objects easier, go right ahead.
> The various front end entities then map as follows:
> struct: optional base, optional local_members, no variants
> simple union: no base, one-element local_members, variants with tag_member
> from local_members
> flat union: base, no local_members, variants with tag_member from base
> alternate: no base, no local_members, variants
>
> With the new local members, we require a bit of finesse to
> avoid assertions in the clients. No change to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v8: new patch
> ---
> scripts/qapi-types.py | 4 +++-
> scripts/qapi-visit.py | 4 +++-
> scripts/qapi.py | 14 ++++++++++----
> tests/qapi-schema/qapi-schema-test.out | 2 ++
> tests/qapi-schema/union-clash-data.out | 1 +
> tests/qapi-schema/union-empty.out | 1 +
> 6 files changed, 20 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index b37900f..a6bf95d 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -269,7 +269,9 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
> def visit_object_type(self, name, info, base, members, variants):
> self._fwdecl += gen_fwd_object_or_array(name)
> if variants:
> - assert not members # not implemented
> + if members:
> + assert len(members) == 1
> + assert members[0] == variants.tag_member
> self.decl += gen_union(name, base, variants)
> else:
> self.decl += gen_struct(name, base, members)
The assertion checks that not passing members to gen_union() won't lose
any. Before the patch, we assert there are none. After the patch, we
assert there's either none or variants.tag_member. Before ans after,
gen_union() takes care of variants.tag_member. Okay.
Let's keep the comment, though. Perhaps like this:
# Members other than variants.tag_member not implemented
assert len(members) == 1
assert members[0] == variants.tag_member
I hope this the whole conditional will eventually be replaced by
self.decl += gen_object(name, base, members, variants)
> diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
> index f40c3c7..318b8e6 100644
> --- a/scripts/qapi-visit.py
> +++ b/scripts/qapi-visit.py
> @@ -360,7 +360,9 @@ class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
> def visit_object_type(self, name, info, base, members, variants):
> self.decl += gen_visit_decl(name)
> if variants:
> - assert not members # not implemented
> + if members:
> + assert len(members) == 1
> + assert members[0] == variants.tag_member
> self.defn += gen_visit_union(name, base, variants)
> else:
> self.defn += gen_visit_struct(name, base, members)
Likewise, let's keep the comment.
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 7c50cc4..84ac151 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -957,6 +957,9 @@ class QAPISchemaArrayType(QAPISchemaType):
>
> class QAPISchemaObjectType(QAPISchemaType):
> def __init__(self, name, info, base, local_members, variants):
> + # struct has local_members, optional base, and no variants
> + # flat union has base, variants, and no local_members
> + # simple union has local_members, variants, and no base
> QAPISchemaType.__init__(self, name, info)
> assert base is None or isinstance(base, str)
> for m in local_members:
> @@ -1048,9 +1051,11 @@ class QAPISchemaObjectTypeVariants(object):
> self.variants = variants
>
> def check(self, schema, members, seen):
> - if self.tag_name:
> + if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
> - else:
> + elif seen: # simple union
> + assert self.tag_member in seen.itervalues()
> + else: # alternate
> self.tag_member.check(schema, members, seen)
> assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> for v in self.variants:
The test for simple union is hackish.
I guess you want to bypass self.tag_member.check() when for simple
unions, because it's now checked when QAPISchemaObjectType.check() calls
QAPISchemaObjectTypeMember.check() for each of self.local_members.
Could we instead make QAPISchemaAlternateType.check() call
QAPISchemaObjectTypeMember.check() for its implicit tag, and drop the
else here?
> @@ -1270,13 +1275,14 @@ class QAPISchema(object):
> if tag_name:
> variants = [self._make_variant(key, value)
> for (key, value) in data.iteritems()]
> + members = []
> else:
> variants = [self._make_simple_variant(key, value, info)
> for (key, value) in data.iteritems()]
> tag_member = self._make_implicit_tag(name, info, variants)
> + members = [tag_member]
> self._def_entity(
> - QAPISchemaObjectType(name, info, base,
> - self._make_members(OrderedDict(), info),
> + QAPISchemaObjectType(name, info, base, members,
> QAPISchemaObjectTypeVariants(tag_name,
> tag_member,
> variants)))
Two steps squashed together:
1. Simplify _make_members(OrderedDict(), info) to [].
2. Put a simple union's implicit tag_member into the members array
passed to QAPISchemaObjectType().
Okay.
[...]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
@ 2015-10-30 13:01 ` Markus Armbruster
2015-10-30 16:36 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-10-30 13:01 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> These two methods are now close enough that we can finally merge
> them, relying on the fact that simple unions now provide a
> reasonable local_members. Change gen_struct() to gen_object()
> that handles all forms of QAPISchemaObjectType, and rename and
> shrink gen_union() to gen_variants() to handle the portion of
> gen_object() needed when variants are present.
>
> gen_struct_fields() now has a single caller, so it no longer
> needs an optional parameter; however, I did not choose to inline
> it into the caller.
>
> No difference to generated code.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v8: new patch
> ---
> scripts/qapi-types.py | 36 +++++++++++-------------------------
> 1 file changed, 11 insertions(+), 25 deletions(-)
>
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index a6bf95d..403768b 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -51,7 +51,7 @@ def gen_struct_field(member):
> return ret
>
>
> -def gen_struct_fields(local_members, base=None):
> +def gen_struct_fields(local_members, base):
> ret = ''
>
> if base:
> @@ -70,7 +70,7 @@ def gen_struct_fields(local_members, base=None):
> return ret
>
>
> -def gen_struct(name, base, members):
> +def gen_object(name, base, members, variants):
> ret = mcgen('''
>
> struct %(c_name)s {
> @@ -79,11 +79,14 @@ struct %(c_name)s {
>
> ret += gen_struct_fields(members, base)
>
> + if variants:
> + ret += gen_variants(variants)
> +
> # Make sure that all structs have at least one field; this avoids
> # potential issues with attempting to malloc space for zero-length
> # structs in C, and also incompatibility with C++ (where an empty
> # struct is size 1).
> - if not (base and base.members) and not members:
> + if not (base and base.members) and not members and not variants:
> ret += mcgen('''
> char qapi_dummy_field_for_empty_struct;
> ''')
> @@ -140,17 +143,7 @@ const int %(c_name)s_qtypes[QTYPE_MAX] = {
> return ret
>
>
> -def gen_union(name, base, variants):
> - ret = mcgen('''
> -
> -struct %(c_name)s {
> -''',
> - c_name=c_name(name))
> - if base:
> - ret += gen_struct_fields([], base)
> - else:
> - ret += gen_struct_field(variants.tag_member)
> -
> +def gen_variants(variants):
> # FIXME: What purpose does data serve, besides preventing a union that
> # has a branch named 'data'? We use it in qapi-visit.py to decide
> # whether to bypass the switch statement if visiting the discriminator
> @@ -159,11 +152,11 @@ struct %(c_name)s {
> # should not be any data leaks even without a data pointer. Or, if
> # 'data' is merely added to guarantee we don't have an empty union,
> # shouldn't we enforce that at .json parse time?
> - ret += mcgen('''
> + ret = mcgen('''
> union { /* union tag is @%(c_name)s */
> void *data;
> ''',
> - c_name=c_name(variants.tag_member.name))
> + c_name=c_name(variants.tag_member.name))
>
> for var in variants.variants:
> # Ugly special case for simple union TODO get rid of it
> @@ -176,7 +169,6 @@ struct %(c_name)s {
>
> ret += mcgen('''
> } u;
> -};
> ''')
>
> return ret
> @@ -268,13 +260,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
>
> def visit_object_type(self, name, info, base, members, variants):
> self._fwdecl += gen_fwd_object_or_array(name)
> - if variants:
> - if members:
> - assert len(members) == 1
> - assert members[0] == variants.tag_member
> - self.decl += gen_union(name, base, variants)
> - else:
> - self.decl += gen_struct(name, base, members)
> + self.decl += gen_object(name, base, members, variants)
> if base:
> self.decl += gen_upcast(name, base)
> self._gen_type_cleanup(name)
> @@ -282,7 +268,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
> def visit_alternate_type(self, name, info, variants):
> self._fwdecl += gen_fwd_object_or_array(name)
> self._fwdefn += gen_alternate_qtypes(name, variants)
> - self.decl += gen_union(name, None, variants)
> + self.decl += gen_object(name, None, [variants.tag_member], variants)
> self.decl += gen_alternate_qtypes_decl(name)
> self._gen_type_cleanup(name)
Turned out nicely.
We could morph gen_struct_field() back into its original shape in a
separate patch:
def gen_struct_fields(members):
ret = ''
for memb in members:
ret += gen_struct_field(memb.name, memb.type, memb.optional)
return ret
with calling code
ret += mcgen('''
/* Members inherited from %(c_name)s: */
''',
c_name=base.c_name())
ret += gen_struct_fields(base.members)
ret += mcgen('''
/* Own members: */
''')
ret += gen_struct_fields(local_members)
Matter of taste.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting
2015-10-30 11:20 ` Markus Armbruster
@ 2015-10-30 15:41 ` Eric Blake
0 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-30 15:41 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 6008 bytes --]
On 10/30/2015 05:20 AM, Markus Armbruster wrote:
> For now, only high-level review.
>
> The main cost of sorting is interface complexity: we need to specify
> which things are sorted, and what the sorting order is (see your TODO
> below). Once we've done so, we can't go back.
>
> There's also implementation complexity, but your patch shows it's low
> enough to be ignored.
>
> Cost needs be justified by benefits. Let's have a look at the benefits.
>
> My qmp-introspect.c has 48 SchemaInfoEnum, ranging from one to 125
> members. Histogram:
>
> #enums with this #members
> 1 27
> 1 43
> 1 125
>
> Mean is less than eight. Median is *three*.
Binary searches have more overhead on small lists; you don't get the
benefits of the better algorithm until you overcome the initial hit of
more expensive lookups (bsearch() has to use a callback function, which
is intrinsically slower than doing direct comparison; the speedups are
only present if the search set is large enough that you are able to skip
enough comparisons to overcome the higher cost of the extra
indirection). So leaving enums unsorted is unlikely to slow clients
down (clients will have to do a linear search, but not the end of the
world since we have few large lists, and even among those, the size is
unlikely to grow much further).
>
> Sorting these member arrays in the client won't have a noticeable
> performance impact. Heck, I suspect even using linear search wouldn't!
> Therefore, we can't really claim client performance as a benefit.
So I'm inclined to agree with your conclusion.
>> Additionally, QMP clients need not know which
>> C value is associated with an enum name, so sorting is an
>> effective way to hide that non-ABI aspect of qapi.
>
> Sorting indeed hides the implementation detail of how enumerations are
> encoded in the server. However, I can't see what would tempt clients
> into relying on it. I can see for type names, and that's why we hide
> them (commit 1a9a507).
You have a point - even if a client figures out what integer value an
enum name maps to, it doesn't really help, because the client never
sends the integer over the wire.
>
> One more potential benefit: when the schema changes in a way that
> affects only order in introspection, sorting can hide the changes from
> clients. Can't think of a practical use of it, though.
Because we send names, not numbers, over the wire, we can insert a new
enum member in slot 0. Clients blindly doing strcmp(enum[0], "magic")
will get thrown off if "magic" moves to a different position - but this
is true whether or not we sort. So the only working clients are those
that check every enum position, not just enum[0]. So I agree - broken
clients are broken whether or not we sort, and working clients don't
miss anything whether or not we sort.
> My qmp-introspect.c has 48 SchemaInfoObject, ranging from zero to 27
> members. Histogram:
>
> #objs with this #members
> 1 15
> 1 16
> 1 27
Even less than the largest enum (and large structs are already unwieldy,
compared to making trees of substructs, so we're less likely to make
them too much bigger).
> Since we visit stuff in a defined order, qmp-introspect.c's order is
> stable. The order just isn't particularly useful for clients, let alone
> specified.
>
>> Document these conventions, so that clients will know what can
>> and cannot be relied on.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> TODO: should the documentation mention that the sorting is done
>> in the C locale?
>
> I'd assume C locale because we're sorting plain ASCII strings. But
> spelling it out can't hurt.
>
>> Is there anything required to ensure that python
>> sorts sanely (ie. that the choice of locale while building
>> doesn't cause inadvertent sorting differences such as turning on
>> case-insensitivity)?
>
> Good question.
And my biggest worry.
>> +++ b/docs/qapi-code-gen.txt
>> @@ -516,6 +516,13 @@ query-qmp-schema. QGA currently doesn't support introspection.
>>
>> query-qmp-schema returns a JSON array of SchemaInfo objects. These
>> objects together describe the wire ABI, as defined in the QAPI schema.
>> +There is no specified order to the SchemaInfo objects returned; a
>> +client must search for a particular name and meta-type throughout the
>> +entire array to learn more about that name. For now, the name for
>> +each SchemaInfo is unique thanks to qapi naming conventions; however
>> +this may be changed in the future (such as allowing a command and an
>> +event with the same name), so it is important that the client check
>> +for the desired meta-type.
>
> I feel separate name spaces aren't necessary and would be confusing, and
> I'd be prepared to cast "single entity name space" in stone now.
Sounds fair to me.
> This all boils down to whether the increase in interface specification
> complexity is justified by the benefits. The cost is small, but I'm
> having a hard time seeing the benefits, to be honest.
>
> Am I missing something?
No, I think you gave a fair review. I put this patch out there for
discussion and to see how hard it would be; but I'm not strongly tied to
it (if I have to pick only one patch out of 3/17 and 4/17, I'd rather
see 3/17 go in). So now that we've looked at the benefits (or rather,
that they aren't very strong), I'm okay with the idea of making my v9
spin of this series use an alternative patch that _just_ documents the
"single entity name space" and the fact that clients cannot rely on any
ordering (we don't sort - we may be deterministic, but we make no
guarantees on what that order is, and we are free to change the order in
the future).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members
2015-10-30 12:54 ` Markus Armbruster
@ 2015-10-30 16:32 ` Eric Blake
0 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-30 16:32 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 4179 bytes --]
On 10/30/2015 06:54 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> We were previously creating all unions with an empty list for
>> local_members. However, it will make it easier to unify struct
>> and union generation if we include the generated tag member in
>> local_members. That way, we can have a common code pattern:
>> visit the base (if any), visit the local members (if any), visit
>> the variants (if any). The local_members of a flat union
>> remains empty (because the discriminator is already visited as
>> part of the base).
>
> Keeping the implicit tag implicit by not including it in local_members
> was a conscious design decision, but if including it makes unifying
> struct and union into objects easier, go right ahead.
>
>> The various front end entities then map as follows:
>> struct: optional base, optional local_members, no variants
>> simple union: no base, one-element local_members, variants with tag_member
>> from local_members
>> flat union: base, no local_members, variants with tag_member from base
>> alternate: no base, no local_members, variants
>>
>> With the new local members, we require a bit of finesse to
>> avoid assertions in the clients. No change to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> +++ b/scripts/qapi-types.py
>> @@ -269,7 +269,9 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
>> def visit_object_type(self, name, info, base, members, variants):
>> self._fwdecl += gen_fwd_object_or_array(name)
>> if variants:
>> - assert not members # not implemented
>> + if members:
>> + assert len(members) == 1
>> + assert members[0] == variants.tag_member
>> self.decl += gen_union(name, base, variants)
>> else:
>> self.decl += gen_struct(name, base, members)
>
> The assertion checks that not passing members to gen_union() won't lose
> any. Before the patch, we assert there are none. After the patch, we
> assert there's either none or variants.tag_member. Before ans after,
> gen_union() takes care of variants.tag_member. Okay.
>
> Let's keep the comment, though. Perhaps like this:
>
> # Members other than variants.tag_member not implemented
> assert len(members) == 1
> assert members[0] == variants.tag_member
Indeed, that sounds nicer.
>
> I hope this the whole conditional will eventually be replaced by
>
> self.decl += gen_object(name, base, members, variants)
Yes. It goes away in 6/17 for qapi-types (as you already saw), and I
have a later patch queued that likewise makes it go away for qapi-visit
(but not in subset C or D, since there's still a bit more to clean up
there first).
>> def check(self, schema, members, seen):
>> - if self.tag_name:
>> + if self.tag_name: # flat union
>> self.tag_member = seen[self.tag_name]
>> - else:
>> + elif seen: # simple union
>> + assert self.tag_member in seen.itervalues()
>> + else: # alternate
>> self.tag_member.check(schema, members, seen)
>> assert isinstance(self.tag_member.type, QAPISchemaEnumType)
>> for v in self.variants:
>
> The test for simple union is hackish.
>
> I guess you want to bypass self.tag_member.check() when for simple
> unions, because it's now checked when QAPISchemaObjectType.check() calls
> QAPISchemaObjectTypeMember.check() for each of self.local_members.
>
> Could we instead make QAPISchemaAlternateType.check() call
> QAPISchemaObjectTypeMember.check() for its implicit tag, and drop the
> else here?
I debated about that when writing my patch. Yes, I can make that change
(it seems a little bit unclean to be calling
self.variants.tag_member.check() in QAPISchemaAlternateType, but not too
much worse; and adding a comment might help).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
2015-10-30 13:01 ` Markus Armbruster
@ 2015-10-30 16:36 ` Eric Blake
0 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-10-30 16:36 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2788 bytes --]
On 10/30/2015 07:01 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> These two methods are now close enough that we can finally merge
>> them, relying on the fact that simple unions now provide a
>> reasonable local_members. Change gen_struct() to gen_object()
>> that handles all forms of QAPISchemaObjectType, and rename and
>> shrink gen_union() to gen_variants() to handle the portion of
>> gen_object() needed when variants are present.
>>
>> gen_struct_fields() now has a single caller, so it no longer
>> needs an optional parameter; however, I did not choose to inline
>> it into the caller.
>>
>> No difference to generated code.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> def visit_object_type(self, name, info, base, members, variants):
>> self._fwdecl += gen_fwd_object_or_array(name)
>> - if variants:
>> - if members:
>> - assert len(members) == 1
>> - assert members[0] == variants.tag_member
>> - self.decl += gen_union(name, base, variants)
>> - else:
>> - self.decl += gen_struct(name, base, members)
>> + self.decl += gen_object(name, base, members, variants)
>> if base:
>> self.decl += gen_upcast(name, base)
>> self._gen_type_cleanup(name)
>> @@ -282,7 +268,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
>> def visit_alternate_type(self, name, info, variants):
>> self._fwdecl += gen_fwd_object_or_array(name)
>> self._fwdefn += gen_alternate_qtypes(name, variants)
>> - self.decl += gen_union(name, None, variants)
>> + self.decl += gen_object(name, None, [variants.tag_member], variants)
>> self.decl += gen_alternate_qtypes_decl(name)
>> self._gen_type_cleanup(name)
>
> Turned out nicely.
Yes, I was pretty pleased with it. It gives more credence to our choice
of introspection representation.
>
> We could morph gen_struct_field() back into its original shape in a
> separate patch:
>
> def gen_struct_fields(members):
> ret = ''
>
> for memb in members:
> ret += gen_struct_field(memb.name, memb.type, memb.optional)
> return ret
>
> with calling code
>
> ret += mcgen('''
> /* Members inherited from %(c_name)s: */
> ''',
> c_name=base.c_name())
> ret += gen_struct_fields(base.members)
> ret += mcgen('''
> /* Own members: */
> ''')
> ret += gen_struct_fields(local_members)
>
> Matter of taste.
Reasonable idea; I'll add it in for the v9 spin.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions Eric Blake
@ 2015-11-02 15:37 ` Markus Armbruster
2015-11-02 17:20 ` Eric Blake
` (6 more replies)
0 siblings, 7 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-02 15:37 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Now that we have separate namespaces for QMP vs. tag values,
What's the "QMP namespace"?
> we can simplify how the QAPISchema*.check() methods check for
> collisions.
I *guess* the point of this patch is dropping checks that are obsolete
now tag values no longer collide with non-variant member names in
generated C, with simplifications on top. Correct?
> Each QAPISchemaObjectTypeMember check() call is
> given a single set of names it must not collide with; this set
> is either the QMP names (when this member is used by an
> ObjectType) or the case names (when this member is used by an
> ObjectTypeVariants). We no longer need an all_members
> parameter, as it can be computed by seen.values(). When used
> by a union, QAPISchemaObjectTypeVariant must also perform a
> QMP collision check for each member of its corresponding type.
I'm afraid this explanation is next to impossible to understand unless
you know how the checking works. I should know, because I wrote it, but
actually don't, at least not by heart. So let me relearn how checking
works before your patch.
We're talking about a single invocation of QAPISchemaObjectType.check().
Its job is to compute self.members and self.base, and do sanity
checking, which includes checking for collisions. It has two variables
of interest:
* members is where we accumulate the list of members that'll become
self.members. It's initialized to the inherited members (empty if no
base, obviously).
* seen is a dictionary mapping names to members. This is merely for
collision checking. It's initialized to contain the inherited members
(whose names must be distinct, by induction, because we make sure the
base type's check() completes first).
For each local member m of self, we call m.check(schema, members, seen).
This is actually QAPISchemaObjectTypeMember.check(), and it uses members
and seen as follows:
* Assert m.name doesn't collide with another member's name, i.e. not in
seen.
* Append m to members.
* Insert m.name: m into seen.
Straightforward so far. Coming up next: variants.
Each variant's members are represented as a single member with the tag
value as name. Its type is an object type that has the variant's
members.
For each variant v:
* Copy seen to vseen. It holds the non-variant members.
* Call QAPISchemaObjectTypeMember.check(schema, [], vseen), which boils
down to assert v.name doesn't collide with a non-variant member's
name. This check is obsolete.
* Assert v.name is a member of tag_type.
Not checked: variant's name doesn't collide with another variant's name.
That's because we copy seen inside the loop instead of before the loop.
Not checked: variant's members don't collide with non-variant members.
I think this check got lost when we simplified
QAPISchemaObjectTypeVariants to hold a single member.
Note that the real checking happens in check_union(), and the checks
here are just sanity checks. As long as that's the case, holes here are
harmless. However, we need them plugged them when we move the real
checking from check_union() to the .check().
Looks like variant checking should be thrown out and redone.
I still don't get your description. Guess I have to read the patch
after all ;)
> The new ObjectType.check_qmp() is an idempotent subset of
> check(), and can be called multiple times over different seen
> sets (useful, since the members of one type can be applied
> into more than one other location via inheritance or flat
> union variants).
I think I get this argument. Except I don't get why the method's named
check_qmp().
> The code needs a temporary hack of passing a 'union' flag
> through Variants.check(), since we do not inline the branches
> of an alternate type into a parent QMP object.
What a "QMP object"? Sounds like a JSON object on the QMP wire, but
that makes no sense :)
> A later patch
> will rework how alternates are laid out, by adding a new
> subclass, and that will allow us to drop the extra parameter.
>
> There are no changes to generated code.
>
> Future patches will enhance testsuite coverage, improve error
> message quality on actual collisions, and move collision
> checks out of ad hoc parse code into the check() methods.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v8: rebase to earlier patches, defer positive test additions to
> later in series
> v7: new patch, although it is a much cleaner implementation of
> what was attempted by subset B v8 15/18
> https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03042.html
> ---
> scripts/qapi.py | 55 +++++++++++++++++++++++++++++++++----------------------
> 1 file changed, 33 insertions(+), 22 deletions(-)
>
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index 84ac151..c571709 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -972,28 +972,28 @@ class QAPISchemaObjectType(QAPISchemaType):
> self.variants = variants
> self.members = None
>
> + # Finish construction, and validate that all members are usable
> def check(self, schema):
> assert self.members is not False # not running in cycles
> if self.members:
> return
> self.members = False # mark as being checked
> + seen = OrderedDict()
Why do you need to switch from {} to OrderedDict()?
> if self._base_name:
> self.base = schema.lookup_type(self._base_name)
> - assert isinstance(self.base, QAPISchemaObjectType)
> - assert not self.base.variants # not implemented
> - self.base.check(schema)
> - members = list(self.base.members)
> - else:
> - members = []
> - seen = {}
> - for m in members:
> - assert c_name(m.name) not in seen
> - seen[m.name] = m
> + self.base.check_qmp(schema, seen)
> for m in self.local_members:
> - m.check(schema, members, seen)
> + m.check(schema, seen)
> if self.variants:
> - self.variants.check(schema, members, seen)
> - self.members = members
> + self.variants.check(schema, seen)
> + self.members = seen.values()
> +
> + # Check that this type does not introduce QMP collisions into seen
> + def check_qmp(self, schema, seen):
> + self.check(schema)
> + assert not self.variants # not implemented
> + for m in self.members:
> + m.check(schema, seen)
You said "ObjectType.check_qmp() is an idempotent subset of check()",
but it's obviously a superset: it calls check(), then does some more
work.
I think I'm now too confused to make further progress on this patch,
probably because I've thought myself into a corner.
Before I give up, a remark on design. The QAPISchemaFOO classes all
implement the same protocol:
* __init__() type-checks arguments and initializes instance variables,
generally either to an argument or to None. This is basically AST
construction.
* check() computes the remaining instance variables. This is semantic
analysis, except it's stubbed out.
QAPISchema.__init__() calls its own check(), which calls all the
others, directly (for entities), or indirect (for members and such).
Each check() runs *once*.
Except for QAPISchemaType.check(). Why? Unlike other entities, the
types need to be checked in a certain order: contained types (base and
variant) before the containing type. For simplicity, we simply call
their check(), and detect cycles. This is basically a topological
sort and the real checking squashed together. The real checking still
runs once.
It follows that you must not call check() except:
- An entity is responsible for calling check() for the non-entities it
owns (e.g. an object type's check() calls its members' check()).
- A type may call check() for a type it contains (e.g. its base type).
That's safe, because no type may contain itself. This kind of call
drives the top-sort.
* Obviously, the instance variables computed by check() have valid
values only after check(). Likewise, certain methods should be called
only after check(), e.g. visit().
Of course, if we find a more suitable protocol, we're free to adopt it.
>
> def is_implicit(self):
> # See QAPISchema._make_implicit_object_type()
> @@ -1027,11 +1027,13 @@ class QAPISchemaObjectTypeMember(object):
> self.type = None
> self.optional = optional
>
> - def check(self, schema, all_members, seen):
> + def check(self, schema, seen):
> + # seen is a map of names we must not collide with (either QMP
> + # names, when called by ObjectType, or case names, when called
> + # by Variant). This method is safe to call over multiple 'seen'.
What are "QMP names"? Member names, perhaps?
> assert self.name not in seen
> self.type = schema.lookup_type(self._type_name)
> assert self.type
> - all_members.append(self)
> seen[self.name] = self
>
>
> @@ -1050,26 +1052,35 @@ class QAPISchemaObjectTypeVariants(object):
> self.tag_member = tag_member
> self.variants = variants
>
> - def check(self, schema, members, seen):
> + # TODO drop union once alternates can be distinguished by tag_member
> + def check(self, schema, seen, union=True):
> if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
> + assert self.tag_member
> elif seen: # simple union
> assert self.tag_member in seen.itervalues()
> else: # alternate
> - self.tag_member.check(schema, members, seen)
> + self.tag_member.check(schema, seen)
> assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> + cases = OrderedDict()
> for v in self.variants:
> - vseen = dict(seen)
> - v.check(schema, self.tag_member.type, vseen)
> + # Reset seen array for each variant, since QMP names from one
> + # branch do not affect another branch, nor add to all_members
> + v.check(schema, self.tag_member.type, dict(seen), cases, union)
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> def __init__(self, name, typ):
> QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
>
> - def check(self, schema, tag_type, seen):
> - QAPISchemaObjectTypeMember.check(self, schema, [], seen)
> + # TODO drop union once alternates can be distinguished by tag_type
> + def check(self, schema, tag_type, seen, cases, union):
> + # cases is case names we must not collide with
> + QAPISchemaObjectTypeMember.check(self, schema, cases)
> assert self.name in tag_type.values
> + if union:
> + # seen is QMP names our members must not collide with
> + self.type.check_qmp(schema, seen)
>
> # This function exists to support ugly simple union special cases
> # TODO get rid of them, and drop the function
> @@ -1090,7 +1101,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
> self.variants = variants
>
> def check(self, schema):
> - self.variants.check(schema, [], {})
> + self.variants.check(schema, {}, False)
>
> def json_type(self):
> return 'value'
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions
2015-11-02 15:37 ` Markus Armbruster
@ 2015-11-02 17:20 ` Eric Blake
2015-11-02 21:24 ` Eric Blake
` (5 subsequent siblings)
6 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-02 17:20 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 13747 bytes --]
On 11/02/2015 08:37 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Now that we have separate namespaces for QMP vs. tag values,
>
> What's the "QMP namespace"?
I guess I need to be more explicit :)
In the generated C struct for a qapi object, we now have two separate
namespaces:
struct Foo {
Type1 name1; /* namespace for QMP (aka non-variant) members */
Type2 name2;
union {
Type1 name1; /* namespace for tag values */
Type2 name2;
} u;
};
>
>> we can simplify how the QAPISchema*.check() methods check for
>> collisions.
>
> I *guess* the point of this patch is dropping checks that are obsolete
> now tag values no longer collide with non-variant member names in
> generated C, with simplifications on top. Correct?
>
Almost. This is not actually dropping any of the old ad hoc checks in
the parser, but you are correct that it IS fixing the check() methods to
quit asserting things that are no longer forbidden now that tag values
no longer collide with non-variant member names.
>> Each QAPISchemaObjectTypeMember check() call is
>> given a single set of names it must not collide with; this set
>> is either the QMP names (when this member is used by an
>> ObjectType) or the case names (when this member is used by an
>> ObjectTypeVariants). We no longer need an all_members
>> parameter, as it can be computed by seen.values().
This is point [1] mentioned below.
>> When used
>> by a union, QAPISchemaObjectTypeVariant must also perform a
>> QMP collision check for each member of its corresponding type.
>
> I'm afraid this explanation is next to impossible to understand unless
> you know how the checking works. I should know, because I wrote it, but
> actually don't, at least not by heart. So let me relearn how checking
> works before your patch.
I guess I get to pick and choose from your wording, to try and make my
commit body more comprehensible.
>
> We're talking about a single invocation of QAPISchemaObjectType.check().
> Its job is to compute self.members and self.base, and do sanity
> checking, which includes checking for collisions. It has two variables
> of interest:
>
> * members is where we accumulate the list of members that'll become
> self.members. It's initialized to the inherited members (empty if no
> base, obviously).
True pre-patch; eliminated by point [1] mentioned above, by observing
that seen.values() is identical to members after processing
local_members, and that we don't further modify seen when processing
variants.
>
> * seen is a dictionary mapping names to members. This is merely for
> collision checking. It's initialized to contain the inherited members
> (whose names must be distinct, by induction, because we make sure the
> base type's check() completes first).
>
> For each local member m of self, we call m.check(schema, members, seen).
> This is actually QAPISchemaObjectTypeMember.check(), and it uses members
> and seen as follows:
>
> * Assert m.name doesn't collide with another member's name, i.e. not in
> seen.
>
> * Append m to members.
>
> * Insert m.name: m into seen.
>
> Straightforward so far. Coming up next: variants.
>
> Each variant's members are represented as a single member with the tag
> value as name. Its type is an object type that has the variant's
> members.
>
> For each variant v:
>
> * Copy seen to vseen. It holds the non-variant members.
>
> * Call QAPISchemaObjectTypeMember.check(schema, [], vseen), which boils
> down to assert v.name doesn't collide with a non-variant member's
> name. This check is obsolete.
>
> * Assert v.name is a member of tag_type.
>
> Not checked: variant's name doesn't collide with another variant's name.
> That's because we copy seen inside the loop instead of before the loop.
>
> Not checked: variant's members don't collide with non-variant members.
> I think this check got lost when we simplified
> QAPISchemaObjectTypeVariants to hold a single member.
Perhaps so; our testsuite wasn't as complete at that time. Or maybe
it's because our ad hoc checks in the parsing portion were preventing us
from realizing that we needed to repeat things in check() methods.
>
> Note that the real checking happens in check_union(), and the checks
> here are just sanity checks. As long as that's the case, holes here are
> harmless. However, we need them plugged them when we move the real
> checking from check_union() to the .check().
Yep, and that's what this patch is trying to do.
>
> Looks like variant checking should be thrown out and redone.
>
> I still don't get your description. Guess I have to read the patch
> after all ;)
>
>> The new ObjectType.check_qmp() is an idempotent subset of
>> check(), and can be called multiple times over different seen
>> sets (useful, since the members of one type can be applied
>> into more than one other location via inheritance or flat
>> union variants).
>
> I think I get this argument. Except I don't get why the method's named
> check_qmp().
Check that the QMP field names introduced by this type into an overall
object do not cause any collisions with QMP field names already present
from other sources. I'm open to suggestions for a better name.
>
>> The code needs a temporary hack of passing a 'union' flag
>> through Variants.check(), since we do not inline the branches
>> of an alternate type into a parent QMP object.
>
> What a "QMP object"? Sounds like a JSON object on the QMP wire, but
> that makes no sense :)
Let's try again:
The code needs a temporary hack of passing a 'union' flag through
Variants.check(), in order to decide whether to check the various fields
of the branch object type against the earlier QMP names in seen. The
check is conditional on being a union, because the QMP field names of
each union branch share the same QMP namespace as the non-variant fields
in the overall JSON object (even though they are in a different C
namespace due to boxing behind the branch type), while an alternate does
not inline any QMP fields (because it is not part of a larger JSON
object). Note that we do not need to distinguish between simple and
flat unions, because simple unions have a generated wrapper type whose
lone QMP field ('data') will be checked against the lone 'type'
non-variant field.
>
>> A later patch
>> will rework how alternates are laid out, by adding a new
>> subclass, and that will allow us to drop the extra parameter.
>>
>> There are no changes to generated code.
>>
>> Future patches will enhance testsuite coverage, improve error
>> message quality on actual collisions, and move collision
>> checks out of ad hoc parse code into the check() methods.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v8: rebase to earlier patches, defer positive test additions to
>> later in series
>> v7: new patch, although it is a much cleaner implementation of
>> what was attempted by subset B v8 15/18
>> https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03042.html
>> ---
>> scripts/qapi.py | 55 +++++++++++++++++++++++++++++++++----------------------
>> 1 file changed, 33 insertions(+), 22 deletions(-)
Okay, let's see what you think of the actual patch, or if you have any
better suggestions for how I should have worded the commit body.
>>
>> diff --git a/scripts/qapi.py b/scripts/qapi.py
>> index 84ac151..c571709 100644
>> --- a/scripts/qapi.py
>> +++ b/scripts/qapi.py
>> @@ -972,28 +972,28 @@ class QAPISchemaObjectType(QAPISchemaType):
>> self.variants = variants
>> self.members = None
>>
>> + # Finish construction, and validate that all members are usable
>> def check(self, schema):
>> assert self.members is not False # not running in cycles
>> if self.members:
>> return
>> self.members = False # mark as being checked
>> + seen = OrderedDict()
>
> Why do you need to switch from {} to OrderedDict()?
Because I'm dropping the all_members argument, and I need seen.values()
to maintain the same order in which members are seen, for it to be a
replacement for all_members.
Maybe I need to split this patch into a few more pieces, each tackling a
smaller cleanup, rather than trying to do it all in one?
>
>> if self._base_name:
>> self.base = schema.lookup_type(self._base_name)
>> - assert isinstance(self.base, QAPISchemaObjectType)
>> - assert not self.base.variants # not implemented
>> - self.base.check(schema)
[2] The old code calls base.check()...
>> - members = list(self.base.members)
x>> - else:
>> - members = []
>> - seen = {}
>> - for m in members:
>> - assert c_name(m.name) not in seen
>> - seen[m.name] = m
...then adds each of base's members into seen.
>> + self.base.check_qmp(schema, seen)
The new code combines all of those actions under the helper
base.check_qmp().
>> for m in self.local_members:
>> - m.check(schema, members, seen)
>> + m.check(schema, seen)
>> if self.variants:
>> - self.variants.check(schema, members, seen)
>> - self.members = members
>> + self.variants.check(schema, seen)
>> + self.members = seen.values()
Nothing else in child calls check_qmp().
>> +
>> + # Check that this type does not introduce QMP collisions into seen
>> + def check_qmp(self, schema, seen):
>> + self.check(schema)
>> + assert not self.variants # not implemented
>> + for m in self.members:
>> + m.check(schema, seen)
>
> You said "ObjectType.check_qmp() is an idempotent subset of check()",
> but it's obviously a superset: it calls check(), then does some more
> work.
See [2] above. I need to reword that then, or split it into a separate
patch for easier reasoning. base.check_qmp() is a subset of the work
done by child.check(). You are correct that it is NOT a subset of the
work done by base.check() (because it calls base.check() and then does
additional work).
>
> I think I'm now too confused to make further progress on this patch,
> probably because I've thought myself into a corner.
>
> Before I give up, a remark on design. The QAPISchemaFOO classes all
> implement the same protocol:
>
> * __init__() type-checks arguments and initializes instance variables,
> generally either to an argument or to None. This is basically AST
> construction.
>
> * check() computes the remaining instance variables. This is semantic
> analysis, except it's stubbed out.
>
> QAPISchema.__init__() calls its own check(), which calls all the
> others, directly (for entities), or indirect (for members and such).
> Each check() runs *once*.
And I'm intentionally reworking things to run a _subset_ of check()
multiple times - namely, calling member.check(seen) to see if member
collides with any names already present in seen, where seen can be
either the set of inherited non-variant fields (for both local members
and for the fields of a variant type from a qapi union), or where seen
can be the set of tag names (for the use of QAPISchemaObjectTypeVariant,
which is a subclass of QAPISchemaObjectTypeMember).
>
> Except for QAPISchemaType.check(). Why? Unlike other entities, the
> types need to be checked in a certain order: contained types (base and
> variant) before the containing type. For simplicity, we simply call
> their check(), and detect cycles. This is basically a topological
> sort and the real checking squashed together. The real checking still
> runs once.
>
> It follows that you must not call check() except:
>
> - An entity is responsible for calling check() for the non-entities it
> owns (e.g. an object type's check() calls its members' check()).
>
> - A type may call check() for a type it contains (e.g. its base type).
> That's safe, because no type may contain itself. This kind of call
> drives the top-sort.
>
> * Obviously, the instance variables computed by check() have valid
> values only after check(). Likewise, certain methods should be called
> only after check(), e.g. visit().
>
> Of course, if we find a more suitable protocol, we're free to adopt it.
>
>>
>> def is_implicit(self):
>> # See QAPISchema._make_implicit_object_type()
>> @@ -1027,11 +1027,13 @@ class QAPISchemaObjectTypeMember(object):
>> self.type = None
>> self.optional = optional
>>
>> - def check(self, schema, all_members, seen):
>> + def check(self, schema, seen):
>> + # seen is a map of names we must not collide with (either QMP
>> + # names, when called by ObjectType, or case names, when called
>> + # by Variant). This method is safe to call over multiple 'seen'.
>
> What are "QMP names"? Member names, perhaps?
Yes, the member names that result from JSON object keys at the same
level of nesting, whether or not those names are attributed to the same
C namespace.
I still like my patch, but I can see it caused some confusion. So I'll
try to break it into smaller pieces for v9.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions
2015-11-02 15:37 ` Markus Armbruster
2015-11-02 17:20 ` Eric Blake
@ 2015-11-02 21:24 ` Eric Blake
2015-11-03 7:56 ` Markus Armbruster
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 0/4] rework of 7/17 Eric Blake
` (4 subsequent siblings)
6 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-11-02 21:24 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1333 bytes --]
On 11/02/2015 08:37 AM, Markus Armbruster wrote:
>
> Not checked: variant's members don't collide with non-variant members.
> I think this check got lost when we simplified
> QAPISchemaObjectTypeVariants to hold a single member.
Yep, I found the culprit: in your v2 proposal for QAPISchema, you had:
+class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
+ def __init__(self, name, typ, flat):
+ QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
+ assert isinstance(flat, bool)
+ self.flat = flat
+ def check(self, schema, tag_type, seen):
+ QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ assert self.name in tag_type.values
+ if self.flat:
+ self.type.check(schema)
+ assert isinstance(self.type, QAPISchemaObjectType)
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg00394.html
but the 'if self.flat' clause was lost in v3:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00450.html
I am in fact reinstating it here, but for v9, will do it in a separate
patch rather than blended in with the rest of the changes.
[wow - we've been hammering at this since July?]
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8.5 0/4] rework of 7/17
2015-11-02 15:37 ` Markus Armbruster
2015-11-02 17:20 ` Eric Blake
2015-11-02 21:24 ` Eric Blake
@ 2015-11-02 22:41 ` Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check() Eric Blake
` (3 subsequent siblings)
6 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-02 22:41 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru
Based on the confusion I caused Markus, I've tried to split
the v8 patch 7/17 into smaller pieces that are hopefully
easier to review in isolation.
Eric Blake (4):
qapi: Drop all_members parameter from check()
qapi: Check for QMP collisions of flat union branches
qapi: Fix check for variant tag values collision
qapi: Consolidate collision detection code
scripts/qapi.py | 62 ++++++++++++++++++++++++++++++++++++---------------------
1 file changed, 39 insertions(+), 23 deletions(-)
--
2.4.3
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-02 15:37 ` Markus Armbruster
` (2 preceding siblings ...)
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 0/4] rework of 7/17 Eric Blake
@ 2015-11-02 22:41 ` Eric Blake
2015-11-03 11:06 ` Markus Armbruster
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 2/4] qapi: Check for QMP collisions of flat union branches Eric Blake
` (2 subsequent siblings)
6 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-11-02 22:41 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
The implementation of QAPISchemaObjectTypeMember.check() always
adds the member currently being checked to both the all_members
and seen parameters. However, the three callers of this method
pass in the following parameters:
QAPISchemaObjectType.check():
- all_members contains all non-variant members seen to date,
for use in populating self.members
- seen contains all non-variant members seen to date, for
use in checking for collisions
QAPISchemaObjectTypeVariant.check():
- all_members is a throwaway empty list
- seen is a throwaway dictionary created as a copy by
QAPISchemaObjectVariants.check() (since the members of
one variant cannot collide with those from another), for
use in checking for collisions (technically, we no longer
need to check for collisions between tag values and QMP
key names, but that's a cleanup for another patch)
QAPISchemaAlternateType.check():
- all_members is a throwaway empty list
- seen is a throwaway empty dict
Therefore, in the one case where we care about all_members
after seen has been populated, we know that it contains the
same members as seen.values(); changing seen to be an
OrderedDict() is sufficient to pick up this information with
one less parameter being passed around.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v9: new patch, split off from v8 7/17
---
scripts/qapi.py | 29 +++++++++++++----------------
1 file changed, 13 insertions(+), 16 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6b6ba5f..fff4adb 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -972,28 +972,26 @@ class QAPISchemaObjectType(QAPISchemaType):
self.variants = variants
self.members = None
+ # Finish construction, and validate that all members are usable
def check(self, schema):
assert self.members is not False # not running in cycles
if self.members:
return
self.members = False # mark as being checked
+ seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
assert isinstance(self.base, QAPISchemaObjectType)
assert not self.base.variants # not implemented
self.base.check(schema)
- members = list(self.base.members)
- else:
- members = []
- seen = {}
- for m in members:
- assert c_name(m.name) not in seen
- seen[m.name] = m
+ for m in self.base.members:
+ assert c_name(m.name) not in seen
+ seen[m.name] = m
for m in self.local_members:
- m.check(schema, members, seen)
+ m.check(schema, seen)
if self.variants:
- self.variants.check(schema, members, seen)
- self.members = members
+ self.variants.check(schema, seen)
+ self.members = seen.values()
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
@@ -1027,11 +1025,10 @@ class QAPISchemaObjectTypeMember(object):
self.type = None
self.optional = optional
- def check(self, schema, all_members, seen):
+ def check(self, schema, seen):
assert self.name not in seen
self.type = schema.lookup_type(self._type_name)
assert self.type
- all_members.append(self)
seen[self.name] = self
@@ -1050,7 +1047,7 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
- def check(self, schema, members, seen):
+ def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
assert self.tag_member
@@ -1067,7 +1064,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
def check(self, schema, tag_type, seen):
- QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ QAPISchemaObjectTypeMember.check(self, schema, seen)
assert self.name in tag_type.values
# This function exists to support ugly simple union special cases
@@ -1090,8 +1087,8 @@ class QAPISchemaAlternateType(QAPISchemaType):
def check(self, schema):
seen = {}
- self.variants.tag_member.check(schema, [], seen)
- self.variants.check(schema, [], seen)
+ self.variants.tag_member.check(schema, seen)
+ self.variants.check(schema, seen)
def json_type(self):
return 'value'
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8.5 2/4] qapi: Check for QMP collisions of flat union branches
2015-11-02 15:37 ` Markus Armbruster
` (3 preceding siblings ...)
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check() Eric Blake
@ 2015-11-02 22:41 ` Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 3/4] qapi: Fix check for variant tag values collision Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 4/4] qapi: Consolidate collision detection code Eric Blake
6 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-02 22:41 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any QMP member names that would
conflict with the non-variant QMP members already present
from the union's base type (see flat-union-clash-member.json).
However, we want QAPISchema*.check() to make the same check,
so we can later reduce some of the ad hoc checks.
Basically, all branches of a flat union must be qapi structs
with no variants, at which point those members appear in the
same JSON object as all non-variant members. We already have
a map 'seen' of all non-variant members passed in to
QAPISchemaObjectTypeVariants.check(), which we clone for each
particular variant (since the members of one variant do not
clash with another); all that is additionally needed is to
actually check the each member of the variant type do not add
any collisions.
For simple unions, the same code happens to work (however, our
synthesized wrapper type has a single member 'data' which will
never collide with the one non-variant member 'type').
But for alternates, we do NOT want to check the type members
for adding collisions (an alternate has no parent JSON object
that is merging in member names, the way a flat union does), so
we have to pass in an extra flag to distinguish whether we are
working with a union or an alternate. The flag is temporary; a
later patch will rework how alternates are laid out by creating
a new subclass of QAPISchemaObjectTypeMember, and detecting the
use of this subclass for variants.tag_member will serve the
same purpose.
Note that an early proposal [1] for what eventually became
commit ac88219a had QAPISchemaObjectTypeVariant.check() ensure
that the variant type was complete, although it was later
removed [2]; the checks added here happen to match what that
earlier attempt meant to do.
[1] https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg00394.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00450.html
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v9: new patch, split off from v8 7/17
---
scripts/qapi.py | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index fff4adb..3cf051f 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1047,7 +1047,8 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
- def check(self, schema, seen):
+ # TODO drop 'union' param once tag_member is sufficient to spot alternates
+ def check(self, schema, seen, union=True):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
assert self.tag_member
@@ -1055,17 +1056,29 @@ class QAPISchemaObjectTypeVariants(object):
assert self.tag_member in seen.itervalues()
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
- vseen = dict(seen)
- v.check(schema, self.tag_member.type, vseen)
+ # Reset seen array for each variant, since QMP names from one
+ # branch do not affect another branch
+ v.check(schema, self.tag_member.type, dict(seen), union)
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
- def check(self, schema, tag_type, seen):
- QAPISchemaObjectTypeMember.check(self, schema, seen)
+ # TODO drop 'union' param once tag_type is sufficient to spot alternates
+ def check(self, schema, tag_type, seen, union):
+ QAPISchemaObjectTypeMember.check(self, schema, dict(seen))
assert self.name in tag_type.values
+ if union:
+ # If this variant is used within a union, then each member
+ # field must avoid collisions with the non-variant members
+ # already present in the union.
+ assert isinstance(self.type, QAPISchemaObjectType)
+ assert not self.type.variants # not implemented
+ self.type.check(schema)
+ for m in self.type.members:
+ assert c_name(m.name) not in seen
+ seen[m.name] = m
# This function exists to support ugly simple union special cases
# TODO get rid of them, and drop the function
@@ -1088,7 +1101,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
def check(self, schema):
seen = {}
self.variants.tag_member.check(schema, seen)
- self.variants.check(schema, seen)
+ self.variants.check(schema, seen, False)
def json_type(self):
return 'value'
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8.5 3/4] qapi: Fix check for variant tag values collision
2015-11-02 15:37 ` Markus Armbruster
` (4 preceding siblings ...)
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 2/4] qapi: Check for QMP collisions of flat union branches Eric Blake
@ 2015-11-02 22:41 ` Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 4/4] qapi: Consolidate collision detection code Eric Blake
6 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-02 22:41 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Now that commit e4ba22b3 has separated the C representation of
qapi unions so that tag values no longer collide with non-variant
members, we must adjust QAPISchemaObjectTypeVariant.check() to
match. The fix is conceptually simple - track a separate
dictionary of tag names we have seen so far, different from the
dictionary of non-variant names. And while the non-variant seen
array gets reset for each new variant (because the JSON object
does not have collisions between separate branches), the map of
tag names is not reset.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v9: new patch, split off from v8 7/17
---
scripts/qapi.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 3cf051f..10bf16f 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1055,10 +1055,11 @@ class QAPISchemaObjectTypeVariants(object):
else: # simple union or alternate
assert self.tag_member in seen.itervalues()
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
+ cases = {}
for v in self.variants:
# Reset seen array for each variant, since QMP names from one
# branch do not affect another branch
- v.check(schema, self.tag_member.type, dict(seen), union)
+ v.check(schema, self.tag_member.type, dict(seen), cases, union)
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
@@ -1066,8 +1067,8 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
# TODO drop 'union' param once tag_type is sufficient to spot alternates
- def check(self, schema, tag_type, seen, union):
- QAPISchemaObjectTypeMember.check(self, schema, dict(seen))
+ def check(self, schema, tag_type, seen, cases, union):
+ QAPISchemaObjectTypeMember.check(self, schema, cases)
assert self.name in tag_type.values
if union:
# If this variant is used within a union, then each member
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH v8.5 4/4] qapi: Consolidate collision detection code
2015-11-02 15:37 ` Markus Armbruster
` (5 preceding siblings ...)
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 3/4] qapi: Fix check for variant tag values collision Eric Blake
@ 2015-11-02 22:41 ` Eric Blake
6 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-02 22:41 UTC (permalink / raw)
To: qemu-devel; +Cc: armbru, Michael Roth
Rather than having three separate places populate the seen map,
it is easier to just factor out a subset of Member.check()
that does this as a new method check_collision(), and have the
remaining places in ObjectType.check() and Variant.check()
call into it. This likewise means a new helper method
ObjectType.check_collision(). Later patches can then change
the handling of the seen array in just one place, when moving
away from ad hoc parser tests.
Note that there was some discrepancy in the existing code on
whether name or c_name(name) could not previously be in the
seen map; a future patch will clean this up to consistently
populate the map via c_name().
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v9: new patch, split off from v8 7/17; name change from
ObjectType.check_qmp() to check_collision(), and new method
Member.check_collision(). I'm open to naming suggestions.
---
scripts/qapi.py | 31 ++++++++++++++++++-------------
1 file changed, 18 insertions(+), 13 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 10bf16f..b519d30 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -981,18 +981,21 @@ class QAPISchemaObjectType(QAPISchemaType):
seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
- assert isinstance(self.base, QAPISchemaObjectType)
- assert not self.base.variants # not implemented
- self.base.check(schema)
- for m in self.base.members:
- assert c_name(m.name) not in seen
- seen[m.name] = m
+ self.base.check_collision(schema, seen)
for m in self.local_members:
m.check(schema, seen)
if self.variants:
self.variants.check(schema, seen)
self.members = seen.values()
+ # Check that the members of this type do not cause duplicate JSON fields,
+ # and update seen to track the members seen so far
+ def check_collision(self, schema, seen):
+ assert not self.variants # not implemented
+ self.check(schema)
+ for m in self.members:
+ m.check_collision(seen)
+
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
return self.name[0] == ':'
@@ -1026,9 +1029,16 @@ class QAPISchemaObjectTypeMember(object):
self.optional = optional
def check(self, schema, seen):
- assert self.name not in seen
self.type = schema.lookup_type(self._type_name)
+ self.check_collision(seen)
+
+ # Check that this member does not collide with anything in seen (the
+ # set of non-variant members when called from QAPISchemaObjectType,
+ # or the set of tag values when called from QAPISchemaObjectTypeVariant),
+ # and update seen accordingly.
+ def check_collision(self, seen):
assert self.type
+ assert self.name not in seen
seen[self.name] = self
@@ -1074,12 +1084,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
# If this variant is used within a union, then each member
# field must avoid collisions with the non-variant members
# already present in the union.
- assert isinstance(self.type, QAPISchemaObjectType)
- assert not self.type.variants # not implemented
- self.type.check(schema)
- for m in self.type.members:
- assert c_name(m.name) not in seen
- seen[m.name] = m
+ self.type.check_collision(schema, seen)
# This function exists to support ugly simple union special cases
# TODO get rid of them, and drop the function
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions
2015-11-02 21:24 ` Eric Blake
@ 2015-11-03 7:56 ` Markus Armbruster
2015-11-03 13:30 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 7:56 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/02/2015 08:37 AM, Markus Armbruster wrote:
>
>>
>> Not checked: variant's members don't collide with non-variant members.
>> I think this check got lost when we simplified
>> QAPISchemaObjectTypeVariants to hold a single member.
>
> Yep, I found the culprit: in your v2 proposal for QAPISchema, you had:
>
> +class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> + def __init__(self, name, typ, flat):
> + QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
> + assert isinstance(flat, bool)
> + self.flat = flat
> + def check(self, schema, tag_type, seen):
> + QAPISchemaObjectTypeMember.check(self, schema, [], seen)
> + assert self.name in tag_type.values
> + if self.flat:
> + self.type.check(schema)
> + assert isinstance(self.type, QAPISchemaObjectType)
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg00394.html
>
> but the 'if self.flat' clause was lost in v3:
>
> https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00450.html
Quoting v3's change log:
- Lower simple variants to flat ones as described on
qapi-code-gen.txt. Replace QAPISchemaObjectType's .flat by
.simple_union_type(), for use by pre-existing code-generation
warts.
> I am in fact reinstating it here, but for v9, will do it in a separate
> patch rather than blended in with the rest of the changes.
Any "is this union flat or simple" check signals a flaw. It's either a
pointless difference in generated code (these should all be marked TODO
by now), or something's wrong with the desugaring of simple to flat
unions.
In this case, it looks like a collision check was lost when I switched
from "simple unions are its own separate type" to "simple unions are
sugar for flat unions".
Reinstating it makes sense, except for the "if self.flat" part. If the
check's correct for flat unions, it must be correct for desugared simple
unions, or else we screwed up the desugaring.
For a genuine flat union case
'tag-value': 'FlatVariant'
self.type is the object type 'FlatVariant'. Its members are the case's
variant members, and they can clash with the flat union's non-variant
members.
For a desugared simple union case
'tag-value': 'SimpleVariant'
self.type is an object type with a single member 'data' of type
'SimpleVariant'. Its member 'data' is the case's only variant member,
and it can clash with the simple union's non-variant members. In theory
only, because the only non-variant member is the implicit tag, and it's
called 'type'.
Therefore, the if self.flat is superfluous. Good, because otherwise our
desugaring must be flawed.
> [wow - we've been hammering at this since July?]
First RFC was in April, but we started hammering in earnest only in
summer.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check() Eric Blake
@ 2015-11-03 11:06 ` Markus Armbruster
2015-11-03 13:26 ` Eric Blake
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
0 siblings, 2 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 11:06 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> The implementation of QAPISchemaObjectTypeMember.check() always
> adds the member currently being checked to both the all_members
> and seen parameters.
QAPISchemaObjectTypeMember.check() does four things:
1. Compute self.type
Precondition: all types are defined.
2. Accumulate members
all_members serves as accumulator.
We'll see that its only actual use is the owning object type's
check(), which uses it to compute self.members.
3. Check for collisions
This works by accumulating names in seen. Precondition: seen
contains the names seen so far.
Note that this part uses seen like a set. See 4.
4. Accumulate a map from names to members
seen serves as accumulator.
We'll see that its only actual user is the owning object type's
variants.check(), which uses it to compute variants.tag_member from
variants.tag_name.
> However, the three callers of this method
> pass in the following parameters:
>
> QAPISchemaObjectType.check():
> - all_members contains all non-variant members seen to date,
> for use in populating self.members
> - seen contains all non-variant members seen to date, for
> use in checking for collisions
Yes, and:
- we're calling it for m in self.local_members
- before the loop, all_members and seen are initialized to the inherited
non-variant members
- after the loop, they therefore contain all non-variant members
This caller uses all four things done by QAPISchemaObjectType.check():
1. Compute m.type
2. Accumulate non-variant members
3. Check for collisions among non-variant members
Before the loop, seen contains the inherited members, which don't
collide (self.base.check() ensures that). The loop adds the local
members one by one, checking for collisions.
4. Accumulate a map from names to non-variant members
Similar argument to 3.
> QAPISchemaObjectTypeVariant.check():
Do you mean QAPISchemaObjectVariants.check()?
> - all_members is a throwaway empty list
> - seen is a throwaway dictionary created as a copy by
> QAPISchemaObjectVariants.check() (since the members of
> one variant cannot collide with those from another), for
> use in checking for collisions (technically, we no longer
> need to check for collisions between tag values and QMP
> key names, but that's a cleanup for another patch)
>
> QAPISchemaAlternateType.check():
> - all_members is a throwaway empty list
> - seen is a throwaway empty dict
I'm afraid you're omitting a few steps here, and I think you missed
QAPISchemaObjectVariants.check()'s self.tag_member.check(). Let me go
through the remaining callers of QAPISchemaObjectTypeMember.check() real
slow.
* QAPISchemaObjectType.check() calls it via v.check(), which is a thin
wrapper that throws away 2. and asserts something of no interest here.
seen is a map from names to non-variant members. Therefore,
QAPISchemaObjectTypeMember.check() gets used here as follows:
1. Compute v.type
2. Thrown away
3. Check for collision of tag value with non-variant members
This is obsolete now.
4. Thrown away
* QAPISchemaAlternateType.check() calls it via v.check(), which is the
same thin wrapper. seen is {} here. Therefore:
1. Compute v.type
2. Thrown away
3. No-op
4. Thrown away
* QAPISchemaObjectTypeVariants.check() calls
self.tag_member.check(schema, members, seen)
where members and seen contain the non-variant members.
However, it gets called only when the owning type is an alternate, and
then members and seen are both empty. Therefore:
1. Compute v.type
2. Thrown away
3. No-op
4. Thrown away
> Therefore, in the one case where we care about all_members
> after seen has been populated, we know that it contains the
> same members as seen.values(); changing seen to be an
> OrderedDict() is sufficient to pick up this information with
> one less parameter being passed around.
I believe the first step should be dropping the obsolete check for
collision of tag value with non-variant members. I believe this should
do:
@@ -1059,8 +1059,7 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member.check(schema, members, seen)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
- vseen = dict(seen)
- v.check(schema, self.tag_member.type, vseen)
+ v.check(schema, self.tag_member.type, {})
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
Then only one caller about 2-4., namely QAPISchemaObjectType.check().
Simplify radically: move 2-4. to the caller that cares, drop parameters
all_members and seen.
Still to do then: non-variant member collision checking. Factor out
3. into a helper function, use it for non-variant members.
What do you think?
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-03 11:06 ` Markus Armbruster
@ 2015-11-03 13:26 ` Eric Blake
2015-11-03 14:02 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
1 sibling, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-11-03 13:26 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 6835 bytes --]
On 11/03/2015 04:06 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> The implementation of QAPISchemaObjectTypeMember.check() always
>> adds the member currently being checked to both the all_members
>> and seen parameters.
>
> QAPISchemaObjectTypeMember.check() does four things:
>
> 1. Compute self.type
>
> Precondition: all types are defined.
Correct, unchanged by this patch.
>
> 2. Accumulate members
>
> all_members serves as accumulator.
>
> We'll see that its only actual use is the owning object type's
> check(), which uses it to compute self.members.
This patch changes it to use seen.values(), which (once you use an
OrderedDict() instead of plain {}) is identical to all_members.
>
> 3. Check for collisions
>
> This works by accumulating names in seen. Precondition: seen
> contains the names seen so far.
>
> Note that this part uses seen like a set. See 4.
Unchanged by this patch; but see also 2/4 and 3/4.
>
> 4. Accumulate a map from names to members
>
> seen serves as accumulator.
>
Unchanged by this patch.
> We'll see that its only actual user is the owning object type's
> variants.check(), which uses it to compute variants.tag_member from
> variants.tag_name.
>
>> However, the three callers of this method
>> pass in the following parameters:
>>
>> QAPISchemaObjectType.check():
>> - all_members contains all non-variant members seen to date,
>> for use in populating self.members
>> - seen contains all non-variant members seen to date, for
>> use in checking for collisions
>
> Yes, and:
>
> - we're calling it for m in self.local_members
> - before the loop, all_members and seen are initialized to the inherited
> non-variant members
> - after the loop, they therefore contain all non-variant members
>
> This caller uses all four things done by QAPISchemaObjectType.check():
>
> 1. Compute m.type
Unchanged by this patch.
> 2. Accumulate non-variant members
Whether the accumulation is done via all_members (pre-patch) or by
seen.values() (post-patch), this step is still done.
> 3. Check for collisions among non-variant members
> Before the loop, seen contains the inherited members, which don't
> collide (self.base.check() ensures that). The loop adds the local
> members one by one, checking for collisions.
Unchanged by this patch.
> 4. Accumulate a map from names to non-variant members
> Similar argument to 3.
Unchanged by this patch.
>
>> QAPISchemaObjectTypeVariant.check():
>
> Do you mean QAPISchemaObjectVariants.check()?
QAPISchemaObjectTypeVariants.check() calls
QAPISchemaObjectTypeVariant.check() for each variant, but with a fresh
copy of seen. We'll later need to expand this copy of seen (patch 2/4),
but for this patch its use is unchanged - we are appending a single
value (the tag value) which is wrong, but no one cares that we appended
it because it was a copy. Patch 3/4 fixes to not append to it.
>
>> - all_members is a throwaway empty list
>> - seen is a throwaway dictionary created as a copy by
>> QAPISchemaObjectVariants.check() (since the members of
>> one variant cannot collide with those from another), for
>> use in checking for collisions (technically, we no longer
>> need to check for collisions between tag values and QMP
>> key names, but that's a cleanup for another patch)
>>
>> QAPISchemaAlternateType.check():
>> - all_members is a throwaway empty list
>> - seen is a throwaway empty dict
>
> I'm afraid you're omitting a few steps here, and I think you missed
> QAPISchemaObjectVariants.check()'s self.tag_member.check().
There is no self.tag_member.check(), any more; rather, my earlier
patches have reworked things so that tag_member is checked by the owner
(a flat union's base type, a simple union's local_members, or directly
in QAPISchemaAlternateType prior to calling Variants.check()). I guess
that's a pitfall of seeing this patch without my rework of 5/17 to
address your comments there.
>> Therefore, in the one case where we care about all_members
>> after seen has been populated, we know that it contains the
>> same members as seen.values(); changing seen to be an
>> OrderedDict() is sufficient to pick up this information with
>> one less parameter being passed around.
>
> I believe the first step should be dropping the obsolete check for
> collision of tag value with non-variant members. I believe this should
> do:
>
> @@ -1059,8 +1059,7 @@ class QAPISchemaObjectTypeVariants(object):
> self.tag_member.check(schema, members, seen)
> assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> for v in self.variants:
> - vseen = dict(seen)
> - v.check(schema, self.tag_member.type, vseen)
> + v.check(schema, self.tag_member.type, {})
Close, but not quite. It should do:
+ cases = {}
for v in self.variants:
vseen = dict(seen)
- v.check(schema, self.tag_member.type, vseen)
+ v.check(schema, self.tag_member.type, vseen, cases)
coupled with this in QAPISchemaObjectTypeVariant:
- def check(self, schema, tag_type, seen):
- QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ def check(self, schema, tag_type, seen, cases):
+ QAPISchemaObjectTypeMember.check(self, schema, [], cases)
so that we are now checking collisions between tag values, rather than
cases. But that's what I did in patch 3/4. And we still need seen
passed to Variant.check(), because that's the checking added in 2/4.
Okay, you've convinced me - when I post v9, I'll reorder these four
patches to put 3/4 first.
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
>
> Then only one caller about 2-4., namely QAPISchemaObjectType.check().
> Simplify radically: move 2-4. to the caller that cares, drop parameters
> all_members and seen.
Nope - because seen (well, a copy of seen) is still important to patch 2/4.
>
> Still to do then: non-variant member collision checking. Factor out
> 3. into a helper function, use it for non-variant members.
Factoring into a helper function is done in 4/4. I can try and
rearrange that earlier, too.
>
> What do you think?
>
Can you at least look at 2, 3, and 4 to see where I'm headed, and then I
can rearrange things for the v9 spin? We're probably talking a bit past
each other, with the same end goal, but a muddle in the middle of how to
get there.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions
2015-11-03 7:56 ` Markus Armbruster
@ 2015-11-03 13:30 ` Eric Blake
0 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-03 13:30 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 2250 bytes --]
On 11/03/2015 12:56 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> On 11/02/2015 08:37 AM, Markus Armbruster wrote:
>>
>>>
>>> Not checked: variant's members don't collide with non-variant members.
>>> I think this check got lost when we simplified
>>> QAPISchemaObjectTypeVariants to hold a single member.
>>
>> Yep, I found the culprit: in your v2 proposal for QAPISchema, you had:
>>
>> +class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
>> + def __init__(self, name, typ, flat):
>> + QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
>> + assert isinstance(flat, bool)
>> + self.flat = flat
>> + def check(self, schema, tag_type, seen):
>> + QAPISchemaObjectTypeMember.check(self, schema, [], seen)
>> + assert self.name in tag_type.values
>> + if self.flat:
>> + self.type.check(schema)
>> + assert isinstance(self.type, QAPISchemaObjectType)
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg00394.html
>>
>> but the 'if self.flat' clause was lost in v3:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00450.html
>
> Quoting v3's change log:
>
> - Lower simple variants to flat ones as described on
> qapi-code-gen.txt. Replace QAPISchemaObjectType's .flat by
> .simple_union_type(), for use by pre-existing code-generation
> warts.
>
>> I am in fact reinstating it here, but for v9, will do it in a separate
>> patch rather than blended in with the rest of the changes.
>
> Any "is this union flat or simple" check signals a flaw. It's either a
> pointless difference in generated code (these should all be marked TODO
> by now), or something's wrong with the desugaring of simple to flat
> unions.
Losing 'if self.flat' was correct, but we still need 'if union'; and
that's what I add in 2/4.
>
> Therefore, the if self.flat is superfluous. Good, because otherwise our
> desugaring must be flawed.
And things correctly work on simple unions due to our wrapper type, so
that 'if union' was sufficient.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-03 13:26 ` Eric Blake
@ 2015-11-03 14:02 ` Markus Armbruster
2015-11-03 14:12 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:02 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/03/2015 04:06 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> The implementation of QAPISchemaObjectTypeMember.check() always
>>> adds the member currently being checked to both the all_members
>>> and seen parameters.
>>
>> QAPISchemaObjectTypeMember.check() does four things:
>>
>> 1. Compute self.type
>>
>> Precondition: all types are defined.
>
> Correct, unchanged by this patch.
>
>>
>> 2. Accumulate members
>>
>> all_members serves as accumulator.
>>
>> We'll see that its only actual use is the owning object type's
>> check(), which uses it to compute self.members.
>
> This patch changes it to use seen.values(), which (once you use an
> OrderedDict() instead of plain {}) is identical to all_members.
>
>>
>> 3. Check for collisions
>>
>> This works by accumulating names in seen. Precondition: seen
>> contains the names seen so far.
>>
>> Note that this part uses seen like a set. See 4.
>
> Unchanged by this patch; but see also 2/4 and 3/4.
>
>>
>> 4. Accumulate a map from names to members
>>
>> seen serves as accumulator.
>>
>
> Unchanged by this patch.
>
>> We'll see that its only actual user is the owning object type's
>> variants.check(), which uses it to compute variants.tag_member from
>> variants.tag_name.
>>
>>> However, the three callers of this method
>>> pass in the following parameters:
>>>
>>> QAPISchemaObjectType.check():
>>> - all_members contains all non-variant members seen to date,
>>> for use in populating self.members
>>> - seen contains all non-variant members seen to date, for
>>> use in checking for collisions
>>
>> Yes, and:
>>
>> - we're calling it for m in self.local_members
>> - before the loop, all_members and seen are initialized to the inherited
>> non-variant members
>> - after the loop, they therefore contain all non-variant members
>>
>> This caller uses all four things done by QAPISchemaObjectType.check():
>>
>> 1. Compute m.type
>
> Unchanged by this patch.
>
>> 2. Accumulate non-variant members
>
> Whether the accumulation is done via all_members (pre-patch) or by
> seen.values() (post-patch), this step is still done.
>
>> 3. Check for collisions among non-variant members
>> Before the loop, seen contains the inherited members, which don't
>> collide (self.base.check() ensures that). The loop adds the local
>> members one by one, checking for collisions.
>
> Unchanged by this patch.
>
>> 4. Accumulate a map from names to non-variant members
>> Similar argument to 3.
>
> Unchanged by this patch.
>
>>
>>> QAPISchemaObjectTypeVariant.check():
>>
>> Do you mean QAPISchemaObjectVariants.check()?
>
> QAPISchemaObjectTypeVariants.check() calls
> QAPISchemaObjectTypeVariant.check() for each variant, but with a fresh
> copy of seen. We'll later need to expand this copy of seen (patch 2/4),
> but for this patch its use is unchanged - we are appending a single
> value (the tag value) which is wrong, but no one cares that we appended
> it because it was a copy. Patch 3/4 fixes to not append to it.
>
>>
>>> - all_members is a throwaway empty list
>>> - seen is a throwaway dictionary created as a copy by
>>> QAPISchemaObjectVariants.check() (since the members of
>>> one variant cannot collide with those from another), for
>>> use in checking for collisions (technically, we no longer
>>> need to check for collisions between tag values and QMP
>>> key names, but that's a cleanup for another patch)
>>>
>>> QAPISchemaAlternateType.check():
>>> - all_members is a throwaway empty list
>>> - seen is a throwaway empty dict
>>
>> I'm afraid you're omitting a few steps here, and I think you missed
>> QAPISchemaObjectVariants.check()'s self.tag_member.check().
>
> There is no self.tag_member.check(), any more; rather, my earlier
> patches have reworked things so that tag_member is checked by the owner
> (a flat union's base type, a simple union's local_members, or directly
> in QAPISchemaAlternateType prior to calling Variants.check()). I guess
> that's a pitfall of seeing this patch without my rework of 5/17 to
> address your comments there.
I'm assuming this patch is based on
[PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
which has
def check(self, schema, members, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
elif seen: # simple union
assert self.tag_member in seen.itervalues()
else: # alternate
---> self.tag_member.check(schema, members, seen)
in QAPISchemaObjectTypeVariants.
>>> Therefore, in the one case where we care about all_members
>>> after seen has been populated, we know that it contains the
>>> same members as seen.values(); changing seen to be an
>>> OrderedDict() is sufficient to pick up this information with
>>> one less parameter being passed around.
>>
>> I believe the first step should be dropping the obsolete check for
>> collision of tag value with non-variant members. I believe this should
>> do:
>>
>> @@ -1059,8 +1059,7 @@ class QAPISchemaObjectTypeVariants(object):
>> self.tag_member.check(schema, members, seen)
>> assert isinstance(self.tag_member.type, QAPISchemaEnumType)
>> for v in self.variants:
>> - vseen = dict(seen)
>> - v.check(schema, self.tag_member.type, vseen)
>> + v.check(schema, self.tag_member.type, {})
>
> Close, but not quite. It should do:
>
> + cases = {}
> for v in self.variants:
> vseen = dict(seen)
> - v.check(schema, self.tag_member.type, vseen)
> + v.check(schema, self.tag_member.type, vseen, cases)
>
> coupled with this in QAPISchemaObjectTypeVariant:
>
> - def check(self, schema, tag_type, seen):
> - QAPISchemaObjectTypeMember.check(self, schema, [], seen)
> + def check(self, schema, tag_type, seen, cases):
> + QAPISchemaObjectTypeMember.check(self, schema, [], cases)
>
> so that we are now checking collisions between tag values, rather than
> cases. But that's what I did in patch 3/4. And we still need seen
> passed to Variant.check(), because that's the checking added in 2/4.
A sanity check "no collisions between tag values" doesn't hurt, but I'd
simply rely on the tag type's .check(). The tag type is an enumeration
type, its check() ensures the enumeration values are distinct. For
variants, checking the tag value is a is a value of the tag type
suffices to ensure they don't collide.
> Okay, you've convinced me - when I post v9, I'll reorder these four
> patches to put 3/4 first.
>
>>
>>
>> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
>>
>> Then only one caller about 2-4., namely QAPISchemaObjectType.check().
>> Simplify radically: move 2-4. to the caller that cares, drop parameters
>> all_members and seen.
>
> Nope - because seen (well, a copy of seen) is still important to patch 2/4.
>
>>
>> Still to do then: non-variant member collision checking. Factor out
>> 3. into a helper function, use it for non-variant members.
>
> Factoring into a helper function is done in 4/4. I can try and
> rearrange that earlier, too.
>
>>
>> What do you think?
>>
>
> Can you at least look at 2, 3, and 4 to see where I'm headed, and then I
> can rearrange things for the v9 spin? We're probably talking a bit past
> each other, with the same end goal, but a muddle in the middle of how to
> get there.
Yes, we're almost certainly headed in the same direction. But I got
thoroughly confused and lost in the details on the way, so I had to hack
things up myself to clear my head. The result looks nice to me, so I'll
send it out in a jiffie in the hope you'll find it useful.
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions
2015-11-03 11:06 ` Markus Armbruster
2015-11-03 13:26 ` Eric Blake
@ 2015-11-03 14:04 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 2/7] qapi: Simplify QAPISchemaObjectTypeMember.check() Markus Armbruster
` (5 more replies)
1 sibling, 6 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
Union tag values can't clash with member names in generated C anymore
since commit e4ba22b, but QAPISchemaObjectTypeVariants.check() still
asserts they don't. Drop it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 84ac151..a303929 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1059,8 +1059,7 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member.check(schema, members, seen)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
- vseen = dict(seen)
- v.check(schema, self.tag_member.type, vseen)
+ v.check(schema, self.tag_member.type, {})
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 2/7] qapi: Simplify QAPISchemaObjectTypeMember.check()
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
@ 2015-11-03 14:04 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 3/7] qapi: Clean up after previous commit Markus Armbruster
` (4 subsequent siblings)
5 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
QAPISchemaObjectTypeMember.check() currently does four things:
1. Compute self.type
2. Accumulate members in all_members
Only one caller cares: QAPISchemaObjectType.check() uses it to
compute self.members. The other callers pass a throw-away
accumulator.
3. Accumulate a map from names to members in seen
Only one caller cares: QAPISchemaObjectType.check() uses it to
compute its local variable seen, for self.variants.check(), which
uses it to compute self.variants.tag_member from
self.variants.tag_name. The other callers pass a throw-away
accumulator.
4. Check for collisions
This piggyback on 3: before adding a new entry, we assert it's new.
Only one caller cares: QAPISchemaObjectType.check() uses it to
assert non-variant members don't clash.
Simplify QAPISchemaObjectType.check(): move 2.-4. to
QAPISchemaObjectType.check(), and drop parameters all_members and
seen.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a303929..20fdfbf 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -990,7 +990,10 @@ class QAPISchemaObjectType(QAPISchemaType):
assert c_name(m.name) not in seen
seen[m.name] = m
for m in self.local_members:
- m.check(schema, members, seen)
+ m.check(schema)
+ assert m.name not in seen
+ seen[m.name] = m
+ members.append(m)
if self.variants:
self.variants.check(schema, members, seen)
self.members = members
@@ -1027,12 +1030,9 @@ class QAPISchemaObjectTypeMember(object):
self.type = None
self.optional = optional
- def check(self, schema, all_members, seen):
- assert self.name not in seen
+ def check(self, schema):
self.type = schema.lookup_type(self._type_name)
assert self.type
- all_members.append(self)
- seen[self.name] = self
class QAPISchemaObjectTypeVariants(object):
@@ -1056,7 +1056,7 @@ class QAPISchemaObjectTypeVariants(object):
elif seen: # simple union
assert self.tag_member in seen.itervalues()
else: # alternate
- self.tag_member.check(schema, members, seen)
+ self.tag_member.check(schema)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
v.check(schema, self.tag_member.type, {})
@@ -1067,7 +1067,7 @@ class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
def check(self, schema, tag_type, seen):
- QAPISchemaObjectTypeMember.check(self, schema, [], seen)
+ QAPISchemaObjectTypeMember.check(self, schema)
assert self.name in tag_type.values
# This function exists to support ugly simple union special cases
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 3/7] qapi: Clean up after previous commit
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 2/7] qapi: Simplify QAPISchemaObjectTypeMember.check() Markus Armbruster
@ 2015-11-03 14:04 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 4/7] qapi: Fix up commit 7618b91's clash sanity checking change Markus Armbruster
` (3 subsequent siblings)
5 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
QAPISchemaObjectTypeVariants.check() parameter members and
QAPISchemaObjectTypeVariant.check() parameter seen are no longer used,
drop them.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 20fdfbf..7f8af41 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -995,7 +995,7 @@ class QAPISchemaObjectType(QAPISchemaType):
seen[m.name] = m
members.append(m)
if self.variants:
- self.variants.check(schema, members, seen)
+ self.variants.check(schema, seen)
self.members = members
def is_implicit(self):
@@ -1050,7 +1050,7 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member = tag_member
self.variants = variants
- def check(self, schema, members, seen):
+ def check(self, schema, seen):
if self.tag_name: # flat union
self.tag_member = seen[self.tag_name]
elif seen: # simple union
@@ -1059,14 +1059,14 @@ class QAPISchemaObjectTypeVariants(object):
self.tag_member.check(schema)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
- v.check(schema, self.tag_member.type, {})
+ v.check(schema, self.tag_member.type)
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
def __init__(self, name, typ):
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
- def check(self, schema, tag_type, seen):
+ def check(self, schema, tag_type):
QAPISchemaObjectTypeMember.check(self, schema)
assert self.name in tag_type.values
@@ -1089,7 +1089,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
- self.variants.check(schema, [], {})
+ self.variants.check(schema, {})
def json_type(self):
return 'value'
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 4/7] qapi: Fix up commit 7618b91's clash sanity checking change
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 2/7] qapi: Simplify QAPISchemaObjectTypeMember.check() Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 3/7] qapi: Clean up after previous commit Markus Armbruster
@ 2015-11-03 14:04 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 5/7] qapi: Eliminate QAPISchemaObjectType.check() variable members Markus Armbruster
` (2 subsequent siblings)
5 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
This hunk
@@ -964,6 +965,7 @@ class QAPISchemaObjectType(QAPISchemaType):
members = []
seen = {}
for m in members:
+ assert c_name(m.name) not in seen
seen[m.name] = m
for m in self.local_members:
m.check(schema, members, seen)
is plainly broken.
Asserting the members inherited from base don't clash is somewhat
redundant, because self.base.check() just checked that. But it
doesn't hurt.
The idea to use c_name(m.name) instead of m.name for collision
checking is sound, because we need to catch clashes between the m.name
and between the c_name(m.name), and when two m.name clash, then their
c_name() also clash.
However, using c_name(m.name) instead of m.name in one of several
places doesn't work. See the very next line.
Keep the assertion, but drop the c_name() for now. A future commit
will bring it back.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 7f8af41..a6fdd76 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -987,7 +987,7 @@ class QAPISchemaObjectType(QAPISchemaType):
members = []
seen = {}
for m in members:
- assert c_name(m.name) not in seen
+ assert m.name not in seen
seen[m.name] = m
for m in self.local_members:
m.check(schema)
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 5/7] qapi: Eliminate QAPISchemaObjectType.check() variable members
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
` (2 preceding siblings ...)
2015-11-03 14:04 ` [Qemu-devel] [PATCH 4/7] qapi: Fix up commit 7618b91's clash sanity checking change Markus Armbruster
@ 2015-11-03 14:04 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 6/7] qapi: Factor out QAPISchemaObjectTypeMember.check_clash() Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 7/7] qapi: QAPISchemaObjectTypeVariants.check() Markus Armbruster
5 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
We can use seen.values() instead if we make it an OrderedDict.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a6fdd76..a46fded 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -977,26 +977,22 @@ class QAPISchemaObjectType(QAPISchemaType):
if self.members:
return
self.members = False # mark as being checked
+ seen = OrderedDict()
if self._base_name:
self.base = schema.lookup_type(self._base_name)
assert isinstance(self.base, QAPISchemaObjectType)
assert not self.base.variants # not implemented
self.base.check(schema)
- members = list(self.base.members)
- else:
- members = []
- seen = {}
- for m in members:
- assert m.name not in seen
- seen[m.name] = m
+ for m in self.base.members:
+ assert m.name not in seen
+ seen[m.name] = m
for m in self.local_members:
m.check(schema)
assert m.name not in seen
seen[m.name] = m
- members.append(m)
if self.variants:
self.variants.check(schema, seen)
- self.members = members
+ self.members = seen.values()
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 6/7] qapi: Factor out QAPISchemaObjectTypeMember.check_clash()
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
` (3 preceding siblings ...)
2015-11-03 14:04 ` [Qemu-devel] [PATCH 5/7] qapi: Eliminate QAPISchemaObjectType.check() variable members Markus Armbruster
@ 2015-11-03 14:04 ` Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 7/7] qapi: QAPISchemaObjectTypeVariants.check() Markus Armbruster
5 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
While there, stick in a TODO change key of seen from QAPI name to C
name. Can't do it right away, because it would fail the assertion for
tests/qapi-schema/args-has-clash.json.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a46fded..f0cbb7e 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -984,12 +984,10 @@ class QAPISchemaObjectType(QAPISchemaType):
assert not self.base.variants # not implemented
self.base.check(schema)
for m in self.base.members:
- assert m.name not in seen
- seen[m.name] = m
+ m.check_clash(seen)
for m in self.local_members:
m.check(schema)
- assert m.name not in seen
- seen[m.name] = m
+ m.check_clash(seen)
if self.variants:
self.variants.check(schema, seen)
self.members = seen.values()
@@ -1030,6 +1028,11 @@ class QAPISchemaObjectTypeMember(object):
self.type = schema.lookup_type(self._type_name)
assert self.type
+ def check_clash(self, seen):
+ # TODO change key of seen from QAPI name to C name
+ assert self.name not in seen
+ seen[self.name] = self
+
class QAPISchemaObjectTypeVariants(object):
def __init__(self, tag_name, tag_member, variants):
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [Qemu-devel] [PATCH 7/7] qapi: QAPISchemaObjectTypeVariants.check()
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
` (4 preceding siblings ...)
2015-11-03 14:04 ` [Qemu-devel] [PATCH 6/7] qapi: Factor out QAPISchemaObjectTypeMember.check_clash() Markus Armbruster
@ 2015-11-03 14:04 ` Markus Armbruster
5 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 14:04 UTC (permalink / raw)
To: qemu-devel; +Cc: mdroth
Avoid the ugly flat union / simple union / alternate conditional by
doing just the essential work here, namely setting self.tag_member.
Move the rest to callers.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
scripts/qapi.py | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi.py b/scripts/qapi.py
index f0cbb7e..86341e6 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -988,9 +988,10 @@ class QAPISchemaObjectType(QAPISchemaType):
for m in self.local_members:
m.check(schema)
m.check_clash(seen)
+ self.members = seen.values()
if self.variants:
self.variants.check(schema, seen)
- self.members = seen.values()
+ assert self.variants.tag_member in self.members
def is_implicit(self):
# See QAPISchema._make_implicit_object_type()
@@ -1050,12 +1051,8 @@ class QAPISchemaObjectTypeVariants(object):
self.variants = variants
def check(self, schema, seen):
- if self.tag_name: # flat union
+ if not self.tag_member: # flat union
self.tag_member = seen[self.tag_name]
- elif seen: # simple union
- assert self.tag_member in seen.itervalues()
- else: # alternate
- self.tag_member.check(schema)
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
for v in self.variants:
v.check(schema, self.tag_member.type)
@@ -1088,6 +1085,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
self.variants = variants
def check(self, schema):
+ self.variants.tag_member.check(schema)
self.variants.check(schema, {})
def json_type(self):
--
2.4.3
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-03 14:02 ` Markus Armbruster
@ 2015-11-03 14:12 ` Eric Blake
2015-11-03 16:19 ` Markus Armbruster
0 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-11-03 14:12 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1973 bytes --]
On 11/03/2015 07:02 AM, Markus Armbruster wrote:
>>> I'm afraid you're omitting a few steps here, and I think you missed
>>> QAPISchemaObjectVariants.check()'s self.tag_member.check().
>>
>> There is no self.tag_member.check(), any more; rather, my earlier
>> patches have reworked things so that tag_member is checked by the owner
>> (a flat union's base type, a simple union's local_members, or directly
>> in QAPISchemaAlternateType prior to calling Variants.check()). I guess
>> that's a pitfall of seeing this patch without my rework of 5/17 to
>> address your comments there.
>
> I'm assuming this patch is based on
> [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
> which has
>
> def check(self, schema, members, seen):
> if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
> elif seen: # simple union
> assert self.tag_member in seen.itervalues()
> else: # alternate
> ---> self.tag_member.check(schema, members, seen)
>
> in QAPISchemaObjectTypeVariants.
That was true in v8, but not in my pending v9 which did essentially what
your 7/7 did (move the tag_member.check() into Alternate.check() back in
patch 5).
At any rate, my first glance of your series shows that it is reasonable,
so my task today is to spit out a v9 of my series, but using your seven
patches in place of my four.
> Yes, we're almost certainly headed in the same direction. But I got
> thoroughly confused and lost in the details on the way, so I had to hack
> things up myself to clear my head. The result looks nice to me, so I'll
> send it out in a jiffie in the hope you'll find it useful.
>
Yes, it should make v9 easier when it includes your patches, because
then you know what to look for :)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-03 14:12 ` Eric Blake
@ 2015-11-03 16:19 ` Markus Armbruster
2015-11-03 17:34 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 16:19 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/03/2015 07:02 AM, Markus Armbruster wrote:
>
>>>> I'm afraid you're omitting a few steps here, and I think you missed
>>>> QAPISchemaObjectVariants.check()'s self.tag_member.check().
>>>
>>> There is no self.tag_member.check(), any more; rather, my earlier
>>> patches have reworked things so that tag_member is checked by the owner
>>> (a flat union's base type, a simple union's local_members, or directly
>>> in QAPISchemaAlternateType prior to calling Variants.check()). I guess
>>> that's a pitfall of seeing this patch without my rework of 5/17 to
>>> address your comments there.
>>
>> I'm assuming this patch is based on
>> [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
>> which has
>>
>> def check(self, schema, members, seen):
>> if self.tag_name: # flat union
>> self.tag_member = seen[self.tag_name]
>> elif seen: # simple union
>> assert self.tag_member in seen.itervalues()
>> else: # alternate
>> ---> self.tag_member.check(schema, members, seen)
>>
>> in QAPISchemaObjectTypeVariants.
>
> That was true in v8, but not in my pending v9 which did essentially what
> your 7/7 did (move the tag_member.check() into Alternate.check() back in
> patch 5).
>
> At any rate, my first glance of your series shows that it is reasonable,
> so my task today is to spit out a v9 of my series, but using your seven
> patches in place of my four.
Buyer beware: I'm not sure my seven do everything your four do.
>> Yes, we're almost certainly headed in the same direction. But I got
>> thoroughly confused and lost in the details on the way, so I had to hack
>> things up myself to clear my head. The result looks nice to me, so I'll
>> send it out in a jiffie in the hope you'll find it useful.
>>
>
> Yes, it should make v9 easier when it includes your patches, because
> then you know what to look for :)
I hope I'll be less easily confused going forward...
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 08/17] qapi: Remove outdated tests related to QMP/branch collisions
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 08/17] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
@ 2015-11-03 16:32 ` Markus Armbruster
0 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 16:32 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Now that branches are in a separate C namespace, we can remove
> the restrictions in the parser that claim a branch name would
> collide with QMP, and delete the negative tests that are no
> longer problematic. A separate patch can then add positive
> tests to qapi-schema-test to test that any corner cases will
> compile correctly.
>
> This reverts the scripts/qapi.py portion of commit 7b2a5c2,
> now that the assertions that it plugged are no longer possible.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
Yay, an easy patch!
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test Eric Blake
@ 2015-11-03 16:43 ` Markus Armbruster
2015-11-03 16:56 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 16:43 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Add positive tests to qapi-schema-test for things that were
> made possible by recent patches but which caused compile errors
> due to collisions prior to that point.
>
> This includes:
> Use of a member 'base' in a struct with a base class
> Use of a member name ending in 'Kind' or 'List'
> Use of a type name starting with 'has_'
> Use of a type named 'u'
> Use of a union branch name of 'u'
> Use of a union branch name starting with 'has_'
> Use of a union branch name of 'type'
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v8: new, but collects portions of subset B v10 patches 2, 3, and
> 16 and subset C v7 patch 6 that were deferred to later.
>
> It might be worth dropping or simplifying this patch, depending
> on how many corner cases we actually want to test.
Let's go through your list:
* Use of a member 'base' in a struct with a base class
'base' is no longer special, and testing it is as useful as testing
any other non-special member name. I'd drop it. Or am I missing
something?
* Use of a member name ending in 'Kind' or 'List'
These aren't special as member names, but they are reserved type
names. The test ensures we don't accidentally reserve them as member
names. Low probability * low damage = very low risk. But since you
wrote the test already, we can just as well keep it.
* Use of a type name starting with 'has_'
* Use of a type named 'u'
* Use of a union branch name of 'u'
* Use of a union branch name starting with 'has_'
Similarly: these are only reserved as member names, very low risk, but
why not keep the test.
* Use of a union branch name of 'type'
As far as I know, 'type' isn't reserved anywhere, it's just the name
of a simple union's implicit tag member. Testing a tag value doesn't
clash with 'type' is as useful as testing any other non-variant member
name. I'd drop it. Or am I missing something?
Anything else being tested here?
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test
2015-11-03 16:43 ` Markus Armbruster
@ 2015-11-03 16:56 ` Eric Blake
0 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-03 16:56 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 3254 bytes --]
On 11/03/2015 09:43 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Add positive tests to qapi-schema-test for things that were
>> made possible by recent patches but which caused compile errors
>> due to collisions prior to that point.
>>
>> This includes:
>> Use of a member 'base' in a struct with a base class
>> Use of a member name ending in 'Kind' or 'List'
>> Use of a type name starting with 'has_'
>> Use of a type named 'u'
>> Use of a union branch name of 'u'
>> Use of a union branch name starting with 'has_'
>> Use of a union branch name of 'type'
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v8: new, but collects portions of subset B v10 patches 2, 3, and
>> 16 and subset C v7 patch 6 that were deferred to later.
>>
>> It might be worth dropping or simplifying this patch, depending
>> on how many corner cases we actually want to test.
>
> Let's go through your list:
>
> * Use of a member 'base' in a struct with a base class
>
> 'base' is no longer special, and testing it is as useful as testing
> any other non-special member name. I'd drop it. Or am I missing
> something?
Broken prior to commit ddf2190, and no reservation exists. I don't mind
dropping it, especially since it quite distinct from the other grouping
of collision testing.
>
> * Use of a member name ending in 'Kind' or 'List'
>
> These aren't special as member names, but they are reserved type
> names. The test ensures we don't accidentally reserve them as member
> names. Low probability * low damage = very low risk. But since you
> wrote the test already, we can just as well keep it.
>
> * Use of a type name starting with 'has_'
> * Use of a type named 'u'
Never have been broken, but this adds insurance that we don't break it
(and that our reservations added earlier aren't too greedy).
> * Use of a union branch name of 'u'
Broken prior to commit e4ba22b3; also proves that the reservation in
5359baf is not too greedy.
> * Use of a union branch name starting with 'has_'
Broken prior to commit e4ba22b3; also proves that the reservation in
9fb081e is not too greedy.
>
> Similarly: these are only reserved as member names, very low risk, but
> why not keep the test.
>
> * Use of a union branch name of 'type'
Broken prior to commit e4ba22b3. Doesn't have a reservation, so...
>
> As far as I know, 'type' isn't reserved anywhere, it's just the name
> of a simple union's implicit tag member. Testing a tag value doesn't
> clash with 'type' is as useful as testing any other non-variant member
> name. I'd drop it. Or am I missing something?
...I could indeed drop this part of the test, on the same grounds as
dropping the 'base' non-collision tests.
>
> Anything else being tested here?
>
I think that covers it. There's also one more positive test added in
10/17, when I rework the layout of alternate types (at that point, it is
possible to have an alternate branch named 'max' because it no longer
collides with a generated enum _MAX value).
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check()
2015-11-03 16:19 ` Markus Armbruster
@ 2015-11-03 17:34 ` Eric Blake
0 siblings, 0 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-03 17:34 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1940 bytes --]
On 11/03/2015 09:19 AM, Markus Armbruster wrote:
>>> I'm assuming this patch is based on
>>> [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union()
>>> which has
>>>
>>> def check(self, schema, members, seen):
>>> if self.tag_name: # flat union
>>> self.tag_member = seen[self.tag_name]
>>> elif seen: # simple union
>>> assert self.tag_member in seen.itervalues()
>>> else: # alternate
>>> ---> self.tag_member.check(schema, members, seen)
>>>
>>> in QAPISchemaObjectTypeVariants.
>>
>> That was true in v8, but not in my pending v9 which did essentially what
>> your 7/7 did (move the tag_member.check() into Alternate.check() back in
>> patch 5).
Caused me some churn in incorporating your patches, but not too bad.
>>
>> At any rate, my first glance of your series shows that it is reasonable,
>> so my task today is to spit out a v9 of my series, but using your seven
>> patches in place of my four.
>
> Buyer beware: I'm not sure my seven do everything your four do.
Yours do the same as my patch 1. You also made a compelling argument for
eliminating my patch 3 (the enum values are already collision-proof,
therefore if we check that a variant name is in the enum, we don't need
to check for collisions) - except that in my 10/17, when I get rid of
the generated enum for alternates, I have to reinstate that check
somewhere; but adding it directly to QAPISchemaAlternateType.check() at
that point feels better.
My patch 2 is still needed, but looks a bit nicer on top of some of your
refactoring. And my patch 4 is still useful, as additional refactoring
to share the code used by my patch 2 and your code for ObjectType.check().
We'll see how it all turns out in v9.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types Eric Blake
@ 2015-11-03 18:30 ` Markus Armbruster
2015-11-03 18:59 ` Eric Blake
0 siblings, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-11-03 18:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> Previously, working with alternates required two enums, and
> some indirection: for type Foo, we created Foo_qtypes[] which
> maps each qtype to a member of FooKind_lookup[], then use
member of FooKind, actually.
> FooKind_lookup[] like we do for other union types.
You probably mean FooKind here as well.
> This has a subtle bug: since the values of FooKind_lookup
> start at zero, all entries of Foo_qtypes that were not
> explicitly initialized map to the same branch of the union as
> the first member of the alternate, rather than triggering a
> failure in visit_get_next_type(). Fortunately, the bug
> seldom bites; the very next thing the input visitor does is
> try to parse the incoming JSON with the wrong parser, which
> fails; the output visitor is not used with a C struct in that
> state, and the dealloc visitor has nothing to clean up (so
> there is no leak).
Yes, I remember us discussing this bug.
While reading code to double-check your description, I stumbled over
this beauty in generated qapi-visit.c:
visit_get_next_type(v, (int*) &(*obj)->type, BlockdevRef_qtypes, name, &err);
This casts enum BlockdevRefKind * to int *, which assumes the compiler
represents the enum BlockdevRefKind as int or unsigned. It is free to
use any integer type, though. Common mistake of programmers with
insufficiently developed wariness of C's subtleties.
visit_get_next_type() passes the fishy int * on to v->get_next_type().
Only implementation is qmp_input_get_next_type(), which uses it so:
*kind = qobjects[qobject_type(qobj)];
Latent death trap.
Does your patch clean this up?
> However, it IS observable in one case: the behavior of an
> alternate that contains a 'number' member but no 'int' member
> differs according to whether the 'number' was first in the
> qapi definition, and when the input being parsed is an integer;
> this is because the 'number' parser accepts QTYPE_QINT in
> addition to the expected QTYPE_QFLOAT. A later patch will worry
> about fixing alternates to parse all inputs that a non-alternate
> 'number' would accept, for now it is still marked FIXME.
>
> This patch fixes the validation bug by deleting the indirection,
> and modifying get_next_type() to directly return a qtype code.
get_next_type() doesn't return anything. Do you mean "store a qtype
code"?
> There is no longer a need to generate an implicit FooKind array
FooKind is an enum, not an array.
> associated with the alternate type (since the QMP wire format
> never uses the stringized counterparts of the C union member
> names); that also means we no longer have a collision with an
> alternate branch named 'max'. Next, the generated visitor is
> fixed to properly detect unexpected qtypes in the switch
> statement. This is done via the use of a new
> QAPISchemaAlternateTypeTag subclass and the use of a new
> member.c_type() method when producing qapi-types. The new
> subtype also allows us to clean up a TODO left in the previous
> commit.
>
> Callers now have to know the QTYPE_* mapping when looking at the
> discriminator; but so far, only the testsuite was even using the
> C struct of an alternate types. If that gets too confusing, we
> could reintroduce FooKind, but initialize it differently than
> most generated arrays, as in:
> typedef enum FooKind {
> FOO_KIND_A = QTYPE_QDICT,
> FOO_KIND_B = QTYPE_QINT,
> } FooKind;
> to create nicer aliases for knowing when to use foo->a or foo->b
> when inspecting foo->type. But without a current client, I
> didn't see the point of doing it now.
>
> There is a user-visible side effect to this change, but I
> consider it to be an improvement. Previously,
> the invalid QMP command:
> {"execute":"blockdev-add", "arguments":{"options":
> {"driver":"raw", "id":"a", "file":true}}}
> failed with:
> {"error": {"class": "GenericError",
> "desc": "Invalid parameter type for 'file', expected: QDict"}}
> Now it fails with:
> {"error": {"class": "GenericError",
> "desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
I wonder how that happens. Perhaps it's obvious in the patch.
QMP introspection isn't affected, because we carefully minimized the
information to expose there.
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v8: no change
> v7: rebase onto earlier changes, rework how subtype makes things work
> v6: rebase onto tag_member subclass, testsuite, gen_err_check(),
> and info improvements
> ---
> docs/qapi-code-gen.txt | 3 ---
> include/qapi/visitor-impl.h | 3 ++-
> include/qapi/visitor.h | 8 +++++-
> qapi/qapi-visit-core.c | 4 +--
> qapi/qmp-input-visitor.c | 4 +--
> scripts/qapi-types.py | 36 +--------------------------
> scripts/qapi-visit.py | 12 +++++----
> scripts/qapi.py | 43 ++++++++++++++++++++++-----------
> tests/qapi-schema/alternate-empty.out | 1 -
> tests/qapi-schema/qapi-schema-test.json | 2 +-
> tests/qapi-schema/qapi-schema-test.out | 10 +-------
> tests/test-qmp-input-visitor.c | 33 +++++++++++++------------
> tests/test-qmp-output-visitor.c | 21 ++++++++++++----
> 13 files changed, 86 insertions(+), 94 deletions(-)
>
> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
> index 163f547..0c1bd36 100644
> --- a/docs/qapi-code-gen.txt
> +++ b/docs/qapi-code-gen.txt
> @@ -383,9 +383,6 @@ where each branch of the union names a QAPI type. For example:
> 'data': { 'definition': 'BlockdevOptions',
> 'reference': 'str' } }
>
> -Just like for a simple union, an implicit C enum 'NameKind' is created
> -to enumerate the branches for the alternate 'Name'.
> -
> Unlike a union, the discriminator string is never passed on the wire
> for the Client JSON Protocol. Instead, the value's JSON type serves
> as an implicit discriminator, which in turn means that an alternate
> diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> index 8c0ba57..6d95b36 100644
> --- a/include/qapi/visitor-impl.h
> +++ b/include/qapi/visitor-impl.h
> @@ -32,7 +32,8 @@ struct Visitor
>
> void (*type_enum)(Visitor *v, int *obj, const char * const strings[],
> const char *kind, const char *name, Error **errp);
> - void (*get_next_type)(Visitor *v, int *kind, const int *qobjects,
> + /* May be NULL; most useful for input visitors. */
> + void (*get_next_type)(Visitor *v, qtype_code *type,
> const char *name, Error **errp);
>
> void (*type_int)(Visitor *v, int64_t *obj, const char *name, Error **errp);
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index cfc19a6..b765993 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -41,7 +41,13 @@ GenericList *visit_next_list(Visitor *v, GenericList **list, Error **errp);
> void visit_end_list(Visitor *v, Error **errp);
> void visit_optional(Visitor *v, bool *present, const char *name,
> Error **errp);
> -void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
> +
> +/**
> + * Determine the qtype of the item @name in the current object visit.
> + * For input visitors, set *@type to the correct qtype of a qapi
> + * alternate type; for other visitors, leave *@type unchanged.
> + */
> +void visit_get_next_type(Visitor *v, qtype_code *type,
> const char *name, Error **errp);
Naive question: what makes a visitor an input visitor?
> void visit_type_enum(Visitor *v, int *obj, const char * const strings[],
> const char *kind, const char *name, Error **errp);
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 59ed506..3f24daa 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -81,11 +81,11 @@ void visit_optional(Visitor *v, bool *present, const char *name,
> }
> }
>
> -void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
> +void visit_get_next_type(Visitor *v, qtype_code *type,
> const char *name, Error **errp)
> {
> if (v->get_next_type) {
> - v->get_next_type(v, obj, qtypes, name, errp);
> + v->get_next_type(v, type, name, errp);
> }
> }
>
> diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
> index 5dd9ed5..803ffad 100644
> --- a/qapi/qmp-input-visitor.c
> +++ b/qapi/qmp-input-visitor.c
> @@ -208,7 +208,7 @@ static void qmp_input_end_list(Visitor *v, Error **errp)
> qmp_input_pop(qiv, errp);
> }
>
> -static void qmp_input_get_next_type(Visitor *v, int *kind, const int *qobjects,
> +static void qmp_input_get_next_type(Visitor *v, qtype_code *type,
> const char *name, Error **errp)
> {
> QmpInputVisitor *qiv = to_qiv(v);
> @@ -218,7 +218,7 @@ static void qmp_input_get_next_type(Visitor *v, int *kind, const int *qobjects,
> error_setg(errp, QERR_MISSING_PARAMETER, name ? name : "null");
> return;
> }
> - *kind = qobjects[qobject_type(qobj)];
> + *type = qobject_type(qobj);
> }
>
> static void qmp_input_type_int(Visitor *v, int64_t *obj, const char *name,
> diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
> index 403768b..3fd07fd 100644
> --- a/scripts/qapi-types.py
> +++ b/scripts/qapi-types.py
> @@ -47,7 +47,7 @@ def gen_struct_field(member):
> ret += mcgen('''
> %(c_type)s %(c_name)s;
> ''',
> - c_type=member.type.c_type(), c_name=c_name(member.name))
> + c_type=member.c_type(), c_name=c_name(member.name))
This uses the new member.c_type() defined further down in the patch.
No change if member is an instance of QAPISchemaObjectTypeMember.
> return ret
>
>
> @@ -111,38 +111,6 @@ static inline %(base)s *qapi_%(c_name)s_base(const %(c_name)s *obj)
> c_name=c_name(name), base=base.c_name())
>
>
> -def gen_alternate_qtypes_decl(name):
> - return mcgen('''
> -
> -extern const int %(c_name)s_qtypes[];
> -''',
> - c_name=c_name(name))
> -
> -
> -def gen_alternate_qtypes(name, variants):
> - ret = mcgen('''
> -
> -const int %(c_name)s_qtypes[QTYPE_MAX] = {
> -''',
> - c_name=c_name(name))
> -
> - for var in variants.variants:
> - qtype = var.type.alternate_qtype()
> - assert qtype
> -
> - ret += mcgen('''
> - [%(qtype)s] = %(enum_const)s,
> -''',
> - qtype=qtype,
> - enum_const=c_enum_const(variants.tag_member.type.name,
> - var.name))
> -
> - ret += mcgen('''
> -};
> -''')
> - return ret
> -
> -
> def gen_variants(variants):
> # FIXME: What purpose does data serve, besides preventing a union that
> # has a branch named 'data'? We use it in qapi-visit.py to decide
> @@ -267,9 +235,7 @@ class QAPISchemaGenTypeVisitor(QAPISchemaVisitor):
>
> def visit_alternate_type(self, name, info, variants):
> self._fwdecl += gen_fwd_object_or_array(name)
> - self._fwdefn += gen_alternate_qtypes(name, variants)
> self.decl += gen_object(name, None, [variants.tag_member], variants)
> - self.decl += gen_alternate_qtypes_decl(name)
> self._gen_type_cleanup(name)
>
> # If you link code generated from multiple schemata, you want only one
> diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
> index 318b8e6..2091a0f 100644
> --- a/scripts/qapi-visit.py
> +++ b/scripts/qapi-visit.py
> @@ -189,7 +189,7 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
> if (err) {
> goto out;
> }
> - visit_get_next_type(v, (int*) &(*obj)->type, %(c_name)s_qtypes, name, &err);
> + visit_get_next_type(v, &(*obj)->type, name, &err);
> if (err) {
> goto out_obj;
> }
Yes, your patch disarms the latent death trap: no more pointer casting.
> @@ -203,14 +203,14 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
> visit_type_%(c_type)s(v, &(*obj)->u.%(c_name)s, name, &err);
> break;
> ''',
> - case=c_enum_const(variants.tag_member.type.name,
> - var.name),
> + case=var.type.alternate_qtype(),
> c_type=var.type.c_name(),
> c_name=c_name(var.name))
>
> ret += mcgen('''
> default:
> - abort();
> + error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
> + "%(name)s");
Okay, this is where the new error message comes from.
Before, default is unreachable, because (*obj)->type got erroneously set
the enum's first member when none of the alternate's variants matches
the qtype.
After, (*obj)->type *is* the qtype, and we do reach default when no
variant matches.
How can name be null?
I really need to finish the QERR_ killing job.
> }
> out_obj:
> error_propagate(errp, err);
> @@ -219,7 +219,8 @@ out_obj:
> out:
> error_propagate(errp, err);
> }
> -''')
> +''',
> + name=name)
>
> return ret
>
> @@ -425,6 +426,7 @@ fdef.write(mcgen('''
>
> fdecl.write(mcgen('''
> #include "qapi/visitor.h"
> +#include "qapi/qmp/qerror.h"
> #include "%(prefix)sqapi-types.h"
>
> ''',
> diff --git a/scripts/qapi.py b/scripts/qapi.py
> index a0639e4..bd74470 100644
> --- a/scripts/qapi.py
> +++ b/scripts/qapi.py
> @@ -627,15 +627,15 @@ def check_union(expr, expr_info):
> def check_alternate(expr, expr_info):
> name = expr['alternate']
> members = expr['data']
> - values = {'MAX': '(automatic)'}
> + values = {}
> types_seen = {}
>
> # Check every branch
> for (key, value) in members.items():
> check_name(expr_info, "Member of alternate '%s'" % name, key)
>
> - # Check for conflicts in the generated enum
> - c_key = camel_to_upper(key)
> + # Check for conflicts in the branch names
> + c_key = c_name(key)
Why c_name()?
> if c_key in values:
> raise QAPIExprError(expr_info,
> "Alternate '%s' member '%s' clashes with '%s'"
> @@ -1029,13 +1029,17 @@ class QAPISchemaObjectTypeMember(object):
> assert self.type
> seen[self.name] = self
>
> + def c_type(self):
> + return self.type.c_type()
> +
>
> class QAPISchemaObjectTypeVariants(object):
> def __init__(self, tag_name, tag_member, variants):
> # Flat unions pass tag_name but not tag_member.
> # Simple unions and alternates pass tag_member but not tag_name.
> # After check(), tag_member is always set, and tag_name remains
> - # a reliable witness of being used by a flat union.
> + # a reliable witness of being used by a flat union, and
> + # tag_member.type being None is a reliable witness of an alternate.
A member without a type? Ugh! I wouldn't dare breaking invariants like
that.
Of course, an alternate's tag member still has a type: qtype_code. It's
just not declared in the schema. Should it be a built-in type then?
> assert bool(tag_member) != bool(tag_name)
> assert (isinstance(tag_name, str) or
> isinstance(tag_member, QAPISchemaObjectTypeMember))
> @@ -1045,8 +1049,7 @@ class QAPISchemaObjectTypeVariants(object):
> self.tag_member = tag_member
> self.variants = variants
>
> - # TODO drop union once alternates can be distinguished by tag_member
> - def check(self, schema, seen, union=True):
> + def check(self, schema, seen):
> if self.tag_name: # flat union
> self.tag_member = seen[self.tag_name]
> assert self.tag_member
> @@ -1054,25 +1057,25 @@ class QAPISchemaObjectTypeVariants(object):
> assert self.tag_member in seen.itervalues()
> else: # alternate
> self.tag_member.check(schema, seen)
> - assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> + if not isinstance(self.tag_member, QAPISchemaAlternateTypeTag):
> + assert isinstance(self.tag_member.type, QAPISchemaEnumType)
> cases = OrderedDict()
> for v in self.variants:
> # Reset seen array for each variant, since QMP names from one
> # branch do not affect another branch, nor add to all_members
> - v.check(schema, self.tag_member.type, dict(seen), cases, union)
> + v.check(schema, self.tag_member.type, dict(seen), cases)
I expect some rebase churn around here, so I'm not reviewing closely.
>
>
> class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
> def __init__(self, name, typ):
> QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
>
> - # TODO drop union once alternates can be distinguished by tag_type
> - def check(self, schema, tag_type, seen, cases, union):
> + def check(self, schema, tag_type, seen, cases):
> # cases is case names we must not collide with
> QAPISchemaObjectTypeMember.check(self, schema, cases)
> - assert self.name in tag_type.values
> - if union:
> + if tag_type:
> # seen is QMP names our members must not collide with
> + assert self.name in tag_type.values
> self.type.check_qmp(schema, seen)
>
My patches move the member name collision checking to
QAPISchemaObjectType.check().
I suspect alternate branch name collision checking should similarly move
to QAPISchemaAlternateType.check().
Union branch name collision checking is superfluous, because they
already get checked by enum member name collision checking.
> # This function exists to support ugly simple union special cases
> @@ -1094,7 +1097,7 @@ class QAPISchemaAlternateType(QAPISchemaType):
> self.variants = variants
>
> def check(self, schema):
> - self.variants.check(schema, {}, False)
> + self.variants.check(schema, {})
>
> def json_type(self):
> return 'value'
> @@ -1103,6 +1106,18 @@ class QAPISchemaAlternateType(QAPISchemaType):
> visitor.visit_alternate_type(self.name, self.info, self.variants)
>
>
> +class QAPISchemaAlternateTypeTag(QAPISchemaObjectTypeMember):
> + def __init__(self):
> + QAPISchemaObjectTypeMember.__init__(self, 'type', '', False)
> +
> + def check(self, schema, seen):
> + assert len(seen) == 0
> + seen[self.name] = self
> +
> + def c_type(self):
> + return 'qtype_code'
> +
> +
This is a hack to work around the lack of a qtype_code type. I suspect
creating such a type would be simpler in the end. Safer, too, because
it would avoid having members without a type, which scares me.
> class QAPISchemaCommand(QAPISchemaEntity):
> def __init__(self, name, info, arg_type, ret_type, gen, success_response):
> QAPISchemaEntity.__init__(self, name, info)
> @@ -1296,7 +1311,7 @@ class QAPISchema(object):
> data = expr['data']
> variants = [self._make_variant(key, value)
> for (key, value) in data.iteritems()]
> - tag_member = self._make_implicit_tag(name, info, variants)
> + tag_member = QAPISchemaAlternateTypeTag()
> self._def_entity(
> QAPISchemaAlternateType(name, info,
> QAPISchemaObjectTypeVariants(None,
> diff --git a/tests/qapi-schema/alternate-empty.out b/tests/qapi-schema/alternate-empty.out
> index 0f153b6..9b010d8 100644
> --- a/tests/qapi-schema/alternate-empty.out
> +++ b/tests/qapi-schema/alternate-empty.out
> @@ -1,4 +1,3 @@
> object :empty
> alternate Alt
> case i: int
> -enum AltKind ['i']
> diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
> index 4354604..4513d94 100644
> --- a/tests/qapi-schema/qapi-schema-test.json
> +++ b/tests/qapi-schema/qapi-schema-test.json
> @@ -131,7 +131,7 @@
> 'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero',
> 'u': 'UserDefZero', 'type': 'UserDefZero' } }
> { 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool',
> - 'myKind': 'has_a' } }
> + 'myKind': 'has_a', 'max': 'str' } }
Here, you add the positive test that alternate name 'max' works.
One, not mentioned in the commit message.
Two, the commit message says we may reintroduce FooKind if working with
qtype_code turns out to be too confusing. If we ever do that, alternate
name 'max' breaks, doesn't it? Shouldn't we keep it reserved then, just
in case?
>
> # testing commands
> { 'command': 'user_def_cmd', 'data': {} }
> diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
> index 29c46da..eda71b9 100644
> --- a/tests/qapi-schema/qapi-schema-test.out
> +++ b/tests/qapi-schema/qapi-schema-test.out
> @@ -60,32 +60,26 @@ object :obj-user_def_cmd2-arg
> alternate AltIntNum
> case i: int
> case n: number
> -enum AltIntNumKind ['i', 'n']
> alternate AltName
> case type: int
> case u: bool
> case myKind: has_a
> -enum AltNameKind ['type', 'u', 'myKind']
> + case max: str
> alternate AltNumInt
> case n: number
> case i: int
> -enum AltNumIntKind ['n', 'i']
> alternate AltNumStr
> case n: number
> case s: str
> -enum AltNumStrKind ['n', 's']
> alternate AltStrBool
> case s: str
> case b: bool
> -enum AltStrBoolKind ['s', 'b']
> alternate AltStrInt
> case s: str
> case i: int
> -enum AltStrIntKind ['s', 'i']
> alternate AltStrNum
> case s: str
> case n: number
> -enum AltStrNumKind ['s', 'n']
> event EVENT_A None
> event EVENT_B None
> event EVENT_C :obj-EVENT_C-arg
> @@ -129,7 +123,6 @@ alternate UserDefAlternate
> case uda: UserDefA
> case s: str
> case i: int
> -enum UserDefAlternateKind ['uda', 's', 'i']
> object UserDefB
> member intb: int optional=False
> member a-b: bool optional=True
> @@ -198,7 +191,6 @@ event __ORG.QEMU_X-EVENT __org.qemu_x-Struct
> alternate __org.qemu_x-Alt
> case __org.qemu_x-branch: str
> case b: __org.qemu_x-Base
> -enum __org.qemu_x-AltKind ['__org.qemu_x-branch', 'b']
> object __org.qemu_x-Base
> member __org.qemu_x-member1: __org.qemu_x-Enum optional=False
> enum __org.qemu_x-Enum ['__org.qemu_x-value']
> diff --git a/tests/test-qmp-input-visitor.c b/tests/test-qmp-input-visitor.c
> index 3f6bc4d..552fc74 100644
> --- a/tests/test-qmp-input-visitor.c
> +++ b/tests/test-qmp-input-visitor.c
> @@ -343,14 +343,14 @@ static void test_visitor_in_alternate(TestInputVisitorData *data,
>
> v = visitor_input_test_init(data, "42");
> visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
> - g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_I);
> + g_assert_cmpint(tmp->type, ==, QTYPE_QINT);
> g_assert_cmpint(tmp->u.i, ==, 42);
> qapi_free_UserDefAlternate(tmp);
> visitor_input_teardown(data, NULL);
>
> v = visitor_input_test_init(data, "'string'");
> visit_type_UserDefAlternate(v, &tmp, NULL, &error_abort);
> - g_assert_cmpint(tmp->type, ==, USER_DEF_ALTERNATE_KIND_S);
> + g_assert_cmpint(tmp->type, ==, QTYPE_QSTRING);
> g_assert_cmpstr(tmp->u.s, ==, "string");
> qapi_free_UserDefAlternate(tmp);
> visitor_input_teardown(data, NULL);
> @@ -386,11 +386,10 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
> qapi_free_AltStrBool(asb);
> visitor_input_teardown(data, NULL);
>
> - /* FIXME: Order of alternate should not affect semantics; asn should
> - * parse the same as ans */
> + /* FIXME: integer should parse as number */
> v = visitor_input_test_init(data, "42");
> visit_type_AltStrNum(v, &asn, NULL, &err);
> - /* FIXME g_assert_cmpint(asn->type, == ALT_STR_NUM_KIND_N); */
> + /* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */
> /* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */
> g_assert(err);
> error_free(err);
> @@ -398,30 +397,34 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
> qapi_free_AltStrNum(asn);
> visitor_input_teardown(data, NULL);
>
> + /* FIXME: integer should parse as number */
> v = visitor_input_test_init(data, "42");
> - visit_type_AltNumStr(v, &ans, NULL, &error_abort);
> - g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
> - g_assert_cmpfloat(ans->u.n, ==, 42);
> + visit_type_AltNumStr(v, &ans, NULL, &err);
> + /* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */
> + /* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */
> + g_assert(err);
> + error_free(err);
> + err = NULL;
What's happening here? Whatever it is, the commit message didn't
prepare me for it...
> qapi_free_AltNumStr(ans);
> visitor_input_teardown(data, NULL);
>
> v = visitor_input_test_init(data, "42");
> visit_type_AltStrInt(v, &asi, NULL, &error_abort);
> - g_assert_cmpint(asi->type, ==, ALT_STR_INT_KIND_I);
> + g_assert_cmpint(asi->type, ==, QTYPE_QINT);
> g_assert_cmpint(asi->u.i, ==, 42);
> qapi_free_AltStrInt(asi);
> visitor_input_teardown(data, NULL);
>
> v = visitor_input_test_init(data, "42");
> visit_type_AltIntNum(v, &ain, NULL, &error_abort);
> - g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_I);
> + g_assert_cmpint(ain->type, ==, QTYPE_QINT);
> g_assert_cmpint(ain->u.i, ==, 42);
> qapi_free_AltIntNum(ain);
> visitor_input_teardown(data, NULL);
>
> v = visitor_input_test_init(data, "42");
> visit_type_AltNumInt(v, &ani, NULL, &error_abort);
> - g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_I);
> + g_assert_cmpint(ani->type, ==, QTYPE_QINT);
> g_assert_cmpint(ani->u.i, ==, 42);
> qapi_free_AltNumInt(ani);
> visitor_input_teardown(data, NULL);
> @@ -438,14 +441,14 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
>
> v = visitor_input_test_init(data, "42.5");
> visit_type_AltStrNum(v, &asn, NULL, &error_abort);
> - g_assert_cmpint(asn->type, ==, ALT_STR_NUM_KIND_N);
> + g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT);
> g_assert_cmpfloat(asn->u.n, ==, 42.5);
> qapi_free_AltStrNum(asn);
> visitor_input_teardown(data, NULL);
>
> v = visitor_input_test_init(data, "42.5");
> visit_type_AltNumStr(v, &ans, NULL, &error_abort);
> - g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
> + g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT);
> g_assert_cmpfloat(ans->u.n, ==, 42.5);
> qapi_free_AltNumStr(ans);
> visitor_input_teardown(data, NULL);
> @@ -460,14 +463,14 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
>
> v = visitor_input_test_init(data, "42.5");
> visit_type_AltIntNum(v, &ain, NULL, &error_abort);
> - g_assert_cmpint(ain->type, ==, ALT_INT_NUM_KIND_N);
> + g_assert_cmpint(ain->type, ==, QTYPE_QFLOAT);
> g_assert_cmpfloat(ain->u.n, ==, 42.5);
> qapi_free_AltIntNum(ain);
> visitor_input_teardown(data, NULL);
>
> v = visitor_input_test_init(data, "42.5");
> visit_type_AltNumInt(v, &ani, NULL, &error_abort);
> - g_assert_cmpint(ani->type, ==, ALT_NUM_INT_KIND_N);
> + g_assert_cmpint(ani->type, ==, QTYPE_QFLOAT);
> g_assert_cmpfloat(ani->u.n, ==, 42.5);
> qapi_free_AltNumInt(ani);
> visitor_input_teardown(data, NULL);
> diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
> index 9364843..695db32 100644
> --- a/tests/test-qmp-output-visitor.c
> +++ b/tests/test-qmp-output-visitor.c
> @@ -449,20 +449,31 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
> const void *unused)
> {
> QObject *arg;
> - Error *err = NULL;
> + UserDefAlternate *tmp;
>
> - UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
> - tmp->type = USER_DEF_ALTERNATE_KIND_I;
> + tmp = g_new0(UserDefAlternate, 1);
> + tmp->type = QTYPE_QINT;
> tmp->u.i = 42;
Coding style touched up. Okay.
>
> - visit_type_UserDefAlternate(data->ov, &tmp, NULL, &err);
> - g_assert(err == NULL);
> + visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
We need to make up our mind whether to use g_assert(err == NULL) or
&error_abort in tests. Wholesale conversion could be in order. I like
&error_abort, because it's more concise.
> arg = qmp_output_get_qobject(data->qov);
>
> g_assert(qobject_type(arg) == QTYPE_QINT);
> g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
>
> qapi_free_UserDefAlternate(tmp);
> +
> + tmp = g_malloc0(sizeof(UserDefAlternate));
g_new0(UserDefAlternate, 1), please.
> + tmp->type = QTYPE_QSTRING;
> + tmp->u.s = g_strdup("hello");
> +
> + visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
> + arg = qmp_output_get_qobject(data->qov);
> +
> + g_assert(qobject_type(arg) == QTYPE_QSTRING);
> + g_assert_cmpstr(qstring_get_str(qobject_to_qstring(arg)), ==, "hello");
> +
> + qapi_free_UserDefAlternate(tmp);
New test, not mentioned in commit message. Separate patch, perhaps,
along with the nearby coding style touch ups?
> }
>
> static void test_visitor_out_empty(TestOutputVisitorData *data,
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-11-03 18:30 ` Markus Armbruster
@ 2015-11-03 18:59 ` Eric Blake
2015-11-04 7:30 ` Markus Armbruster
2015-11-04 16:03 ` Markus Armbruster
0 siblings, 2 replies; 56+ messages in thread
From: Eric Blake @ 2015-11-03 18:59 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 16181 bytes --]
On 11/03/2015 11:30 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
>
>> Previously, working with alternates required two enums, and
>> some indirection: for type Foo, we created Foo_qtypes[] which
>> maps each qtype to a member of FooKind_lookup[], then use
>
> member of FooKind, actually.
Or entry in the FooKind_lookup[] array.
>
>> FooKind_lookup[] like we do for other union types.
>
> You probably mean FooKind here as well.
I'll play with the wording.
>
>> This has a subtle bug: since the values of FooKind_lookup
>> start at zero, all entries of Foo_qtypes that were not
>> explicitly initialized map to the same branch of the union as
>> the first member of the alternate, rather than triggering a
>> failure in visit_get_next_type(). Fortunately, the bug
>> seldom bites; the very next thing the input visitor does is
>> try to parse the incoming JSON with the wrong parser, which
>> fails; the output visitor is not used with a C struct in that
>> state, and the dealloc visitor has nothing to clean up (so
>> there is no leak).
>
> Yes, I remember us discussing this bug.
>
> While reading code to double-check your description, I stumbled over
> this beauty in generated qapi-visit.c:
>
> visit_get_next_type(v, (int*) &(*obj)->type, BlockdevRef_qtypes, name, &err);
>
> This casts enum BlockdevRefKind * to int *, which assumes the compiler
> represents the enum BlockdevRefKind as int or unsigned. It is free to
> use any integer type, though. Common mistake of programmers with
> insufficiently developed wariness of C's subtleties.
>
> visit_get_next_type() passes the fishy int * on to v->get_next_type().
> Only implementation is qmp_input_get_next_type(), which uses it so:
>
> *kind = qobjects[qobject_type(qobj)];
>
> Latent death trap.
>
> Does your patch clean this up?
Yes, and I need to also document that this is an additional bug fix.
>
>> However, it IS observable in one case: the behavior of an
>> alternate that contains a 'number' member but no 'int' member
>> differs according to whether the 'number' was first in the
>> qapi definition, and when the input being parsed is an integer;
>> this is because the 'number' parser accepts QTYPE_QINT in
>> addition to the expected QTYPE_QFLOAT. A later patch will worry
>> about fixing alternates to parse all inputs that a non-alternate
>> 'number' would accept, for now it is still marked FIXME.
See [1] below.
>>
>> This patch fixes the validation bug by deleting the indirection,
>> and modifying get_next_type() to directly return a qtype code.
>
> get_next_type() doesn't return anything. Do you mean "store a qtype
> code"?
Yes.
>
>> There is no longer a need to generate an implicit FooKind array
>
> FooKind is an enum, not an array.
...to generate an implicit FooKind enum, nor FooKind_lookup[] array.
>
>> associated with the alternate type (since the QMP wire format
>> never uses the stringized counterparts of the C union member
>> names); that also means we no longer have a collision with an
>> alternate branch named 'max'. Next, the generated visitor is
>> fixed to properly detect unexpected qtypes in the switch
>> statement. This is done via the use of a new
>> QAPISchemaAlternateTypeTag subclass and the use of a new
>> member.c_type() method when producing qapi-types. The new
>> subtype also allows us to clean up a TODO left in the previous
>> commit.
>>
>> Callers now have to know the QTYPE_* mapping when looking at the
>> discriminator; but so far, only the testsuite was even using the
>> C struct of an alternate types. If that gets too confusing, we
>> could reintroduce FooKind, but initialize it differently than
>> most generated arrays, as in:
>> typedef enum FooKind {
>> FOO_KIND_A = QTYPE_QDICT,
>> FOO_KIND_B = QTYPE_QINT,
>> } FooKind;
>> to create nicer aliases for knowing when to use foo->a or foo->b
>> when inspecting foo->type. But without a current client, I
>> didn't see the point of doing it now.
You have a point below that we either need to reserve MAX and require no
case-insensitive clashes, or that we will never want to add it. I'm
leaning towards never going back, because the new way feels so much nicer.
>>
>> There is a user-visible side effect to this change, but I
>> consider it to be an improvement. Previously,
>> the invalid QMP command:
>> {"execute":"blockdev-add", "arguments":{"options":
>> {"driver":"raw", "id":"a", "file":true}}}
>> failed with:
>> {"error": {"class": "GenericError",
>> "desc": "Invalid parameter type for 'file', expected: QDict"}}
>> Now it fails with:
>> {"error": {"class": "GenericError",
>> "desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
>
> I wonder how that happens. Perhaps it's obvious in the patch.
I think you found it below.
>
> QMP introspection isn't affected, because we carefully minimized the
> information to expose there.
Ooh, nice tidbit to add.
>> +/**
>> + * Determine the qtype of the item @name in the current object visit.
>> + * For input visitors, set *@type to the correct qtype of a qapi
>> + * alternate type; for other visitors, leave *@type unchanged.
>> + */
>> +void visit_get_next_type(Visitor *v, qtype_code *type,
>> const char *name, Error **errp);
>
> Naive question: what makes a visitor an input visitor?
I've got a later patch in my queue that adds a lot more documentation:
http://repo.or.cz/qemu/ericb.git/commitdiff/f7674a87e72
+/* This file describes the client view for visiting a map between
+ * generated QAPI C structs and another representation (command line
+ * options, strings, or QObjects). An input visitor converts from
+ * some other form into QAPI representation; an output visitor
+ * converts from QAPI back into another form. In the descriptions
+ * below, an object or dictionary refers to a JSON '{}', and an array
+ * or list refers to a JSON '[]'. These functions seldom need to be
+ * called directly, but are instead used by code generated by
+ * scripts/qapi-visit.py. For the visitor callback contracts, see
+ * visitor-impl.h. */
>> +++ b/scripts/qapi-visit.py
>> @@ -189,7 +189,7 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
>> if (err) {
>> goto out;
>> }
>> - visit_get_next_type(v, (int*) &(*obj)->type, %(c_name)s_qtypes, name, &err);
>> + visit_get_next_type(v, &(*obj)->type, name, &err);
>> if (err) {
>> goto out_obj;
>> }
>
> Yes, your patch disarms the latent death trap: no more pointer casting.
Indeed, I noticed the cleanup as well (I'm quite familiar with the
unsafe nature of casting enum* because you cannot guarantee its size),
but failed to call out the trap in my commit message.
>
>> @@ -203,14 +203,14 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
>> visit_type_%(c_type)s(v, &(*obj)->u.%(c_name)s, name, &err);
>> break;
>> ''',
>> - case=c_enum_const(variants.tag_member.type.name,
>> - var.name),
>> + case=var.type.alternate_qtype(),
>> c_type=var.type.c_name(),
>> c_name=c_name(var.name))
>>
>> ret += mcgen('''
>> default:
>> - abort();
>> + error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>> + "%(name)s");
>
> Okay, this is where the new error message comes from.
>
> Before, default is unreachable, because (*obj)->type got erroneously set
> the enum's first member when none of the alternate's variants matches
> the qtype.
>
> After, (*obj)->type *is* the qtype, and we do reach default when no
> variant matches.
>
> How can name be null?
When you have the qapi representation ['MyAlternate'], you will have
qapi_visit_type_MyAlternateList() which passes NULL for the name of each
list member (because names are only present for objects, not lists).
>
> I really need to finish the QERR_ killing job.
Agreed. But shouldn't stall this patch, though.
>> # Check every branch
>> for (key, value) in members.items():
>> check_name(expr_info, "Member of alternate '%s'" % name, key)
>>
>> - # Check for conflicts in the generated enum
>> - c_key = camel_to_upper(key)
>> + # Check for conflicts in the branch names
>> + c_key = c_name(key)
>
> Why c_name()?
So that 'a-b' and 'a_b' are properly flagged as conflicting (they map to
the same c_name).
>> class QAPISchemaObjectTypeVariants(object):
>> def __init__(self, tag_name, tag_member, variants):
>> # Flat unions pass tag_name but not tag_member.
>> # Simple unions and alternates pass tag_member but not tag_name.
>> # After check(), tag_member is always set, and tag_name remains
>> - # a reliable witness of being used by a flat union.
>> + # a reliable witness of being used by a flat union, and
>> + # tag_member.type being None is a reliable witness of an alternate.
>
> A member without a type? Ugh! I wouldn't dare breaking invariants like
> that.
>
> Of course, an alternate's tag member still has a type: qtype_code. It's
> just not declared in the schema. Should it be a built-in type then?
It's not a builtin that can ever be referenced in the .json files. But
I could probably come up with something, if it would make you feel better.
>> for v in self.variants:
>> # Reset seen array for each variant, since QMP names from one
>> # branch do not affect another branch, nor add to all_members
>> - v.check(schema, self.tag_member.type, dict(seen), cases, union)
>> + v.check(schema, self.tag_member.type, dict(seen), cases)
>
> I expect some rebase churn around here, so I'm not reviewing closely.
>
Indeed. All the more reason for me to post a v9 spin (and maybe defer
the question of a non-None type for tag_member until after that post).
> My patches move the member name collision checking to
> QAPISchemaObjectType.check().
>
> I suspect alternate branch name collision checking should similarly move
> to QAPISchemaAlternateType.check().
>
Yep, already that way in my pending v9 series after incorporating your
patches.
>> +class QAPISchemaAlternateTypeTag(QAPISchemaObjectTypeMember):
>> + def __init__(self):
>> + QAPISchemaObjectTypeMember.__init__(self, 'type', '', False)
>> +
>> + def check(self, schema, seen):
>> + assert len(seen) == 0
>> + seen[self.name] = self
>> +
>> + def c_type(self):
>> + return 'qtype_code'
>> +
>> +
>
> This is a hack to work around the lack of a qtype_code type. I suspect
> creating such a type would be simpler in the end. Safer, too, because
> it would avoid having members without a type, which scares me.
I can play with dropping c_type() here in favor of adding a qtype_code
special class, but I may still need to keep this
QAPISchemaAlternateTypeTag subclass.
>> +++ b/tests/qapi-schema/qapi-schema-test.json
>> @@ -131,7 +131,7 @@
>> 'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero',
>> 'u': 'UserDefZero', 'type': 'UserDefZero' } }
>> { 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool',
>> - 'myKind': 'has_a' } }
>> + 'myKind': 'has_a', 'max': 'str' } }
>
> Here, you add the positive test that alternate name 'max' works.
>
> One, not mentioned in the commit message.
D'oh.
>
> Two, the commit message says we may reintroduce FooKind if working with
> qtype_code turns out to be too confusing. If we ever do that, alternate
> name 'max' breaks, doesn't it? Shouldn't we keep it reserved then, just
> in case?
>
See my comment above; at this point, I doubt we'll ever want to go back,
so maybe I just need to be more definitive in stating that.
>> +++ b/tests/test-qmp-input-visitor.c
>> @@ -386,11 +386,10 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
>> qapi_free_AltStrBool(asb);
>> visitor_input_teardown(data, NULL);
>>
>> - /* FIXME: Order of alternate should not affect semantics; asn should
>> - * parse the same as ans */
>> + /* FIXME: integer should parse as number */
>> v = visitor_input_test_init(data, "42");
>> visit_type_AltStrNum(v, &asn, NULL, &err);
>> - /* FIXME g_assert_cmpint(asn->type, == ALT_STR_NUM_KIND_N); */
>> + /* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */
>> /* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */
>> g_assert(err);
>> error_free(err);
>> @@ -398,30 +397,34 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
>> qapi_free_AltStrNum(asn);
>> visitor_input_teardown(data, NULL);
>>
>> + /* FIXME: integer should parse as number */
>> v = visitor_input_test_init(data, "42");
>> - visit_type_AltNumStr(v, &ans, NULL, &error_abort);
>> - g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
>> - g_assert_cmpfloat(ans->u.n, ==, 42);
>> + visit_type_AltNumStr(v, &ans, NULL, &err);
>> + /* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */
>> + /* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */
>> + g_assert(err);
>> + error_free(err);
>> + err = NULL;
>
> What's happening here? Whatever it is, the commit message didn't
> prepare me for it...
See [1] above. 'asn' is now parsing the same as 'ans' (we are no longer
sensitive to whether 'number' was the first member of the alternate),
but it isn't until patch 11/17 that we fix things that 'ans' and 'asn'
both properly parse '1' as a number instead of rejecting it as an integer.
>> +++ b/tests/test-qmp-output-visitor.c
>> @@ -449,20 +449,31 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
>> const void *unused)
>> {
>> QObject *arg;
>> - Error *err = NULL;
>> + UserDefAlternate *tmp;
>>
>> - UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
>> - tmp->type = USER_DEF_ALTERNATE_KIND_I;
>> + tmp = g_new0(UserDefAlternate, 1);
>> + tmp->type = QTYPE_QINT;
>> tmp->u.i = 42;
>
> Coding style touched up. Okay.
>
>>
>> - visit_type_UserDefAlternate(data->ov, &tmp, NULL, &err);
>> - g_assert(err == NULL);
>> + visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
>
> We need to make up our mind whether to use g_assert(err == NULL) or
> &error_abort in tests. Wholesale conversion could be in order. I like
> &error_abort, because it's more concise.
Wholesale conversion dead-ahead, in 14/17 of this subset.
>
>> arg = qmp_output_get_qobject(data->qov);
>>
>> g_assert(qobject_type(arg) == QTYPE_QINT);
>> g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
>>
>> qapi_free_UserDefAlternate(tmp);
>> +
>> + tmp = g_malloc0(sizeof(UserDefAlternate));
>
> g_new0(UserDefAlternate, 1), please.
>
>> + tmp->type = QTYPE_QSTRING;
>> + tmp->u.s = g_strdup("hello");
>> +
>> + visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
>> + arg = qmp_output_get_qobject(data->qov);
>> +
>> + g_assert(qobject_type(arg) == QTYPE_QSTRING);
>> + g_assert_cmpstr(qstring_get_str(qobject_to_qstring(arg)), ==, "hello");
>> +
>> + qapi_free_UserDefAlternate(tmp);
>
> New test, not mentioned in commit message. Separate patch, perhaps,
> along with the nearby coding style touch ups?
Yes, I will split this portion of the test changes out to a separate commit.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-11-03 18:59 ` Eric Blake
@ 2015-11-04 7:30 ` Markus Armbruster
2015-11-04 16:03 ` Markus Armbruster
1 sibling, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-04 7:30 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/03/2015 11:30 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Previously, working with alternates required two enums, and
>>> some indirection: for type Foo, we created Foo_qtypes[] which
>>> maps each qtype to a member of FooKind_lookup[], then use
>>
>> member of FooKind, actually.
>
> Or entry in the FooKind_lookup[] array.
The generated
const int BlockdevRef_qtypes[QTYPE_MAX] = {
[QTYPE_QDICT] = BLOCKDEV_REF_KIND_DEFINITION,
[QTYPE_QSTRING] = BLOCKDEV_REF_KIND_REFERENCE,
};
maps from qtype_code to BlockdevRefKind, except it uses int instead of
BlockdevRefKind, so it can be passed to visit_get_next_type().
>>> FooKind_lookup[] like we do for other union types.
>>
>> You probably mean FooKind here as well.
>
> I'll play with the wording.
>
>>
>>> This has a subtle bug: since the values of FooKind_lookup
>>> start at zero, all entries of Foo_qtypes that were not
>>> explicitly initialized map to the same branch of the union as
>>> the first member of the alternate, rather than triggering a
>>> failure in visit_get_next_type(). Fortunately, the bug
>>> seldom bites; the very next thing the input visitor does is
>>> try to parse the incoming JSON with the wrong parser, which
>>> fails; the output visitor is not used with a C struct in that
>>> state, and the dealloc visitor has nothing to clean up (so
>>> there is no leak).
>>
>> Yes, I remember us discussing this bug.
>>
>> While reading code to double-check your description, I stumbled over
>> this beauty in generated qapi-visit.c:
>>
>> visit_get_next_type(v, (int*) &(*obj)->type, BlockdevRef_qtypes,
>> name, &err);
>>
>> This casts enum BlockdevRefKind * to int *, which assumes the compiler
>> represents the enum BlockdevRefKind as int or unsigned. It is free to
>> use any integer type, though. Common mistake of programmers with
>> insufficiently developed wariness of C's subtleties.
>>
>> visit_get_next_type() passes the fishy int * on to v->get_next_type().
>> Only implementation is qmp_input_get_next_type(), which uses it so:
>>
>> *kind = qobjects[qobject_type(qobj)];
>>
>> Latent death trap.
>>
>> Does your patch clean this up?
>
> Yes, and I need to also document that this is an additional bug fix.
>
>>
>>> However, it IS observable in one case: the behavior of an
>>> alternate that contains a 'number' member but no 'int' member
>>> differs according to whether the 'number' was first in the
>>> qapi definition, and when the input being parsed is an integer;
>>> this is because the 'number' parser accepts QTYPE_QINT in
>>> addition to the expected QTYPE_QFLOAT. A later patch will worry
>>> about fixing alternates to parse all inputs that a non-alternate
>>> 'number' would accept, for now it is still marked FIXME.
>
> See [1] below.
>
>>>
>>> This patch fixes the validation bug by deleting the indirection,
>>> and modifying get_next_type() to directly return a qtype code.
>>
>> get_next_type() doesn't return anything. Do you mean "store a qtype
>> code"?
>
> Yes.
>
>>
>>> There is no longer a need to generate an implicit FooKind array
>>
>> FooKind is an enum, not an array.
>
> ...to generate an implicit FooKind enum, nor FooKind_lookup[] array.
Yes.
>>> associated with the alternate type (since the QMP wire format
>>> never uses the stringized counterparts of the C union member
>>> names); that also means we no longer have a collision with an
>>> alternate branch named 'max'. Next, the generated visitor is
>>> fixed to properly detect unexpected qtypes in the switch
>>> statement. This is done via the use of a new
>>> QAPISchemaAlternateTypeTag subclass and the use of a new
>>> member.c_type() method when producing qapi-types. The new
>>> subtype also allows us to clean up a TODO left in the previous
>>> commit.
>>>
>>> Callers now have to know the QTYPE_* mapping when looking at the
>>> discriminator; but so far, only the testsuite was even using the
>>> C struct of an alternate types. If that gets too confusing, we
>>> could reintroduce FooKind, but initialize it differently than
>>> most generated arrays, as in:
>>> typedef enum FooKind {
>>> FOO_KIND_A = QTYPE_QDICT,
>>> FOO_KIND_B = QTYPE_QINT,
>>> } FooKind;
>>> to create nicer aliases for knowing when to use foo->a or foo->b
>>> when inspecting foo->type. But without a current client, I
>>> didn't see the point of doing it now.
>
> You have a point below that we either need to reserve MAX and require no
> case-insensitive clashes, or that we will never want to add it. I'm
> leaning towards never going back, because the new way feels so much nicer.
Perhaps un-reserving MAX and making the collision checking more lenient
should be a separate follow-up patch we could easily revert. Probably
useful only if this patch doesn't make 2.5.
>>>
>>> There is a user-visible side effect to this change, but I
>>> consider it to be an improvement. Previously,
>>> the invalid QMP command:
>>> {"execute":"blockdev-add", "arguments":{"options":
>>> {"driver":"raw", "id":"a", "file":true}}}
>>> failed with:
>>> {"error": {"class": "GenericError",
>>> "desc": "Invalid parameter type for 'file', expected: QDict"}}
>>> Now it fails with:
>>> {"error": {"class": "GenericError",
>>> "desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
>>
>> I wonder how that happens. Perhaps it's obvious in the patch.
>
> I think you found it below.
>
>>
>> QMP introspection isn't affected, because we carefully minimized the
>> information to expose there.
>
> Ooh, nice tidbit to add.
>
>
>>> +/**
>>> + * Determine the qtype of the item @name in the current object visit.
>>> + * For input visitors, set *@type to the correct qtype of a qapi
>>> + * alternate type; for other visitors, leave *@type unchanged.
>>> + */
>>> +void visit_get_next_type(Visitor *v, qtype_code *type,
>>> const char *name, Error **errp);
>>
>> Naive question: what makes a visitor an input visitor?
>
> I've got a later patch in my queue that adds a lot more documentation:
> http://repo.or.cz/qemu/ericb.git/commitdiff/f7674a87e72
>
> +/* This file describes the client view for visiting a map between
> + * generated QAPI C structs and another representation (command line
> + * options, strings, or QObjects). An input visitor converts from
> + * some other form into QAPI representation; an output visitor
> + * converts from QAPI back into another form. In the descriptions
> + * below, an object or dictionary refers to a JSON '{}', and an array
> + * or list refers to a JSON '[]'. These functions seldom need to be
> + * called directly, but are instead used by code generated by
> + * scripts/qapi-visit.py. For the visitor callback contracts, see
> + * visitor-impl.h. */
That's a useful step towards a visitors contract.
>>> +++ b/scripts/qapi-visit.py
>>> @@ -189,7 +189,7 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
>>> if (err) {
>>> goto out;
>>> }
>>> - visit_get_next_type(v, (int*) &(*obj)->type, %(c_name)s_qtypes, name, &err);
>>> + visit_get_next_type(v, &(*obj)->type, name, &err);
>>> if (err) {
>>> goto out_obj;
>>> }
>>
>> Yes, your patch disarms the latent death trap: no more pointer casting.
>
> Indeed, I noticed the cleanup as well (I'm quite familiar with the
> unsafe nature of casting enum* because you cannot guarantee its size),
> but failed to call out the trap in my commit message.
>
>>
>>> @@ -203,14 +203,14 @@ void visit_type_%(c_name)s(Visitor *v, %(c_name)s **obj, const char *name, Error
>>> visit_type_%(c_type)s(v, &(*obj)->u.%(c_name)s, name, &err);
>>> break;
>>> ''',
>>> - case=c_enum_const(variants.tag_member.type.name,
>>> - var.name),
>>> + case=var.type.alternate_qtype(),
>>> c_type=var.type.c_name(),
>>> c_name=c_name(var.name))
>>>
>>> ret += mcgen('''
>>> default:
>>> - abort();
>>> + error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
>>> + "%(name)s");
>>
>> Okay, this is where the new error message comes from.
>>
>> Before, default is unreachable, because (*obj)->type got erroneously set
>> the enum's first member when none of the alternate's variants matches
>> the qtype.
>>
>> After, (*obj)->type *is* the qtype, and we do reach default when no
>> variant matches.
>>
>> How can name be null?
>
> When you have the qapi representation ['MyAlternate'], you will have
> qapi_visit_type_MyAlternateList() which passes NULL for the name of each
> list member (because names are only present for objects, not lists).
Examining the occurences of visit_type_BlockdevRef* in generated
qapi-visit.c shows that parameter name is commonly the member name. It
appears to be used only for error messages.
Works fine when the object containing the member is obvious. With
multiple objects, the recipient of the error message has to guess the
object containing the member. If the member name isn't unique, the
error message is ambiguous. Crap, but not crap we can afford to improve
right now.
Array elements are anonymous, and I guess that's what led to NULL.
Stupid choice. Make it "array element" or something. Code becomes
slightly simpler, error message becomes slightly less cruel. This could
be simple enough to improve now.
>> I really need to finish the QERR_ killing job.
>
> Agreed. But shouldn't stall this patch, though.
Certainly not.
>>> # Check every branch
>>> for (key, value) in members.items():
>>> check_name(expr_info, "Member of alternate '%s'" % name, key)
>>>
>>> - # Check for conflicts in the generated enum
>>> - c_key = camel_to_upper(key)
>>> + # Check for conflicts in the branch names
>>> + c_key = c_name(key)
>>
>> Why c_name()?
>
> So that 'a-b' and 'a_b' are properly flagged as conflicting (they map to
> the same c_name).
>
>>> class QAPISchemaObjectTypeVariants(object):
>>> def __init__(self, tag_name, tag_member, variants):
>>> # Flat unions pass tag_name but not tag_member.
>>> # Simple unions and alternates pass tag_member but not tag_name.
>>> # After check(), tag_member is always set, and tag_name remains
>>> - # a reliable witness of being used by a flat union.
>>> + # a reliable witness of being used by a flat union, and
>>> + # tag_member.type being None is a reliable witness of an alternate.
>>
>> A member without a type? Ugh! I wouldn't dare breaking invariants like
>> that.
>>
>> Of course, an alternate's tag member still has a type: qtype_code. It's
>> just not declared in the schema. Should it be a built-in type then?
>
> It's not a builtin that can ever be referenced in the .json files. But
> I could probably come up with something, if it would make you feel better.
>
>
>>> for v in self.variants:
>>> # Reset seen array for each variant, since QMP names from one
>>> # branch do not affect another branch, nor add to all_members
>>> - v.check(schema, self.tag_member.type, dict(seen), cases, union)
>>> + v.check(schema, self.tag_member.type, dict(seen), cases)
>>
>> I expect some rebase churn around here, so I'm not reviewing closely.
>>
>
> Indeed. All the more reason for me to post a v9 spin (and maybe defer
> the question of a non-None type for tag_member until after that post).
>
>
>> My patches move the member name collision checking to
>> QAPISchemaObjectType.check().
>>
>> I suspect alternate branch name collision checking should similarly move
>> to QAPISchemaAlternateType.check().
>>
>
> Yep, already that way in my pending v9 series after incorporating your
> patches.
>
>>> +class QAPISchemaAlternateTypeTag(QAPISchemaObjectTypeMember):
>>> + def __init__(self):
>>> + QAPISchemaObjectTypeMember.__init__(self, 'type', '', False)
>>> +
>>> + def check(self, schema, seen):
>>> + assert len(seen) == 0
>>> + seen[self.name] = self
>>> +
>>> + def c_type(self):
>>> + return 'qtype_code'
>>> +
>>> +
>>
>> This is a hack to work around the lack of a qtype_code type. I suspect
>> creating such a type would be simpler in the end. Safer, too, because
>> it would avoid having members without a type, which scares me.
>
> I can play with dropping c_type() here in favor of adding a qtype_code
> special class, but I may still need to keep this
> QAPISchemaAlternateTypeTag subclass.
Let's give it a try and see.
>>> +++ b/tests/qapi-schema/qapi-schema-test.json
>>> @@ -131,7 +131,7 @@
>>> 'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero',
>>> 'u': 'UserDefZero', 'type': 'UserDefZero' } }
>>> { 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool',
>>> - 'myKind': 'has_a' } }
>>> + 'myKind': 'has_a', 'max': 'str' } }
>>
>> Here, you add the positive test that alternate name 'max' works.
>>
>> One, not mentioned in the commit message.
>
> D'oh.
>
>>
>> Two, the commit message says we may reintroduce FooKind if working with
>> qtype_code turns out to be too confusing. If we ever do that, alternate
>> name 'max' breaks, doesn't it? Shouldn't we keep it reserved then, just
>> in case?
>>
>
> See my comment above; at this point, I doubt we'll ever want to go back,
> so maybe I just need to be more definitive in stating that.
>
>>> +++ b/tests/test-qmp-input-visitor.c
>
>>> @@ -386,11 +386,10 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
>>> qapi_free_AltStrBool(asb);
>>> visitor_input_teardown(data, NULL);
>>>
>>> - /* FIXME: Order of alternate should not affect semantics; asn should
>>> - * parse the same as ans */
>>> + /* FIXME: integer should parse as number */
>>> v = visitor_input_test_init(data, "42");
>>> visit_type_AltStrNum(v, &asn, NULL, &err);
>>> - /* FIXME g_assert_cmpint(asn->type, == ALT_STR_NUM_KIND_N); */
>>> + /* FIXME g_assert_cmpint(asn->type, ==, QTYPE_QFLOAT); */
>>> /* FIXME g_assert_cmpfloat(asn->u.n, ==, 42); */
>>> g_assert(err);
>>> error_free(err);
>>> @@ -398,30 +397,34 @@ static void test_visitor_in_alternate_number(TestInputVisitorData *data,
>>> qapi_free_AltStrNum(asn);
>>> visitor_input_teardown(data, NULL);
>>>
>>> + /* FIXME: integer should parse as number */
>>> v = visitor_input_test_init(data, "42");
>>> - visit_type_AltNumStr(v, &ans, NULL, &error_abort);
>>> - g_assert_cmpint(ans->type, ==, ALT_NUM_STR_KIND_N);
>>> - g_assert_cmpfloat(ans->u.n, ==, 42);
>>> + visit_type_AltNumStr(v, &ans, NULL, &err);
>>> + /* FIXME g_assert_cmpint(ans->type, ==, QTYPE_QFLOAT); */
>>> + /* FIXME g_assert_cmpfloat(ans->u.n, ==, 42); */
>>> + g_assert(err);
>>> + error_free(err);
>>> + err = NULL;
>>
>> What's happening here? Whatever it is, the commit message didn't
>> prepare me for it...
>
> See [1] above. 'asn' is now parsing the same as 'ans' (we are no longer
> sensitive to whether 'number' was the first member of the alternate),
> but it isn't until patch 11/17 that we fix things that 'ans' and 'asn'
> both properly parse '1' as a number instead of rejecting it as an integer.
Recommend to stick a brief note into the commit message that this
temporarily breaks test so-and-so, marked FIXME.
>>> +++ b/tests/test-qmp-output-visitor.c
>>> @@ -449,20 +449,31 @@ static void test_visitor_out_alternate(TestOutputVisitorData *data,
>>> const void *unused)
>>> {
>>> QObject *arg;
>>> - Error *err = NULL;
>>> + UserDefAlternate *tmp;
>>>
>>> - UserDefAlternate *tmp = g_malloc0(sizeof(UserDefAlternate));
>>> - tmp->type = USER_DEF_ALTERNATE_KIND_I;
>>> + tmp = g_new0(UserDefAlternate, 1);
>>> + tmp->type = QTYPE_QINT;
>>> tmp->u.i = 42;
>>
>> Coding style touched up. Okay.
>>
>>>
>>> - visit_type_UserDefAlternate(data->ov, &tmp, NULL, &err);
>>> - g_assert(err == NULL);
>>> + visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
>>
>> We need to make up our mind whether to use g_assert(err == NULL) or
>> &error_abort in tests. Wholesale conversion could be in order. I like
>> &error_abort, because it's more concise.
>
> Wholesale conversion dead-ahead, in 14/17 of this subset.
>
>>
>>> arg = qmp_output_get_qobject(data->qov);
>>>
>>> g_assert(qobject_type(arg) == QTYPE_QINT);
>>> g_assert_cmpint(qint_get_int(qobject_to_qint(arg)), ==, 42);
>>>
>>> qapi_free_UserDefAlternate(tmp);
>>> +
>>> + tmp = g_malloc0(sizeof(UserDefAlternate));
>>
>> g_new0(UserDefAlternate, 1), please.
>>
>>> + tmp->type = QTYPE_QSTRING;
>>> + tmp->u.s = g_strdup("hello");
>>> +
>>> + visit_type_UserDefAlternate(data->ov, &tmp, NULL, &error_abort);
>>> + arg = qmp_output_get_qobject(data->qov);
>>> +
>>> + g_assert(qobject_type(arg) == QTYPE_QSTRING);
>>> + g_assert_cmpstr(qstring_get_str(qobject_to_qstring(arg)), ==, "hello");
>>> +
>>> + qapi_free_UserDefAlternate(tmp);
>>
>> New test, not mentioned in commit message. Separate patch, perhaps,
>> along with the nearby coding style touch ups?
>
> Yes, I will split this portion of the test changes out to a separate commit.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-11-03 18:59 ` Eric Blake
2015-11-04 7:30 ` Markus Armbruster
@ 2015-11-04 16:03 ` Markus Armbruster
2015-11-04 20:52 ` Eric Blake
1 sibling, 1 reply; 56+ messages in thread
From: Markus Armbruster @ 2015-11-04 16:03 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/03/2015 11:30 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>>
>>> Previously, working with alternates required two enums, and
>>> some indirection: for type Foo, we created Foo_qtypes[] which
>>> maps each qtype to a member of FooKind_lookup[], then use
>>
>> member of FooKind, actually.
>
> Or entry in the FooKind_lookup[] array.
>
>>
>>> FooKind_lookup[] like we do for other union types.
>>
>> You probably mean FooKind here as well.
>
> I'll play with the wording.
>
>>
>>> This has a subtle bug: since the values of FooKind_lookup
>>> start at zero, all entries of Foo_qtypes that were not
>>> explicitly initialized map to the same branch of the union as
>>> the first member of the alternate, rather than triggering a
>>> failure in visit_get_next_type(). Fortunately, the bug
>>> seldom bites; the very next thing the input visitor does is
>>> try to parse the incoming JSON with the wrong parser, which
>>> fails; the output visitor is not used with a C struct in that
>>> state, and the dealloc visitor has nothing to clean up (so
>>> there is no leak).
>>
>> Yes, I remember us discussing this bug.
>>
>> While reading code to double-check your description, I stumbled over
>> this beauty in generated qapi-visit.c:
>>
>> visit_get_next_type(v, (int*) &(*obj)->type, BlockdevRef_qtypes,
>> name, &err);
>>
>> This casts enum BlockdevRefKind * to int *, which assumes the compiler
>> represents the enum BlockdevRefKind as int or unsigned. It is free to
>> use any integer type, though. Common mistake of programmers with
>> insufficiently developed wariness of C's subtleties.
>>
>> visit_get_next_type() passes the fishy int * on to v->get_next_type().
>> Only implementation is qmp_input_get_next_type(), which uses it so:
>>
>> *kind = qobjects[qobject_type(qobj)];
>>
>> Latent death trap.
>>
>> Does your patch clean this up?
>
> Yes, and I need to also document that this is an additional bug fix.
>
>>
>>> However, it IS observable in one case: the behavior of an
>>> alternate that contains a 'number' member but no 'int' member
>>> differs according to whether the 'number' was first in the
>>> qapi definition, and when the input being parsed is an integer;
>>> this is because the 'number' parser accepts QTYPE_QINT in
>>> addition to the expected QTYPE_QFLOAT. A later patch will worry
>>> about fixing alternates to parse all inputs that a non-alternate
>>> 'number' would accept, for now it is still marked FIXME.
>
> See [1] below.
>
>>>
>>> This patch fixes the validation bug by deleting the indirection,
>>> and modifying get_next_type() to directly return a qtype code.
>>
>> get_next_type() doesn't return anything. Do you mean "store a qtype
>> code"?
>
> Yes.
>
>>
>>> There is no longer a need to generate an implicit FooKind array
>>
>> FooKind is an enum, not an array.
>
> ...to generate an implicit FooKind enum, nor FooKind_lookup[] array.
>
>>
>>> associated with the alternate type (since the QMP wire format
>>> never uses the stringized counterparts of the C union member
>>> names); that also means we no longer have a collision with an
>>> alternate branch named 'max'. Next, the generated visitor is
>>> fixed to properly detect unexpected qtypes in the switch
>>> statement. This is done via the use of a new
>>> QAPISchemaAlternateTypeTag subclass and the use of a new
>>> member.c_type() method when producing qapi-types. The new
>>> subtype also allows us to clean up a TODO left in the previous
>>> commit.
>>>
>>> Callers now have to know the QTYPE_* mapping when looking at the
>>> discriminator; but so far, only the testsuite was even using the
>>> C struct of an alternate types. If that gets too confusing, we
>>> could reintroduce FooKind, but initialize it differently than
>>> most generated arrays, as in:
>>> typedef enum FooKind {
>>> FOO_KIND_A = QTYPE_QDICT,
>>> FOO_KIND_B = QTYPE_QINT,
>>> } FooKind;
>>> to create nicer aliases for knowing when to use foo->a or foo->b
>>> when inspecting foo->type. But without a current client, I
>>> didn't see the point of doing it now.
>
> You have a point below that we either need to reserve MAX and require no
> case-insensitive clashes, or that we will never want to add it. I'm
> leaning towards never going back, because the new way feels so much nicer.
[...]
>>> +++ b/tests/qapi-schema/qapi-schema-test.json
>>> @@ -131,7 +131,7 @@
>>> 'data': { 'value1': 'UserDefZero', 'has_a': 'UserDefZero',
>>> 'u': 'UserDefZero', 'type': 'UserDefZero' } }
>>> { 'alternate': 'AltName', 'data': { 'type': 'int', 'u': 'bool',
>>> - 'myKind': 'has_a' } }
>>> + 'myKind': 'has_a', 'max': 'str' } }
>>
>> Here, you add the positive test that alternate name 'max' works.
>>
>> One, not mentioned in the commit message.
>
> D'oh.
>
>>
>> Two, the commit message says we may reintroduce FooKind if working with
>> qtype_code turns out to be too confusing. If we ever do that, alternate
>> name 'max' breaks, doesn't it? Shouldn't we keep it reserved then, just
>> in case?
>>
>
> See my comment above; at this point, I doubt we'll ever want to go back,
> so maybe I just need to be more definitive in stating that.
Fair enough.
However, we should also consider QAPI language regularity and
simplicity. To make that argument, I need to back up a bit.
There are three kinds of name collisions: QAPI schema, QMP wire,
generated C.
QAPI schema collisions are the obvious ones:
* JSON object member names must be distinct (enforced by our JSON
parser).
* Enumeration values must be distinct.
* Each flat union's variant's members must be distinct from the
non-variant members.
A translation of the schema into a protocol or code can add additional
restrictions.
I believe the translation to QMP wire doesn't add any. We don't mangle
names, we don't add object members or enumeration values.
The translation to C does add some:
* Mangled names can collide even when unmangled names don't.
* Names can collide with names used by the implementation.
It can also remove collision possibilities (e.g. variant and non-variant
members end up in separate name spaces), but that's not important here.
One possible approach would be to let qapi.py worry about QAPI schema /
QMP wire collisions, and the C compiler about generated C collisions.
We rejected that approach, because navigating from the C compiler's
error messages to the broken spot in the QAPI schema is a bother.
Instead, qapi.py knows enough about the generators to predict collisions
in generated C. This is feasible only because the generators follow
simple and regular rules:
* Names are mangled by c_name(), prefixes or suffixes may be tacked on.
* Exception: enumeration values are mangled by camel_to_upper().
Although the way c_enum_const() is defined, you have to look closely
to see it.
* Any pair of names that clashes before mangling also clashes after.
Permits omitting collision checks for unmangled names when the mangled
names are checked.
* Tag values are mangled *both* ways, because they occur both as C union
members and as C enum members.
* We reserve a bunch of names for generator use: mangled names can't
start with 'q_', mangled member names can't start with 'has_', mangled
enumeration values can't be 'MAX', and so forth.
Remarks:
* A simple union's member names are also tag values, and are therefore
mangled both ways, and both reserved member and enum value names
apply.
* The moment you use an enumeration as type of a flat union tag, the
other mangling and reserved names kicks in.
Conclusions:
* Having two different name manglers is a headache we could do without,
especially since the second one camel_to_upper() is pretty magic.
We have it only to get
typedef enum BlockDeviceIoStatus {
BLOCK_DEVICE_IO_STATUS_OK = 0,
BLOCK_DEVICE_IO_STATUS_FAILED = 1,
BLOCK_DEVICE_IO_STATUS_NOSPACE = 2,
BLOCK_DEVICE_IO_STATUS_MAX = 3,
} BlockDeviceIoStatus;
instead of
typedef enum BlockDeviceIoStatus {
BlockDeviceIoStatus_ok = 0,
BlockDeviceIoStatus_failed = 1,
BlockDeviceIoStatus_nospace = 2,
BlockDeviceIoStatus_MAX = 3,
} BlockDeviceIoStatus;
Bah! CODING_STYLE doesn't even ask for shouting enumeration
constants. Can't see why we do.
* Keeping the complexity of the rules under control is good both for
qapi.py and for the QAPI schema language.
To that end, I think we should consider reserving the same set of
names both for members and tag values. It gets rid of complications
like enumerations you can't use as flat union tags.
Additionally, the question whether to keep the door open for
generating an enum for the alternate cases becomes moot.
What do you think?
[...]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-11-04 16:03 ` Markus Armbruster
@ 2015-11-04 20:52 ` Eric Blake
2015-11-05 7:17 ` Markus Armbruster
0 siblings, 1 reply; 56+ messages in thread
From: Eric Blake @ 2015-11-04 20:52 UTC (permalink / raw)
To: Markus Armbruster; +Cc: qemu-devel, Michael Roth
[-- Attachment #1: Type: text/plain, Size: 1825 bytes --]
On 11/04/2015 09:03 AM, Markus Armbruster wrote:
> Conclusions:
>
> * Having two different name manglers is a headache we could do without,
> especially since the second one camel_to_upper() is pretty magic.
>
> We have it only to get
>
> typedef enum BlockDeviceIoStatus {
> BLOCK_DEVICE_IO_STATUS_OK = 0,
> BLOCK_DEVICE_IO_STATUS_FAILED = 1,
> BLOCK_DEVICE_IO_STATUS_NOSPACE = 2,
> BLOCK_DEVICE_IO_STATUS_MAX = 3,
> } BlockDeviceIoStatus;
>
> instead of
>
> typedef enum BlockDeviceIoStatus {
> BlockDeviceIoStatus_ok = 0,
> BlockDeviceIoStatus_failed = 1,
> BlockDeviceIoStatus_nospace = 2,
> BlockDeviceIoStatus_MAX = 3,
> } BlockDeviceIoStatus;
>
> Bah! CODING_STYLE doesn't even ask for shouting enumeration
> constants. Can't see why we do.
Interesting idea. In fact, if we went one step further:
BlockDeviceIoStatus_ok = 0,
...
BlockDeviceIoStatusMAX = 3.
(that is, typename + '_' + value for user values, and typename + 'MAX'
for the sentinel), then the max value _cannot_ collide with any of the
other values.
>
> * Keeping the complexity of the rules under control is good both for
> qapi.py and for the QAPI schema language.
>
> To that end, I think we should consider reserving the same set of
> names both for members and tag values. It gets rid of complications
> like enumerations you can't use as flat union tags.
>
> Additionally, the question whether to keep the door open for
> generating an enum for the alternate cases becomes moot.
>
> What do you think?
I like the idea. Don't know if it's too late for 2.5, though.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types
2015-11-04 20:52 ` Eric Blake
@ 2015-11-05 7:17 ` Markus Armbruster
0 siblings, 0 replies; 56+ messages in thread
From: Markus Armbruster @ 2015-11-05 7:17 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, Michael Roth
Eric Blake <eblake@redhat.com> writes:
> On 11/04/2015 09:03 AM, Markus Armbruster wrote:
>
>> Conclusions:
>>
>> * Having two different name manglers is a headache we could do without,
>> especially since the second one camel_to_upper() is pretty magic.
>>
>> We have it only to get
>>
>> typedef enum BlockDeviceIoStatus {
>> BLOCK_DEVICE_IO_STATUS_OK = 0,
>> BLOCK_DEVICE_IO_STATUS_FAILED = 1,
>> BLOCK_DEVICE_IO_STATUS_NOSPACE = 2,
>> BLOCK_DEVICE_IO_STATUS_MAX = 3,
>> } BlockDeviceIoStatus;
>>
>> instead of
>>
>> typedef enum BlockDeviceIoStatus {
>> BlockDeviceIoStatus_ok = 0,
>> BlockDeviceIoStatus_failed = 1,
>> BlockDeviceIoStatus_nospace = 2,
>> BlockDeviceIoStatus_MAX = 3,
>> } BlockDeviceIoStatus;
>>
>> Bah! CODING_STYLE doesn't even ask for shouting enumeration
>> constants. Can't see why we do.
>
> Interesting idea. In fact, if we went one step further:
>
> BlockDeviceIoStatus_ok = 0,
> ...
> BlockDeviceIoStatusMAX = 3.
>
> (that is, typename + '_' + value for user values, and typename + 'MAX'
> for the sentinel), then the max value _cannot_ collide with any of the
> other values.
Another arbitrary rule killed.
>> * Keeping the complexity of the rules under control is good both for
>> qapi.py and for the QAPI schema language.
>>
>> To that end, I think we should consider reserving the same set of
>> names both for members and tag values. It gets rid of complications
>> like enumerations you can't use as flat union tags.
>>
>> Additionally, the question whether to keep the door open for
>> generating an enum for the alternate cases becomes moot.
>>
>> What do you think?
>
> I like the idea. Don't know if it's too late for 2.5, though.
The only risk with renaming obviously unique identifiers is producing
merge conflicts. Annoying, but easy enough to resolve.
How much churn exactly? I count 1423 occurences of generated
enumeration constants in 143 out of 4442 source files. Top-scorer is of
course QKeyCode, with 471 occurences in 9 files, almost all in tables.
I've seen worse. I wouldn't ask for such a pull during hard freeze, of
course. During soft freeze, merge conflict risk might be lower than
usual.
^ permalink raw reply [flat|nested] 56+ messages in thread
end of thread, other threads:[~2015-11-05 7:17 UTC | newest]
Thread overview: 56+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-10-28 17:14 [Qemu-devel] [PATCH v8 00/17] alternate layout (post-introspection cleanups, subset C) Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 01/17] qapi: Use generated TestStruct machinery in tests Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 02/17] qapi: Strengthen test of TestStructList Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 03/17] qapi: Provide nicer array names in introspection Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 04/17] qapi-introspect: Guarantee particular sorting Eric Blake
2015-10-30 11:20 ` Markus Armbruster
2015-10-30 15:41 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 05/17] qapi: Track simple union tag in object.local_members Eric Blake
2015-10-30 12:54 ` Markus Armbruster
2015-10-30 16:32 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 06/17] qapi-types: Consolidate gen_struct() and gen_union() Eric Blake
2015-10-30 13:01 ` Markus Armbruster
2015-10-30 16:36 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 07/17] qapi: Rework collision assertions Eric Blake
2015-11-02 15:37 ` Markus Armbruster
2015-11-02 17:20 ` Eric Blake
2015-11-02 21:24 ` Eric Blake
2015-11-03 7:56 ` Markus Armbruster
2015-11-03 13:30 ` Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 0/4] rework of 7/17 Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 1/4] qapi: Drop all_members parameter from check() Eric Blake
2015-11-03 11:06 ` Markus Armbruster
2015-11-03 13:26 ` Eric Blake
2015-11-03 14:02 ` Markus Armbruster
2015-11-03 14:12 ` Eric Blake
2015-11-03 16:19 ` Markus Armbruster
2015-11-03 17:34 ` Eric Blake
2015-11-03 14:04 ` [Qemu-devel] [PATCH 1/7] qapi: Drop obsolete tag value collision assertions Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 2/7] qapi: Simplify QAPISchemaObjectTypeMember.check() Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 3/7] qapi: Clean up after previous commit Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 4/7] qapi: Fix up commit 7618b91's clash sanity checking change Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 5/7] qapi: Eliminate QAPISchemaObjectType.check() variable members Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 6/7] qapi: Factor out QAPISchemaObjectTypeMember.check_clash() Markus Armbruster
2015-11-03 14:04 ` [Qemu-devel] [PATCH 7/7] qapi: QAPISchemaObjectTypeVariants.check() Markus Armbruster
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 2/4] qapi: Check for QMP collisions of flat union branches Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 3/4] qapi: Fix check for variant tag values collision Eric Blake
2015-11-02 22:41 ` [Qemu-devel] [PATCH v8.5 4/4] qapi: Consolidate collision detection code Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 08/17] qapi: Remove outdated tests related to QMP/branch collisions Eric Blake
2015-11-03 16:32 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 09/17] qapi: Add positive tests to qapi-schema-test Eric Blake
2015-11-03 16:43 ` Markus Armbruster
2015-11-03 16:56 ` Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 10/17] qapi: Simplify visiting of alternate types Eric Blake
2015-11-03 18:30 ` Markus Armbruster
2015-11-03 18:59 ` Eric Blake
2015-11-04 7:30 ` Markus Armbruster
2015-11-04 16:03 ` Markus Armbruster
2015-11-04 20:52 ` Eric Blake
2015-11-05 7:17 ` Markus Armbruster
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 11/17] qapi: Fix alternates that accept 'number' but not 'int' Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 12/17] qapi: Remove dead visitor code Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 13/17] qapi: Plug leaks in test-qmp-* Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 14/17] qapi: Simplify error testing " Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 15/17] qapi: Test failure in middle of array parse Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 16/17] qapi: More tests of input arrays Eric Blake
2015-10-28 17:14 ` [Qemu-devel] [PATCH v8 17/17] qapi: Simplify visits of optional fields Eric Blake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.